Wednesday, October 31, 2012

My Daily Used Commands

Share it Please
Count the Number of Files
find . -type f | wc -l

Empty a File
echo > filename
>FileName

Tail Last Tailed File
!tail

Find and Remove Files Matching a pattern
find $PWD -type f -name "*Jul-*.ESS-A1.log" -exec rm -f {} \;

Find and Zip files matching a Pattern
find $PWD –type f –name “ess_admin.controllermessages.log.” | xargs tar zcvf one.tar

Differences between 2 files in remote hosts
diff <(ssh alice cat /etc/apt/sources.list) <(ssh bob cat /etc/apt/sources.list)

Find the Symbolic Link
readlink -f LinkFIle

Monitor the active thread count of a process (jvm) on linux
Dev:vx1423:djbs002-bin $ ps uH -p 10343 | wc -l
40

Show Process belong to all Users
ps -ef | grep java

Password Generator
echo "my Password" | md5sum

Find the Command Line Of the Process
ps ux : find the Process ID
cd /proc/<pId of Process>
cat cmdline

List all the Jars loaded by a Process
lsof -p <PID> | grep jar

All process Running as a Specific User
pgrep -l -f -x -u root

Search The File From Multiple Jar Files
find . -name "*.jar" | xargs -tn1 jar tvf | grep --color "log4j.xml"

Show Ports that belong to this PID
netstat --all --program | grep <PID>

Find Time Zone Of the System
date +%Z

Drop all the Connections available now for a Port
iptables -I INPUT -p tcp -dport 80 -j DROP

Find from which package the tool is provided
yum provides /bin/ls

Find all the Specific Process
ps -fp $(pgrep -d, -x java)

List all files starting with dots
ls -ld .[1-9A-Z]*

Reverse A String
rev<<<"This is a test"

Find and Grep a File
find $PWD -type f -exec egrep PAVAN {} \;

Find the Time took Process To Complete
/usr/bin/time -v <command>

Memory Information ( Much Better than Free Command)
vmstat -s -S M | grep mem

Find the Process Based On Command
ps -f -C java

See the Memory Usage of Process
ps aux --sort pmem ( See the MEM column)

find out which cpu processor a process is running on
ps -eo pid,args,psr

Return Number of kernel Threads Owned by a process
ps -o thcount –p

Display Date of the Process Started
ps -o lstart

List Threads By PID along with Thread Start Time
ps -o lwp,lstart --pid <PID>

Find Information of the Parent Process
ps -o ppid= | xargs ps -p <PID>

Count Threads OF a JVM Process
ps uH p | wc –l

Print a stack trace of a running process
Pstack <PID>

Find how many files and application is using
lsof +c 0 | cut -d' ' -f1 | sort | uniq –c

Find and Kill a Process
Ps ux | grep | grep –v grep | awk ‘{print $2}’ | xargs –r kill -9

Print Lines Between 404 and 487
sed -n '404,480p' <File Name>

List all files opened by a particular command
lsof -c java

Determine if port is open or not
lsof -i :22

Kill a Process on the Port
kill -9 `lsof -t -i :port_number`

List all files opened by a particular command
lsof -c java

Check open ports
lsof -Pni4 | grep LISTEN

View details of network activity
lsof -i :7000-8000

show the working directories of running processes
lsof -bw -d cwd -a -c java

find the deleted file ,which is in use
lsof -n |grep delete

Grep from Open Files
lsof | grep "stuff"

View Open File Descriptors for a Process
lsof -p | wc -l

Find The Command Line Of the Process Using Specific Port
cat /proc/$(lsof -ti:631)/cmdline

Find out what is listening on a series of ports
/sbin/fuser -n tcp {7000..8000}

Get the 10 Biggest Files
du -sh * | sort -n | tail

Convert Of Bytes to Mbs
units --terse "3415014314 bytes" "MB"

SSh and Execute a Command
ssh root@<Remote Machine> -q 'echo $MYDIR’

Find out which process is using up your memory using ps, awk, sort.
ps aux | awk '{if ($5 != 0 ) print $2,$5,$6,$11}' | sort -k2n

Create Multiple Files at a Time
touch $(seq -f "file%02g" 10)

Find Whether The Process is 32Bit or 64Bit
file -L /proc/<PID>/exe

The Output would be some thing like this
/proc/6462/exe: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped

Find Whether a port is open on a Remote Host
nc -z <Ip Address> <port>; echo $?
1

1:Running and Open
0:No

Last Boot Time
who -b
system boot May 31 23:59

Find the Operating System Type
cat /etc/issue

tmpwatch
Remote Files which havenot been accessed for a period of Time
tmpwatch -umc -v 24 /test : Delete Files that are not accessed under 24 hours

Show Ports that Belong to a Process
netstat --all --program | grep <PID>

HostIP
/usr/bin/gethostip www.google.com
www.google.com 74.125.227.50 4A7DE332

Find the File Permission of a file
stat –printf="%a\n" <File Name>

Execute a Script on the Remote Host
ssh <User ID>@<Host Name> 'sh' < $PWD/<Script Name>

M4 : Replace
This command takes an input file and substitutes strings inside it with the parameters passed, similar to substituting for variables. For example, here is an input file:
$ cat temp

The COLOR fox jumped over the TYPE fence.
Were you to substitute the strings "COLOR" by "brown" and "TYPE" by "broken", you could use:

$ m4 -DCOLOR=brown -DTYPE=broken temp
The brown fox jumped over the broken fence.
Else, if you want to substitute "white" and "high" for the same:

$ m4 -DCOLOR=white -DTYPE=high temp
The white fox jumped over the high fence.

Status Of the HTTP
curl -o /dev/null --silent --head --write-out '%{http_code}\n' <url>
200
-o /dev/null throws away the usual output
-silent throws away the progress meter
-head makes a HEAD HTTP request, instead of GET
-write-out '%{http_code}\n' prints the required status code

Number Of Active Physical Processors
Dev:vx1423:djbs002-~ $ cat /proc/cpuinfo | grep physical.id | sort -u | wc -l
2

Number of CPU cores per Processor
If this number is 1 and physical CPUs is 2, then each of the 2 physical processors has 1 CPU cores, leading to a total of 2 cores.

Dev:vx1423:djbs002-~ $ grep cpu.cores /proc/cpuinfo | sort -u
cpu cores : 1

Number of Logical Processors
For each processor core that is physically present, the operating system addresses two virtual or logical cores, and shares the workload between them when possible. The main function of hyper-threading is to decrease the number of dependent instructions on the pipeline

Dev:vx1423:djbs002-~ $ grep processor /proc/cpuinfo | wc -l
2

Find Whether a Port is Open On a Remote Machine
nmap -p 16162 vx1528

Interesting ports on eth0.vx1528.uprr.com (67.206.10.243):
PORT STATE SERVICE
16162/tcp open unknown

Find OS and Other Information about a Remote Host
[root@vx111a ~]# nmap -O 172.16.11.178

Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2013-03-19 18:52 IST
Interesting ports on 172.16.11.178:
Not shown: 1672 closed ports
PORT STATE SERVICE
25/tcp open smtp
80/tcp open http
135/tcp open msrpc
139/tcp open netbios-ssn
443/tcp open https
445/tcp open microsoft-ds
1026/tcp open LSA-or-nterm
3389/tcp open ms-term-serv
MAC Address: 00:25:64:BB:D8:66 (Unknown)
Device type: general purpose
Running: Microsoft Windows 2003/.NET|NT/2K/XP
OS details: Microsoft Windows 2003 Server or XP SP2

Finding Allocations of Memory For a Proces
pmap -x <PID>| grep rwx | sort -n -k3 | less

List Process By memory usage
ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS

List all the Installed Perl packages
perl -MFile::Find=find -MFile::Spec::Functions -Tlwe 'find { wanted => sub { print canonpath $_ if /\.pm\z/ }, no_chdir => 1 }, @INC'

Total Number of Process that Can Be Created
                                        RHEL 2.1 RHEL-3 RHEL-4 RHEL-5 RHEL-6
Max number of processes 11000     32000  32000  32000 32768[1]

RHEL6: [1] default and limit on 32 bit kernels. On 64 bit kernels, limit is 4194304, configurable via /proc/sys/kernel/pid_max

All Open Tomcat Threads
Dev:vx11111:jbs002-~ $ ps -ALcf | grep org.apache.catalina.startup.Bootstrap | wc -l
87

ac - print statistics about users connect time
Dev:vx1111:jbs002-jas $ ac -p
jbs002 37.60
pma002 0.02
root 170.69
jbs001 841.64
total 1049.95

Convert horizontal lines to vertical lines
Dev:vx1379:djbs002-jas $ echo "123456"| fold -w 11
123456

Shows the Java Process along with Command Line
pgrep -fl java

Find The MAC Address
[root@vx111a test]# cat /sys/class/net/eth0/address
00:25:64:b3:19:fd

Find The Command Line Of the Process Using Specific Port
cat /proc/$(lsof -ti:631)/cmdline

Ping a Host Using Specific Interface ( Or can check Internet Connection for an Interface)
ping -I eth0 www.yahoo.com

Scp To transfer a File
Scp /root/file1 root@10.5.11.193:.
.’ Says to the root folder. You can use /tmp or any thing to copy to a specific location

scp -r /dir root@10.5.11.193:. ( Copies a Entire Directory Structure)

Delete a File
Unlink fileName

Describe a Command
[root@vx111a ~]$ type ls
ls is aliased to `ls --color=tty'

DMI table Decoder
[root@vx111a ~]# dmidecode -type bios
Invalid type keyword: ype
Valid type keywords are:
bios
system
baseboard
chassis
processor
memory
cache
connector
slot

ex: dmidecode –type memory : Prints memory relegated things

Last Reboot Information
[root@vx111a ~]# last reboot
reboot system boot 2.6.18-238.el5 Mon Jul 30 12:44 (06:45)
reboot system boot 2.6.18-238.el5 Fri Jul 27 12:39 (06:36)
reboot system boot 2.6.18-238.el5 Thu Jul 26 13:38 (06:59)
reboot system boot 2.6.18-238.el5 Fri Jul 20 13:03 (06:29)

wtmp begins Tue Jul 3 15:22:01 2012

Files Opened By Application
In Order to find the files that are opened by an application we can use the lsof command like,

[root@vx111a bin]# lsof +f | head
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
init 1 root cwd DIR 8,8 4096 2 /

How do I find how many files an application is using
In order to find how many files and application is using , we can use
lsof +c 0 | cut -d' ' -f1 | sort | uniq –c

Find Out What Partition a File Belongs To
We can use the df command to find out what partition a file belongs,

[root@vx111a perl]# df -T file1
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda7 ext3 49594196 5499736 41534504 12% /

Check the Multi Cast Address Subscribed
There are times where we need to test various address including the multi cast address.To find the multi cast address that was subscribed for the system we can use

[root@vx111a local]# ip maddr
1: lo
inet 224.0.0.1
2: eth0
link 01:00:5e:00:00:fb
link 01:00:5e:00:00:01
inet 224.0.0.251
inet 224.0.0.1

Total Number Of Threads In a Linux Process
For a single-threaded process, you will find all the information about the state of the process under /proc/pid itself. For a multi-threaded process, the /proc/pid will have a sub directory called task. This directory contains contains one sub directory per thread, including the main thread: /proc/pid/task/tid1, /proc/pid/task/tid2, etc. The main thread has the thread ID same as the process ID.

The easiest way to find out if a given process is multi threaded or not is to examine the /proc/pid/task directory. If it contains many sub directories, its a multi-threaded process. Otherwise, it could be a single threaded one.

The Number of Threads that processes have can be given using
cat /proc/<PID>/status

Working Directory of a Current Process
There are many times for a sys admin or even a developer to get the current working directory of a process. We can get this information by visiting /proc/pid/cwd or even by using pwdx command.

According to the man pages, a pwdx command provides the information of the working directory of a process.

If we need to find the current working directory of a process 3349, we can use

[root@vx111a local]# pwdx 3349
3349: /root

Or We can also use

[root@vx111a local]# cat /proc/$(lsof -ti:3349)/cmdline
ro root=LABEL=/ rhgb quiet

Find and Kill a Process
There are some times where we need to find a process and kill it in a single line , here is how it can be done
Ps ux | grep | grep –v grep | awk ‘{print $2}’ | xargs –r kill -9

Find the Command Line Of a Process
Dev:vx1379:djbs002-7996 $ /usr/bin/strings /proc/7728/cmdline
/software/java32/jdk1.6.0_16/bin/java
-Djbs.name=JAS-A2
-server
-Xms512m
-Xmx512m
-XX:MaxPermSize=256m
-Dcom.sun.management.jmxremote.port=10009
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djava.security.auth.login.config=/config/jboss/ews/1.0/domains/jas/JAS-A2/conf/login.config
-Djbs.logdir=/logs/jboss/ews/1.0/domains/jas
-javaagent:/software/jboss/ews32/1.0/lib/spring-agent.jar
.
.
start

Scheduler class for a process in linux
ps ax --format uname,pid,ppid,tty,cmd,cls,pri,rtprio

Details about Process
ps -fp $(pgrep -d, -x java)

ps -ef : This output shows that the first process that is executed when the system boots

Highest CPU
ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10 | awk "{ print $2 }"

Tar file correctly Copied or Not
tar -ztf /path/to/archive.tar.gz > /dev/null && echo $? If echo $? = 0

Memory Usage
sar -q 1 | tail -1 | awk '{ print "" $3}' | sed 's/%//g'

Finding the CPU Threshold
top -b -n 1 | awk -F'[:,]' '/^Cpu/{sub("\\..*","",$2); print $2}'

Current User With Session Count
who |
awk '
{ User [$1]++; }
END { for (i in User) printf "%-9s %s\n", i, User [i] }

Top 10 Process
ps -efF "%x %p %P %U %u %y %a" | sort -r | head

Memory Space Details
free -t -m | grep "Total" | awk '{ print "Total Memory space : "$2 " MB";
print "Used Memory Space : "$3" MB";
print "Free Memory : "$4" MB";
}'

Swap memory Details
free -t -m | grep "Swap" | awk '{ print "Total Swap space : "$2 " MB";
print "Used Swap Space : "$3" MB";
print "Free Swap : "$4" MB";
}'

Remove Files Using Find
Dev:vx1111:bs002-jas $ find . -type f -atime +60 -print > ./deadFiles
Dev:vx1111:bs002-jas $ cat ./deadFiles
./JAS-B2/conf/logs/juli.2010-02-11.log
./JAS-B2/bin/logs/juli.2010-01-06.log
./JAS-B2/bin/logs/juli.2009-11-05.log
./JAS-B2/bin/logs/juli.2010-02-11.log
./JAS-C2/catalina.out
./JAS-C2/conf/catalina.out

Dev:vx1111:bs002-jas $ rm `cat ./deadFiles`

How to Find Files That Exceed a Specified Size Limit
find . -size +400 -print

How to List the Newest Files
ls -t

Most recently Created or Changed
ls -tl

List Process By memory usage
ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS

Find how many Files are opened for a process
lsof -p <PID> | wc -l
741

Detailed Information
Dev:vx1379:djbs002-~ $ ps -fp $(pgrep -d, -x java)
UID PID PPID C STIME TTY TIME CMD
djbs002 20366 1 0 May13 ? 00:01:03 /software/java32/jdk1.6.0_16/bin/java -Djbs.name=JAS-B2 -server -Xms512m -Xmx512m -XX:MaxPermSize=256m -Dcom.sun.management.jmxre

View What PID is Touching
lsof -p PID Or lsof -p 2424

Also, /proc/PID/ provides lots of info:
Code:
cd /proc/2324
ls –l

Finally, you can attach PID to a file and record all calls:
strace -s 2000 -p PID -o /tmp/pid.log
If you just need to trace a lib calls, try ltrace.

Total number of unique users logged in currently
cat /tmp/a| sed '1,2d' | awk '{print $1}' | uniq | wc -l

List of unique users logged in currently
cat /tmp/a | sed '1,2d'| awk '{print $1}' | uniq

User who is using high %cpu
cat /tmp/a | sed '1,2d' | awk '$7 > maxuid { maxuid=$7; maxline=$0 }; END { print maxuid, maxline }'

Smallest to Largest
Dev:vx1379:djbs002-~ $ ls -s | sort -nr | more

View Command Line
Dev:vx1379:djbs002-7996 $ /usr/bin/strings /proc/7728/cmdline
/software/java32/jdk1.6.0_16/bin/java
-Djbs.name=JAS-A2
-server
-Xms512m
-Xmx512m
-XX:MaxPermSize=256m
-Dcom.sun.management.jmxremote.port=10009

Show Number of Java Process
top -b -n 1|grep java|wc -l
processes per user counter
Dev:vx1379:djbs002-jas $ ps hax -o user | sort | uniq -c
2 avahi
1 dbus
.
.

Cpu & Memory Usage
ps aux --sort=%mem,%cpu

Who Started this process
Dev:vx1379:djbs002-jas $ ps -o comm= -p $(ps -o ppid= -p 28453)
init

How Much Ram Is Being Used
Dev:vx1379:djbs002-jas $ ps -o rss -C java | tail -n +2 | (sed 's/^/x+=/'; echo x) | bc
2245952

Find When a Process Was Started
Dev:vx1379:djbs002-jas $ ps -o lstart 28453
STARTED
Fri May 3 16:41:01 2013

Print ENVIRONMENT Variables associated with a Process
ps ewwo command <PID> | tr ' ' '\n' | grep \=

List Threads by Pid along with Thread Start Time
Dev:vx1379:djbs002-jas $ ps -o pid,lwp,lstart --pid 28453 -L

PID LWP STARTED
28453 28453 Fri May 3 16:41:01 2013
28453 28455 Fri May 3 16:41:01 2013
28453 28456 Fri May 3 16:41:01 2013
..

Show CPU usage for EACH cores
djbs002-jas $ ps ax -L -o pid,tid,psr,pcpu,args | sort -nr -k4| head -15 | cut -c 1-90

26903 5658 0 0.1 /software/java64/jdk1.6.0_24/bin/java -Dprogram.name=run.sh -Dapp.nam
..

Memory Usage
Dev:vx1379:djbs002-jas $ ps -o comm,%mem,args 28453
COMMAND %MEM COMMAND
java 1.8 /software/java64/jdk1.6.0_24/bin/java

Total CPU Usage Percentage
omhq126a:dwls999-~ $ ps aux | awk {'sum+=$3;print sum'} | tail -n 1
261.4

DF in Cloumn Format
df -hP|column -t

Show the working directories of running processes
Dev:vx1379:djbs002-~ $ lsof -bw -d cwd -a -c java
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 9582 djbs002 cwd DIR 253,2 1024 6149 /config/jboss/ews/1.0/domains/jas
java 26417 djbs002 cwd DIR 253,9 4096 131093 /privdir/djbs002

Find the Class File in jars
Dev:vx1379:djbs002-lib $ find . -name "*.jar" | while read line; do unzip -l $line; done | grep AlgDH.class
486 03-26-10 14:20 com/tera/tdgss/jalgapi/AlgDH.class

Java Path
Dev:vx1379:djbs002-lib $ ls -l $(type -path -all java)
-rwxr-xr-x 1 djvm999 jvmgrp 47308 Jul 31 2009 /software/java32/jdk1.6.0_16/bin/java

Discard Ping Output
ping -c 1 $host >/dev/null || { echo " The remote Host is unavailable" ; exit; }

Extract the tar and bz2 file at same time
tar jxf firefox-bin.tar.bz2

Get response from a URL
wget --spider -nv -T 5 -t 1 http://localhost:10011/instance-monitor

Get the Socket Status
Dev:vx1379:djbs002-~ $ netstat -t -n | cut -c 68- | sort | uniq -c | sort -n
      1
      1          State
      4          ESTABLISHED

Test the RPM
rpm -i --test eject-1.2-2.i386.rpm

List File descriptors used by process
lsof -Fpcn -u djhe002 | awk '
 /^p/ { pid=substr($0,2) }
 /^c/ { cmd=substr($0,2) }
 /^n/ { fd[cmd"["pid"]"]++ }
 END  { for (cc in fd) printf("%-20s %i\n",cc,fd[cc]) } ' | sort -n -k 2

File currently Open For writing
lsof | grep -e "[[:digit:]]\+w"

Get the Socket Status
netstat -anl | awk '/^tcp/ {t[$NF]++}END{for(state in t){print state, t[state]} }'
CLOSE_WAIT 429

Happy learning