SAR ( System Activity Reporter ) is an excellent command available on Linux for obtaining various information regarding the System. SAR tool can be used to
Monitor System Real Time Performance ( When the System is Running ).Information like CPU,Memory , I/O and many more Details.
Collect Performance Data on Back Ground and allows to use that data to identify Bottlenecks.
SAR
is available from the sysstat package.Here are some of the Things
that can be done using the SAR command,
- Collective CPU usage
- Individual CPU statistics
- Memory used and available
- Swap space used and available
- Overall I/O activities of the system
- Individual device I/O activities
- Context switch statistics
- Run queue and load average data
- Network statistics
- Report sar data from a specific time
Options
for the sar Command
Option
Actions
-a
: Checks file access operations
-b
: Checks buffer activity
-c
: Checks system calls
-d
: Checks activity for each block device
-g
: Checks page-out and memory freeing
-k
: Checks kernel memory allocation
-m
: Checks interprocess communication
-v
: Checks system table status
-p
: Checks swap and dispatch activity
-q
: Checks queue activity
-r
: Checks unused memory
-u
: Checks CPU utilization
-w
: Checks swapping and switching volume
-y
: Checks terminal activity
-A
: Reports overall system performance, which is the same as entering
all options.
Lets
see a few examples on how we can use sar command in obtaining various
information.
CPU
Usage (sar -u)
rootLocal:root002-~ $
sar -u 1 3
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:05:35 PM CPU
%user %nice %system %iowait %steal %idle
10:05:36 PM all
0.75 4.00 0.17 0.00 0.00
95.09
10:05:37 PM all
0.58 27.33 0.42 0.00 0.00 71.67
10:05:38 PM all
1.67 9.16 0.75 0.00 0.00
88.43
Average:
all 1.00 13.49 0.44 0.00 0.00 85.06
The
above output gives you the real time usage of all the CPUs available
in the System.“1 3″ reports for every 1 seconds a total of 3
times. The Important thing we need to focus up here was the %IDLE
which gives the idle time of the CPU.
Following
are some of the other ways of using the sar command for the CPU
information
sar
-u : Displays the CPU Information for the Current day until at this
point
sar
-u 1 3 : Displays real time CPU usage every 1 second for 3 times.
sar
-u ALL : Same as “sar -u” but displays additional fields.
sar
-u ALL 1 3 : Same as “sar -u 1 3″ but displays additional
fields.
Individual
CPU Usage (sar -P)
Todays
System has many CPU cores available,by using the sar utility we can
find the information regarding every CPU core available like,
rootLocal:root002-~
$ sar -P ALL 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:14:33
PM CPU %user %nice %system %iowait %steal
%idle
10:14:34
PM all 1.00 7.66 0.25 0.00 0.00
91.09
10:14:34
PM 0 0.00 0.00 0.00 0.00 0.00
100.00
10:14:34
PM 1 0.99 0.00 0.00 0.00 0.00
99.01
10:14:34
PM 2 8.00 0.00 0.00 0.00 0.00
92.00
10:14:34
PM 3 0.00 9.09 0.00 0.00 0.00
90.91
10:14:34
PM 4 0.99 16.83 0.99 0.00 0.00
81.19
10:14:34
PM 5 1.00 13.00 1.00 0.00 0.00
85.00
10:14:34
PM 6 0.98 37.25 0.98 0.00 0.00
60.78
10:14:34
PM 7 0.00 0.00 0.00 0.00 0.00
100.00
10:14:34
PM 8 0.00 0.00 0.00 0.00 0.00
100.00
10:14:34
PM 9 1.00 0.00 0.00 0.00 0.00
99.00
10:14:34
PM 10 0.00 0.00 0.99 0.00 0.00
99.01
10:14:34
PM 11 1.00 15.00 0.00 0.00 0.00
84.00
In
the above example under “CPU” column we can see 0, 1, 2, and 3
indicates the corresponding CPU core numbers. so we have 12 core CPU
available.
Here
are some of the Other ways of using the sar -P
sar
-P ALL : Displays CPU usage broken down by all cores for the current
day.
sar
-P ALL 1 3 : Displays real time CPU usage for ALL cores every 1
second for 3 times
sar
-P 1 : Displays CPU usage for core number 1 for the current day.
sar
-P 1 1 3 : Displays real time CPU usage for core number 1, every 1
second for 3 times.
Memory
and Swap(sar -r)
Sar
command can also be used in finding the memory usage like,
rootLocal:root002-~
$ sar -r 1 2
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:20:27
PM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree
kbswpused %swpused kbswpcad
10:20:28
PM 796732 65034268 98.79 2836608 21328880
4194296 0 0.00 0
10:20:29
PM 796780 65034220 98.79 2836608 21328896
4194296 0 0.00 0
Average:
796756 65034244 98.79 2836608 21328888 4194296
0 0.00 0
Here
are some of the Other ways of using the sar -r
sar
-r : Display Complete memory details
sar
-r 1 3 : Memory Details for every 1 second for 3 times.
Swap
Usage (sar -W)
rootLocal:root002-~
$ sar -W
Linux
2.6.18-348.4.1.el5 (omhq1970) 08/18/2013
12:00:01
AM pswpin/s pswpout/s
12:10:06
AM 0.00 0.00
12:20:01
AM 0.00 0.00
12:30:01
AM 0.00 0.00
12:40:01
AM 0.00 0.00
12:50:01
AM 0.00 0.00
01:00:01
AM 0.00 0.00
Memory
In pages (sar -R)
Sar
command can also help in obtaining details in pages like,
rootLocal:root002-~
$ sar -R 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:40:53
PM frmpg/s bufpg/s campg/s
10:40:54
PM -54.46 0.00 10.89
Average:
-54.46 0.00 10.89
This
can be used to identify number of memory pages freed, used, and
cached per second by the system.
Here
are some of the Other ways ,
Use
“sar -H” to identify the hugepages (in KB) that are used and
available.
Use
“sar -B” to generate paging statistics. i.e Number of KB paged in
(and out) from disk per second.
Use
“sar -W” to generate page swap statistics. i.e Page swap in (and
out) per second.
I/O
activities (sar -b)
“sar
-b” allows us to find the I/O statistics like,
rootLocal:root002-~
$ sar -b 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:48:37
PM tps rtps wtps bread/s bwrtn/s
10:48:38
PM 71.00 0.00 71.00 0.00 664.00
Average:
71.00 0.00 71.00 0.00 664.00
The
columns say
tps
– Transactions per second (this includes both read and write)
rtps
– Read transactions per second
wtps
– Write transactions per second
bread/s
– Bytes read per second
bwrtn/s
– Bytes written per second
Individual
Device Activities (sar -d)
Sar
command can also be used in identifying various information regarding
individual devices like mount point , partition etc
rootLocal:root002-~
$ sar -d 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:51:12
PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz
await svctm %util
10:51:13
PM dev8-0 22.22 0.00 454.55 20.45 0.00
0.00 0.00 0.00
10:51:13
PM dev8-2 22.22 0.00 454.55 20.45 0.00
0.00 0.00 0.00
10:51:13
PM dev253-0 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00
DEV
column indicates the device.For example: “dev253-0″ means a block
device with 253 as major number, and 0 as minor number.
The
DEV can actually be a device name like sda , sda1 etc like
rootLocal:root002-~
$ sar -p -d 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:53:12
PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz
await svctm %util
10:53:13
PM sda 27.72 0.00 639.60 23.07 0.01
0.54 0.18 0.50
10:53:13
PM sda1 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00
10:53:13
PM sda2 27.72 0.00 639.60 23.07 0.01
0.54 0.18 0.50
Context
Switch (sar -w)
rootLocal:root002-~
$ sar -w 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:55:13
PM cswch/s
10:55:14
PM 11051.49
Average:
11051.49
Sar
command can also be used to find how many context switches are
happening.The above output says how many process are being created
and how many context switches happened for a Second.
Run
Queue and Load Average(sar -q)
Run
Queue and Load average can also be obtained using the sar commnad
like,
rootLocal:root002-~
$ sar -q 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
10:58:59
PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
10:59:00
PM 3 7444 0.26 0.66 1.68
Average:
3 7444 0.26 0.66 1.68
Network
Statistics(sar -n)
Network
statistics can also be obtained using “sar -n”.This works in a
different way. The Syntax is
sar
-n KEYWORD
We
need to pass a KEYWORD for obtaining the Details. The Available
keywords are
DEV
– Displays network devices vital statistics for eth0, eth1, etc.,
EDEV
– Display network device failure statistics
NFS
– Displays NFS client activities
NFSD
– Displays NFS server activities
SOCK
– Displays sockets in use for IPv4
IP
– Displays IPv4 network traffic
EIP
– Displays IPv4 network errors
ICMP
– Displays ICMPv4 network traffic
EICMP
– Displays ICMPv4 network errors
TCP
– Displays TCPv4 network traffic
ETCP
– Displays TCPv4 network errors
UDP
– Displays UDPv4 network traffic
SOCK6,
IP6, EIP6, ICMP6, UDP6 are for IPv6
ALL
– This displays all of the above information
rootLocal:root002-~
$ sar -n DEV 1 1
Linux
2.6.18-348.4.1.el5 (rootLocal) 08/18/2013
11:04:08
PM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s
txcmp/s rxmcst/s
11:04:09
PM lo 0.00 0.00 0.00 0.00 0.00
0.00 0.00
11:04:09
PM eth0 0.00 408.00 0.00 50481.00 0.00
0.00 0.00
11:04:09
PM eth1 0.00 0.00 0.00 0.00 0.00
0.00 0.00
11:04:09
PM sit0 0.00 0.00 0.00 0.00 0.00
0.00 0.00
Average:
IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s
txcmp/s rxmcst/s
Average:
lo 0.00 0.00 0.00 0.00 0.00
0.00 0.00
File
System Usage (sar -v)
File
System details can also be obtained using ,
05:00:01
PM dentunusd file-sz inode-sz super-sz %super-sz dquot-sz
%dquot-sz rtsig-sz %rtsig-sz
05:10:01
PM 192010 1530 194516 0 0.00 0
0.00 0 0.00
05:20:01
PM 192010 1530 194516 0 0.00 0
0.00 0 0.00
05:30:01
PM 191997 1530 194500 0 0.00 0
0.00 0 0.00
“sar
-v” to display number of inode handlers, file handlers, and
pseudo-terminals used by the system.
TTY
activity (sar -y)
Using
sar command we can find various information regarding the TTY like
[root@vx111a
soa]# sar -y | more
Linux
2.6.32-131.0.15.el6.x86_64 (vx111a.jas.com) 08/19/2013
_x86_64_ (4 CPU)
12:00:01
AM TTY rcvin/s xmtin/s framerr/s prtyerr/s brk/s
ovrun/s
12:10:01
AM 0 0.00 0.00 0.00 0.00 0.00
0.00
12:10:01
AM 1 0.00 0.00 0.00 0.00 0.00
0.00
12:20:01
AM 0 0.00 0.00 0.00 0.00 0.00
0.00
12:20:01
AM 1 0.00 0.00 0.00 0.00 0.00
0.00
Historical
Data With SAR
Sar
also provides a way to parse historical data, I.e data regarding the
system that is stored for previous days.
Usually
The system information like CPU,Memory , I/O details that are parsed
by Sar command are stored in /var/log/sa/ location.if we go to this
location we can see various files like
sa01
sa02
sa03
sa04
sa05
sa06
sa07
sa30
They
are created as sadd where dd is
a numeric value for the day of the week (starting at 01).
The
file from the current day is Used.We can restrict the time stamps by
using the -s and -e parmaters
If
you need to access the data of any Day , we can invoke the sar
command as
sar
-f /var/log/sysstat/sa04
sar
-u -f /var/log/sa/sa10 :
Displays CPU usage for the 10day of the month from the sa10 file.
sar
-P ALL -f /var/log/sa/sa10
: Displays CPU usage broken down by all cores for the 10day day of
the month from sa10 file.
Few
Other Uses
sar
-r -f /var/log/sa/sa10
sar
-b -f /var/log/sa/sa10
sar
-d -f /var/log/sa/sa10
sar
-w -f /var/log/sa/sa10
sar
-q -f /var/log/sa/sa10
sar
-s 09:00:00 -e 10:30:00
Between
Time Using SAR(sar -s and -e)
Using
-s (start) and -e (end) options you can retrieve data from past
days.If you want to get information a few days in the past. For
example if you want data from 12:00 to 12:30 your syntax would be:
Dev:rootLocal:root002-~
$ sar -s 12:00:00 -e 12:30:00
Linux
2.6.18-348.el5xen (vx1379) 08/18/2013
12:00:01
PM CPU %user %nice %system %iowait %steal
%idle
12:10:01
PM all 0.49 0.00 0.19 1.09
0.13 98.10
12:20:01
PM all 0.58 0.00 0.17 0.82
0.12 98.31
Average:
all 0.54 0.00 0.18 0.95
0.13 98.21
You
also can add all of the normal sar options when pulling from past
logfiles, so you could run the same command and add the -r argument
to get RAM statistics:
$
sar -s 17:00:00 -e 17:30:00 -f /var/log/sysstat/sa01 -r
Collecting
System Activity Data using SAR at any time
Using
Sar we can collect the system activity to a data file using
sar
-u -o ./datafile 2 3
The
-u option specifies our interest in the CPU subsystem. The -o option
will create an output file that contains binary data. Finally, we
will take 3 samples at two-second intervals. Upon completion of the
sampling, sar will report the results to the screen. This provides us
with a snapshot of current system activity.
To
read the datafile that we created we can use
Sar
-u -f ./datafile
Display
Data In Multiple Formats
[root@vx111a
~]# sadf -d ./datafile -- -B
#
hostname;interval;timestamp;pgpgin/s;pgpgout/s;fault/s;majflt/s;pgfree/s;pgscank/s;pgscand/s;pgsteal/s;%vmeff
vx111a.jas.com;2;2013-08-19
09:50:22 UTC;0.00;25.00;56.25;0.00;442.05;0.00;0.00;0.00;0.00
vx111a.jas.com;2;2013-08-19
09:50:24 UTC;0.00;20.22;28.09;0.00;900.56;0.00;0.00;0.00;0.00
vx111a.jas.com;2;2013-08-19
09:50:26 UTC;0.00;18.80;255.56;0.00;1098.72;0.00;0.00;0.00;0.00
9642534599
Keep
Files For More Than 7 Days
To
make SAR track data for more than 7 days, simply change the
configuration file:
[root@example
~]# vim /etc/sysconfig/sysstat
HISTORY=7
# How long to keep log files (days), maximum is a month
Happy
learning , More To Come :-)
