UNIX Support: Analyzing and Improving System Loads For System Admin

Do you have a UNIX computer that is unusually slow and do not know why? Let me introduce a few UNIX commands that can help.

The first UNIX command is called the system activity reporter or "sar". Sar will report the history of a UNIX computer’s load for several system activities. Let’s take a look at this example:

 $ sar

 HP-UX machinename B.11.11 U 9000/800    08/13/07

 00:00:00    %usr    %sys    %wio   %idle
 00:20:00       4       2       0      64
 00:40:00       4       2       0      90
 01:00:00       5       3       0      92
 01:20:00       6       4       1      88
 01:40:00       6       3       2      85
 02:00:00       5       3       3      80
 02:20:00       7       3       2      87
 02:40:00       6       3       2      80
 03:00:00       5       3       2      80
 03:20:00       5       3       3      81
 03:40:00       6       3       3      78
 04:00:00       8       3       6      72
 04:20:00       6       3       5      74

 Average        6       3       1      81

The first row of information is the basic information about the machine, such as the OS, machine name, OS version, etc. The interesting data is in the following columns. The first column (time) is the timestamp as to when the data was collected. By default, this is every 20 minutes. The second column (%user) is the load that the users are putting on the system. The third column (%system) is the load that the system or root (including daemons) processes are putting on the system. And fourth column (%IO) is the load or processes that are waiting for hardware reads or writes (input and outputs, also known as I/O). The final column (%idle) is the average idle load of the system; in other words, how often is the system waiting for tasks to start.

If the idle percentage load is consistently below 20% for long periods of time, then the server’s users would be noticing a slower than normal performance from the machine. Thus, they will be contacting the system administrator, saying that something is wrong, and we would start by running the "sar" UNIX command to find out why the server is slow.

If the idle percentage is low thus there is a server issue, then you will need to look at the other columns for more information. Next, check the last two rows of data. Use the average load information to determine if the problem is a recurring issue that needs a long term solution or just a one time occurrence. You may need to look at the sar history of several days to determine this. Use the second to last row to determine if the performance of the machine is currently good or bad. If the most recent idle load is high, then there is nothing presently running that is using a lot of system resources.

If the system or user percentage loads are high, then there is probably a rogue or zombie process(es) that is eating up a lot of system resources. To list all the process running on the machine sorted from the highest load first, run the "top" UNIX command (exit this tool by pressing Control-C). Here is an example of this command:

  $ top

 Load averages: 0.06, 0.06, 0.07
 845 processes: 813 sleeping, 22 running, 6 stopped, 6 zombies
 Cpu states:
 CPU   LOAD   USER   NICE    SYS   IDLE  BLOCK  SWAIT   INTR   SSYS
  0    0.06   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%
  1    0.06   0.0%   0.0%   1.0%  99.0%   0.0%   0.0%   0.0%   0.0%
  2    0.04   1.0%   0.0%   0.0%  98.0%   0.0%   0.0%   0.0%   0.0%
  3    0.06   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%
 ---   ----  -----  -----  -----  -----  -----  -----  -----  -----
 avg   0.06   1.0%   0.0%   0.0%  99.0%   0.0%   0.0%   0.0%   0.0%

 Memory: 1093116K (283720K) real, 1320320K (332308K) virtual, 10717400K free  Page# 1/30

 CPU TTY        PID USERNAME PRI NI   SIZE    RES STATE    TIME %WCPU  %CPU COMMAND
  3 pts/th    18029 jim   154 20 39732K 18844K sleep    3:24  0.69  0.69 perl
  3   ?          39 root     152 20  1888K  1888K run      2:36  0.65 0.65 vxfsd
  3   ?         752 root     154 20  4180K  1756K sleep   34:04  0.64 0.64 autom
  0   ?        9074 john   154 20 16812K  8892K sleep    2:09  0.63 0.63 Xvnc
  1 pts/11    28419 phil   178 20    32K    32K zomb     0:00  0.42 0.42 perl

The upper part of the output shows the status of each CPU and their averages, in addition to how many zombie processes (processes without valid parent processes) and other useful information. Consider killing all zombie processes if they appear to be using too many resources and are no longer needed. The next section shows how much memory the computer has and is using, including virtual memory. This will show if the system needs more RAM installed, especially if there is a lot of thrashing from RAM onto a disk drive. Finally, the bottom section will report the list of processes in order of the most load intensive processes first. Note that this tool will update this data quite often and quickly, so look for processes that are consistently high and that can be killed without any bad effects. If you do not know how to kill a UNIX process, then please speak with a system administrator. Be careful to not terminate good processes that normally have a high load. If no particular process or set of processes are running amok, then it is probably time to upgrade your RAM or the entire machine.

Finally, if the I/O percentages are high, then that means the hardware resources are doing so many reads and writes that application processes are consistently waiting on the hardware to finish. This is a common issue with high traffic websites, where the high rate of reading of webpages taxes the capability of the hardware. Once you have determined that the I/O load is high, you must determine which drive(s) have a high load by running the "iostat" UNIX command. Here is an example of this command:

$ iostat

 device    bps     sps    msps  

 c2t0d0      0     0.0     1.0  
 c2t1d0      0     0.0     1.0  
 c4t1d1      0     0.0     1.0  
 c6t1d1      0     0.0     1.0  
 c8t2d0      0     0.0     1.0  
 c6t2d0      0     0.0     1.0  
 c4t2d1      0     0.0     1.0

This command will list each device by their internal names, followed by the kilobytes transferred per second (bps), number of seeks per second (sps), and milliseconds per average seek time (msps). Look for any unusually high numbers for any particular device; therefore, you should keep history records of this data to look for data trends that are higher than normal. If the computer transfers a lot of data, such as downloading and uploading files, the closely monitor the transfer rate (bps). If the computer reads in a lot of small packets of information scattered throughout different databases, then look at the other two data columns. Either way, this will quickly reveal any overloaded drives that need upgrading or offloading to other drives that have lower loads.

That pretty much summarizes an introductory to monitoring, analyzing, and improving UNIX system loads. Don’t forget to read the manual pages for these commands to see additional options by using these commands: "man sar", "man top", and "man iostat".

Comparison between Linux And Unix Operating System

Many a times we get confused with Unix and Linux also threat them as synonyms but that is not true. UNIX is copyrighted name only few of the companies are allowed it i.e. IBM AIX and Sun Solaris and HP-UX. This quote from Official Linux kernel README file confuses most:
Linux is a Unix clone written from scratch by Linus Torvalds with assistance from a loosely-knit team of hackers across the Net. It aims towards POSIX compliance.

In this article I am trying to describe the various differences between Unix and Linux under the following headings -

History
Usage
Cost
GUI
Security
Future OS

History

UNIX was created in the 1960s by the AT&T Bell Labs (Ken Thompson and Dennis Ritchie) and first commercially UNIX OS became available in 1970. First release of Unix was having a command line interpreter and some small utility programs with future extension for the multitasking and UNIX evolved from there.

In 1991, Linus Torvalds began working on a UNIX like OS known as Minix. The Minix source code was freely available under GNU GPL and he successfully modified this code for use on x86 PC systems. By September 1991, Torvalds has successfully completed the first Linux kernel designed for PCs. This kernel included various system utilities and libraries from the GNU project to create a usable operating system based completely on open source code.

Usage

The UNIX operating system is used in internet servers, workstations & PCs. Backbone of the majority of finance infastructure and many 24x365 high availability solutions.
Linux can be installed on a wide variety of computer hardware, ranging from mobile phones, tablet computers and video game consoles, to mainframes and supercomputers.

Cost

Linux is Free (as it is opensource, except the case where you need a supported version i.e. RHEL). You can download it from the Internet or redistribute it under GNU licenses. You will see the best community support for Linux. Most UNIX like operating systems are not free (but this is changing fast, for example OpenSolaris UNIX). However, some Linux distributions such as Redhat / Novell provides additional Linux support, consultancy, bug fixing, and training for additional fees.

GUI

Any modern system discussion can not be completed untill unless we discuss about graphical interface. Linux typically provides KDE and Gnome, but there are many alternatives such as LXDE, Xfce, Unity, Mate, twm, ect. where as Unix most distributions now ship with Gnome (initial GUI used to be called CDE i.e. Common Desktop Environment)

Security

Linux comes with open source netfilter/iptables based firewall tool to protect your server and desktop from the crackers and hackers. UNIX operating systems comes with its own firewall product (for example Solaris UNIX comes with ipfilter based firewall) or you need to purchase a 3rd party software such as Checkpoint UNIX firewall.

Future OS

Linux is the fastest moving operating system if we exclude the Android (android can be thought of evolution from UBuntu) whereas Unix is slowly becoming obsolete.

BSD (Berkeley Source Distribution) License Definition for Unix, Linux, Windows or Mac

The BSD license is a class of extremely simple and very liberal licenses for computer software that was originally developed at the University of California at Berkeley (UCB). It was first used in 1980 for the Berkeley Source Distribution (BSD), also known as BSD UNIX, an enhanced version of the original UNIX operating system that was first written in 1969 by Ken Thompson at Bell Labs.

BSD restrictions can be summarized as:

(1) one should not claim that they wrote the software if they did not write it .
(2) one should not sue the developer if the software does not function as expected or as desired.

Some BSD licenses additionally include a clause that restricts the use of the name of the project (or the names of its contributors) for endorsing or promoting derivative works.

Besides the original license used in BSD, there are several derivative licenses that are commonly referred to as a "BSD license". Today, the typical BSD license is the 3-clause version, which is revised from the original 4-clause version.

Note: In all BSD licences as following, is the organization of the or just the , and is the year of the copyright. As published in BSD, is "Regents of the University of California", and is "University of California, Berkeley".

Examples of BSD-Style Licenses

Below are three examples of BSD-style licenses:
(1) The BSD license as it is used by the FreeBSD operating system.
(2) A BSD license as it is used by Sudo (a free utility program for Unix-like operating systems).
(3) A template of a BSD-style license that can be applied to any appropriate project.

Proprietary software licenses compatibility:
The BSD License allows proprietary use and allows the software released under the license to be incorporated into proprietary products. Works based on the material may be released under a proprietary license as closed source software

BSD Licenses Versus the GPL

The GPL (GNU General Public License) is by far the most widely used license for free software (i.e., software whose source code is available at no cost for anyone to use for any purpose). The Linux kernel (i.e., the core of the operating system) as well as much of the other software generally included in Linux distributions have been released under the terms of the GPL.

Although far fewer programs are released under BSD-style licenses, this class of licenses is disproportionately important because of the widespread use of BSD-licensed code in both free and proprietary operating systems.

Possibly the biggest difference between the GPL and BSD licenses is the fact that the former is a copyleft license and the latter is not. Copyleft is the application of copyright law to permit the free creation of derivative works but requiring that such works be redistributable under the same terms (i.e., the same license) as the original work.