Pages

Monday, February 4, 2013

File Descriptors In Linux


As we all know that every thing in Linux is a file. Any program that runs on a Linux machine has access to something called a €œFile Descriptor Table. A This table acts as a map providing the process access to files, directories, unnamed pipes, named pipes, sockets and kernel-level data structures. This table exists for each process.

There are 3 standard file descriptors accessible inside the bash shell. They are
0 - Input - Keyboard (stdin) - Standard input
1 - Output - Screen (stdout) - Standard output
2 - Error - Screen (stderr) -Standard error

The above three numbers are standard POSIX numbers and also known as file descriptors. Every Linux command at least open the above streams to talk with users or other system programs. These 3 are character devices. Character devices provide a mechanism to send a stream of characters or bytes. A stream provides sequential access that is, it provides output in the order in which it was received, this is also known as a FIFO pipe, which stands for First-In First-Out.

stdin 0 Read input from a file (the default is keyboard)
cat < File Name
stdout 1 Send data to a file (the default is screen).
date > Output File Name
cat Output File Name
stderr 2 Send all error messages to a file (the default is screen).
rm /tmp/File Name 2> error.txt
cat Error.txt
Using the Above FD Numbers, 2> redirects file descriptor 2, or standard error. &n is the syntax for redirecting to a specific open file (Because 0 (stdin), 1 (stdout) and 2 (stderr) are actually file descriptors the shell requires an ampersand put in front of them for redirection. It duplicates the file descriptor in this case effectively merging the two streams of information together. if you just had "1" with no ampersand, the shell would create a file named "1" and redirect stderr output to it).

Now when you try to execute the below command which gives an error like,
Dev:vx1423:djbs002-jas $ ls -l myfile.txt > test.txt
ls: myfile.txt: No such file or directory

We can redirect the error to a file like
Dev:vx1423:djbs002-jas $ ls -l myfile.txt 2> test.txt
Check the Contents of the Error File
Dev:vx1423:djbs002-jas $ cat test.txt
ls: myfile.txt: No such file or directory


For example 2>&1 redirects 2 (standard error) to 1 (standard output); if 1 has been redirected to a file, 2 goes there too.

Character Action
>         Redirect standard output
2>       Redirect standard error
2>&1   Redirect standard error to standard output
<        Redirect standard input
|         Pipe standard output to another command
>>      Append to standard output
2>&1| Pipe standard output and standard error to another command

/dev/null & /dev/zero
All data written on a /dev/null or /dev/zero special file is discarded by the system. Use /dev/null to send any unwanted output from program/command

command >/dev/null
This syntax redirects the command standard output messages to /dev/null where it is ignored by the shell.

command 2>/dev/null
This syntax redirects the command error output messages to /dev/null where it is ignored by the shell.

command &>/dev/null
This syntax redirects both standard output and error output messages to /dev/null where it is ignored by the shell.

As an Example
Localhost:Root$ grep root /etc/passwd && echo "Found" || "Not Found"
root:x:7282:1566:Nohting:/privdir/root:/bin/bash
Found

This Command shows the Standard Output as well as the Result. If you need only the result ,then send the output to /dev/null like

Localhost:Root $ grep root /etc/passwd > /dev/null && echo "Found" || "Not Found"
Found

A Standard Error Re-direction include
command-name 2>error.log

LocalHost:Root$ find /home -name .profile 2>/tmp/error
LocalHost:Root $ cat /tmp/error
find: /home/lost+found: Permission denied
find: /home/dsig999: Permission denied

Redirect Script Errors

You can redirect script error to a log file called scripts.err:
./script.sh 2>scripts.err
/path/to/example.pl 2>scripts.err

Append To Error Log
You can append standard error to end of error.log file using >> operator:
command-name 2>>error.log
./script.sh 2>>error.log
/path/to/example.pl 2>>error.log

2>&1

The 1 denotes standard output (stdout). The 2 denotes standard error (stderr).

command-name >/dev/null 2>&1

So 2>&1 says to send standard error to where ever standard output is being redirected as well.
Which since it's being sent to /dev/null is akin to ignoring any output at all.

Input redirection can be useful if you have written a program which expects input from the terminal and you want to provide it from a file.

$ myprog < Input File > Output File

To redirect standard error and output to different files (note that grouping is not necessary in Bourne shell):

$ cat myfile > Output File 2> Error File

Assigns the file descriptor (fd) to file for output
Besides 1,2 and 3 there are also file Descriptors from 3 to 1023 which are free to use. For each one a symlink in /dev/fd is created as soon as it is initialized.

For creating a new FD , we need to use the “exec” command.

The internal exec command replaces the current shell process with the specified command. Normally, when you run a command a new process is spawned. The exec command does not spawn a new process. Instead, the current process is overlaid with the new command.

You can assign a file descriptor to an output file with the following syntax:
exec fd> output.txt
where, fd >= 3

Dev:vx1423:djbs002-jas $ exec 4> /config/jboss/ews/1.0/domains/jas/noter
Dev:vx1423:djbs002-jas $ echo "This is a test" >&4
Dev:vx1423:djbs002-jas $ cat noter
This is a test
exec 4<&-

Assigns the file descriptor (fd) to file for input
exec fd< input.txt
exec 3< /etc/resolv.conf
# Executes cat commands and read input from
# the file descriptor (fd) # 3 i.e. read input from /etc/resolv.conf file
cat <&3
# Close fd # 3
exec 3<&-

To Create a read/Write FD for a File use,

exec 3<> file

You can use exec to perform I/O redirection commands. The following examples illustrate the use of exec for redirection purposes.

exec 3< inputfile # Opens inputfile with file descriptor 3 for reading.
exec 4> outputfile # Opens outputfile with file descriptor 4 for writing.
exec 5<&0 # Makes fd 5 a copy of fd 0 (standard input).
exec 6>&p # Attach fd 6 to co-process.