 Miscellaneous Unix Tips: Answering Novice Shell Questions
Ed Schaefer and John Spurgeon
As Sys Admin contributing editors, we've had the opportunity to answer a number of novice shell questions. In this column, we'll cover a few of the questions we've been asked:
- Using the exec and eval commands
- Combining files using the paste command
- Processing a string a character at a time
- Deleting a file named dash
Using the exec & eval Commands
One novice asked when it was suitable to exec and eval Unix commands:
exec mycommand
eval mycommand
The exec command works differently depending on the shell; in the Bourne and Korn shells, the exec command replaces the current shell with the command being exec'ed. Consider this stub script:
exec echo "Hello John"
echo "Hello Ed"
# end stub
When the above stub executes, the current shell will be replaced when exec'ing the echo "Hello John" command. The echo "Hello Ed" command never gets the chance to execute. Obviously, this capability has limited uses.
You might design a shell menu where the requirement is to execute an option that never returns to the menu. Another use would be restricting the user from obtaining the command line. The following last line in the user's .profile file logs the user out as soon as my_executable terminates:
exec my_executable
However, most systems administrators probably use the exit command instead:
my_executable
exit
Don't confuse exec'ing a Unix command with using exec to assign a file to a file descriptor. Remember that the default descriptors are standard input, output, and error or 0, 1, and 2, respectively.
Let's consider an example where you need to read a file with a while loop and ask the user for input in the same loop. Assign the file to an unused descriptor -- 3 in this case -- and obtain the user input from standard input:
exec 3< file.txt
while read line <&3
do
echo "$line"
echo "what is your input? "
read answer
.
.
done
exec 3<&- # close the file descriptor when done
The eval command is more interesting. A common eval use is to build a dynamic string containing valid Unix commands and then use eval to execute the string. Why do we need eval? Often, you can build a command that doesn't require eval:
evalstr="myexecutable"
$evalstr # execute the command string
However, chances are the above command won't work if "myexecutable" requires command-line arguments. That's where eval comes in.
Our man page says that the arguments to the eval command are "read as input to the shell and the resulting commands executed". What does that mean? Think of it as the eval command forcing a second pass so the string's arguments become the arguments of the spawned child shell.
In a previous column, we built a dynamic sed command that skipped 3 header lines, printed 5 lines, and skipped 3 more lines until the end of the file:
evalstr="sed -n '4,\${p;n;p;n;p;n;p;n;p;n;n;n;}' data.file"
eval $evalstr # execute the command string
This command fails without eval. When the sed command executes in the child shell, eval forces the remainder of the string to become arguments to the child.
Possibly the coolest eval use is building dynamic Unix shell variables. The following stub script dynamically creates shell variables user1 and user2 setting them equal to the strings John and Ed, respectively:
COUNT=1
eval user${COUNT}=John
echo $user1
COUNT=2
eval user${COUNT}=Ed
echo $user2
Pasting Files with paste
Another novice asked how to line up three files line by line sending the output to another file. Given the following:
file1:
1
2
3
file2:
a
b
c
file3:
7
8
9
the output file should look like this:
1a7
2b8
3c9
The paste command is a ready-made solution:
paste file1 file2 file3
By default, the delimiter character between the columns is a tab key. The paste command provides a -d delimiter option. Everything after -d is treated as a list. For example, this paste rendition uses the pipe symbol and ampersand characters as a list:
paste -d"|&" file1 file2 file3
The command produces this output:
1|a&7
2|b&8
3|c&9
The pipe symbol character, |, is used between columns 1 and 2, while the ampersand, &, separates column 2 and 3. If the list is completely used, and if the paste command contains more files arguments, then paste starts at the beginning of the list.
To satisfy our original requirement, paste provides a null character, \0, signifying no character. To prevent the shell from interpreting the character, it must also be quoted:
paste -d"\0" file1 file2 file3
Process a String One Character at a Time
Still another user asked how to process a string in a shell script one character at a time. Certainly, advanced scripting languages such as Perl and Ruby can solve this problem, but the cut command's -b option, which specifies the byte position, is a simple alternative:
#!/bin/ksh
mystring="teststring"
length=${#mystring}
count=0
until [ $count -eq $length ]
do
((count+=1))
char=$(echo $mystring|cut -b"$count")
echo $char
done
In the stub above, string mystring's length is determined using the advanced pattern-matching capabilities of the bash and ksh shells. Any number of external Unix commands can provide a string length, but probably the command with the smallest foot print is expr:
length=$(expr "$mystring" : '.*')
Also, the bash shell contains a substring expansion parameter:
${parameter:offset:length}
According to the bash man page, the substring expansion expands "up to length characters of parameter starting at the character specified offset". Note that the offset starts counting from zero:
#!/bin/bash
mystring="teststring"
length=${#mystring}
ol=1
offset=0
until [ $offset -eq $length ]
do
echo "${mystring:${offset}:${ol}}"
((offset+=1))
done
# end script
Deleting a File Named dash
Finally, a novice inadvertently created a file named with the single character dash, and asked us how to delete the file. No matter how he escaped the dash in the rm command, it still was considered an rm option.
It's easy enough to create the file using the touch command:
touch -
To remove it, use a path to the file -- either full or relative. Assuming the dash file exists in the mydir directory, provide a full path to the file:
rm /pathto/mydir/-
Or if the file exists in the current directory, provide a relative path:
rm ./-
Of course, our old friend find can clobber that file everywhere:
find . -name "-" |xargs rm
Conclusion
None of these tips are earth-shaking -- unless you don't know them. Let us know if you have alternative solutions.
Ed Schaefer is a frequent contributor to Sys Admin. He is a software developer and DBA for Intel's Factory Integrated Information Systems, FIIS, in Aloha, Oregon. Ed also hosts the monthly Shell Corner column on UnixReview.com. He can be reached at: shellcorner@comcast.net.
John Spurgeon is a software developer and systems administrator for Intel's Factory Information Control Systems, IFICS, in Aloha, Oregon. Outside of work, he enjoys turfgrass management, triathlons, ultra-marathon cycling, and spending time with his family.
|