Questions and Answers

Amy Rich

Q When we first installed our Solaris 8 machines, it seemed like 1G would be plenty of space for /var. However, we've gotten to the point now where we need to delete logs early or we run out of disk space. The biggest disk hog seems to be the /var/sadm directory, which claims to hold package and patch information. Is there any way we can reclaim some of this space for our log files?

A By default, the Solaris patching mechanism creates backup copies of the packages replaced by the patch. Every time you install a patch using patchadd, the current package is compressed into a file called /var/sadm/pkg/<PKGNAME>/save/<PATCHNAME>/undo.Z. If you install newer versions of the same patch, the undo.Z file in the older version of the patch directory is renamed to obsolete.Z. Let's say you've applied three different revisions of patch 117350, which replaces the packages SUNWcsu, SUNWcsr, SUNWcarx, SUNWcar, SUNWcsxu, and SUNWhea:

117350-12
117350-18
117350-25

You'll find the following save files under /var/sadm/pkg:

   SIZE  FILE NAME

     20  SUNWcsu/save/117350-12/obsoleted_by
  25947  SUNWcsu/save/117350-12/obsolete.Z
     10  SUNWcsu/save/117350-18/obsoleted_by
  26724  SUNWcsu/save/117350-18/obsolete.Z
  28263  SUNWcsu/save/117350-25/undo.Z
     20  SUNWcsr/save/117350-12/obsoleted_by
 807215  SUNWcsr/save/117350-12/obsolete.Z
     10  SUNWcsr/save/117350-18/obsoleted_by
 819144  SUNWcsr/save/117350-18/obsolete.Z
 820046  SUNWcsr/save/117350-25/undo.Z
     20  SUNWcarx/save/117350-12/obsoleted_by
3035397  SUNWcarx/save/117350-12/obsolete.Z
     10  SUNWcarx/save/117350-18/obsoleted_by
3057987  SUNWcarx/save/117350-18/obsolete.Z
3087793  SUNWcarx/save/117350-25/undo.Z
     20  SUNWcar/save/117350-12/obsoleted_by
1842577  SUNWcar/save/117350-12/obsolete.Z
     10  SUNWcar/save/117350-18/obsoleted_by
1848698  SUNWcar/save/117350-18/obsolete.Z
1851817  SUNWcar/save/117350-25/undo.Z
     20  SUNWcsxu/save/117350-12/obsoleted_by
  18948  SUNWcsxu/save/117350-12/obsolete.Z
     10  SUNWcsxu/save/117350-18/obsoleted_by
  19671  SUNWcsxu/save/117350-18/obsolete.Z
  20579  SUNWcsxu/save/117350-25/undo.Z
     20  SUNWhea/save/117350-12/obsoleted_by
  48723  SUNWhea/save/117350-12/obsolete.Z
     10  SUNWhea/save/117350-18/obsoleted_by
  87479  SUNWhea/save/117350-18/obsolete.Z
  93245  SUNWhea/save/117350-25/undo.Z

If you know that you don't want to back out of any of these patches, you can safely delete the files named obsolete.Z and undo.Z. Removing the backup files for path 117350 alone would free up almost 17 megabytes.

If you want to avoid removing these files in the future, you can use patchrm to remove obsolete versions of a patch before installing a new one. For 117350, this would have meant performing the following when adding 117350-18:

patchrm 117350-12
patchadd 117350-18

Then later when 117350-25 came out:

patchrm 117350-18
patchadd 117350-25

There's a great deal of administrative overhead and chance for error involved in this method, though. When removing obsolete versions of 117350 in preparation for an upgrade, you also need to worry about other patches that may depend on it.

Another option is to run patchadd with the -d flag, which tells it not to make any backup copies of the current revision. This can be risky with patches you haven't tested on a similar system, though, because there's no way you can back out. If you wanted to restore to a pre-patch state, you'd have to recover from an OS backup.

The best option is probably to automate removal of the obsolete.Z and undo.Z files with a find cron job, which looks for files of that name older than, say, a month or two. Usually within a month or two of installing a patch on a production system, you'll have found any bugs that would cause you to back out.

Q We use FreeBSD 6.0 on x86 hardware. We'd like to keep up with the ports tree, but our site is locked down behind a squid proxy, and we can't open port 5999. Is there any way to pull the ports tree across via http or https instead of using cvs/cvsup?

A You can use portsnap to update your ports tree instead of using cvs or rsync directly. The portsnap program downloads a compressed snapshot of the FreeBSD ports tree to /usr/local/portsnap. You can then extract the snapshot into /usr/ports or use update to bring your current ports tree up to speed.

FreeBSD 6.0 comes with a newer version of portsnap in the default base system, and the configuration file is located at /etc/portsnap.conf. If you're installing portsnap on an older system, you can do a make install from the /usr/ports/sysutils/portsnap directory to get a slightly older version. The ports version places a sample configuration file, including a key and download URL, in /usr/local/etc/portsnap.conf.sample. Move this file to /usr/local/etc/portsnap.conf prior to using the portsnap port for the first time.

Here's the usage statement from portsnap:

usage: portsnap [options] command [URL | path]

Options:
  -d workdir   -- Store working files in workdir
                  (default: /usr/local/portsnap/)
  -f conffile  -- Read configuration options from conffile
                  (default: /usr/local/etc/portsnap.conf)
  -I           -- Update INDEX only. (update command only)
  -k KEY       -- Trust an RSA key with SHA1 hash of KEY
  -p portsdir  -- Location of uncompressed ports tree
                  (default: /usr/ports/)
  URL          -- Fetch updates from given URL.  
                  (fetch / cron commands only)
  path         -- Extract only parts of the tree starting with the 
                  given string.  (extract command only)
Commands:
  fetch        -- Fetch a compressed snapshot of the ports tree,
                  or update an existing snapshot.
  cron         -- Sleep rand(3600) seconds, and then fetch updates.
  extract      -- Extract snapshot of ports tree, replacing existing
                  files and directories.
  update       -- Update ports tree to match current snapshot, replacing
                  files and directories which have changed.

Even if you already have a copy of the ports tree, perhaps from the install CDROM, you must first download and extract the snapshot so that the correct portsnap metadata is added to /usr/ports:

portsnap fetch
portsnap extract

Now you can interactively run portsnap fetch update or schedule automatic updates from cron. It's recommended that you don't run portsnap cron update, since there's a chance of updating the tree while you're in the middle of a build or install. Instead only update the index from cron: root portsnap -I cron update, and later perform the update interactively: portsnap update.

Q We keep data transfer statistics for an in-house software application in a comma-separated file. The first column in the file is a date stamp, and each column in the file represents a day of the week. We have a number of rows equal to one month's worth of data (so this varies based on the month). We'd like to parse this file and get total transfer statistics for each day of the week. Here's some sample data, which probably shows what I mean more clearly than my attempt to explain it:

1106888400,242574,345724,298451,389164,360195,406682,201988
1106910000,11345,26179,25165,29844,27820,30682,10332
1106931600,6782195,8719573,8479931,8896571,8477045,8546094,7143607
1106953200,9073,10846,11743,10572,10966,12322,9254
1106974800,252574,444039,430619,362729,412532,388076,229699
1106996400,12301,31603,23779,21686,26615,34230,11833
1107018000,7187597,8094294,8096775,8093265,8455820,8365924,7284681
1107039600,9290,13216,11341,12945,14920,14794,10504

I want to sum the columns (except for the date stamp):

242574+11345+6782195+9073+252574+12301+7187597+9290      = 14506949
345724+26179+8719573+10846+444039+31603+8094294+13216    = 17685474
298451+25165+8479931+11743+430619+23779+8096775+11341    = 17377804
389164+29844+8896571+10572+362729+21686+8093265+12945    = 17816776
360195+27820+8477045+10966+412532+26615+8455820+14920    = 17785913
406682+30682+8546094+12322+388076+34230+8365924+14794    = 17798804
201988+10332+7143607+9254+229699+11833+7284681+10504     = 14901898

I'd just want the output of the command to be the sums, though, not the entire sequence of numbers I added up, for example:

14506949 17685474 17377804 17816776 17785913 17798804 14901898

Is there a simple, portable script that will do what I'm looking for?

A Since you have a fixed number of columns, you can sum each column with a simple awk command line. In the following example, datafile is the comma-separated file in which your data resides:

awk -F, '{su+=$2;m+=$3;t+=$4;w+=$5;th+=$6;f+=$7;sa+=$8 } END \
 {print su,m,t,w,th,f,sa }' datafile

Q I'm trying to compile GNU coreutils 5.93 on a Solaris 9 machine using gcc 3.4.3 and autoconf 2.59. Everything seems to go fine with configure, and then it runs coreutils/src/build-aux/missing --run aclocal-1.9 -I m4 and throws a whole lot of errors:

/usr/local/share/aclocal/libmcrypt.m4:17: warning: underquoted 
  definition of AM_PATH_LIBMCRYPT
run info '(automake)Extending aclocal'
or see http://sources.redhat.com/automake/automake.html#Extending-aclocal
configure.ac:51: warning: gl_CHECK_HEADER_sys_m_h is m4_require'd 
  but is not m4_defun'd
configure.ac:51: gl_CHECK_HEADER_sys_m_h is required by...
m4/onceonly_2_57.m4:48: AC_CHECK_HEADERS_ONCE is expanded from...
m4/jm-macros.m4:207: gl_CHECK_ALL_HEADERS is expanded from...
configure.ac:51: gl_CHECK_ALL_HEADERS is required by...
m4/jm-macros.m4:266: gl_CHECK_ALL_TYPES is expanded from...
m4/jm-macros.m4:182: gl_MACROS is expanded from...
configure.ac:51: the top level

There are hundreds of lines of output similar to this, which complain about the following macros and definitions:

gl_CANON_HOST                    gl_CHECK_ALL_HEADERS
gl_CHECK_ALL_TYPES               gl_CHECK_HEADER__avf_h
gl_CHECK_HEADER__macro_h         gl_CHECK_HEADER__tatvf_h
gl_CHECK_HEADER_e_h              gl_CHECK_HEADER_ntdb_h
gl_CHECK_HEADER_priv_            gl_CHECK_HEADER_ss_es_h
gl_CHECK_HEADER_stdio_xt_h       gl_CHECK_HEADER_stlib_h
gl_CHECK_HEADER_string_h         gl_CHECK_HEADER_stropts_h
gl_CHECK_HEADER_sys_e_h          gl_CHECK_HEADER_sys_fs_yps_h
gl_CHECK_HEADER_sys_m_h          gl_CHECK_HEADER_sys_socket_h
gl_CHECK_HEADER_sys_systeminfo_h gl_CHECK_HEADER_sys_tim_
gl_CHECK_HEADER_sys_time_h       gl_CHECK_HEADER_sys_vfs_h
gl_CHECK_HEADER_td_ext_h         gl_CHECK_HEADER_termios_h
gl_CHECK_HEADER_umpx_h           gl_CHECK_HEADER_unistd_h
gl_CHECK_HEADER_wchar_h          gl_CHECK_HEADER_y_et_h
gl_CHECK_HEADER_y_time_h         gl_CHECK_TYPE_STRUCT_UTIMBUF
gl_FSUSAGE                       gl_FUNC_GETPASS_GNU
gl_FUNC_MKSTEMP                  gl_GETADDRINFO
gl_GETPAGESIZE                   gl_GETPAGESIZE   
gl_LOCALCHARSET                  gl_MACROS
gl_MBCHAR                        gl_PATHMAX
gl_PREREQ                        gl_PREREQ_CANON_HOST
gl_PREREQ_GETADDRINFO            gl_PREREQ_GETPASS
gl_PREREQ_REGEX                  gl_PREREQ_SETENV
gl_PREREQ_TEMPNAME               gl_READUTMP
gl_REGEX                         gl_SOCKLEN_T
gl_USERSPEC                      gl_UTIMENS

The final lines just before configure exits are:

configure:6714: error: possibly undefined macro: gl_CHECK_HEADER_priv_
   If this token and others are legitimate, please use m4_pattern_allow.
   See the Autoconf documentation.
configure:6715: error: possibly undefined macro: gl_CHECK_HEADER_stropts_h
configure:6734: error: possibly undefined macro: gl_CHECK_HEADER__avf_h
configure:6735: error: possibly undefined macro: \
   gl_CHECK_HEADER_sys_systeminfo_h
configure:6736: error: possibly undefined macro: gl_CHECK_HEADER_sys_time_h
configure:12125: error: possibly undefined macro: gl_CHECK_HEADER_sys_socket_h
configure:12126: error: possibly undefined macro: gl_CHECK_HEADER_ntdb_h
configure:17931: error: possibly undefined macro: gl_CHECK_HEADER_stdio_xt_h
configure:21381: error: possibly undefined macro: gl_CHECK_HEADER_unistd_h
configure:27430: error: possibly undefined macro: gl_CHECK_HEADER_sys_m_h
configure:27952: error: possibly undefined macro: gl_CHECK_HEADER_string_h
configure:28099: error: possibly undefined macro: gl_CHECK_HEADER_wchar_h
configure:31274: error: possibly undefined macro: gl_CHECK_HEADER_umpx_h
configure:33651: error: possibly undefined macro: gl_CHECK_HEADER_e_h
configure:41574: error: possibly undefined macro: gl_CHECK_HEADER__tatvf_h
configure:44348: error: possibly undefined macro: gl_CHECK_HEADER_ss_es_h
configure:44349: error: possibly undefined macro: gl_CHECK_HEADER_y_et_h

I've never seen this kind of output before, so I'm not entirely sure where to start. It clearly thinks there's some sort of an issue with autoconf, but any other software I've attempted to build has worked just fine.

A The error messages you describe are usually indicative of an autoconf built with a buggy version of GNU m4. GNU m4 versions 1.4 and 1.4.1 both had some memory corruption issues under Solaris. If you upgrade to the newest GNU m4 and rebuild any tools that require it (autoconf, et al.), then the errors above should go away and your coreutils build should be successful.

Amy Rich has more than a decade of Unix systems administration experience in various types of environments. Her current roles include that of Senior Systems Administrator for the University Systems Group at Tufts University, Unix systems administration consultant, author, and charter member of LOPSA. She can be reached at: qna@oceanwave.com.