 VPNSCAN: Extending the Audit and Compliance Perimeter
Rob VandenBrink
In my work as a security consultant, I have a large number of clients with remote access or remote computing security policies. All of these policies have wording that encompasses some or all of:
- All VPN or dial-up connections to the corporate network will be made from corporately owned hardware.
- Any Internet connection made from a non-corporate location will use a properly configured (or corporately owned and configured) hardware firewall.
- All corporately owned laptops will have a corporate-approved, properly configured personal firewall installed.
However, what struck me was that none of these companies had a good method of auditing these policies to ensure compliance. After some research, I found that there are not a lot of solutions to accomplish this without a large budget commitment. The VPNSCAN tool that I will present in this article addresses this issue and has been deployed at several customer sites to date.
Solution Overview
VPNSCAN uses several tools to accomplish this auditing goal:
- Swatch is used to monitor syslog from the VPN gateway. It waits for a successful VPN connection, then feeds that entire syslog event to a shell script
- The shell script parses out the command-line arguments and then uses Nessus to scan the external (public Internet IP) of the person who just VPN'd in.
- The Nessus scan is saved to an HTML report file.
- If the Nessus scan indicates a violation of company policy (i.e., if the remote firewall fails the scan on some or all identified tests), an alert email is sent to the IT team responsible for security.
In this article, I will outline specifically how VPNSCAN was built, describing policy and implementation issues found in various customer environments. Figure 1 outlines the process flow, showing all processes, data, and people involved. Processes are numbered in order, starting with a client VPN'ing into the corporate network.
Audit, Policy Violations, and Compliance
The purpose of VPNSCAN is to audit remote IP addresses as clients VPN in, to test them for compliance against an existing remote access policy (or a remote access section or paragraph in an "Acceptable Use Policy"). Policies of this type typically state something similar to "Users who access the corporate network remotely will not connect to the public Internet without a corporate-approved firewall". Corporate-approved firewalls typically are a short list of home/SOHO hardware firewalls; some policies also list personal (software-based) firewalls.
The challenge is to audit a remote client's public IP address against this phrase. VPNSCAN employs the Nessus security scanner running on a Linux host, so a reasonable toolset is available to accomplish this task. I decided simply to use grep to look for strings in the Nessus output to evaluate for two conditions:
1. The remote user has no firewall.
2. The remote user's firewall is configured incorrectly.
The grep search strings used to evaluate these conditions are outlined in Table 1. The presence of any of these ports or strings would normally indicate a Microsoft Windows PC with no firewall present, or a very badly configured firewall. Other ports would generally indicate a firewall that has been poorly configured or in some cases connected with an internal port facing the Internet (see Table 2).
Policy Violation and Remediation
As VPNSCAN is written currently, the only automated result is an email to the IT group responsible for security. In many environments, almost all VPN connections are made after hours, so dealing with events of this type in a timely fashion can be a challenge.
There is a continuum of responses to alerts of the type VPNSCAN generates -- a surprising number of IT departments are still completely project-centric and do not apply significant resources to security. In situations like this, alerts are routinely ignored. In almost all implementations, VPNSCAN alerts are simply dealt with via a face-to-face discussion during business hours the following day, where a policy compliant solution is negotiated. However, other options are certainly available.
VPNSCAN alerts could also be sent directly to the USERID involved in the security event. However, end users do not typically have the appreciation to interpret the alert email into practical action to resolve the issue without assistance. This option is not normally implemented.
If the IT group views events of this type to be of the same importance as hostile intrusion events, then responses to VPNSCAN alerts could result in on-call pager events and phone calls to the user's home or cell phone within a time period defined in the company security procedures.
Another option would be to have VPNSCAN perform a more active function, perhaps modifying the firewall or VPN gateway's Internet-facing access list/shun list to deny VPN access from that IP or perhaps disabling internal accounts to deny VPN access (and internal access as well). These approaches are typically not deployed, as false positives can have drastic affects. Another downside to this is that if active methods are deployed, the VPNSCAN scripts would need administrative access to either the firewall/VPN gateway or the internal user database (Active Directory in most cases).
Decision Points in Design
I decided to implement the pilot for VPNSCAN on VMware, using Centos for a guest operating system. VMware was chosen so that VPNSCAN could be easily moved to additional clients -- the entire solution can be image backed up to a single DVD and restored at a different site. Also, no hardware purchase is required if the client has a VMware environment with some capacity available. If deployed to a physical platform, the memory, CPU, and disk requirements are extremely modest for VPNSCAN; almost any workstation or server hardware could be used.
I chose CentOS for an operating system because it is a Linux distribution with a goal of binary compatibility with Red Hat Enterprise Linux (although they no longer mention Red Hat by name on their site). This gives some assurance that the base OS is stable and that the binaries are well tested in combination. It also provides a reasonable path to a commercially supported OS (Red Hat Enterprise). CentOS is freely available, as are all the tools used in VPNSCAN. This ensures that VPNSCAN can be built for and delivered to clients for testing without worrying about software budgets or license fees of any kind. However, if your client has a budget, or if you are building VPNSCAN as part of a consulting engagement, you should ensure that a donation is made to continue the excellent work at CentOS (most easily done by PayPal).
Swatch was chosen because it is a simple, widely deployed tool that will monitor syslog logs and trigger configurable events in near real-time.
Nessus was chosen as the security scanner because it is still freely available, widely used in the security community, and has an excellent reputation. Nessus could, however, be replaced by any security scanner and could in fact be replaced or augmented with any other command-line security assessment tool, command-line virus scanner, or other command-line tool that might be required to assess a remote station. Nessus is available at no charge (registration required) or can be purchased (for newer signature files and commercial support).
Build Procedure
OS Installation
To begin, a basic OS is required -- CentOS in a VMWare virtual machine in this case. The virtual machine was allocated 512 MB Dram and 6 GB of disk. The operating system in the VM was installed without a GUI; however, VMware tools are installed. This improves the console responsiveness, memory, and CPU utilization and enables the configuration of a vmxnet network card (among other benefits).
The detailed mechanics of installing and securing Linux will not be covered in this article -- there are a large number of excellent resources that outline how to do this, and requirements will vary depending on the environment. However, I will mention key decision points along the way that have a bearing on the final solution.
Very little is required for this build beyond the operating system. We installed our image with no GUI, no Web server, and no delivered network services. The OS firewall was enabled, permitting only SSH inbound. The final footprint resulting from these selections fits nicely on the 6-GB disk image allocated for this build.
Linux Configs
After installation, the next step is to install VMware tools, then run netconfig to configure the network card. To bring up the nic, run "ifup eth0" or "service network restart". Ensure that a host name and domain are set (/etc/sysconfig/network and /etc/hosts).
Sendmail is NOT required to send the alert emails (or any mail for that matter). Disable sendmail entirely using chkconfig (or edit the init directories manually if the preference is for doing things "the hard way"). Disabling sendmail is an excellent security measure on almost all Linux hosts (or at least on any Linux host that isn't receiving mail):
[root@vpnscan ~]# chkconfig --list sendmail
sendmail 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[root@vpnscan ~]# chkconfig --level 2345 sendmail off
[root@vpnscan ~]# chkconfig --list sendmail
sendmail 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@vpnscan ~]#
What IS required to send mail is a simple update /etc/submit.cf to point it to a valid mail host, because sendmail is no longer routing SMTP mail via DNS. In this case we'll point it at mail bastion host/spam filter. By default, submit.cf points to localhost, so that sendmail sends it:
D{MTAHost}[127.0.0.1]
This is the modified entry:
D{MTAHost}[172.16.1.22]
If for some reason an email does not leave the Linux host (e.g., if the target host is down at the time of a send), it will be queued. To process mail queues, a cron job will be required -- add this line to /etc/crontab to process the mail queues at the top of each hour:
0 * * * * root /usr/sbin/sendmail -Ac -q
Package Installation
Download and install the latest Nessus version (installation from the OS CDs is not recommended). Note that the Nessus install will alert you to performance issues in the VMware environment. In our case it is not normally an issue, as the scan volume is relatively low (in most environments less than 1,000 scans per day).
Activate the Nessus account (go to www.nessus.org and register to get the activation code for this installation). Then install the Nessus client so that this host can initiate scans:
[root@vpnscan install]# rpm -iv NessusClient-1.0.0.RC5-es4.i386.rpm
Preparing packages for installation...
NessusClient-1.0.0.RC5-es4
[root@vpnscan install]#
Start the service (this will take a while)
[root@vpnscan nessus]# service nessusd start
Starting Nessus services: [ OK ]
[root@vpnscan nessus]#
To start the Swatch install:
# tar -xvzf swatch-3.2.1.tar.gz
# cd swatch-3.2.1/
The command sequence for swatch, and for each of the Perl module dependencies for swatch, is:
perl Makefile.PL
make
make test
make install
make realclean
However, after running Makefile.PL, you'll see that some prerequisites are missing:
[root@vpnscan swatch-3.2.1]# perl Makefile.PL
Checking if your kit is complete...
Looks good
Warning: prerequisite Date::Calc 0 not found.
Warning: prerequisite Date::Format 0 not found.
Warning: prerequisite Date::Manip 0 not found.
Warning: prerequisite File::Tail 0 not found.
Warning: prerequisite Time::HiRes 1.12 not found.
Writing Makefile for swatch
[root@vpnscan swatch-3.2.1]#
These modules can all be found at:
http://search.cpan.org
After trying to install the prerequisites, additional prerequisites for these are discovered! When this phase of the installation is complete, the following additional Perl packages will be installed:
TimeDate
Bit::Vector
Carp::Clan
Mount Points
A mountpoint is required to read the syslog file. It's certainly possible to direct syslog to the vpnscan host, but in most environments a centralized syslog server is already configured for centralized event management and alerting. A mountpoint is also required to write the Nessus reports to. In this installation, they will be written directly to the IT group's private intranet. These files will be http-browsable as they are posted.
Edit /etc/fstab to contain the mounts required for the installation:
//sysloghost/logs /opt/vpnscan/mnt/syslog smbfs \
username=syslogview,password=Abc123 0 0
//fileprintserver/vpnscans /opt/vpnscan/mnt/vpnscans smbfs \
username=dom\vpnscans,password=Abc123 0 0
To do an immediate mount as a check, execute "mount -a".
The Scripts
The scripts are built in /opt/vpnscan. This keeps it away from the operating system, in its own volume. This approach prevents it from filling up the root, and also clearly delineates "who is where" to simplify backups and documentation.
Swatch.conf
The file /opt/etc/swatch.conf contains only two lines:
watchfor /109005/
exec /opt/vpnscan/bin/swatch_scan_ext.sh $_
The trigger event is an occurrence of the string "109005" in syslog. Event id 109005 is what a Cisco PIX writes to syslog when a user successfully connects to the VPN gateway. The event has the format:
2006-09-06 21:35:11 Local4.Info 172.16.1.2 Sep 06 2006 21:35:11: \
%PIX-6-109005: Authentication succeeded for user username from \
111.222.33.44/0 to 99.88.77.66/0 on interface outside
This event is used to trigger swatch because it provides all the information required for the scan in a single line -- the user who is authenticating and his or her public Internet IP (either native on the user's pc or the firewall's IP).
When the event is seen in syslog, swatch executes the command shown. "$_" indicates that the entire syslog message should be passed to the command. The swatch documentation indicates that discrete arguments can be passed (which would work better here). This function has worked in previous versions, but is broken in the current release of swatch.
Swatcher.sh
The script file /opt/vpnscan/bin/swatcher.sh actually calls swatch:
#!/bin/bash
swatch -c /opt/vpnscan/etc/swatch.conf -t \
/opt/vpnscan/mnt/syslog/swatchlog.txt --pid-fil \
/var/run/swatcher.pid -daemon
Note the filename being monitored (or "swatched") is swatchlog.txt. It was found that "swatching" a logfile in this environment actually locked the file on the windows host, which in turn broke log rotation (the log would never be moved to archive and simply grew forever). It was decided to create a second logfile for swatch monitoring, which could then be deleted periodically from the Linux side (more on that later). It was decided to go this route because the syslog host already has the appropriate security and space allocated, and rebuilding this on the Linux host would result in two locations hosting this sensitive information. Additionally, this approach leaves the VPN gateway with a single log trap host, which (theoretically) reduces the CPU load on the VPN gateway by some small amount.
The "-daemon" parameter instructs swatch to run as a daemon -- it is not dependent on any terminal session being logged in to run.
Also, note the PID (Process ID) file "/var/run/swatcher.pid". This provides a handy way to reset the daemon from a script (again, more on this later).
Swatchrst.sh, rc.local and crontab
The file /opt/vpnscan/bin/Swatchrst.sh provides an automated method of resetting the swatch process that is now running independent of any console or vty session:
if /bin/ps -ef | grep swatch | grep pid; then kill \
$(cat /var/run/swatcher.pid ); fi
echo 'a' >/opt/vpnscan/mnt/syslog/swatchlog.txt
/opt/vpnscan/bin/swatcher.sh
In short, this script first checks whether the swatch process is running (checking for the pid string filters out children and the ps parent command itself). If the process is running, it kills it using the contents of the pid file. The echo command shown both deletes the swatchlog.txt file and creates a new one (swatch will not start if the logfile does not exist). Finally, the swatch process itself restarts.
Many of the examples that are listed on the internet use pkill -HUP to restart swatch. In fact, swatch also has a command-line argument to restart the process periodically. The decision to use kill with no HUP parameter was made for two reasons. The process actually needs to be stopped so that the swatchlog file can be deleted before restarting the daemon (remember the file lock discussed?). Using the -HUP (Hangup) parameter restarts the process, but does not address the logfile issue.
By scheduling this hourly, the size of swatchlog.txt is kept under tight control; this also ensures that if the daemon stops for some reason, it will not be stopped for too long. Add this to /etc/crontab to restart at 10 minutes past each hour:
10 * * * * root /opt/vpnscan/bin/swatchrst.sh > /dev/null 2>&1
Also, add swatchrst.sh to /etc/rc.d/rc.local so that it will start if this server is reloaded:
opt/vpnscan/bin/swatchrst.sh > /dev/null 2>&1
On some operating systems, there can be a race condition between the execution of /etc/fstab, and rc.local. If this is the case during a given install, adding the line mount -a into rc.local, ahead of swatchrst.sh will resolve this.
The "> /dev/null 2>&1" string sends both STDOUT (1) and STDERR (2) to /dev/null. In some cases, text output from a daemon-type process can hang the process.
swatch_scan_ext.sh
The /opt/vpnscan/bin/swatch_scan_ext.sh script is what actually runs when a user VPN's in. It scans the public IP of the remote user and then tests for policy compliance. If a policy violation is found, an email is sent to the IT group, who follows up with the remote user. See Listing 1.
/opt/vpscan/etc/vpscan.cf
This shell script that is treated as a config file -- it is used to set all of the variables used in swatch_scan_ext.sh. This file is commented heavily, so it should not require additional documentation. All site-specific information is located in this file, so deploying VPNSCAN at additional sites should not require editing of the main scripts.
#
# These Variables are typically ok as-is
#
#This is where the app is installed
BASEDIR=/opt/vpnscan
# The Nessus Server - typically this is localhost
NESSUSSERVER=127.0.0.1
# Credentials for accessing Nessus
NESSUSUSR=vpnscan
NESSUSPWD=Passw0rd123
# Where should we deposit the Nessus Reports
REPORTDIR=/opt/vpnscan/mnt/vpnscans
# Where should we look for our syslog file
SYSLOGDIR=/opt/vpnscan/mnt/syslog
#
# These variables are site-specific and should be tailored
#
#
# The Corporate mail server, spam filter or other valid smtp host
SMTPSRV=172.16.1.22
# Which user or group should receive alerts
ALERTUSR=itservices@metafore.ca
/opt/vpnscan/etc/services.deny
This file (see Table 3) holds a list of strings that are used as triggers for alerts. In most cases, they are tcp or udp protocols, but note that the last one is a simple ASCII string "Security Hole Found". Similar to the vpnscan.cf file, this decouples the events that trigger alerts from the actual code of the script.
Ancillary Files
Sed input files:
Fixip.sed.in
s/\/0// -- Delete the trailing "/0" from the ip address that syslog gives us.
fixdate.sed.in
s/ /_/g -- Replace all occurences of the space character in the date with underscores s/:/_/g -- Similarly, replace all colons with underscores.
Failscan.msg.in
This user failed the security scan that is done on a VPN connection
The remote host has one or more of the services below open to
the public internet:
tcp/445 Microsoft DS Access
tcp or udp/135, 137, 139 Microsoft / SMB Netbios / RPC
tcp/3389 RDP
tcp/5631 PCAnywhere data
tcp/2967 Symantec Managed AV Client
tcp/5900-5901 VNC
tcp/23 telnet
udp/161 snmp
6000-6004 X-windows
tcp or udp/2049 NFS
Please contact them as soon as possible to ensure that they install a corporate approved
firewall at their remote location, or correctly reconfigure their existing firewall.
Related Files
The Nessus client was run with its default config file. One key thing that might be considered for change would be to expand the port range (the default is to scan the contents of the services file). Note also that "safe" checks are used for our scans -- this process is meant to be as non-intrusive as possible, so crashing a remote firewall would not be desirable. In this installation, the default config file was deemed adequate to identify any station in violation of the remote access policy.
Immediate Futures
The swatch.conf and the shell scripts should be updated to include support for:
- Cisco VPN 3000
- Cisco IOS router as a VPN gateway (this will require support for multi-line events)
- Cisco PIX / ASA5500 version 7.x
- Cisco dial devices
- Shiva dial devices
- Checkpoint Secure/Remote VPN
I hope to implement an active response module, probably written in EXPECT, to update. Implementation of VPNSCAN on a Windows host is also desired. This could easily be implemented using tools such as Kiwi Syslog and Nessus for Windows.
The decision was made not to write any "active response" type extensions to VPNScan, because the feedback from clients and users has been uniformly negative. The chance of denying VPN service because of a false positive (or in many cases even a "real" positive) is deemed an unacceptable risk in almost all corporate environments.
References
CENTOS -- http://www.centos.org
Cisco Documentation -- http://www.cisco.com/univercd
CPAN Net::SSH Perl Module -- http://search.cpan.org/~ivan/Net-SSH-0.08/SSH.pm
CPAN Net::Telnet Perl Module -- http://search.cpan.org/~jrogers/Net-Telnet-3.03/ \
lib/Net/Telnet.pm
Expect -- http://expect.nist.gov/ KIWI Syslog -- http://www.kiwisyslog.com/syslog-info.ph
Nessus -- http://www.nessus.org Swatch -- http://sourceforge.net/projects/swatch/
VMWare -- http://www.vmware.com
Rob VandenBrink is a Security and Network Consultant in Ontario Canada. He currently works for Metafore IT Solutions, and can be reached at: rvandenbrink@metafore.ca. This article began life as a paper for SANS GSEC Gold Certification. The original paper can be found at the SANS Reading Room at: http://www.sans.org/reading_room/.
|