Article Figure 1 Listing 1 aug2006.tar

Web Server Vulnerability Assessments: Using NiktoRAT to Analyze Nikto Reports

Brian Reilly

Open source contributors have provided an abundance of tools for security assessments. From port scanners like Nmap and vulnerability scanners like the open source versions of Nessus, to more specialized tools that target specific applications and protocols, it is possible to build a comprehensive assessment process strictly using open source. Beyond looking for open ports and missing patches, one important part of any vulnerability assessment is a review of Web servers and Web applications. Nikto remains one of the best tools out there for this purpose. Yet with Nikto, as with many similar tools, running it is the easy part. The real value comes from being able to efficiently review and analyze the output, to cast aside false positives and minor findings and focus on the high-risk, high-impact ones.

After manually reviewing many Nikto reports, I wanted a better solution than having to visit and verify each URL one by one. NiktoRAT (Nikto Results Analysis Tool) was developed to meet this need. It parses a Nikto report and then presents the user with a series of navigable Web pages that contain an HTML dump of the vulnerability URL, the HTTP request and response headers, and other useful information for each finding. This information can then be reviewed by a sys admin, analyst, or penetration tester who can make an assessment on the accuracy and importance of each finding.

An Introduction to Nikto

Nikto is a Web vulnerability scanner that was first released in 2001 and is available from CIRT.net. Written in Perl, Nikto makes use of Rain Forest Puppy's LibWhisker v1.8 module and checks for thousands of Web server and application security holes, vulnerabilities, and misconfigurations. In addition to the base set of plugins provided, it is possible for users to write their own plugins to check for new vulnerabilities or organization-specific requirements.

Installing Nikto and Its Prerequisites

A recent Perl interpreter is required to run Nikto. If you want to be able to scan HTTPS Web servers, OpenSSL and Net::SSLeay are required. Detailed instructions on how to build and install OpenSSL have been well documented elsewhere and are beyond the scope of this article. However, it is worth noting that due to incompatibilities with OpenSSL 0.9.8.a and some Net::SSLeay fuctions, using a version of OpenSSL in the 0.9.7.x branch when building Net::SSLeay is recommended at this time.

If you already have an unsupported version of OpenSSL on your system, it is very easy to install an additional version. When running the OpenSSL config script, set the values for the --prefix and --openssldir options as appropriate. For example, to install OpenSSL in /usr/local/openssl-0.9.7i/ you would do the following: 

% ./config -openssldir=/usr/local/openssl-0.9.7i
% make
% make test
# make install (as root)
Net::SSLeay is available from CPAN and can be installed with the standard perl Makefile.PL, make, make test, make install process. If you need to specify an alternate path to your OpenSSL libraries (such as in the case that you have two versions installed), you can modify the first command as follows: 

% perl Makefile.PL /alternate/path/to/openssl
Installing Nikto is very easy. Simply download the latest version and extract the files to the desired location: 

# gzip -dc nikto-1.35.tar.gz | tar xvf -
Before running Nikto, you may want to modify the value of the $CFG{configfile} variable in nikto.pl from the relative path of the config.txt file to the full path of that file. This will allow you to execute Nikto from a working directory other than where Nikto was installed. Change the line: 

$CFG{configfile}="config.txt";
to something like: 

$CFG{configfile}="/full/path/to/my/nikto/config.txt";
Running Nikto

Here's an example of how to run a basic Nikto scan and save the results to a file called nikto.out: 

% ./nikto.pl -host 10.1.1.1 -output nikto.out
By default, Nikto will assume the host is running an HTTP server on port 80/TCP. However, the following options are supported: 

Options:
    -Cgidirs+      Scan these CGI dirs: 'none', 'all', or a value 
                   like '/cgi/'
    -cookies       print cookies found
    -evasion+      ids evasion technique (1-9, see below)
    -findonly      find http(s) ports only, don't perform a full scan
    -Format        save file (-o) Format: htm, csv or txt (assumed)
    -generic       force full (generic) scan
    -host+         target host
    -id+           host authentication to use, format is 
                   userid:password
    -mutate+       mutate checks (see below)
    -nolookup      skip name lookup
    -output+       write output to this file
    -port+         port to use (default 80)
    -root+         prepend root value to all requests, format is 
                   /directory
    -ssl           force ssl mode on port
    -timeout       timeout (default 10 seconds)
    -useproxy      use the proxy defined in config.txt
    -Version       print plugin and database versions
    -vhost+        virtual host (for Host header)
+ requires a value

These options cannot be abbreviated:
    -config+       use this config file
    -debug         debug mode
    -dbcheck       syntax check scan_database.db and  
                   user_scan_database.db
    -update        update databases and plugins from cirt.net
    -verbose       verbose mode

IDS Evasion Techniques:
     1       Random URI encoding (non-UTF8)
     2       Directory self-reference (/./)
     3       Premature URL ending
     4       Prepend long random string
     5       Fake parameter
     6       TAB as request spacer
     7       Random case sensitivity
     8       Use Windows directory separator (\)
     9       Session splicing

Mutation Techniques:
     1       Test all files with all root directories
     2       Guess for password file names
     3       Enumerate user names via Apache (/~user type requests)
     4       Enumerate user names via cgiwrap 
             (/cgi-bin/cgiwrap/~user type requests)
Additional information about these options and their use can be found in the documentation included with Nikto.

Nikto can generate reports in plaintext, CSV, and HTML formats. For the purposes of this article, we're going to focus on plaintext output. The output from the scan above against a fairly generic Apache Web server looks like the following: 

--------------------------------------------------------------------
- Nikto 1.35/1.34     -     www.cirt.net
+ Target IP:       10.1.1.1
+ Target Hostname: testweb.xyzzycorp.tld
+ Target Port:     80
+ Start Time:      Fri Apr 28 02:21:45 2006
--------------------------------------------------------------------
- Scan is dependent on "Server" string which can be faked, use -g 
  to override
+ Server: Apache/2.2.0 (Unix) mod_ssl/2.2.0 OpenSSL/0.9.8a
+ Allowed HTTP Methods: GET,HEAD,POST,OPTIONS,TRACE
+ HTTP method 'TRACE' is typically only used for debugging. It  
  should be disabled. OSVDB-877.
+ mod_ssl/2.2.0 appears to be outdated (current is at least 2.8.22)  
  (may depend on 
server version)
+ 2.2.0 (Unix) mod_ssl/2.2.0 OpenSSL/0.9.8a - TelCondex  
  Simpleserver 2.13.31027 
Build 3289 and below allow directory  
  traversal with '/.../' entries.
+ mod_ssl/2.2.0 OpenSSL/0.9.8a - mod_ssl 2.8.7 and lower are  
  vulnerable to a 
remote buffer overflow which may allow a remote  
  shell (difficult to exploit). CAN-2002-0082.
+ /cgi-bin/test-cgi - Apache 2.0 default script is executable and  
  reveals system information. 
All default scripts should be removed.  
  (GET)
+ /cgi-bin/phf - Old and vulnerable? Or is someone messing with  
  us...? (GET)
+ / - TRACE option appears to allow XSS or credential theft. See  
  http://www.cgisecurity.com/whitehat-mirror/WhitePaper_screen.pdf 
 
  for details (TRACE)
+ 2563 items checked - 3 item(s) found on remote host(s)
+ End Time:        Fri Apr 28 02:22:31 2006 (46 seconds)
---------------------------------------------------------------------
+ 1 host(s) tested
As we can see, three potential items were found during this scan: one for test-cgi, one for phf, and one for a potential cross-site scripting stemming from the TRACE method. Reviewing a few potential vulnerabilities or misconfigurations from one host is an easy enough task. However, having to do this for a few dozen or more hosts quickly becomes a much greater undertaking.

Dealing with False Positives

Like other security tools, Nikto will occasionally generate false positives. This can be caused by both plugin logic and the configuration of the target Web server. Consider, for example, a cross-site-scripting (XSS) check for the IBM Traffic Express Caching Proxy. Nikto sends a GET request for the following string: 

/\"?<img%20src=\"javascript:alert(document.domain)\">", \ 
  "alert(document.domain)
and looks for an HTTP response containing the same text. If a Web server includes the path of the requested resource in "File Not Found" response, this signature is prone to return a false positive.

There are also a number of circumstances in which Web server configuration can influence the accuracy of the scan. Web server behavior such as returning an unexpected response code (e.g., something other than 404 for unfound pages) or re-directing all requests (regardless of success or failure) can increase the number of false positives generated by a Nikto scan. Some lightweight Web servers, such as those running on various embedded devices and only expect to see a limited syntax of URLs, may return unexpected results including a number of false positives when scanned. Security-conscious Web administrators may also intentionally modify or remove HTTP server banners. Although Nikto can be configured to test all plugins regardless of banner content, this can also affect its ability to correctly identify real vulnerabilities.

Nikto's --verbose and --debug options provide a great deal of information that can be used to see the HTTP response codes for each plugin, view some HTML output, and view raw output generated by LibWhisker. However, using these options often provides so much raw information that it can be unmanageable outside of specific instances of troubleshooting.

NiktoRAT does not do anything to automatically detect these false positives, but it does make them much easier for a human analyst to spot.

A Look at NiktoRAT

My primary objectives when designing NiktoRAT were simple:

  • Parse a Nikto Report.
  • Display enough HTML content to verify each result.
  • Make the interface functional but not complicated.
  • Keep the requirements for running the script as minimal as possible.

NiktoRAT is written in Perl and requires the URI::Escape and HTML::Entities modules to ensure that certain non-alphanumeric characters are handled correctly. These modules are available from CPAN and can be installed using the CPAN shell interface or by manually compiling them from source.

NiktoRAT reads in a Nikto report, sends an HTTP(S) request for each finding, and outputs a series of Web pages that contain detailed raw data. NiktoRAT uses curl when sending Web requests. Curl is an extremely useful and versatile tool that can be used to transfer information via a number of protocols, including but not limited to HTTP and HTTPS. I'm not going to cover curl installation, as it is fairly straightforward and has been documented elsewhere. Note that for HTTPS support, curl has to be compiled with OpenSSL. Although using LWP or LibWhisker was also an option, I opted to use curl to keep things simple and because the wealth of features supported by curl, including SSL and proxy support, kept me from having to re-code the wheel.

I considered a few different options for the user interface and underlying output file structure. I settled on HTML frames -- although not the prettiest and far from ideal from a design standpoint, they met the functional requirement of being able to display HTML content exactly as received from a Web server, with no modification or re-formatting.

There are four frames in the NiktoRAT Web interface: the navigation frame along the left side of the browser (named nikframe-nav.html), a top frame that contains the vulnerability URL and summary as reported by Nikto (nikframe-top-n.html, where n is incremented with each vulnerability URL processed), a middle frame that contains the HTTP request and response headers (nikframe-headers-n.txt), and a bottom frame that contains the HTML content from the vulnerability URL as reported by the Web server (nikframe-bottom-n.html). The navigation frame shows an aggregate of the vulnerability information from the Nikto report, and individual pages for the other frames are generated as NiktoRAT parses the vulnerability URLs.

Listing 1 contains the source code for NiktoRAT.

Running NiktoRAT

Running NiktoRAT is straightforward. The script takes two arguments -- the name of the Nikto report and the output directory for the HTML files. As NiktoRAT parses the report, it will announce when it identifies new hosts and processes vulnerability URLs. Below is sample output from processing the Nikto report from our example above with NiktoRAT: 

% ./niktorat.pl nikto.out /usr/local/niktorat/testweb

************
* NiktoRAT *
************

Reading file: nikto.out

IP Address Found: 10.1.1.1
Hostname Found: testweb.xyzzycorp.tld
Port Found:  80
Processing URLs........

Done parsing file. Results begin here:  
  /usr/local/niktorat/testweb/nikframe-0.html
From here, the user would point a frames-capable Web browser at file:///usr/local/niktorat/testweb/nikframe-0.html or a corresponding http-accessible location to begin reviewing the results. A sample of the Web interface is shown in Figure 1. In this example, we're looking at the content generated by the Nikto finding that the server is vulnerable to an old phf bug. The navigation frame allows one to view all the results, only view the results for a particular HTTP response code, or jump to the results for a specific host. The navigation frame also has a link to the unparsed lines from the Nikto report that did not appear to contain vulnerability URLs. Looking at the right-hand side of the screen, the top frame shows the vulnerability details and provides links to go to the next or previous vulnerability finding. The middle frame shows the URL that was requested with curl and the HTTP response headers sent from the server. The bottom frame shows the HTML content of the requested URL. As we can see, the server that was scanned is not actually vulnerable to the phf bug, and this finding can be confirmed as a false positive. At this point the analyst would continue through the rest of the results, verifying the accuracy and importance of each one.

Using NiktoRAT During Vulnerability Assessments and Penetration Tests

NiktoRAT can be run immediately following a Nikto scan. Note that it will send its own HTTP(S) requests for the vulnerability URLs, so it may generate traffic that is significant to firewalls, intrusion detection/prevention systems, and diligent human log-watchers. As with any activity that is part of a vulnerability scan or penetration test, you should only run NiktoRAT against systems for which you have permission and authority to do so.

Future Work

Additional features that could be added to NiktoRAT include performing more analysis on the results in an attempt to pull out false positives. Since it is pulling HTML content for each result, NiktoRAT could generate a checksum for each page, and give a high false-positive rating to results with the same checksum. A secondary approach could be to calculate the ASCII character values for a page that should not exist (such as Nikto's unique fingerprint request), add the values together, and identify pages that have ASCII sums that are different no more than a set amount. This could be helpful to identify error pages that contained similar but not identical content -- such as "File Not Found" pages that listed the file name and path of the requested resource. The effectiveness of these techniques is still under review and may be added to a future version. A final area for future development is the way the output files are created and stored. Currently, all of the output is stored in flat files. This keeps things simple but isn't the most scalable implementation. Storing the results into a database would require additional work when installing and running the scripts, but would result in being able to sort, index, search, and display the findings in a more robust and scalable manner.

Conclusion

I hope this article has demonstrated the usefulness of Nikto and NiktoRAT when conducting security assessments of Web servers and Web applications. Automating some of the repetitive work and heavy lifting during the initial review and analysis should make the process more efficient, and allow one to focus more energy on identifying, prioritizing, and fixing the real vulnerabilities.

Resources

curl -- http://curl.haxx.se

HTML::Entities -- http://search.cpan.org/search?module=HTML::Entities

LibWhisker -- http://www.wiretrip.net/rfp/lw1.asp

Net::SSLeay -- http://search.cpan.org/search?module=Net::SSLeay

Nikto -- http://www.cirt.net/code/nikto.shtml

NiktoRAT -- http://niktorat.sourceforge.net

OpenSSL -- http://www.openssl.org

URI::Escape -- http://search.cpan.org/search?module=URI::Escape

Brian Reilly is a Consulting Services Technical Lead with Symantec. He previously worked at Georgetown University for more that nine years in a variety of information security and systems administration roles. His background includes performing and managing incident response, network security monitoring, vulnerability assessments, policy development, and regulatory compliance for enterprise organizations. He holds degrees from Georgetown University and George Washington University. Brian can be contacted at: reillyb@georgetown.edu.