Search This Blog

Thursday, May 29, 2014

Nagios/Icinga Windows Server and Domain Controller Performance Monitoring


Introduction

Windows Server ships with a excellent monitoring and trend analysis tool: Performance Monitor.  As illustrated below, it allows administrators to select and graph counters that include list of system metrics.  These measurements may also be saved as delimited text files for future analysis and visualization.  A centralized server may connect to other servers to remotely collect data.  Since the API is well-documented, it is integrated into other value-added systems monitoring software.





The illustration below depicts the text format of a Windows Performance Monitor counter.  Commands conforming to this syntax may be sent from remote monitoring servers whose applications comply with the Windows API.
Nagios and Icinga include a plugin that complies with Windows Performance Monitor APIs as well as several other Windows systems performance indicators.  The balance of this document will describe how to implement the check_nt command to develop a comprehensive monitoring, alerting and trend analysis system for Windows Server 2008 R2 and Windows Domain Controllers.

Prerequisites


A thorough knowledge of Nagios/Icinga, including PNP4Nagios, is necessary.  Several articles that describe how to configure the necessary components are available here.

Windows Server 2008 R2 Monitoring and Trend Analysis


Check_nt Commands and Services

The command set is based upon the Microsoft recommended 

There are three check_nt variables (-v) used in the command definitions: Windows Services, Windows Performance Counters and the CPULOAD check.  The commands are used in corresponding service definitions that assign templates, service groups and hostgroups.  Sample commands to illustrate the formats include:

Windows Server Services Format

define command {
command_name Win_2008R2_Server_Services
command_line /usr/lib/nagios/plugins/check_nt -H $HOSTADDRESS$ -s <password> -p 12489 -v SERVICESTATE -l MSDTC,gpsvc,Netlogon,netprofm,nlasvc,nsi,RpcEptMapper,SamSs,LanmanServer,eventlog,MpsSvc,W32Time,LanmanWorkstation,Dnscache
}

define service{

use                               generic-service,pnpsvc
servicegroups               win_2008r2_servers_servicegroup
hostgroup_name          win_2008r2_servers_hostgroup
service_description      Windows 2008R2 Server Services
check_command          Win_2008R2_Server_Services
notification_interval      0

}

Windows Performance Monitor Counters Format

define command {
command_name Win_2008R2_Processor_Processor_Percent_Idle_Time
command_line /usr/lib/nagios/plugins/check_nt -H $HOSTADDRESS$ -s <password> -p 12489 -v COUNTER -l "\\Processor(_Total)\% Idle Time","Processor Percent Idle Time is %.f"
}

define service{
use                                  generic-service,pnpsvc
servicegroups                 win_2008r2_servers_servicegroup
hostgroup_name            win_2008r2_servers_hostgroup
service_description        Windows 2008R2 Server Processor Percent Idle Time
check_command            Win_2008R2_Processor_Processor_Percent_Idle_Time
notification_interval        0
}

Windows Server CPU Load Format

define command{
command_name Win_2008R2_CPU_Avg
command_line /usr/lib/nagios/plugins//check_nt -H $HOSTADDRESS$ -s <password> -p 12489 -v CPULOAD -l 1,60,95,5,60,95,15,60,95,60,60,95
}

define service{
use                                  generic-service,pnpsvc
servicegroups                 win_2008r2_servers_servicegroup
hostgroup_name            win_2008r2_servers_hostgroup
service_description        Windows 2008R2 Server Processor Averaged Load
check_command            Win_2008R2_CPU_Avg
notification_interval        0
}



Download Windows Server 2008R2 Nagios/Icinga Command and Service Definitions
Download Windows Domain Controller Nagios/Icinga Command and Service Definitions

Summary of Monitored Services and Counters

Windows Server Services

Distributed Transaction Coordinator (MSDTC)
Group Policy Client (gpsvc)
Netlogon (Netlogon)
Network List Service (netprofm)
Network Location Awareness Service (nlasvc)
Network Store Interface (nsi)
RPC Endpoint Mapper (RpcEptMapper)
Security Accounts manager (SamSs)
Server Service (LanmanServer)
Event Log Service (eventlog)
Windows Firewall Service (MpsSvc)
Windows Time Service (W32Time)
Workstation Service (LanmanWorkstation)
DNS Cache (Dnscache)

Processor

Current work queue (Processor 0)
Current work queue (Processor 1)
Processor Percent C1 TimeProcessor Percent C2 TimeProcessor Percent C3 TimeProcessor Percent DPC TimeProcessor Percent Idle TimeProcessor Percent Interrupt TimeProcessor Percent Maximum Frequency TimeProcessor Percent Priority TimeProcessor Percent Privileged TimeProcessor Percent Processor TimeProcessor Percent User TimeSystem Context Switches/sec
System Processor Queue Length
CPULOAD (1-, 5-, 15- and 60-minute averages)

Memory

Memory Available MBytes
Memory Free System Page Table Entries
Memory Pages Input/sec
Memory Pages/sec
Memory Pool Nonpaged Bytes
Memory Pool Paged Bytes

Disk

LogicalDisk Avg. Disk sec/Read
LogicalDisk Avg. Disk sec/Write
LogicalDisk Disk Transfers/sec
PhysicalDisk Avg. Disk sec/Read
PhysicalDisk Avg. Disk sec/Write

Network

Network Interface Output Queue Length
Network Interface Packets Outbound Discarded
Network Interface Packets Outbound Errors
Network Interface Packets Inbound Discarded
Network Interface Packets Inbound Errors
Network Interface Bytes Total/sec
Server Bytes Total per sec

Other

Percent Registry Quota in Use
Process IO Data Operations/sec (Typically Disk)
Process IO Other Operations/sec (Typically Disk)
Process IO Read Operations/sec (Typically Disk)
Process IO Write Operations/sec (Typically Disk)

Description of the Command and Service Definitions

Using the check commands requires a working knowledge of both Nagios/Icinga and Windows Performance Monitor.  For instance, the included definitions are relatively generic, using counters such as _Total instead of specific targets.  For instance, instead of monitoring the C:\, D:\, etc. and individual physical RAID arrays, the commands merely monitor total activity thus:
\\LogicalDisk(_Total)\Avg. Disk sec/Read
\\PhysicalDisk(_Total)\Avg. Disk sec/Read
A more precise definition would replace _Total with specific Logical and Physical Array definitions.

Network adapter counters must be specified by name, consulting the name used in the Performance Monitor Counter.  Thus, the files used in this configuration define an Intel Desktop NIC:
\\Network Interface(Intel[R] PRO_1000 MT Desktop Adapter)\Output Queue Length
The adapter name must be changed for each specific server.  In fact, it is necessary to write individual check commands and service definitions for each type of adapter and assign it on a host-by-host basis.

Windows Domain Controller Monitoring and Trend Analysis

The format and use of the command and service definitions are the same as those described above.

Summary of Monitored Services and Counters

Windows Server Services

DNS Server (DNS)
Intersite Messaging Service (IsmServ)
Kerberos Key Distribution Center (kdc)

Windows Server Domain Controller (NTDS) Counters

NTDS Address Book Client Sessions
NTDS ATQ Estimated Queue Delay
NTDS ATQ Outstanding Queued Requests
NTDS ATQ Request Latency
NTDS DRA Pending Replication Synchronizations
NTDS DRA Inbound Full Sync Objects Remaining
NTDS DRA Outbound Values (DNs only) per sec
NTDS DS Threads in Use
NTDS DS Directory Reads per sec
NTDS DS Directory Writes per sec
NTDS DS Name Cache hit rate
NTDS DS Notify Queue Size
NTDS LDAP Active Threads
NTDS LDAP Bind Time
NTDS LDAP Client Sessions
NTDS LDAP Successful Binds per sec
NTDS LDAP Searches per sec
NTDS NTLM Binds per second
NTDS SAM Account Group Evaluation Latency
NTDS SAM Enumerations per sec
NTDS Simple Binds per sec

The configuration files are straightforward.

Configuration and CPU Stress Test Demonstration

The two videos below illustrate the configuration files (as viewed through NConf) and a CPU stress test with a warning alert and graphing of relevant counters.







Download Windows Server 2008R2 Nagios/Icinga Command and Service Definitions
Download Windows Domain Controller Nagios/Icinga Command and Service Definitions

Friday, May 9, 2014

Suricata IDS -- Arachni Vulnerability Scan

Introduction

This article describes an Arachni vulnerability scan of a Linux - Exchange 2010 messaging and collaboration system.  A Linux workstation will perform the scan of a Linux security appliance in the DMZ that protects an Exchange infrastructure in the private network.

Several articles are prerequisite reading for this one:
Postfix Mail Gateway to Exchange
Apache Reverse Proxy Server to Exchange Client Access Web Services
Linux Security Appliance for Microsoft Services
Installing Suricata, Barnyard2 and Snorby

Background

The illustration below depicts the test environment. A Linux workstation equipped with OpenVAS will scan the address servicing HTTP/HTTPS and SMTP services.  A Linux Security appliance is located in the DMZ and services these traffic types and then forwards them to the Exchange Edge Services (also HTTP/HTTPS and SMTP).  This traffic is then managed by the Exchange infrastructure.

Suricata is installed on a hub in the DMZ.  Hubs operate at Layer 1 of the OSI model and forward all traffic to every port on the device.  The attached servers and workstations then decide what traffic to accept or reject based upon the destination MAC address in the Ethernet frame; they also negotiate media access.

Switches operate at Layer 2 of the OSI model.  They maintain a table of connected MAC addresses and forward traffic only to the port to which the destination MAC address is attached.  In this example, a switch would only forward inbound Internet traffic to the port to which the Linux Security Device is attached.  However, higher-end switches also support port forwarding, in which specified traffic is also forwarded to a defined port, which in this case would be the Suricata server's port.

For an IDS to see traffic if interest, it must either be on a hub or a switch with port forwarding enabled and configured to forward to the Suricata server's port.

 Arachni Results

Arachni operates primarily at higher layers of the OSI model, targeting applications running only on HTTP and HTTPS servers.  In this case, it was able to identify the /owa redirect from the Apache reverse proxy to Exchange 2010 Outlook Web Access.  This information, alone, is adequate to identify well-known targets to further investigate.

Scans of the default Exchange Client Access Services web directories (/owa, /Microsoft-Server-ActiveSync and /ecp) disclosed significant vulnerabilities.  The most serious was a cross-site request forgery vulnerability present on the unpatched Exchange 2010 Client Access Server.


A PDF of the scan results is available here.

Suricata Results

Suricata detected none of the Arachni scans.  Arachni only used valid HTTP requests to enumerate directories present below the web root.  Additional scans were conducted using the SSL-encrypted HTTPS protocol; Suricata can not decrypt this traffic and was unable to read, let alone fingerprint, potentially malicious traffic.

Discussion

In this case, a well-crafted vulnerability scan was conducted that disclosed significant risks and was not detected by the IDS.  In a previous article, Suricata IDS -- OpenVAS Vulnerability Scan, the security appliance and IDS identified a generalized reconnaissance and vulnerability scan.  However, a more precisely-conducted Arachni vulnerability scan went completely undetected.

IDS systems as implemented in this model are not effective at identifying well-crafted vulnerability scans.  Information may be enumerated using valid HTTP requests and the actual penetration conducted using encrypted HTTPS that Suricata can not read.

Under these circumstances, Suricata may provide a false sense of security.  Patch management of the targeted Exchange system is required in addition to IDS systems.


The video below documents the Arachni scan and Suricata-Barnyard2-Snorby IDS logging.
 

Suricata IDS -- OpenVAS Vulnerability Scan

Introduction

This article describes an OpenVAS vulnerability scan of a Linux - Exchange 2010 messaging and collaboration system.  A Linux workstation will perform the scan of a Linux security appliance in the DMZ that protects an Exchange infrastructure in the private network.

Several articles are prerequisite reading for this one:
Postfix Mail Gateway to Exchange
Apache Reverse Proxy Server to Exchange Client Access Web Services
Linux Security Appliance for Microsoft Services
Installing Suricata, Barnyard2 and Snorby

Background

The illustration below depicts the test environment. A Linux workstation equipped with OpenVAS will scan the address servicing HTTP/HTTPS and SMTP services.  A Linux Security appliance is located in the DMZ and services these traffic types and then forwards them to the Exchange Edge Services (also HTTP/HTTPS and SMTP).  This traffic is then managed by the Exchange infrastructure.

Suricata is installed on a hub in the DMZ.  Hubs operate at Layer 1 of the OSI model and forward all traffic to every port on the device.  The attached servers and workstations then decide what traffic to accept or reject based upon the destination MAC address in the Ethernet frame; they also negotiate media access.

Switches operate at Layer 2 of the OSI model.  They maintain a table of connected MAC addresses and forward traffic only to the port to which the destination MAC address is attached.  In this example, a switch would only forward inbound Internet traffic to the port to which the Linux Security Device is attached.  However, higher-end switches also support port forwarding, in which specified traffic is also forwarded to a defined port, which in this case would be the Suricata server's port.

For an IDS to see traffic if interest, it must either be on a hub or a switch with port forwarding enabled and configured to forward to the Suricata server's port.

OpenVAS Results

OpenVAS operates primarily at Layer 3 (Network - IP) and Layer 4 (Transport - TCP) of the OSI mode.  It first determines which well-known ports are listening on the target IP address.
Open TCP ports: 80, 443, 25
It then attempts to fingerprint the applications running on those ports.  In this case it determined the web server is Apache version 2.2.22, but it was unable to fingerprint the mail server (Postfix).
Detected Apache version: 2.2.22
Location: 80/tcp
CPE: cpe:/a:apache:http_server:2.2.22
Concluded from version identification result:
Server: Apache/2.2.22

Nmap service detection result for this port: smtp
This is a guess. A confident identification of the service was not possible.
Once OpenVAS has narrowed the potential ports and applications, it then runs checks for only those vulnerabilities associated with the fingerprinted applications.   For this scan, it identified 4 Medium Risk and 2 Low Risk Vulnerabilities of Apache and logged 27 other results.

A PDF of the scan results is available here.

Suricata Results

Suricata was able to detect suspicious traffic signatures.  It first identified the OpenVAS port scans.  It then identified several different attempts to identify Apache server weaknesses (such as malformed requests).  It also identified HTTP traffic targeting specific application vulnerabilities (e.g. ColdFusion and Magneto), such as attempts to gain elevated privileges and access user names and passwords.  It also detected an attempt to exploit a Heartbleed vulnerability.

Suricata detected a total of 43 High, 1 Medium and 60 Low Severity intrusion attempts.

Discussion

The overall performance of the Linux-Apache-Postfix gateway to Microsoft Exchange 2010 services was good.  OpenVAS focused on the Linux-Apache server when attempting to identify vulnerabilities.  These were identified and are of no more than Medium risk and relatively easy to fix.  Better still, the scan did not identify Microsoft Exchange web and mail services at all.

The video below documents the OpenVAS scan and Suricata-Barnyard2-Snorby IDS logging.



Linux with Suricata, Barnyard2 and Snorby

Introduction

Suricata, like the older and better-known Snort, is an intrusion detection / intrusion prevention system (IDS/IPS) that operates by capturing packets and searching for signatures of potentially malicious payloads.  It uses Snort-compatible rule sets and interacts with other software -- such as Barnyard2, Snorby and MySQL -- for presentation.

This article describes installing the Suricata IDS/IPS, Barnyard2 log-exporting daemon and Snorby web-based front- and back-end built upon Apache, MySQL and Ruby on Rails.  There are many other considerations -- such as hardware and application configuration tuning -- that must also be considered, but these will be addressed in other articles.

The installation shall consist of two servers: Suricata and Barnyard2 on the first at address 10.64.0.3 and Snorby, Apache, Ruby ob Rails and MySQL on the second at address 10.64.0.4.

Suricata

The following series of commands will install Suricata:
#apt-get -y install libpcre3 libpcre3-dbg libpcre3-dev build-essential autoconf automake libtool libpcap-dev libnet1-dev libyaml-0-2 libyaml-dev pkg-config zlib1g zlib1g-dev libcap-ng-dev libcap-ng0 make libmagic-dev git-core libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev libnfnetlink0

#mkdir suricata && cd suricata && git clone git://phalanx.openinfosecfoundation.org/oisf.git && cd oisf && git clone https://github.com/ironbee/libhtp.git -b 0.5.x

#./autogen.sh && ./configure --enable-nfqueue --prefix=/usr --sysconfdir=/etc --localstatedir=/var && make && make install-full && ldconfig

Network Card Modifications

First, the NIC must be set to promiscuous mode, in which it accepts all packets, not just those whose destination is its own MAC address.  For a VirtualBox VM, this is set in the VM configuration.  That is, the VM's NIC is connected to a virtual hub (which sends each packet to every port) instead of a virtual switch (which maintains a MAC address table and only forwards a packet to the port to which the packet's destination MAC address is attached).

Network interface cards (NICs) are designed to optimize system performance by offloading work from the CPU to dedicated logic on the NIC -- the TCO Offload Engine (TOE).  Processing TCP/IP processing requires approximately 1 Hz processing per 1 bit/second network traffic.  Thus, a fully utilized Gigabit Ethernet NIC, transferring 1 Gigibit/second, consumes 1 GHz processing cycles.  That represents approximately 40% of a single 2.5 GHz processor's capacity.  The dedicated logic on the NIC frees up a significant amount of processor resources by offloading the TCP/IP processing from the CPU to the logic on the NIC.

TOE also enhances PCIe bus performance.  PCIe is inefficient at small-data transfers, such as unassembled TCP/IP traffic.  By performing the TCP/IP processing and reassembling the data into larger streams, data transfers on the PCIe bus are more efficient.

Unfortunately, the packet capture functionality of an IDS system operates not on the NIC, but with the operating system on the CPU.  If the TCP/IP processing occurs on the NIC, the packet capture will not see the Ethernet traffic as packets but as reassembled data.  This breaks the ability of the packet capturing software to read the data.  Therefore, it is necessary to disable the driver functionality that performs TOE and allows the unprocessed TCP/IP packets to reach the operating system and packet capture software.

The ethtool package provides access to driver information and configurations.  Use the ethtool -k <interface_name> command to view the current settings, as this default for an Intel PRO/1000 MT Desktop (82540EM) NIC:
# ethtool -k eth1
Features for eth1:
rx-checksumming: off
tx-checksumming: on
    tx-checksum-ipv4: off [fixed]
    tx-checksum-unneeded: off [fixed]
    tx-checksum-ip-generic: on
    tx-checksum-ipv6: off [fixed]
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: on
    tx-scatter-gather: on
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
    tx-tcp-segmentation: on
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
 You may also check the ring parameters for the NIC with the ethtool -g <interface_name> command:
root@cou-gateway:~# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 256
RX Mini: 0
RX Jumbo: 0
TX: 256

The TOE parameters are a problem and packet capturing will not work correctly.  The TOE functionality must be disabled.  The ring parameters are not optimal because the NIC supports jumbo frames (4096), but the driver is configured for 256.  If you have configured Suricata to support jumbo frames, you may increase the receive ring parameters to 4096, if not, you may increase them to 1512.  A simple port-up script applied to the end of each Ethernet interface definition in /etc/network/interfaces will optimize the NIC for packet capture:
post-up ethtool -G $IFACE rx 1512; for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done
This is the output for a properly-configured packet capture NIC:

# ethtool -k eth0
Features for eth0:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-unneeded: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]

# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 1512
RX Mini: 0
RX Jumbo: 0
TX: 1512
Keep in mind that these necessary modifications will negatively impact system performance.  A live IDS/IPS system, with the large TCP/IP processing overhead, is one of the most hardware-intensive applications available.  It's hardware specifications are similar to (or exceed) a database server because it is so intensive of all major server components: processor, disk, network and memory IO.

Barnyard2

Barnyard2 reads Suricata log file output (in our case in the unified2 format), formats and exports it to a database (in our case MySQL).  The following series of commands will install Barnyard2:
apt-get install mysql-client libmysqlclient-dev libprelude-dev

git clone git://github.com/firnsy/barnyard2.git && cd barnyard2 && ./autogen.sh && ./configure --with-mysql --with-mysql-libraries=/usr/lib/x86_64-linux-gnu/ && make && make install

cd etc && cp barnyard2.conf /etc/suricata


Modify /etc/suricata/barnyard2.conf

Change:

config reference_file: /etc/snort/reference.config
config classification_file: /etc/snort/classification.config
config gen_file: /etc/snort/gen-msg.map
config sid_file: /etc/snort/sid-msg.map
To:
config reference_file: /etc/suricata/reference.config
config classification_file: /etc/suricata/classification.config
config gen_file: /etc/suricata/rules/gen-msg.map
config sid_file: /etc/suricata/rules/sid-msg.map
Change:
# output database: log, mysql, user=root password=test dbname=db host=localhost
To:
output database: log, mysql, user=root password=secretpassword dbname=snorby host=10.64.0.4
Change:
output alert_fast: stdout
To:
output alert_fast
Make sure the following lines are present:
config logdir: /var/log/suricata

config waldo_file: /var/log/suricata/barnyard2.waldo

config hostname: cou-ids
config interface: eth0
config daemon

Start the barnyard2 process in daemon mode with the following command:
#barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/suricata -f unified2.alert -D


Oinkmaster

Oinkmaster is a simple application that retrieves signature files for the Suricata IDS/IPS.  By default, Suricata looks to the /etc/suricata/rules directory for signatures and Oinkmaster will place them there.  The following command installs Oinkmaster:
#apt-get install oinkmaster
At this point, simply modify the /etc/oinkmaster.conf file to configure it to retrieve:
http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz
and place them in:
/etc/suricata/rules 

MySQL

The following command installs MySQL:
#apt-get install mysql-server mysql-client
By default, the mysql server listens on localhost only. Edit /etc/mysql/my.cnf to change the default behavior from listening on the bind-address 127.0.0.1 to the server's IP address:
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 10.64.0.4
#service mysql restart

Snorby

Snorby is a configuration and presentation application for Snort / Suricata data.  It requires the Apache web server with Ruby on Rails and the MySQL database.

Make sure the linux-headers for the system architecture are installed.

Debian Packages:

apt-get install gcc g++ build-essential libssl-dev libreadline6-dev zlib1g-dev libsqlite3-dev libxslt1-dev libxml2-dev imagemagick git-core libmysqlclient-dev mysql-server libmagickwand-dev default-jre ruby1.9.3 rubygems ruby-dev

Install wkhtmltopdf from Google Code:

wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.10.0_rc2-static-amd64.tar.bz2
bunzip2 wkhtmltopdf-0.10.0_rc2-static-amd64.tar.bz2
tar xvf wkhtmltopdf-0.10.0_rc2-static-amd64.tar
cp wkhtmltopdf-amd64 /usr/bin/wkhtmltopdf

Install Ruby Gems:

gem install thor i18n bundler tzinfo builder memcache-client rack rack-test erubis mail text-format rack-mount rails sqlite3

gem install rake --version=0.9.2

Install Snorby from github:

git clone http://github.com/Snorby/snorby.git /var/www/snorby

cp /var/www/snorby/config/database.yml.example /var/www/snorby/config/database.yml
cp /var/www/snorby/config/snorby_config.yml.example /var/www/snorby/config/snorby_config.yml


Edit /var/www/snorby/config/database.yml : look for the "snorby" entry and enter the mysql root username & password here :

nano /var/www/snorby/config/database.yml
snorby: &snorby
adapter: mysql
username: root
password: "mysqlrootpassword"
host: localhost

Edit /var/www/snorby/config/snorby_config.yml : set the correct path to wkhtmltopdf ( if you need to find it use which wkhtmltopdf ), make it look like this:

development:
domain: localhost:3000
wkhtmltopdf: /usr/bin/wkhtmltopdf

test:
domain: localhost:3000
wkhtmltopdf: /usr/bin/wkhtmltopdf

production:
domain: localhost:3000
wkhtmltopdf: /usr/bin/wkhtmltopdf

Then

cd /var/www/snorby
bundle update activesupport railties rails
gem install arel ezprint && bundle install
bundle exec rake snorby:setup



apt-get install apache2 apache2-prefork-dev libapr1-dev libaprutil1-dev libopenssl-ruby libcurl4-openssl-dev

service apache2 start

gem install --no-ri --no-rdoc passenger
/usr/local/bin/passenger-install-apache2-module -a


Edit /etc/apache2/mods-available/passenger.load (or create if it does not exits) :

to find what you need you can use :

find / -name "*mod_passenger*"
/var/lib/gems/1.9.1/gems/passenger-4.0.41/buildout/apache2/mod_passenger.so


Then put that in the file :

#nano /etc/apache2/mods-available/passenger.load
LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-4.0.41/buildout/apache2/mod_passenger.so

<IfModule mod_passenger.c>
PassengerRoot /var/lib/gems/1.9.1/gems/passenger-4.0.41
PassengerRuby /usr/bin/ruby
</IfModule>



#a2enmod passenger
#a2enmod rewrite
#a2enmod ssl
#chown www-data:www-data /var/www/snorby -R

Create a file "snorby" under /etc/apache2/sites-available :
#nano /etc/apache2/sites-available
<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName snorby.Server2
DocumentRoot /var/www/snorby/public

<Directory "/var/www/snorby/public">
AllowOverride all
Order deny,allow
Allow from all
Options -MultiViews
</Directory>

</VirtualHost>

Enable the new website :

#ln -s /etc/apache2/sites-available/snorby /etc/apache2/sites-enabled/snorby_config
#service apache2 restart
#cd /var/www/snorby
#bundle pack && bundle install --path vender/cache

Make sure cou-ids-web.mydomain.com points at your local apache2 server in the /etc/hosts file, and navigate to that website :

127.0.0.1 localhost
127.0.1.1 debian.localhost
127.0.0.1 cou-ids-web.mydomain.com

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

sudo service apache2 restart.

Conclusion

These two servers constitute a base IDS/IPS installation. We will look at the tuning and operational issues in subsequent articles.

Postscript

An example of Suricata successfully detecting an OpenVAS scan is available here.  It is quite apparent that something -- a port scan and vulnerability test -- is occurring.  However, OpenVAS is a rather blunt tool.  An attacker who is targeting web services will use a more focused tool.  An example of Suricata failing to detect an Arachni scan of Microsoft Exchange HTTPS is available here.  In this case, network intrusion detection fails; integration with a host intrusion detection system is warranted.