Suricata: Instalasi Snorby & barnyard2

Cheat sheet : Installing Snorby 2.2 with Apache2 and Suricata with Barnyard2 on Ubuntu 10.x Published February 27, 2011 | By Corelan Team (corelanc0d3r) Introduction

After spending a few hours fighting a battle against Snorby and Apache2 + Passenger, I finally managed to get it to run properly on my Ubunty 10.x box (32bit). Looking back, I figured I might not be the only one who is having issues with this.

So I decided to publish the notes I took while setting everything up, and as a little bonus, explain how to install and configure Suricata as well (configured in combination with barnyard2 which will pick up local logs and send them to the remote MySQL server).

There are the components that will be installed :

   Snorby 2.x (latest revision from git)
   MySql 5
   Ruby 1.9.2p0
   Apache2
   Passenger 3
   Barnyard 2
   Suricata 1.1beta1 with emerging-threat ruleset

Install dependencies / prerequisites for Snorby Packages

First, make sure your system is up to date :

aptitude update apt-get update apt-get upgrade apt-get dist-upgrade

Then install new packages :

apt-get install gcc g++ build-essential libssl-dev libreadline5-dev \

    zlib1g-dev linux-headers-generic libsqlite3-dev libxslt-dev libxml2-dev \
    imagemagick git-core libmysqlclient-dev mysql-server libmagickwand-dev \
    default-jre

wkhtmlpdf with QT patch

cd /tmp wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.10.0_rc2-static-i386.tar.bz2 bunzip2 wkhtmltopdf-0.10.0_rc2-static-i386.tar.bz2 tar xvf wkhtmltopdf-0.10.0_rc2-static-i386.tar cp wkhtmltopdf-i386 /usr/bin/wkhtmltopdf

Ruby 1.9.2p0

cd /tmp wget http://ftp.ruby-lang.org//pub/ruby/1.9/ruby-1.9.2-p0.tar.gz tar -xvzf ruby-1.9.2-p0.tar.gz cd ruby-1.9.2-p0 ./configure make && make install ln -s /usr/local/ruby/bin/bundle /usr/bin

Run "ruby – v" and verify that it returns the correct version :

ruby 1.9.2p0 (2010-08-18 revision 29036) [i686-linux]

(If this shows a different version, then verify that /usr/local/ruby/bin/ruby -v is version 1.9.2p0) gems

gem install thor i18n bundler gem install tzinfo builder memcache-client rack rack-test erubis mail text-format gem install rack-mount --version=0.4.0 gem install rails sqlite3-ruby

Installing Snorby

git clone http://github.com/Snorby/snorby.git /var/www/snorby

Edit configuration files :

Edit /var/www/snorby/config/database.yml : look for the "snorby" entry and enter the mysql root username & password here :

snorby: &snorby

 adapter: mysql
 username: root
 password: <enter the mysql root password here>
 host: localhost

(don’t worry, we’ll get rid of the root username/password later on)

Edit /var/www/snorby/config/snorby_config.yml : set the correct path to wkhtmltopdf

development:

 domain: localhost:3000
 wkhtmltopdf: /usr/bin/wkhtmltopdf

test:

 domain: localhost:3000
 wkhtmltopdf: /usr/bin/wkhtmltopdf

production:

 domain: localhost:3000
 wkhtmltopdf: /usr/bin/wkhtmltopdf

Run Snorby setup :

cd /var/www/snorby rake snorby:setup

It is very likely that you will get the following error :

(in /var/www/snorby) You have requested:

 activesupport = 3.0.3

The bundle currently has activesupport locked at 3.0.4. Try running `bundle update activesupport` Try running `bundle install`.

Fix : run the following commands in the /var/www/snorby folder :

bundle update activesupport railties rails gem install arel gem install ezprint bundle install

Run the setup again :

cd /var/www/snorby rake snorby:setup

If all goes well, the snorby database should get created/populated now. Since we used the mysql root username/password in the database.yml configuration file, the necessary database and tables should be created successfully.

root@server:/var/www/snorby# rake snorby:setup (in /var/www/snorby) <...long key....> [datamapper] Created database 'snorby' [datamapper] Finished auto_upgrade! for :default repository 'snorby'

If you get an error about ezprint:

(in /var/www/snorby) rake aborted! http://github.com/mephux/ezprint.git (at rails3) is not checked out. \

  Please run `bundle install`

/var/www/snorby/Rakefile:4 (See full trace by running task with --trace)

Solution : run this from /var/www/snorby

bundle pack bundle install --path vender/cache

then run bundle install the rake snorby:setup command again

Configure mysql

We used the root user / password to allow snorby to create the necessary tables. If you prefer to use a mysql user account that has less privileges, then you can add a new mysql user, grant privileges, and edit the snorby configuration again :

mysql -u root -p

creat user 'snorbyuser'@'localhost' IDENTIFIED BY 'some_pass'; grant all privileges on snorby.* to 'snorbyuser'@'localhost' with grant option; flush privileges;

Now edit /var/www/snorby/config/database.yml again and replace the username and password with the newly created user

snorby: &snorby

 adapter: mysql
 username: snorbyuser
 password: some_pass
 host: localhost

You will need to create a useraccount for your (remote) suricata/snorby sensors too. The procedure is exactly the same as indicated above, but you will have to replace ‘localhost’ with the IP address of the remote sensor. If the sensor is local, you can use the snorbyuser@localhost mysql user account as well.

By default, the mysql server listens on localhost only. Edit /etc/mysql/my.cnf to change the default behaviour :

Instead of skip-networking the default is now to listen only on
localhost which is more compatible and is not less secure.

bind-address = 127.0.0.1

Comment the bind-address statement (add a # in front of the line) and restart mysql

service mysql restart

Verify that the server is now listening on all ip addresses :

root@server:/# lsof -i | grep mysqld mysqld 21309 mysql 10u IPv4 16405476 0t0 TCP *:mysql (LISTEN)

TCP *:mysql => listening on all interfaces

Apache2 & Passenger Install packages & dependencies

apt-get install apache2 apache2-prefork-dev libapr1-dev libaprutil1-dev libopenssl-ruby apt-get install libcurl4-openssl-dev

Start apache2 and make sure the default webpage loads

service apache2 start

Install passenger

gem install --no-ri --no-rdoc --version 3.0.3 passenger

Install passenger module for apache2

/usr/local/ruby/lib/ruby/gems/1.9.1/gems/passenger-3.0.3/bin/passenger-install-apache2-module -a

Edit /etc/apache2/mods-available/passenger.load (or create if it does not exits) :

LoadModule passenger_module /usr/local/ruby/lib/ruby/gems/1.9.1/gems/passenger-3.0.3/ext/apache2/mod_passenger.so

Edit /etc/apache2/mods-available/passenger.conf :

  PassengerRoot /usr/local/ruby/lib/ruby/gems/1.9.1/gems/passenger-3.0.3
  PassengerRuby /usr/local/ruby/bin/ruby

</IfModule>

Enable the module (and some other modules you might need) :

a2enmod passenger a2enmod rewrite a2enmod ssl

Set file/folder permissions on the snorby folder :

chown www-data:www-data /var/www/snorby -R

Integrate Snorby with Apache2

Suppose we want the snorby frontend to be reachable using virtualhost snorby.corelan.be :

Create a file "snorby" under /etc/apache2/sites-available :

       ServerAdmin webmaster@localhost
       ServerName snorby.corelan.be
       DocumentRoot /var/www/snorby/public

       <Directory "/var/www/snorby/public">
               AllowOverride all
               Order deny,allow
               Allow from all
               Options -MultiViews
       </Directory>

</VirtualHost>

Enable the new website :

ln -s /etc/apache2/sites-available/snorby /etc/apache2/sites-enabled/snorby

Restart apache2 :

service apache2 restart

Make sure snorby.corelan.be points at your local apache2 server, and navigate to that website :

image

(log in with user snorby@snorby.org and password snorby)

If you get an error page instead of the login page :

image

-> complaining about ezprint.git not being installed, then go to the /var/www/snorby folder and run the following 2 commands :

bundle pack bundle install --path vender/cache

Wait until the process has finished.

Restart apache2, and then try to access the website again, you should now be able to log on.

If you get a message about the "worker" not being started :

image

Solution : click "Administation", Click "Worker Options" Administration menu and select "Start worker".

Now click on "Worker Options" and start the 2 jobs

image

If you go back to the main page now, you may see "Currently caching" for a brief moment (depending on the number of events already in the database) :

image

   Tip : if, at any given time, the dashboard continues to show 0 events (or an incorrect number of events in general), but the Events view shows that all entries are inside the database, then you may have to clear the caches and rebuild it from scratch :

   mysql -u root -p

   use snorby;
   truncate table caches;
   exit

   Now remove the 2 worker jobs (use the little trash can icon next to each worker job to remove the job)

   image

   Recreate the jobs via Worker Options, and the main dashboard should eventually get populated again.

Okay, the server is now ready to receive data from local/remote sensors (Snort, Suricata, …).

Updating Snorby

Updating snorby is as easy as running the following commands :

cd /var/www/snorby git pull origin master rake snorby:update

Installing Suricata & Barnyard2 Dependencies

apt-get install libpcre3 libpcre3-dbg libpcre3-dev \

           build-essential autoconf automake libtool \
           libpcap-dev libnet1-dev mysql-client libmysqlclient16-dev

Set up yaml :

yaml : cd /tmp wget http://pyyaml.org/download/libyaml/yaml-0.1.3.tar.gz tar xvfz yaml-0.1.3.tar.gz cd yaml-0.1.3 ./configure && make && make install

Install barnyard2 :

cd /tmp wget http://www.securixlive.com/download/barnyard2/barnyard2-1.9.tar.gz tar xvfz barnyard2-1.9.tar.gz cd barnyard2-1.9 ./configure --with-mysql && make && make install

Do NOT delete the /tmp/barnyard2-1.9 folder yet. Install suricata:

cd /tmp wget http://www.openinfosecfoundation.org/download/suricata-1.1beta1.tar.gz tar xvfz suricata-1.1beta1.tar.gz cd suricata-1.1beta1 mkdir /var/log/suricata ./configure && make && make install

Do NOT remove the /tmp/suricata-1.1beta1 folder yet, we need some files from this folder later on.

Try to run suricata :

suricata

If you get the following message :

suricata: error while loading shared libraries: libhtp-0.2.so.1: cannot open shared object file: No such file or directory

then add "/usr/local/lib" to /etc/ld.so.conf and run ldconfig.

root@server:/# cat /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf /usr/local/lib root@server:/# ldconfig

Run "suricata" again :

suricata [14005] 27/2/2011 -- 22:08:28 - (suricata.c:440) <Info> (main) -- This is Suricata version 1.1beta1 [14005] 27/2/2011 -- 22:08:28 - (util-cpu.c:171) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 2 [14005] 27/2/2011 -- 22:08:28 - (suricata.c:765) <Error> (main) -- [ERRCODE: SC_ERR_OPENING_FILE(40)]

   - Configuration file has not been provided

Suricata 1.1beta1 USAGE: suricata

       -c <path>                    : path to configuration file
       -i <dev or ip>               : run in pcap live mode
       -r <path>                    : run in pcap file/offline mode
       -s <path>                    : path to signature file (optional)
       -l <dir>                     : default log directory
       -D                           : run as daemon
       --engine-analysis            : print reports on analysis of different sections in the engine and exit.
                                      Please have a look at the conf parameter engine-analysis on what reports
                                      can be printed
       --pidfile <file>             : write pid to this file (only for daemon mode)
       --init-errors-fatal          : enable fatal failure on signature init error
       --dump-config                : show the running configuration
       --pcap-buffer-size           : size of the pcap buffer value from 0 - 2147483647
       --user <user>                : run suricata as this user after init
       --group <group>              : run suricata as this group after init
       --erf-in <path>              : process an ERF file

To run the engine with default configuration on interface eth0 with signature file "signatures.rules", run the command as:

suricata -c suricata.yaml -s signatures.rules -i eth0

Get suricata rules (emerging-threats)

mkdir /etc/suricata cd /etc/suricata wget http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz tar xvfz emerging.rules.tar.gz

Configure suricata :

cd /tmp/suricata-1.1beta1 cp suricata.yaml /etc/suricata/ cp classification.config /etc/suricata/ cp reference.config /etc/suricata/

(note : After copying those files, you can remove the installation folder from /tmp)

Edit /etc/suricata/suricata.yaml

Make sure alert output for barnyard2 is enabled (it is enabled by default) :

 # alert output for use with Barnyard2
 - unified2-alert:
     enabled: yes
     filename: unified2.alert

     # Limit in MB.
     #limit: 32

Scroll down until you reach "default-rule-path:" and enable/put the emerging-threat rules files that are relevant to your system under "rule-files:". (You can find the list with rules under /etc/suricata/rules). Example :

default-rule-path: /etc/suricata/rules/ rule-files:

- emerging-attack_response.rules
- emerging-dos.rules
- emerging-exploit.rules
- emerging-games.rules
- emerging-inappropriate.rules
- emerging-malware.rules
- emerging-p2p.rules
- emerging-policy.rules
- emerging-scada.rules
- emerging-smtp.rules
- emerging-virus.rules
- emerging-voip.rules
- emerging-web_client.rules
- emerging-web_server.rules
- emerging-web_specific_apps.rules
- emerging-worm.rules
- emerging-user_agents.rules
- emerging-current_events.rules

Next, edit the HOME_NET variable and set it to your local IP or IP subnet

Example :

HOME_NET: "[192.168.0.0/24]"

That’s the basic config. Keeping suricata up to date

You can use this optional simple script to grab a copy of the git master and update the suricata binaries :

!/bin/bash

cd /tmp rm -rf /tmp/suricata mkdir suricata cd suricata /usr/bin/git clone git://phalanx.openinfosecfoundation.org/oisf.git cd oisf ./autogen.sh ./configure && make && make install

Configure barnyard2 :

Get the sample config file from the installation folder :

cp /tmp/barnyard2-1.9/etc/barnyard2.conf /etc/suricata/

(note : After copying the file, you can remove the installation folder from /tmp)

Edit the conf file and set the following parameters :

(we’ll assume you are installing suricata on the same box as the snorby engine)

config reference_file: /etc/suricata/reference.config config classification_file: /etc/suricata/classification.config config gen_file: /etc/suricata/rules/gen-msg.map config sid_file: /etc/suricata/rules/sid-msg.map

output database: log, mysql, user=snorbyuser password=some_pass /

  dbname=snorby host=localhost sensor_name=sensor1

(obviously the output database configuration must be placed on one line, remove the / between the password and dbname.)

If you are installing remote suricate sensors (remote from the mysql server / snorby engine point of view), then you will have to configure mysql and grant access to the remote mysqluser, from the IP of the sensor. The "host" entry in the barnyard2.conf file needs to point at the remote mysql server.

Finally, create the log folder for barnyard2 :

mkdir /var/log/barnyard2

Run barnyard2 :

barnyard2 -c /etc/suricata/barnyard2.conf -d /var/log/suricata -f unified2.alert -w /var/log/suricata/suricata.waldo -D

This will run barnyard2 in daemon mode. If barnyard2 does not appear to be working, omit the -D parameter and you will be able to see any errors that might prevent barnyard2 from running.

When barnyard2 is running, you should see a new sensor in Snorby. If you don’t like the display name of the sensor, you can change the name via Administration Menu – Sensors

When barnyard2 is running, you can launch suricata too : Run suricata :

suricata -c /etc/suricata/suricata.yaml -i eth0 -D

(change interface accordingly. -D will make suricate run in daemon mode)

As soon as suricata starts generating alerts, barnyard2 should pick them up, and use the mysql connector to write them into the events table of the snorby database. You should be able to see these new events in the "events" view of Snorby.

In the background (every 30 mins), the snorby worker jobs will pick up the events, process them, add them to the caches table, and show them on the dashboard too.

Test IDS

If you want to test your setup, then run :

lynx www.testmyids.com

(if lynx was not installed, run apt-get install lynx and try again)

Watch the /var/log/suricata folder. You should see something similar to this :

root@server:/var/log/suricata# ls -al total 88 drwxr-xr-x 2 root root 4096 2011-02-28 05:38 . drwxr-xr-x 18 root root 4096 2011-02-28 05:30 .. -rw-r----- 1 root root 194 2011-02-28 05:37 fast.log -rw-r----- 1 root root 0 2011-02-28 05:35 http.log -rw-r--r-- 1 root root 66873 2011-02-28 05:39 stats.log -rw------- 1 root root 2056 2011-02-28 05:38 suricata.waldo -rw-r----- 1 root root 0 2011-02-28 05:34 unified2.alert.1298867650 -rw-r----- 1 root root 60 2011-02-28 05:37 unified2.alert.1298867720

If the fast.log file, suricata.waldo and unified2.alert files are growing, then the IDS is picking up the test alerts from www.testmyids.com

Addendum (march 2nd 2011)

After running this setup for a few days, I noticed that Snorby only appears to be seeing "low severity" events, no matter how hard I try.

Something must be wrong.

I did some tests and it appears that suricata 1.1 might not be able to properly classify events.

I tried with snort, and that seems to work well.

Update : I filed a bug report with suricata and it looks like they have fixed the issue. You can use the suricata "update" script to grab the latest version.

Snort

Follow the installation guide at http://www.snort.org/assets/158/snortinstallguide2904.pdf with the exception of creating a "snort" database. Make sure to set up barnyard2 logging and tell it to use "snorby" instead of the "snort" database.

In short, the easiest way to get snort to run on ubuntu 10.x is to run apt-get install snort (or apt-get install snort-mysql if you want to have snort log events directly into the MySql database. This is not the recommended configuration and barnyard2 will be able to handle MySQL downtime)

   Twitter36
   Reddit
   Facebook4
   Google
   LinkedIn9
   StumbleUpon

Referensi

https://www.corelan.be/index.php/2011/02/27/cheat-sheet-installing-snorby-2-2-with-apache2-and-suricata-with-barnyard2-on-ubuntu-10-x/

Suricata: Instalasi Snorby & barnyard2

Referensi

Navigation menu

Search