Difference between revisions of "Curl-loader: Install di Ubuntu"

Latest revision as of 13:42, 7 June 2015

Instalasi curl-loader

Cek Versi

uname -ri; tail -1 /etc/lsb-release

3.13.0-24-generic x86_64
DISTRIB_DESCRIPTION="Ubuntu 14.04 LTS"

Instalasi Aplikasi Pendukung

apt-get install libssl-dev build-essential

Download curl-loader

Dari

http://curl-loader.sourceforge.net/

Atau

cd /usr/local/src
wget http://sourceforge.net/projects/curl-loader/files/curl-loader-stable/curl-loader-0.56/curl-loader-0.56.tar.bz2

Instalasi

Lakukan

cd /usr/local/stc
bunzip2 curl-loader-0.56.tar.bz2
tar xvf curl-loader-0.56.tar
cd /usr/local/src/curl-loader-0.56/
make
make install

mkdir -p /usr/bin
mkdir -p /usr/share/man/man1
mkdir -p /usr/share/man/man5
mkdir -p /usr/share/doc/curl-loader/
cp -f curl-loader /usr/bin
cp -f doc/curl-loader.1 /usr/share/man/man1/  
cp -f doc/curl-loader-config.5 /usr/share/man/man5/
cp -f doc/* /usr/share/doc/curl-loader/
cp -rf conf-examples /usr/share/doc/curl-loader/

Help curl-load

curl-loader -h

Note, to run your load, create your batch configuration file.

usage: run as a root:
./curl-loader -f <configuration file name> with [other options below]:
 -c[onnection establishment timeout, seconds]
 -d[etailed logging; outputs to logfile headers and bodies of requests/responses. Good for text pages/files]
 -e[rror drop client (smooth mode). Client on error doesn't attempt next cycle]
 -i[ntermediate (snapshot) statistics time interval (default 3 sec)]
 -l[ogfile max size in MB (default 1024). On the size reached, file pointer rewinded]
 -m[ode of loading, 0 - hyper  (default), 1 - smooth]
 -r[euse onnections disabled. Close connections and re-open them. Try with and without]
 -t[hreads number to run batch clients as sub-batches in several threads. Works to utilize SMP/m-core HW]
 -v[erbose output to the logfiles; includes info about headers sent/received]
 -u[rl logging - logs url names to logfile, when -v verbose option is used]
 -w[arnings skip]
 -x[set|unset proxy] "<proxy:port>"

For more examples of configuration files please, look at "conf-examples" directory.

Running thousands and more clients, please do not forget to consider the options:
- to increase limit of open descriptors in shell by running e.g.    ulimit -n 19999:
- to increase total limit of  open descriptors in systeme somewhere in /proc
- to consider reusing sockets in time-wait state: by     echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
- and/or    echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

Ubah Sysctl

ulimit -n 10000
echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

Contoh Konfigurasi

cd /usr/local/src
ls curl-loader-0.56/conf-examples/

10K.conf                          delete.conf                  ipv6.conf                      resp-errors.conf
60K.conf                          fetch-probability.conf       log-hdr-body.conf              some.xml
bax.conf                          fetch-probability-once.conf  login-uas-cycling.conf         timers-random-range.conf
bax-login-logoff-noncycling.conf  ftp.conf                     login-uas-logoff-cycling.conf  url-randomize.conf
bax-logs.conf                     ftp-http.conf                multipart-formdata-post.conf   url-template-file.txt
big-load.conf                     ftp-upload.conf              post-form-tokens-fr-file.conf  url-template-fr-file.conf
bulk.conf                         get-forms.conf               post-xml.conf                  url-template-resp-dynamic.conf
bulk-dual-url.conf                get-post-login.conf          put.conf
credentials.cred                  get-post-login-cycling.conf  random_file_records.conf
custom-hdrs.conf                  https.conf                   rate-limit.conf

Membuat konfigurasi sendiri

cd /usr/local/src/curl-loader-0.56/conf-examples
cp bulk.conf ~
cd ~

Edit bulk.conf

vi bulk.conf

Isi dengan

########### GENERAL SECTION ################################

BATCH_NAME= bulk  
CLIENTS_NUM_MAX=200 # Same as CLIENTS_NUM
#CLIENTS_NUM_START=10
CLIENTS_RAMPUP_INC=5
INTERFACE=eth1
NETMASK=24
IP_ADDR_MIN= 192.168.0.90
IP_ADDR_MAX= 192.168.0.99
IP_SHARED_NUM=10
CYCLES_NUM= -1
URLS_NUM= 1

########### URL SECTION #################################### 

URL=http://192.168.0.80/wiki/index.php/Main_Page
#URL=http://localhost/apache2-default/ACE-INSTALL.html
#URL=http://localhost/ACE-INSTALL.html
URL_SHORT_NAME="local-apache"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 5000
TIMER_AFTER_URL_SLEEP = 500

Set ifconfig

ifconfig eth0:90 192.168.0.90
ifconfig eth0:91 192.168.0.91
ifconfig eth0:92 192.168.0.92
ifconfig eth0:93 192.168.0.93
ifconfig eth0:94 192.168.0.94
ifconfig eth0:95 192.168.0.95
ifconfig eth0:96 192.168.0.96
ifconfig eth0:97 192.168.0.97
ifconfig eth0:98 192.168.0.98
ifconfig eth0:99 192.168.0.99

run test

curl-loader -v -u -f ~/bulk.conf

Tampilan

============  loading batch is: bulk       ======================================
--------------------------------------------------------------------------------
Interval stats (latest:3 sec, clients:200, CAPS-curr:43):
H/F   Req:18,1xx:0,2xx:0,3xx:0,4xx:0,5xx:8,Err:10,T-Err:113,D:4711ms,D-2xx:0ms,Ti:3776B/s,To:5031B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
--------------------------------------------------------------------------------
Summary stats (runs:51 secs, CAPS-average:27):
H/F   Req:794,1xx:0,2xx:78,3xx:349,4xx:0,5xx:154,Err:16,T-Err:970,D:3011ms,D-2xx:2532ms,Ti:38039B/s,To:4322B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
=================================================================================
 Manual: clients:max[200],curr[200]. Inc num: [+|*].
=================================================================================

Setelah di tekan ^C

======= SIGINT Received ============.
H/F   Req:10,1xx:0,2xx:0,3xx:0,4xx:0,5xx:2,Err:8,T-Err:105,D:4397ms,D-2xx:0ms,Ti:944B/s,To:5375B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s

=======================================================================================
End of the test for batch: bulk      
=======================================================================================
 

Test total duration was 54 seconds and CAPS average 26:
H/F   Req:804,1xx:0,2xx:78,3xx:349,4xx:0,5xx:156,Err:24,T-Err:1075,D:3015ms,D-2xx:2532ms,Ti:35978B/s,To:4381B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s

Exited. For details look in the files:
- bulk.log for errors and traces;
- bulk.txt for loading statistics;
- bulk.ctx for virtual client based statistics.
- bulk.ops for operational statistics.
Add -v and -u options to the command line for verbose output to bulk.log file.

Yang penting di perhatikan kapan mulai ada

Err
T-Err

Log File

more bulk.log

Akan keluar

# 1433659405554 Sun Jun  7 13:43:25 2015
# msec_offset cycle_no url_no client_no (ip) indic info
0 0 0 1 == About to connect() to 192.168.0.80 port 80 (#0) eff-url: url http://192.168.0.80/wiki/index.php
0 0 0 2 == About to connect() to 192.168.0.80 port 80 (#1) eff-url: url http://192.168.0.80/wiki/index.php
1 0 0 3 == About to connect() to 192.168.0.80 port 80 (#2) eff-url: url http://192.168.0.80/wiki/index.php
1 0 0 4 == About to connect() to 192.168.0.80 port 80 (#3) eff-url: url http://192.168.0.80/wiki/index.php
2 0 0 5 == About to connect() to 192.168.0.80 port 80 (#4) eff-url: url http://192.168.0.80/wiki/index.php
73 0 0 1 !! RDR 301 eff-url: url http://192.168.0.80/wiki/index.php
77 0 0 3 !! RDR 301 eff-url: url http://192.168.0.80/wiki/index.php
78 0 0 2 !! RDR 301 eff-url: url http://192.168.0.80/wiki/index.php
80 0 0 4 !! RDR 301 eff-url: url http://192.168.0.80/wiki/index.php
80 0 0 5 !! RDR 301 eff-url: url http://192.168.0.80/wiki/index.php
453 0 0 5 !! OK 200 eff-url: url http://192.168.0.80/wiki/index.php/Main_Page url: url http://192.168.0.80/wiki/index.php
454 0 0 3 !! OK 200 eff-url: url http://192.168.0.80/wiki/index.php/Main_Page url: url http://192.168.0.80/wiki/index.php
455 0 0 1 !! OK 200 eff-url: url http://192.168.0.80/wiki/index.php/Main_Page url: url http://192.168.0.80/wiki/index.php
456 0 0 4 !! OK 200 eff-url: url http://192.168.0.80/wiki/index.php/Main_Page url: url http://192.168.0.80/wiki/index.php
457 0 0 2 !! OK 200 eff-url: url http://192.168.0.80/wiki/index.php/Main_Page url: url http://192.168.0.80/wiki/index.php
1014 0 0 6 == About to connect() to 192.168.0.80 port 80 (#5) eff-url: url http://192.168.0.80/wiki/index.php
1014 0 0 7 == About to connect() to 192.168.0.80 port 80 (#6) eff-url: url http://192.168.0.80/wiki/index.php
1015 0 0 8 == About to connect() to 192.168.0.80 port 80 (#7) eff-url: url http://192.168.0.80/wiki/index.php
1015 0 0 9 == About to connect() to 192.168.0.80 port 80 (#8) eff-url: url http://192.168.0.80/wiki/index.php
1016 0 0 10 == About to connect() to 192.168.0.80 port 80 (#9) eff-url: url http://192.168.0.80/wiki/index.php

Cek Access Log di Server

tail -5 /var/log/apache2/access.log

Akan terlihat akses dari banyak client

192.168.0.93 - - [07/Jun/2015:13:31:50 +0700] "GET /wiki/index.php/Main_Page HTTP/1.1" 200 20418 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
192.168.0.97 - - [07/Jun/2015:13:31:58 +0700] "GET /wiki/index.php/Main_Page HTTP/1.1" 200 20418 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
192.168.0.90 - - [07/Jun/2015:13:32:12 +0700] "GET /wiki/index.php/Main_Page HTTP/1.1" 500 1416 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
192.168.0.97 - - [07/Jun/2015:13:31:58 +0700] "GET /wiki/index.php/Main_Page HTTP/1.1" 200 20418 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
192.168.0.97 - - [07/Jun/2015:13:31:51 +0700] "GET /wiki/index.php/Main_Page HTTP/1.1" 200 20418 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

Referensi

http://lost-and-found-narihiro.blogspot.com/2014/09/http-traffic-generator-install-curl.html