Difference between revisions of "MRTG: Advanced"
Onnowpurbo (talk | contribs) |
Onnowpurbo (talk | contribs) |
||
Line 30: | Line 30: | ||
[root@bigboy tmp]# | [root@bigboy tmp]# | ||
− | + | Konfigurasi MIB adalah text file yang bisa di search menggunakan keyword di perintah grep. Contoh berikut mencari MIB yang berhubungan dengan connections, | |
grep -i tcp /usr/share/snmp/mibs/*.txt | grep connections | grep -i tcp /usr/share/snmp/mibs/*.txt | grep connections | ||
Line 43: | Line 43: | ||
[root@silent mibs]# | [root@silent mibs]# | ||
− | + | Kita bisa membaca MIB menggunakan text editor. JANGAN DIUBAH, karena mengubah akan menyebabkan SNMP gagal. MIB sangat kompelks, tapi untung-nya bagian penting biasanya di commented. | |
− | |||
− | + | Setiap nilai yang dilacak dalam MIB disebut object dan sering disebut dengan Object ID atau OID. Dalam cuplikan file RFC1213-MIB.txt ini, kita dapat melihat bahwa dengan meminta objek tcpActiveOpens mengembalikan jumlah koneksi TCP terbuka yang aktif ke server. Field SYNTAX menunjukkan bahwa ini adalah counter value. | |
+ | |||
+ | MIB biasanya melacak dua jenis nilai. Counter Value digunakan untuk nilai yang terus meningkat seiring berjalannya waktu, seperti jumlah paket yang melewati NIC atau jumlah waktu sibuk CPU sejak waktu boot. Nilai Integer berubah instant by instant dan berguna untuk melacak statistik seperti jumlah memori yang sedang digunakan. | ||
tcpActiveOpens OBJECT-TYPE | tcpActiveOpens OBJECT-TYPE | ||
Line 59: | Line 60: | ||
::= { tcp 5 } | ::= { tcp 5 } | ||
− | + | Kita perlu mengeksplorasi perbedaan antara terminologi SNMP dan MRTG lebih terinci nanti. Memahami mereka akan menjadi penting dalam memahami bagaimana menggunakan MRTG untuk track nilai MIB. | |
==Testing Your MIB Value== | ==Testing Your MIB Value== |
Revision as of 11:24, 19 May 2020
Dalam banyak hal menggunakan basic configuration MRTG untuk memantau volume lalu lintas jaringan ke server saja tidak cukup. Kita mungkin juga ingin melihat grafik penggunaan CPU, disk, dan memori. Bagian ini menjelaskan cara menemukan nilai yang ingin dipantau dalam file SNMP MIB dan kemudian bagaimana menggunakan informasi ini untuk mengonfigurasi MRTG.
Dalam bagian ini contoh yang digunakan mengasumsikan SNMP Read Only string adalah craz33guy dan paket snmpd snmp libsnmp-dev di Ubuntu atau paket net-snmp-utils RPM di Fedora.
Di Ubuntu 20.04, instalasi menggunakan
apt update apt -y install snmpd snmp libsnmp-dev
Mencari dan Melihat isi MIB
Berada di memori, MIB adalah struktur data yang terus diperbarui melalui daemon SNMP. File text konfigurasi MIB terletak di hard disk dan dimuat ke dalam memori setiap kali SNMP restart.
Kita dapat dengan mudah menemukan MIB di Linux dengan menggunakan perintah locate dan memfilter output untuk menyertakan hanya nilai dengan kata "snmp" di dalamnya. Seperti yang dapat Anda lihat dalam kasus ini, MIB terletak di direktori /usr/share/snmp/mibs:
locate mib | grep snmp
[root@bigboy tmp]# locate mib | grep snmp /usr/share/doc/net-snmp-5.0.6/README.mib2c /usr/share/snmp/mibs /usr/share/snmp/mibs/DISMAN-SCHEDULE-MIB.txt ... ... [root@bigboy tmp]#
Konfigurasi MIB adalah text file yang bisa di search menggunakan keyword di perintah grep. Contoh berikut mencari MIB yang berhubungan dengan connections,
grep -i tcp /usr/share/snmp/mibs/*.txt | grep connections
... RFC1213-MIB.txt: "The limit on the total number of TCP connections RFC1213-MIB.txt: "The number of times TCP connections have made a ... TCP-MIB.txt: "The number of times TCP connections have made a ... ... [root@silent mibs]#
Kita bisa membaca MIB menggunakan text editor. JANGAN DIUBAH, karena mengubah akan menyebabkan SNMP gagal. MIB sangat kompelks, tapi untung-nya bagian penting biasanya di commented.
Setiap nilai yang dilacak dalam MIB disebut object dan sering disebut dengan Object ID atau OID. Dalam cuplikan file RFC1213-MIB.txt ini, kita dapat melihat bahwa dengan meminta objek tcpActiveOpens mengembalikan jumlah koneksi TCP terbuka yang aktif ke server. Field SYNTAX menunjukkan bahwa ini adalah counter value.
MIB biasanya melacak dua jenis nilai. Counter Value digunakan untuk nilai yang terus meningkat seiring berjalannya waktu, seperti jumlah paket yang melewati NIC atau jumlah waktu sibuk CPU sejak waktu boot. Nilai Integer berubah instant by instant dan berguna untuk melacak statistik seperti jumlah memori yang sedang digunakan.
tcpActiveOpens OBJECT-TYPE SYNTAX Counter ACCESS read-only STATUS mandatory DESCRIPTION "The number of times TCP connections have made a direct transition to the SYN-SENT state from the CLOSED state." ::= { tcp 5 }
Kita perlu mengeksplorasi perbedaan antara terminologi SNMP dan MRTG lebih terinci nanti. Memahami mereka akan menjadi penting dalam memahami bagaimana menggunakan MRTG untuk track nilai MIB.
Testing Your MIB Value
Once you have identified an interesting MIB value for your Linux system you can then use the snmpwalk command to poll it. Many times the text aliases in a MIB only reference the OID branch and not the OID the data located in a leaf ending in an additional number like a ".0" or ".1". The snmpget command doesn't work with branches giving an error stating that the MIB variable couldn't be found.
In the example below, the ssCpuRawUser OID alias was found to be interesting, but the snmpget command fails to get a value. Follow up with the snmpwalk command shows that the value is located in ssCpuRawUser.0 instead. The snmpget is then successful in retrieving the "counter32" type data with a current value of 396271.
[root@bigboy tmp]# snmpget -v1 -c craz33guy localhost ssCpuRawUser Error in packet Reason: (noSuchName) There is no such variable name in this MIB. Failed object: UCD-SNMP-MIB::ssCpuRawUser [root@bigboy tmp]# [root@bigboy tmp]# snmpwalk -v1 -c craz33guy localhost ssCpuRawUser UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 396241 [root@bigboy tmp]# snmpget -v1 -c craz33guy localhost ssCpuRawUser.0 UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 396271 [root@bigboy tmp]#
The MIB values that work successfully with snmpget are the ones you should use with MRTG.
Differences In MIB And MRTG Terminology
Always keep in mind that MRTG refers to MIB counter values as counter values. It refers to MIB integer and gauge values as gauge. By default, MRTG considers all values to be counters.
MRTG doesn't plot counter values as a constantly increasing graph, it plots only how much the value has changed since the last polling cycle. CPU usage is typically tracked by MIBs as a counter value; fortunately, you can edit your MRTG configuration file to make it graph this information in a percentage use format (more on this later).
The syntax type, the MIB object name, and the description of what it does are the most important things you need to know when configuring MRTG; I'll come back to these later. The CPU And Memory Monitoring MIB
The UCD-SNMP-MIB MIB keeps track of a number of key performance MIB objects, including the commonly used ones in Table 23-1.
Table 23-1 Important Objects In The UCD-SNMP-MIB MIB UCD-SNMP-MIB Object Variable MIB Type MRTG Type Description ssCpuRawUser Counter Counter
Total CPU usage by applications run by nonprivileged users since the system booted. Adding the user, system, and nice values can give a good approximation of total CPU usage..
ssCpuRawSystem Counter Counter Total CPU usage by applications run by privileged system processes since the system booted. ssCpuRawNice Counter Counter Total CPU usage by applications running at a nondefault priority level. ssCpuRawIdle Counter Counter
The percentage of the time the CPU is running idle. Subtracting this value from 100 can give a good approximation of total CPU usage.
memAvailReal Integer Gauge Available Physical Memory Space On The Host
The TCP/IP Monitoring MIB
The TCP-MIB MIB keeps track of data connection information and contains the very useful tcpActiveOpens and tcpCurrEstab objects. Table 23.2 details the most important objects in TCP-MIB. Table 23-2 Important Objects In The TCP-MIB MIB
UCD-SNMP-MIB Object Variable MIB Type MRTG Type Description tcpActiveOpens Counter Counter Measures the number of completed TCP connections. tcpCurrEstab Gauge Gauge Measures the number of TCP connections in the established state tcpInErrs Counter Counter Total number of TCP segments with bad checksum errors
Manually Configuring Your MRTG File
The MRTG cfgmaker program creates configuration files for network interfaces only, simultaneously tracking two OIDs: the NIC's input and output data statistics. The mrtg program then uses these configuration files to determine the type of data to record in its data directory. The indexmaker program also uses this information to create the overview, or Summary View Web page for the MIB OIDs you're monitoring.
This Summary View page shows daily statistics only. You have to click on the Summary View graphs to get the Detailed View page behind it with the daily, weekly, monthly, and annual graphs. Some of the parameters in the configuration file refer to the Detailed View, others refer to the Summary View.
If you want to monitor any other pairs of OIDs, you have to manually create the configuration files, because cfgmaker isn't aware of any OIDs other than those related to a NIC. The mrtg and indexmaker program can be fed individual OIDs from a customized configuration file and will function as expected if you edit the file correctly.
Parameter Formats
MRTG configuration parameters are always followed by a graph name surrounded by square brackets and a colon. The format looks like this:
Parameter[graph name]: value
For ease of editing, the parameters for a particular graph are usually grouped together. Each graph can track two OIDS listed in the Target parameter, which is usually placed at the very top of the graph name list. The two OID values are separated by an & symbol; the first one can be is the input OID, and the second one is the output OID.
Legend Parameters
On the Detailed View Web page, each graph has a legend that shows the max, average, and current values of the graph's OID statistics. You can use the legendI parameter for the description of the input graph (first graph OID) and the legendO for the output graph (second graph OID).
The space available under each graph's legend is tiny so MRTG also has legend1 and legend2 parameters that are placed at the very bottom of the page to provide more details. Parameter legend1 is the expansion of legendI, and legend2 is the expansion of legendO.
The Ylegend is the legend for the Y axis, the value you are trying to compare. In the case of a default MRTG configuration this would be the data flow through the interface in bits or bytes per second. Here is an example of the legends of a default MRTG configuration:
YLegend[graph1]: Bits per second Legend1[graph1]: Incoming Traffic in Bits per Second Legend2[graph1]: Outgoing Traffic in Bits per Second LegendI[graph1]: In LegendO[graph1]: Out
You can prevent MRTG from printing the legend at the bottom of the graph by leaving the value of the legend blank like this:
LegendI[graph1]:
Later you'll learn how to match the legends to the OIDs for a variety of situations.
Options Parameters
Options parameters provide MRTG with graph formatting information. The growright option makes sure the data at the right of the screen is for the most current graph values. This usually makes the graphs more intuitively easy to read. MRTG defaults to growing from the left.
The nopercent option prevents MRTG from printing percentage style statistics in the legends at the bottom of the graph. The gauge option alerts MRTG to the fact that the graphed values are of the gauge type. If the value you are monitoring is in bytes, then you can convert the output to bits using the bits option. Likewise, you can convert per second values to per minute graphs using the perminute option. Here are some examples for two different graphs:
options[graph1]: growright,nopercent,perminute options[graph2]: gauge,bits
If you place this parameter at the top with a label of [_] it gets applied to all the graphs defined in the file. Here's an example.
options[_]: growright
Title Parameters
The title on the Summary Page is provided by the Title parameter, the PageTop parameter tells the title for the Detailed View page. The PageTop string must start with < H1 > and end with < H1 >.
Title[graph1]: Interface eth0 PageTop[graph1]: < H1 >Detailed Statistics For Interface eth0 < H1 >
Scaling Parameters
The MaxBytes parameter is the maximum amount of data MRTG will plot on a graph. Anything more than this seems to disappear over the edge of the graph.
MRTG also tries to adjust its graphs so that the largest value plotted on the graph is always close to the top. This is so even if you set the MaxBytes parameter.
When you are plotting a value that has a known maximum and you always want to have this value at the top of the vertical legend, you may want to turn off MRTG's auto scaling. If you are plotting percentage CPU usage, and the server reaches a maximum of 60%, with scaling, MRTG will have a vertical plot of 0% to 60%, so that the vertical peak is near the top of the graph image.
When scaling is off, and MaxBytes is set to 100, then the peak will be only 60% of the way up as the graph plots from 0% to 100%. The example removes scaling from the yearly, monthly, weekly, and daily views on the Detailed View page and gives them a maximum value of 100.
Unscaled[graph1]: ymwd MaxBytes[graph1]: 100
Defining The MIB Target Parameters
As stated before, MRTG always tries to compare two MIB OID values that are defined by the Target parameter. You have to specify the two MIB OID objects, the SNMP password and the IP address of the device you are querying in this parameter, and separate them with an & character:
Target[graph1]: mib-object-1.0&mib-object-2.0:<SNMP-password>@<IP-address>
The numeric value, in this case .0, at the end of the MIB is required. The next example uses the SNMP command to return the user mode CPU utilization of a Linux server. Notice how the .0 is tagged onto the end of the output.
[root@silent mibs]# snmpwalk -v 1 -c craz33guy localhost ssCpuRawUser UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 926739 [root@silent mibs]#
The MRTG legends map to the MIBs listed in the target as shown in Table 23-3.
Table 23-3 Mapping MIBs To The Graph Legends Legend Maps To Target MIB Legend1 #1 Legend2 #2 LegendI #1 LegendO #2
So in the example below, legend1 and legendI describe mib-object-1.0 and legend2 and legendO describe mib-object-2.0.
Target[graph1]: mib-object-1.0&mib-object-2.0:<SNMP-password>@<IP-address>
Plotting Only One MIB Value
If you want to plot only one MIB value, you can just repeat the target MIB in the definition as in the next example, which plots only mib-object-1. The resulting MRTG graph actually superimposes the input and output graphs one on top of the other.
Target[graph1]: mib-object-1.0&mib-object-1.0:<SNMP-password>@<IP-address>
Adding MIB Values Together For a Graph
You can use the plus sign between the pairs of MIB object values to add them together. The next example adds mib-object-1.0 and mib-object-3.0 for one graph and adds mib-object-2.0 and mib-object-4.0 for the other.
Target[graph1]: mib-object-1.0&mib-object-2.0:<SNMP-password>@<IP-address> + mib-object-3.0&mib-object-4.0:<SNMP-password>@<IP-address>
You can use other mathematical operators, such as subtract (-), multiply (*), and divide (%). Left and right parentheses are also valid. There must be white spaces before and after all these operators for MRTG to work correctly. If not, you'll get oddly shaded graphs.
Sample Target: Total CPU Usage
Linux CPU usage is occupied by system processes, user mode processes, and a few processes running in nice mode. This example adds them all together in a single plot.
Target[graph1]:ssCpuRawUser.0&ssCpuRawUser.0:<SNMP-password>@<IP-address> + ssCpuRawSystem.0&ssCpuRawSystem.0:<SNMP-password>@<IP-address> + ssCpuRawNice.0&ssCpuRawNice.0:<SNMP-password>@<IP-address>
Be sure to place this command on a single line
Sample Target: Memory Usage
Here is an example for the plotting the amount of free memory versus the total RAM installed in the server. Notice that this is a gauge type variable.
Target[graph1]: memAvailReal.0&memTotalReal.0:<SNMP-password>@<IP-address> options[graph1]: nopercent,growright,gauge
Next, plot the percentage of available memory. Notice how the mandatory white spaces separate the mathematical operators from the next target element.
Target[graph1]: ( memAvailReal.0& memAvailReal.0:<SNMP-password>@<IP-Address> ) * 100 / ( memTotalReal.0&memTotalReal.0:<SNMP-password>@<IP-Address> ) options[graph1]: nopercent,growright,gauge
Sample Target: Newly Created Connections
HTTP traffic caused by Web browsing usually consists of many very short lived connections. The tcpPassiveOpens MIB object tracks newly created connections and is suited for this type of data transfer. The tcpActiveOpens MIB object monitors new connections originating from the server. On smaller Web sites you may want to use the perminute option to make the graphs more meaningful.
Target[graph1]: tcpPassiveOpens.0& tcpPassiveOpens.0:<SNMP-password>@<IP-address> MaxBytes[graph1]: 1000000 Options[graph1]: perminute
Sample Target: Total TCP Established Connections
Other protocols such as FTP and SSH create longer established connections while people download large files or stay logged into the server. The tcpCurrEstab MIB object measures the total number of connections in the established state and is a gauge value.
Target[graph1]: tcpCurrEstab.0&tcpCurrEstab.0:<SNMP-password>@<IP-address> MaxBytes[graph1]: 1000000 Options[graph1]: gauge
Sample Target: Disk Partition Usage
In this example, you'll monitor the /var and /home disk partitions on the system.
1) First use the df -k command to get a list of the partitions in use.
[root@bigboy tmp]# df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/hda8 505605 128199 351302 27% / /dev/hda1 101089 19178 76692 21% /boot /dev/hda5 1035660 122864 860188 13% /home /dev/hda6 505605 8229 471272 2% /tmp /dev/hda3 3921436 890092 2832140 24% /usr /dev/hda2 1510060 171832 1261520 73% /var [root@bigboy tmp]#
2) Add two entries to your snmpd.conf file.
disk /home disk /var
3) Restart the SNMP daemon to reload the values.
[root@bigboy tmp]# systemctl restart snmpd.service
4) Use the snmpwalk command to query the the dskPercent MIB. Object dskPercent.1 refers to the first disk entry in snmpd.conf (/home), and dskPercent.2 refers to the second (/var).
[root@bigboy tmp]# snmpwalk -v 1 -c craz33guy localhost dskPercent.1 UCD-SNMP-MIB::dskPercent.1 = INTEGER: 13 [root@bigboy tmp]# snmpwalk -v 1 -c craz33guy localhost dskPercent.2 UCD-SNMP-MIB::dskPercent.2 = INTEGER: 73 [root@bigboy tmp]#
Your MRTG target for these gauge MIB objects should look like this:
Target[graph1]: dskPercent.1& dskPercent.1:<SNMP-password>@<IP-address> options[graph1]: growright,gauge
Defining Global Variables
You have to make sure MRTG knows where the MIBs you're using are located. The default location MRTG uses may not be valid. Specify their locations with the global LoadMIBs parameter. You must also define where the HTML files will be located; the example specifies the default Fedora MRTG HTML directory.
LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt, /usr/share/snmp/mibs/TCP-MIB.txt workdir: /var/www/mrtg/
Implementing Advanced Server Monitoring
You now can combine all you have learned to create a configuration file that monitors all these variables, and then you can integrate it into the existing MRTG configuration.
A Complete Sample Configuration
Here is a sample configuration file that is used to query server localhost for CPU, memory, disk, and TCP connection information.
# # File: /etc/mrtg/server-info.cfg # # Configuration file for non bandwidth server statistics # # # Define global options # LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt,/usr/share/snmp/mibs/TCP-MIB.txt workdir: /var/www/mrtg/ # # CPU Monitoring # (Scaled so that the sum of all three values doesn't exceed 100) # Target[server.cpu]:ssCpuRawUser.0&ssCpuRawUser.0:craz33guy@localhost + ssCpuRawSystem.0&ssCpuRawSystem.0:craz33guy@localhost + ssCpuRawNice.0&ssCpuRawNice.0:craz33guy@localhost Title[server.cpu]: Server CPU Load PageTop[server.cpu]: < H1 >CPU Load - System, User and Nice Processes< /H1 > MaxBytes[server.cpu]: 100 ShortLegend[server.cpu]: % YLegend[server.cpu]: CPU Utilization Legend1[server.cpu]: Current CPU percentage load LegendI[server.cpu]: Used LegendO[server.cpu]: Options[server.cpu]: growright,nopercent Unscaled[server.cpu]: ymwd # # Memory Monitoring (Total Versus Available Memory) # Target[server.memory]: memAvailReal.0&memTotalReal.0:craz33guy@localhost Title[server.memory]: Free Memory PageTop[server.memory]: < H1 >Free Memory< /H1 > MaxBytes[server.memory]: 100000000000 ShortLegend[server.memory]: B YLegend[server.memory]: Bytes LegendI[server.memory]: Free LegendO[server.memory]: Total Legend1[server.memory]: Free memory, not including swap, in bytes Legend2[server.memory]: Total memory Options[server.memory]: gauge,growright,nopercent kMG[server.memory]: k,M,G,T,P,X # # Memory Monitoring (Percentage usage) # Title[server.mempercent]: Percentage Free Memory PageTop[server.mempercent]: < H1 >Percentage Free Memory< /H1 > Target[server.mempercent]: ( memAvailReal.0&memAvailReal.0:craz33guy@localhost ) * 100 / ( memTotalReal.0&memTotalReal.0:craz33guy@localhost ) options[server.mempercent]: growright,gauge,transparent,nopercent Unscaled[server.mempercent]: ymwd MaxBytes[server.mempercent]: 100 YLegend[server.mempercent]: Memory % ShortLegend[server.mempercent]: Percent LegendI[server.mempercent]: Free LegendO[server.mempercent]: Free Legend1[server.mempercent]: Percentage Free Memory Legend2[server.mempercent]: Percentage Free Memory # # New TCP Connection Monitoring (per minute) # Target[server.newconns]: tcpPassiveOpens.0&tcpActiveOpens.0:craz33guy@localhost Title[server.newconns]: Newly Created TCP Connections PageTop[server.newconns]: < H1 >New TCP Connections< /H1 > MaxBytes[server.newconns]: 10000000000 ShortLegend[server.newconns]: c/s YLegend[server.newconns]: Conns / Min LegendI[server.newconns]: In LegendO[server.newconns]: Out Legend1[server.newconns]: New inbound connections Legend2[server.newconns]: New outbound connections Options[server.newconns]: growright,nopercent,perminute # # Established TCP Connections # Target[server.estabcons]: tcpCurrEstab.0&tcpCurrEstab.0:craz33guy@localhost Title[server.estabcons]: Currently Established TCP Connections PageTop[server.estabcons]: < H1 >Established TCP Connections< /H1 > MaxBytes[server.estabcons]: 10000000000 ShortLegend[server.estabcons]: YLegend[server.estabcons]: Connections LegendI[server.estabcons]: In LegendO[server.estabcons]: Legend1[server.estabcons]: Established connections Legend2[server.estabcons]: Options[server.estabcons]: growright,nopercent,gauge # # Disk Usage Monitoring # Target[server.disk]: dskPercent.1&dskPercent.2:craz33guy@localhost Title[server.disk]: Disk Partition Usage PageTop[server.disk]: < H1 >Disk Partition Usage /home and /var< /H1 > MaxBytes[server.disk]: 100 ShortLegend[server.disk]: % YLegend[server.disk]: Utilization LegendI[server.disk]: /home LegendO[server.disk]: /var Options[server.disk]: gauge,growright,nopercent Unscaled[server.disk]: ymwd
Testing The Configuration
The next step is to test that MRTG can load the configuration file correctly.
Restart SNMP to make sure the disk monitoring commands in the snmpd.conf file are activated. Run the /usr/bin/mrtg command followed by the name of the configuration file three times. If all goes well, MRTG will complain only about the fact that certain database files don't exist. MRTG then creates the files. By the third run, all the files are created and MRTG should operate smoothly.
[root@bigboy tmp]# systemctl restart snmpd.service [root@bigboy tmp]# env LANG=C /usr/bin/mrtg /etc/mrtg/server-stats.cfg
Creating A New MRTG Index Page To Include This File
Use the indexmaker command and include your original MRTG configuration file from Chapter 22, " Monitoring Server Performance", (/etc/mrtg/mrtg.cfg) plus the new one you created (/etc/mrtg/server-stats.cfg).
[root@bigboy tmp]# indexmaker --output=/var/www/mrtg/index.html \ /etc/mrtg/mrtg.cfg /etc/mrtg/server-stats.cfg
Configuring cron To Use The New MRTG File
The final step is to make sure that MRTG is configured to poll your server every five minutes using this new configuration file. To do so, add this line to your /etc/cron.d/mrtg file.
0-59/5 * * * * root env LANG=C /usr/bin/mrtg /etc/mrtg/server-stats.cfg
Some versions of Linux require you to edit your /etc/crontab file instead. See Chapter 22, " Monitoring Server Performance", for more details. You will also have to restart cron with the service crond restart for it to read its new configuration file that tells it to additionally run MRTG every five minutes using the new MRTG configuration file.
[root@bigboy tmp]# systemctl restart crond.service
Monitoring Non Linux MIB Values
All the MIBs mentioned so far are for Linux systems; other types of systems will need additional MIBs whose correct installation may be unclear in user guides or just not available. In such cases, you'll need to know the exact value of the OID. Scenario
Imagine that your small company has purchased a second-hand Cisco switch to connect its Web site servers to the Internet. The basic MRTG configuration shown in Chapter 22, " Monitoring Server Performance", provides the data bandwidth statistics, but you want to measure the CPU load the traffic is having on the device, as well. Downloading MIBs from Cisco and using them with the snmpget command was not a success. You do not know what to do next. Find The OIDs
When MIB values fail, it is best to try to find the exact OID value. Like most network equipment manufacturers, Cisco has an FTP site from which you can download both MIBs and OIDs. The SNMP files for Cisco's devices can be found at ftp.cisco.com in the /pub/mibs directory; OIDs are in the oid directory beneath that.
After looking at all the OID files, you decide that the file CISCO-PROCESS-MIB.oid will contain the necessary values and find these entries inside it.
"cpmCPUTotalPhysicalIndex" "1.3.6.1.4.1.9.9.109.1.1.1.1.2" "cpmCPUTotal5sec" "1.3.6.1.4.1.9.9.109.1.1.1.1.3" "cpmCPUTotal1min" "1.3.6.1.4.1.9.9.109.1.1.1.1.4" "cpmCPUTotal5min" "1.3.6.1.4.1.9.9.109.1.1.1.1.5" "cpmCPUTotal5secRev" "1.3.6.1.4.1.9.9.109.1.1.1.1.6" "cpmCPUTotal1minRev" "1.3.6.1.4.1.9.9.109.1.1.1.1.7" "cpmCPUTotal5minRev" "1.3.6.1.4.1.9.9.109.1.1.1.1.8"
Testing The OIDs
As you can see, all the OIDs are a part of the same tree starting with 1.3.6.1.4.1.9.9.109.1.1.1.1. The OIDs provided may be incomplete, so it is best to use the snmpwalk command to try to get all the values below this root first.
[root@bigboy tmp]# snmpwalk -v1 -c craz33guy cisco-switch 1.3.6.1.4.1..9.9.109.1.1.1.1 SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.2.1 = INTEGER: 0 SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.3.1 = Gauge32: 32 SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.4.1 = Gauge32: 32 SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.5.1 = Gauge32: 32 [root@bigboy tmp]#
Although listed in the OID file, 1.1.1.1.6, 1.1.1.1.7, and 1.1.1.1.8 are not supported. Notice also how SNMP has determined that the first part of the OID value (1.3.6.1.4.1) in the original OID file maps to the word "enterprise".
Next, you can use one the snmpget command to set only one of the OID values returned by snmpwalk.
[root@bigboy tmp]# snmpget -v1 -c craz33guy cisco-switch \ enterprises.9.9.109.1.1.1.1.5.1 SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.5.1 = Gauge32: 33 [root@bigboy tmp]#
Success! Now you can use this OID value, enterprises.9.9.109.1.1.1.1.5.1, for your MRTG queries.
Speeding up MRTG with RRDtool
MRTG is a very useful program but it has a limitation. All the graphs and web pages are recreated each time a device is polled. This can potentially overload your MRTG server especially if you have a large number of monitored devices and the graphs take more than five minutes to generate. RRDtool is an application written by the creator of MRTG that can store general purpose data, but generates graphs on demand. Integrating MRTG with RRDtool can have very noticeable performance benefits. The example that follows will show you how to quickly implement a general purpose solution.
Scenari
The use of RRDtool is needed to reduce the load on a monitoring server that has been experiencing very sluggish performance due to the amount of MRTG graphs it has to regenerate every polling cycle.
- Due to space constraints, the RRD database needs to be located in the /var partition.
- The server has a default Apache configuration with the CGI files needed for dynamically generated content being located in the /var/www/cgi-bin directory.
- A CGI script is required that will read the new MRTG data in RRDtool format.
- The MRTG configuration file is /etc/mrtg/mrtg.cfg.
Here's how to proceed.
Installing RRDtool
The RRDtool and RRDtool PERL module file can be downloaded from its website at http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/, but installation can be tricky as the installation program may look for certain supporting libraries in the wrong directories.
Fortunately the prerequisite rrdtool and rrdtool-perl packages now come as part of most Linux distributions. For more details on installing packages, see Chapter 6, "Installing Linux Software"). Storing the MRTG Data in RRDtool Format
This phase of the integration process can be done in a few minutes, but the steps can be tricky:
The first step is to add some new options to your cfgmaker command. The first indicates that MRTG should only store rrdtool formatted data, and the second defines the /var/mrtg directory in which it should be stored. For added security, the directory should be external to your web server's document root.
--global 'LogFormat: rrdtool' --global "workdir: /var/mrtg" --global 'IconDir: /mrtg'
Finally, you should also specify an icon directory which specifies the location of all miscellaneous MRTG web page icons. The RRD web interface script we'll install later uses an incorrect location. The icon directory /mrtg is actually a partial URL location. In this Fedora scenario we are using the default Apache configuration which locates the MRTG icon files in the /var/www/mrtg directory. If you are using a non default Apache MRTG configuration or are using other Linux distributions or versions you may have to copy the icons to the custom directory in which the MRTG PNG format icon files are located.
The cfgmaker program is simple to use and is covered in in Chapter 22, "Monitoring Server Performance".
The next step is to create the data repository directory /var/mrtg and make it be owned by the apache user and process that runs the default Linux web server application.
[root@bigboy tmp]# mkdir /var/mrtg [root@bigboy tmp]# chown apache /var/mrtg [root@bigboy tmp]#
Note: If you are using SELinux you'll have to change the context of this directory to match that of the /var/www/html directory so that the apache process will be able to read the database files when your CGI script needs them. These commands compare the contexts of the both directories and apply the correct set to /var/mrtg.
Please refer to Chapter 20, " The Apache Web Server" for more details on file contexts with Apache.
[root@bigboy tmp]# ls -alZ /var/www | grep html drwxr-xr-x root root system_u:object_r:httpd_sys_content_t html [root@bigboy tmp]# ls -alZ /var | grep mrtg drwxr-xr-x apache root root:object_r:var_t mrtg [root@bigboy tmp]# chcon -R -u system_u -r object_r -t httpd_sys_content_t /var/mrtg [root@bigboy tmp]#
We now need to test that the RRD files are being created correctly. Run MRTG using the /etc/mrtg/mrtg.cfg file as the source configuration file then test to see if the contents of the /var/mrtg directory have changed. Success!
[root@bigboy tmp]# ls /var/mrtg/ localhost_192.168.1.100.rrd [root@bigboy tmp]#
The files are being created properly. Now we need to find a script to read the new data format and present it in a web format. This will be discussed next.
The MRTG / RRDtool Integration Script
The MRTG website recommends the script located on the mrtg-rrd website (http://www.fi.muni.cz/~kas/mrtg-rrd/) as being a good one to use. Let's go ahead and install it.
Download the script using wget. The site lists several versions; make sure you get the latest one.
[root@bigboy tmp]# wget ftp://ftp.linux.cz/pub/linux/people/jan_kasprzak/mrtg-rrd/mrtg-rrd-0.7.tar.gz --12:42:12-- ftp://ftp.linux.cz/pub/linux/people/jan_kasprzak/mrtg-rrd/mrtg-rrd-0.7.tar.gz
=> `mrtg-rrd-0.7.tar.gz'
Resolving ftp.linux.cz... 147.251.48.205 Connecting to ftp.linux.cz|147.251.48.205|:21... connected. Logging in as anonymous ... Logged in! ... ... ... 15:24:50 (53.53 KB/s) - `mrtg-rrd-0.7.tar.gz' saved [20863] [root@bigboy tmp]# ls mrtg-rrd-0.7.tar.gz [root@bigboy tmp]#
Extract the contents of the tar file.
[root@bigboy tmp]# tar -xzvf mrtg-rrd-0.7.tar.gz mrtg-rrd-0.7/ mrtg-rrd-0.7/COPYING mrtg-rrd-0.7/FAQ mrtg-rrd-0.7/TODO mrtg-rrd-0.7/Makefile mrtg-rrd-0.7/mrtg-rrd.cgi mrtg-rrd-0.7/ChangeLog [root@bigboy tmp]#
Create the /var/www/cgi-bin/mrtg directory and copy the mrtg-rrd.cgi file to it.
[root@bigboy tmp]# mkdir -p /var/www/cgi-bin/mrtg [root@bigboy tmp]# cp mrtg-rrd-0.7/mrtg-rrd.cgi /var/www/cgi-bin/mrtg/ [root@bigboy tmp]#
Edit the mrtg-rrd.cgi file and make it refer to the /etc/mrtg/mrtg.cfg file for its configuration details, or you can specify all the .cfg files in your /etc/mrtg directory.
- File: mrtg-rrd.cgi (Single File)
- EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = qw(/etc/mrtg/mrtg.cfg); }
- File: mrtg-rrd.cgi (multipl .cfg files)
- EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = </etc/mrtg/*.cfg>; }
You should now be able to access your MRTG RRD graphs by visiting this URL:
http://www.my-web-site.org/cgi-bin/mrtg/mrtg-rrd.cgi
Once installed, RRDtool operates transparently with MRTG. You'll have to remember to add the RRD statements to any new MRTG configurations and also add the configuration file to the CGI script. Our monitoring server can now breathe a little easier.
Troubleshooting
The troubleshooting techniques for advanced MRTG are similar to those mentioned in Chapter 22, " Monitoring Server Performance", but because you have done some customizations you'll have to go the extra mile.
Verify the IP address and community string of the target device you intend to poll. Make sure you can do an SNMP walk of the target device. If not, revise your access controls on the target device and any firewall rules that may impede SNMP traffic. Ensure you can do an SNMP get of the specific OID value listed in your MRTG configuration file. Check your MRTG parameters to make sure they are correct. Gauge values defined as counter and vice versa will cause your graphs to have continuous zero values. Graph results that are eight times what you expect may have the bits parameter set. There are a few errors common to initial RRDtool integration.
Web messages like this where the reference to the MRTG configuration file in the CGI script was incorrect
Error: Cannot open config file: No such file or directory
"Permission Denied" web messages are usually caused by incorrect file permissions and / or SELinux contexts
Error: RRDs::graph failed, opening '/var/mrtg/localhost_192.168.1.100.rrd': Permission denied
Errors in the /var/log/httpd/errorlog file referring to files or directories that don't exist can be caused by an incorrect IconDir statement in the MRTG configuration file.
[Wed Jan 04 15:42:13 2006] [error] [client 192.168.1.102] File does not exist: /var/www/html/var, referer: http://bigboy/cgi-bin/mrtg/mrtg-rrd.cgi/
[Wed Jan 04 15:45:46 2006] [error] [client 192.168.1.102] script not found or unable to stat:
/var/www/cgi-bin/mrtg/mrtg-l.png, referer: http://bigboy/cgi-bin/mrtg/mrtg-rrd.cgi/
Errors caused by not installing the pre-requisite RRD RPM modules rrdtool, perl-RRD-Simple and rrdtool-perl.
ERROR: could not find RRDs.pm. Use LibAdd: in mrtg.cfg to help mrtg find RRDs.pm
These quick steps should be sufficient in most cases and will reward you with a more manageable network. Conclusion
Using the guidelines in this chapter you should be able to graph most SNMP MIB values available on any type of device. MRTG is an excellent, flexible monitoring tool and should be considered as a part of any systems administrator's server management plans.