snmp__if_combined

#!/usr/bin/perl -w
# -*- perl -*-

=head1 NAME

snmp__if_combined - SNMP card plugin to monitor the network interfaces of any networked equipment.

=head1 APPLICABLE SYSTEMS

Any SNMP capable networked computer equipment.  Using a command such
as

    munin-node-configure --snmp switch.langfeldt.net --snmpversion 2c --snmpcommunity public | sh -x

should auto-detect all applicable interfaces.  On a typical switch you will get
one plugin pr. ethernet port.  On a router you might get one plugin per VLAN
interface.

=head2 NOTABLE DIFFERENCE WITH SNMP__IF_MULTI

The `smnp__if_multi` plugin records data in _bytes_ per second, and shows them
as bps by using CDEFs when graphing. This plugin performs the multiplication
upon reporting the value, and therefore doesn't have a CDEF in the graphing
path.

This is important when migration from `snmp__if_multi`, for at least two
aspects:

- Reusing RRDs: it is not possible to just rename the old RRD files from
`smnp__if_multi`, as historical data will be graphed as 8 times smaller than in
reality. The stored values would need to be adjusted.
- Data loaning: borrowing graphs no longer need to include a CDEF to convert
bytes to bits. Conversely, they do now need a CDEF to convert bits to bytes.

To reflect this aspect explicitely, the root graph's totals are named
recv_bits and send_bits.

=head1 CONFIGURATION

As a rule SNMP plugins need site specific configuration.  The default
configuration (shown here) will only work on insecure sites/devices:

   [snmp_*]
   env.version 2
   env.community public
   env.ifTypeOnly ethernetCsmacd
   env.stackedRoot 1

In general SNMP is not very secure at all unless you use SNMP version
3 which supports authentication and privacy (encryption).  But in any
case the community string for your devices should not be "public".

Please see 'perldoc Munin::Plugin::SNMP' for further configuration
information.

The ifTypeOnly is a space separated list of media types to show. To
see what interface types your switch, router, or other net thing uses
you can use this command:

  snmpwalk -c public -v 2c switch 1.3.6.1.2.1.2.2.1.3

It may show something like this:

  IF-MIB::ifType.1 = INTEGER: ethernetCsmacd(6)
  IF-MIB::ifType.4 = INTEGER: mplsTunnel(150)
  IF-MIB::ifType.5 = INTEGER: other(1)
  IF-MIB::ifType.6 = INTEGER: softwareLoopback(24)
  IF-MIB::ifType.8 = INTEGER: tunnel(131)
  IF-MIB::ifType.13 = INTEGER: propVirtual(53)
  IF-MIB::ifType.123 = INTEGER: l2vlan(135)

propVirtual or l2vlan is usually used for VLAN interfaces.  Tunnel
would normally be for VPNs.  A minor horde of different interface types
are supposted, please see IANAifType-MIB (on your system or find with
Google) for a full list.

The `stackedRoot` option determines whether the root summary graph shows the
traffic on each interface separately, or stacks them on top of one another to
show the total traffic through the device.

=head1 INTERPRETATION

A single graph is generated with all the interfaces overlaid one over
the other: incoming traffic is received on the interface from the
connected device, outgoing is sent to it instead.

With the `stackedRoot` option, traffic is plotted as stacked areas instead, and
two additional series are calculated to show the total usage of the device.
Those series are useful for loaning into higher-level summary graphs (but see
BUGS below).

Sub-graphs are created, one per interface, akin to `snmp__if_multi`
plugin.

=head1 MIB INFORMATION

This plugin requires the IF-MIB the standard IETF MIB for network
interfaces.

It reports the contents of the
IF-MIB::ifHCInOctets/IF-MIB::ifHCOutOctets if available,
IF-MIB::ifInOctets/IF-MIB::ifOutOctets if not.  The former are 64 bit
counters only available with SNMP 2 and later.  The later are 32 bit
counters (see FEATURES below).

=head1 MAGIC MARKERS

  #%# family=snmpauto
  #%# capabilities=snmpconf

=head1 VERSION

  $Id: snmp__if_.in 1818 2009-01-03 19:29:30Z janl $

=head1 BUGS

=head2 Missing indexing options

Should support indexing by

  - ifIndex
  - ifName
  - ifDescr
  - ifAlias
  - mac-address

(and anything else MRTG can index by) in addition to OID-index as now.

=head2 Total traffic oddities in stackedRoot mode.

When using the `stackedRoot` option, the total lines (`send_bits` and
`recv_bits`) may sometimes jump to nonsensical values. This is a bug shared with
the original `snmp__if_multi`.

This seems to be due to the way those series are calculated, by summing the
current values of the other counters. The first issue is that of summing 64-bit
values into a single variable that is likely to also be 64 bits. The second
issue (compounded by the first) is that of wraparound. When one of the counters
wraps around, the sum jumps backwards. With a `min` set to 0, and other counters
having kept increasing, this looks like a huge increase the total counter.
Depending on the `max` (which should match the backplane bandwidth), this may
not be correctly recognised as a spurious value, just reconded as valid.

There is no clear solution to this bug at the moment, save for trying to salvage
the data after the fact. This can be done my either clipping all values beyond a
maximum (e.g., the known use of the switch, rather that its full backplane
bandwith).

    RANGE='[0-9.]\+e+\(0[789]\|[1-9][0-9]\)'  # Anything >=1e07
    rrdtool dump ${RRD_FILE} \
      | sed "s/<v>${RANGE}/<v>NaN/" \
      | rrdtool restore - ${RRD_FILE}.new
    mv ${RRD_FILE}.new ${RRD_FILE}  # Make sure you have a backup!

Alternatively, if you can determine the time at whiche the spurious value was
recorded, you can just make that value a NaN.

    TIMESTAMP="1623492000"
    rrdtool dump "${RRD_FILE}" \
      | sed "/${TIMESTAMP}/s/<v>.*<\/v>/<v>NaN<\/v>/" \
      | rrdtool restore - "${RRD_FILE}.new"
    mv ${RRD_FILE}.new ${RRD_FILE}  # Make sure you have a backup!


=head1 FEATURES

You may get strange results if you use SNMPv1, or SNMPv2 on
switches that do not support 64 bit byte counters.  If the interface
traffic exceeds about 50Mbps a 32 bit byte counter will wrap around in
less than 5 minutes making the graph for the interface show random
results.

If you have a switch/device that supports 64 bit byte counters this
plugin will use them and the graph will be fine.  The graph
information will inform about this.  You must use SNMPv2c or SNMPv3 to
be able to use 64 bit counters - if the device supports them.

This problem is a feature of the device SNMP implementation or your
usage of it, it is nothing the plugin can fix.  In the future Munin
may be able to run the plugin more often than the counter wraps
around.

=head1 AUTHORS

Copyright (C) 2012 Diego Elio Pettenò.

Original snmp__if_multi: Copyright (C) 2004-2010 Jimmy Olsen, Dagfinn
Ilmari Mannsåker, Nicolai Langfeldt, Redpill Linpro AS and others.

Original snmp__if_ plugin: Copyright (C) 2004-2009 Jimmy Olsen, Dagfinn
Ilmari Mannsåker.

Initial SNMPv3 support by "Confusedhacker".

Documentation, porting to Munin::Plugin::SNMP and
further grooming by Nicolai Langfeldt.

Reworked to snmp__if_multi by Nicolai Langfeldt.

Reworked to snmp__if_combined by Diego Elio Pettenò.

Stacked option by Olivier Mehani <shtrom+munin@ssji.net>

=head1 LICENSE

GPLv2

=cut

use strict;
use Munin::Plugin;
use Munin::Plugin::SNMP;

my $response;
my $iface;

# This is the snmpwalk:
# .1.3.6.1.2.1.2.1.0 = INTEGER: 2
# .1.3.6.1.2.1.2.2.1.1.1 = INTEGER: 1
# .1.3.6.1.2.1.2.2.1.1.65539 = INTEGER: 65539
# .1.3.6.1.2.1.2.2.1.2.1 = STRING: MS TCP Loopback interface
# .1.3.6.1.2.1.2.2.1.2.65539 = STRING: Broadcom NetXtreme Gigabit Ethernet
# .1.3.6.1.2.1.2.2.1.3.1 = INTEGER: softwareLoopback(24)
# .1.3.6.1.2.1.2.2.1.3.65539 = INTEGER: ethernetCsmacd(6)
# .1.3.6.1.2.1.2.2.1.4.1 = INTEGER: 1520
# .1.3.6.1.2.1.2.2.1.4.65539 = INTEGER: 1500
# .1.3.6.1.2.1.2.2.1.5.1 = Gauge32: 10000000
# .1.3.6.1.2.1.2.2.1.5.65539 = Gauge32: 1000000000
# .1.3.6.1.2.1.2.2.1.6.1 = STRING:
# .1.3.6.1.2.1.2.2.1.6.65539 = STRING: 0:30:48:75:65:5e
# .1.3.6.1.2.1.2.2.1.7.1 = INTEGER: up(1)
# .1.3.6.1.2.1.2.2.1.7.65539 = INTEGER: up(1)
# .1.3.6.1.2.1.2.2.1.8.1 = INTEGER: up(1)
# .1.3.6.1.2.1.2.2.1.8.65539 = INTEGER: up(1)
#
# 64 bit counters:
# .1.3.6.1.2.1.31.1.1.1.6.   Counter64 ifHCInOctets
# .1.3.6.1.2.1.31.1.1.1.10.  Counter64 ifHCOutOctets

if (defined $ARGV[0] and $ARGV[0] eq "snmpconf") {
    print "require 1.3.6.1.2.1.2.2.1.1.\n";
    exit 0;
}

my $stackedRoot = 0;
if (exists $ENV{'stackedRoot'}) {
    $stackedRoot = $ENV{'stackedRoot'};
}

my $sysDescr           = '1.3.6.1.2.1.1.1.0';
my $sysLocation        = '1.3.6.1.2.1.1.6.0';
my $sysContact         = '1.3.6.1.2.1.1.4.0';
my $sysName            = '1.3.6.1.2.1.1.5.0';

my $ifOIDBase          = "1.3.6.1.2.1.2.2.1";		# ifEntry
my $ifv2OIDBase        = "1.3.6.1.2.1.31.1.1.1";	# ifXEntry

my %ifTypes = (
    other => 1,
    regular1822 => 2,
    hdh1822 => 3,
    ddnX25 => 4,
    rfc877x25 => 5,
    ethernetCsmacd => 6,
    iso88023Csmacd => 7,
    iso88024TokenBus => 8,
    iso88025TokenRing => 9,
    iso88026Man => 10,
    starLan => 11,
    proteon10Mbit => 12,
    proteon80Mbit => 13,
    hyperchannel => 14,
    fddi => 15,
    lapb => 16,
    sdlc => 17,
    ds1 => 18,
    e1 => 19,
    basicISDN => 20,
    primaryISDN => 21,
    propPointToPointSerial => 22,
    ppp => 23,
    softwareLoopback => 24,
    eon => 25,
    ethernet3Mbit => 26,
    nsip => 27,
    slip => 28,
    ultra => 29,
    ds3 => 30,
    sip => 31,
    frameRelay => 32,
    rs232 => 33,
    para => 34,
    arcnet => 35,
    arcnetPlus => 36,
    atm => 37,
    miox25 => 38,
    sonet => 39,
    x25ple => 40,
    iso88022llc => 41,
    localTalk => 42,
    smdsDxi => 43,
    frameRelayService => 44,
    v35 => 45,
    hssi => 46,
    hippi => 47,
    modem => 48,
    aal5 => 49,
    sonetPath => 50,
    sonetVT => 51,
    smdsIcip => 52,
    propVirtual => 53,
    propMultiplexor => 54,
    ieee80212 => 55,
    fibreChannel => 56,
    hippiInterface => 57,
    frameRelayInterconnect => 58,
    aflane8023 => 59,
    aflane8025 => 60,
    cctEmul => 61,
    fastEther => 62,
    isdn => 63,
    v11 => 64,
    v36 => 65,
    g703at64k => 66,
    g703at2mb => 67,
    qllc => 68,
    fastEtherFX => 69,
    channel => 70,
    ieee80211 => 71,
    ibm370parChan => 72,
    escon => 73,
    dlsw => 74,
    isdns => 75,
    isdnu => 76,
    lapd => 77,
    ipSwitch => 78,
    rsrb => 79,
    atmLogical => 80,
    ds0 => 81,
    ds0Bundle => 82,
    bsc => 83,
    async => 84,
    cnr => 85,
    iso88025Dtr => 86,
    eplrs => 87,
    arap => 88,
    propCnls => 89,
    hostPad => 90,
    termPad => 91,
    frameRelayMPI => 92,
    x213 => 93,
    adsl => 94,
    radsl => 95,
    sdsl => 96,
    vdsl => 97,
    iso88025CRFPInt => 98,
    myrinet => 99,
    voiceEM => 100,
    voiceFXO => 101,
    voiceFXS => 102,
    voiceEncap => 103,
    voiceOverIp => 104,
    atmDxi => 105,
    atmFuni => 106,
    atmIma => 107,
    pppMultilinkBundle => 108,
    ipOverCdlc => 109,
    ipOverClaw => 110,
    stackToStack => 111,
    virtualIpAddress => 112,
    mpc => 113,
    ipOverAtm => 114,
    iso88025Fiber => 115,
    tdlc => 116,
    gigabitEthernet => 117,
    hdlc => 118,
    lapf => 119,
    v37 => 120,
    x25mlp => 121,
    x25huntGroup => 122,
    trasnpHdlc => 123,
    interleave => 124,
    fast => 125,
    ip => 126,
    docsCableMaclayer => 127,
    docsCableDownstream => 128,
    docsCableUpstream => 129,
    a12MppSwitch => 130,
    tunnel => 131,
    coffee => 132,
    ces => 133,
    atmSubInterface => 134,
    l2vlan => 135,
    l3ipvlan => 136,
    l3ipxvlan => 137,
    digitalPowerline => 138,
    mediaMailOverIp => 139,
    dtm => 140,
    dcn => 141,
    ipForward => 142,
    msdsl => 143,
    ieee1394 => 144,
    'if-gsn' => 145,
    dvbRccMacLayer => 146,
    dvbRccDownstream => 147,
    dvbRccUpstream => 148,
    atmVirtual => 149,
    mplsTunnel => 150,
    srp => 151,
    voiceOverAtm => 152,
    voiceOverFrameRelay => 153,
    idsl => 154,
    compositeLink => 155,
    ss7SigLink => 156,
    propWirelessP2P => 157,
    frForward => 158,
    rfc1483 => 159,
    usb => 160,
    ieee8023adLag => 161,
    bgppolicyaccounting => 162,
    frf16MfrBundle => 163,
    h323Gatekeeper => 164,
    h323Proxy => 165,
    mpls => 166,
    mfSigLink => 167,
    hdsl2 => 168,
    shdsl => 169,
    ds1FDL => 170,
    pos => 171,
    dvbAsiIn => 172,
    dvbAsiOut => 173,
    plc => 174,
    nfas => 175,
    tr008 => 176,
    gr303RDT => 177,
    gr303IDT => 178,
    isup => 179,
    propDocsWirelessMaclayer => 180,
    propDocsWirelessDownstream => 181,
    propDocsWirelessUpstream => 182,
    hiperlan2 => 183,
    propBWAp2Mp => 184,
    sonetOverheadChannel => 185,
    digitalWrapperOverheadChannel => 186,
    aal2 => 187,
    radioMAC => 188,
    atmRadio => 189,
    imt => 190,
    mvl => 191,
    reachDSL => 192,
    frDlciEndPt => 193,
    atmVciEndPt => 194,
    opticalChannel => 195,
    opticalTransport => 196,
    propAtm => 197,
    voiceOverCable => 198,
    infiniband => 199,
    teLink => 200,
    q2931 => 201,
    virtualTg => 202,
    sipTg => 203,
    sipSig => 204,
    docsCableUpstreamChannel => 205,
    econet => 206,
    pon155 => 207,
    pon622 => 208,
    bridge => 209,
    linegroup => 210,
    voiceEMFGD => 211,
    voiceFGDEANA => 212,
    voiceDID => 213,
    mpegTransport => 214,
    sixToFour => 215,
    gtp => 216,
    pdnEtherLoop1 => 217,
    pdnEtherLoop2 => 218,
    opticalChannelGroup => 219,
    homepna => 220,
    gfp => 221,
    ciscoISLvlan => 222,
    actelisMetaLOOP => 223,
    fcipLink => 224,
    rpr => 225,
    qam => 226,
    lmp => 227,
    cblVectaStar => 228,
    docsCableMCmtsDownstream => 229,
    adsl2 => 230,
    macSecControlledIF => 231,
    macSecUncontrolledIF => 232,
    aviciOpticalEther => 233,
    atmbond => 234,
    voiceFGDOS => 235,
    mocaVersion1 => 236,
    ieee80216WMAN => 237,
    adsl2plus => 238,
    dvbRcsMacLayer => 239,
    dvbTdm => 240,
    dvbRcsTdma => 241,
    x86Laps => 242,
    wwanPP => 243,
    wwanPP2 => 244
);

my %ifTypeByNum = map { $ifTypes{$_} => $_; } keys %ifTypes;

# Needed as globals
my $snmpinfo;
my $snmpinfoX;

sub do_collect {
    # Collect information from SNMP agent

    my $session = Munin::Plugin::SNMP->session();

    $snmpinfo = $session->get_hash(
        -baseoid => $ifOIDBase,
        -cols    => {
            2  => 'ifDescr',
            # Type: See above
            3  => 'ifType',
            5  => 'ifSpeed',
            # Oper: 1) up 2) down 3) testing
            # 4) unknown, 5) dormant 6) not present
            # 7) lowerLayerDown
            8  => 'ifOperStatus',
            10 => 'ifInOctets',
            13 => 'ifInDiscards',
            14 => 'ifInErrors',
            16 => 'ifOutOctets',
            19 => 'ifOutDiscards',
            20 => 'ifOutErrors',
        });

    # ifXEntry - SNMP v2 and up only - on some devices
    $snmpinfoX = $session->get_hash(
        -baseoid => $ifv2OIDBase,
        -cols    => {
            6  => 'ifHCInOctets',
            10 => 'ifHCOutOctets',
            15 => 'ifHighSpeed',
            18 => 'ifAlias',
        });
}



sub do_preprocess_if {
    my ($mediatype, $if) = @_;

    my $response = $snmpinfo->{$if}->{ifType} || 1;

    if (defined($mediatype)) {
        if (defined($mediatype->{$response})) {
            # This is one of the interesting media types
        } else {
            # This media type is not interesting.  Remove.
            delete $snmpinfo->{$if} if exists $snmpinfo->{$if};
            delete $snmpinfoX->{$if} if exists $snmpinfoX->{$if};
            return;
        }
    }

    if (defined ($response = $snmpinfo->{$if}->{ifOperStatus})) {
        # 1 = up,  2 = down,  7 = lowerLayerDown
        if ($response == 2 or $response == 7) {
            # Fold recognized down states into one.
            $response = $snmpinfo->{$if}->{ifOperStatus} = 2;
        } elsif ($response != 1) {
            # This interface is fishy, remove and forget.
            delete $snmpinfo->{$if} if exists $snmpinfo->{$if};
            delete $snmpinfoX->{$if} if exists $snmpinfoX->{$if};
            return;
        }
    }
}


sub do_preprocess {
    my $mediatypes = 'ALL';

    if (exists( $ENV{'ifTypeOnly'} )) {
        $mediatypes = $ENV{'ifTypeOnly'};
    }

    my @mediatypes = split(/[ ,]+/,$mediatypes);

    # Hash of numerical media types the user finds interesting.
    my $mediatype={};

    if ($mediatypes eq 'ALL') {
        $mediatype = undef;
    } else {
        foreach my $type (@mediatypes) {
            if (exists($ifTypes{$type})) {
                $mediatype->{$ifTypes{$type}} = 1;
            } else {
                die "Unknown media type '$type' specified in ifTypeOnly\n";
            }
        }
    }

    foreach my $if (keys %{$snmpinfo}) {
        do_preprocess_if($mediatype, $if);
    }
}


sub do_config_root {
    my ($host, $version) = @_;

    print <<END;

multigraph snmp_if_combined
graph_title All interfaces traffic
graph_args --base 1000
graph_vlabel bits in (-) / out (+) per \${graph_period}
graph_category network
graph_info This graph shows the total traffic for $host.
END

    print "graph_order";
    my @ifs;
    foreach my $if (sort {$a <=> $b} keys %{$snmpinfo}) {
        print " recv$if=snmp_if_combined.$if.recv send$if=snmp_if_combined.$if.send";
        push @ifs, "snmp_if_combined.$if";
    }
    if ($stackedRoot) {
        print " recv_bits send_bits";
    }
    print "\n";

    # XXX: For some reason, Munin doesn't resolve AREASTACK properly in this case...
    my $area_or_stack = "AREA";
    foreach my $if (sort {$a <=> $b} keys %{$snmpinfo}) {
        my $alias = $snmpinfoX->{$if}->{ifAlias} || $snmpinfo->{$if}->{ifDescr} || "Interface $if";

        if (! ($alias =~ /\d+/) ) {
            # If there are no numbers in the $alias add the if index
            $alias .=" (if $if)";
        }

        print <<END;
recv$if.label $alias
recv$if.graph no
send$if.label $alias
send$if.negative recv$if
END
        if ($stackedRoot) {
            print "send$if.draw $area_or_stack\n";
            $area_or_stack = "STACK";
        }
    }
    if ($stackedRoot) {

        # If a stacked graph is requested, plot the total. Rather than
        # aggregating them with CDEFs of SUMs, we sum them in the script, which
        # allows us to create real series than can be easily borrowed by other
        # loaning graphs.
        #
        # Using CDEFs and SUMs proved difficult anyway. Due to bugs in Munin
        # 2.0's GraphOld core  (negatives and multigraph), the graph_total
        # behaves weirdly: - only the positive total is plotted (but both
        # positive and negatives are calculated); - the graph_total is also
        # included in the single graphs, where it is clearly unecessary.

        print <<END;
recv_bits.label Total traffic
recv_bits.type DERIVE
recv_bits.min 0
recv_bits.graph no
send_bits.label Total traffic
send_bits.type DERIVE
send_bits.min 0
send_bits.draw LINE1
send_bits.colour 000000
send_bits.negative recv_bits
END
    }

    print <<END;
multigraph snmp_if_combined_err
graph_title All interfaces errors
graph_args --base 1000
graph_vlabel errors in (-) / out (+) per \${graph_period}
graph_category network
END

    print "graph_order";
    foreach my $if (sort {$a <=> $b} keys %{$snmpinfo}) {
        print " recv$if=snmp_if_combined_err.$if.total_in send$if=snmp_if_combined_err.$if.total_out";
    }
    print "\n";

    foreach my $if (sort {$a <=> $b} keys %{$snmpinfo}) {
        my $alias = $snmpinfoX->{$if}->{ifAlias} || $snmpinfo->{$if}->{ifDescr} || "Interface $if";

        if (! ($alias =~ /\d+/) ) {
            # If there are no numbers in the $alias add the if index
            $alias .=" (if $if)";
        }

        print <<END;
recv$if.label $alias
recv$if.graph no
send$if.label $alias
send$if.negative recv$if
END
    }
}

sub do_config_if {
    my ($host,$version,$if) = @_;

    my $alias = $snmpinfoX->{$if}->{ifAlias} || $snmpinfo->{$if}->{ifDescr} || "Interface $if";

    if (! ($alias =~ /\d+/) ) {
        # If there are no numbers in the $alias add the if index
        $alias .=" (if $if)";
    }

    my $warn = undef;
    my $speed = 0;

    my $extrainfo="";

    if (defined ($snmpinfoX->{$if}->{ifHCInOctets})) {
        # If we get an answer at the 64 bit OID then this switch
        # supports the extended MIB

        $extrainfo .= " This switch supports 64 bit byte counters and these are used by this plugin.";
    } else {
        # If not we only have a 32 bit counter and are lost.
        $extrainfo .= " NOTE! This switch supports only 32 bit byte counters which makes the plugin unreliable and unsuitable for most 100Mb (or faster) interfaces, where bursts are expected to exceed 50Mbps.  This means that for interfaces where much traffic is sent this plugin will report false throughputs and cannot be trusted.";

        # unless perhaps the operator can get us snmp version 2c or 3?
        $extrainfo .= " I notice that you use SNMP version 1 which does not support 64 bit quantities.  You may get better results if you switch to SNMP version 2c or 3.  Please refer to the plugin documentation."
        if $version == 1;
    }

    if (defined ($speed = $snmpinfoX->{$if}->{ifHighSpeed}) and $speed) {
        # Speed in 1,000,000 bits per second
        $speed = $speed * 1000000;
        $warn = $speed / 75 * 100;

        my $textspeed = scaleNumber($speed,'bps','',
            'The interface speed is %.1f%s%s.');

        $extrainfo .= ' '.$textspeed if $textspeed;
    } elsif (defined ($speed = $snmpinfo->{$if}->{ifSpeed}) and $speed) {
        # Speed in bits pr. second
        $warn = $speed*100/75;

        my $textspeed = scaleNumber($speed,'bps','',
            'The interface speed is %.1f%s%s.');

        $extrainfo .= " ".$textspeed if $textspeed;
    }

    $response = $snmpinfo->{$if}->{ifType};
    $extrainfo .= " This is a '".$ifTypeByNum{$response}."' interface.";

    print <<END;
multigraph snmp_if_combined.$if
graph_title $alias traffic
graph_info This graph shows traffic for the \"$alias\" network interface.$extrainfo
graph_args --base 1000
graph_vlabel bits in (-) / out (+) per \${graph_period}
graph_category network

recv.label bps
recv.type DERIVE
recv.graph no
recv.min 0
send.label bps
send.type DERIVE
send.negative recv
send.min 0
END

    if ( defined($speed) ) {
        printf("recv.max %s\nsend.max %s\n", $speed, $speed);
    }

    if ( defined($warn) ) {
        printf("recv.warning %s\nsend.warning %s\n", $warn, $warn);
    }

    print <<END;
multigraph snmp_if_combined_err.$if
graph_title $alias errors
graph_info This graph shows errors for the \"$alias\" network interface.$extrainfo
graph_args --base 1000
graph_vlabel errors in (-) / out (+) per \${graph_period}
graph_category network

errors_in.label $alias
errors_in.graph no
errors_in.label recv
errors_in.type DERIVE
errors_in.min 0
errors_out.label Errors
errors_out.negative errors_in
errors_out.type DERIVE
errors_out.min 0

discards_in.label $alias
discards_in.graph no
discards_in.type DERIVE
discards_in.min 0
discards_out.label Discards
discards_out.negative discards_in
discards_out.type DERIVE
discards_out.min 0

total_in.label $alias
total_in.graph no
total_in.type DERIVE
total_in.min 0
total_out.label $alias
total_out.graph no
total_out.type DERIVE
total_out.min 0
END
}

sub do_fetch_if {
    my($if) = @_;

    my $status = $snmpinfo->{$if}->{ifOperStatus} || 1;

    # 1 means up
    # 2 means set to down
    # Everything else we ignore.

    if ($status != 1) {
        # Interface is down
        print <<END;
multigraph snmp_if_combined.$if
recv.value U
send.value U
send.extinfo This interface is currently down.
multigraph snmp_if_combined_err.$if
errors_in.value U
errors_out.value U
discards_in.value U
discards_out.value U
total_in.value U
total_out.value U
errors_out.extinfo This interface is currently down
END
        return (undef, undef);
    }

    my $response;
    my $recv;
    my $send;

    print "multigraph snmp_if_combined.$if\n";

    $response = $snmpinfoX->{$if}->{ifHCInOctets} || $snmpinfo->{$if}->{ifInOctets};
    $recv = defined($response) ? ($response * 8) : undef;
    printf("recv.value %s\n", $recv || "U");

    $response = $snmpinfoX->{$if}->{ifHCOutOctets} || $snmpinfo->{$if}->{ifOutOctets};
    $send = defined($response) ? ($response * 8) : undef;
    printf("send.value %s\n", $send || "U");

    print "multigraph snmp_if_combined_err.$if\n";

    my $errors = $snmpinfo->{$if}->{ifInErrors};
    my $discards = $snmpinfo->{$if}->{ifInDiscards};
    printf("errors_in.value %s\n".
        "discards_in.value %s\n".
        "total_in.value %s\n",
        $errors || "U",
        $discards || "U",
        ($errors || 0) + ($discards || 0)
    );

    $errors = $snmpinfo->{$if}->{ifOutErrors};
    $discards = $snmpinfo->{$if}->{ifOutDiscards};
    printf("errors_out.value %s\n".
        "discards_out.value %s\n".
        "total_out.value %s\n",
        $errors || "U",
        $discards || "U",
        ($errors || 0) + ($discards || 0)
    );

    return ($recv, $send);
}

sub do_config {

    my ($host,undef,$version) = Munin::Plugin::SNMP->config_session();

    print "host_name $host\n" unless $host eq 'localhost';

    foreach my $if (sort {$a <=> $b} keys %{$snmpinfo}) {
        do_config_if($host,$version,$if);
    }

    do_config_root($host,$version);
}

# ############################## MAIN ################################

do_collect;
do_preprocess;

if ($ARGV[0] and $ARGV[0] eq "config") {
    do_config();
    exit 0;
}

my $recv = 0;
my $send = 0,
my $recv_if;
my $send_if;

foreach my $if (sort {$a <=> $b} keys %{$snmpinfo}) {
    ($recv_if, $send_if) = do_fetch_if($if);
    $recv += ($recv_if || 0);
    $send += ($send_if || 0);
}
if ($stackedRoot) {
    print "multigraph snmp_if_combined\n";
    print "recv_bits.value $recv\n";
    print "send_bits.value $send\n";
}