meminfo

#!/usr/bin/perl

=head1 NAME

meminfo - Plugin to monitor memory usage.

=head1 WARNING

Do not enable this plugin unless you need it.  It alone produces
significantly more data than any normal munin-node in sum.  Therefore
it is not automatically enabled.

=head1 APPLICABLE SYSTEMS

All Linux systems

=head1 INTERPRETATION

This plugin shows many graphs (on my pc ~90 graphs with total ~750
values).  Graphs are split to 8 groups, every group has sub graphs:

  Application groups memory usage                  - show applications groups memory using by group (env.) [env.applications_group]
    Group `groupName1` applications memory usage     v
    ...                                            - show applications groups memory using per application in every group
    Group `groupNameX` applications memory usage     ^
    Summary group `groupName1` memory usage          v
    ...                                            - show applications groups memory using as summ of every parameter of application in every group
    Summary group `groupNameX` memory usage          ^
  Application memory usage                         - show memory usage per application [env.applications]
    Application processes                          - show processes count of every application
    Summary `applicationName1` memory usage          v
    ...                                            - Summary application processes memory using
    Summary `applicationNameX` memory usage          ^
  External fragmentation: Buddyinfo                - Buddyinfo/ Nodes and zones summary
     Node X, zone ZZZ                              - Chunks for each zone
  External fragmentation: Page type info           - Page type info, summary node-zone-type pages
    Node X, zone ZZZ, type TTT                     - Free pages per node-zone-page by order
  Physical memory usage                            - Main graph, show physical memory usage
    Active/Inactive memory                         - subj :)
    Memory usage by cashes and buffers             - subj :)
    mmap                                           - mmapp'ed memory
    HugePages count                                - subj :)
    HugePages size                                 - subj :)
    Kernel memory                                  - subj :)
    Low and high memory                            - subj :)
  Slab objects size                                - subj :)
    Slabs size [GroupName1]                          v
    ...                                            - show slabs size. I try split fields by this types.
    Slabs size [GroupNameX]                          ^
    Slabs sizes of groups                          - "Groups" sizes
    Slab objects                                   - Active/inactive objects
    Slabs                                          - Active/inactive and shared slabs
    Reclaim                                        - Reclaimed
  Swap usage                                       - subj :)
    First candidates to swap out                   - subj :)
    Writeback                                      - writeback and dirty
  Virtual memory usage                             - subj :)
    Vmalloc                                        - show vmalloc usage
    Vmalloc allocates                              - allocates per object

=head1 CONFIGURATION

The plugin automatically selects which graphics drawing.
But you can select graphs to draw by enabled_graphs environment.
enabled_graphs is a regexp, where you include graphs to draw.

Default: undefined

Examples:

  env.enabled_graphs meminfo_\w+\.?       # Draw only meminfo graphs
  env.enabled_graphs (meminfo|swapinfo)$  # Draw only meminfo and swapinfo graphs, without 'childs'

Also you can select applications to monitor it by applications
environment applications environment is regexp

Default: undefined

Examples:

  env.applications (firefox|\w{1,3}?sh) # Monitor firefox and shells (bash, zsh, sh, etc...)

Also you can group applications and show memory using of this groups,
per application in group and summmary applications in group. Use
env.applications_group for this.

Format: env.applications_group regexp:groupName;regexp:groupName;...

where regexp - it applications names regexp, and groupName - on-screen
group name

Default: undefined

Examples:

  env.applications_group ^(firefox|chrome|opera|konqu|arora):Browsers;(^lx|openbox|menu|gnome|slim|^X):X;^(hal|console-kit|syslog|cron|dhcp|udev|dbus|bluetoo|agett|login|automount):System;

And you can select time to display data after application close (all
values return as 'NaN'), for this you must used environment
env.application_wait Value - secunds

Default: 1800

Example:

  env.application_wait 86400 #24h

=head2 PERMISSIONS NOTE

Please check, can your munin user read files such as

  /proc/meminfo
  /proc/slabinfo
  /proc/vmallocinfo
  /proc/buddyinfo
  /proc/pagetypeinfo
  /proc/[pid]/status

If no access, just write in plugin-config

  [meminfo]
  user root
  group root

=head2 WARNING AND CRITICAL SETTINGS

You can set warning and critical levels for *each* of the data
series the plugin reports.

Template for limits:

  env.limit_%field% (warning_num:crytical_num|critical_num)[kMG]

where %field% - field name. You can see it in graph view, 'field
internal name' field

  warning_num - number, warning value
  crytical_num - number, critical value
  k - kilobytes, M - megabytes, G - gigabytes

Examples:

  env.limit_apps 300M:500M    # Applications, warning - 300M, critical - 500m
  env.limit_committed_as 700M # Committed AS, critical - 700M


=head1 MAGIC MARKERS

  #%# family=contrib
  #%# capabilities=autoconf


=head1 VERSION

  1.0

=head1 BUGS

none known

=head1 AUTHOR

Gorlow Maxim aka Sheridan <sheridan@sheridan-home.ru> (email and jabber)

=head1 LICENSE

GPLv2

=cut



use strict;
use warnings;
use Munin::Plugin;
use IO::Dir;
use Data::Dumper;

# -----------------------------------------------------------------------------------------------------------------------------------------------------
my $graphs_source =
{
  # ----------------------------------------------
  'meminfo_physical' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Physical memory usage',
      'vtitle'   => 'bytes',
      'category' => 'memory'
     },
     'fields' => [qw(apps:AREASTACK:- slab:AREASTACK:- buffers:AREASTACK:- cached:AREASTACK:- page_tables:AREASTACK:- cached_swap:AREASTACK:- free:AREASTACK:FFFFCCCC swap_used:AREASTACK:- used:LINE1:- usedwswap:LINE1:- hardware_corrupted:LINE1:- total:LINE1:000000)]
  },

  # ----------------------------------------------
  'meminfo_physical.cb' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Memory usage by cashes and buffers',
      'vtitle'   => 'bytes',
      'category' => 'memory',
      'total'    => 'Total'
    },
    'fields' => [qw(cached:AREASTACK:- buffers:AREASTACK:- cached_swap:AREASTACK:- nfs_unstable:AREASTACK:- writeback_fuse:AREASTACK:- bounce:AREASTACK:-)]
  },
  # ----------------------------------------------
  'meminfo_physical.active_inactive' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Active/Inactive memory',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(active_file:AREA:FF444488 active_anon:STACK:44FF4488 active:LINE1:000000 inactive_file:AREA:4444FF88 inactive_anon:STACK:FFFF4499 inactive:LINE1:FFFFFF)]
  },
  # ----------------------------------------------
  'meminfo_physical.mmap' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'mmap',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(direct_map_4k:LINE1:- direct_map_2m:LINE1:- direct_map_4m:LINE1:- direct_map_1g:LINE1:- mapped:LINE1:- mmap_copy:LINE1:-)]
  },
  # ----------------------------------------------
  'meminfo_physical.kernel' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Kernel memory',
      'vtitle'   => 'bytes',
      'category' => 'memory',
      'total'    => 'Total'
    },
    'fields' => [qw(kernel_stack:AREASTACK:- slab:AREASTACK:- qlists:LINE1:-)]
  },
  # ----------------------------------------------
  'meminfo_physical.low_and_high' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Low and high memory',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(low_used:AREASTACK:- low_free:AREASTACK:CCFFCCCC high_used:AREASTACK:- high_free:AREASTACK:FFFFCCCC low_total:LINE1:- high_total:LINE1:- l_h_total:LINE1:000000)]
  },
  # ----------------------------------------------
  'meminfo_physical.hugepages' =>
  {
    'munin' =>
    {
      'args'     => '--base 1000 ',
      'title'    => 'HugePages count',
      'vtitle'   => 'count',
      'category' => 'memory'
    },
    'fields' => [qw(huge_pages_total:LINE1:000000 huge_pages_free:AREASTACK:- huge_pages_rsvd:AREASTACK:- huge_pages_surp:AREASTACK:-)]
  },
  # ----------------------------------------------
  'meminfo_physical.hugepages_sizes' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'HugePages size',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(huge_page_size:LINE1:00FF00 huge_pages_total_size:LINE1:000000 huge_pages_free_size:AREASTACK:- huge_pages_rsvd_size:AREASTACK:- huge_pages_surp_size:AREASTACK:-)]
  },
  # ----------------------------------------------
  'meminfo_virtual' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Virtual memory usage',
      'vtitle'   => 'bytes',
      'category' => 'memory'
     },
     'fields' => [qw(commit_limit:AREA:88884477 committed_as:AREA:- mlocked:LINE1:- shmem:LINE1:- vmalloc_used:LINE1:-)]
  },
  # ----------------------------------------------
  'meminfo_virtual.vmalloc' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Vmalloc',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(vmalloc_used:AREASTACK:- vmalloc_free:AREASTACK:FFFFCCCC vmalloc_total:LINE1:000000 vmalloc_chunk:LINE1:-)]
  },
  # ----------------------------------------------
  'meminfo_virtual.vmalloc_allocates' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Vmalloc allocates',
      'vtitle'   => 'bytes',
      'category' => 'memory',
      'total'    => 'Total'
    },
    'fields' => []
  },
  # ----------------------------------------------
  'slab' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Slab objects size',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(min_obj_size:AREA:00000000 max_obj_size:STACK:- avg_obj_size:LINE1:-)]
  },
  # ----------------------------------------------
  'slab.slab_objects' =>
  {
    'munin' =>
    {
      'args'     => '--base 1000 ',
      'title'    => 'Slab objects',
      'vtitle'   => 'count',
      'category' => 'memory'
    },
    'fields' => [qw(active_objects:AREASTACK:- inactive_objects:AREASTACK:FFFFCCCC total_objects:LINE1:000000)]
  },
  # ----------------------------------------------
  'slab.slabs' =>
  {
    'munin' =>
    {
      'args'     => '--base 1000 ',
      'title'    => 'Slabs',
      'vtitle'   => 'count',
      'category' => 'memory'
    },
    'fields' => [qw(active_slabs:AREASTACK:- inactive_slabs:AREASTACK:FFFFCCCC shared_slabs:LINE1:- total_slabs:LINE1:000000)]
  },
  # ----------------------------------------------
  'slab.reclaim' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Reclaim',
      'vtitle'   => 'bytes',
      'category' => 'memory',
      'total'    => 'Total',
    },
    'fields' => [qw(slab_reclaimable:AREA:- slab_unreclaimable:STACK:-)]
  },
  # ----------------------------------------------
  'slab.slabs_size_all_groups' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => "Slabs sizes of groups",
      'vtitle'   => 'bytes',
      'total'    => 'Total',
      'category' => 'memory'
    },
    'fields' => []
  },
  # ----------------------------------------------
  'swapinfo' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Swap usage',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(swap_used:AREA:- swap_free:STACK:FFFFCCCC cached_swap:LINE1:- swap_total:LINE1:000000)]
  },
  # ----------------------------------------------
  'swapinfo.wrback' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Writeback',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(dirty:LINE1:- writeback:LINE1:-)]
  },
  # ----------------------------------------------
  'swapinfo.candidates' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'First candidates to swap out',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => [qw(inactive_file:AREASTACK:- inactive_anon:AREASTACK:- anon_pages:AREASTACK:- inactive:LINE1:-)]
  },
  # ----------------------------------------------
  'buddyinfo' =>
  {
    'munin' =>
    {
      'args'     => '--base 1000',
      'title'    => 'External fragmentation: Buddyinfo',
      'vtitle'   => 'chunks',
      'category' => 'memory',
      'info'     => 'External fragmentation is a problem under some workloads, and buddyinfo is a useful tool for helping diagnose these problems. Buddyinfo will give you a clue as to how big an area you can safely allocate, or why a previous allocation failed.',
      'total'    => 'Total'
    },
    'fields' => []
  },
  # ----------------------------------------------
  'appinfo' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Application memory usage',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => []
  },
  # ----------------------------------------------
  'appinfo.processes' =>
  {
    'munin' =>
    {
      'args'     => '--base 1000',
      'title'    => 'Application processes',
      'vtitle'   => 'count',
      'category' => 'processes'
    },
    'fields' => []
  },
  # ----------------------------------------------
  'appgroupinfo' =>
  {
    'munin' =>
    {
      'args'     => '--base 1024',
      'title'    => 'Application groups memory usage',
      'vtitle'   => 'bytes',
      'category' => 'memory'
    },
    'fields' => []
  },
  # ----------------------------------------------
  'pagetypeinfo' =>
  {
    'munin' =>
    {
      'args'     => '--base 1000',
      'title'    => 'External fragmentation: Page type info',
      'vtitle'   => 'pages',
      'info'     => 'The pagetypinfo gives the same type of information as buddyinfo except broken down by migrate-type and finishes with details on how many page blocks of each type exist.',
      'category' => 'memory'
    },
    'fields' => []
  }
};

my $fields_source =
{
  # ----------------------------------------------
  'apps' =>  {
    'src' =>
    {
      # y - must exists, n - no
      'calculated' => 'used:y - buffers:y - cached:y - slab:n - page_tables:n - cached_swap:n',
    },
    'munin' =>
    {
      'label' => 'Applications',
      'info'  => 'Memory, used by applications'
    }
  },
  # ----------------------------------------------
  'free' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'MemFree'
    },
    'munin' =>
    {
      'label' => 'Free',
      'info'  => 'The sum of LowFree+HighFree'
    }
  },
  # ----------------------------------------------
  'used' =>  {
    'src' =>
    {
      'calculated' => 'total:y - free:y',
    },
    'munin' =>
    {
      'label' => 'Used',
      'info'  => 'Used memory (see "Total" and "Free")'
    }
  },
  # ----------------------------------------------
  'usedwswap' =>  {
    'src' =>
    {
      'calculated' => 'used:y + swap_used:y',
    },
    'munin' =>
    {
      'label' => 'Used + Swap used',
      'info'  => 'See "Used" and "Swap used"'
    }
  },
  # ----------------------------------------------
  'total' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'MemTotal'
    },
    'munin' =>
    {
      'label' => 'Total',
      'info'  => 'Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code)'
    }
  },
  # ----------------------------------------------
  'buffers' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Buffers'
    },
    'munin' =>
    {
      'label' => 'Buffers',
      'info'  => 'Relatively temporary storage for raw disk blocks shouldn\'t get tremendously large (20MB or so)'
    }
  },
  # ----------------------------------------------
  'cached' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Cached'
    },
    'munin' =>
    {
      'label' => 'Cached',
      'info'  => 'In-memory cache for files read from the disk (the pagecache).  Doesn\'t include SwapCached'
    }
  },
  # ----------------------------------------------
  'cached_swap' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'SwapCached'
    },
    'munin' =>
    {
      'label' => 'Swap cached',
      'info'  => 'Memory that once was swapped out, is swapped back in but still also is in the swapfile (if memory is needed it doesn\'t need to be swapped out AGAIN because it is already in the swapfile. This saves I/O)'
    }
  },
  # ----------------------------------------------
  'qlists' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Quicklists'
    },
    'munin' =>
    {
      'label' => 'Quicklists',
      'info'  => 'QuickLists'
    }
  },
  # ----------------------------------------------
  'active' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Active'
    },
    'munin' =>
    {
      'label' => 'Active',
      'info'  => 'Memory that has been used more recently and usually not reclaimed unless absolutely necessary.'
    }
  },
  # ----------------------------------------------
  'inactive' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Inactive'
    },
    'munin' =>
    {
      'label' => 'Inactive',
      'info'  => 'Memory which has been less recently used.  It is more eligible to be reclaimed for other purposes'
    }
  },
  # ----------------------------------------------
  'active_file' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Active(file)'
    },
    'munin' =>
    {
      'label' => 'Active (file)',
      'info'  => 'See "Active"'
    }
  },
  # ----------------------------------------------
  'inactive_file' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Inactive(file)'
    },
    'munin' =>
    {
      'label' => 'Inactive (file)',
      'info'  => 'See "Inactive"'
    }
  },
  # ----------------------------------------------
  'active_anon' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Active(anon)'
    },
    'munin' =>
    {
      'label' => 'Active (anonymous)',
      'info'  => 'See "Active"'
    }
  },
  # ----------------------------------------------
  'inactive_anon' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Inactive(anon)'
    },
    'munin' =>
    {
      'label' => 'Inactive (anonymous)',
      'info'  => 'See "Inactive"'
    }
  },
  # ----------------------------------------------
  'high_total' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HighTotal'
    },
    'munin' =>
    {
      'label' => 'High total',
      'info'  => 'See "High Free"'
    }
  },
  # ----------------------------------------------
  'high_free' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HighFree'
    },
    'munin' =>
    {
      'label' => 'High free',
      'info'  => 'Highmem is all memory above ~860MB of physical memory Highmem areas are for use by userspace programs, or for the pagecache. The kernel must use tricks to access this memory, making it slower to access than lowmem.'
    }
  },
  # ----------------------------------------------
  'high_used' =>  {
    'src' =>
    {
      'calculated' => 'high_total:y - high_free:y'
    },
    'munin' =>
    {
      'label' => 'High used',
      'info'  => 'Used high memory'
    }
  },
  # ----------------------------------------------
  'low_total' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'LowTotal'
    },
    'munin' =>
    {
      'label' => 'Low total',
      'info'  => 'See "Low Free"'
    }
  },
  # ----------------------------------------------
  'low_free' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'LowFree'
    },
    'munin' =>
    {
      'label' => 'Low free',
      'info'  => 'Lowmem is memory which can be used for everything that highmem can be used for, but it is also available for the kernel\'s use for its own data structures. Among many other things, it is where everything from the Slab is allocated.  Bad things happen when you\'re out of lowmem.'
    }
  },
  # ----------------------------------------------
  'low_used' =>  {
    'src' =>
    {
      'calculated' => 'low_total:y - low_free:y'
    },
    'munin' =>
    {
      'label' => 'Low used',
      'info'  => 'Used low memory'
    }
  },
  # ----------------------------------------------
  'l_h_total' =>  {
    'src' =>
    {
      'calculated' => 'low_total:y + high_total:y'
    },
    'munin' =>
    {
      'label' => 'Total',
      'info'  => 'Total physical memory'
    }
  },
  # ----------------------------------------------
  'swap_total' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'SwapTotal'
    },
    'munin' =>
    {
      'label' => 'Swap total',
      'info'  => 'Total amount of swap space available'
    }
  },
  # ----------------------------------------------
  'swap_free' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'SwapFree'
    },
    'munin' =>
    {
      'label' => 'Swap free',
      'info'  => 'Memory which has been evicted from RAM, and is temporarily on the disk'
    }
  },
  # ----------------------------------------------
  'swap_used' =>  {
    'src' =>
    {
      'calculated' => 'swap_total:y - swap_free:y'
    },
    'munin' =>
    {
      'label' => 'Swap used',
      'info'  => 'Memory which has been evicted from RAM, and is temporarily on the disk'
    }
  },
  # ----------------------------------------------
  'dirty' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Dirty'
    },
    'munin' =>
    {
      'label' => 'Dirty',
      'info'  => 'Memory which is waiting to get written back to the disk'
    }
  },
  # ----------------------------------------------
  'writeback' =>  {
    'src'  =>
    {
      'target' => 'meminfo',
      'field' => 'Writeback'
    },
    'munin' =>
    {
      'label' => 'Writeback',
      'info'  => 'Memory which is actively being written back to the disk'
    }
  },
  # ----------------------------------------------
  'anon_pages' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'AnonPages'
    },
    'munin' =>
    {
      'label' => 'Anonymous pages',
      'info'  => 'Non-file backed pages mapped into userspace page tables'
    }
  },
  # ----------------------------------------------
  'mapped' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Mapped'
    },
    'munin' =>
    {
      'label' => 'Mapped',
      'info'  => 'Files which have been mapped, such as libraries'
    }
  },
  # ----------------------------------------------
  'mmap_copy' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'MmapCopy'
    },
    'munin' =>
    {
      'label' => 'mmap copy',
      'info'  => 'Indicates the amount of RAM currently allocated by mmap to hold mappable regions that can\'t be mapped directly.  These are copies of the backing device or file if not anonymous.'
    }
  },
  # ----------------------------------------------
  'slab' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Slab'
    },
    'munin' =>
    {
      'label' => 'Slab',
      'info'  => 'In-kernel data structures cache'
    }
  },
  # ----------------------------------------------
  'slab_reclaimable' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'SReclaimable'
    },
    'munin' =>
    {
      'label' => 'Slab reclaimable',
      'info'  => 'Part of Slab, that might be reclaimed, such as caches'
    }
  },
  # ----------------------------------------------
  'slab_unreclaimable' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'SUnreclaim'
    },
    'munin' =>
    {
      'label' => 'Slab unreclaimable',
      'info'  => 'Part of Slab, that cannot be reclaimed on memory pressure'
    }
  },
  # ----------------------------------------------
  'page_tables' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'PageTables'
    },
    'munin' =>
    {
      'label' => 'Page tables',
      'info'  => 'amount of memory dedicated to the lowest level of page tables'
    }
  },
  # ----------------------------------------------
  'nfs_unstable' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'NFS_Unstable'
    },
    'munin' =>
    {
      'label' => 'NFS unstable',
      'info'  => 'NFS pages sent to the server, but not yet committed to stable <--> storage'
    }
  },
  # ----------------------------------------------
  'bounce' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Bounce'
    },
    'munin' =>
    {
      'label' => 'Bounce',
      'info'  => 'Memory used for block device "bounce buffers"'
    }
  },
  # ----------------------------------------------
  'writeback_fuse' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'WritebackTmp'
    },
    'munin' =>
    {
      'label' => 'FUSE buffers',
      'info'  => 'Memory used by FUSE for temporary writeback buffers'
    }
  },
  # ----------------------------------------------
  'commit_limit' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'CommitLimit'
    },
    'munin' =>
    {
      'label' => 'Commit limit',
      'info'  => 'Based on the overcommit ratio ("vm.overcommit_ratio"), this is the total amount of  memory currently available to be allocated on the system. This limit is only adhered to if strict overcommit accounting is enabled (mode 2 in "vm.overcommit_memory"). The CommitLimit is calculated with the following formula=> CommitLimit = ("vm.overcommit_ratio" * Physical RAM) + Swap For example, on a system with 1G of physical RAM and 7G of swap with a `vm.overcommit_ratio` of 30 it would yield a CommitLimit of 7.3G. For more details, see the memory overcommit documentation in vm/overcommit-accounting.',
    }
  },
  # ----------------------------------------------
  'committed_as' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Committed_AS'
    },
    'munin' =>
    {
      'label' => 'Committed AS',
      'info'  => 'The amount of memory presently allocated on the system. The committed memory is a sum of all of the memory which has been allocated by processes, even if it has not been "used" by them as of yet. A process which malloc()\'s 1G of memory, but only touches 300M of it will only show up as using 300M of memory even if it has the address space allocated for the entire 1G. This 1G is memory which has been "committed" to by the VM and can be used at any time by the allocating application. With strict overcommit enabled on the system (mode 2 in "vm.overcommit_memory"), allocations which would exceed the CommitLimit (detailed above) will not be permitted. This is useful if one needs to guarantee that processes will not fail due to lack of memory once that memory has been successfully allocated.'
    }
  },
  # ----------------------------------------------
  'vmalloc_total' =>  {
    'src'  =>
    {
      'target' => 'meminfo',
      'field' => 'VmallocTotal'
    },
    'munin' =>
    {
      'label' => 'Vmalloc total',
      'info'  => 'Total size of vmalloc memory area'
    }
  },
  # ----------------------------------------------
  'vmalloc_used' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'VmallocUsed'
    },
    'munin' =>
    {
      'label' => 'Vmalloc used',
      'info'  => 'Amount of vmalloc area which is used'
    }
  },
  # ----------------------------------------------
  'vmalloc_chunk' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'VmallocChunk'
    },
    'munin' =>
    {
      'label' => 'Vmalloc chunk',
      'info'  => 'largest contiguous block of vmalloc area which is free'
    }
  },
  # ----------------------------------------------
  'vmalloc_free' =>  {
    'src' =>
    {
      # y - must exists, n - no
      'calculated' => 'vmalloc_total:y - vmalloc_used:y'
    },
    'munin' =>
    {
      'label' => 'Vmalloc free',
      'info'  => 'Free size of vmalloc memory area'
    }
  },
  # ----------------------------------------------
  'mlocked' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Mlocked'
    },
    'munin' =>
    {
      'label' => 'Mlocked',
      'info'  => 'Mlocked'
    }
  },
  # ----------------------------------------------
  'shmem' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Shmem'
    },
    'munin' =>
    {
      'label' => 'Shared memory',
      'info'  => 'Shared memory'
    }
  },
  # ----------------------------------------------
  'kernel_stack' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'KernelStack'
    },
    'munin' =>
    {
      'label' => 'Kernel stack',
      'info'  => 'Kernel stack'
    }
  },
  # ----------------------------------------------
  'hardware_corrupted' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HardwareCorrupted'
    },
    'munin' =>
    {
      'label' => 'Hardware corrupted',
      'info'  => 'Hardware corrupted'
    }
  },
  # ----------------------------------------------
  'huge_pages_total' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HugePages_Total'
    },
    'munin' =>
    {
      'label' => 'HugePages total',
      'info'  => 'Huge Pages is a feature available in later Linux kernels that provides two important benefits: 1. Locks the memory available to huge pages, so it cannot be paged to disk 2. Make the TLB (translation lookaside buffer) much smaller on the processor, as the number of entries is much smaller. This is due to the fact the standard page size in Linux is 4K, whereas a huge page is either 2MB of 4MB in size. This makes managing virtual memory much more efficient, as the processor does not have to work as hard switching pages in and out.'
    }
  },
  # ----------------------------------------------
  'huge_pages_free' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HugePages_Free'
    },
    'munin' =>
    {
      'label' => 'HugePages free',
      'info'  => 'See "HugePages_Total"'
    }
  },
  # ----------------------------------------------
  'huge_pages_rsvd' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HugePages_Rsvd'
    },
    'munin' =>
    {
      'label' => 'HugePages reserved',
      'info'  => 'See "HugePages_Total"'
    }
  },
  # ----------------------------------------------
  'huge_pages_surp' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'HugePages_Surp'
    },
    'munin' =>
    {
      'label' => 'HugePages surp',
      'info'  => 'See "HugePages_Total"'
    }
  },
  # ----------------------------------------------
  'huge_pages_total_size' =>  {
    'src' =>
    {
      'calculated' => 'huge_page_size:y * huge_pages_total:y'
    },
    'munin' =>
    {
      'label' => 'HugePages size',
      'info'  => 'Total size of HugePages'
    }
  },
  # ----------------------------------------------
  'huge_pages_free_size' =>  {
    'src' =>
    {
      'calculated' => 'huge_page_size:y * huge_pages_free:y'
    },
    'munin' =>
    {
      'label' => 'HugePages free',
      'info'  => 'Free HugePages memory size'
    }
  },
  # ----------------------------------------------
  'huge_pages_rsvd_size' =>  {
    'src' =>
    {
      'calculated' => 'huge_page_size:y * huge_pages_rsvd:y'
    },
    'munin' =>
    {
      'label' => 'HugePages rsvd',
      'info'  => 'HugePages reserved size'
    }
  },
  # ----------------------------------------------
  'huge_pages_surp_size' =>  {
    'src' =>
    {
      'calculated' => 'huge_page_size:y * huge_pages_surp:y'
    },
    'munin' =>
    {
      'label' => 'HugePages surp',
      'info'  => 'HugePages surp size'
    }
  },
  # ----------------------------------------------
  'huge_page_size' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'Hugepagesize'
    },
    'munin' =>
    {
      'label' => 'HugePage size',
      'info'  => 'HugePage size'
    }
  },
  # ----------------------------------------------
  'direct_map_4k' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'DirectMap4k'
    },
    'munin' =>
    {
      'label' => 'Direct map 4k',
      'info'  => 'Direct map 4k'
    }
  },
  # ----------------------------------------------
  'direct_map_2m' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'DirectMap2M'
    },
    'munin' =>
    {
      'label' => 'Direct map 2M',
      'info'  => 'Direct map 2M'
    }
  },
  # ----------------------------------------------
  'direct_map_4m' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'DirectMap4M'
    },
    'munin' =>
    {
      'label' => 'Direct map 4M',
      'info'  => 'Direct map 4M'
    }
  },
  # ----------------------------------------------
  'direct_map_1g' =>  {
    'src' =>
    {
      'target' => 'meminfo',
      'field' => 'DirectMap1G'
    },
    'munin' =>
    {
      'label' => 'Direct map 1G',
      'info'  => 'Direct map 1G'
    }
  },
  # ---------------------------------------------- slabinfo -------------
  # ----------------------------------------------
  'min_obj_size' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'min_obj_size'
    },
    'munin' =>
    {
      'label' => 'Min. object size',
      'info'  => 'Minimal object size'
    }
  },
  # ----------------------------------------------
  'avg_obj_size' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'avg_obj_size'
    },
    'munin' =>
    {
      'label' => 'Avg. object size',
      'info'  => 'Average object size'
    }
  },
  # ----------------------------------------------
  'max_obj_size' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'max_obj_size'
    },
    'munin' =>
    {
      'label' => 'Max. object size',
      'info'  => 'Maximal object size'
    }
  },
  # ----------------------------------------------
  'total_objects' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'total_objects'
    },
    'munin' =>
    {
      'label' => 'Total objects',
      'info'  => 'Total objects'
    }
  },
  # ----------------------------------------------
  'active_objects' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'active_objects'
    },
    'munin' =>
    {
      'label' => 'Active objects',
      'info'  => 'Active objects'
    }
  },
  # ----------------------------------------------
  'inactive_objects' =>  {
    'src' =>
    {
      'calculated' => 'total_objects:y - active_objects:y'
    },
    'munin' =>
    {
      'label' => 'Inactive objects',
      'info'  => 'Inactive objects'
    }
  },
  # ----------------------------------------------
  'total_slabs' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'total_slabs'
    },
    'munin' =>
    {
      'label' => 'Total slabs',
      'info'  => 'Total slabs'
    }
  },
  # ----------------------------------------------
  'active_slabs' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'active_slabs'
    },
    'munin' =>
    {
      'label' => 'Active slabs',
      'info'  => 'Active slabs'
    }
  },
  # ----------------------------------------------
  'shared_slabs' =>  {
    'src' =>
    {
      'target' => 'slabinfo',
      'field' => 'shared_slabs'
    },
    'munin' =>
    {
      'label' => 'Shared slabs',
      'info'  => 'Shared slabs'
    }
  },
  # ----------------------------------------------
  'inactive_slabs' =>  {
    'src' =>
    {
      'calculated' => 'total_slabs:y - active_slabs:y'
    },
    'munin' =>
    {
      'label' => 'Inactive slabs',
      'info'  => 'Inactive slabs'
    }
  }
};

# appdata descriptions
my $application_fields =
{
  'label' =>
  {
    'VmLck'  => 'Locked in mem',
    'VmHWM'  => 'Peak resident',
    'VmPeak' => 'Peak virtual',
    'VmSwap' => 'Swap usage',
    'VmExe'  => 'Code segment',
    'VmPTE'  => 'Page table entries',
    'VmData' => 'Data segment',
    'VmLib'  => 'Shared library',
    'VmSize' => 'Allocated virtual',
    'VmRSS'  => 'Mapped in RAM',
    'VmStk'  => 'Stack'
   },
  'info' =>
  {
    'VmLck'  => 'The amount of locked memory',
    'VmHWM'  => 'Peak resident set size ("high water mark")',
    'VmPeak' => 'Peak virtual memory size',
    'VmSwap' => 'Size of swap usage (the number of referred swapents)',
    'VmExe'  => 'The size of the executable segment',
    'VmPTE'  => 'Size of page table entries',
    'VmData' => 'The size of the Data segment',
    'VmLib'  => 'Size of shared library code',
    'VmSize' => 'The size of the virtual memory allocated to the process',
    'VmRSS'  => 'The amount of memory mapped in RAM ( instead of swapped out )',
    'VmStk'  => 'The stack size'
   },
  'draw' =>
  {
    'VmSize' => 'AREA',
    'VmRSS'  => 'AREA',
    'VmStk'  => 'STACK',
    'VmData' => 'STACK',
    'VmExe'  => 'LINE1.5',
    'VmLib'  => 'LINE1.5',
    'VmLck'  => 'LINE1',
    'VmPTE'  => 'LINE1',
    'VmSwap' => 'AREA',
    'VmHWM'  => 'LINE2',
    'VmPeak' => 'LINE2'

   },
   'color' =>
   {
     'VmSize' => '0066CCBB',
     'VmPeak' => '000000',
     'VmRSS'  => '00FF33BB',
     'VmStk'  => 'FFCC55BB',
     'VmData' => '00FFFFBB',
     'VmExe'  => 'FF1100',
     'VmLib'  => 'DD9900',
     'VmLck'  => 'AA00AA',
     'VmPTE'  => '7E7E7E',
     'VmSwap' => 'FF447788',
     'VmHWM'  => 'FFFFFF',
   },
   'order' => [qw(VmSize VmPeak VmRSS VmStk VmData VmSwap VmExe VmLib VmLck VmPTE VmHWM)]
};


# ----------------- main ----------------
need_multigraph();

my @graphs_to_sort = ();
my $stats          = {};
my $storage        = {};
my $applications   = {};
my $enabled_graphs       = $ENV{'enabled_graphs'}     || '.*';
$applications->{'show'}  = $ENV{'applications'}       || undef; # '(psi|chrome|mc|\w{1,3}sh|firefox)';
$applications->{'group'} = $ENV{'applications_group'} || undef; # '(chrome|firefox):browsers;\w{1,3}sh:sh';
$applications->{'wait'}  = $ENV{'application_wait'}   || 1800;

restore_state_data();
load_stats();
save_state_data();

if (defined($ARGV[0]) and ($ARGV[0] eq 'autoconf'))
{
  printf("%s\n", -e "/proc/meminfo" ? "yes" : "no (/proc/meminfo not exists)");
  exit (0);
}

if (defined($ARGV[0]) and ($ARGV[0] eq 'config'))
{
  sort_graphs_fields();
  print_config();
  exit (0);
}

print_values();
exit(0);

# ---------------------------------------- data load ---------------------------------
# ----------------------------------- saving state data using munin --------------------
sub save_state_data
{
  $storage->{'timestamp'}{'previous'} = time();
  my $d = Data::Dumper->new([$storage]);
  $d->Indent(0);
  save_state($d->Dump);
}

# -------------------------------- loading previous state data using munin -------------------
sub restore_state_data
{
  my $VAR1;
  my $states = (restore_state())[0];
  eval $states if defined $states;
  $storage = $VAR1;
  $storage->{'timestamp'}{'current'} = time();
}

# -------------------------- loading memory info ---------------------------------
sub load_meminfo
{
  my $file = "/proc/meminfo";
  my $result = {};
  open (FH, '<', $file) or die "$! $file \n";
  my ($var, $val) = ('',0);
  for my $line (<FH>)
  {
    chomp $line;
    ($var, $val) = split(/:?\s+/, $line);
    $val = $val * 1024 unless ($var =~ m/HugePages_.*/);
    $result->{$var} = $val;
  }
  close (FH);
  return $result;
}
# -------------------------- loading vmalloc info ---------------------------------
sub load_vmallocinfo
{
  my $file = "/proc/vmallocinfo";
  my $result = {};
  open (FH, '<', $file) or die "$! $file \n";
  my @tokens = ();
  my $name = '';
  for my $line (<FH>)
  {
    chomp $line;
    @tokens =  split(/\s+/, $line);
    $name     = (split(/\+/ , $tokens[2]))[0];
    $result->{$name}  = 0 unless (exists($result->{$name}));
    $result->{$name} += $tokens[1];

    my $field_name = sprintf("vi_%s", clean_fieldname($name));
    append_field         ($field_name, 'vmallocinfo', $name, $name, $name);
    append_field_to_graph($field_name, 'meminfo_virtual.vmalloc_allocates', 'AREASTACK', '-');
  }
  push(@graphs_to_sort, 'meminfo_virtual.vmalloc_allocates');
  close (FH);
  #print Dumper $result;

  return $result;
}
# -------------------------- loading slab info ---------------------------------
sub load_slabinfo
{
  my $file = "/proc/slabinfo";
  my $result = {};
  open (FH, '<', $file) or die "$! $file \n";
  my $groups = {};

  my ($temp, $objects, $tunables, $slabdata) = ('','','','');
  # objects
  my ($name, $active_objs, $num_objs, $objsize, $objperslab, $pagesperslab) = ('','','','','','');
  # tunables
  #my ($limit, $batchcount, $sharedfactor) = ('','','');
  # slabdata
  my ($active_slabs, $num_slabs, $sharedavail) = ('','','');

  my ($min_obj_size, $max_obj_size, $total_active_objects, $total_objects_size, $total_objects) = (1024, 0, 0, 0);
  my ($total_slabs, $total_active_slabs, $total_shared_slabs) = (0, 0, 0);
  # name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
  for my $line (<FH>)
  {
    next if $line =~ m/^(#\s+|slabinfo)/;
    chomp $line;
    ($objects, $tunables, $slabdata)                                       = split(/\s+:\s+/, $line    );
    ($name, $active_objs, $num_objs, $objsize, $objperslab, $pagesperslab) = split(/\s+/    , $objects );
    #($temp, $limit, $batchcount, $sharedfactor)                            = split(/\s+/    , $tunables);
    ($temp, $active_slabs, $num_slabs, $sharedavail)                       = split(/\s+/    , $slabdata);
    # splitting slabs to groups
    if ($name =~ m/((TC|UD|SCT)P|RAW|UNIX|inet|ip6)/i)
    {
      push(@{$groups->{'network'}}, $name);
    }
    elsif($name =~ m/[-_]/)
    {
      my @parts = split(/[-_]/, $name);
      push(@{$groups->{$parts[0]}}, $name);
    }
    else
    {
      push(@{$groups->{'other'}}, $name);
    }

    if($num_objs > 0)
    {
      $min_obj_size          = $objsize if $objsize < $min_obj_size;
      $max_obj_size          = $objsize if $objsize > $max_obj_size;
      $total_objects_size   += $objsize * $num_objs;
      $total_objects        += $num_objs;
      $total_active_objects += $active_objs;
    }
    if($num_slabs > 0)
    {
      $total_slabs        += $num_slabs;
      $total_active_slabs += $active_slabs;
      $total_shared_slabs += $sharedavail;
    }
    # foreach slab field append this field to global fields list
    my $field_name = sprintf("slab_%s_size", clean_fieldname($name));
    append_field($field_name, 'slabinfo', $field_name, $name, '');
    $result->{$field_name} = $objsize*$num_objs;
  }
  # second iteration for better splitting slabs to groups
  for my $group ( keys %{$groups} )
  {
    if ( scalar(@{$groups->{$group}}) == 1 )
    {
      my $name = $groups->{$group}[0];
      if  ($name =~ m/cache/)  { push(@{$groups->{'caches'}}, $name); }
      else                     { push(@{$groups->{'other'}}, $name) ; }
      delete($groups->{$group});
    }
  }
  # appending grsaph of slabs groups to global graphs list

  # groups graphs
  for my $group ( sort keys %{$groups} )
  {
    my @group_size_formula = ();
    # appending group graph to global graphs list
    my $graph_name = sprintf("slab.slabs_size_%s", $group);
    append_graph($graph_name, '--base 1024',
                 sprintf("Slabs size [%s]"                , $group),
                 sprintf("Slabs size jf group, named '%s'", $group),
                 'bytes', 'groups_size', 'Total');
    # appending fields to graph and calculating formula
    for my $group_field ($groups->{$group}[0] =~ m/-\d+(\(.*\))?$/ ? @{$groups->{$group}} : sort @{$groups->{$group}})
    {
      my $group_field_name = sprintf("slab_%s_size", clean_fieldname($group_field));
      append_field_to_graph($group_field_name, $graph_name, 'AREASTACK', '-');
      push (@group_size_formula, $group_field_name)
    }
    # appending `summ of group` field to global fields list
    my $summ_field_name = sprintf("slab_size_summ_%s", clean_fieldname($group));
    append_formula_field ($summ_field_name, join(":n + ", @group_size_formula).':n', $group, '');
    append_field_to_graph($summ_field_name, 'slab.slabs_size_all_groups', 'AREASTACK', '-');
  }
  #$graphs_source->{'slab.slabs_size_all_groups'}{'fields'} = sort(@{$graphs_source->{'slab.slabs_size_all_groups'}{'fields'}});
  #print Dumper $groups;
  $result->{'min_obj_size'}   = $min_obj_size;
  $result->{'avg_obj_size'}   = $total_objects_size / $total_objects;
  $result->{'max_obj_size'}   = $max_obj_size;
  $result->{'total_objects'}  = $total_objects;
  $result->{'active_objects'} = $total_active_objects;
  $result->{'total_slabs'}    = $total_slabs;
  $result->{'active_slabs'}   = $total_active_slabs;
  $result->{'shared_slabs'}   = $total_shared_slabs;
  close (FH);
  return $result;
}
# -------------------------- loading buddyinfo ---------------------------------
sub load_buddyinfo
{
  my $file = "/proc/buddyinfo";
  my $result = {};
  open (FH, '<', $file) or die "$! $file \n";
  my ($node, $zone, $info) = ('','','');
  my ($graph_name, $field_name, $summ_field_name, $num) = ('','','',0);
  my @splitted_info = ();
  for my $line (<FH>)
  {
    chomp $line;
    ($node, $info) = split(/,\s+zone\s+/, $line);
    @splitted_info = split(/\s+/, $info);
    $zone = shift (@splitted_info);
    $summ_field_name = clean_fieldname(sprintf("summ_%s_%s", $node, $zone));
    append_field         ($summ_field_name, 'buddyinfo', $summ_field_name, sprintf("%s, zone %s", $node, $zone), 'Chunks count');
    append_field_to_graph($summ_field_name, 'buddyinfo', 'AREASTACK', '-');
    $result->{$summ_field_name} = 0;
    $graph_name = 'buddyinfo.'.clean_fieldname(sprintf("%s_%s", $node, $zone));
    append_graph($graph_name, '--base 1000',
                 sprintf("%s, zone %s"          , $node, $zone),
                 sprintf("Chunks of %s, zone %s", $node, $zone),
                 'chunks', $node, 'Total');
    $num = 0;

    for my $chunk (@splitted_info)
    {
      $result->{$summ_field_name} += $chunk;
      $field_name = clean_fieldname(sprintf("chunk_%s_%s_%s", $node, $zone, $num));
      append_field         ($field_name, 'buddyinfo', $field_name, sprintf("Chunks of (2^%2d) * Page size", $num), 'Chunks count');
      append_field_to_graph($field_name, $graph_name, 'AREASTACK', '-');
      $num++;
      $result->{$field_name} = $chunk;
    }
  }
  close (FH);
  return $result;
}

# ---------------- applications info -----------
sub summ_app_usage
{
  my $app_data = $_[0];
  return exists($app_data->{'data'}{'VmData'}) ? $app_data->{'data'}{'VmData'} : 0 +
         exists($app_data->{'data'}{'VmRSS'})  ? $app_data->{'data'}{'VmRSS'}  : 0 +
         exists($app_data->{'data'}{'VmStk'})  ? $app_data->{'data'}{'VmStk'}  : 0;
}
sub load_appinfo
{
  my $result = {};
  my $procdir = IO::Dir->new("/proc");
  if(defined $procdir)
  {
    push(@graphs_to_sort, qw(appinfo appgroupinfo appinfo.processes));
    my $current_time = time();
    my $status_file = '';
    my ($pid, $label, $data, $app_name, $field_name, $summ_field_name, $processes_field_name, $group_summ_field_name, $graph_name, $bytes) = (0,'','','','','','','','',0);
    #loading application groups
    my $groups = {};
    if(defined($applications->{'group'}))
    {
      my ($g_name, $g_reg) = ('','');
      for my $gr (split(/;/, $applications->{'group'}))
      {
        ($g_reg, $g_name) = split(/:/, $gr);
        $groups->{$g_name} = $g_reg;
      }
    }

    # for each /proc/{pid}
    while (defined ($pid = $procdir->read))
    {
      next unless $pid =~ m/\d+/; # only pids
      my $status_file = sprintf("/proc/%s/status", $pid);
      my $app_data = {};
      open (FH, '<', $status_file) or next;
      # collect application data
      for my $line (<FH>)
      {
        chomp $line;
        ($label, $data) = split(/:\s+/, $line);
        $app_data->{'name'} = $data if $label eq 'Name';
        $app_data->{'data'}{$label} = (split(/\s+/, $data))[0] * 1024 if($label =~ m/^Vm.*/);
      }
      close (FH);
      # application data has memory information?
      if(exists($app_data->{'data'}{'VmSize'}))
      {

        # generate graphs/fields, collect values
        # application in list?
        if(defined($applications->{'show'}) and $app_data->{'name'} =~ m/$applications->{'show'}/)
        {
          $storage->{'applications'}{'single'}{$app_data->{'name'}} = $current_time;
          append_application($app_data, $result);
        }
        for my $group (keys %{$groups})
        {
          if($app_data->{'name'} =~ m/$groups->{$group}/)
          {
            $storage->{'applications'}{'groups'}{$group}{$app_data->{'name'}} = $current_time;
            append_application_to_group($app_data, $group, $result);
          }
        }
      }
    }
    # detect and show applications that are not currently running
    my $dummy_app_data = {};
    for my $app (keys %{$storage->{'applications'}{'single'}})
    {
      next if($storage->{'applications'}{'single'}{$app} == $current_time);
      if($current_time - $storage->{'applications'}{'single'}{$app} >= $applications->{'wait'})
      {
        delete ($storage->{'applications'}{'single'}{$app});
        next;
      }
      $dummy_app_data->{'name'} = $app;
      append_application($dummy_app_data, $result);
    }
    for my $group (keys %{$storage->{'applications'}{'groups'}})
    {
      for my $app (keys %{$storage->{'applications'}{'groups'}{$group}})
      {
        next if($storage->{'applications'}{'groups'}{$group}{$app} == $current_time);
        if ($current_time - $storage->{'applications'}{'groups'}{$group}{$app} >= $applications->{'wait'})
        {
          delete ($storage->{'applications'}{'groups'}{$group}{$app});
          next;
        }
        $dummy_app_data->{'name'} = $app;
        append_application_to_group($dummy_app_data, $group, $result);
      }
    }
  }
  return $result;
}

sub append_application
{
  my ($app_data, $result) = @_[0..1];

  # field for summary
  my $summ_field_name = clean_fieldname(sprintf("summ_%s", $app_data->{'name'}));
  append_field         ($summ_field_name, 'appinfo', $summ_field_name, $app_data->{'name'}, $app_data->{'name'});
  append_field_to_graph($summ_field_name, 'appinfo', 'LINE1', '-');

  # field for processes count
  my $processes_field_name = clean_fieldname(sprintf("pr_%s", $app_data->{'name'}));
  append_field         ($processes_field_name, 'appinfo', $processes_field_name, $app_data->{'name'}, sprintf("%s processes count", $app_data->{'name'}));
  append_field_to_graph($processes_field_name, 'appinfo.processes', 'LINE1', '-');

  # values
  $result->{$summ_field_name}       = 0     unless  (exists($result->{$summ_field_name}));
  $result->{$processes_field_name}  = 0     unless  (exists($result->{$processes_field_name}));
  $result->{$processes_field_name} ++       if      (exists($app_data->{'data'}));
  $result->{$summ_field_name}      += summ_app_usage($app_data) if exists($app_data->{'data'});
  $result->{$summ_field_name}       = 'NaN' unless  (exists($app_data->{'data'}));

  # per-application graphs
  my $graph_name = sprintf("appinfo.app_%s", clean_fieldname($app_data->{'name'}));
  append_graph($graph_name, '--base 1024',
               sprintf("Summary `%s` memory usage"                , $app_data->{'name'}),
               sprintf("All `%s` processes summ by each parameter", $app_data->{'name'}),
              'bytes', 'summary');

  for my $label (@{$application_fields->{'order'}})
  {
    my $field_name = clean_fieldname(sprintf("app_%s_%s", $app_data->{'name'}, $label));
    append_field         ($field_name, 'appinfo', $field_name, $application_fields->{'label'}{$label}, $application_fields->{'info'}{$label});
    append_field_to_graph($field_name, $graph_name, $application_fields->{'draw'}{$label}, $application_fields->{'color'}{$label});

    $result->{$field_name}  =     0 unless (exists($result->{$field_name}));
    $result->{$field_name} += $app_data->{'data'}{$label} if (exists($app_data->{'data'}{$label}));
    $result->{$field_name}  = 'NaN' unless (exists($app_data->{'data'}{$label}));
  }
  #return $graph_name;
}

sub append_application_to_group
{
  my ($app_data, $group, $result) = @_[0..2];
  my $group_summ_field_name = clean_fieldname(sprintf("group_summ_%s", $group));
  append_field         ($group_summ_field_name, 'appinfo', $group_summ_field_name, $group, $group);
  append_field_to_graph($group_summ_field_name, 'appgroupinfo', 'LINE1', '-');

  # per process in group
  my $graph_name = sprintf("appgroupinfo.group_%s_applications", clean_fieldname($group));
  append_graph($graph_name, '--base 1024',
               sprintf("Group `%s` applications memory usage", $group),
               sprintf("All group `%s` applications summ"    , $group),
               'bytes', 'applications', 'Total');
  push(@graphs_to_sort, $graph_name);

  my $summ_field_name = clean_fieldname(sprintf("group_%s_proc_%s", $group, $app_data->{'name'}));
  append_field         ($summ_field_name, 'appinfo', $summ_field_name, $app_data->{'name'}, $app_data->{'name'});
  append_field_to_graph($summ_field_name, $graph_name, 'AREASTACK', '-');

  $result->{$summ_field_name}        = 0     unless (exists($result->{$summ_field_name}));
  $result->{$summ_field_name}       += summ_app_usage($app_data) if exists($app_data->{'data'});
  $result->{$summ_field_name}        = 'NaN' unless (exists($app_data->{'data'}));
  $result->{$group_summ_field_name}  = 0     unless (exists($result->{$group_summ_field_name}));
  $result->{$group_summ_field_name} += $result->{$summ_field_name} if exists($app_data->{'data'});

  # per group summary
  $graph_name = sprintf("appgroupinfo.group_%s_summ", clean_fieldname($group));
  append_graph($graph_name, '--base 1024',
                 sprintf("Summary group `%s` memory usage"                , $group),
                 sprintf("All group `%s` processes summ by each parameter", $group),
                 'bytes', 'summary');

  for my $label (@{$application_fields->{'order'}})
  {
    my $field_name = clean_fieldname(sprintf("group_%s_%s", $group, $label));
    append_field         ($field_name, 'appinfo', $field_name, $application_fields->{'label'}{$label}, $application_fields->{'info'}{$label});
    append_field_to_graph($field_name, $graph_name, $application_fields->{'draw'}{$label}, $application_fields->{'color'}{$label});

    $result->{$field_name}  = 0 unless (exists($result->{$field_name}));
    $result->{$field_name} += $app_data->{'data'}{$label} if exists($app_data->{'data'}{$label});
  }
}

# -------------------------- loading pagetype info ---------------------------------
sub load_pagetypeinfo
{
  my $file = "/proc/pagetypeinfo";
  my $result = {};
  my ($temp, $node, $zone, $type) = ('','','','');
  my ($field_name, $graph_name) = ('','');
  my @types = ();
  my @values = ();
  open (FH, '<', $file) or die "$! $file \n";
  for my $line (<FH>)
  {
    chomp $line;
    next if($line =~ m/^(Page|Free|\s*)$/);
    if ($line =~ m/^Number/)
    {
      my $num = 0;
      for $temp (split(/\s+/, $line))
      {
        push(@types, $temp) if ($num>3);
        $num++;
      }
      next;
    }
    if ($line =~ m/,\s+type/)
    {
      ($node, $zone, $temp) = split(/,\s+/, $line);
      $node   = (split(/\s+/, $node))[1];
      $zone   = (split(/\s+/, $zone))[1];
      @values =  split(/\s+/, $temp);
      shift(@values); $type = shift(@values);
      $graph_name = 'pagetypeinfo.'.clean_fieldname(sprintf("n%s_z%s_t%s", $node, $zone, $type));
      append_graph($graph_name, '--base 1024',
                   sprintf("Node %s, zone %s, type %s"                                    , $node, $zone, $type),
                   sprintf("Free pages count per migrate type (Node %s, zone %s, type %s)", $node, $zone, $type),
                   'pages', 'free');
      my $num = 0;
      for my $value (@values)
      {
        $field_name = clean_fieldname(sprintf("fp_n%s_z%s_t%s_o%s", $node, $zone, $type, $num));
        append_field         ($field_name, 'pagetypeinfo', $field_name, sprintf("Free pages at order %2s", $num), sprintf("Free pages count per migrate type at order %2s", $num));
        append_field_to_graph($field_name, $graph_name, 'AREASTACK', '-');
        $result->{$field_name} = $value;
        $num++;
      }
      next;
    }
    if($line =~ m/,\s+zone/)
    {
      ($node, $temp) = split(/,\s+/, $line);
      $node   = (split(/\s+/, $node))[1];
      @values =  split(/\s+/, $temp);
      shift(@values); $zone = shift(@values);
      my $num = 0;
      for my $value (@values)
      {
        $field_name = clean_fieldname(sprintf("blocks_n%s_z%s_t%s", $node, $zone, $types[$num]));
        append_field         ($field_name, 'pagetypeinfo', $field_name, sprintf("Node %2s, zone %9s, type %11s", $node, $zone, $types[$num]), sprintf("Number of blocks type %s", $types[$num]));
        append_field_to_graph($field_name, 'pagetypeinfo', 'AREASTACK', '-');
        $result->{$field_name} = $value;
        $num++;
      }
    }
  }
  close (FH);
  return $result;
}


# ----------------------- all stats ------------------
sub load_stats
{
  $stats->{'meminfo'}      = load_meminfo()      if (-e "/proc/meminfo"       and -r "/proc/meminfo"                      );
  $stats->{'slabinfo'}     = load_slabinfo()     if (-e "/proc/slabinfo"      and -r "/proc/slabinfo"                     );
  $stats->{'vmallocinfo'}  = load_vmallocinfo()  if (-e "/proc/vmallocinfo"   and -r "/proc/vmallocinfo"                  );
  $stats->{'buddyinfo'}    = load_buddyinfo()    if (-e "/proc/buddyinfo"     and -r "/proc/buddyinfo"                    );
  $stats->{'pagetypeinfo'} = load_pagetypeinfo() if (-e "/proc/pagetypeinfo"  and -r "/proc/pagetypeinfo"                 );
  $stats->{'appinfo'}      = load_appinfo()      if (defined($applications->{'show'}) or defined($applications->{'group'}));
  #print Dumper $stats;
}
# appending graphs and fields
sub append_field
{
  my ($field_name, $target, $name, $label, $info) = @_[0..4];
  unless (exists($fields_source->{$field_name}))
  {
    $info = "$name" unless defined $info && $info ne '';
    $fields_source->{$field_name} =
    {
      'src'   => { 'target' => $target, 'field' => $name },
      'munin' => { 'label'  => $label,  'info'  => $info }
    };
  }
}

sub append_formula_field
{
  my ($field_name, $formula, $label, $info) = @_[0..3];
  unless (exists($fields_source->{$field_name}))
  {
    $fields_source->{$field_name} =
    {
      'src'   => { 'calculated' => $formula },
      'munin' => { 'label' => $label, 'info'  => $info }
    };
  }
}

sub append_field_to_graph
{
  my ($field_name, $graph_name, $draw, $color) = @_[0..3];
  my $item = sprintf("%s:%s:%s", $field_name, $draw, $color);
  push(@{$graphs_source->{$graph_name}{'fields'}}, $item) unless (grep $_ eq $item, @{$graphs_source->{$graph_name}{'fields'}});
}

sub append_graph
{
  my ($graph_name, $args, $title, $info, $vtitle, $category, $total) = @_[0..6];
  unless (exists($graphs_source->{$graph_name}))
  {
    $graphs_source->{$graph_name} =
    {
      'munin' =>
      {
        'args'     => $args,    'title'    => $title,
        'info'     => $info,    'vtitle'   => $vtitle,
        'category' => $category
      },
      'fields' => []
    };
    if (defined($total)) { $graphs_source->{$graph_name}{'munin'}{'total'} = $total; }
  }
}

sub sort_graphs_fields
{
  for my $graph_name (@graphs_to_sort)
  {
      my @fields = ();
      push  (@fields, @{$graphs_source->{$graph_name}{'fields'}});
      delete($graphs_source->{$graph_name}{'fields'});
      push  (@{$graphs_source->{$graph_name}{'fields'}}, sort(@fields));
  }
}

# --------- convert num[kMG] values to numbers --------------------------------
sub convert_postfixed
{
  my $src = $_[0];
  $src =~ s/,/./ig;
  my $result = $src;
  my ($num, $suff) = $src =~ /([\d\.]+)(\w)/;
  if   ($suff eq 'k') { $result = $num * 1024;               }
  elsif($suff eq 'M') { $result = $num * 1024 * 1024;        }
  elsif($suff eq 'G') { $result = $num * 1024 * 1024 * 1024; }
  return $result;
}

# ---------------- loading limits -------------
sub load_limits
{
  for my $field_name (keys (%{$fields_source}))
  {
    my $limit_name = sprintf("limit_%s", $field_name);
    my $limit = $ENV{$limit_name} || undef;
    if(defined($limit))
    {
      my ($crit, $warn) = (undef, undef);
      if($limit =~ m/:/) { ($warn, $crit) = split(/:/, $limit); }
      else               {  $crit = $limit;                     }
      $fields_source->{$field_name}{'munin'}{'warning'}  = convert_postfixed($warn) if defined($warn);
      $fields_source->{$field_name}{'munin'}{'critical'} = convert_postfixed($crit) if defined($crit);
    }
  }
}


# --------------------------------------- both -----------------------------
# check graph enabled or not
sub graph_is_enabled
{
  return $_[0] =~ m/$enabled_graphs/ig;
}
# extract calculated fields
my $calculated_sources = {};
sub calculated_sources
{
  my $field_name = $_[0];
  unless(exists($calculated_sources->{$field_name}))
  {
    my $field = $fields_source->{$field_name};
    $calculated_sources->{$field_name}{'must_count'} = 0;
    my ($fld, $must) = ('','');
    for my $val (split(/\s.\s/, $field->{'src'}{'calculated'}))
    {
     ($fld, $must) = split(/:/, $val);
      push(@{$calculated_sources->{$field_name}{'fields'}}, $fld);
      $calculated_sources->{$field_name}{$fld} = 0;
      if($must eq 'y')
      {
        $calculated_sources->{$field_name}{$fld} = 1;
        $calculated_sources->{$field_name}{'must_count'} ++;
      }
    }
    #print Dumper $calculated_sources->{$field_name};
  }
  return $calculated_sources->{$field_name};
}
# --- graph fields ----
sub graph_fields
{
  my $graph_name = $_[0];
  my $graph_fields = {};
  my ($field_name, $draw, $colour) = ('','','');
  my $graph = $graphs_source->{$graph_name};
  for my $field (@{$graph->{'fields'}})
  {
    ($field_name, $draw, $colour) = split(/:/, $field);
    push(@{$graph_fields->{'fields'}}, $field_name);
    $graph_fields->{$field_name}{'draw'}   = $draw;
    $graph_fields->{$field_name}{'colour'} = $colour if $colour ne '-';
  }
  return $graph_fields;
}
# ---- exists field or not --------------
my $fields_exists = {};
sub field_exists
{
  my $field_name = $_[0];
  unless(exists($fields_exists->{$field_name}))
  {
    my $field = $fields_source->{$field_name};
    if (exists($field->{'src'}{'target'}))
    {
      $fields_exists->{$field_name} = exists($stats->{$field->{'src'}{'target'}}{$field->{'src'}{'field'}});
    }
    elsif (exists($field->{'src'}{'calculated'}))
    {
      my $sources = calculated_sources($field_name);
      my $must_count = 0;
      for my $fld (@{$sources->{'fields'}})
      {
        $must_count++ if ($sources->{$fld} && field_exists($fld));
      }
      $fields_exists->{$field_name} = ($must_count == $sources->{'must_count'});
    }
  }
  return $fields_exists->{$field_name};
}
# ---------------------------------------- config -------------------------------------
sub print_config
{
  my $gr = {};
  load_limits();
  for my $graph_name (keys(%{$graphs_source}))
  {
    my $graph = $graphs_source->{$graph_name};
    my $order = '';
    my $graph_fields = graph_fields($graph_name);
    for my $field_name (@{$graph_fields->{'fields'}})
    {
      my $field = $fields_source->{$field_name};
      if (field_exists($field_name))
      {
        for my $option (keys(%{$field->{'munin'}}))
        {
          $gr->{$graph_name}{'fields'}{$field_name}{$option} = $field->{'munin'}{$option};
        }
        $gr->{$graph_name}{'fields'}{$field_name}{'draw'} = $graph_fields->{$field_name}{'draw'};
        $gr->{$graph_name}{'fields'}{$field_name}{'colour'} = $graph_fields->{$field_name}{'colour'} if exists($graph_fields->{$field_name}{'colour'});
        $order .= $field_name.' ';
      }
    }
    if(exists($gr->{$graph_name}{'fields'}))
    {
      $gr->{$graph_name}{'munin'} = $graph->{'munin'};
      $gr->{$graph_name}{'munin'}{'order'} = $order;
    }
  }
  #print Dumper $gr;
  for my $graph (sort keys %{$gr})
  {
    next unless (graph_is_enabled($graph));
    printf ("multigraph %s\n", $graph);
    for my $option (sort keys %{$gr->{$graph}{'munin'}})
    {
      printf ("graph_%s %s\n", $option, $gr->{$graph}{'munin'}{$option});
    }
    for my $field (sort keys %{$gr->{$graph}{'fields'}})
    {
      for my $type (sort keys %{$gr->{$graph}{'fields'}{$field}})
      {
        printf ("%s.%s %s\n", $field, $type, $gr->{$graph}{'fields'}{$field}{$type});
      }
    }
    print "\n";
  }
}

# -------------------------------------- values --------------------------------
my $values = {};
sub field_value
{
  my $field_name = $_[0];
  unless(exists($values->{$field_name}))
  {
    my $field = $fields_source->{$field_name};
    if (exists($field->{'src'}{'target'}))
    {
      $values->{$field_name} = $stats->{$field->{'src'}{'target'}}{$field->{'src'}{'field'}};
    }
    elsif (exists($field->{'src'}{'calculated'}))
    {
      my $sources = calculated_sources($field_name);
      my $formula = $field->{'src'}{'calculated'};
      for my $form_field (@{$sources->{'fields'}})
      {
        my $val = field_exists($form_field) ? field_value($form_field) : 0;
        $formula =~ s/$form_field:./$val/;
      }
      #print $formula." !!\n";
      my $result = 0;
      eval '$result = '.$formula.';';
      $values->{$field_name} = $result;
    }
  }
  return $values->{$field_name};
}

sub print_values
{
  my $gr = {};
  for my $graph_name (keys(%{$graphs_source}))
  {
    my $graph = $graphs_source->{$graph_name};
    my $graph_fields = graph_fields($graph_name);
    for my $field_name (@{$graph_fields->{'fields'}})
    {
      my $field = $fields_source->{$field_name};
      $gr->{$graph_name}{$field_name} = field_value($field_name) if field_exists($field_name);
    }
  }

  for my $graph_name (sort (keys %{$gr}))
  {
    next unless (graph_is_enabled($graph_name));
    printf ("multigraph %s\n", $graph_name);
    for my $field_name (sort keys %{$gr->{$graph_name}})
    {
      printf("%s.value %s\n", $field_name, $gr->{$graph_name}{$field_name});
    }
    print "\n";
  }

}