[Python-modules-commits] [python-protobix] 01/01: New upstream version 1.0.0
Jean Baptiste Favre
jbfavre-guest at moszumanska.debian.org
Tue Nov 22 13:14:23 UTC 2016
This is an automated email from the git hooks/post-receive script.
jbfavre-guest pushed a commit to annotated tag upstream/1.0.0
in repository python-protobix.
commit 6c0ebb2eaf5437c06135d0963fb987a9873c167c
Author: Jean Baptiste Favre <debian at jbfavre.org>
Date: Tue Nov 22 13:36:35 2016 +0100
New upstream version 1.0.0
---
.gitignore | 5 +
.travis.yml | 4 +-
README.md | 311 +++++++----
conftest.py | 18 -
protobix/__init__.py | 5 +
protobix/datacontainer.py | 372 +++++++------
protobix/sampleprobe.py | 419 ++++++++++----
protobix/senderprotocol.py | 542 ++++++++++--------
protobix/zabbixagentconfig.py | 335 +++++++++++
pytest.ini | 2 +
requirements-test.txt | 3 +
requirements.txt | 3 -
setup.py | 33 +-
tests/ZabbixServerTrapper.py | 240 --------
tests/__init__.py | 20 +
tests/docker-tests.sh | 47 ++
tests/test_01Params.py | 190 -------
tests/test_01Params_Deprecated.py | 74 ---
tests/test_02Items.py | 152 -----
tests/test_02Items_Deprecated.py | 116 ----
tests/test_03LLD.py | 159 ------
tests/test_03LLD_Deprecated.py | 75 ---
tests/test_04ReadZabbixFile.py | 100 ----
tests/test_datacontainer.py | 269 +++++++++
tests/test_memory_leak.py | 127 +++++
tests/test_sampleprobe.py | 615 +++++++++++++++++++++
tests/test_senderprotocol.py | 324 +++++++++++
tests/test_zabbixagentconfig.py | 576 +++++++++++++++++++
tests/tls_ca/protobix-ca.cert.pem | 29 +
tests/tls_ca/protobix-client.cert.pem | 32 ++
tests/tls_ca/protobix-client.key.pem | 52 ++
.../tls_ca/protobix-client.not-yet-valid.cert.pem | 33 ++
tests/tls_ca/protobix-client.not-yet-valid.key.pem | 52 ++
tests/tls_ca/protobix-client.revoked.cert.pem | 32 ++
tests/tls_ca/protobix-client.revoked.key.pem | 52 ++
tests/tls_ca/protobix-zabbix-server.cert.pem | 32 ++
tests/tls_ca/protobix-zabbix-server.key.pem | 52 ++
tests/tls_ca/protobix.crl | 16 +
tests/tls_ca/rogue-protobix-ca.cert.pem | 36 ++
tests/tls_ca/rogue-protobix-client.cert.pem | 36 ++
tests/tls_ca/rogue-protobix-client.key.pem | 52 ++
.../rogue-protobix-client.not-yet-valid.cert.pem | 33 ++
.../rogue-protobix-client.not-yet-valid.key.pem | 52 ++
.../tls_ca/rogue-protobix-client.revoked.cert.pem | 32 ++
tests/tls_ca/rogue-protobix-client.revoked.key.pem | 52 ++
tests/tls_ca/rogue-protobix.crl | 20 +
tests/tls_ca/rogue-zabbix-server.cert.pem | 32 ++
tests/tls_ca/rogue-zabbix-server.key.pem | 52 ++
tests/zabbix/zabbix_server.conf | 31 ++
tests/zabbix/zabbix_server_mysql.sql | 49 ++
tests/zabbix/zabbix_server_sqlite3.sql | 3 +
tests/zabbix_agentd.conf | 240 --------
52 files changed, 4230 insertions(+), 2008 deletions(-)
diff --git a/.gitignore b/.gitignore
index eff9bc7..2ff4086 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,6 +1,11 @@
.coverage
+.cache
*.pyc
+*.swp
/__pycache__/
+/.eggs/
MANIFEST
dist/
*.egg-info
+*.xdb
+*.db
diff --git a/.travis.yml b/.travis.yml
index b7f42ff..cd8753d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -3,7 +3,7 @@ python:
- "2.7"
- "3.4"
- "3.5"
-# command to install dependencies
-install: pip install -r requirements.txt
+# comand to install requirements
+install: pip install -r requirements.txt -r requirements-test.txt
# command to run tests
script: python setup.py test
diff --git a/README.md b/README.md
index eafd269..795c973 100644
--- a/README.md
+++ b/README.md
@@ -1,25 +1,48 @@
# python-protobix
-dev Branch: [](https://travis-ci.org/jbfavre/python-protobix)
-upstream Branch: [](https://travis-ci.org/jbfavre/python-protobix)
+* `dev` Branch: [](https://travis-ci.org/jbfavre/python-protobix)
+* `upstream` Branch (default): [](https://travis-ci.org/jbfavre/python-protobix)
-Very simple python module implementing Zabbix Sender protocol.
-It allows one to build list of items and send them as trapper.
-It currently supports `items` as well as [`Low Level Discovery`](https://www.zabbix.com/documentation/2.4/manual/discovery/low_level_discovery).
+`python-protobix` is a very simple python module which implements [Zabbix Sender protocol 2.0](https://www.zabbix.org/wiki/Docs/protocols/zabbix_sender/2.0).
+It allows to build a list of Zabbix items and send them as `trappers`.
-## Test
+Currently `python-protobix` supports "classics" `items` as well as [`Low Level Discovery`](https://www.zabbix.com/documentation/2.4/manual/discovery/low_level_discovery) ones.
+
+Please note that `python-protobix` is developped and tested on Debian GNU/Linux only.
+I can't enforce compatibility with other distributions, though it should work on any distribution providing Python 2.7 or Python 3.x.
-First, launch provided Zabbix Trapper Server:
+Any feedback on this is, of course, welcomed.
- python tests/ZabbixServerTrapper.py
+## Test
-Then, launch test suite:
+To install all required dependencies and launch test suite
python setup.py test
-## Install
+By default, all tests named like `*need_backend*` are disabled, since they need a working Zabbix Server.
+
+If you want to run theses tests as well, you will need:
+* a working Zabbix Server 3.x configuration file like the one in `tests/zabbix/zabbix_server.conf`
+* SQL statements in `tests/zabbix/zabbix_server_mysql.sql` with all informations to create testing hosts & items
+
+You can then start Zabbix Server with `zabbix_server -c tests/zabbix/zabbix_server.conf -f` and launch test suite with
+
+ py.test --cov protobix --cov-report term-missing
+
+### Using a docker container
+
+You can also use docker to run test suite on any Linux distribution of your choice.
+You can use provided script `docker-tests.sh` as entrypoint example:
+
+ docker run --volume=$(pwd):/home/python-protobix --entrypoint=/home/python-protobix/tests/docker-tests.sh -ti debian:jessie
-With `pip`:
+Currently, entrypoint `docker-tests.sh` only supports Debian GNU/Linux.
+
+__Please note that this docker entrypoint does not provide a way to execute test that need a backend__.
+
+## Installation
+
+With `pip` (stable version):
pip install protobix
@@ -27,20 +50,109 @@ With `pip` (test version):
pip install -i https://testpypi.python.org/simple/ protobix
-Build `Debian` package:
+Python is available as Debian package for Debian GNU/Linux sid and testing.
+
+## Usage
- apt-get install python-stdeb python-setuptools
+Once module is installed, you can use it either extending `protobix.SampleProbe` or directly using `protobix.Datacontainer`.
- cd module
- python setup.py --command-packages=stdeb.command bdist_deb
- apt-get install python-simplejson
- dpkg -i deb_dist/python-zabbix_0.0.1-1_all.deb
+### Extend `protobix.SampleProbe`
-## Usage
+`python-protobix` provides a convenient sample probe you can extend to fit your own needs.
+
+Using `protobix.SampleProbe` allows you to concentrate on getting metrics or Low Level Discovery items without taking care of anything related to `protobix` itself.
+This is the recommanded way of using `python-protobix`.
+
+`protobix.SampleProbe` provides a `run` method which take care of everything related to `protobix`.
-Once module is installed, you can use it as follow
+Some probes are available from my Github repository [`python-zabbix`](https://github.com/jbfavre/python-zabbix)
-## Send items as trappers
+```python
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+''' Copyright (c) 2013 Jean Baptiste Favre.
+ Sample Python class which extends protobix.SampleProbe
+'''
+import protobix
+import argparse
+import socket
+import sys
+
+class ExampleProbe(protobix.SampleProbe):
+
+ __version__ = '1.0.0'
+ # discovery_key is *not* the one declared in Zabbix Agent configuration
+ # it's the one declared in Zabbix template's "Discovery rules"
+ discovery_key = "example.probe.llddiscovery"
+
+ def _parse_probe_args(self, parser):
+ # Parse the script arguments
+ # parser is an instance of argparse.parser created by SampleProbe._parse_args method
+ # you *must* return parser to SampleProbe so that your own options are taken into account
+ example_probe_options = parser.add_argument_group('ExampleProbe configuration')
+ example_probe_options.add_argument(
+ "-o", "--option", default="default_value",
+ help="WTF do this option"
+ )
+ return parser
+
+ def _init_probe(self):
+ # Whatever you need to initiliaze your probe
+ # Can be establishing a connection
+ # Or reading a configuration file
+ # If you have nothing special to do
+ # Just do not override this method
+ # Or use:
+ pass
+
+ def _get_discovery(self):
+ # Whatever you need to do to discover LLD items
+ # this method is mandatory
+ # If not declared, calling the probe ith --discovery option will resut in a NotimplementedError
+ # If you get discovery infos for only one node you should return data as follow
+ return { self.hostname: data }
+ # If you get discovery infos for many hosts, then you should build data dict by yourself
+ # and return result as follow
+ return data
+
+ def _get_metrics(self):
+ # Whatever you need to do to collect metrics
+ # this method is mandatory
+ # If not declared, calling the probe with --update-items option will resut in a NotimplementedError
+ # If you get metrics for only one node you should return data as follow
+ return { self.hostname: data }
+ # If you get metrics for many hosts, then you should build data dict by your self
+ # and return result as follow
+ return data
+
+if __name__ == '__main__':
+ ret = RedisServer().run()
+ print ret
+ sys.exit(ret)
+```
+
+Declare your newly created probe as `Zabbix Agent` user parameters:
+
+ UserParameter=example.probe.check,/usr/local/bin/example_probe.py --update-items
+ UserParameter=example.probe.discovery,/usr/local/bin/example_probe.py --discovery
+
+You're done.
+
+The `protobix.SampleProbe` exit code will be sent to Zabbix.
+You'll be able to setup triggers if needed.
+
+__Exit codes mapping__:
+* 0: everything went well
+* 1: probe failed at step 1 (probe initialization)
+* 2: probe failed at step 2 (probe data collection)
+* 3: probe failed at step 3 (add data to DataContainer)
+* 4: probe failed at step 4 (send data to Zabbix)
+
+### Use `protobix.Datacontainer`
+
+If you don't want or can't use `protobix.SampleProbe`, you can also directly use `protobix.Datacontainer`.
+
+__How to send items updates__
```python
#!/usr/bin/env python
@@ -48,47 +160,24 @@ Once module is installed, you can use it as follow
''' import module '''
import protobix
-''' create DataContainer, providing data_type, zabbix server and port '''
-zbx_container = protobix.DataContainer(data_type = "items",
- zbx_host = '127.0.0.1',
- zbx_port = 10051,
- debug = False,
- dryrun = False)
-''' set debug '''
-zbx_container.debug = True
-
-''' set dryrun for testing purpose. Won't send anything to Zabbix '''
-zbx_container.dryrun = True
-
-''' Add items one after the other '''
-hostname="myhost"
-item="my.zabbix.item"
-value=0
-zbx_container.add_item( hostname, item, value)
-
-''' or use bulk insert '''
-data = {
- "myhost1": {
- "my.zabbix.item1": 0,
- "my.zabbix.item2": "item string"
+DATA = {
+ "protobix.host1": {
+ "my.protobix.item.int": 0,
+ "my.protobix.item.string": "item string"
},
- "myhost2": {
- "my.zabbix.item1": 0,
- "my.zabbix.item2": "item string"
+ "protobix.host2": {
+ "my.protobix.item.int": 0,
+ "my.protobix.item.string": "item string"
}
}
-zbx_container.add(data)
-
-''' Send data to zabbix '''
-try:
- zbx_container.send()
-except SendException as e:
- print str(e)
-print "Everything is OK"
+zbx_datacontainer = protobix.DataContainer()
+zbx_datacontainer.data_type = 'items'
+zbx_datacontainer.add(DATA)
+zbx_datacontainer.send()
```
-## Send Low Level Discovery as trappers
+__How to send Low Level Discovery__
```python
#!/usr/bin/env python
@@ -96,50 +185,78 @@ print "Everything is OK"
''' import module '''
import protobix
-''' create DataContainer, providing data_type, zabbix server and port '''
-zbx_container = protobix.DataContainer(data_type = "lld",
- zbx_host = '127.0.0.1',
- zbx_port = 10051,
- debug = False,
- dryrun = False)
-''' set debug '''
-zbx_container.debug = True
-
-''' Add items one after the other '''
-hostname="myhost"
-item="my.zabbix.lld_item1"
-value=[
- { 'my.zabbix.ldd_key1': 0,
- 'my.zabbix.ldd_key2': 'lld string' },
- { 'my.zabbix.ldd_key3': 1,
- 'my.zabbix.ldd_key4': 'another lld string' }
-]
-zbx_container.add_item( hostname, item, value)
-
-''' or use bulk insert '''
-data = {
- 'myhost1': {
- 'my.zabbix.lld_item1': [
- { '{#ZBX_LLD_KEY11}': 0,
- '{#ZBX_LLD_KEY12}': 'lld string' },
- { '{#ZBX_LLD_KEY11}': 1,
- '{#ZBX_LLD_KEY12}': 'another lld string' }
+DATA = {
+ 'protobix.host1': {
+ 'my.protobix.lld_item1': [
+ { '{#PBX_LLD_KEY11}': 0,
+ '{#PBX_LLD_KEY12}': 'lld string' },
+ { '{#PBX_LLD_KEY11}': 1,
+ '{#PBX_LLD_KEY12}': 'another lld string' }
+ ],
+ 'my.protobix.lld_item2': [
+ { '{#PBX_LLD_KEY21}': 10,
+ '{#PBX_LLD_KEY21}': 'yet an lld string' },
+ { '{#PBX_LLD_KEY21}': 2,
+ '{#PBX_LLD_KEY21}': 'yet another lld string' }
]
- 'myhost2':
- 'my.zabbix.lld_item2': [
- { '{#ZBX_LLD_KEY21}': 10,
- '{#ZBX_LLD_KEY21}': 'yet an lld string' },
- { '{#ZBX_LLD_KEY21}': 2,
- '{#ZBX_LLD_KEY21}': 'yet another lld string' }
+ },
+ 'protobix.host2': {
+ 'my.protobix.lld_item1': [
+ { '{#PBX_LLD_KEY11}': 0,
+ '{#PBX_LLD_KEY12}': 'lld string' },
+ { '{#PBX_LLD_KEY11}': 1,
+ '{#PBX_LLD_KEY12}': 'another lld string' }
+ ],
+ 'my.protobix.lld_item2': [
+ { '{#PBX_LLD_KEY21}': 10,
+ '{#PBX_LLD_KEY21}': 'yet an lld string' },
+ { '{#PBX_LLD_KEY21}': 2,
+ '{#PBX_LLD_KEY21}': 'yet another lld string' }
]
+ }
}
-zbx_container.add(data)
-''' Send data to zabbix '''
-try:
- zbx_container.send()
-except SendException as e:
- print str(e)
-
-print "Everything is OK"
+zbx_datacontainer = protobix.DataContainer()
+zbx_datacontainer.data_type = 'items'
+zbx_datacontainer.add(DATA)
+zbx_datacontainer.send()
```
+
+## Advanced configuration
+
+`python-protobix` behaviour can be altered in many ways using options.
+All configuration options are stored in a `protobix.ZabbixAgentConfig` instance.
+
+__Protobix specific configuration options__
+
+| Option name | Default value | ZabbixAgentConfig property | Command-line option (SampleProbe) |
+|--------------|---------------|----------------------------|-----------------------------------|
+| `data_type` | `None` | `data_type` | `--update-items` or `--discovery` |
+| `dryrun` | `False` | `dryrun` | `-d` or `--dryrun` |
+
+__Zabbix Agent configuration options__
+
+| Option name | Default value | ZabbixAgentConfig property | Command-line option (SampleProbe) |
+|------------------------|--------------------------|----------------------------|-----------------------------------|
+| `ServerActive` | `127.0.0.1` | `server_active` | `-z` or `--zabbix-server` |
+| `ServerPort` | `10051` | `server_port` | `-p` or `--port` |
+| `LogType` | `file` | `log_type` | none |
+| `LogFile` | `/tmp/zabbix_agentd.log` | `log_file` | none |
+| `DebugLevel` | `3` | `debug_level` | `-v` (from none to `-vvvvv`) |
+| `Timeout` | `3` | `timeout` | none |
+| `Hostname` | `socket.getfqdn()` | `hostname` | none |
+| `TLSConnect` | `unencrypted` | `tls_connect` | `--tls-connect` |
+| `TLSCAFile` | `None` | `tls_ca_file` | `--tls-ca-file` |
+| `TLSCertFile` | `None` | `tls_cert_file` | `--tls-cert-file` |
+| `TLSCRLFile` | `None` | `tls_crl_file` | `--tls-crl-file` |
+| `TLSKeyFile` | `None` | `tls_key_file` | `--tls-key-file` |
+| `TLSServerCertIssuer` | `None` | `tls_server_cert_issuer` | `--tls-server-cert-issuer` |
+| `TLSServerCertSubject` | `None` | `tls_server_cert_subject` | `--tls-server-cert-subject` |
+
+## How to contribute
+
+You can contribute to `protobix`:
+* fork this repository
+* write tests and documentation (tests __must__ pass for both Python 2.7 & 3.x)
+* implement the feature you need
+* open a pull request against __`upstream`__ branch
diff --git a/conftest.py b/conftest.py
deleted file mode 100644
index 3a79b77..0000000
--- a/conftest.py
+++ /dev/null
@@ -1,18 +0,0 @@
-import pytest
-import os
-import subprocess
-import time
-
- at pytest.fixture(scope="session", autouse=True)
-def start_zbx_server (request):
- zbx_cmd = [
- 'python',
- 'tests/ZabbixServerTrapper.py'
- ]
- zbx_proc = subprocess.Popen(zbx_cmd,
- stdout=open(os.devnull),
- stderr=open(os.devnull),
- shell=False
- )
- request.addfinalizer(zbx_proc.kill)
- time.sleep(0.5)
diff --git a/protobix/__init__.py b/protobix/__init__.py
index 79ecf54..3afac30 100644
--- a/protobix/__init__.py
+++ b/protobix/__init__.py
@@ -1,3 +1,8 @@
+"""
+Protobix is a simple module which implement Zabbix Sender protocol
+It provides a sample probe you can extend to monitor any software with Zabbix
+"""
from .datacontainer import DataContainer
from .senderprotocol import SenderProtocol
from .sampleprobe import SampleProbe
+from .zabbixagentconfig import ZabbixAgentConfig
diff --git a/protobix/datacontainer.py b/protobix/datacontainer.py
index cd541b6..78780c1 100644
--- a/protobix/datacontainer.py
+++ b/protobix/datacontainer.py
@@ -1,199 +1,237 @@
-import time
-import configobj
import logging
-import warnings, functools
try: import simplejson as json
-except ImportError: import json
-from datetime import datetime
-
-import sys
-if sys.version_info < (3,):
- def deprecated(func):
- '''This is a decorator which can be used to mark functions
- as deprecated. It will result in a warning being emitted
- when the function is used.'''
-
- @functools.wraps(func)
- def new_func(*args, **kwargs):
- warnings.warn_explicit(
- "Call to deprecated function {}.".format(func.__name__),
- category=DeprecationWarning,
- filename=func.func_code.co_filename,
- lineno=func.func_code.co_firstlineno + 1
- )
- return func(*args, **kwargs)
- return new_func
-else:
- def deprecated(func):
- '''This is a decorator which can be used to mark functions
- as deprecated. It will result in a warning being emitted
- when the function is used.'''
-
- @functools.wraps(func)
- def new_func(*args, **kwargs):
- warnings.warn_explicit(
- "Call to deprecated function {}.".format(func.__name__),
- category=DeprecationWarning,
- filename=func.__code__.co_filename,
- lineno=func.__code__.co_firstlineno + 1
- )
- return func(*args, **kwargs)
- return new_func
+except ImportError: import json # pragma: no cover
+from .zabbixagentconfig import ZabbixAgentConfig
from .senderprotocol import SenderProtocol
+# For both 2.0 & >2.2 Zabbix version
+# ? 1.8: Processed 0 Failed 1 Total 1 Seconds spent 0.000057
+# 2.0: Processed 0 Failed 1 Total 1 Seconds spent 0.000057
+# 2.2: processed: 50; failed: 1000; total: 1050; seconds spent: 0.09957
+# 2.4: processed: 50; failed: 1000; total: 1050; seconds spent: 0.09957
+ZBX_DBG_SEND_RESULT = "Send result [%s-%s-%s] for key [%s] item [%s]. Server's response is %s"
+ZBX_TRAPPER_MAX_VALUE = 250
+
class DataContainer(SenderProtocol):
- REQUEST = "sender data"
- _data_type = None
_items_list = []
+ _result = []
+ _logger = None
+ _config = None
+ socket = None
- def __init__(self, data_type = None,
- zbx_file = '/etc/zabbix/zabbix_agentd.conf',
- zbx_host = None,
- zbx_port = None,
- log_level = None,
- log_output = None,
- dryrun = False,
- logger = None):
- super(DataContainer,self).__init__()
-
+ def __init__(self,
+ config=None,
+ logger=None):
# Loads config from zabbix_agentd file
# If no file, it uses the default _config as configuration
- self._load_zabbix_config(zbx_file)
- # Override default values with the ones provided
- if log_level:
- self._config['log_level'] = log_level
- if log_output != None:
- self._config['log_output'] = log_output
- self._config['dryrun'] = dryrun
- if zbx_host:
- self._config['server'] = zbx_host
- if zbx_port:
- self._config['port'] = zbx_port
- if data_type:
- self._config['data_type'] = data_type
- self._logger = logger
-
- @property
- def data_type(self):
- return self._config['data_type']
-
- @data_type.setter
- def data_type(self, value):
- if self._logger:
- self._logger.debug("Setting value %s as data_type" % value)
- if value in ['lld', 'items']:
- self._config['data_type'] = value
- # Clean _items_list & _result when changing _data_type
- # Incompatible format
- self._items_list = []
- self._result = []
- else:
- if self._logger:
- self._logger.error('data_type requires either "items" or "lld"')
- raise ValueError('data_type requires either "items" or "lld"')
-
- @property
- def log_level(self):
- return int(self._config['log_level'])
+ self._config = config
+ if config is None:
+ self._config = ZabbixAgentConfig()
+ if logger:
+ self.logger = logger
+ self._items_list = []
- @log_level.setter
- def log_level(self, value):
- if isinstance(value, int) and value >= 0 and value < 5:
- self._config['log_level'] = value
+ def add_item(self, host, key, value, clock=None):
+ """
+ Add a single item into DataContainer
+
+ :host: hostname to which item will be linked to
+ :key: item key as defined in Zabbix
+ :value: item value
+ :clock: timestemp as integer. If not provided self.clock()) will be used
+ """
+ if clock is None:
+ clock = self.clock
+ if self._config.data_type == "items":
+ item = {"host": host, "key": key,
+ "value": value, "clock": clock}
+ elif self._config.data_type == "lld":
+ item = {"host": host, "key": key, "clock": clock,
+ "value": json.dumps({"data": value})}
else:
- if self._logger:
- self._logger.error('log_level parameter must be less than 5')
- raise ValueError('log_level parameter must be less than 5')
+ if self.logger: # pragma: no cover
+ self.logger.error("Setup data_type before adding data")
+ raise ValueError('Setup data_type before adding data')
+ self._items_list.append(item)
- @property
- def dryrun(self):
- return self._config['dryrun']
+ def add(self, data):
+ """
+ Add a list of item into the container
- @dryrun.setter
- def dryrun(self, value):
- if value in [True, False]:
- self._config['dryrun'] = value
+ :data: dict of items & value per hostname
+ """
+ for host in data:
+ for key in data[host]:
+ if not data[host][key] == []:
+ self.add_item(host, key, data[host][key])
+
+ def send(self):
+ """
+ Entrypoint to send data to Zabbix
+ If debug is enabled, items are sent one by one
+ If debug isn't enable, we send items in bulk
+ Returns a list of results (1 if no debug, as many as items in other case)
+ """
+ if self.logger: # pragma: no cover
+ self.logger.info("Starting to send %d items" % len(self._items_list))
+ try:
+ # Zabbix trapper send a maximum of 250 items in bulk
+ # We have to respect that, in case of enforcement on zabbix server side
+ # Special case if debug is enabled: we need to send items one by one
+ max_value = ZBX_TRAPPER_MAX_VALUE
+ if self.debug_level >= 4:
+ max_value = 1
+ if self.logger: # pragma: no cover
+ self.logger.debug("Bulk limit is %d items" % max_value)
+ else:
+ if self.logger: # pragma: no cover
+ self.logger.info("Bulk limit is %d items" % max_value)
+ # Initialize offsets & counters
+ max_offset = len(self._items_list)
+ run = 0
+ start_offset = 0
+ stop_offset = min(start_offset + max_value, max_offset)
+ server_success = server_failure = processed = failed = total = time = 0
+ while start_offset < stop_offset:
+ run += 1
+ if self.logger: # pragma: no cover
+ self.logger.debug(
+ 'run %d: start_offset is %d, stop_offset is %d' %
+ (run, start_offset, stop_offset)
+ )
+
+ # Extract items to be send from global item's list'
+ _items_to_send = self.items_list[start_offset:stop_offset]
+
+ # Send extracted items
+ run_response, run_processed, run_failed, run_total, run_time = self._send_common(_items_to_send)
+
+ # Update counters
+ if run_response == 'success':
+ server_success += 1
+ elif run_response == 'failed':
+ server_failure += 1
+ processed += run_processed
+ failed += run_failed
+ total += run_total
+ time += run_time
+ if self.logger: # pragma: no cover
+ self.logger.info("%d items sent during run %d" % (run_total, run))
+ self.logger.debug(
+ 'run %d: processed is %d, failed is %d, total is %d' %
+ (run, run_processed, run_failed, run_total)
+ )
+
+ # Compute next run's offsets
+ start_offset = stop_offset
+ stop_offset = min(start_offset + max_value, max_offset)
+
+ # Reset socket, which is likely to be closed by server
+ self._socket_reset()
+ except:
+ self._reset()
+ self._socket_reset()
+ raise
+ if self.logger: # pragma: no cover
+ self.logger.info('All %d items have been sent in %d runs' % (total, run))
+ self.logger.debug(
+ 'Total run is %d; item processed: %d, failed: %d, total: %d, during %f seconds' %
+ (run, processed, failed, total, time)
+ )
+ # Everything has been sent.
+ # Reset DataContainer & return results_list
+ self._reset()
+ return server_success, server_failure, processed, failed, total, time
+
+ def _send_common(self, item):
+ """
+ Common part of sending operations
+ Calls SenderProtocol._send_to_zabbix
+ Returns result as provided by _handle_response
+
+ :item: either a list or a single item depending on debug_level
+ """
+ total = len(item)
+ processed = failed = time = 0
+ if self._config.dryrun is True:
+ total = len(item)
+ processed = failed = time = 0
+ response = 'dryrun'
else:
- if self._logger:
- self._logger.error('dryrun parameter requires boolean')
- raise ValueError('dryrun parameter requires boolean')
+ self._send_to_zabbix(item)
+ response, processed, failed, total, time = self._read_from_zabbix()
+
+ output_key = '(bulk)'
+ output_item = '(bulk)'
+ if self.debug_level >= 4:
+ output_key = item[0]['key']
+ output_item = item[0]['value']
+ if self.logger: # pragma: no cover
+ self.logger.info(
+ "" +
+ ZBX_DBG_SEND_RESULT % (
+ processed,
+ failed,
+ total,
+ output_key,
+ output_item,
+ response
+ )
+ )
+ return response, processed, failed, total, time
+
+ def _reset(self):
+ """
+ Reset main DataContainer properties
+ """
+ # Reset DataContainer to default values
+ # So that it can be reused
+ if self.logger: # pragma: no cover
+ self.logger.info("Reset DataContainer")
+ self._items_list = []
+ self._config.data_type = None
@property
def logger(self):
+ """
+ Returns logger instance
+ """
return self._logger
@logger.setter
def logger(self, value):
+ """
+ Set logger instance for the class
+ """
if isinstance(value, logging.Logger):
self._logger = value
else:
- if self._logger:
- self._logger.error('logger requires a logging instance')
+ if self._logger: # pragma: no cover
+ self._logger.error("logger requires a logging instance")
raise ValueError('logger requires a logging instance')
+ # ZabbixAgentConfig getter & setter
+ # Avoid using private property _config from outside
@property
- def log_output(self):
- return self._config['log_output']
-
- def _load_zabbix_config(self,config_file):
- # Load zabbix agent configuration as default values
- # Default values are set in self._config
- # - ServerActive (default: 127.0.0.1)
- # - LogFile (default: /tmp/zabbix_agentd.log)
- # - DebugLevel (default: 3, Allowed: 0-4)
- # 0 -> logging.NOTSET
- # 1 -> logging.CRITICAL
- # 2 -> logging.ERROR
- # 3 -> logging.WARNING
- # 4 -> logging.DEBUG
- # - Timeout (default: 3, Allowed: 1-30)
-
- # list_values=False argument bellow is needed because of UserParameter with spaces
- tmp_config = configobj.ConfigObj(config_file, list_values=False)
-
- if 'ServerActive' in tmp_config:
- # Because of list_values=False above,
- # we have to check ServerActive and make the split manually
- tmp_server = tmp_config['ServerActive'].split(',')[0] \
- if "," in tmp_config['ServerActive'] \
- else tmp_config['ServerActive']
- self._config['server'], self._config['port'] = tmp_server.split(':') \
- if ":" in tmp_server else (tmp_server, 10051)
- self._config['port'] = int(self._config['port'])
-
- if 'LogFile' in tmp_config:
- self._config['log_output'] = tmp_config['LogFile']
-
- if 'DebugLevel' in tmp_config:
- self._config['log_level'] = int(tmp_config['DebugLevel'])
-
- if 'Timeout' in tmp_config:
- self._config['timeout'] = int(tmp_config['Timeout'])
-
- def add_item(self, host, key, value, clock=None):
- if clock is None:
- clock = self.clock
- if self._config['data_type'] == "items":
- item = { "host": host, "key": key,
- "value": value, "clock": clock}
- elif self._config['data_type'] == "lld":
- item = { "host": host, "key": key, "clock": clock,
- "value": json.dumps({"data": value}) }
- else:
- if self._logger:
- self._logger.error('Setup data_type before adding data')
- raise ValueError('Setup data_type before adding data')
- self._items_list.append(item)
+ def dryrun(self):
+ """
+ Returns dryrun
+ """
+ return self._config.dryrun
- def add(self, data):
- for host in data:
- for key in data[host]:
- if not data[host][key] == []:
- self.add_item( host, key, data[host][key])
+ @dryrun.setter
+ def dryrun(self, value):
+ """
+ Set dryrun
+ """
+ self._config.dryrun = value
- @deprecated
- def set_type(self, value):
- self.data_type = value
+ @dryrun.setter
+ def data_type(self, value):
+ """
+ Set data_type
+ """
+ self._config.data_type = value
diff --git a/protobix/sampleprobe.py b/protobix/sampleprobe.py
old mode 100644
new mode 100755
index 535d873..8c45fa2
--- a/protobix/sampleprobe.py
+++ b/protobix/sampleprobe.py
@@ -1,178 +1,367 @@
-import logging
-import optparse
+import argparse
+from argparse import RawTextHelpFormatter
import socket
import sys
import traceback
+import logging
from logging import handlers
from .datacontainer import DataContainer
+from .zabbixagentconfig import ZabbixAgentConfig
class SampleProbe(object):
- __version__ = '0.0.9'
+ __version__ = '1.0.0'
# Mapping between zabbix-agent Debug option & logging level
LOG_LEVEL = [
logging.NOTSET,
logging.CRITICAL,
logging.ERROR,
logging.INFO,
- logging.DEBUG
+ logging.DEBUG,
+ logging.DEBUG,
]
logger = None
probe_config = None
+ hostname = None
+ options = None
- def _parse_args(self):
+ def _parse_args(self, args):
+ if self.logger:
+ self.logger.info(
+ "Read command line options"
+ )
# Parse the script arguments
- # Common part
- parser = optparse.OptionParser()
-
- parser.add_option('-c', '--config', default = None,
- help='Probe config file location. '
- 'Can be either absolute or relative path')
- parser.add_option('-d', '--dry', action='store_true', default = False,
- help='Performs CDH API calls but do not send '
- 'anything to the Zabbix server. Can be used '
- 'for both Update & Discovery mode')
- parser.add_option('-D', '--debug', action='store_true', default = False,
- help='Enable debug mode. This will prevent bulk send '
- 'operations and force sending items one after the '
- 'other, displaying result for each one')
-
- zabbix_options = optparse.OptionGroup(parser, 'Zabbix configuration')
- zabbix_options.add_option('-z', '--zabbix-server', default='127.0.0.1',
- help='The hostname of Zabbix server or '
- 'proxy, default is 127.0.0.1.')
- zabbix_options.add_option('-p', '--zabbix-port', default=10051,
- help='The port on which the Zabbix server or '
- 'proxy is running, default is 10051.')
- zabbix_options.add_option('--update-items', action='store_const',
- dest='mode', const='update_items',
- help='Get & send items to Zabbix. This is the default '
- 'behaviour even if option is not specified')
- zabbix_options.add_option('--discovery', action='store_const',
- dest='mode', const='discovery',
- help='If specified, will perform Zabbix Low Level '
- 'Discovery on Hadoop Cloudera Manager API. '
- 'Default is to get & send items')
- parser.add_option_group(zabbix_options)
- parser.set_defaults(mode='update_items')
+ parser = argparse.ArgumentParser(
+ usage='%(prog)s [options]',
+ formatter_class=RawTextHelpFormatter,
+ epilog='Protobix - copyright 2016 - Jean Baptiste Favre (www.jbfavre.org)'
+ )
+ # Probe operation mode
+ probe_mode = parser.add_argument_group('Probe commands')
+ probe_mode.add_argument(
+ '--update-items', action='store_true', dest='update',
+ help="Get & send items to Zabbix.\nThis is the default behaviour"
+ )
+ probe_mode.add_argument(
+ '--discovery', action='store_true',
+ help="If specified, will perform Zabbix Low Level Discovery."
+ )
+ # Common options
+ common = parser.add_argument_group('Common options')
+ common.add_argument(
+ '-d', '--dryrun', action='store_true',
+ help="Do not send anything to Zabbix. Usefull to debug with\n"
+ "--verbose option"
... 5832 lines suppressed ...
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/python-modules/packages/python-protobix.git
More information about the Python-modules-commits
mailing list