[med-svn] [Git][med-team/ctdopts][master] 13 commits: routine-update: New upstream version

Michael R. Crusoe gitlab at salsa.debian.org
Sat Apr 4 08:42:38 BST 2020



Michael R. Crusoe pushed to branch master at Debian Med / ctdopts


Commits:
6ef25749 by Michael R. Crusoe at 2020-03-29T12:37:29+02:00
routine-update: New upstream version

- - - - -
30fee58c by Michael R. Crusoe at 2020-03-29T12:37:30+02:00
New upstream version 1.3
- - - - -
ede64b4b by Michael R. Crusoe at 2020-03-29T12:37:30+02:00
Update upstream source from tag 'upstream/1.3'

Update to upstream version '1.3'
with Debian dir 57e7877581a2e0b2415639fad66d22e39da8f694
- - - - -
a15bfbd3 by Michael R. Crusoe at 2020-03-29T12:37:30+02:00
routine-update: Standards-Version: 4.5.0

- - - - -
594b3fe0 by Michael R. Crusoe at 2020-03-29T12:37:31+02:00
routine-update: debhelper-compat 12

- - - - -
1f5bdafb by Michael R. Crusoe at 2020-03-29T12:43:41+02:00
routine-update: DEB_BUILD_OPTIONS allow override_dh_auto_test

- - - - -
90dc61b9 by Michael R. Crusoe at 2020-03-29T12:43:42+02:00
routine-update: Add salsa-ci file

- - - - -
2cc1b842 by Michael R. Crusoe at 2020-03-29T12:43:42+02:00
routine-update: Rules-Requires-Root: no

- - - - -
80f841a0 by Michael R. Crusoe at 2020-03-29T12:43:51+02:00
Set upstream metadata fields: Bug-Database, Bug-Submit, Repository, Repository-Browse.

Fixes: lintian: upstream-metadata-file-is-missing
See-also: https://lintian.debian.org/tags/upstream-metadata-file-is-missing.html

- - - - -
c5be9c59 by Michael R. Crusoe at 2020-03-29T12:43:52+02:00
Use canonical URL in Vcs-Browser.

Fixes: lintian: vcs-field-not-canonical
See-also: https://lintian.debian.org/tags/vcs-field-not-canonical.html

- - - - -
a7c9c443 by Michael R. Crusoe at 2020-03-29T12:45:42+02:00
debian/clean: *.ctd

- - - - -
332879ae by Michael R. Crusoe at 2020-04-04T09:37:37+02:00
Adapt autopkgtest to upstream changes

- - - - -
4a39ca24 by Michael R. Crusoe at 2020-04-04T09:41:58+02:00
release 1.3-1 to unstable

- - - - -


13 changed files:

- + .travis.yml
- CTDopts/CTDopts.py
- debian/changelog
- debian/clean
- − debian/compat
- debian/control
- debian/rules
- + debian/salsa-ci.yml
- debian/tests/run-example
- + debian/upstream/metadata
- example.py
- + setup.cfg
- setup.py


Changes:

=====================================
.travis.yml
=====================================
@@ -0,0 +1,19 @@
+language: python
+python:
+  - "3.6"
+
+install:
+  - sudo apt-get update
+  - pip install pycodestyle
+#  - pip install lxml ruamel.yaml
+  - cd $TRAVIS_BUILD_DIR
+  - python setup.py install
+
+jobs:
+  include:
+    - stage: "pycodestyle"
+      name: "pycodestyle"
+      script: pycodestyle CTDopts
+    - stage: "run example"
+      name: "run example"
+      script: ls -la && python example.py


=====================================
CTDopts/CTDopts.py
=====================================
@@ -17,6 +17,8 @@ class _ASingleton(type):
     def __call__(cls, *args, **kwargs):
         if cls not in cls._instances:
             cls._instances[cls] = super(_ASingleton, cls).__call__(*args, **kwargs)
+        else:
+            cls._instances[cls].__init__(*args, **kwargs)
         return cls._instances[cls]
 
 
@@ -26,6 +28,9 @@ class _Null(object):
     """
     __metaclass__ = _ASingleton
 
+    def __str__(self):
+        return ""
+
 
 class _InFile(str):
     """Dummy class for input-file CTD type. I think most users would want to just get the file path
@@ -42,15 +47,26 @@ class _OutFile(str):
 
 
 # module globals for some common operations (python types to CTD-types back and forth)
-TYPE_TO_CTDTYPE = {int: 'int', float: 'float', str: 'string', bool: 'boolean',
+TYPE_TO_CTDTYPE = {int: 'int', float: 'double', str: 'string', bool: 'bool',
                    _InFile: 'input-file', _OutFile: 'output-file'}
 CTDTYPE_TO_TYPE = {'int': int, 'float': float, 'double': float, 'string': str, 'boolean': bool, 'bool': bool,
                    'input-file': _InFile, 'output-file': _OutFile, int: int, float: float, str: str,
                    bool: bool, _InFile: _InFile, _OutFile: _OutFile}
 PARAM_DEFAULTS = {'advanced': False, 'required': False, 'restrictions': None, 'description': None,
                   'supported_formats': None, 'tags': None, 'position': None}  # unused. TODO.
-# a boolean type caster to circumvent bool('false')==True when we cast CTD 'value' attributes to their correct type
-CAST_BOOLEAN = lambda x: bool(x) if not isinstance(x, str) else (x in ('true', 'True', '1'))
+
+
+def CAST_BOOLEAN(x):
+    """
+    a boolean type caster to circumvent bool('false')==True when we cast CTD
+    'value' attributes to their correct type
+    """
+    if not isinstance(x, str):
+        return bool(x)
+    else:
+        return x in ('true', 'True', '1')
+
+
 # instead of using None or _Null, we define non-present 'position' attribute values as -1
 NO_POSITION = -1
 
@@ -98,7 +114,7 @@ def flatten_dict(arg_dict, as_string=False):
     def flattener(subgroup, level):
         # recursive closure that accesses and modifies result dict and registers nested elements
         # as it encounters them
-        for key, value in subgroup.iteritems():
+        for key, value in subgroup.items():
             if isinstance(value, Mapping):  # collections.Mapping instead of dict for generality
                 flattener(value, level + [key])
             else:
@@ -106,7 +122,7 @@ def flatten_dict(arg_dict, as_string=False):
 
     flattener(arg_dict, [])
     if as_string:
-        return {':'.join(keylist): value for keylist, value in result.iteritems()}
+        return {':'.join(keylist): value for keylist, value in result.items()}
     else:
         return result
 
@@ -117,9 +133,9 @@ def override_args(*arg_dicts):
 
     combined_args = override_args(args_from_ctd, args_from_commandline)
     """
-    overridden_args = dict(chain(*(flatten_dict(d).iteritems() for d in arg_dicts)))
+    overridden_args = dict(chain(*(flatten_dict(d).items() for d in arg_dicts)))
     result = {}
-    for keylist, value in overridden_args.iteritems():
+    for keylist, value in overridden_args.items():
         set_nested_key(result, keylist, value)
     return result
 
@@ -211,16 +227,30 @@ class ModelError(Exception):
         super(ModelError, self).__init__()
 
 
+class ModelTypeError(ModelError):
+    """Exception if file of wrong type is provided
+    """
+    def __init__(self, message):
+        super(ModelTypeError, self).__init__()
+        self.message = message
+
+    def __str__(self):
+        return "An error occurred while parsing the CTD file: %s" % self.message
+
+    def __repr__(self):
+        return str(self)
+
+
 class ModelParsingError(ModelError):
     """Exception for errors related to CTD parsing
     """
     def __init__(self, message):
         super(ModelParsingError, self).__init__()
         self.message = message
-        
+
     def __str__(self):
         return "An error occurred while parsing the CTD file: %s" % self.message
-    
+
     def __repr__(self):
         return str(self)
 
@@ -298,7 +328,7 @@ class _FileFormat(_Restriction):
     def __init__(self, formats):
         super(_FileFormat, self).__init__()
         if isinstance(formats, str):  # to handle ['txt', 'csv', 'tsv'] and '*.txt,*.csv,*.tsv'
-            formats = map(lambda x: x.replace('*.', '').strip(), formats.split(','))
+            formats = [x.replace('*.', '').strip() for x in formats.split(',')]
         self.formats = formats
 
     def ctd_restriction_string(self):
@@ -336,7 +366,25 @@ class _Choices(_Restriction):
 
 class Parameter(object):
 
-    def __init__(self, name, parent, **kwargs):
+    def __init__(self, name=None, parent=None, node=None, **kwargs):
+        if node is None:
+            kwargs["name"] = name
+            self._init_from_kwargs(parent, **kwargs)
+        else:
+            self._init_from_node(parent, node)
+
+    def _init_from_node(self, parent, nd):
+        setup = _translate_ctd_to_param(dict(nd.attrib))
+        assert nd.tag in ["ITEM", "ITEMLIST"], "Tried to init Parameter from %s" % nd.tag
+        if nd.tag == 'ITEMLIST':
+            if len(nd) > 0:
+                setup['default'] = [listitem.attrib['value'] for listitem in nd]
+            else:
+                setup['default'] = []
+            setup['is_list'] = True
+        self._init_from_kwargs(parent, **setup)
+
+    def _init_from_kwargs(self, parent, **kwargs):
         """Required positional arguments: `name` string and `parent` ParameterGroup object
 
         Optional keyword arguments:
@@ -353,13 +401,13 @@ class Parameter(object):
             `short_name`: string for short name annotation
             `position`: index (1-based) of the position on which the parameter appears on the command-line
         """
-        self.name = name
+        assert "name" in kwargs, "Parameter initialisation without name"
+        self.name = kwargs["name"]
         self.parent = parent
-        self.short_name = kwargs.get('short_name', _Null)
-
+        self.short_name = kwargs.get('short_name', _Null())
         try:
             self.type = CTDTYPE_TO_TYPE[kwargs.get('type', str)]
-        except:
+        except KeyError:
             raise UnsupportedTypeError(kwargs.get('type'))
 
         self.tags = kwargs.get('tags', [])
@@ -371,34 +419,34 @@ class Parameter(object):
         self.advanced = CAST_BOOLEAN(kwargs.get('advanced', False))
         self.position = int(kwargs.get('position', str(NO_POSITION)))
 
-        default = kwargs.get('default', _Null)
+        default = kwargs.get('default', _Null())
 
         self._validate_numerical_defaults(default)
-                    
+
         # TODO 1_6_3: right now the CTD schema requires the 'value' attribute to be present for every parameter.
         # So every time we build a model from a CTD file, we find at least a default='' or default=[]
         # for every parameter. This should change soon, but for the time being, we have to get around this
         # and disregard such default attributes. The below two lines will be deleted after fixing 1_6_3.
         if default == '' or (self.is_list and default == []):
-            default = _Null
+            default = _Null()
 
         # enforce that default is the correct type if exists. Elementwise for lists
-        if default is _Null:
-            self.default = _Null
+        if type(default) is _Null:
+            self.default = _Null()
         elif default is None:
             self.default = None
         else:
             if self.is_list:
-                self.default = map(self.type, default)
+                self.default = list(map(self.type, default))
             else:
                 self.default = self.type(default)
+
         # same for choices. I'm starting to think it's really unpythonic and we should trust input. TODO
 
-        if self.type == bool:
+        if self.type is bool:
             assert self.is_list is False, "Boolean flag can't be a list type"
             self.required = False  # override whatever we found. Boolean flags can't be required...
             self.default = CAST_BOOLEAN(default)
-
         # Default value should exist IFF argument is not required.
         # TODO: if we can have optional list arguments they don't have to have a default? (empty list)
         # TODO: CTD Params 1.6.3 have a required value attrib. That's very wrong for parameters that are required.
@@ -423,14 +471,14 @@ class Parameter(object):
                 raise ModelParsingError("Provided range [%s, %s] is not of type %s" %
                                         (num_range[0], num_range[1], self.type))
         elif 'choices' in kwargs:
-            self.restrictions = _Choices(map(self.type, kwargs['choices']))
+            self.restrictions = _Choices(list(map(self.type, kwargs['choices'])))
         elif 'file_formats' in kwargs:
             self.restrictions = _FileFormat(kwargs['file_formats'])
 
     # perform some basic validation on the provided default values...
-    # an empty string IS NOT a float/int!        
+    # an empty string IS NOT a float/int!
     def _validate_numerical_defaults(self, default):
-        if default is not None and default is not _Null:
+        if default is not None and type(default) is not _Null:
             if self.type is int or self.type is float:
                 defaults_to_validate = []
                 errors_so_far = []
@@ -460,14 +508,21 @@ class Parameter(object):
         ie. the nesting lineage of the Parameter object. With `name_only` setting on, it only returns
         the names of said objects. For top level parameters, it's a list with a single element.
         """
-        lineage = []
-        i = self
-        while i.parent is not None:
-            # Exclude ParameterGroup here, since they do not have a short_name attribute (lzimmermann)
-            lineage.append(i.short_name if short_name and not isinstance(i, ParameterGroup) else i.name if name_only else i)
-            i = i.parent
-        lineage.reverse()
-        return lineage
+        if name_only:
+            n = self.name
+        elif short_name:
+            n = self.short_name
+        else:
+            n = self
+        if self.parent is None:
+            return [n]
+        else:
+            return self.parent.get_lineage(name_only, short_name) + [n]
+
+    def get_parameters(self, nodes=False):
+        """return an iterator over all parameters
+        """
+        yield self
 
     def __repr__(self):
         info = []
@@ -492,7 +547,6 @@ class Parameter(object):
                 value = self.default
         else:  # otherwise take the parameter default
             value = self.default
-
         # XML attributes to be created (depending on whether they are needed or not):
         # name, value, type, description, tags, restrictions, supported_formats
 
@@ -501,14 +555,16 @@ class Parameter(object):
         if not self.is_list:  # we'll deal with list parameters later, now only normal:
             # TODO: once Param_1_6_3.xsd gets fixed, we won't have to set an empty value='' attrib.
             # but right now value is a required attribute.
-            attribs['value'] = '' if value is _Null else str(value)
-            if self.type is bool:  # for booleans str(True) returns 'True' but the XS standard is lowercase
+            attribs['value'] = str(value)
+            if self.type is bool or type(value) is bool:  # for booleans str(True) returns 'True' but the XS standard is lowercase
                 attribs['value'] = 'true' if value else 'false'
         attribs['type'] = TYPE_TO_CTDTYPE[self.type]
         if self.description:
             attribs['description'] = self.description
         if self.tags:
             attribs['tags'] = ','.join(self.tags)
+        attribs['required'] = str(self.required).lower()
+        attribs['advanced'] = str(self.advanced).lower()
 
         # Choices and NumericRange restrictions go in the 'restrictions' attrib, FileFormat has
         # its own attribute 'supported_formats' for whatever historic reason.
@@ -519,9 +575,10 @@ class Parameter(object):
 
         if self.is_list:  # and now list parameters
             top = Element('ITEMLIST', attribs)
-            
             # (lzimmermann) I guess _Null has to be exluded here, too
-            if value is not None and value is not _Null:
+            if value is None or type(value) is _Null:
+                pass
+            elif type(value) is list:
                 for d in value:
                     SubElement(top, 'LISTITEM', {'value': str(d)})
             return top
@@ -530,8 +587,8 @@ class Parameter(object):
 
     def _cli_node(self, parent_name, prefix='--'):
         lineage = self.get_lineage(name_only=True)
-        top_node = Element('clielement', {"optionIdentifier": prefix+':'.join(lineage)})
-        SubElement(top_node, 'mapping', {"referenceName": parent_name+"."+self.name})
+        top_node = Element('clielement', {"optionIdentifier": prefix + ':'.join(lineage)})
+        SubElement(top_node, 'mapping', {"referenceName": parent_name + "." + self.name})
         return top_node
 
     def is_positional(self):
@@ -539,11 +596,23 @@ class Parameter(object):
 
 
 class ParameterGroup(object):
-    def __init__(self, name, parent, description=None):
+    def __init__(self, name=None, parent=None, node=None, description=None):
+        self.parameters = OrderedDict()
         self.name = name
         self.parent = parent
         self.description = description
-        self.parameters = OrderedDict()
+        if node is None:
+            return
+
+        validate_contains_keys(node.attrib, ['name'], 'NODE')
+        self.name = node.attrib['name']
+        if "description" in node.attrib:
+            self.description = node.attrib['description']
+        for c in node:
+            if c.tag == 'NODE':
+                self.parameters[c.attrib['name']] = ParameterGroup(parent=self, node=c)
+            elif c.tag in ["ITEMLIST", "ITEM"]:
+                self.parameters[c.attrib['name']] = Parameter(parent=self, node=c)
 
     def add(self, name, **kwargs):
         """Registers a parameter in a ParameterGroup. Required: `name` string.
@@ -568,12 +637,12 @@ class ParameterGroup(object):
         """Registers a child parameter group under a ParameterGroup. Required: `name` string. Optional: `description`
         """
         # TODO assertion if name already exists? It just overrides now, but I'm not sure if allowing this behavior is OK
-        self.parameters[name] = ParameterGroup(name, self, description)
+        self.parameters[name] = ParameterGroup(name, parent=self, description=description)
         return self.parameters[name]
 
     def _get_children(self):
         children = []
-        for child in self.parameters.itervalues():
+        for child in self.parameters.values():
             if isinstance(child, Parameter):
                 children.append(child)
             elif isinstance(child, ParameterGroup):
@@ -590,7 +659,7 @@ class ParameterGroup(object):
         # Of course this should never happen if the argument tree is built properly but it would be
         # nice to take care of it if a user happens to randomly define his arguments and groups.
         # So first we could sort self.parameters (Items first, Groups after them).
-        for arg in self.parameters.itervalues():
+        for arg in self.parameters.values():
             top.append(arg._xml_node(arg_dict))
         return top
 
@@ -600,17 +669,41 @@ class ParameterGroup(object):
         :param arg_dict: dafualt values for elements
         :return: list of clielements
         """
-        for arg in self.parameters.itervalues():
-            yield arg._cli_node(parent_name=parent_name+"."+self.name, prefix=prefix)
+        for arg in self.parameters.values():
+            yield arg._cli_node(parent_name=parent_name + "." + self.name, prefix=prefix)
 
     def __repr__(self):
         info = []
         info.append('PARAMETER GROUP %s (' % self.name)
-        for subparam in self.parameters.itervalues():
+        for subparam in self.parameters.values():
             info.append(subparam.__repr__())
         info.append(')')
         return '\n'.join(info)
 
+    def get_lineage(self, name_only=False, short_name=False):
+        """Returns a list of zero or more ParameterGroup objects plus this one object at the end,
+        ie. the nesting lineage of the ParameterGroup object. With `name_only` setting on, it only returns
+        the names of said objects. For top level parameters, it's a list with a single element.
+        """
+        if name_only:
+            n = self.name
+        elif short_name:
+            n = self.short_name
+        else:
+            n = self
+        if self.parent is None:
+            return [n]
+        else:
+            return self.parent.get_lineage(name_only, short_name) + [n]
+
+    def get_parameters(self, nodes=False):
+        """return an iterator over all parameters
+        """
+        if nodes:
+            yield self
+        for p in self.parameters.values():
+            yield from p.get_parameters(nodes)
+
 
 class Mapping(object):
     def __init__(self, reference_name=None):
@@ -628,6 +721,190 @@ class CLI(object):
         self.cli_elements = cli_elements
 
 
+class Parameters(ParameterGroup):
+    def __init__(self, name=None, version=None, from_file=None, from_node=None, **kwargs):
+        self.name = None
+        self.version = None
+        self.description = None
+        self.opt_attribs = dict()  # little helper to have similar access as to CTDModel;
+
+        if from_file is not None or from_node is not None:
+            if from_file is not None:
+                root = parse(from_file).getroot()
+            else:
+                root = from_node
+            if root.tag != 'PARAMETERS':
+                raise ModelTypeError("Invalid PARAMETERS file root is not <PARAMETERS>")
+            # tool_element.attrib['version'] == '1.6.2'  # check whether the schema matches the one CTDOpts uses?
+            params_container_node = root.find('NODE')
+
+            one = root.find('./NODE/NODE[@name="1"]')
+            version = root.find('./NODE/ITEM[@name="version"]')
+
+            if one is not None and version is not None:
+                super(Parameters, self).__init__(name=None, parent=None, node=one, description=None)
+            else:
+                super(Parameters, self).__init__(name=None, parent=None, node=params_container_node, description=None)
+            self.description = params_container_node.attrib.get("description", "")
+            self.name = params_container_node.attrib.get("name", "")
+            if version is not None:
+                self.version = version.attrib["value"]
+        else:
+            self.name = name
+            self.version = version
+            super(Parameters, self).__init__(name=name, parent=None, node=None, description=kwargs.get("description", ""))
+
+        self.opt_attribs['description'] = self.description
+
+    def _xml_node(self, arg_dict):
+        params = Element('PARAMETERS', {
+            'version': "1.7.0",
+            'xmlns:xsi': "http://www.w3.org/2001/XMLSchema-instance",
+            'xsi:noNamespaceSchemaLocation': "https://github.com/genericworkflownodes/CTDopts/raw/master/schemas/Param_1_7_0.xsd"
+        })
+        node = Element("NODE", {"name": self.name, 'description': self.description})
+        params.append(node)
+
+        if self.version is not None:
+            node.append(Element("ITEM", {"name": "version", 'value': self.version, 'type': "string", 'description': "Version of the tool that generated this parameters file.", "required": "false", "advanced": "true"}))
+        one_node = Element("NODE", {"name": "1", 'description': "Instance '1' section for '%s'" % self.name})
+        node.append(one_node)
+
+        for arg in self.parameters.values():
+            n = arg._xml_node(arg_dict)
+            one_node.append(n)
+
+        return params
+
+    def get_lineage(self, name_only=False, short_name=False):
+        """Returns a list of zero or more ParameterGroup objects plus this one object at the end,
+        ie. the nesting lineage of the ParameterGroup object. With `name_only` setting on, it only returns
+        the names of said objects. For top level parameters, it's a list with a single element.
+        """
+        return []
+
+    def get_parameters(self, nodes=False):
+        """return an iterator over all parameters
+        """
+        for p in self.parameters.values():
+            yield from p.get_parameters(nodes)
+
+    def parse_cl_args(self, cl_args=None, prefix='--', short_prefix="-", get_remaining=False, ignore_required=False):
+        """Parses command line arguments `cl_args` (either a string or a list like sys.argv[1:])
+        assuming that parameter names are prefixed by `prefix` (default '--').
+
+        Returns a nested dictionary with found arguments. Note that parameters have to be registered
+        in the model to be parsed and returned.
+
+        Remaining (unmatchable) command line arguments can be accessed if the method is called with
+        `get_remaining`. In this case, the method returns a tuple, whose first element is the
+        argument dictionary, the second a list of unmatchable command line options.
+        """
+
+        class StoreFirst(argparse._StoreAction):
+            """
+            OpenMS command line parser uses the value of the first
+            occurence of an argument. This action does the same
+            (contrary to the default behaviour of the store action)
+            see also https://github.com/OpenMS/OpenMS/issues/4545
+            """
+            def __init__(self, option_strings, dest, nargs=None, **kwargs):
+                self._seen_args = set()
+                super(StoreFirst, self).__init__(option_strings, dest, nargs, **kwargs)
+
+            def __call__(self, parser, namespace, values, option_strings=None):
+                if self.dest not in self._seen_args:
+                    self._seen_args.add(self.dest)
+                    setattr(namespace, self.dest, values)
+
+        cl_arg_list = cl_args.split() if isinstance(cl_args, str) else cl_args
+        # if no arguments are given print help
+        if not cl_arg_list:
+            cl_arg_list.append("-h")
+
+        cl_parser = argparse.ArgumentParser()
+        for param in self.get_parameters():
+            lineage = param.get_lineage(name_only=True)
+            short_lineage = param.get_lineage(name_only=True, short_name=True)
+            cli_param = prefix + ':'.join(lineage)
+            cli_short_param = short_prefix + ':'.join(short_lineage)
+            idx = -1
+            if cli_param in cl_arg_list:
+                idx = cl_arg_list.index(cli_param)
+            elif cli_short_param in cl_arg_list:
+                idx = cl_arg_list.index(cli_short_param)
+
+            cl_arg_kws = {}  # argument processing info passed to argparse in keyword arguments, we build them here
+            if idx >= 0 and idx + 1 < len(cl_arg_list) and cl_arg_list[idx + 1] in ['true', 'false']:
+                cl_arg_kws['type'] = str
+                cl_arg_kws['action'] = StoreFirst
+            elif param.type is bool or (param.type is str and type(param.restrictions) is _Choices and set(param.restrictions.choices) == set(["true", "false"])):  # boolean flags are not followed by a value, only their presence is required
+                cl_arg_kws['action'] = 'store_true'
+            else:
+                # we take every argument as string and cast them only later in validate_args() if
+                # explicitly asked for. This is because we don't want to deal with type exceptions
+                # at this stage, and prefer the multi-leveled strictness settings in validate_args()
+                cl_arg_kws['type'] = str
+                cl_arg_kws['action'] = StoreFirst
+
+            if param.is_list:
+                # or '+' rather? Should we allow empty lists here? If default is a proper list with elements
+                # that we want to clear, this would be the only way to do it so I'm inclined to use '*'
+                cl_arg_kws['nargs'] = '*'
+
+            if type(param.default) is not _Null():
+                cl_arg_kws['default'] = param.default
+
+            if param.required and not ignore_required:
+                cl_arg_kws['required'] = True
+
+            # hardcoded 'group:subgroup:param'
+            if all(type(a) is not _Null for a in short_lineage):
+                cl_parser.add_argument(cli_short_param, cli_param, **cl_arg_kws)
+            else:
+                cl_parser.add_argument(cli_param, **cl_arg_kws)
+
+        parsed_args, rest = cl_parser.parse_known_args(cl_arg_list)
+        res_args = {}  # OrderedDict()
+        for param_name, value in vars(parsed_args).items():
+            # None values are created by argparse if it didn't find the argument or default=None, we skip params
+            # that dont have a default value
+            if value is not None or value == self.parameters.parameters[param_name].default:
+                set_nested_key(res_args, param_name.split(':'), value)
+        return res_args if not get_remaining else (res_args, rest)
+
+    def generate_ctd_tree(self, arg_dict=None, *args):
+        """Generates an XML ElementTree from the parameters model and returns
+        the top <parameters> Element object, that can be output to a file
+        (Parameters.write_ctd() does everything needed if the user
+        doesn't need access to the actual element-tree).
+        Calling this function without any arguments generates the tool-describing CTD with default
+        values. For parameter-storing and logging optional arguments can be passed:
+
+        `arg_dict`: nested dictionary with values to be used instead of defaults.
+        other arguments are irnored
+        """
+        return self._xml_node(arg_dict)
+
+    def write_ctd(self, out_file, arg_dict=None, log=None, cli=False):
+        """Generates a CTD XML from the model and writes it to `out_file`, which is either a string
+        to a file path or a stream with a write() method.
+
+        Calling this function without any arguments besides `out_file` generates the tool-describing
+        CTD with default values. For parameter-storing and logging optional arguments can be passed:
+
+        `arg_dict`: nested dictionary with values to be used instead of defaults.
+        `log`: dictionary with the following optional keys:
+            'time_start' and 'time_finish': proper XML date strings (eg. datetime.datetime.now(pytz.utc).isoformat())
+            'status': exit status
+            'output': standard output or whatever output the user intends to log
+            'warning': warning logs
+            'error': standard error or whatever error log the user wants to store
+        `cli`: boolean whether or not cli elements should be generated (needed for GenericKNIMENode for example)
+        """
+        write_ctd(self, out_file, arg_dict, log, cli)
+
+
 class CTDModel(object):
     def __init__(self, name=None, version=None, from_file=None, **kwargs):
         """The parameter model of a tool.
@@ -646,14 +923,15 @@ class CTDModel(object):
             self.version = version
             # TODO: check whether optional attributes in kwargs are all allowed or just ignore the rest?
             self.opt_attribs = kwargs  # description, manual, docurl, category (+executable stuff).
-            self.parameters = ParameterGroup('1', None, 'Parameters of %s' % self.name)  # openMS legacy, top group named "1"
+            self.parameters = Parameters(name=self.name, version=version, **kwargs)
             self.cli = []
 
     def _load_from_file(self, filename):
         """Builds a CTDModel from a CTD XML file.
         """
         root = parse(filename).getroot()
-        assert root.tag == 'tool', "Invalid CTD file, root is not <tool>"  # TODO: own exception
+        if root.tag != 'tool':
+            raise ModelTypeError("Invalid CTD file, root is not <tool>")
 
         self.opt_attribs = {}
         self.cli = []
@@ -667,28 +945,15 @@ class CTDModel(object):
                 self.opt_attribs[tool_opt_attrib] = root.attrib[tool_opt_attrib]
 
         for tool_element in root:
+            # ignoring: cli, logs, relocators. cli and relocators might be useful later.
             if tool_element.tag in ['manual', 'description', 'executableName', 'executablePath']:
-                                    # ignoring: cli, logs, relocators. cli and relocators might be useful later.
                 self.opt_attribs[tool_element.tag] = tool_element.text
 
             if tool_element.tag == 'cli':
                 self._build_cli(tool_element.findall('clielement'))
 
             if tool_element.tag == 'PARAMETERS':
-                # tool_element.attrib['version'] == '1.6.2'  # check whether the schema matches the one CTDOpts uses?
-                params_container_node = tool_element.find('NODE')
-                # we have to check the case in which the parent node contains 
-                # item/itemlist elements AND node element children
-                params_container_node_contains_items = params_container_node.find('ITEM') is not None or params_container_node.find('ITEMLIST')                 
-                # assert params_container_node.attrib['name'] == self.name
-                # check params_container_node's first ITEM child's tool version information again? (OpenMS legacy?)
-                params = params_container_node.find('NODE')  # OpenMS legacy again, NODE with name="1" on top
-                # check for the case when we have PARAMETERS/NODE/ITEM
-                if params is None or params_container_node_contains_items:                    
-                    self.parameters = self._build_param_model(params_container_node, base=None)
-                else:
-                    # OpenMS legacy again, PARAMETERS/NODE/NODE/ITEM
-                    self.parameters = self._build_param_model(params, base=None)
+                self.parameters = Parameters(from_node=tool_element)
 
     def _build_cli(self, xml_cli_elements):
         for xml_cli_element in xml_cli_elements:
@@ -750,7 +1015,7 @@ class CTDModel(object):
     def get_defaults(self):
         """Returns a nested dictionary with all parameters of the model having default values.
         """
-        params_w_default = (p for p in self.list_parameters() if p.default is not _Null)
+        params_w_default = (p for p in self.list_parameters() if type(p.default) is not _Null)
         defaults = {}
         for param in params_w_default:
             set_nested_key(defaults, param.get_lineage(name_only=True), param.default)
@@ -775,7 +1040,7 @@ class CTDModel(object):
                 # boolean values are the only ones that don't get casted correctly with, say, bool('false')
                 typecast = param.type if param.type is not bool else CAST_BOOLEAN
                 try:
-                    validated_value = map(typecast, arg) if param.is_list else typecast(arg)
+                    validated_value = list(map(typecast, arg)) if param.is_list else typecast(arg)
                 except ValueError:  # type casting failed
                     validated_value = arg  # just keep it as a string (or list of strings)
                     if enforce_type:  # but raise a warning or exception depending on enforcement level
@@ -805,60 +1070,10 @@ class CTDModel(object):
                     set_nested_key(validated_args, lineage, param.default)
         return validated_args
 
-    def parse_cl_args(self, cl_args=None, prefix='--', short_prefix="-", get_remaining=False):
-        """Parses command line arguments `cl_args` (either a string or a list like sys.argv[1:])
-        assuming that parameter names are prefixed by `prefix` (default '--').
-
-        Returns a nested dictionary with found arguments. Note that parameters have to be registered
-        in the model to be parsed and returned.
-
-        Remaining (unmatchable) command line arguments can be accessed if the method is called with
-        `get_remaining`. In this case, the method returns a tuple, whose first element is the
-        argument dictionary, the second a list of unmatchable command line options.
-        """
-        cl_parser = argparse.ArgumentParser()
-        for param in self.list_parameters():
-            lineage = param.get_lineage(name_only=True)
-            short_lineage = param.get_lineage(name_only=True, short_name=True)
-            cl_arg_kws = {}  # argument processing info passed to argparse in keyword arguments, we build them here
-            if param.type is bool:  # boolean flags are not followed by a value, only their presence is required
-                cl_arg_kws['action'] = 'store_true'
-            else:
-                # we take every argument as string and cast them only later in validate_args() if
-                # explicitly asked for. This is because we don't want to deal with type exceptions
-                # at this stage, and prefer the multi-leveled strictness settings in validate_args()
-                cl_arg_kws['type'] = str
-
-            if param.is_list:
-                # or '+' rather? Should we allow empty lists here? If default is a proper list with elements
-                # that we want to clear, this would be the only way to do it so I'm inclined to use '*'
-                cl_arg_kws['nargs'] = '*'
-
-            if param.default is not _Null():
-                cl_arg_kws['default'] = param.default
-
-            if param.required:
-                cl_arg_kws['required'] = True
-
-            # hardcoded 'group:subgroup:param1'
-            if all(a is not _Null for a in short_lineage):
-                cl_parser.add_argument(short_prefix+':'.join(short_lineage), prefix + ':'.join(lineage), **cl_arg_kws)
-            else:
-                cl_parser.add_argument(prefix + ':'.join(lineage), **cl_arg_kws)
-
-
-        cl_arg_list = cl_args.split() if isinstance(cl_args, str) else cl_args
-        #if no arguments are given print help
-        if not cl_arg_list:
-            cl_arg_list.append("-h")
-        parsed_args, rest = cl_parser.parse_known_args(cl_arg_list)
-        res_args = {}  # OrderedDict()
-        for param_name, value in vars(parsed_args).iteritems():
-            # None values are created by argparse if it didn't find the argument or default=None, we skip params
-            # that dont have a default value
-            if value is not None or value == self.parameters.parameters[param_name].default:
-                set_nested_key(res_args, param_name.split(':'), value)
-        return res_args if not get_remaining else (res_args, rest)
+    def parse_cl_args(self, cl_args=None, prefix='--', short_prefix="-",
+                      get_remaining=False, ignore_required=False):
+        return self.parameters.parse_cl_args(cl_args, prefix, short_prefix,
+                                             get_remaining, ignore_required)
 
     def generate_ctd_tree(self, arg_dict=None, log=None, cli=False, prefix='--'):
         """Generates an XML ElementTree from the model and returns the top <tool> Element object,
@@ -913,27 +1128,8 @@ class CTDModel(object):
             if 'error' in log:
                 SubElement(log_node, 'executionError').text = log['error']
 
-        # XML.ETREE SYNTAX
-        params = SubElement(tool, 'PARAMETERS', {
-            'version': '1.6.2',
-            'xmlns:xsi': "http://www.w3.org/2001/XMLSchema-instance",
-            'xsi:noNamespaceSchemaLocation': "https://github.com/genericworkflownodes/CTDopts/raw/master/schemas/Param_1_6_2.xsd"
-        })
-
-        # This seems to be some OpenMS hack (defining name, description, version for the second time)
-        # but I'll stick to it for consistency
-        top_node = SubElement(params, 'NODE', name=self.name, description=self.opt_attribs.get('description', ''))
-
-        SubElement(top_node, 'ITEM',
-            name='version',
-            value=self.version,
-            type='string',
-            description='Version of the tool that generated this parameters file.',
-            tags='advanced')
-
         # all the above was boilerplate, now comes the actual parameter tree generation
-        args_top_node = self.parameters._xml_node(arg_dict)
-        top_node.append(args_top_node)
+        tool.append(self.parameters._xml_node(arg_dict))
 
         if cli:
             cli_node = SubElement(tool, "cli")
@@ -946,6 +1142,11 @@ class CTDModel(object):
         # xml.etree syntax (no pretty print available, so we use xml.dom.minidom stuff)
         return tool
 
+    def get_parameters(self, nodes=False):
+        """return an iterator over all parameters
+        """
+        yield from self.parameters.get_parameters(nodes)
+
     def write_ctd(self, out_file, arg_dict=None, log=None, cli=False):
         """Generates a CTD XML from the model and writes it to `out_file`, which is either a string
         to a file path or a stream with a write() method.
@@ -962,13 +1163,17 @@ class CTDModel(object):
             'error': standard error or whatever error log the user wants to store
         `cli`: boolean whether or not cli elements should be generated (needed for GenericKNIMENode for example)
         """
-        xml_content = parseString(tostring(self.generate_ctd_tree(arg_dict, log, cli), encoding="UTF-8")).toprettyxml()
+        write_ctd(self, out_file, arg_dict, log, cli)
 
-        if isinstance(out_file, str):  # if out_file is a string, we create and write the file
-            with open(out_file, 'w') as f:
-                f.write(xml_content)
-        else:  # otherwise we assume it's a writable stream and write into that.
-            out_file.write(xml_content)
+
+def write_ctd(model, out_file, arg_dict=None, log=None, cli=False):
+    xml_content = parseString(tostring(model.generate_ctd_tree(arg_dict, log, cli), encoding="UTF-8")).toprettyxml(indent="  ")
+
+    if isinstance(out_file, str):  # if out_file is a string, we create and write the file
+        with open(out_file, 'w') as f:
+            f.write(xml_content)
+    else:  # otherwise we assume it's a writable stream and write into that.
+        out_file.write(xml_content)
 
 
 def args_from_file(filename):
@@ -1011,7 +1216,7 @@ def args_from_file(filename):
 
 
 def parse_cl_directives(cl_args, write_tool_ctd='write_tool_ctd', write_param_ctd='write_param_ctd',
-                       input_ctd='input_ctd', prefix='--'):
+                        input_ctd='input_ctd', prefix='--'):
     '''Parses command line CTD processing directives. `write_tool_ctd`, `write_param_ctd` and `input_ctd`
     string are customizable, and will be parsed for in command line. `prefix` should be one or two dashes,
     default is '--'.
@@ -1021,6 +1226,14 @@ def parse_cl_directives(cl_args, write_tool_ctd='write_tool_ctd', write_param_ct
         'write_param_ctd': if flag set, either True or the filename provided in command line. Otherwise None.
         'input_ctd': filename if found, otherwise None
     '''
+    def transform(x):
+        if x is None:
+            return None
+        elif x == []:
+            return True
+        else:
+            return x[0]
+
     parser = argparse.ArgumentParser()
     parser.add_argument(prefix + write_tool_ctd, nargs='*')
     parser.add_argument(prefix + write_param_ctd, nargs='*')
@@ -1030,8 +1243,6 @@ def parse_cl_directives(cl_args, write_tool_ctd='write_tool_ctd', write_param_ct
     directives, rest = parser.parse_known_args(cl_arg_list)
     directives = vars(directives)
 
-    transform = lambda x: None if x is None else True if x == [] else x[0]
-
     parsed_directives = {}
     parsed_directives['write_tool_ctd'] = transform(directives[write_tool_ctd])
     parsed_directives['write_param_ctd'] = transform(directives[write_param_ctd])
@@ -1045,4 +1256,4 @@ def validate_contains_keys(dictionary, keys, element_tag):
     for key in keys:
         assert key in dictionary, "Missing required attribute '%s' in %s element. Present attributes: %s" % \
                                   (key, element_tag,
-                                   ', '.join(['{0}="{1}"'.format(k, v) for k, v in dictionary.iteritems()]))
+                                   ', '.join(['{0}="{1}"'.format(k, v) for k, v in dictionary.items()]))


=====================================
debian/changelog
=====================================
@@ -1,8 +1,23 @@
-ctdopts (1.2-4) UNRELEASED; urgency=medium
+ctdopts (1.3-1) unstable; urgency=medium
 
+  [ Jelmer Vernooij ]
   * Remove unnecessary X-Python{,3}-Version field in debian/control.
 
- -- Jelmer Vernooij <jelmer at debian.org>  Sat, 20 Oct 2018 14:03:18 +0000
+  [ Michael R. Crusoe ]
+  * New upstream version
+  * Standards-Version: 4.5.0 (routine-update)
+  * debhelper-compat 12 (routine-update)
+  * Respect DEB_BUILD_OPTIONS in override_dh_auto_test target (routine-
+    update)
+  * Add salsa-ci file (routine-update)
+  * Rules-Requires-Root: no (routine-update)
+  * Set upstream metadata fields: Bug-Database, Bug-Submit, Repository,
+    Repository-Browse.
+  * Use canonical URL in Vcs-Browser.
+  * debian/clean: *.ctd
+  * Adapt autopkgtest to upstream changes
+
+ -- Michael R. Crusoe <michael.crusoe at gmail.com>  Sat, 04 Apr 2020 09:41:42 +0200
 
 ctdopts (1.2-3) unstable; urgency=medium
 


=====================================
debian/clean
=====================================
@@ -1,2 +1,3 @@
 debian/exampleTool*
 debian/example.py
+*.ctd


=====================================
debian/compat deleted
=====================================
@@ -1 +0,0 @@
-11


=====================================
debian/control
=====================================
@@ -3,16 +3,17 @@ Maintainer: Debian Med Packaging Team <debian-med-packaging at lists.alioth.debian.
 Uploaders: Michael R. Crusoe <michael.crusoe at gmail.com>
 Section: python
 Priority: optional
-Build-Depends: debhelper (>= 11),
+Build-Depends: debhelper-compat (= 12),
                dh-python,
                python3,
                python3-setuptools,
                2to3,
                python3-tz
-Standards-Version: 4.1.3
-Vcs-Browser: https://salsa.debian.org/med-team/ctdopts.git
+Standards-Version: 4.5.0
+Vcs-Browser: https://salsa.debian.org/med-team/ctdopts
 Vcs-Git: https://salsa.debian.org/med-team/ctdopts.git
 Homepage: https://github.com/WorkflowConversion/CTDopts
+Rules-Requires-Root: no
 
 Package: python3-ctdopts
 Architecture: all


=====================================
debian/rules
=====================================
@@ -27,4 +27,6 @@ override_dh_auto_build:
 	2to3 --write --nobackups --no-diffs --output-dir=debian example.py
 
 override_dh_auto_test:
+ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
 	export PYBUILD_SYSTEM=custom; dh_auto_test
+endif


=====================================
debian/salsa-ci.yml
=====================================
@@ -0,0 +1,4 @@
+---
+include:
+  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
+  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/pipeline-jobs.yml


=====================================
debian/tests/run-example
=====================================
@@ -1,10 +1,7 @@
 #!/bin/sh -ex
-cd "$AUTOPKGTEST_TMP"
-gunzip -c /usr/share/doc/python3-ctdopts/examples/example.py.gz > example.py
-2to3 --write --nobackups --no-diffs example.py
 for py in $(py3versions -r 2>/dev/null)
 	do echo "Testing with $py:"
 	${py} -c "import CTDopts; print(CTDopts)"
-	${py} example.py
+	${py} /usr/share/doc/python3-ctdopts/examples/example.py
 done
 


=====================================
debian/upstream/metadata
=====================================
@@ -0,0 +1,4 @@
+Bug-Database: https://github.com/WorkflowConversion/CTDopts/issues
+Bug-Submit: https://github.com/WorkflowConversion/CTDopts/issues/new
+Repository: https://github.com/WorkflowConversion/CTDopts.git
+Repository-Browse: https://github.com/WorkflowConversion/CTDopts


=====================================
example.py
=====================================
@@ -1,7 +1,6 @@
 # or for easier access of certain commonly used module methods
 import datetime
 import pprint
-import pytz
 
 import CTDopts.CTDopts  # once you installed it, it's just CTDopts
 from CTDopts.CTDopts import CTDModel, args_from_file, parse_cl_directives, flatten_dict, override_args, ArgumentRestrictionError
@@ -25,7 +24,7 @@ model = CTDModel(
     category='testing',
     executableName='exampletool',
     executablePath='/path/to/exec/exampletool-1.0/exampletool'
-    )
+)
 
 # The parameters of the tool have to be registered the following way:
 model.add(
@@ -35,7 +34,7 @@ model.add(
     default=5,
     tags=['advanced', 'magic'],  # for certain workflow engines that make use of parameter tags
     description='A positive integer parameter'
-    )
+)
 
 model.add(
     'input_files',
@@ -44,7 +43,7 @@ model.add(
     is_list=True,  # for list parameters with an arbitrary number of values
     file_formats=['fastq', 'fastq.gz'],  # filename restrictions
     description='A list of filenames to feed this dummy tool with'
-    )
+)
 
 model.add(
     'this_that',
@@ -52,7 +51,7 @@ model.add(
     choices=['this', 'that'],  # controlled vocabulary
     default='this',
     description='A controlled vocabulary parameter. Allowed values `this` or `that`.'
-    )
+)
 
 # Certain tools may want to group parameters together. One can define them like this:
 subparams = model.add_group('subparams', 'Grouped settings')
@@ -63,7 +62,7 @@ subparams.add(
     type=float,
     default=5.5,
     description='Some floating point setting.'
-    )
+)
 
 subparams.add(
     'param_2',
@@ -72,7 +71,7 @@ subparams.add(
     tags=['advanced'],
     default=[0.0, 2.5, 5.0],
     description='A list of floating point settings'
-    )
+)
 
 subsubparams = subparams.add_group('subsubparam', 'A group of sub-subsettings')
 subsubparams.add(
@@ -80,137 +79,137 @@ subsubparams.add(
     type=int,
     default=2,
     description="A subsetting's subsetting"
-    )
+)
 
 # Now we have a CTDModel. To write the model to a CTD (xml) file:
-print 'Model being written to exampleTool.ctd...\n'
+print('Model being written to exampleTool.ctd...\n')
 model.write_ctd('exampleTool.ctd')
 
 # However, if we already have a CTD model for a tool, we can spare the pain of defining it like above, we can just
 # load it from a file directly. Like this:
-print 'Model loaded from exampleTool.ctd...\n'
+print('Model loaded from exampleTool.ctd...\n')
 model_2 = CTDModel(from_file='exampleTool.ctd')
 
 # We can list all the model's parameters. The below call will get a list of all Parameter objects registered in the model.
 # These objects store name, type, default, restriction, parent group etc. information we set above.
 params = model.list_parameters()
 
-print "For debugging purposes we can output a human readable representation of Parameter objects. Here's the first one:"
-print params[0]
+print("For debugging purposes we can output a human readable representation of Parameter objects. Here's the first one:")
+print(params[0])
 print
 
 # Let's print out the name attributes of these parameters.
-print 'The following parameters were registered in the model:'
-print [p.name for p in params]
+print('The following parameters were registered in the model:')
+print([p.name for p in params])
 print
 
 # In the above model, certain parameters were registered under parameter groups. We can access their 'lineage' and see
 # their nesting levels. Let's display nesting levels separated by colons:
-print 'The same parameters with subgroup information, if they were registered under parameter groups:'
-print [':'.join(p.get_lineage(name_only=True)) for p in params]
-print
+print('The same parameters with subgroup information, if they were registered under parameter groups:')
+print([':'.join(p.get_lineage(name_only=True)) for p in params])
+print()
 # (Parameter.get_lineage() returns a list of ParameterGroups down to the leaf Parameter. `name_only` setting returns
 # only the names of the objects, instead of the actual Parameter objects..
 
 # Some of the parameters had default values in the model. We can get those:
-print 'A dictionary of parameters with default values, returned by CTDModel.get_defaults():'
+print('A dictionary of parameters with default values, returned by CTDModel.get_defaults():')
 defaults = model_2.get_defaults()
 pretty_print(defaults)
-print
+print()
 
-print ('As you can see, parameter values are usually stored in nested dictionaries. If you want a flat dictionary, you can'
-    'get that using CTDopts.flatten_dict(). Flat keys can be either tuples of tree node (subgroup) names down to the parameter...')
+print('As you can see, parameter values are usually stored in nested dictionaries. If you want a flat dictionary, you can'
+      'get that using CTDopts.flatten_dict(). Flat keys can be either tuples of tree node (subgroup) names down to the parameter...')
 flat_defaults = flatten_dict(defaults)
 pretty_print(flat_defaults)
-print
+print()
 
-print '...or they can be strings where nesing levels are separated by colons:'
+print('...or they can be strings where nesing levels are separated by colons:')
 flat_defaults_colon = flatten_dict(defaults, as_string=True)
 pretty_print(flat_defaults_colon)
-print
+print()
 
-print ('We can create dictionaries of arguments on our own that we want to validate against the model.'
-'CTDopts can read them from argument-storing CTD files or from the command line, but we can just define them in a '
-'nested dictionary on our own as well. We start with defining them explicitly.')
+print('We can create dictionaries of arguments on our own that we want to validate against the model.'
+      'CTDopts can read them from argument-storing CTD files or from the command line, but we can just define them in a '
+      'nested dictionary on our own as well. We start with defining them explicitly.')
 new_values = {
-'positive_int': 111,
-'input_files': ['file1.fastq', 'file2.fastq', 'file3.fastq'],
-'subparams': {'param_1': '999.0'}
+    'positive_int': 111,
+    'input_files': ['file1.fastq', 'file2.fastq', 'file3.fastq'],
+    'subparams': {'param_1': '999.0'}
 }
 pretty_print(new_values)
-print
+print()
 
-print ("We can validate these arguments against the model, and get a dictionary with parameter types correctly casted "
-"and defaults set. Note that subparams:param_1 was casted from string to a floating point number because that's how it "
-"was defined in the model.")
+print("We can validate these arguments against the model, and get a dictionary with parameter types correctly casted "
+      "and defaults set. Note that subparams:param_1 was casted from string to a floating point number because that's how it "
+      "was defined in the model.")
 validated = model.validate_args(new_values)
 pretty_print(validated)
-print
+print()
 
-print ('We can write a CTD file containing these validated argument values. Just call CTDModel.write_ctd() with an extra '
-    'parameter: the nested argument dictionary containing the actual values.')
+print('We can write a CTD file containing these validated argument values. Just call CTDModel.write_ctd() with an extra '
+      'parameter: the nested argument dictionary containing the actual values.')
 model.write_ctd('exampleTool_preset_params.ctd', validated)
-print
+print()
 
-print ('As mentioned earlier, CTDopts can load argument values from CTD files. Feel free to change some values in '
-    "exampleTool_preset_params.ctd you've just written, and load it back.")
+print('As mentioned earlier, CTDopts can load argument values from CTD files. Feel free to change some values in '
+      "exampleTool_preset_params.ctd you've just written, and load it back.")
 args_from_ctd = args_from_file('exampleTool_preset_params.ctd')
 pretty_print(args_from_ctd)
-print
-print ("Notice that all the argument values are strings now. This is because we didn't validate them against the model, "
-    "just loaded some stuff from a file into a dictionary. If you want to cast them, call CTDModel.validate_args():")
+print()
+print("Notice that all the argument values are strings now. This is because we didn't validate them against the model, "
+      "just loaded some stuff from a file into a dictionary. If you want to cast them, call CTDModel.validate_args():")
 validated_2 = model.validate_args(args_from_ctd)
 pretty_print(validated_2)
-print
+print()
 
-print ("Now certain parameters may have restrictions that we might want to validate for as well. Let's set the parameter "
-    "positive_int to a negative value, and try to validate it with a strictness level enforce_restrictions=1. This "
-    "will register a warning, but still accept the value.")
+print("Now certain parameters may have restrictions that we might want to validate for as well. Let's set the parameter "
+      "positive_int to a negative value, and try to validate it with a strictness level enforce_restrictions=1. This "
+      "will register a warning, but still accept the value.")
 validated_2['positive_int'] = -5
 _ = model.validate_args(validated_2, enforce_restrictions=1)
-print
+print()
 
-print ("Validation enforcement levels can be 0, 1 or 2 for type-casting, restriction-checking and required argument presence. "
-    "They can be set with the keywords enforce_type, enforce_restrictions and enforce_required respectively. Let's increase "
-    "strictness for restriction checking. CTDModel.validate_args() will now raise an exception that we'll catch:\n")
+print("Validation enforcement levels can be 0, 1 or 2 for type-casting, restriction-checking and required argument presence. "
+      "They can be set with the keywords enforce_type, enforce_restrictions and enforce_required respectively. Let's increase "
+      "strictness for restriction checking. CTDModel.validate_args() will now raise an exception that we'll catch:\n")
 try:
     model.validate_args(validated_2, enforce_restrictions=2)  # , enforce_type=0, enforce_required=0
 except ArgumentRestrictionError as ee:
     # other exceptions: ArgumentTypeError, ArgumentMissingError, all subclasses of Argumenterror
-    print ee
+    print(ee)
 
-print
-print ("One might want to combine arguments loaded from a CTD file with arguments coming from elsewhere, like the command line."
-    "In that case, the method CTDopts.override_args(*arg_dicts) creates a combined argument dictionary where argument values "
-    "are always taken from the rightmost (last) dictionary that has them. Let's override a few parameters:")
+print()
+print("One might want to combine arguments loaded from a CTD file with arguments coming from elsewhere, like the command line."
+      "In that case, the method CTDopts.override_args(*arg_dicts) creates a combined argument dictionary where argument values "
+      "are always taken from the rightmost (last) dictionary that has them. Let's override a few parameters:")
 override = {
     'this_that': 'that',
     'positive_int': 777
-    }
+}
 overridden = override_args(validated, override)
 pretty_print(overridden)
-print
+print()
 
-print ("So how to deal with command line arguments? If we have a model, we can look for its arguments. "
-    "Call CTDModel.parse_cl_args() with either a string of the command line call or a list with the split words. "
-    "By default, it will assume a '--' prefix before parameter names, but it can be overridden with prefix='-'."
-    "Grouped parameters are expected in --group:subgroup:param_x format.")
+print("So how to deal with command line arguments? If we have a model, we can look for its arguments. "
+      "Call CTDModel.parse_cl_args() with either a string of the command line call or a list with the split words. "
+      "By default, it will assume a '--' prefix before parameter names, but it can be overridden with prefix='-'."
+      "Grouped parameters are expected in --group:subgroup:param_x format.")
 cl_args = model.parse_cl_args('--positive_int 44 --subparams:param_2 5.0 5.5 6.0 --input_files a.fastq b.fastq')
 pretty_print(cl_args)
-print
+print()
 # # you can get unmatchable command line arguments with get_remaining=True like:
 # cl_args, unparsed = model.parse_cl_args('--positive_int 44 --subparams:param_2 5.0 5.5 6.0 --unrelated_stuff abc', get_remaining=True)
 
-print ("Override other parameters with them, and validate it against the model:")
+print("Override other parameters with them, and validate it against the model:")
 overridden_with_cl = override_args(validated, cl_args)
 validated_3 = model.validate_args(overridden_with_cl)
 pretty_print(validated_3)
 print
 
-print ("One last thing: certain command line directives that are specific to CTD functionality can be parsed for, "
-    "to help your script performing common tasks. These are CTD argument input, CTD model file writing and CTD argument "
-    "file writing. CTDopts.parse_cl_directives() can also be customized as to what directives to look for if the defaults "
-    "--input_ctd, --write_tool_ctd and --write_param_ctd respectively don't satisfy you.")
+print("One last thing: certain command line directives that are specific to CTD functionality can be parsed for, "
+      "to help your script performing common tasks. These are CTD argument input, CTD model file writing and CTD argument "
+      "file writing. CTDopts.parse_cl_directives() can also be customized as to what directives to look for if the defaults "
+      "--input_ctd, --write_tool_ctd and --write_param_ctd respectively don't satisfy you.")
 directives_1 = parse_cl_directives('--input_ctd exampleTool_preset_params.ctd --write_param_ctd new_preset_params.ctd')
 pretty_print(directives_1)
 directives_2 = parse_cl_directives('-inctd exampleTool_preset_params_2.ctd -toolctd ', input_ctd='inctd', write_tool_ctd='toolctd', prefix='-')
@@ -227,23 +226,23 @@ pretty_print(directives_2)
 # If I found directives['write_tool_ctd'], I'd immediately output the tool's CTD model with
 # model.write_ctd(directives['write_tool_ctd']), etc.
 
-print ("Finally, writing CTDs with logging information, passing a dictionary with a 'log' keyword, using any or all of the "
-    "fields shown below.")
-time_start = datetime.datetime.now(pytz.utc).isoformat()
+print("Finally, writing CTDs with logging information, passing a dictionary"
+      "with a 'log' keyword, using any or all of the fields shown below.")
+time_start = datetime.datetime.now(datetime.timezone.utc).isoformat()
 # do stuff
 output = 'Output of my program, however I generated or logged it'
 errors = 'Standard error output of my program, however I caught or redirected them'
 warnings = 'Warnings of my program'
 exitstatus = '1'
-time_finish = datetime.datetime.now(pytz.utc).isoformat()
+time_finish = datetime.datetime.now(datetime.timezone.utc).isoformat()
 
 log = {
-'time_start': time_start,  # make sure to give it a legal XML date string if you can.
-'time_finish': time_finish,  # You can generate them with datetime.datetime.now(pytz.utc).isoformat()
-'status': exitstatus,
-'output': output,
-'warning': warnings,
-'error': errors
+    'time_start': time_start,  # make sure to give it a legal XML date string if you can.
+    'time_finish': time_finish,  # You can generate them with datetime.datetime.now(pytz.utc).isoformat()
+    'status': exitstatus,
+    'output': output,
+    'warning': warnings,
+    'error': errors
 }
 
 model.write_ctd('exampleTool_w_logging.ctd', validated_3, log)
@@ -252,4 +251,3 @@ model.write_ctd('exampleTool_w_logging.ctd', validated_3, log)
 # Methods you might find helpful to deal with argument dictionaries (see docstrings):
 # CTDopts.set_nested_key(dictionary, tuple_w_levels, value) and
 # CTDopts.get_nested_key(dictionary, tuple_w_levels for nested dictionaries
-


=====================================
setup.cfg
=====================================
@@ -0,0 +1,2 @@
+[pycodestyle]
+ignore = E501


=====================================
setup.py
=====================================
@@ -2,7 +2,7 @@ from distutils.core import setup
 
 setup(
     name='CTDopts',
-    version='1.1',
+    version='1.3',
     packages=['CTDopts'],
     url='https://github.com/genericworkflownodes/CTDopts',
     license='',



View it on GitLab: https://salsa.debian.org/med-team/ctdopts/-/compare/27cdc56cdc2b419276df2bd9d3dba66b734fa405...4a39ca24de64183bd3bff202f4992c83ce5fd891

-- 
View it on GitLab: https://salsa.debian.org/med-team/ctdopts/-/compare/27cdc56cdc2b419276df2bd9d3dba66b734fa405...4a39ca24de64183bd3bff202f4992c83ce5fd891
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20200404/04559784/attachment-0001.html>


More information about the debian-med-commit mailing list