[Git][debian-gis-team/trollsift][upstream] New upstream version 0.5.0
Antonio Valentino (@antonio.valentino)
gitlab at salsa.debian.org
Wed Nov 23 06:52:59 GMT 2022
Antonio Valentino pushed to branch upstream at Debian GIS Project / trollsift
Commits:
b556eb74 by Antonio Valentino at 2022-11-23T06:47:31+00:00
New upstream version 0.5.0
- - - - -
10 changed files:
- + .github/dependabot.yml
- .github/workflows/ci.yaml
- .github/workflows/deploy-sdist.yaml
- AUTHORS.md
- CHANGELOG.md
- RELEASING.md
- trollsift/parser.py
- trollsift/tests/integrationtests/test_parser.py
- trollsift/tests/unittests/test_parser.py
- trollsift/version.py
Changes:
=====================================
.github/dependabot.yml
=====================================
@@ -0,0 +1,11 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+ - package-ecosystem: "github-actions" # See documentation for possible values
+ directory: "/" # Location of package manifests
+ schedule:
+ interval: "weekly"
=====================================
.github/workflows/ci.yaml
=====================================
@@ -18,10 +18,10 @@ jobs:
steps:
- name: Checkout source
- uses: actions/checkout at v2
+ uses: actions/checkout at v3
- name: Set up Python ${{ matrix.python-version }}
- uses: actions/setup-python at v2
+ uses: actions/setup-python at v4
with:
python-version: ${{ matrix.python-version }}
@@ -38,7 +38,7 @@ jobs:
pytest --cov=trollsift trollsift/tests --cov-report=xml
- name: Upload unittest coverage to Codecov
- uses: codecov/codecov-action at v1
+ uses: codecov/codecov-action at v3
with:
flags: unittests
file: ./coverage.xml
=====================================
.github/workflows/deploy-sdist.yaml
=====================================
@@ -11,7 +11,7 @@ jobs:
steps:
- name: Checkout source
- uses: actions/checkout at v2
+ uses: actions/checkout at v3
- name: Create sdist
shell: bash -l {0}
@@ -19,7 +19,7 @@ jobs:
- name: Publish package to PyPI
if: github.event.action == 'published'
- uses: pypa/gh-action-pypi-publish at v1.4.1
+ uses: pypa/gh-action-pypi-publish at v1.5.1
with:
user: __token__
password: ${{ secrets.pypi_password }}
\ No newline at end of file
=====================================
AUTHORS.md
=====================================
@@ -11,3 +11,4 @@ The following people have made contributions to this project:
- [Hrobjartur Thorsteinsson (thorsteinssonh)](https://github.com/thorsteinssonh)
- [Stephan Finkensieper (sfinkens)](https://github.com/sfinkens)
- [Paulo Medeiros (paulovcmedeiros)](https://github.com/paulovcmedeiros)
+- [Regan Koopmans (Regan-Koopmans)](https://github.com/Regan-Koopmans)
\ No newline at end of file
=====================================
CHANGELOG.md
=====================================
@@ -1,3 +1,28 @@
+## Version 0.5.0 (2022/11/21)
+
+### Issues Closed
+
+* [Issue 45](https://github.com/pytroll/trollsift/issues/45) - Provide simple access to defined keys of a parser instance ([PR 46](https://github.com/pytroll/trollsift/pull/46) by [@carloshorn](https://github.com/carloshorn))
+* [Issue 37](https://github.com/pytroll/trollsift/issues/37) - Global instances of formatters ([PR 38](https://github.com/pytroll/trollsift/pull/38) by [@Regan-Koopmans](https://github.com/Regan-Koopmans))
+* [Issue 36](https://github.com/pytroll/trollsift/issues/36) - Alignment marker is not optional for numbers when it should
+* [Issue 34](https://github.com/pytroll/trollsift/issues/34) - Trollsift doesn't parse hex numbers ([PR 35](https://github.com/pytroll/trollsift/pull/35) by [@mraspaud](https://github.com/mraspaud))
+
+In this release 4 issues were closed.
+
+### Pull Requests Merged
+
+#### Bugs fixed
+
+* [PR 38](https://github.com/pytroll/trollsift/pull/38) - Replace global RegexFormatter with memoized function ([37](https://github.com/pytroll/trollsift/issues/37))
+* [PR 35](https://github.com/pytroll/trollsift/pull/35) - Add hex, octal, and binary parsing ([34](https://github.com/pytroll/trollsift/issues/34))
+
+#### Features added
+
+* [PR 46](https://github.com/pytroll/trollsift/pull/46) - Add keys method to Parser class ([45](https://github.com/pytroll/trollsift/issues/45))
+
+In this release 3 pull requests were closed.
+
+
## Version 0.4.0 (2022/02/03)
### Issues Closed
=====================================
RELEASING.md
=====================================
@@ -5,21 +5,27 @@
3. run the unittests
4. run `loghub` and update the `CHANGELOG.md` file:
-```
-loghub pytroll/trollsift --token $LOGHUB_GITHUB_TOKEN -st v0.8.0 -plg bug "Bugs fixed" -plg enhancement "Features added" -plg documentation "Documentation changes" -plg backwards-incompatibility "Backwards incompatible changes"
-```
+ ```
+ loghub pytroll/trollsift --token $LOGHUB_GITHUB_TOKEN -st $(git tag --sort=-version:refname --list 'v*' | head -n 1) -plg bug "Bugs fixed" -plg enhancement "Features added" -plg documentation "Documentation changes" -plg backwards-incompatibility "Backwards incompatible changes"
+ ```
-Don't forget to commit!
+ Don't forget to commit!
5. Create a tag with the new version number, starting with a 'v', eg:
-```
-git tag -a v0.22.45 -m "Version 0.22.45"
-```
-
-See [semver.org](http://semver.org/) on how to write a version number.
-
+ ```
+ git tag -a v0.22.45 -m "Version 0.22.45"
+ ```
+ See [semver.org](http://semver.org/) on how to write a version number.
6. push changes to github `git push --follow-tags`
-7. Verify travis tests passed and deployed sdist and wheel to PyPI
+7. Verify tests pass on GitHub Actions
+8. Create a "Release" on GitHub by going to
+ https://github.com/pytroll/pyresample/releases and clicking "Draft a new release".
+ On the next page enter the newly created tag in the "Tag version" field,
+ "Version X.Y.Z" in the "Release title" field, and paste the markdown from
+ the changelog (the portion under the version section header) in the
+ "Describe this release" box. Finally click "Publish release".
+9. Verify the GitHub actions for deployment succeed and the release is on PyPI.
+
=====================================
trollsift/parser.py
=====================================
@@ -33,6 +33,11 @@ class Parser(object):
def __str__(self):
return self.fmt
+ def keys(self):
+ """Get parameter names defined in the format string."""
+ convert_dict = get_convert_dict(self.fmt)
+ return convert_dict.keys()
+
def parse(self, stri, full_match=True):
'''Parse keys and corresponding values from *stri* using format
described in *fmt* string.
@@ -67,7 +72,7 @@ class Parser(object):
def validate(self, stri):
"""
Validates that string *stri* is parsable and therefore complies with
- this string format definition. Useful for filtering strings, or to
+ this string format definition. Useful for filtering strings, or to
check if a string if compatible before passing it to the
parser function.
"""
@@ -83,9 +88,9 @@ class Parser(object):
or loss in information.
Note: This test only applies to sensible usage of the format string.
- If string or numeric data is causes overflow, e.g.
- if composing "abcd" into {3s}, one to one correspondence will always
- be broken in such cases. This off course also applies to precision
+ If string or numeric data is causes overflow, e.g.
+ if composing "abcd" into {3s}, one to one correspondence will always
+ be broken in such cases. This off course also applies to precision
losses when using datetime data.
"""
return is_one2one(self.fmt)
@@ -142,6 +147,7 @@ formatter = StringFormatter()
# taken from https://docs.python.org/3/library/re.html#simulating-scanf
spec_regexes = {
+ 'b': r'[-+]?[0-1]',
'c': r'.',
'd': r'[-+]?\d',
'f': {
@@ -164,7 +170,7 @@ spec_regexes['E'] = spec_regexes['f']
spec_regexes['g'] = spec_regexes['f']
spec_regexes['X'] = spec_regexes['x']
spec_regexes[''] = spec_regexes['s']
-allow_multiple = ['c', 'd', 'o', 's', '', 'x', 'X']
+allow_multiple = ['b', 'c', 'd', 'o', 's', '', 'x', 'X']
fixed_point_types = ['f', 'e', 'E', 'g']
# format_spec ::= [[fill]align][sign][#][0][width][,][.precision][type]
# https://docs.python.org/3.4/library/string.html#format-specification-mini-language
@@ -195,7 +201,7 @@ def _get_fixed_point_regex(regex_dict, width, precision):
class RegexFormatter(string.Formatter):
"""String formatter that converts a format string to a regular expression.
-
+
>>> regex_formatter = RegexFormatter()
>>> regex_str = regex_formatter.format('{field_one:5d}_{field_two}')
@@ -295,7 +301,7 @@ class RegexFormatter(string.Formatter):
if fill is None:
if width is not None and width[0] == '0':
fill = '0'
- elif ftype in ['s', '', 'd']:
+ elif ftype in ['s', '', 'd', 'x', 'X', 'o', 'b']:
fill = ' '
char_type = spec_regexes[ftype]
@@ -349,26 +355,30 @@ class RegexFormatter(string.Formatter):
field_name, value = value
return self.regex_field(field_name, value, format_spec)
- def extract_values(self, fmt, stri, full_match=True):
- """Extract information from string matching format.
- Args:
- fmt (str): Python format string to match against
- stri (str): String to extract information from
- full_match (bool): Force the match of the whole string. Default
- to ``True``.
+ at lru_cache()
+def regex_format(fmt):
+ # We create a new instance of RegexFormatter here to prevent concurrent calls to
+ # format interfering with one another.
+ return RegexFormatter().format(fmt)
- """
- regex = self.format(fmt)
- if full_match:
- regex = '^' + regex + '$'
- match = re.match(regex, stri)
- if match is None:
- raise ValueError("String does not match pattern.")
- return match.groupdict()
+def extract_values(fmt, stri, full_match=True):
+ """Extract information from string matching format.
-regex_formatter = RegexFormatter()
+ Args:
+ fmt (str): Python format string to match against
+ stri (str): String to extract information from
+ full_match (bool): Force the match of the whole string. Default
+ to ``True``.
+ """
+ regex = regex_format(fmt)
+ if full_match:
+ regex = '^' + regex + '$'
+ match = re.match(regex, stri)
+ if match is None:
+ raise ValueError("String does not match pattern.")
+ return match.groupdict()
def _get_number_from_fmt(fmt):
@@ -394,6 +404,12 @@ def _convert(convdef, stri):
result = _strip_padding(convdef, stri)
if 'd' in convdef:
result = int(result)
+ elif 'x' in convdef or 'X' in convdef:
+ result = int(result, 16)
+ elif 'o' in convdef:
+ result = int(result, 8)
+ elif 'b' in convdef:
+ result = int(result, 2)
elif any(float_type_marker in convdef for float_type_marker in fixed_point_types):
result = float(result)
@@ -447,7 +463,7 @@ def parse(fmt, stri, full_match=True):
"""
convdef = get_convert_dict(fmt)
- keyvals = regex_formatter.extract_values(fmt, stri, full_match=full_match)
+ keyvals = extract_values(fmt, stri, full_match=full_match)
for key in convdef.keys():
keyvals[key] = _convert(convdef[key], keyvals[key])
@@ -556,7 +572,7 @@ def globify(fmt, keyvals=None):
def validate(fmt, stri):
"""
Validates that string *stri* is parsable and therefore complies with
- the format string, *fmt*. Useful for filtering string, or to
+ the format string, *fmt*. Useful for filtering string, or to
check if string if compatible before passing the string to the
parser function.
"""
@@ -632,8 +648,8 @@ def is_one2one(fmt):
or loss in information.
Note: This test only applies to sensible usage of the format string.
- If string or numeric data is causes overflow, e.g.
- if composing "abcd" into {3s}, one to one correspondence will always
+ If string or numeric data is causes overflow, e.g.
+ if composing "abcd" into {3s}, one to one correspondence will always
be broken in such cases. This of course also applies to precision
losses when using datetime data.
"""
@@ -663,7 +679,7 @@ def purge():
is very limited.
"""
- regex_formatter.format.cache_clear()
+ regex_format.cache_clear()
get_convert_dict.cache_clear()
=====================================
trollsift/tests/integrationtests/test_parser.py
=====================================
@@ -42,15 +42,14 @@ class TestParser(unittest.TestCase):
def test_cache_clear(self):
"""Test we can clear the internal cache properly"""
- from trollsift.parser import purge
- from trollsift.parser import regex_formatter
+ from trollsift.parser import purge, regex_format
# Run
result = self.p.parse(self.string)
# Assert
self.assertDictEqual(result, self.data)
- assert regex_formatter.format.cache_info()[-1] != 0
+ assert regex_format.cache_info()[-1] != 0
purge()
- assert regex_formatter.format.cache_info()[-1] == 0
+ assert regex_format.cache_info()[-1] == 0
def test_compose(self):
# Run
=====================================
trollsift/tests/unittests/test_parser.py
=====================================
@@ -2,9 +2,9 @@ import unittest
import datetime as dt
import pytest
-from trollsift.parser import get_convert_dict, regex_formatter
+from trollsift.parser import get_convert_dict, extract_values
from trollsift.parser import _convert
-from trollsift.parser import parse, globify, validate, is_one2one, compose
+from trollsift.parser import parse, globify, validate, is_one2one, compose, Parser
class TestParser(unittest.TestCase):
@@ -17,6 +17,12 @@ class TestParser(unittest.TestCase):
self.string3 = "/somedir/otherdir/hrpt_noaa16_20140210_1004_69022"
self.string4 = "/somedir/otherdir/hrpt_noaa16_20140210_1004_69022"
+ def test_parser_keys(self):
+ parser = Parser(self.fmt)
+ keys = {"directory", "platform", "platnum", "time", "orbit"}
+ self.assertTrue(keys.issubset(parser.keys())
+ and keys.issuperset(parser.keys()))
+
def test_get_convert_dict(self):
# Run
result = get_convert_dict(self.fmt)
@@ -31,54 +37,54 @@ class TestParser(unittest.TestCase):
def test_extract_values(self):
fmt = "/somedir/{directory}/hrpt_{platform:4s}{platnum:2s}_{time:%Y%m%d_%H%M}_{orbit:d}.l1b"
- result = regex_formatter.extract_values(fmt, self.string)
+ result = extract_values(fmt, self.string)
self.assertDictEqual(result, {'directory': 'otherdir',
'platform': 'noaa', 'platnum': '16',
'time': '20140210_1004', 'orbit': '69022'})
def test_extract_values_end(self):
fmt = "/somedir/{directory}/hrpt_{platform:4s}{platnum:2s}_{time:%Y%m%d_%H%M}_{orbit:d}"
- result = regex_formatter.extract_values(fmt, self.string3)
+ result = extract_values(fmt, self.string3)
self.assertDictEqual(result, {'directory': 'otherdir',
'platform': 'noaa', 'platnum': '16',
'time': '20140210_1004', 'orbit': '69022'})
def test_extract_values_beginning(self):
fmt = "{directory}/hrpt_{platform:4s}{platnum:2s}_{time:%Y%m%d_%H%M}_{orbit:d}"
- result = regex_formatter.extract_values(fmt, self.string4)
+ result = extract_values(fmt, self.string4)
self.assertDictEqual(result, {'directory': '/somedir/otherdir',
'platform': 'noaa', 'platnum': '16',
'time': '20140210_1004', 'orbit': '69022'})
def test_extract_values_s4spair(self):
fmt = "{directory}/hrpt_{platform:4s}{platnum:s}_{time:%Y%m%d_%H%M}_{orbit:d}"
- result = regex_formatter.extract_values(fmt, self.string4)
+ result = extract_values(fmt, self.string4)
self.assertDictEqual(result, {'directory': '/somedir/otherdir',
'platform': 'noaa', 'platnum': '16',
'time': '20140210_1004', 'orbit': '69022'})
def test_extract_values_ss2pair(self):
fmt = "{directory}/hrpt_{platform:s}{platnum:2s}_{time:%Y%m%d_%H%M}_{orbit:d}"
- result = regex_formatter.extract_values(fmt, self.string4)
+ result = extract_values(fmt, self.string4)
self.assertDictEqual(result, {'directory': '/somedir/otherdir',
'platform': 'noaa', 'platnum': '16',
'time': '20140210_1004', 'orbit': '69022'})
def test_extract_values_ss2pair_end(self):
fmt = "{directory}/hrpt_{platform:s}{platnum:2s}"
- result = regex_formatter.extract_values(fmt, "/somedir/otherdir/hrpt_noaa16")
+ result = extract_values(fmt, "/somedir/otherdir/hrpt_noaa16")
self.assertDictEqual(result, {'directory': '/somedir/otherdir',
'platform': 'noaa', 'platnum': '16'})
def test_extract_values_sdatetimepair_end(self):
fmt = "{directory}/hrpt_{platform:s}{date:%Y%m%d}"
- result = regex_formatter.extract_values(fmt, "/somedir/otherdir/hrpt_noaa20140212")
+ result = extract_values(fmt, "/somedir/otherdir/hrpt_noaa20140212")
self.assertDictEqual(result, {'directory': '/somedir/otherdir',
'platform': 'noaa', 'date': '20140212'})
def test_extract_values_everything(self):
fmt = "{everything}"
- result = regex_formatter.extract_values(fmt, self.string)
+ result = extract_values(fmt, self.string)
self.assertDictEqual(
result, {'everything': '/somedir/otherdir/hrpt_noaa16_20140210_1004_69022.l1b'})
@@ -88,7 +94,7 @@ class TestParser(unittest.TestCase):
# {'platform': '4s'}, {'platnum': '2s'},
# '_', {'time': '%Y%m%d_%H%M'}, '_',
# {'orbit': '0>5d'}, '.l1b']
- result = regex_formatter.extract_values(fmt, self.string2)
+ result = extract_values(fmt, self.string2)
# Assert
self.assertDictEqual(result, {'directory': 'otherdir',
'platform': 'noaa', 'platnum': '16',
@@ -96,15 +102,15 @@ class TestParser(unittest.TestCase):
def test_extract_values_fails(self):
fmt = '/somedir/{directory}/hrpt_{platform:4s}{platnum:2s}_{time:%Y%m%d_%H%M}_{orbit:4d}.l1b'
- self.assertRaises(ValueError, regex_formatter.extract_values, fmt, self.string)
+ self.assertRaises(ValueError, extract_values, fmt, self.string)
def test_extract_values_full_match(self):
"""Test that a string must completely match."""
fmt = '{orbit:05d}'
- val = regex_formatter.extract_values(fmt, '12345')
+ val = extract_values(fmt, '12345')
self.assertEqual(val, {'orbit': '12345'})
- self.assertRaises(ValueError, regex_formatter.extract_values, fmt, '12345abc')
- val = regex_formatter.extract_values(fmt, '12345abc', full_match=False)
+ self.assertRaises(ValueError, extract_values, fmt, '12345abc')
+ val = extract_values(fmt, '12345abc', full_match=False)
self.assertEqual(val, {'orbit': '12345'})
def test_convert_digits(self):
@@ -451,3 +457,34 @@ class TestParserFixedPoint:
"""Test cases expected to not be matched."""
with pytest.raises(ValueError):
parse(fmt, string)
+
+
+ at pytest.mark.parametrize(
+ ('fmt', 'string', 'expected'),
+ [
+ # Decimal
+ ("{foo:d}", "123", 123),
+ # Hex with small letter
+ ("{foo:x}", "7b", 123),
+ # Hex with big letter
+ ("{foo:X}", "7B", 123),
+ # Fixed length hex
+ ("{foo:03x}", "07b", 123),
+ ("{foo:3x}", " 7b", 123),
+ ("{foo:3X}", " 7B", 123),
+ # Octal
+ ("{foo:o}", "173", 123),
+ # Free size with octal
+ ("{bar:s}{foo:o}", "something173", 123),
+ # Fixed length with octal
+ ("{foo:_>4o}", "_173", 123),
+ ("{foo:4o}", " 173", 123),
+ # Binary
+ ("{foo:b}", "1111011", 123),
+ # Fixed length with binary
+ ("{foo:8b}", " 1111011", 123),
+ ("{foo:_>8b}", "_1111011", 123),
+ ]
+)
+def test_parse_integers(fmt, string, expected):
+ assert parse(fmt, string)["foo"] == expected
=====================================
trollsift/version.py
=====================================
@@ -23,9 +23,9 @@ def get_keywords():
# setup.py/versioneer.py will grep for the variable names, so they must
# each be defined on a line of their own. _version.py will just call
# get_keywords().
- git_refnames = " (HEAD -> main, tag: v0.4.0)"
- git_full = "77c3efcc58eda33ae8ffa3579db811912e09de6a"
- git_date = "2022-02-03 19:55:29 -0600"
+ git_refnames = " (HEAD -> main, tag: v0.5.0)"
+ git_full = "07df6df7f906ecffe85ba523438562d7b063de2b"
+ git_date = "2022-11-21 07:50:03 -0600"
keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
return keywords
View it on GitLab: https://salsa.debian.org/debian-gis-team/trollsift/-/commit/b556eb74bb0a58d1bcfa64f1025799459242dc87
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/trollsift/-/commit/b556eb74bb0a58d1bcfa64f1025799459242dc87
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20221123/4a091548/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list