[protozero] 01/06: Imported Upstream version 1.5.0
Bas Couwenberg
sebastic at debian.org
Thu Jan 12 14:32:55 UTC 2017
This is an automated email from the git hooks/post-receive script.
sebastic pushed a commit to branch master
in repository protozero.
commit 2c92fb9ac06d7c2fd5c27eebde92ebcaa72771c0
Author: Bas Couwenberg <sebastic at xs4all.nl>
Date: Thu Jan 12 14:23:45 2017 +0100
Imported Upstream version 1.5.0
---
.travis.yml | 21 ++--
CHANGELOG.md | 28 ++++-
README.md | 17 +--
doc/advanced.md | 172 +++++++++++++++++++++++++++++
doc/macros.md | 40 -------
doc/tutorial.md | 50 +--------
include/protozero/pbf_builder.hpp | 5 +
include/protozero/pbf_reader.hpp | 68 ++++++++++++
include/protozero/pbf_writer.hpp | 66 ++++++++---
include/protozero/types.hpp | 15 ++-
include/protozero/varint.hpp | 12 +-
include/protozero/version.hpp | 6 +-
package.json | 2 +-
test/include/catch.hpp | 14 ++-
test/t/bytes/test_cases.cpp | 52 +++++++++
test/t/repeated_packed_bool/test_cases.cpp | 40 +++++++
test/t/vector_tile/test_cases.cpp | 60 +++++++++-
test/t/zigzag/test_cases.cpp | 40 ++++++-
18 files changed, 560 insertions(+), 148 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index de09019..224d250 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -49,43 +49,43 @@ matrix:
osx_image: xcode8
compiler: clang
- os: linux
- compiler: "clang35"
+ compiler: "clang-3.5"
env: CXX=clang++-3.5
addons: *clang35
- os: linux
- compiler: "clang38"
+ compiler: "clang-3.8"
env: CXX=clang++-3.8
addons: *clang38
- os: linux
- compiler: "clang38"
+ compiler: "clang-3.8"
env: CXX=clang++-3.8 CXX_STD=c++14
addons: *clang38
- os: linux
- compiler: "gcc47"
+ compiler: "gcc-4.7"
env: CXX=g++-4.7
addons: *gcc47
- os: linux
- compiler: "gcc48"
+ compiler: "gcc-4.8"
env: CXX=g++-4.8
addons: *gcc48
- os: linux
- compiler: "gcc49"
+ compiler: "gcc-4.9"
env: CXX=g++-4.9 COVERAGE=gcov-4.9
addons: *gcc49
- os: linux
- compiler: "gcc49"
+ compiler: "gcc-4.9"
env: CXX=g++-4.9 CXX_STD=c++14
addons: *gcc49
- os: linux
- compiler: "gcc5"
+ compiler: "gcc-5"
env: CXX=g++-5 CXXFLAGS="-D_GLIBCXX_USE_CXX11_ABI=0"
addons: *gcc5
- os: linux
- compiler: "gcc5"
+ compiler: "gcc-5"
env: CXX=g++-5 CXXFLAGS="-D_GLIBCXX_USE_CXX11_ABI=1"
addons: *gcc5
- os: linux
- compiler: "gcc6"
+ compiler: "gcc-6"
env: CXX=g++-6
addons: *gcc6
@@ -98,6 +98,7 @@ before_install:
export PYTHONPATH=$(pwd)/.local/lib/python/site-packages;
fi
- if [ -n "${COVERAGE}" ]; then
+ PYTHONUSERBASE=$(pwd)/.local pip install --user urllib3[secure];
PYTHONUSERBASE=$(pwd)/.local pip install --user cpp-coveralls;
fi
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b8dbbca..938fd4c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -13,6 +13,31 @@ This project adheres to [Semantic Versioning](http://semver.org/).
### Fixed
+## [1.5.0] - 2017-01-12
+
+### Added
+
+- Add `add_bytes_vectored()` methods to `pbf_writer` and `pbf_builder`. This
+ allows single-copy scatter-gather type adding of data that has been prepared
+ in pieces to a protobuf message.
+- New functions to check the tag and wire type at the same time: Two parameter
+ version of `pbf_reader::next()` and `pbf_reader::tag_and_type()` can be used
+ together with the free function `tag_and_type()` to easily and quickly check
+ that not only the tag but also the wire type is correct for a field.
+
+### Changed
+
+- `packed_field_*` classes now work with `pbf_builder`.
+- Reorganized documentation. Advanced docs are now under doc/advanced.md.
+
+### Fixed
+
+- `packed_field` class is now non-copyable because data can get corrupted if
+ you copy it around.
+- Comparison operators of `data_view` now have const& parameters.
+- Make zigzag encoding/decoding functions constexpr.
+
+
## [1.4.5] - 2016-11-18
### Fixed
@@ -131,7 +156,8 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- Make pbf reader and writer code endianess-aware.
-[unreleased]: https://github.com/osmcode/libosmium/compare/v1.4.5...HEAD
+[unreleased]: https://github.com/osmcode/libosmium/compare/v1.5.0...HEAD
+[1.5.0]: https://github.com/osmcode/libosmium/compare/v1.4.5...v1.5.0
[1.4.5]: https://github.com/osmcode/libosmium/compare/v1.4.4...v1.4.5
[1.4.4]: https://github.com/osmcode/libosmium/compare/v1.4.3...v1.4.4
[1.4.3]: https://github.com/osmcode/libosmium/compare/v1.4.2...v1.4.3
diff --git a/README.md b/README.md
index 0fa752b..5d19fff 100644
--- a/README.md
+++ b/README.md
@@ -48,9 +48,9 @@ You have to have a working knowledge of how
* Read the [tutorial](doc/tutorial.md) for an introduction on how to use
Protozero.
+* Some advanced topics are described in an [extra document](doc/advanced.md).
* There is a table of all types and functions in the
[cheat sheet](doc/cheatsheet.md).
-* [Macros defined or used by Protozero](doc/macros.md).
* Read the [upgrading instructions](UPGRADING.md) if you are upgrading from
an older version of Protozero.
@@ -65,21 +65,6 @@ Call `make install` to install include files in `/usr/include/protozero`. Call
`make install DESTDIR=/usr/local` or similar to change install directory.
-## Limitations
-
-* A protobuf message has to fit into memory completely, otherwise it can not
- be parsed with this library. There is no streaming support.
-* The length of a string, bytes, or submessage can't be more than 2^31-1.
-* The Google Protobuf spec documents that a non-repeated field can actually
- appear several times in a message and the implementation is required to
- return the value of the last version of that field in this case.
- `pbf_reader.hpp` does not enforce this. If this feature is needed in your
- case, you have to do this yourself.
-* There is no specific support for maps but they can be used as described in
- the "Backwards compatibility" section of
- https://developers.google.com/protocol-buffers/docs/proto3#maps.
-
-
## Endianness
Protozero uses a very simplistic test to check the byte order of the system it
diff --git a/doc/advanced.md b/doc/advanced.md
new file mode 100644
index 0000000..8032bcb
--- /dev/null
+++ b/doc/advanced.md
@@ -0,0 +1,172 @@
+
+# Protozero Advanced Topics
+
+This documentation contains some mixed advanced topics for Protozero users.
+Read the [tutorial](tutorial.md) first if you are new to Protozero.
+
+
+## Limitations of Protozero
+
+* A protobuf message has to fit into memory completely, otherwise it can not
+ be parsed with this library. There is no streaming support.
+* The length of a string, bytes, or submessage can't be more than 2^31-1.
+* There is no specific support for maps but they can be used as described in
+ the "Backwards compatibility" section of
+ https://developers.google.com/protocol-buffers/docs/proto3#maps.
+
+
+## Checking the Protozero version number
+
+If `protozero/version.hpp` is included, the following macros are set:
+
+| Macro | Example | Description |
+| -------------------------- | ------- | ---------------------------------------------- |
+| `PROTOZERO_VERSION_MAJOR` | 1 | Major version number |
+| `PROTOZERO_VERSION_MINOR` | 3 | Minor version number |
+| `PROTOZERO_VERSION_PATCH` | 2 | Patch number |
+| `PROTOZERO_VERSION_CODE` | 10302 | Version (major * 10,000 + minor * 100 + patch) |
+| `PROTOZERO_VERSION_STRING` | "1.3.2" | Version string |
+
+
+## Changing Protozero behaviour with macros
+
+The behaviour of Protozero can be changed by defining the following macros.
+They have to be set before including any of the Protozero headers.
+
+## `PROTOZERO_STRICT_API`
+
+If this is set, you will get some extra warnings or errors during compilation
+if you are using an old (deprecated) interface to Protozero. Enable this if
+you want to make sure your code will work with future versions of Protozero.
+
+## `PROTOZERO_USE_VIEW`
+
+Protozero uses the class `protozero::data_view` as the return type of the
+`pbf_reader::get_view()` method and a few other functions take a
+`protozero::data_view` as parameter.
+
+If `PROTOZERO_USE_VIEW` is unset, `protozero::data_view` is Protozero's own
+implementation of a *string view* class.
+
+Set this macro if you want to use a different implementation such as the C++17
+`std::string_view` class. In this case `protozero::data_view` will simply be
+an alias to the class you specify.
+
+ #define PROTOZERO_USE_VIEW std::string_view
+
+
+## Repeated fields in messages
+
+The Google Protobuf spec documents that a non-repeated field can actually
+appear several times in a message and the implementation is required to return
+the value of the last version of that field in this case. `pbf_reader.hpp` does
+not enforce this. If this feature is needed in your case, you have to do this
+yourself.
+
+The [spec also
+says](https://developers.google.com/protocol-buffers/docs/encoding#packed)
+that you must be able to read a packed repeated field where a not-packed
+repeated field is expected and vice versa. Also there can be several (packed or
+not-packed) repeated fields with the same tag and their contents must be
+concatenated. It is your responsibility to do this, Protozero doesn't do that
+for you.
+
+
+## Reserving memory when writing messages
+
+If you know beforehand how large a message will become or can take an educated
+guess, you can call the usual `std::string::reserve()` on the underlying string
+before you give it to an `pbf_writer` or `pbf_builder` object.
+
+Or you can (at any time) call `reserve()` on the `pbf_writer` or `pbf_builder`.
+This will reserve the given amount of bytes *in addition to whatever is already
+in that message*. (Note that this behaviour is different then what `reserve()`
+does on `std::string` or `std::vector`.)
+
+In the general case it is not easy to figure out how much memory you will need
+because of the varint packing of integers. But sometimes you can make at least
+a rough estimate. Still, you should probably only use this facility if you have
+benchmarks proving that it actually makes your program faster.
+
+
+## Using the Low-Level Varint and Zigzag Encoding and Decoding Functions
+
+Protozero gives you access to the low-level functions for encoding and
+decoding varint and zigzag integer encodings, because these functions can
+sometimes be useful outside the Protocol Buffer context.
+
+### Using Low-Level Functions
+
+To use the low-level functions, add this include to your C++ program:
+
+```cpp
+#include <protozero/varint.hpp>
+```
+
+### Functions
+
+The following functions are then available:
+
+```cpp
+decode_varint()
+write_varint()
+encode_zigzag32()
+encode_zigzag64()
+decode_zigzag32()
+decode_zigzag64()
+```
+
+See the reference documentation created by `make doc` for details.
+
+
+## Vectored input for length-delimited fields
+
+Length-delimited fields (like string fields, byte fields and messages) are
+usually set by calling `add_string()`, `add_message()`, etc. These functions
+have several forms, but they basically all take a *tag*, a *size*, and a
+*pointer to the data*. They write the length of the data into the message
+and then copy the data over.
+
+Sometimes you have the data not in one place, but spread over several
+buffers. In this case you have to consolidate those buffers first, which needs
+an extra copy. Say you have two very long strings that should be concatenated
+into a message:
+
+```cpp
+std::string a{"very long string..."};
+std::string b{"another very long string..."};
+
+std::string data;
+protozero::pbf_writer writer{data};
+
+a.append(b); // expensive extra copy
+
+writer.add_string(1, a);
+```
+
+To avoid this, the function `add_bytes_vectored()` can be used which allows
+vectored (or scatter/gather) input like this:
+
+```cpp
+std::string a{"very long string..."};
+std::string b{"another very long string..."};
+
+std::string data;
+protozero::pbf_writer writer{data};
+
+writer.add_bytes_vectored(1, a, b);
+```
+
+`add_bytes_vectored()` will add up the sizes of all its arguments and copy over
+all the data only once.
+
+The function takes any number of arguments. The arguments must be of a type
+supporting the `data()` and `size()` methods like `protozero::data_view()`,
+`std::string` or the C++17 `std::string_view`.
+
+Note that there is only one version of the function which can be used for any
+length-delimited field including strings, bytes, messages and repeated packed
+fields.
+
+The function is also available in the `pbf_builder` class.
+
diff --git a/doc/macros.md b/doc/macros.md
deleted file mode 100644
index e16ab1e..0000000
--- a/doc/macros.md
+++ /dev/null
@@ -1,40 +0,0 @@
-
-# Macros
-
-## Version number
-
-If `protozero/version.hpp` is included, the following macros are set:
-
-| Macro | Example | Description |
-| -------------------------- | ------- | ---------------------------------------------- |
-| `PROTOZERO_VERSION_MAJOR` | 1 | Major version number |
-| `PROTOZERO_VERSION_MINOR` | 3 | Minor version number |
-| `PROTOZERO_VERSION_PATCH` | 2 | Patch number |
-| `PROTOZERO_VERSION_CODE` | 10302 | Version (major * 10,000 + minor * 100 + patch) |
-| `PROTOZERO_VERSION_STRING` | "1.3.2" | Version string |
-
-## Changing Protozero behaviour
-
-The behaviour of Protozero can be changed by defining the following macros.
-They have to be set before including any of the Protozero headers.
-
-## `PROTOZERO_STRICT_API`
-
-If this is set, you will get some extra warnings or errors during compilation
-if you are using an old (deprecated) interface to Protozero. Enable this if
-you want to make sure your code will work with future versions of Protozero.
-
-## `PROTOZERO_USE_VIEW`
-
-If this is unset `protozero::data_view` is Protozero's own implementation of
-a *string view* class.
-
-Set this if you want to use a different implementation such as the C++17
-`std::string_view` class. In this case `protozero::data_view` will simply be
-an alias to the class you specify.
-
- #define PROTOZERO_USE_VIEW std::string_view
-
-This affects the result type of the `pbf_reader::get_view()` method and a few
-others taking a `protozero::data_view` as parameters.
-
diff --git a/doc/tutorial.md b/doc/tutorial.md
index 65c00c9..0254eba 100644
--- a/doc/tutorial.md
+++ b/doc/tutorial.md
@@ -21,7 +21,7 @@ and [the cheat sheet](cheatsheet.md) if you are getting lost.
## Prerequisites
-You need a C++11-capable compiler for protozero to work. Copy the files in the
+You need a C++11-capable compiler for Protozero to work. Copy the files in the
`include/protozero` directory somewhere where your build system can find them.
Keep the `protozero` directory and include the files in the form
@@ -231,7 +231,7 @@ the `get_enum()` function to get the value of the enum, you have to translate
this into the symbolic name yourself. See the `enum` test case for an example.
-### Asserts and exceptions in the protozero library
+### Asserts and exceptions in the Protozero library
Protozero uses `assert()` liberally to help you find bugs in your own code when
compiled in debug mode (ie with `NDEBUG` not set). If such an assert "fires",
@@ -241,7 +241,7 @@ this is a very strong indication that there is a bug in your code somewhere.
own test code. This is done to make sure the asserts actually work as intended.
Your test code will not need this!)
-Exceptions, on the other hand, are thrown by protozero if some kind of data
+Exceptions, on the other hand, are thrown by Protozero if some kind of data
corruption was detected while it is trying to parse the data. This could also
be an indicator for a bug in the user code, but because it can happen if the
data was (intentionally or not intentionally) been messed with, it is reported
@@ -471,7 +471,7 @@ protozero::pbf_writer pw(data);
```
Of course you can add as many elements as you want. If you add no elements
-at all, this code will still work, protozero detects this special case and
+at all, this code will still work, Protozero detects this special case and
pretends you never even initialized this field.
The nested scope is important in this case, because the destructor of the
@@ -610,45 +610,3 @@ integers.
See the `test/t/complex` test case for a complete example using this interface.
-## Reserving memory
-
-If you know beforehand how large a message will become or can take an educated
-guess, you can call the usual `std::string::reserve()` on the underlying string
-before you give it to an `pbf_writer` or `pbf_builder` object.
-
-Or you can (at any time) call `reserve()` on the `pbf_writer` or `pbf_builder`.
-This will reserve the given amount of bytes in addition to whatever is already
-in that message.
-
-In the general case it is not easy to figure out how much memory you will need
-because of the varint packing of integers. But sometimes you can make at least
-a rough estimate. Still, you should probably only use this facility if you have
-benchmarks proving that it actually makes your program faster.
-
-## Using the Low-Level Varint and Zigzag Encoding and Decoding Functions
-
-Protozero gives you access to the low-level functions for encoding and
-decoding varint and zigzag integer encodings, because these functions can
-sometimes be useful outside the Protocol Buffer context.
-
-### Using Low-Level Functions
-
-To use the low-level, add this include to your C++ program:
-
- #include <protozero/varint.hpp>
-
-### Functions
-
-The following functions are then available:
-
-```cpp
-decode_varint()
-write_varint()
-encode_zigzag32()
-encode_zigzag64()
-decode_zigzag32()
-decode_zigzag64()
-```
-
-See the reference documentation created by `make doc` for details.
-
diff --git a/include/protozero/pbf_builder.hpp b/include/protozero/pbf_builder.hpp
index 39af53f..fef53e3 100644
--- a/include/protozero/pbf_builder.hpp
+++ b/include/protozero/pbf_builder.hpp
@@ -87,6 +87,11 @@ public:
pbf_writer::add_bytes(pbf_tag_type(tag), value);
}
+ template <typename... Ts>
+ void add_bytes_vectored(T tag, Ts&&... values) {
+ pbf_writer::add_bytes_vectored(pbf_tag_type(tag), std::forward<Ts>(values)...);
+ }
+
void add_string(T tag, const char* value, std::size_t size) {
pbf_writer::add_string(pbf_tag_type(tag), value, size);
}
diff --git a/include/protozero/pbf_reader.hpp b/include/protozero/pbf_reader.hpp
index 8754cab..ff4ccb9 100644
--- a/include/protozero/pbf_reader.hpp
+++ b/include/protozero/pbf_reader.hpp
@@ -321,6 +321,9 @@ public:
* }
* @endcode
*
+ * Note that this will not check the wire type. The two-argument version
+ * of this function will also check the wire type.
+ *
* @returns `true` if there is a next field with this tag.
* @pre There must be no current field.
* @post If it returns `true` there is a current field now with the given tag.
@@ -337,6 +340,45 @@ public:
}
/**
+ * Set next field with given tag and wire type in the message as the
+ * current field. Fields with other tags are skipped. This is usually
+ * called in a while loop for repeated fields:
+ *
+ * @code
+ * pbf_reader message(...);
+ * while (message.next(17, pbf_wire_type::varint)) {
+ * // handle field
+ * }
+ * @endcode
+ *
+ * or you can call it just once to get the one field with this tag:
+ *
+ * @code
+ * pbf_reader message(...);
+ * if (message.next(17, pbf_wire_type::varint)) {
+ * // handle field
+ * }
+ * @endcode
+ *
+ * Note that this will also check the wire type. The one-argument version
+ * of this function will not check the wire type.
+ *
+ * @returns `true` if there is a next field with this tag.
+ * @pre There must be no current field.
+ * @post If it returns `true` there is a current field now with the given tag.
+ */
+ bool next(pbf_tag_type next_tag, pbf_wire_type wire_type) {
+ while (next()) {
+ if (m_tag == next_tag && m_wire_type == wire_type) {
+ return true;
+ } else {
+ skip();
+ }
+ }
+ return false;
+ }
+
+ /**
* The tag of the current field. The tag is the field number from the
* description in the .proto file.
*
@@ -369,6 +411,32 @@ public:
}
/**
+ * Get the tag and wire type of the current field in one integer suitable
+ * for comparison with a switch statement.
+ *
+ * Use it like this:
+ *
+ * @code
+ * pbf_reader message(...);
+ * while (message.next()) {
+ * switch (message.tag_and_type()) {
+ * case tag_and_type(17, pbf_wire_type::length_delimited):
+ * ....
+ * break;
+ * case tag_and_type(21, pbf_wire_type::varint):
+ * ....
+ * break;
+ * default:
+ * message.skip();
+ * }
+ * }
+ * @endcode
+ */
+ uint32_t tag_and_type() const noexcept {
+ return protozero::tag_and_type(tag(), wire_type());
+ }
+
+ /**
* Check the wire type of the current field.
*
* @returns `true` if the current field has the given wire type.
diff --git a/include/protozero/pbf_writer.hpp b/include/protozero/pbf_writer.hpp
index 77cc9d0..39dd795 100644
--- a/include/protozero/pbf_writer.hpp
+++ b/include/protozero/pbf_writer.hpp
@@ -226,8 +226,7 @@ public:
*/
explicit pbf_writer(std::string& data) noexcept :
m_data(&data),
- m_parent_writer(nullptr),
- m_pos(0) {
+ m_parent_writer(nullptr) {
}
/**
@@ -236,8 +235,7 @@ public:
*/
pbf_writer() noexcept :
m_data(nullptr),
- m_parent_writer(nullptr),
- m_pos(0) {
+ m_parent_writer(nullptr) {
}
/**
@@ -252,8 +250,7 @@ public:
*/
pbf_writer(pbf_writer& parent_writer, pbf_tag_type tag, std::size_t size=0) :
m_data(parent_writer.m_data),
- m_parent_writer(&parent_writer),
- m_pos(0) {
+ m_parent_writer(&parent_writer) {
m_parent_writer->open_submessage(tag, size);
}
@@ -505,6 +502,37 @@ public:
}
/**
+ * Add "bytes" field to data using vectored input. All the data in the
+ * 2nd and further arguments is "concatenated" with only a single copy
+ * into the final buffer.
+ *
+ * This will work with objects of any type supporting the data() and
+ * size() methods like std::string or protozero::data_view.
+ *
+ * Example:
+ * @code
+ * std::string data1 = "abc";
+ * std::string data2 = "xyz";
+ * writer.add_bytes_vectored(1, data1, data2);
+ * @endcode
+ *
+ * @tparam Ts List of types supporting data() and size() methods.
+ * @param tag Tag (field number) of the field
+ * @param values List of objects of types Ts with data to be appended.
+ */
+ template <typename... Ts>
+ void add_bytes_vectored(pbf_tag_type tag, Ts&&... values) {
+ protozero_assert(m_pos == 0 && "you can't add fields to a parent pbf_writer if there is an existing pbf_writer for a submessage");
+ protozero_assert(m_data);
+ size_t sum_size = 0;
+ (void)std::initializer_list<size_t>{sum_size += values.size()...};
+ protozero_assert(sum_size <= std::numeric_limits<pbf_length_type>::max());
+ add_length_varint(tag, pbf_length_type(sum_size));
+ m_data->reserve(m_data->size() + sum_size);
+ (void)std::initializer_list<int>{(m_data->append(values.data(), values.size()), 0)...};
+ }
+
+ /**
* Add "string" field to data.
*
* @param tag Tag (field number) of the field
@@ -814,6 +842,12 @@ namespace detail {
public:
+ packed_field(const packed_field&) = delete;
+ packed_field& operator=(const packed_field&) = delete;
+
+ packed_field(packed_field&&) = default;
+ packed_field& operator=(packed_field&&) = default;
+
packed_field(pbf_writer& parent_writer, pbf_tag_type tag) :
m_writer(parent_writer, tag) {
}
@@ -833,12 +867,14 @@ namespace detail {
public:
- packed_field_fixed(pbf_writer& parent_writer, pbf_tag_type tag) :
- packed_field(parent_writer, tag) {
+ template <typename P>
+ packed_field_fixed(pbf_writer& parent_writer, P tag) :
+ packed_field(parent_writer, static_cast<pbf_tag_type>(tag)) {
}
- packed_field_fixed(pbf_writer& parent_writer, pbf_tag_type tag, std::size_t size) :
- packed_field(parent_writer, tag, size * sizeof(T)) {
+ template <typename P>
+ packed_field_fixed(pbf_writer& parent_writer, P tag, std::size_t size) :
+ packed_field(parent_writer, static_cast<pbf_tag_type>(tag), size * sizeof(T)) {
}
void add_element(T value) {
@@ -852,8 +888,9 @@ namespace detail {
public:
- packed_field_varint(pbf_writer& parent_writer, pbf_tag_type tag) :
- packed_field(parent_writer, tag) {
+ template <typename P>
+ packed_field_varint(pbf_writer& parent_writer, P tag) :
+ packed_field(parent_writer, static_cast<pbf_tag_type>(tag)) {
}
void add_element(T value) {
@@ -867,8 +904,9 @@ namespace detail {
public:
- packed_field_svarint(pbf_writer& parent_writer, pbf_tag_type tag) :
- packed_field(parent_writer, tag) {
+ template <typename P>
+ packed_field_svarint(pbf_writer& parent_writer, P tag) :
+ packed_field(parent_writer, static_cast<pbf_tag_type>(tag)) {
}
void add_element(T value) {
diff --git a/include/protozero/types.hpp b/include/protozero/types.hpp
index 431c207..5e14972 100644
--- a/include/protozero/types.hpp
+++ b/include/protozero/types.hpp
@@ -46,6 +46,17 @@ enum class pbf_wire_type : uint32_t {
};
/**
+ * Get the tag and wire type of the current field in one integer suitable
+ * for comparison with a switch statement.
+ *
+ * See pbf_reader.tag_and_type() for an example how to use this.
+ */
+template <typename T>
+constexpr inline uint32_t tag_and_type(T tag, pbf_wire_type wire_type) noexcept {
+ return (static_cast<uint32_t>(static_cast<pbf_tag_type>(tag)) << 3) | static_cast<uint32_t>(wire_type);
+}
+
+/**
* The type used for length values, such as the length of a field.
*/
using pbf_length_type = uint32_t;
@@ -166,7 +177,7 @@ inline void swap(data_view& lhs, data_view& rhs) noexcept {
* @param lhs First object.
* @param rhs Second object.
*/
-inline bool operator==(data_view& lhs, data_view& rhs) noexcept {
+inline bool operator==(const data_view& lhs, const data_view& rhs) noexcept {
return lhs.size() == rhs.size() && !std::strcmp(lhs.data(), rhs.data());
}
@@ -177,7 +188,7 @@ inline bool operator==(data_view& lhs, data_view& rhs) noexcept {
* @param lhs First object.
* @param rhs Second object.
*/
-inline bool operator!=(data_view& lhs, data_view& rhs) noexcept {
+inline bool operator!=(const data_view& lhs, const data_view& rhs) noexcept {
return !(lhs == rhs);
}
diff --git a/include/protozero/varint.hpp b/include/protozero/varint.hpp
index f6142d1..6d2a043 100644
--- a/include/protozero/varint.hpp
+++ b/include/protozero/varint.hpp
@@ -157,29 +157,29 @@ inline int write_varint(T data, uint64_t value) {
/**
* ZigZag encodes a 32 bit integer.
*/
-inline uint32_t encode_zigzag32(int32_t value) noexcept {
+inline constexpr uint32_t encode_zigzag32(int32_t value) noexcept {
return (static_cast<uint32_t>(value) << 1) ^ (static_cast<uint32_t>(value >> 31));
}
/**
* ZigZag encodes a 64 bit integer.
*/
-inline uint64_t encode_zigzag64(int64_t value) noexcept {
+inline constexpr uint64_t encode_zigzag64(int64_t value) noexcept {
return (static_cast<uint64_t>(value) << 1) ^ (static_cast<uint64_t>(value >> 63));
}
/**
* Decodes a 32 bit ZigZag-encoded integer.
*/
-inline int32_t decode_zigzag32(uint32_t value) noexcept {
- return int32_t(value >> 1) ^ -int32_t(value & 1);
+inline constexpr int32_t decode_zigzag32(uint32_t value) noexcept {
+ return static_cast<int32_t>(value >> 1) ^ -static_cast<int32_t>(value & 1);
}
/**
* Decodes a 64 bit ZigZag-encoded integer.
*/
-inline int64_t decode_zigzag64(uint64_t value) noexcept {
- return int64_t(value >> 1) ^ -int64_t(value & 1);
+inline constexpr int64_t decode_zigzag64(uint64_t value) noexcept {
+ return static_cast<int64_t>(value >> 1) ^ -static_cast<int64_t>(value & 1);
}
} // end namespace protozero
diff --git a/include/protozero/version.hpp b/include/protozero/version.hpp
index d7cea48..b5da95b 100644
--- a/include/protozero/version.hpp
+++ b/include/protozero/version.hpp
@@ -20,16 +20,16 @@ documentation.
#define PROTOZERO_VERSION_MAJOR 1
/// The minor version number
-#define PROTOZERO_VERSION_MINOR 4
+#define PROTOZERO_VERSION_MINOR 5
/// The patch number
-#define PROTOZERO_VERSION_PATCH 5
+#define PROTOZERO_VERSION_PATCH 0
/// The complete version number
#define PROTOZERO_VERSION_CODE (PROTOZERO_VERSION_MAJOR * 10000 + PROTOZERO_VERSION_MINOR * 100 + PROTOZERO_VERSION_PATCH)
/// Version number as string
-#define PROTOZERO_VERSION_STRING "1.4.5"
+#define PROTOZERO_VERSION_STRING "1.5.0"
#endif // PROTOZERO_VERSION_HPP
diff --git a/package.json b/package.json
index 1a81c51..38963cb 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "protozero",
- "version": "1.4.5",
+ "version": "1.5.0",
"description": "Minimalist protocol buffer decoder and encoder in C++",
"main": "include_dirs.js",
"repository" : {
diff --git a/test/include/catch.hpp b/test/include/catch.hpp
index 2e6fe8d..3d18ead 100644
--- a/test/include/catch.hpp
+++ b/test/include/catch.hpp
@@ -1,6 +1,6 @@
/*
- * Catch v1.5.8
- * Generated: 2016-10-26 12:07:30.938259
+ * Catch v1.5.9
+ * Generated: 2016-11-29 12:14:38.049276
* ----------------------------------------------------------
* This file has been merged from multiple headers. Please don't edit it directly
* Copyright (c) 2012 Two Blue Cubes Ltd. All rights reserved.
@@ -3428,6 +3428,7 @@ namespace Catch {
#include <streambuf>
#include <ostream>
#include <fstream>
+#include <memory>
namespace Catch {
@@ -3995,9 +3996,12 @@ namespace Clara {
inline void convertInto( std::string const& _source, std::string& _dest ) {
_dest = _source;
}
+ char toLowerCh(char c) {
+ return static_cast<char>( ::tolower( c ) );
+ }
inline void convertInto( std::string const& _source, bool& _dest ) {
std::string sourceLC = _source;
- std::transform( sourceLC.begin(), sourceLC.end(), sourceLC.begin(), ::tolower );
+ std::transform( sourceLC.begin(), sourceLC.end(), sourceLC.begin(), toLowerCh );
if( sourceLC == "y" || sourceLC == "1" || sourceLC == "true" || sourceLC == "yes" || sourceLC == "on" )
_dest = true;
else if( sourceLC == "n" || sourceLC == "0" || sourceLC == "false" || sourceLC == "no" || sourceLC == "off" )
@@ -7578,7 +7582,7 @@ namespace Catch {
return os;
}
- Version libraryVersion( 1, 5, 8, "", 0 );
+ Version libraryVersion( 1, 5, 9, "", 0 );
}
@@ -9166,6 +9170,7 @@ namespace Catch {
public:
XmlReporter( ReporterConfig const& _config )
: StreamingReporterBase( _config ),
+ m_xml(_config.stream()),
m_sectionDepth( 0 )
{
m_reporterPrefs.shouldRedirectStdOut = true;
@@ -9185,7 +9190,6 @@ namespace Catch {
virtual void testRunStarting( TestRunInfo const& testInfo ) CATCH_OVERRIDE {
StreamingReporterBase::testRunStarting( testInfo );
- m_xml.setStream( stream );
m_xml.startElement( "Catch" );
if( !m_config->name().empty() )
m_xml.writeAttribute( "name", m_config->name() );
diff --git a/test/t/bytes/test_cases.cpp b/test/t/bytes/test_cases.cpp
index b335b26..05d772d 100644
--- a/test/t/bytes/test_cases.cpp
+++ b/test/t/bytes/test_cases.cpp
@@ -92,3 +92,55 @@ TEST_CASE("write bytes field") {
}
+TEST_CASE("write bytes field using vectored approach") {
+
+ std::string buffer;
+ protozero::pbf_writer pw(buffer);
+
+ SECTION("using two strings") {
+ std::string d1{"foo"};
+ std::string d2{"bar"};
+
+ pw.add_bytes_vectored(1, d1, d2);
+ }
+
+ SECTION("using a string and a dataview") {
+ std::string d1{"foo"};
+ std::string d2{"bar"};
+ protozero::data_view dv{d2};
+
+ pw.add_bytes_vectored(1, d1, dv);
+ }
+
+ SECTION("using three strings") {
+ std::string d1{"foo"};
+ std::string d2{"ba"};
+ std::string d3{"r"};
+
+ pw.add_bytes_vectored(1, d1, d2, d3);
+ }
+
+ SECTION("with empty string") {
+ std::string d1{"foo"};
+ std::string d2{""};
+ std::string d3{"bar"};
+
+ pw.add_bytes_vectored(1, d1, d2, d3);
+ }
+
+ REQUIRE(buffer == load_data("bytes/data-string"));
+}
+
+TEST_CASE("write bytes field using vectored approach with builder") {
+ enum class foo : protozero::pbf_tag_type { bar = 1 };
+ std::string buffer;
+ protozero::pbf_builder<foo> pw(buffer);
+
+ std::string d1{"foo"};
+ std::string d2{"bar"};
+
+ pw.add_bytes_vectored(foo::bar, d1, d2);
+
+ REQUIRE(buffer == load_data("bytes/data-string"));
+}
+
diff --git a/test/t/repeated_packed_bool/test_cases.cpp b/test/t/repeated_packed_bool/test_cases.cpp
index 9ce2f79..dd9ccaa 100644
--- a/test/t/repeated_packed_bool/test_cases.cpp
+++ b/test/t/repeated_packed_bool/test_cases.cpp
@@ -119,3 +119,43 @@ TEST_CASE("write repeated packed bool field using packed_field_bool") {
}
+TEST_CASE("write repeated packed bool field using packed_field_bool with pbf_builder") {
+
+ enum class msg : protozero::pbf_tag_type {
+ f = 1
+ };
+
+ std::string buffer;
+ protozero::pbf_builder<msg> pw(buffer);
+
+ SECTION("empty - should do rollback") {
+ {
+ protozero::packed_field_bool field{pw, msg::f};
+ }
+
+ REQUIRE(buffer == load_data("repeated_packed_bool/data-empty"));
+ }
+
+ SECTION("one") {
+ {
+ protozero::packed_field_bool field{pw, msg::f};
+ field.add_element(true);
+ }
+
+ REQUIRE(buffer == load_data("repeated_packed_bool/data-one"));
+ }
+
+ SECTION("many") {
+ {
+ protozero::packed_field_bool field{pw, msg::f};
+ field.add_element(true);
+ field.add_element(true);
+ field.add_element(false);
+ field.add_element(true);
+ }
+
+ REQUIRE(buffer == load_data("repeated_packed_bool/data-many"));
+ }
+
+}
+
diff --git a/test/t/vector_tile/test_cases.cpp b/test/t/vector_tile/test_cases.cpp
index 1b31904..dc39c7c 100644
--- a/test/t/vector_tile/test_cases.cpp
+++ b/test/t/vector_tile/test_cases.cpp
@@ -53,7 +53,7 @@ TEST_CASE("reading vector tiles") {
REQUIRE(layer_names == expected_layer_names);
}
- SECTION("iterate over message using next(type)") {
+ SECTION("iterate over message using next(tag)") {
while (item.next(3)) { // repeated message Layer
protozero::pbf_reader layermsg{item.get_message()};
while (layermsg.next(1)) { // required string name
@@ -64,9 +64,21 @@ TEST_CASE("reading vector tiles") {
REQUIRE(layer_names == expected_layer_names);
}
- SECTION("iterate over features in road layer") {
+ SECTION("iterate over message using next(tag, type)") {
+ while (item.next(3, protozero::pbf_wire_type::length_delimited)) { // repeated message Layer
+ protozero::pbf_reader layermsg{item.get_message()};
+ while (layermsg.next(1, protozero::pbf_wire_type::length_delimited)) { // required string name
+ layer_names.push_back(layermsg.get_string());
+ }
+ }
+ REQUIRE(layer_names == expected_layer_names);
+ }
+
+ SECTION("iterate over features in road layer") {
int n=0;
+ int n_id = 0;
+ int n_geomtype = 0;
while (item.next(3)) { // repeated message Layer
protozero::pbf_reader layer{item.get_message()};
std::string name = get_name(layer);
@@ -80,12 +92,54 @@ TEST_CASE("reading vector tiles") {
const auto id = feature.get_uint64();
REQUIRE(id >= 1ULL);
REQUIRE(id <= 504ULL);
+ ++n_id;
break;
}
case 3: { // optional GeomType
const auto geom_type = feature.get_uint32();
REQUIRE(geom_type >= 1UL);
REQUIRE(geom_type <= 3UL);
+ ++n_geomtype;
+ break;
+ }
+ default:
+ feature.skip();
+ }
+ }
+ }
+ }
+ }
+
+ REQUIRE(n == 502);
+ REQUIRE(n_id == 502);
+ REQUIRE(n_geomtype == 502);
+ }
+
+ SECTION("iterate over features in road layer using tag_and_type") {
+ int n=0;
+ int n_id = 0;
+ int n_geomtype = 0;
+ while (item.next(3)) { // repeated message Layer
+ protozero::pbf_reader layer{item.get_message()};
+ std::string name = get_name(layer);
+ if (name == "road") {
+ while (layer.next(2)) { // repeated Feature
+ ++n;
+ protozero::pbf_reader feature{layer.get_message()};
+ while (feature.next()) {
+ switch (feature.tag_and_type()) {
+ case protozero::tag_and_type(1, protozero::pbf_wire_type::varint): { // optional uint64 id
+ const auto id = feature.get_uint64();
+ REQUIRE(id >= 1ULL);
+ REQUIRE(id <= 504ULL);
+ ++n_id;
+ break;
+ }
+ case protozero::tag_and_type(3, protozero::pbf_wire_type::varint): { // optional GeomType
+ const auto geom_type = feature.get_uint32();
+ REQUIRE(geom_type >= 1UL);
+ REQUIRE(geom_type <= 3UL);
+ ++n_geomtype;
break;
}
default:
@@ -97,6 +151,8 @@ TEST_CASE("reading vector tiles") {
}
REQUIRE(n == 502);
+ REQUIRE(n_id == 502);
+ REQUIRE(n_geomtype == 502);
}
}
diff --git a/test/t/zigzag/test_cases.cpp b/test/t/zigzag/test_cases.cpp
index e877493..1e333a4 100644
--- a/test/t/zigzag/test_cases.cpp
+++ b/test/t/zigzag/test_cases.cpp
@@ -1,11 +1,43 @@
#include <test.hpp>
-inline int32_t zz32(int32_t val) {
+static_assert(protozero::encode_zigzag32( 0L) == 0UL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag32(-1L) == 1UL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag32( 1L) == 2UL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag32(-2L) == 3UL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag32( 2L) == 4UL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag32( std::numeric_limits<int32_t>::max()) == 2 * static_cast<uint32_t>(std::numeric_limits<int32_t>::max()), "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag32(-std::numeric_limits<int32_t>::max()) == 2 * static_cast<uint32_t>(std::numeric_limits<int32_t>::max()) - 1, "test constexpr zigzag functions");
+
+static_assert(protozero::encode_zigzag64( 0LL) == 0ULL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag64(-1LL) == 1ULL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag64( 1LL) == 2ULL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag64(-2LL) == 3ULL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag64( 2LL) == 4ULL, "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag64( std::numeric_limits<int64_t>::max()) == 2 * static_cast<uint64_t>(std::numeric_limits<int64_t>::max()), "test constexpr zigzag functions");
+static_assert(protozero::encode_zigzag64(-std::numeric_limits<int64_t>::max()) == 2 * static_cast<uint64_t>(std::numeric_limits<int64_t>::max()) - 1, "test constexpr zigzag functions");
+
+static_assert(protozero::decode_zigzag32(0UL) == 0L, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag32(1UL) == -1L, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag32(2UL) == 1L, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag32(3UL) == -2L, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag32(4UL) == 2L, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag32(2 * static_cast<uint32_t>(std::numeric_limits<int32_t>::max()) ) == std::numeric_limits<int32_t>::max(), "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag32(2 * static_cast<uint32_t>(std::numeric_limits<int32_t>::max()) - 1) == -std::numeric_limits<int32_t>::max(), "test constexpr zigzag functions");
+
+static_assert(protozero::decode_zigzag64(0ULL) == 0LL, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag64(1ULL) == -1LL, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag64(2ULL) == 1LL, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag64(3ULL) == -2LL, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag64(4ULL) == 2LL, "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag64(2 * static_cast<uint64_t>(std::numeric_limits<int64_t>::max()) ) == std::numeric_limits<int64_t>::max(), "test constexpr zigzag functions");
+static_assert(protozero::decode_zigzag64(2 * static_cast<uint64_t>(std::numeric_limits<int64_t>::max()) - 1) == -std::numeric_limits<int64_t>::max(), "test constexpr zigzag functions");
+
+inline constexpr int32_t zz32(int32_t val) {
return protozero::decode_zigzag32(protozero::encode_zigzag32(val));
}
-inline int64_t zz64(int64_t val) {
+inline constexpr int64_t zz64(int64_t val) {
return protozero::decode_zigzag64(protozero::encode_zigzag64(val));
}
@@ -15,6 +47,8 @@ TEST_CASE("zigzag encode some 32 bit values") {
REQUIRE(protozero::encode_zigzag32( 1L) == 2UL);
REQUIRE(protozero::encode_zigzag32(-2L) == 3UL);
REQUIRE(protozero::encode_zigzag32( 2L) == 4UL);
+ REQUIRE(protozero::encode_zigzag32( std::numeric_limits<int32_t>::max()) == 2 * static_cast<uint32_t>(std::numeric_limits<int32_t>::max()));
+ REQUIRE(protozero::encode_zigzag32(-std::numeric_limits<int32_t>::max()) == 2 * static_cast<uint32_t>(std::numeric_limits<int32_t>::max()) - 1);
}
TEST_CASE("zigzag encode some 64 bit values") {
@@ -23,6 +57,8 @@ TEST_CASE("zigzag encode some 64 bit values") {
REQUIRE(protozero::encode_zigzag64( 1LL) == 2ULL);
REQUIRE(protozero::encode_zigzag64(-2LL) == 3ULL);
REQUIRE(protozero::encode_zigzag64( 2LL) == 4ULL);
+ REQUIRE(protozero::encode_zigzag64( std::numeric_limits<int64_t>::max()) == 2 * static_cast<uint64_t>(std::numeric_limits<int64_t>::max()));
+ REQUIRE(protozero::encode_zigzag64(-std::numeric_limits<int64_t>::max()) == 2 * static_cast<uint64_t>(std::numeric_limits<int64_t>::max()) - 1);
}
TEST_CASE("zigzag and back - 32bit") {
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-grass/protozero.git
More information about the Pkg-grass-devel
mailing list