* Update protobuf pregenerated files
* Update grpcio-tools to latest version
* Implement proto3 field presence
* Fix to_dict with None optional fields.
* Add test with optional enum
* Properly support optional enums
* Add tests for 64-bit ints and floats
* Support field presence for int64 types
* Fix oneof serialization with proto3 field presence (#292)
= Description
The serialization of a oneof message that contains a message with fields
with explicit presence was buggy.
For example:
```
message A {
oneof kind {
B b = 1;
C c = 2;
}
}
message B {}
message C {
optional bool z = 1;
}
```
Serializing `A(b=B())` would lead to this payload:
```
0A # tag1, length delimited
00 # length: 0
12 # tag2, length delimited
00 # length: 0
```
Which when deserialized, leads to the message `A(c=C())`.
= Explanation
The issue lies in the post_init method. All fields are introspected, and
if different from PLACEHOLDER, the message is marked as having been
"serialized_on_wire".
Then, when serializing `A(b=B())`, we go through each field of the
oneof:
- field 'b': this is the selected field from the group, so it is
serialized
- field 'c': marked as 'serialized_on_wire', so it is added as well.
= Fix
The issue is that support for explicit presence changed the default
value from PLACEHOLDER to None. This breaks the post_init method in that
case, which is relatively easy to fix: if a field is optional, and set
to None, this is considered as the default value (which it is).
This fix however has a side-effect: the group_current for this field (the
oneof trick for explicit presence) is no longer set. This changes the
behavior when serializing the message in JSON: as the value is the
default one (None), and the group is not set (which would force the
serialization of the field), so None fields are no longer serialized in
JSON. This break one test, and will be fixed in the next commit.
* fix: do not serialize None fields in JSON format
This is linked to the fix from the previous commit: after it, scalar
None fields were not included in the JSON format, but some were still
included.
This is all cleaned up: None fields are not added in JSON by default,
as they indicate the default value of fields with explicit presence.
However, if `include_default_values is set, they are included.
* Fix: use builtin annotation prefix
* Remove comment
Co-authored-by: roblabla <unfiltered@roblab.la>
Co-authored-by: Vincent Thiberville <vthib@pm.me>
* Revert "Fix compilation of fields named 'bytes' or 'str' (#226)"
This reverts commit deb623ed14cea65f0a0d17e9c770426d71198ae0.
* Fix compilation of fileds with name identical to their type
* Added test for field-name identical to python type
Co-authored-by: Guy Szweigman <guysz@nvidia.com>
* if you have a field named "bytes" using the bytes type, it doesn't work.
* Enable existing use-case & generalize solution to cover it
Co-authored-by: Spencer <spencer@sf-n.com>
- Enable oneof_enum test case that passes now (removed the xfail)
- Switch from toml to tomlkit as a dev dep for better toml support
- upgrade poethepoet to latest stable release
- use full table format for poe tasks to avoid long lines in pyproject.toml
- remove redundant _WrappedMessage class
- fix various Mypy warnings
- reformat some comments for consistent line length
- Added support for the custom double values from
the protobuf json spec: "Infinity", "-Infinity", and "NaN"
- Added `infinite_floats` test data
- Updated Message.__eq__ to consider nan values
equal
- Updated `test_message_json` and
`test_binary_compatibility` to replace NaN float
values in dictionaries before comparison
(because two NaN values are not equal)
This means the betterproto plugin no longer needs to depend durectly on
protobuf.
This requires a small runtime hack to monkey patch some google types to
get around the fact that the compiler uses proto2, but betterproto
expects proto3.
Also:
- regenerate google.protobuf package
- fix a regex bug in the logic for determining whether to use a google
wrapper type.
- fix a bug causing comments to get mixed up when multiple proto files
generate code into a single python module
- Remove plugin dependency on protobuf since it's no longer required.
- Update poethepoet to for better pyproject toml syntax support
- Add handy generate_lib poe task for maintaining generated libs
This means the betterproto plugin no longer needs to depend durectly on
protobuf.
This requires a small runtime hack to monkey patch some google types to
get around the fact that the compiler uses proto2, but betterproto
expects proto3.
Also:
- regenerate google.protobuf package
- fix a regex bug in the logic for determining whether to use a google
wrapper type.
- fix a bug causing comments to get mixed up when multiple proto files
generate code into a single python module
* Implement Message.__bool__ with similar semantics to a collection, such that any value being set on the message (i.e. having a non-default value) make the Message value truthy .
Co-authored-by: nat <n@natn.me>
Changes message initialization (`__post_init__`) so that default values
are no longer eagerly created to prevent infinite recursion when
initializing recursive messages.
As a result, `PLACEHOLDER` will be present in the message for any
uninitialized fields. So, an implementation of `__get_attribute__` is
added that checks for `PLACEHOLDER` and lazily creates and stores
default field values.
And, because `PLACEHOLDER` values don't compare equal with zero values,
a custom implementation of `__eq__` is provided, and the code generation
template is updated so that messages generate with `@dataclass(eq=False)`.
Also add new Message __repr__ implementation that skips PLACEHOLDER
values and orders keys by number from the proto.
Co-authored-by: Christopher Chambers <chris@peanutcode.com>
Co-authored-by: nat <n@natn.me>
Co-authored-by: James <50501825+Gobot1234@users.noreply.github.com>
* Serialize default values in oneofs when calling to_dict() or to_json()
This change is consistent with the official protobuf implementation. If
a default value is set when using a oneof, and then a message is
translated from message -> JSON -> message, the default value is kept in
tact. Also, if no default value is set, they remain null.
* Some cleanup + testing for nested messages with oneofs
* Cleanup oneof_enum test cases, they should be fixed
This _should_ address:
https://github.com/danielgtaylor/python-betterproto/issues/63
* Include default value oneof fields when serializing to bytes
This will cause oneof fields with default values to explicitly be sent
to clients. Note that does not mean that all fields are serialized and
sent to clients, just those that _could_ be null and are not.
* Remove assignment when populating a sub-message within a proto
Also, move setattr out one indentation level
* Properly transform proto with empty string in oneof to bytes
Also, updated tests to ensure that which_one_of picks up the set field
* Formatting betterproto/__init__.py
* Adding test cases demonstrating equivalent behaviour with google impl
* Removing a temporary file I made locally
* Adding some clarifying comments
* Fixing tests for python38
