Compare commits
157 Commits
Author | SHA1 | Date | |
---|---|---|---|
|
a157f05480 | ||
|
69dfe9cafc | ||
|
a8a082e4e7 | ||
|
e44de6da06 | ||
|
a5e0ef910f | ||
|
8f7af272cc | ||
|
bf9412e083 | ||
|
4630c1cc67 | ||
|
d3e4fbb311 | ||
|
58556e0eb6 | ||
|
a3f5f21738 | ||
|
0028cc384a | ||
|
034e2e7da0 | ||
|
ca16b6ed34 | ||
|
16d554db75 | ||
|
9ef5503728 | ||
|
c93351ef21 | ||
|
80bef7c94f | ||
|
804805f0f5 | ||
|
43c134d27c | ||
|
0cd9510b54 | ||
|
beafc812ff | ||
|
3d8c0cb713 | ||
|
c513853301 | ||
|
c1a76a5f5e | ||
|
2745953a8e | ||
|
b5dcac1250 | ||
|
cbd3437080 | ||
|
2585a07fcf | ||
|
6c29771f4c | ||
|
0ba0692dec | ||
|
42e197f985 | ||
|
459d12b24d | ||
|
cebf9176a3 | ||
|
8864f4fdbd | ||
|
03211604bc | ||
|
1d7ba850e9 | ||
|
b2651335ce | ||
|
5a591ef2a4 | ||
|
8d7d0efb9b | ||
|
b891d257f6 | ||
|
8bcb67b66f | ||
|
72d72b4603 | ||
|
3273ae4d2c | ||
|
6fe666473d | ||
|
0338fcba29 | ||
|
0f3ad25770 | ||
|
586e28d2dc | ||
|
a8d8159d27 | ||
|
3f519d4fb1 | ||
|
dedead048f | ||
|
87b3a4b86d | ||
|
f2e87192b0 | ||
|
98d00f0d21 | ||
|
bde6d06835 | ||
|
23dcbc2695 | ||
|
0af0cf4bfb | ||
|
eaa4f7f5d9 | ||
|
cdddb2f42a | ||
|
d21cd6e391 | ||
|
af7115429a | ||
|
0d9387abec | ||
|
f4ebcb0f65 | ||
|
81711d2427 | ||
|
e3135ce766 | ||
|
9532844929 | ||
|
0c5d1ff868 | ||
|
5fb4b4b7ff | ||
|
4f820b4a6a | ||
|
75a4c230da | ||
|
5c9a12e2f6 | ||
|
e1ccd540a9 | ||
|
4e78fe9579 | ||
|
50bb67bf5d | ||
|
1ecbf1a125 | ||
|
0814729c5a | ||
|
f7aa6150e2 | ||
|
159c30ddd8 | ||
|
c8229e53a7 | ||
|
3185c67098 | ||
|
52eea5ce4c | ||
|
4b6f55dce5 | ||
|
fdbe0205f1 | ||
|
09f821921f | ||
|
a757da1b29 | ||
|
e2d672a422 | ||
|
63f5191f02 | ||
|
87f4b34930 | ||
|
2c360a55f2 | ||
|
04dce524aa | ||
|
8edec81b11 | ||
|
32c8e77274 | ||
|
d9fa6d2dd3 | ||
|
c88edfd093 | ||
|
a46979c8a6 | ||
|
83e13aa606 | ||
|
3ca75dadd7 | ||
|
5d2f3a2cd9 | ||
|
65c1f366ef | ||
|
34c34bd15a | ||
|
fb54917f2c | ||
|
1a95a7988e | ||
|
76db2f153e | ||
|
8567892352 | ||
|
3105e952ea | ||
|
7c8d47de6d | ||
|
c00e2aef19 | ||
|
fdf3b2e764 | ||
|
f7c2fd1194 | ||
|
d8abb850f8 | ||
|
d7ba27de2b | ||
|
57523a9e7f | ||
|
e5e61c873c | ||
|
9fd1c058e6 | ||
|
d336153845 | ||
|
9a45ea9f16 | ||
|
bb7f5229fb | ||
|
f7769a19d1 | ||
|
28a288924f | ||
|
5c700618fd | ||
|
a914306f33 | ||
|
67422db6b9 | ||
|
061bf86a9c | ||
|
d31f90be6b | ||
|
919b0a6a7d | ||
|
7ecf3fe0e6 | ||
|
ff14948a4e | ||
|
cb00273257 | ||
|
973d68a154 | ||
|
ab9857b5fd | ||
|
2f658df666 | ||
|
b813d1cedb | ||
|
f5ce1b7108 | ||
|
62fc421d60 | ||
|
eeed1c0db7 | ||
|
2a3e1e1827 | ||
|
53ce1255d3 | ||
|
e8991339e9 | ||
|
4556d67503 | ||
|
f087c6c9bd | ||
|
eec24e4ee8 | ||
|
91111ab7d8 | ||
|
fcff3dff74 | ||
|
5c4969ff1c | ||
|
ed33a48d64 | ||
|
b354aeb692 | ||
|
6d9e3fc580 | ||
|
72de590651 | ||
|
3c70f21074 | ||
|
4b7d5d3de4 | ||
|
142e976c40 | ||
|
382fabb96c | ||
|
18598e77d4 | ||
|
6871053ab2 | ||
|
5bb6931df7 | ||
|
e8a9960b73 | ||
|
f25c66777a |
23
.github/CONTRIBUTING.md
vendored
Normal file
23
.github/CONTRIBUTING.md
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Contributing
|
||||
|
||||
There's lots to do, and we're working hard, so any help is welcome!
|
||||
|
||||
- :speech_balloon: Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!
|
||||
|
||||
What can you do?
|
||||
|
||||
- :+1: Vote on [issues](https://github.com/danielgtaylor/python-betterproto/issues).
|
||||
- :speech_balloon: Give feedback on [Pull Requests](https://github.com/danielgtaylor/python-betterproto/pulls) and [Issues](https://github.com/danielgtaylor/python-betterproto/issues):
|
||||
- Suggestions
|
||||
- Express approval
|
||||
- Raise concerns
|
||||
- :small_red_triangle: Create an issue:
|
||||
- File a bug (please check its not a duplicate)
|
||||
- Propose an enhancement
|
||||
- :white_check_mark: Create a PR:
|
||||
- [Creating a failing test-case](https://github.com/danielgtaylor/python-betterproto/blob/master/betterproto/tests/README.md) to make bug-fixing easier
|
||||
- Fix any of the open issues
|
||||
- [Good first issues](https://github.com/danielgtaylor/python-betterproto/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
|
||||
- [Issues with tests](https://github.com/danielgtaylor/python-betterproto/issues?q=is%3Aissue+is%3Aopen+label%3A%22has+test%22)
|
||||
- New bugfix or idea
|
||||
- If you'd like to discuss your idea first, join us on Slack!
|
115
.github/workflows/ci.yml
vendored
115
.github/workflows/ci.yml
vendored
@@ -1,72 +1,69 @@
|
||||
name: CI
|
||||
|
||||
on: [push, pull_request]
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
|
||||
jobs:
|
||||
|
||||
check-formatting:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
name: Consult black on python formatting
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v1
|
||||
- uses: actions/setup-python@v1
|
||||
with:
|
||||
python-version: 3.7
|
||||
- uses: dschep/install-pipenv-action@v1
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
pipenv install --dev --python ${pythonLocation}/python
|
||||
- name: Run black
|
||||
run: |
|
||||
pipenv run black . --check --diff --exclude tests/output_
|
||||
|
||||
run-tests:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
tests:
|
||||
name: ${{ matrix.os }} / ${{ matrix.python-version }}
|
||||
runs-on: ${{ matrix.os }}-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: [ '3.6', '3.7' ]
|
||||
|
||||
name: Python ${{ matrix.python-version }} test
|
||||
|
||||
os: [Ubuntu, MacOS, Windows]
|
||||
python-version: [3.6, 3.7, 3.8, 3.9]
|
||||
exclude:
|
||||
- os: Windows
|
||||
python-version: 3.6
|
||||
steps:
|
||||
- uses: actions/checkout@v1
|
||||
- uses: actions/setup-python@v1
|
||||
- uses: actions/checkout@v2
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
- uses: dschep/install-pipenv-action@v1
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
sudo apt install protobuf-compiler libprotobuf-dev
|
||||
pipenv install --dev --python ${pythonLocation}/python
|
||||
- name: Run tests
|
||||
run: |
|
||||
cp .env.default .env
|
||||
pipenv run pip install -e .
|
||||
pipenv run generate
|
||||
pipenv run test
|
||||
|
||||
build-release:
|
||||
runs-on: ubuntu-latest
|
||||
- name: Get full Python version
|
||||
id: full-python-version
|
||||
shell: bash
|
||||
run: echo ::set-output name=version::$(python -c "import sys; print('-'.join(str(v) for v in sys.version_info))")
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v1
|
||||
- uses: actions/setup-python@v1
|
||||
with:
|
||||
python-version: 3.7
|
||||
- uses: dschep/install-pipenv-action@v1
|
||||
- name: Install dependencies
|
||||
- name: Install poetry
|
||||
shell: bash
|
||||
run: |
|
||||
sudo apt install protobuf-compiler libprotobuf-dev
|
||||
pipenv install --dev --python ${pythonLocation}/python
|
||||
- name: Build package
|
||||
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
|
||||
run: pipenv run python setup.py sdist
|
||||
- name: Publish package
|
||||
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
|
||||
uses: pypa/gh-action-pypi-publish@v1.0.0a0
|
||||
python -m pip install poetry
|
||||
echo "$HOME/.poetry/bin" >> $GITHUB_PATH
|
||||
|
||||
- name: Configure poetry
|
||||
shell: bash
|
||||
run: poetry config virtualenvs.in-project true
|
||||
|
||||
- name: Set up cache
|
||||
uses: actions/cache@v2
|
||||
id: cache
|
||||
with:
|
||||
user: __token__
|
||||
password: ${{ secrets.pypi }}
|
||||
path: .venv
|
||||
key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
|
||||
|
||||
- name: Ensure cache is healthy
|
||||
if: steps.cache.outputs.cache-hit == 'true'
|
||||
shell: bash
|
||||
run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
|
||||
|
||||
- name: Install dependencies
|
||||
shell: bash
|
||||
run: |
|
||||
poetry run python -m pip install pip -U
|
||||
poetry install
|
||||
|
||||
- name: Generate code from proto files
|
||||
shell: bash
|
||||
run: poetry run python -m tests.generate -v
|
||||
|
||||
- name: Execute test suite
|
||||
shell: bash
|
||||
run: poetry run pytest tests/
|
||||
|
26
.github/workflows/code-quality.yml
vendored
Normal file
26
.github/workflows/code-quality.yml
vendored
Normal file
@@ -0,0 +1,26 @@
|
||||
name: Code Quality
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
|
||||
jobs:
|
||||
check-formatting:
|
||||
name: Check code/doc formatting
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- name: Run Black
|
||||
uses: lgeiger/black-action@master
|
||||
with:
|
||||
args: --check src/ tests/ benchmarks/
|
||||
|
||||
- name: Install rST dependcies
|
||||
run: python -m pip install doc8
|
||||
- name: Lint documentation for errors
|
||||
run: python -m doc8 docs --max-line-length 88 --ignore-path-errors "docs/migrating.rst;D001"
|
||||
# it has a table which is longer than 88 characters long
|
31
.github/workflows/release.yml
vendored
Normal file
31
.github/workflows/release.yml
vendored
Normal file
@@ -0,0 +1,31 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
tags:
|
||||
- '**'
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
|
||||
jobs:
|
||||
packaging:
|
||||
name: Distribution
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- name: Set up Python 3.8
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
- name: Install poetry
|
||||
run: python -m pip install poetry
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
- name: Publish package to PyPI
|
||||
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
|
||||
env:
|
||||
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.pypi }}
|
||||
run: poetry publish -n
|
11
.gitignore
vendored
11
.gitignore
vendored
@@ -1,12 +1,19 @@
|
||||
.coverage
|
||||
.DS_Store
|
||||
.env
|
||||
.vscode/settings.json
|
||||
.mypy_cache
|
||||
.pytest_cache
|
||||
.python-version
|
||||
build/
|
||||
betterproto/tests/output_*
|
||||
tests/output_*
|
||||
**/__pycache__
|
||||
dist
|
||||
**/*.egg-info
|
||||
output
|
||||
.idea
|
||||
.idea
|
||||
.DS_Store
|
||||
.tox
|
||||
.venv
|
||||
.asv
|
||||
venv
|
||||
|
17
.readthedocs.yml
Normal file
17
.readthedocs.yml
Normal file
@@ -0,0 +1,17 @@
|
||||
version: 2
|
||||
formats: []
|
||||
|
||||
build:
|
||||
image: latest
|
||||
|
||||
sphinx:
|
||||
configuration: docs/conf.py
|
||||
fail_on_warning: false
|
||||
|
||||
python:
|
||||
version: 3.7
|
||||
install:
|
||||
- method: pip
|
||||
path: .
|
||||
extra_requirements:
|
||||
- dev
|
47
CHANGELOG.md
47
CHANGELOG.md
@@ -5,6 +5,53 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
- Versions suffixed with `b*` are in `beta` and can be installed with `pip install --pre betterproto`.
|
||||
|
||||
## [2.0.0b2] - 2020-11-24
|
||||
|
||||
- Add support for deprecated message and fields [#126](https://github.com/danielgtaylor/python-betterproto/pull/126)
|
||||
- Add support for recursive messages [#130](https://github.com/danielgtaylor/python-betterproto/pull/130)
|
||||
- Add support for `bool(Message)` [#142](https://github.com/danielgtaylor/python-betterproto/pull/142)
|
||||
- Improve support for Python 3.9 [#140](https://github.com/danielgtaylor/python-betterproto/pull/140) [#173](https://github.com/danielgtaylor/python-betterproto/pull/173)
|
||||
- Improve keyword sanitisation for generated code [#137](https://github.com/danielgtaylor/python-betterproto/pull/137)
|
||||
|
||||
- Fix missing serialized_on_wire when message contains only lists [#81](https://github.com/danielgtaylor/python-betterproto/pull/81)
|
||||
- Fix circular dependencies [#100](https://github.com/danielgtaylor/python-betterproto/pull/100)
|
||||
- Fix to_dict enum fields when numbering is not consecutive [#102](https://github.com/danielgtaylor/python-betterproto/pull/102)
|
||||
- Fix argument generation for stub methods when using `import` with proto definition [#103](https://github.com/danielgtaylor/python-betterproto/pull/103)
|
||||
- Fix missing async/await keywords when casing [#104](https://github.com/danielgtaylor/python-betterproto/pull/104)
|
||||
- Fix mutable default arguments in generated code [#105](https://github.com/danielgtaylor/python-betterproto/pull/105)
|
||||
- Fix serialisation of default values in oneofs when calling to_dict() or to_json() [#110](https://github.com/danielgtaylor/python-betterproto/pull/110)
|
||||
- Fix static type checking for grpclib client [#124](https://github.com/danielgtaylor/python-betterproto/pull/124)
|
||||
- Fix python3.6 compatibility issue with dataclasses [#124](https://github.com/danielgtaylor/python-betterproto/pull/124)
|
||||
- Fix handling of trailer-only responses [#127](https://github.com/danielgtaylor/python-betterproto/pull/127)
|
||||
|
||||
- Refactor plugin.py to use modular dataclasses in tree-like structure to represent parsed data [#121](https://github.com/danielgtaylor/python-betterproto/pull/121)
|
||||
- Refactor template compilation logic [#136](https://github.com/danielgtaylor/python-betterproto/pull/136)
|
||||
|
||||
- Replace use of platform provided protoc with development dependency on grpcio-tools [#107](https://github.com/danielgtaylor/python-betterproto/pull/107)
|
||||
- Switch to using `poe` from `make` to manage project development tasks [#118](https://github.com/danielgtaylor/python-betterproto/pull/118)
|
||||
- Improve CI platform coverage [#128](https://github.com/danielgtaylor/python-betterproto/pull/128)
|
||||
|
||||
## [2.0.0b1] - 2020-07-04
|
||||
|
||||
[Upgrade Guide](./docs/upgrading.md)
|
||||
|
||||
> Several bugfixes and improvements required or will require small breaking changes, necessitating a new version.
|
||||
> `2.0.0` will be released once the interface is stable.
|
||||
|
||||
- Add support for gRPC and **stream-stream** [#83](https://github.com/danielgtaylor/python-betterproto/pull/83)
|
||||
- Switch from `pipenv` to `poetry` for development [#75](https://github.com/danielgtaylor/python-betterproto/pull/75)
|
||||
- Fix two packages with the same name suffix should not cause naming conflict [#25](https://github.com/danielgtaylor/python-betterproto/issues/25)
|
||||
|
||||
- Fix Import child package from root [#57](https://github.com/danielgtaylor/python-betterproto/issues/57)
|
||||
- Fix Import child package from package [#58](https://github.com/danielgtaylor/python-betterproto/issues/58)
|
||||
- Fix Import parent package from child package [#59](https://github.com/danielgtaylor/python-betterproto/issues/59)
|
||||
- Fix Import root package from child package [#60](https://github.com/danielgtaylor/python-betterproto/issues/60)
|
||||
- Fix Import root package from root [#61](https://github.com/danielgtaylor/python-betterproto/issues/61)
|
||||
|
||||
- Fix ALL_CAPS message fields are parsed incorrectly. [#11](https://github.com/danielgtaylor/python-betterproto/issues/11)
|
||||
|
||||
## [1.2.5] - 2020-04-27
|
||||
|
||||
- Add .j2 suffix to python template names to avoid confusing certain build tools [#72](https://github.com/danielgtaylor/python-betterproto/pull/72)
|
||||
|
32
Pipfile
32
Pipfile
@@ -1,32 +0,0 @@
|
||||
[[source]]
|
||||
name = "pypi"
|
||||
url = "https://pypi.org/simple"
|
||||
verify_ssl = true
|
||||
|
||||
[dev-packages]
|
||||
flake8 = "*"
|
||||
mypy = "*"
|
||||
isort = "*"
|
||||
pytest = "*"
|
||||
pytest-asyncio = "*"
|
||||
rope = "*"
|
||||
|
||||
[packages]
|
||||
protobuf = "*"
|
||||
jinja2 = "*"
|
||||
grpclib = "*"
|
||||
stringcase = "*"
|
||||
black = "*"
|
||||
backports-datetime-fromisoformat = "*"
|
||||
dataclasses = "*"
|
||||
|
||||
[requires]
|
||||
python_version = "3.6"
|
||||
|
||||
[scripts]
|
||||
plugin = "protoc --plugin=protoc-gen-custom=betterproto/plugin.py --custom_out=output"
|
||||
generate = "python betterproto/tests/generate.py"
|
||||
test = "pytest ./betterproto/tests"
|
||||
|
||||
[pipenv]
|
||||
allow_prereleases = true
|
444
Pipfile.lock
generated
444
Pipfile.lock
generated
@@ -1,444 +0,0 @@
|
||||
{
|
||||
"_meta": {
|
||||
"hash": {
|
||||
"sha256": "44ae793965dc2b6ec17f0435a388846248b8a703cf857470b66af84227535950"
|
||||
},
|
||||
"pipfile-spec": 6,
|
||||
"requires": {
|
||||
"python_version": "3.6"
|
||||
},
|
||||
"sources": [
|
||||
{
|
||||
"name": "pypi",
|
||||
"url": "https://pypi.org/simple",
|
||||
"verify_ssl": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"default": {
|
||||
"appdirs": {
|
||||
"hashes": [
|
||||
"sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41",
|
||||
"sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"
|
||||
],
|
||||
"version": "==1.4.4"
|
||||
},
|
||||
"attrs": {
|
||||
"hashes": [
|
||||
"sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
|
||||
"sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
|
||||
],
|
||||
"version": "==19.3.0"
|
||||
},
|
||||
"backports-datetime-fromisoformat": {
|
||||
"hashes": [
|
||||
"sha256:9577a2a9486cd7383a5f58b23bb8e81cf0821dbbc0eb7c87d3fa198c1df40f5c"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==1.0.0"
|
||||
},
|
||||
"black": {
|
||||
"hashes": [
|
||||
"sha256:1b30e59be925fafc1ee4565e5e08abef6b03fe455102883820fe5ee2e4734e0b",
|
||||
"sha256:c2edb73a08e9e0e6f65a0e6af18b059b8b1cdd5bef997d7a0b181df93dc81539"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==19.10b0"
|
||||
},
|
||||
"click": {
|
||||
"hashes": [
|
||||
"sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
|
||||
"sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
|
||||
],
|
||||
"version": "==7.1.2"
|
||||
},
|
||||
"dataclasses": {
|
||||
"hashes": [
|
||||
"sha256:454a69d788c7fda44efd71e259be79577822f5e3f53f029a22d08004e951dc9f",
|
||||
"sha256:6988bd2b895eef432d562370bb707d540f32f7360ab13da45340101bc2307d84"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==0.6"
|
||||
},
|
||||
"grpclib": {
|
||||
"hashes": [
|
||||
"sha256:b27d56c987b89023d5640fe9668943e49b46703fc85d8182a58c9f3b19120cdc"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==0.3.2rc1"
|
||||
},
|
||||
"h2": {
|
||||
"hashes": [
|
||||
"sha256:61e0f6601fa709f35cdb730863b4e5ec7ad449792add80d1410d4174ed139af5",
|
||||
"sha256:875f41ebd6f2c44781259005b157faed1a5031df3ae5aa7bcb4628a6c0782f14"
|
||||
],
|
||||
"version": "==3.2.0"
|
||||
},
|
||||
"hpack": {
|
||||
"hashes": [
|
||||
"sha256:0edd79eda27a53ba5be2dfabf3b15780928a0dff6eb0c60a3d6767720e970c89",
|
||||
"sha256:8eec9c1f4bfae3408a3f30500261f7e6a65912dc138526ea054f9ad98892e9d2"
|
||||
],
|
||||
"version": "==3.0.0"
|
||||
},
|
||||
"hyperframe": {
|
||||
"hashes": [
|
||||
"sha256:5187962cb16dcc078f23cb5a4b110098d546c3f41ff2d4038a9896893bbd0b40",
|
||||
"sha256:a9f5c17f2cc3c719b917c4f33ed1c61bd1f8dfac4b1bd23b7c80b3400971b41f"
|
||||
],
|
||||
"version": "==5.2.0"
|
||||
},
|
||||
"jinja2": {
|
||||
"hashes": [
|
||||
"sha256:c10142f819c2d22bdcd17548c46fa9b77cf4fda45097854c689666bf425e7484",
|
||||
"sha256:c922560ac46888d47384de1dbdc3daaa2ea993af4b26a436dec31fa2c19ec668"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==3.0.0a1"
|
||||
},
|
||||
"markupsafe": {
|
||||
"hashes": [
|
||||
"sha256:06358015a4dee8ee23ae426bf885616ab3963622defd829eb45b44e3dee3515f",
|
||||
"sha256:0b0c4fc852c5f02c6277ef3b33d23fcbe89b1b227460423e3335374da046b6db",
|
||||
"sha256:267677fc42afed5094fc5ea1c4236bbe4b6a00fe4b08e93451e65ae9048139c7",
|
||||
"sha256:303cb70893e2c345588fb5d5b86e0ca369f9bb56942f03064c5e3e75fa7a238a",
|
||||
"sha256:3c9b624a0d9ed5a5093ac4edc4e823e6b125441e60ef35d36e6f4a6fdacd5054",
|
||||
"sha256:42033e14cae1f6c86fc0c3e90d04d08ce73ac8e46ba420a0d22d545c2abd4977",
|
||||
"sha256:4e4a99b6af7bdc0856b50020c095848ec050356a001e1f751510aef6ab14d0e0",
|
||||
"sha256:4eb07faad54bb07427d848f31030a65a49ebb0cec0b30674f91cf1ddd456bfe4",
|
||||
"sha256:63a7161cd8c2bc563feeda45df62f42c860dd0675e2b8da2667f25bb3c95eaba",
|
||||
"sha256:68e0fd039b68d2945b4beb947d4023ca7f8e95b708031c345762efba214ea761",
|
||||
"sha256:8092a63397025c2f655acd42784b2a1528339b90b987beb9253f22e8cdbb36c3",
|
||||
"sha256:841218860683c0f2223e24756843d84cc49cccdae6765e04962607754a52d3e0",
|
||||
"sha256:94076b2314bd2f6cfae508ad65b4d493e3a58a50112b7a2cbb6287bdbc404ae8",
|
||||
"sha256:9d22aff1c5322e402adfb3ce40839a5056c353e711c033798cf4f02eb9f5124d",
|
||||
"sha256:b0e4584f62b3e5f5c1a7bcefd2b52f236505e6ef032cc508caa4f4c8dc8d3af1",
|
||||
"sha256:b1163ffc1384d242964426a8164da12dbcdbc0de18ea36e2c34b898ed38c3b45",
|
||||
"sha256:beac28ed60c8e838301226a7a85841d0af2068eba2dcb1a58c2d32d6c05e440e",
|
||||
"sha256:c29f096ce79c03054a1101d6e5fe6bf04b0bb489165d5e0e9653fb4fe8048ee1",
|
||||
"sha256:c58779966d53e5f14ba393d64e2402a7926601d1ac8adeb4e83893def79d0428",
|
||||
"sha256:cfe14b37908eaf7d5506302987228bff69e1b8e7071ccd4e70fd0283b1b47f0b",
|
||||
"sha256:e834249c45aa9837d0753351cdca61a4b8b383cc9ad0ff2325c97ff7b69e72a6",
|
||||
"sha256:eed1b234c4499811ee85bcefa22ef5e466e75d132502226ed29740d593316c1f"
|
||||
],
|
||||
"version": "==2.0.0a1"
|
||||
},
|
||||
"multidict": {
|
||||
"hashes": [
|
||||
"sha256:1ece5a3369835c20ed57adadc663400b5525904e53bae59ec854a5d36b39b21a",
|
||||
"sha256:275ca32383bc5d1894b6975bb4ca6a7ff16ab76fa622967625baeebcf8079000",
|
||||
"sha256:3750f2205b800aac4bb03b5ae48025a64e474d2c6cc79547988ba1d4122a09e2",
|
||||
"sha256:4538273208e7294b2659b1602490f4ed3ab1c8cf9dbdd817e0e9db8e64be2507",
|
||||
"sha256:5141c13374e6b25fe6bf092052ab55c0c03d21bd66c94a0e3ae371d3e4d865a5",
|
||||
"sha256:51a4d210404ac61d32dada00a50ea7ba412e6ea945bbe992e4d7a595276d2ec7",
|
||||
"sha256:5cf311a0f5ef80fe73e4f4c0f0998ec08f954a6ec72b746f3c179e37de1d210d",
|
||||
"sha256:6513728873f4326999429a8b00fc7ceddb2509b01d5fd3f3be7881a257b8d463",
|
||||
"sha256:7388d2ef3c55a8ba80da62ecfafa06a1c097c18032a501ffd4cabbc52d7f2b19",
|
||||
"sha256:9456e90649005ad40558f4cf51dbb842e32807df75146c6d940b6f5abb4a78f3",
|
||||
"sha256:c026fe9a05130e44157b98fea3ab12969e5b60691a276150db9eda71710cd10b",
|
||||
"sha256:d14842362ed4cf63751648e7672f7174c9818459d169231d03c56e84daf90b7c",
|
||||
"sha256:e0d072ae0f2a179c375f67e3da300b47e1a83293c554450b29c900e50afaae87",
|
||||
"sha256:f07acae137b71af3bb548bd8da720956a3bc9f9a0b87733e0899226a2317aeb7",
|
||||
"sha256:fbb77a75e529021e7c4a8d4e823d88ef4d23674a202be4f5addffc72cbb91430",
|
||||
"sha256:fcfbb44c59af3f8ea984de67ec7c306f618a3ec771c2843804069917a8f2e255",
|
||||
"sha256:feed85993dbdb1dbc29102f50bca65bdc68f2c0c8d352468c25b54874f23c39d"
|
||||
],
|
||||
"version": "==4.7.6"
|
||||
},
|
||||
"pathspec": {
|
||||
"hashes": [
|
||||
"sha256:7d91249d21749788d07a2d0f94147accd8f845507400749ea19c1ec9054a12b0",
|
||||
"sha256:da45173eb3a6f2a5a487efba21f050af2b41948be6ab52b6a1e3ff22bb8b7061"
|
||||
],
|
||||
"version": "==0.8.0"
|
||||
},
|
||||
"protobuf": {
|
||||
"hashes": [
|
||||
"sha256:04d0b2bd99050d09393875a5a25fd12337b17f3ac2e29c0c1b8e65b277cbfe72",
|
||||
"sha256:05288e44638e91498f13127a3699a6528dec6f9d3084d60959d721bfb9ea5b98",
|
||||
"sha256:175d85370947f89e33b3da93f4ccdda3f326bebe3e599df5915ceb7f804cd9df",
|
||||
"sha256:440a8c77531b3652f24999b249256ed01fd44c498ab0973843066681bd276685",
|
||||
"sha256:49fb6fab19cd3f30fa0e976eeedcbf2558e9061e5fa65b4fe51ded1f4002e04d",
|
||||
"sha256:4c7cae1f56056a4a2a2e3b00b26ab8550eae738bd9548f4ea0c2fcb88ed76ae5",
|
||||
"sha256:519abfacbb421c3591d26e8daf7a4957763428db7267f7207e3693e29f6978db",
|
||||
"sha256:60f32af25620abc4d7928d8197f9f25d49d558c5959aa1e08c686f974ac0b71a",
|
||||
"sha256:613ac49f6db266fba243daf60fb32af107cfe3678e5c003bb40a381b6786389d",
|
||||
"sha256:954bb14816edd24e746ba1a6b2d48c43576393bbde2fb8e1e3bd6d4504c7feac",
|
||||
"sha256:9b1462c033a2cee7f4e8eb396905c69de2c532c3b835ff8f71f8e5fb77c38023",
|
||||
"sha256:c0767f4d93ce4288475afe0571663c78870924f1f8881efd5406c10f070c75e4",
|
||||
"sha256:c45f5980ce32879391144b5766120fd7b8803129f127ce36bd060dd38824801f",
|
||||
"sha256:eeb7502f59e889a88bcb59f299493e215d1864f3d75335ea04a413004eb4fe24",
|
||||
"sha256:fdb1742f883ee4662e39fcc5916f2725fec36a5191a52123fec60f8c53b70495",
|
||||
"sha256:fe554066c4962c2db0a1d4752655223eb948d2bfa0fb1c4a7f2c00ec07324f1c"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==3.12.1"
|
||||
},
|
||||
"regex": {
|
||||
"hashes": [
|
||||
"sha256:1386e75c9d1574f6aa2e4eb5355374c8e55f9aac97e224a8a5a6abded0f9c927",
|
||||
"sha256:27ff7325b297fb6e5ebb70d10437592433601c423f5acf86e5bc1ee2919b9561",
|
||||
"sha256:329ba35d711e3428db6b45a53b1b13a0a8ba07cbbcf10bbed291a7da45f106c3",
|
||||
"sha256:3a9394197664e35566242686d84dfd264c07b20f93514e2e09d3c2b3ffdf78fe",
|
||||
"sha256:51f17abbe973c7673a61863516bdc9c0ef467407a940f39501e786a07406699c",
|
||||
"sha256:579ea215c81d18da550b62ff97ee187b99f1b135fd894a13451e00986a080cad",
|
||||
"sha256:70c14743320a68c5dac7fc5a0f685be63bc2024b062fe2aaccc4acc3d01b14a1",
|
||||
"sha256:7e61be8a2900897803c293247ef87366d5df86bf701083b6c43119c7c6c99108",
|
||||
"sha256:8044d1c085d49673aadb3d7dc20ef5cb5b030c7a4fa253a593dda2eab3059929",
|
||||
"sha256:89d76ce33d3266173f5be80bd4efcbd5196cafc34100fdab814f9b228dee0fa4",
|
||||
"sha256:99568f00f7bf820c620f01721485cad230f3fb28f57d8fbf4a7967ec2e446994",
|
||||
"sha256:a7c37f048ec3920783abab99f8f4036561a174f1314302ccfa4e9ad31cb00eb4",
|
||||
"sha256:c2062c7d470751b648f1cacc3f54460aebfc261285f14bc6da49c6943bd48bdd",
|
||||
"sha256:c9bce6e006fbe771a02bda468ec40ffccbf954803b470a0345ad39c603402577",
|
||||
"sha256:ce367d21f33e23a84fb83a641b3834dd7dd8e9318ad8ff677fbfae5915a239f7",
|
||||
"sha256:ce450ffbfec93821ab1fea94779a8440e10cf63819be6e176eb1973a6017aff5",
|
||||
"sha256:ce5cc53aa9fbbf6712e92c7cf268274eaff30f6bd12a0754e8133d85a8fb0f5f",
|
||||
"sha256:d466967ac8e45244b9dfe302bbe5e3337f8dc4dec8d7d10f5e950d83b140d33a",
|
||||
"sha256:d881c2e657c51d89f02ae4c21d9adbef76b8325fe4d5cf0e9ad62f850f3a98fd",
|
||||
"sha256:e565569fc28e3ba3e475ec344d87ed3cd8ba2d575335359749298a0899fe122e",
|
||||
"sha256:ea55b80eb0d1c3f1d8d784264a6764f931e172480a2f1868f2536444c5f01e01"
|
||||
],
|
||||
"version": "==2020.5.14"
|
||||
},
|
||||
"six": {
|
||||
"hashes": [
|
||||
"sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
|
||||
"sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
|
||||
],
|
||||
"version": "==1.15.0"
|
||||
},
|
||||
"stringcase": {
|
||||
"hashes": [
|
||||
"sha256:48a06980661908efe8d9d34eab2b6c13aefa2163b3ced26972902e3bdfd87008"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==1.2.0"
|
||||
},
|
||||
"toml": {
|
||||
"hashes": [
|
||||
"sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f",
|
||||
"sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88"
|
||||
],
|
||||
"version": "==0.10.1"
|
||||
},
|
||||
"typed-ast": {
|
||||
"hashes": [
|
||||
"sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355",
|
||||
"sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919",
|
||||
"sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa",
|
||||
"sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652",
|
||||
"sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75",
|
||||
"sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01",
|
||||
"sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d",
|
||||
"sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1",
|
||||
"sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907",
|
||||
"sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c",
|
||||
"sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3",
|
||||
"sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b",
|
||||
"sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614",
|
||||
"sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb",
|
||||
"sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b",
|
||||
"sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41",
|
||||
"sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6",
|
||||
"sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34",
|
||||
"sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe",
|
||||
"sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4",
|
||||
"sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7"
|
||||
],
|
||||
"version": "==1.4.1"
|
||||
}
|
||||
},
|
||||
"develop": {
|
||||
"attrs": {
|
||||
"hashes": [
|
||||
"sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
|
||||
"sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
|
||||
],
|
||||
"version": "==19.3.0"
|
||||
},
|
||||
"flake8": {
|
||||
"hashes": [
|
||||
"sha256:c69ac1668e434d37a2d2880b3ca9aafd54b3a10a3ac1ab101d22f29e29cf8634",
|
||||
"sha256:ccaa799ef9893cebe69fdfefed76865aeaefbb94cb8545617b2298786a4de9a5"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==3.8.2"
|
||||
},
|
||||
"importlib-metadata": {
|
||||
"hashes": [
|
||||
"sha256:2a688cbaa90e0cc587f1df48bdc97a6eadccdcd9c35fb3f976a09e3b5016d90f",
|
||||
"sha256:34513a8a0c4962bc66d35b359558fd8a5e10cd472d37aec5f66858addef32c1e"
|
||||
],
|
||||
"markers": "python_version < '3.8'",
|
||||
"version": "==1.6.0"
|
||||
},
|
||||
"isort": {
|
||||
"hashes": [
|
||||
"sha256:54da7e92468955c4fceacd0c86bd0ec997b0e1ee80d97f67c35a78b719dccab1",
|
||||
"sha256:6e811fcb295968434526407adb8796944f1988c5b65e8139058f2014cbe100fd"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==4.3.21"
|
||||
},
|
||||
"mccabe": {
|
||||
"hashes": [
|
||||
"sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42",
|
||||
"sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"
|
||||
],
|
||||
"version": "==0.6.1"
|
||||
},
|
||||
"more-itertools": {
|
||||
"hashes": [
|
||||
"sha256:558bb897a2232f5e4f8e2399089e35aecb746e1f9191b6584a151647e89267be",
|
||||
"sha256:7818f596b1e87be009031c7653d01acc46ed422e6656b394b0f765ce66ed4982"
|
||||
],
|
||||
"version": "==8.3.0"
|
||||
},
|
||||
"mypy": {
|
||||
"hashes": [
|
||||
"sha256:15b948e1302682e3682f11f50208b726a246ab4e6c1b39f9264a8796bb416aa2",
|
||||
"sha256:219a3116ecd015f8dca7b5d2c366c973509dfb9a8fc97ef044a36e3da66144a1",
|
||||
"sha256:3b1fc683fb204c6b4403a1ef23f0b1fac8e4477091585e0c8c54cbdf7d7bb164",
|
||||
"sha256:3beff56b453b6ef94ecb2996bea101a08f1f8a9771d3cbf4988a61e4d9973761",
|
||||
"sha256:7687f6455ec3ed7649d1ae574136835a4272b65b3ddcf01ab8704ac65616c5ce",
|
||||
"sha256:7ec45a70d40ede1ec7ad7f95b3c94c9cf4c186a32f6bacb1795b60abd2f9ef27",
|
||||
"sha256:86c857510a9b7c3104cf4cde1568f4921762c8f9842e987bc03ed4f160925754",
|
||||
"sha256:8a627507ef9b307b46a1fea9513d5c98680ba09591253082b4c48697ba05a4ae",
|
||||
"sha256:8dfb69fbf9f3aeed18afffb15e319ca7f8da9642336348ddd6cab2713ddcf8f9",
|
||||
"sha256:a34b577cdf6313bf24755f7a0e3f3c326d5c1f4fe7422d1d06498eb25ad0c600",
|
||||
"sha256:a8ffcd53cb5dfc131850851cc09f1c44689c2812d0beb954d8138d4f5fc17f65",
|
||||
"sha256:b90928f2d9eb2f33162405f32dde9f6dcead63a0971ca8a1b50eb4ca3e35ceb8",
|
||||
"sha256:c56ffe22faa2e51054c5f7a3bc70a370939c2ed4de308c690e7949230c995913",
|
||||
"sha256:f91c7ae919bbc3f96cd5e5b2e786b2b108343d1d7972ea130f7de27fdd547cf3"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==0.770"
|
||||
},
|
||||
"mypy-extensions": {
|
||||
"hashes": [
|
||||
"sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d",
|
||||
"sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"
|
||||
],
|
||||
"version": "==0.4.3"
|
||||
},
|
||||
"packaging": {
|
||||
"hashes": [
|
||||
"sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8",
|
||||
"sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181"
|
||||
],
|
||||
"version": "==20.4"
|
||||
},
|
||||
"pluggy": {
|
||||
"hashes": [
|
||||
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
|
||||
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
|
||||
],
|
||||
"version": "==0.13.1"
|
||||
},
|
||||
"py": {
|
||||
"hashes": [
|
||||
"sha256:5e27081401262157467ad6e7f851b7aa402c5852dbcb3dae06768434de5752aa",
|
||||
"sha256:c20fdd83a5dbc0af9efd622bee9a5564e278f6380fffcacc43ba6f43db2813b0"
|
||||
],
|
||||
"version": "==1.8.1"
|
||||
},
|
||||
"pycodestyle": {
|
||||
"hashes": [
|
||||
"sha256:2295e7b2f6b5bd100585ebcb1f616591b652db8a741695b3d8f5d28bdc934367",
|
||||
"sha256:c58a7d2815e0e8d7972bf1803331fb0152f867bd89adf8a01dfd55085434192e"
|
||||
],
|
||||
"version": "==2.6.0"
|
||||
},
|
||||
"pyflakes": {
|
||||
"hashes": [
|
||||
"sha256:0d94e0e05a19e57a99444b6ddcf9a6eb2e5c68d3ca1e98e90707af8152c90a92",
|
||||
"sha256:35b2d75ee967ea93b55750aa9edbbf72813e06a66ba54438df2cfac9e3c27fc8"
|
||||
],
|
||||
"version": "==2.2.0"
|
||||
},
|
||||
"pyparsing": {
|
||||
"hashes": [
|
||||
"sha256:67199f0c41a9c702154efb0e7a8cc08accf830eb003b4d9fa42c4059002e2492",
|
||||
"sha256:700d17888d441604b0bd51535908dcb297561b040819cccde647a92439db5a2a"
|
||||
],
|
||||
"version": "==3.0.0a1"
|
||||
},
|
||||
"pytest": {
|
||||
"hashes": [
|
||||
"sha256:95c710d0a72d91c13fae35dce195633c929c3792f54125919847fdcdf7caa0d3",
|
||||
"sha256:eb2b5e935f6a019317e455b6da83dd8650ac9ffd2ee73a7b657a30873d67a698"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==5.4.2"
|
||||
},
|
||||
"pytest-asyncio": {
|
||||
"hashes": [
|
||||
"sha256:475bd2f3dc0bc11d2463656b3cbaafdbec5a47b47508ea0b329ee693040eebd2"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==0.12.0"
|
||||
},
|
||||
"rope": {
|
||||
"hashes": [
|
||||
"sha256:658ad6705f43dcf3d6df379da9486529cf30e02d9ea14c5682aa80eb33b649e1"
|
||||
],
|
||||
"index": "pypi",
|
||||
"version": "==0.17.0"
|
||||
},
|
||||
"six": {
|
||||
"hashes": [
|
||||
"sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
|
||||
"sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
|
||||
],
|
||||
"version": "==1.15.0"
|
||||
},
|
||||
"typed-ast": {
|
||||
"hashes": [
|
||||
"sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355",
|
||||
"sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919",
|
||||
"sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa",
|
||||
"sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652",
|
||||
"sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75",
|
||||
"sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01",
|
||||
"sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d",
|
||||
"sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1",
|
||||
"sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907",
|
||||
"sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c",
|
||||
"sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3",
|
||||
"sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b",
|
||||
"sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614",
|
||||
"sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb",
|
||||
"sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b",
|
||||
"sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41",
|
||||
"sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6",
|
||||
"sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34",
|
||||
"sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe",
|
||||
"sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4",
|
||||
"sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7"
|
||||
],
|
||||
"version": "==1.4.1"
|
||||
},
|
||||
"typing-extensions": {
|
||||
"hashes": [
|
||||
"sha256:6e95524d8a547a91e08f404ae485bbb71962de46967e1b71a0cb89af24e761c5",
|
||||
"sha256:79ee589a3caca649a9bfd2a8de4709837400dfa00b6cc81962a1e6a1815969ae",
|
||||
"sha256:f8d2bd89d25bc39dabe7d23df520442fa1d8969b82544370e03d88b5a591c392"
|
||||
],
|
||||
"version": "==3.7.4.2"
|
||||
},
|
||||
"wcwidth": {
|
||||
"hashes": [
|
||||
"sha256:cafe2186b3c009a04067022ce1dcd79cb38d8d65ee4f4791b8888d6599d1bbe1",
|
||||
"sha256:ee73862862a156bf77ff92b09034fc4825dd3af9cf81bc5b360668d425f3c5f1"
|
||||
],
|
||||
"version": "==0.1.9"
|
||||
},
|
||||
"zipp": {
|
||||
"hashes": [
|
||||
"sha256:aa36550ff0c0b7ef7fa639055d797116ee891440eac1a56f378e2d3179e0320b",
|
||||
"sha256:c599e4d75c98f6798c509911d08a22e6c021d074469042177c8c86fb92eefd96"
|
||||
],
|
||||
"version": "==3.1.0"
|
||||
}
|
||||
}
|
||||
}
|
162
README.md
162
README.md
@@ -37,21 +37,26 @@ This project exists because I am unhappy with the state of the official Google p
|
||||
- Uses `SerializeToString()` rather than the built-in `__bytes__()`
|
||||
- Special wrapped types don't use Python's `None`
|
||||
- Timestamp/duration types don't use Python's built-in `datetime` module
|
||||
|
||||
This project is a reimplementation from the ground up focused on idiomatic modern Python to help fix some of the above. While it may not be a 1:1 drop-in replacement due to changed method names and call patterns, the wire format is identical.
|
||||
|
||||
## Installation & Getting Started
|
||||
## Installation
|
||||
|
||||
First, install the package. Note that the `[compiler]` feature flag tells it to install extra dependencies only needed by the `protoc` plugin:
|
||||
|
||||
```sh
|
||||
# Install both the library and compiler
|
||||
$ pip install "betterproto[compiler]"
|
||||
pip install "betterproto[compiler]"
|
||||
|
||||
# Install just the library (to use the generated code output)
|
||||
$ pip install betterproto
|
||||
pip install betterproto
|
||||
```
|
||||
|
||||
*Betterproto* is under active development. To install the latest beta version, use `pip install --pre betterproto`.
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Compiling proto files
|
||||
|
||||
Now, given you installed the compiler and have a proto file, e.g `example.proto`:
|
||||
|
||||
```protobuf
|
||||
@@ -65,17 +70,25 @@ message Greeting {
|
||||
}
|
||||
```
|
||||
|
||||
You can run the following:
|
||||
You can run the following to invoke protoc directly:
|
||||
|
||||
```sh
|
||||
$ protoc -I . --python_betterproto_out=. example.proto
|
||||
mkdir lib
|
||||
protoc -I . --python_betterproto_out=lib example.proto
|
||||
```
|
||||
|
||||
This will generate `hello.py` which looks like:
|
||||
or run the following to invoke protoc via grpcio-tools:
|
||||
|
||||
```py
|
||||
```sh
|
||||
pip install grpcio-tools
|
||||
python -m grpc_tools.protoc -I . --python_betterproto_out=lib example.proto
|
||||
```
|
||||
|
||||
This will generate `lib/hello/__init__.py` which looks like:
|
||||
|
||||
```python
|
||||
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
||||
# sources: hello.proto
|
||||
# sources: example.proto
|
||||
# plugin: python-betterproto
|
||||
from dataclasses import dataclass
|
||||
|
||||
@@ -83,7 +96,7 @@ import betterproto
|
||||
|
||||
|
||||
@dataclass
|
||||
class Hello(betterproto.Message):
|
||||
class Greeting(betterproto.Message):
|
||||
"""Greeting represents a message you can tell a user."""
|
||||
|
||||
message: str = betterproto.string_field(1)
|
||||
@@ -91,23 +104,23 @@ class Hello(betterproto.Message):
|
||||
|
||||
Now you can use it!
|
||||
|
||||
```py
|
||||
>>> from hello import Hello
|
||||
>>> test = Hello()
|
||||
```python
|
||||
>>> from lib.hello import Greeting
|
||||
>>> test = Greeting()
|
||||
>>> test
|
||||
Hello(message='')
|
||||
Greeting(message='')
|
||||
|
||||
>>> test.message = "Hey!"
|
||||
>>> test
|
||||
Hello(message="Hey!")
|
||||
Greeting(message="Hey!")
|
||||
|
||||
>>> serialized = bytes(test)
|
||||
>>> serialized
|
||||
b'\n\x04Hey!'
|
||||
|
||||
>>> another = Hello().parse(serialized)
|
||||
>>> another = Greeting().parse(serialized)
|
||||
>>> another
|
||||
Hello(message="Hey!")
|
||||
Greeting(message="Hey!")
|
||||
|
||||
>>> another.to_dict()
|
||||
{"message": "Hey!"}
|
||||
@@ -119,7 +132,7 @@ Hello(message="Hey!")
|
||||
|
||||
The generated Protobuf `Message` classes are compatible with [grpclib](https://github.com/vmagamedov/grpclib) so you are free to use it if you like. That said, this project also includes support for async gRPC stub generation with better static type checking and code completion support. It is enabled by default.
|
||||
|
||||
Given an example like:
|
||||
Given an example service definition:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
@@ -146,22 +159,37 @@ service Echo {
|
||||
}
|
||||
```
|
||||
|
||||
You can use it like so (enable async in the interactive shell first):
|
||||
A client can be implemented as follows:
|
||||
```python
|
||||
import asyncio
|
||||
import echo
|
||||
|
||||
```py
|
||||
>>> import echo
|
||||
>>> from grpclib.client import Channel
|
||||
from grpclib.client import Channel
|
||||
|
||||
>>> channel = Channel(host="127.0.0.1", port=1234)
|
||||
>>> service = echo.EchoStub(channel)
|
||||
>>> await service.echo(value="hello", extra_times=1)
|
||||
EchoResponse(values=["hello", "hello"])
|
||||
|
||||
>>> async for response in service.echo_stream(value="hello", extra_times=1)
|
||||
async def main():
|
||||
channel = Channel(host="127.0.0.1", port=50051)
|
||||
service = echo.EchoStub(channel)
|
||||
response = await service.echo(value="hello", extra_times=1)
|
||||
print(response)
|
||||
|
||||
async for response in service.echo_stream(value="hello", extra_times=1):
|
||||
print(response)
|
||||
|
||||
EchoStreamResponse(value="hello")
|
||||
EchoStreamResponse(value="hello")
|
||||
# don't forget to close the channel when done!
|
||||
channel.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.run_until_complete(main())
|
||||
|
||||
```
|
||||
which would output
|
||||
```python
|
||||
EchoResponse(values=['hello', 'hello'])
|
||||
EchoStreamResponse(value='hello')
|
||||
EchoStreamResponse(value='hello')
|
||||
```
|
||||
|
||||
### JSON
|
||||
@@ -173,8 +201,8 @@ Both serializing and parsing are supported to/from JSON and Python dictionaries
|
||||
|
||||
For compatibility the default is to convert field names to `camelCase`. You can control this behavior by passing a casing value, e.g:
|
||||
|
||||
```py
|
||||
>>> MyMessage().to_dict(casing=betterproto.Casing.SNAKE)
|
||||
```python
|
||||
MyMessage().to_dict(casing=betterproto.Casing.SNAKE)
|
||||
```
|
||||
|
||||
### Determining if a message was sent
|
||||
@@ -256,6 +284,7 @@ Google provides several well-known message types like a timestamp, duration, and
|
||||
| `google.protobuf.duration` | [`datetime.timedelta`][td] | `0` |
|
||||
| `google.protobuf.timestamp` | Timezone-aware [`datetime.datetime`][dt] | `1970-01-01T00:00:00Z` |
|
||||
| `google.protobuf.*Value` | `Optional[...]` | `None` |
|
||||
| `google.protobuf.*` | `betterproto.lib.google.protobuf.*` | `None` |
|
||||
|
||||
[td]: https://docs.python.org/3/library/datetime.html#timedelta-objects
|
||||
[dt]: https://docs.python.org/3/library/datetime.html#datetime.datetime
|
||||
@@ -296,25 +325,39 @@ datetime.datetime(2019, 1, 1, 11, 59, 58, 800000, tzinfo=datetime.timezone.utc)
|
||||
|
||||
## Development
|
||||
|
||||
First, make sure you have Python 3.6+ and `pipenv` installed, along with the official [Protobuf Compiler](https://github.com/protocolbuffers/protobuf/releases) for your platform. Then:
|
||||
- _Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!_
|
||||
- _See how you can help → [Contributing](.github/CONTRIBUTING.md)_
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python (3.6 or higher)
|
||||
|
||||
- [poetry](https://python-poetry.org/docs/#installation)
|
||||
*Needed to install dependencies in a virtual environment*
|
||||
|
||||
- [poethepoet](https://github.com/nat-n/poethepoet) for running development tasks as defined in pyproject.toml
|
||||
- Can be installed to your host environment via `pip install poethepoet` then executed as simple `poe`
|
||||
- or run from the poetry venv as `poetry run poe`
|
||||
|
||||
### Setup
|
||||
|
||||
```sh
|
||||
# Get set up with the virtual env & dependencies
|
||||
$ pipenv install --dev
|
||||
poetry run pip install --upgrade pip
|
||||
poetry install
|
||||
|
||||
# Link the local package
|
||||
$ pipenv shell
|
||||
$ pip install -e .
|
||||
# Activate the poetry environment
|
||||
poetry shell
|
||||
```
|
||||
|
||||
### Code style
|
||||
|
||||
This project enforces [black](https://github.com/psf/black) python code formatting.
|
||||
|
||||
Before commiting changes run:
|
||||
Before committing changes run:
|
||||
|
||||
```bash
|
||||
pipenv run black .
|
||||
```sh
|
||||
poe format
|
||||
```
|
||||
|
||||
To avoid merge conflicts later, non-black formatted python code will fail in CI.
|
||||
@@ -332,7 +375,7 @@ Adding a standard test case is easy.
|
||||
|
||||
- Create a new directory `betterproto/tests/inputs/<name>`
|
||||
- add `<name>.proto` with a message called `Test`
|
||||
- add `<name>.json` with some test data
|
||||
- add `<name>.json` with some test data (optional)
|
||||
|
||||
It will be picked up automatically when you run the tests.
|
||||
|
||||
@@ -347,11 +390,34 @@ Custom tests are found in `tests/test_*.py` and are run with pytest.
|
||||
Here's how to run the tests.
|
||||
|
||||
```sh
|
||||
# Generate assets from sample .proto files
|
||||
$ pipenv run generate
|
||||
# Generate assets from sample .proto files required by the tests
|
||||
poe generate
|
||||
# Run the tests
|
||||
poe test
|
||||
```
|
||||
|
||||
# Run all tests
|
||||
$ pipenv run test
|
||||
To run tests as they are run in CI (with tox) run:
|
||||
|
||||
```sh
|
||||
poe full-test
|
||||
```
|
||||
|
||||
### (Re)compiling Google Well-known Types
|
||||
|
||||
Betterproto includes compiled versions for Google's well-known types at [betterproto/lib/google](betterproto/lib/google).
|
||||
Be sure to regenerate these files when modifying the plugin output format, and validate by running the tests.
|
||||
|
||||
Normally, the plugin does not compile any references to `google.protobuf`, since they are pre-compiled. To force compilation of `google.protobuf`, use the option `--custom_opt=INCLUDE_GOOGLE`.
|
||||
|
||||
Assuming your `google.protobuf` source files (included with all releases of `protoc`) are located in `/usr/local/include`, you can regenerate them as follows:
|
||||
|
||||
```sh
|
||||
protoc \
|
||||
--plugin=protoc-gen-custom=betterproto/plugin.py \
|
||||
--custom_opt=INCLUDE_GOOGLE \
|
||||
--custom_out=betterproto/lib \
|
||||
-I /usr/local/include/ \
|
||||
/usr/local/include/google/protobuf/*.proto
|
||||
```
|
||||
|
||||
### TODO
|
||||
@@ -384,16 +450,20 @@ $ pipenv run test
|
||||
- [x] Enum strings
|
||||
- [x] Well known types support (timestamp, duration, wrappers)
|
||||
- [x] Support different casing (orig vs. camel vs. others?)
|
||||
- [ ] Async service stubs
|
||||
- [x] Async service stubs
|
||||
- [x] Unary-unary
|
||||
- [x] Server streaming response
|
||||
- [ ] Client streaming request
|
||||
- [x] Client streaming request
|
||||
- [x] Renaming messages and fields to conform to Python name standards
|
||||
- [x] Renaming clashes with language keywords
|
||||
- [x] Python package
|
||||
- [x] Automate running tests
|
||||
- [ ] Cleanup!
|
||||
|
||||
## Community
|
||||
|
||||
Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!
|
||||
|
||||
## License
|
||||
|
||||
Copyright © 2019 Daniel G. Taylor
|
||||
|
157
asv.conf.json
Normal file
157
asv.conf.json
Normal file
@@ -0,0 +1,157 @@
|
||||
{
|
||||
// The version of the config file format. Do not change, unless
|
||||
// you know what you are doing.
|
||||
"version": 1,
|
||||
|
||||
// The name of the project being benchmarked
|
||||
"project": "python-betterproto",
|
||||
|
||||
// The project's homepage
|
||||
"project_url": "https://github.com/danielgtaylor/python-betterproto",
|
||||
|
||||
// The URL or local path of the source code repository for the
|
||||
// project being benchmarked
|
||||
"repo": ".",
|
||||
|
||||
// The Python project's subdirectory in your repo. If missing or
|
||||
// the empty string, the project is assumed to be located at the root
|
||||
// of the repository.
|
||||
// "repo_subdir": "",
|
||||
|
||||
// Customizable commands for building, installing, and
|
||||
// uninstalling the project. See asv.conf.json documentation.
|
||||
//
|
||||
"install_command": ["python -m pip install ."],
|
||||
"uninstall_command": ["return-code=any python -m pip uninstall -y {project}"],
|
||||
"build_command": ["python -m pip wheel -w {build_cache_dir} {build_dir}"],
|
||||
|
||||
// List of branches to benchmark. If not provided, defaults to "master"
|
||||
// (for git) or "default" (for mercurial).
|
||||
// "branches": ["master"], // for git
|
||||
// "branches": ["default"], // for mercurial
|
||||
|
||||
// The DVCS being used. If not set, it will be automatically
|
||||
// determined from "repo" by looking at the protocol in the URL
|
||||
// (if remote), or by looking for special directories, such as
|
||||
// ".git" (if local).
|
||||
// "dvcs": "git",
|
||||
|
||||
// The tool to use to create environments. May be "conda",
|
||||
// "virtualenv" or other value depending on the plugins in use.
|
||||
// If missing or the empty string, the tool will be automatically
|
||||
// determined by looking for tools on the PATH environment
|
||||
// variable.
|
||||
"environment_type": "virtualenv",
|
||||
|
||||
// timeout in seconds for installing any dependencies in environment
|
||||
// defaults to 10 min
|
||||
//"install_timeout": 600,
|
||||
|
||||
// the base URL to show a commit for the project.
|
||||
// "show_commit_url": "http://github.com/owner/project/commit/",
|
||||
|
||||
// The Pythons you'd like to test against. If not provided, defaults
|
||||
// to the current version of Python used to run `asv`.
|
||||
// "pythons": ["2.7", "3.6"],
|
||||
|
||||
// The list of conda channel names to be searched for benchmark
|
||||
// dependency packages in the specified order
|
||||
// "conda_channels": ["conda-forge", "defaults"],
|
||||
|
||||
// The matrix of dependencies to test. Each key is the name of a
|
||||
// package (in PyPI) and the values are version numbers. An empty
|
||||
// list or empty string indicates to just test against the default
|
||||
// (latest) version. null indicates that the package is to not be
|
||||
// installed. If the package to be tested is only available from
|
||||
// PyPi, and the 'environment_type' is conda, then you can preface
|
||||
// the package name by 'pip+', and the package will be installed via
|
||||
// pip (with all the conda available packages installed first,
|
||||
// followed by the pip installed packages).
|
||||
//
|
||||
// "matrix": {
|
||||
// "numpy": ["1.6", "1.7"],
|
||||
// "six": ["", null], // test with and without six installed
|
||||
// "pip+emcee": [""], // emcee is only available for install with pip.
|
||||
// },
|
||||
|
||||
// Combinations of libraries/python versions can be excluded/included
|
||||
// from the set to test. Each entry is a dictionary containing additional
|
||||
// key-value pairs to include/exclude.
|
||||
//
|
||||
// An exclude entry excludes entries where all values match. The
|
||||
// values are regexps that should match the whole string.
|
||||
//
|
||||
// An include entry adds an environment. Only the packages listed
|
||||
// are installed. The 'python' key is required. The exclude rules
|
||||
// do not apply to includes.
|
||||
//
|
||||
// In addition to package names, the following keys are available:
|
||||
//
|
||||
// - python
|
||||
// Python version, as in the *pythons* variable above.
|
||||
// - environment_type
|
||||
// Environment type, as above.
|
||||
// - sys_platform
|
||||
// Platform, as in sys.platform. Possible values for the common
|
||||
// cases: 'linux2', 'win32', 'cygwin', 'darwin'.
|
||||
//
|
||||
// "exclude": [
|
||||
// {"python": "3.2", "sys_platform": "win32"}, // skip py3.2 on windows
|
||||
// {"environment_type": "conda", "six": null}, // don't run without six on conda
|
||||
// ],
|
||||
//
|
||||
// "include": [
|
||||
// // additional env for python2.7
|
||||
// {"python": "2.7", "numpy": "1.8"},
|
||||
// // additional env if run on windows+conda
|
||||
// {"platform": "win32", "environment_type": "conda", "python": "2.7", "libpython": ""},
|
||||
// ],
|
||||
|
||||
// The directory (relative to the current directory) that benchmarks are
|
||||
// stored in. If not provided, defaults to "benchmarks"
|
||||
// "benchmark_dir": "benchmarks",
|
||||
|
||||
// The directory (relative to the current directory) to cache the Python
|
||||
// environments in. If not provided, defaults to "env"
|
||||
"env_dir": ".asv/env",
|
||||
|
||||
// The directory (relative to the current directory) that raw benchmark
|
||||
// results are stored in. If not provided, defaults to "results".
|
||||
"results_dir": ".asv/results",
|
||||
|
||||
// The directory (relative to the current directory) that the html tree
|
||||
// should be written to. If not provided, defaults to "html".
|
||||
"html_dir": ".asv/html",
|
||||
|
||||
// The number of characters to retain in the commit hashes.
|
||||
// "hash_length": 8,
|
||||
|
||||
// `asv` will cache results of the recent builds in each
|
||||
// environment, making them faster to install next time. This is
|
||||
// the number of builds to keep, per environment.
|
||||
// "build_cache_size": 2,
|
||||
|
||||
// The commits after which the regression search in `asv publish`
|
||||
// should start looking for regressions. Dictionary whose keys are
|
||||
// regexps matching to benchmark names, and values corresponding to
|
||||
// the commit (exclusive) after which to start looking for
|
||||
// regressions. The default is to start from the first commit
|
||||
// with results. If the commit is `null`, regression detection is
|
||||
// skipped for the matching benchmark.
|
||||
//
|
||||
// "regressions_first_commits": {
|
||||
// "some_benchmark": "352cdf", // Consider regressions only after this commit
|
||||
// "another_benchmark": null, // Skip regression detection altogether
|
||||
// },
|
||||
|
||||
// The thresholds for relative change in results, after which `asv
|
||||
// publish` starts reporting regressions. Dictionary of the same
|
||||
// form as in ``regressions_first_commits``, with values
|
||||
// indicating the thresholds. If multiple entries match, the
|
||||
// maximum is taken. If no entry matches, the default is 5%.
|
||||
//
|
||||
// "regressions_thresholds": {
|
||||
// "some_benchmark": 0.01, // Threshold of 1%
|
||||
// "another_benchmark": 0.5, // Threshold of 50%
|
||||
// },
|
||||
}
|
1
benchmarks/__init__.py
Normal file
1
benchmarks/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
|
59
benchmarks/benchmarks.py
Normal file
59
benchmarks/benchmarks.py
Normal file
@@ -0,0 +1,59 @@
|
||||
import betterproto
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestMessage(betterproto.Message):
|
||||
foo: int = betterproto.uint32_field(0)
|
||||
bar: str = betterproto.string_field(1)
|
||||
baz: float = betterproto.float_field(2)
|
||||
|
||||
|
||||
class BenchMessage:
|
||||
"""Test creation and usage a proto message."""
|
||||
|
||||
def setup(self):
|
||||
self.cls = TestMessage
|
||||
self.instance = TestMessage()
|
||||
self.instance_filled = TestMessage(0, "test", 0.0)
|
||||
|
||||
def time_overhead(self):
|
||||
"""Overhead in class definition."""
|
||||
|
||||
@dataclass
|
||||
class Message(betterproto.Message):
|
||||
foo: int = betterproto.uint32_field(0)
|
||||
bar: str = betterproto.string_field(1)
|
||||
baz: float = betterproto.float_field(2)
|
||||
|
||||
def time_instantiation(self):
|
||||
"""Time instantiation"""
|
||||
self.cls()
|
||||
|
||||
def time_attribute_access(self):
|
||||
"""Time to access an attribute"""
|
||||
self.instance.foo
|
||||
self.instance.bar
|
||||
self.instance.baz
|
||||
|
||||
def time_init_with_values(self):
|
||||
"""Time to set an attribute"""
|
||||
self.cls(0, "test", 0.0)
|
||||
|
||||
def time_attribute_setting(self):
|
||||
"""Time to set attributes"""
|
||||
self.instance.foo = 0
|
||||
self.instance.bar = "test"
|
||||
self.instance.baz = 0.0
|
||||
|
||||
def time_serialize(self):
|
||||
"""Time serializing a message to wire."""
|
||||
bytes(self.instance_filled)
|
||||
|
||||
|
||||
class MemSuite:
|
||||
def setup(self):
|
||||
self.cls = TestMessage
|
||||
|
||||
def mem_instance(self):
|
||||
return self.cls()
|
@@ -1,41 +0,0 @@
|
||||
import stringcase
|
||||
|
||||
|
||||
def safe_snake_case(value: str) -> str:
|
||||
"""Snake case a value taking into account Python keywords."""
|
||||
value = stringcase.snakecase(value)
|
||||
if value in [
|
||||
"and",
|
||||
"as",
|
||||
"assert",
|
||||
"break",
|
||||
"class",
|
||||
"continue",
|
||||
"def",
|
||||
"del",
|
||||
"elif",
|
||||
"else",
|
||||
"except",
|
||||
"finally",
|
||||
"for",
|
||||
"from",
|
||||
"global",
|
||||
"if",
|
||||
"import",
|
||||
"in",
|
||||
"is",
|
||||
"lambda",
|
||||
"nonlocal",
|
||||
"not",
|
||||
"or",
|
||||
"pass",
|
||||
"raise",
|
||||
"return",
|
||||
"try",
|
||||
"while",
|
||||
"with",
|
||||
"yield",
|
||||
]:
|
||||
# https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles
|
||||
value += "_"
|
||||
return value
|
@@ -1,2 +0,0 @@
|
||||
@SET plugin_dir=%~dp0
|
||||
@python %plugin_dir%/plugin.py %*
|
@@ -1,474 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import itertools
|
||||
import os.path
|
||||
import sys
|
||||
import textwrap
|
||||
from collections import defaultdict
|
||||
from typing import Dict, List, Optional, Type
|
||||
|
||||
try:
|
||||
import black
|
||||
except ImportError:
|
||||
print(
|
||||
"Unable to import `black` formatter. Did you install the compiler feature with `pip install betterproto[compiler]`?"
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
import jinja2
|
||||
import stringcase
|
||||
|
||||
from google.protobuf.compiler import plugin_pb2 as plugin
|
||||
from google.protobuf.descriptor_pb2 import (
|
||||
DescriptorProto,
|
||||
EnumDescriptorProto,
|
||||
FieldDescriptorProto,
|
||||
)
|
||||
|
||||
from betterproto.casing import safe_snake_case
|
||||
|
||||
import google.protobuf.wrappers_pb2 as google_wrappers
|
||||
|
||||
WRAPPER_TYPES: Dict[str, Optional[Type]] = defaultdict(
|
||||
lambda: None,
|
||||
{
|
||||
"google.protobuf.DoubleValue": google_wrappers.DoubleValue,
|
||||
"google.protobuf.FloatValue": google_wrappers.FloatValue,
|
||||
"google.protobuf.Int64Value": google_wrappers.Int64Value,
|
||||
"google.protobuf.UInt64Value": google_wrappers.UInt64Value,
|
||||
"google.protobuf.Int32Value": google_wrappers.Int32Value,
|
||||
"google.protobuf.UInt32Value": google_wrappers.UInt32Value,
|
||||
"google.protobuf.BoolValue": google_wrappers.BoolValue,
|
||||
"google.protobuf.StringValue": google_wrappers.StringValue,
|
||||
"google.protobuf.BytesValue": google_wrappers.BytesValue,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def get_ref_type(
|
||||
package: str, imports: set, type_name: str, unwrap: bool = True
|
||||
) -> str:
|
||||
"""
|
||||
Return a Python type name for a proto type reference. Adds the import if
|
||||
necessary. Unwraps well known type if required.
|
||||
"""
|
||||
# If the package name is a blank string, then this should still work
|
||||
# because by convention packages are lowercase and message/enum types are
|
||||
# pascal-cased. May require refactoring in the future.
|
||||
type_name = type_name.lstrip(".")
|
||||
|
||||
# Check if type is wrapper.
|
||||
wrapper_class = WRAPPER_TYPES[type_name]
|
||||
|
||||
if unwrap:
|
||||
if wrapper_class:
|
||||
wrapped_type = type(wrapper_class().value)
|
||||
return f"Optional[{wrapped_type.__name__}]"
|
||||
|
||||
if type_name == "google.protobuf.Duration":
|
||||
return "timedelta"
|
||||
|
||||
if type_name == "google.protobuf.Timestamp":
|
||||
return "datetime"
|
||||
elif wrapper_class:
|
||||
imports.add(f"from {wrapper_class.__module__} import {wrapper_class.__name__}")
|
||||
return f"{wrapper_class.__name__}"
|
||||
|
||||
if type_name.startswith(package):
|
||||
parts = type_name.lstrip(package).lstrip(".").split(".")
|
||||
if len(parts) == 1 or (len(parts) > 1 and parts[0][0] == parts[0][0].upper()):
|
||||
# This is the current package, which has nested types flattened.
|
||||
# foo.bar_thing => FooBarThing
|
||||
cased = [stringcase.pascalcase(part) for part in parts]
|
||||
type_name = f'"{"".join(cased)}"'
|
||||
|
||||
if "." in type_name:
|
||||
# This is imported from another package. No need
|
||||
# to use a forward ref and we need to add the import.
|
||||
parts = type_name.split(".")
|
||||
parts[-1] = stringcase.pascalcase(parts[-1])
|
||||
imports.add(f"from .{'.'.join(parts[:-2])} import {parts[-2]}")
|
||||
type_name = f"{parts[-2]}.{parts[-1]}"
|
||||
|
||||
return type_name
|
||||
|
||||
|
||||
def py_type(
|
||||
package: str,
|
||||
imports: set,
|
||||
message: DescriptorProto,
|
||||
descriptor: FieldDescriptorProto,
|
||||
) -> str:
|
||||
if descriptor.type in [1, 2, 6, 7, 15, 16]:
|
||||
return "float"
|
||||
elif descriptor.type in [3, 4, 5, 13, 17, 18]:
|
||||
return "int"
|
||||
elif descriptor.type == 8:
|
||||
return "bool"
|
||||
elif descriptor.type == 9:
|
||||
return "str"
|
||||
elif descriptor.type in [11, 14]:
|
||||
# Type referencing another defined Message or a named enum
|
||||
return get_ref_type(package, imports, descriptor.type_name)
|
||||
elif descriptor.type == 12:
|
||||
return "bytes"
|
||||
else:
|
||||
raise NotImplementedError(f"Unknown type {descriptor.type}")
|
||||
|
||||
|
||||
def get_py_zero(type_num: int) -> str:
|
||||
zero = 0
|
||||
if type_num in []:
|
||||
zero = 0.0
|
||||
elif type_num == 8:
|
||||
zero = "False"
|
||||
elif type_num == 9:
|
||||
zero = '""'
|
||||
elif type_num == 11:
|
||||
zero = "None"
|
||||
elif type_num == 12:
|
||||
zero = 'b""'
|
||||
|
||||
return zero
|
||||
|
||||
|
||||
def traverse(proto_file):
|
||||
def _traverse(path, items, prefix=""):
|
||||
for i, item in enumerate(items):
|
||||
# Adjust the name since we flatten the heirarchy.
|
||||
item.name = next_prefix = prefix + item.name
|
||||
yield item, path + [i]
|
||||
|
||||
if isinstance(item, DescriptorProto):
|
||||
for enum in item.enum_type:
|
||||
enum.name = next_prefix + enum.name
|
||||
yield enum, path + [i, 4]
|
||||
|
||||
if item.nested_type:
|
||||
for n, p in _traverse(path + [i, 3], item.nested_type, next_prefix):
|
||||
yield n, p
|
||||
|
||||
return itertools.chain(
|
||||
_traverse([5], proto_file.enum_type), _traverse([4], proto_file.message_type)
|
||||
)
|
||||
|
||||
|
||||
def get_comment(proto_file, path: List[int], indent: int = 4) -> str:
|
||||
pad = " " * indent
|
||||
for sci in proto_file.source_code_info.location:
|
||||
# print(list(sci.path), path, file=sys.stderr)
|
||||
if list(sci.path) == path and sci.leading_comments:
|
||||
lines = textwrap.wrap(
|
||||
sci.leading_comments.strip().replace("\n", ""), width=79 - indent
|
||||
)
|
||||
|
||||
if path[-2] == 2 and path[-4] != 6:
|
||||
# This is a field
|
||||
return f"{pad}# " + f"\n{pad}# ".join(lines)
|
||||
else:
|
||||
# This is a message, enum, service, or method
|
||||
if len(lines) == 1 and len(lines[0]) < 79 - indent - 6:
|
||||
lines[0] = lines[0].strip('"')
|
||||
return f'{pad}"""{lines[0]}"""'
|
||||
else:
|
||||
joined = f"\n{pad}".join(lines)
|
||||
return f'{pad}"""\n{pad}{joined}\n{pad}"""'
|
||||
|
||||
return ""
|
||||
|
||||
|
||||
def generate_code(request, response):
|
||||
env = jinja2.Environment(
|
||||
trim_blocks=True,
|
||||
lstrip_blocks=True,
|
||||
loader=jinja2.FileSystemLoader("%s/templates/" % os.path.dirname(__file__)),
|
||||
)
|
||||
template = env.get_template("template.py.j2")
|
||||
|
||||
output_map = {}
|
||||
for proto_file in request.proto_file:
|
||||
out = proto_file.package
|
||||
if out == "google.protobuf":
|
||||
continue
|
||||
|
||||
if not out:
|
||||
out = os.path.splitext(proto_file.name)[0].replace(os.path.sep, ".")
|
||||
|
||||
if out not in output_map:
|
||||
output_map[out] = {"package": proto_file.package, "files": []}
|
||||
output_map[out]["files"].append(proto_file)
|
||||
|
||||
# TODO: Figure out how to handle gRPC request/response messages and add
|
||||
# processing below for Service.
|
||||
|
||||
for filename, options in output_map.items():
|
||||
package = options["package"]
|
||||
# print(package, filename, file=sys.stderr)
|
||||
output = {
|
||||
"package": package,
|
||||
"files": [f.name for f in options["files"]],
|
||||
"imports": set(),
|
||||
"datetime_imports": set(),
|
||||
"typing_imports": set(),
|
||||
"messages": [],
|
||||
"enums": [],
|
||||
"services": [],
|
||||
}
|
||||
|
||||
type_mapping = {}
|
||||
|
||||
for proto_file in options["files"]:
|
||||
# print(proto_file.message_type, file=sys.stderr)
|
||||
# print(proto_file.service, file=sys.stderr)
|
||||
# print(proto_file.source_code_info, file=sys.stderr)
|
||||
|
||||
for item, path in traverse(proto_file):
|
||||
# print(item, file=sys.stderr)
|
||||
# print(path, file=sys.stderr)
|
||||
data = {"name": item.name, "py_name": stringcase.pascalcase(item.name)}
|
||||
|
||||
if isinstance(item, DescriptorProto):
|
||||
# print(item, file=sys.stderr)
|
||||
if item.options.map_entry:
|
||||
# Skip generated map entry messages since we just use dicts
|
||||
continue
|
||||
|
||||
data.update(
|
||||
{
|
||||
"type": "Message",
|
||||
"comment": get_comment(proto_file, path),
|
||||
"properties": [],
|
||||
}
|
||||
)
|
||||
|
||||
for i, f in enumerate(item.field):
|
||||
t = py_type(package, output["imports"], item, f)
|
||||
zero = get_py_zero(f.type)
|
||||
|
||||
repeated = False
|
||||
packed = False
|
||||
|
||||
field_type = f.Type.Name(f.type).lower()[5:]
|
||||
|
||||
field_wraps = ""
|
||||
if f.type_name.startswith(
|
||||
".google.protobuf"
|
||||
) and f.type_name.endswith("Value"):
|
||||
w = f.type_name.split(".").pop()[:-5].upper()
|
||||
field_wraps = f"betterproto.TYPE_{w}"
|
||||
|
||||
map_types = None
|
||||
if f.type == 11:
|
||||
# This might be a map...
|
||||
message_type = f.type_name.split(".").pop().lower()
|
||||
# message_type = py_type(package)
|
||||
map_entry = f"{f.name.replace('_', '').lower()}entry"
|
||||
|
||||
if message_type == map_entry:
|
||||
for nested in item.nested_type:
|
||||
if (
|
||||
nested.name.replace("_", "").lower()
|
||||
== map_entry
|
||||
):
|
||||
if nested.options.map_entry:
|
||||
# print("Found a map!", file=sys.stderr)
|
||||
k = py_type(
|
||||
package,
|
||||
output["imports"],
|
||||
item,
|
||||
nested.field[0],
|
||||
)
|
||||
v = py_type(
|
||||
package,
|
||||
output["imports"],
|
||||
item,
|
||||
nested.field[1],
|
||||
)
|
||||
t = f"Dict[{k}, {v}]"
|
||||
field_type = "map"
|
||||
map_types = (
|
||||
f.Type.Name(nested.field[0].type),
|
||||
f.Type.Name(nested.field[1].type),
|
||||
)
|
||||
output["typing_imports"].add("Dict")
|
||||
|
||||
if f.label == 3 and field_type != "map":
|
||||
# Repeated field
|
||||
repeated = True
|
||||
t = f"List[{t}]"
|
||||
zero = "[]"
|
||||
output["typing_imports"].add("List")
|
||||
|
||||
if f.type in [1, 2, 3, 4, 5, 6, 7, 8, 13, 15, 16, 17, 18]:
|
||||
packed = True
|
||||
|
||||
one_of = ""
|
||||
if f.HasField("oneof_index"):
|
||||
one_of = item.oneof_decl[f.oneof_index].name
|
||||
|
||||
if "Optional[" in t:
|
||||
output["typing_imports"].add("Optional")
|
||||
|
||||
if "timedelta" in t:
|
||||
output["datetime_imports"].add("timedelta")
|
||||
elif "datetime" in t:
|
||||
output["datetime_imports"].add("datetime")
|
||||
|
||||
data["properties"].append(
|
||||
{
|
||||
"name": f.name,
|
||||
"py_name": safe_snake_case(f.name),
|
||||
"number": f.number,
|
||||
"comment": get_comment(proto_file, path + [2, i]),
|
||||
"proto_type": int(f.type),
|
||||
"field_type": field_type,
|
||||
"field_wraps": field_wraps,
|
||||
"map_types": map_types,
|
||||
"type": t,
|
||||
"zero": zero,
|
||||
"repeated": repeated,
|
||||
"packed": packed,
|
||||
"one_of": one_of,
|
||||
}
|
||||
)
|
||||
# print(f, file=sys.stderr)
|
||||
|
||||
output["messages"].append(data)
|
||||
elif isinstance(item, EnumDescriptorProto):
|
||||
# print(item.name, path, file=sys.stderr)
|
||||
data.update(
|
||||
{
|
||||
"type": "Enum",
|
||||
"comment": get_comment(proto_file, path),
|
||||
"entries": [
|
||||
{
|
||||
"name": v.name,
|
||||
"value": v.number,
|
||||
"comment": get_comment(proto_file, path + [2, i]),
|
||||
}
|
||||
for i, v in enumerate(item.value)
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
output["enums"].append(data)
|
||||
|
||||
for i, service in enumerate(proto_file.service):
|
||||
# print(service, file=sys.stderr)
|
||||
|
||||
data = {
|
||||
"name": service.name,
|
||||
"py_name": stringcase.pascalcase(service.name),
|
||||
"comment": get_comment(proto_file, [6, i]),
|
||||
"methods": [],
|
||||
}
|
||||
|
||||
for j, method in enumerate(service.method):
|
||||
if method.client_streaming:
|
||||
raise NotImplementedError("Client streaming not yet supported")
|
||||
|
||||
input_message = None
|
||||
input_type = get_ref_type(
|
||||
package, output["imports"], method.input_type
|
||||
).strip('"')
|
||||
for msg in output["messages"]:
|
||||
if msg["name"] == input_type:
|
||||
input_message = msg
|
||||
for field in msg["properties"]:
|
||||
if field["zero"] == "None":
|
||||
output["typing_imports"].add("Optional")
|
||||
break
|
||||
|
||||
data["methods"].append(
|
||||
{
|
||||
"name": method.name,
|
||||
"py_name": stringcase.snakecase(method.name),
|
||||
"comment": get_comment(proto_file, [6, i, 2, j], indent=8),
|
||||
"route": f"/{package}.{service.name}/{method.name}",
|
||||
"input": get_ref_type(
|
||||
package, output["imports"], method.input_type
|
||||
).strip('"'),
|
||||
"input_message": input_message,
|
||||
"output": get_ref_type(
|
||||
package,
|
||||
output["imports"],
|
||||
method.output_type,
|
||||
unwrap=False,
|
||||
).strip('"'),
|
||||
"client_streaming": method.client_streaming,
|
||||
"server_streaming": method.server_streaming,
|
||||
}
|
||||
)
|
||||
|
||||
if method.server_streaming:
|
||||
output["typing_imports"].add("AsyncGenerator")
|
||||
|
||||
output["services"].append(data)
|
||||
|
||||
output["imports"] = sorted(output["imports"])
|
||||
output["datetime_imports"] = sorted(output["datetime_imports"])
|
||||
output["typing_imports"] = sorted(output["typing_imports"])
|
||||
|
||||
# Fill response
|
||||
f = response.file.add()
|
||||
# print(filename, file=sys.stderr)
|
||||
f.name = filename.replace(".", os.path.sep) + ".py"
|
||||
|
||||
# Render and then format the output file.
|
||||
f.content = black.format_str(
|
||||
template.render(description=output),
|
||||
mode=black.FileMode(target_versions=set([black.TargetVersion.PY37])),
|
||||
)
|
||||
|
||||
inits = set([""])
|
||||
for f in response.file:
|
||||
# Ensure output paths exist
|
||||
# print(f.name, file=sys.stderr)
|
||||
dirnames = os.path.dirname(f.name)
|
||||
if dirnames:
|
||||
os.makedirs(dirnames, exist_ok=True)
|
||||
base = ""
|
||||
for part in dirnames.split(os.path.sep):
|
||||
base = os.path.join(base, part)
|
||||
inits.add(base)
|
||||
|
||||
for base in inits:
|
||||
name = os.path.join(base, "__init__.py")
|
||||
|
||||
if os.path.exists(name):
|
||||
# Never overwrite inits as they may have custom stuff in them.
|
||||
continue
|
||||
|
||||
init = response.file.add()
|
||||
init.name = name
|
||||
init.content = b""
|
||||
|
||||
filenames = sorted([f.name for f in response.file])
|
||||
for fname in filenames:
|
||||
print(f"Writing {fname}", file=sys.stderr)
|
||||
|
||||
|
||||
def main():
|
||||
"""The plugin's main entry point."""
|
||||
# Read request message from stdin
|
||||
data = sys.stdin.buffer.read()
|
||||
|
||||
# Parse request
|
||||
request = plugin.CodeGeneratorRequest()
|
||||
request.ParseFromString(data)
|
||||
|
||||
# Create response
|
||||
response = plugin.CodeGeneratorResponse()
|
||||
|
||||
# Generate code
|
||||
generate_code(request, response)
|
||||
|
||||
# Serialise response message
|
||||
output = response.SerializeToString()
|
||||
|
||||
# Write to stdout
|
||||
sys.stdout.buffer.write(output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@@ -1,97 +0,0 @@
|
||||
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
||||
# sources: {{ ', '.join(description.files) }}
|
||||
# plugin: python-betterproto
|
||||
from dataclasses import dataclass
|
||||
{% if description.datetime_imports %}
|
||||
from datetime import {% for i in description.datetime_imports %}{{ i }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||
|
||||
{% endif%}
|
||||
{% if description.typing_imports %}
|
||||
from typing import {% for i in description.typing_imports %}{{ i }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||
|
||||
{% endif %}
|
||||
|
||||
import betterproto
|
||||
{% if description.services %}
|
||||
import grpclib
|
||||
{% endif %}
|
||||
|
||||
{% for i in description.imports %}
|
||||
{{ i }}
|
||||
{% endfor %}
|
||||
|
||||
|
||||
{% if description.enums %}{% for enum in description.enums %}
|
||||
class {{ enum.py_name }}(betterproto.Enum):
|
||||
{% if enum.comment %}
|
||||
{{ enum.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for entry in enum.entries %}
|
||||
{% if entry.comment %}
|
||||
{{ entry.comment }}
|
||||
{% endif %}
|
||||
{{ entry.name }} = {{ entry.value }}
|
||||
{% endfor %}
|
||||
|
||||
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% for message in description.messages %}
|
||||
@dataclass
|
||||
class {{ message.py_name }}(betterproto.Message):
|
||||
{% if message.comment %}
|
||||
{{ message.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for field in message.properties %}
|
||||
{% if field.comment %}
|
||||
{{ field.comment }}
|
||||
{% endif %}
|
||||
{{ field.py_name }}: {{ field.type }} = betterproto.{{ field.field_type }}_field({{ field.number }}{% if field.field_type == 'map'%}, betterproto.{{ field.map_types[0] }}, betterproto.{{ field.map_types[1] }}{% endif %}{% if field.one_of %}, group="{{ field.one_of }}"{% endif %}{% if field.field_wraps %}, wraps={{ field.field_wraps }}{% endif %})
|
||||
{% endfor %}
|
||||
{% if not message.properties %}
|
||||
pass
|
||||
{% endif %}
|
||||
|
||||
|
||||
{% endfor %}
|
||||
{% for service in description.services %}
|
||||
class {{ service.py_name }}Stub(betterproto.ServiceStub):
|
||||
{% if service.comment %}
|
||||
{{ service.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for method in service.methods %}
|
||||
async def {{ method.py_name }}(self{% if method.input_message and method.input_message.properties %}, *, {% for field in method.input_message.properties %}{{ field.py_name }}: {% if field.zero == "None" and not field.type.startswith("Optional[") %}Optional[{{ field.type }}]{% else %}{{ field.type }}{% endif %} = {{ field.zero }}{% if not loop.last %}, {% endif %}{% endfor %}{% endif %}) -> {% if method.server_streaming %}AsyncGenerator[{{ method.output }}, None]{% else %}{{ method.output }}{% endif %}:
|
||||
{% if method.comment %}
|
||||
{{ method.comment }}
|
||||
|
||||
{% endif %}
|
||||
request = {{ method.input }}()
|
||||
{% for field in method.input_message.properties %}
|
||||
{% if field.field_type == 'message' %}
|
||||
if {{ field.py_name }} is not None:
|
||||
request.{{ field.py_name }} = {{ field.py_name }}
|
||||
{% else %}
|
||||
request.{{ field.py_name }} = {{ field.py_name }}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
|
||||
{% if method.server_streaming %}
|
||||
async for response in self._unary_stream(
|
||||
"{{ method.route }}",
|
||||
request,
|
||||
{{ method.output }},
|
||||
):
|
||||
yield response
|
||||
{% else %}
|
||||
return await self._unary_unary(
|
||||
"{{ method.route }}",
|
||||
request,
|
||||
{{ method.output }},
|
||||
)
|
||||
{% endif %}
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
@@ -1,91 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
import glob
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
from typing import Set
|
||||
|
||||
from betterproto.tests.util import (
|
||||
get_directories,
|
||||
inputs_path,
|
||||
output_path_betterproto,
|
||||
output_path_reference,
|
||||
protoc_plugin,
|
||||
protoc_reference,
|
||||
)
|
||||
|
||||
# Force pure-python implementation instead of C++, otherwise imports
|
||||
# break things because we can't properly reset the symbol database.
|
||||
os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
|
||||
|
||||
|
||||
def clear_directory(path: str):
|
||||
for file_or_directory in glob.glob(os.path.join(path, "*")):
|
||||
if os.path.isdir(file_or_directory):
|
||||
shutil.rmtree(file_or_directory)
|
||||
else:
|
||||
os.remove(file_or_directory)
|
||||
|
||||
|
||||
def generate(whitelist: Set[str]):
|
||||
path_whitelist = {os.path.realpath(e) for e in whitelist if os.path.exists(e)}
|
||||
name_whitelist = {e for e in whitelist if not os.path.exists(e)}
|
||||
|
||||
test_case_names = set(get_directories(inputs_path))
|
||||
|
||||
for test_case_name in sorted(test_case_names):
|
||||
test_case_input_path = os.path.realpath(
|
||||
os.path.join(inputs_path, test_case_name)
|
||||
)
|
||||
|
||||
if (
|
||||
whitelist
|
||||
and test_case_input_path not in path_whitelist
|
||||
and test_case_name not in name_whitelist
|
||||
):
|
||||
continue
|
||||
|
||||
test_case_output_path_reference = os.path.join(
|
||||
output_path_reference, test_case_name
|
||||
)
|
||||
test_case_output_path_betterproto = os.path.join(
|
||||
output_path_betterproto, test_case_name
|
||||
)
|
||||
|
||||
print(f"Generating output for {test_case_name}")
|
||||
os.makedirs(test_case_output_path_reference, exist_ok=True)
|
||||
os.makedirs(test_case_output_path_betterproto, exist_ok=True)
|
||||
|
||||
clear_directory(test_case_output_path_reference)
|
||||
clear_directory(test_case_output_path_betterproto)
|
||||
|
||||
protoc_reference(test_case_input_path, test_case_output_path_reference)
|
||||
protoc_plugin(test_case_input_path, test_case_output_path_betterproto)
|
||||
|
||||
|
||||
HELP = "\n".join(
|
||||
[
|
||||
"Usage: python generate.py",
|
||||
" python generate.py [DIRECTORIES or NAMES]",
|
||||
"Generate python classes for standard tests.",
|
||||
"",
|
||||
"DIRECTORIES One or more relative or absolute directories of test-cases to generate classes for.",
|
||||
" python generate.py inputs/bool inputs/double inputs/enum",
|
||||
"",
|
||||
"NAMES One or more test-case names to generate classes for.",
|
||||
" python generate.py bool double enums",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
if set(sys.argv).intersection({"-h", "--help"}):
|
||||
print(HELP)
|
||||
return
|
||||
whitelist = set(sys.argv[1:])
|
||||
|
||||
generate(whitelist)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@@ -1,3 +0,0 @@
|
||||
{
|
||||
"greeting": "HEY"
|
||||
}
|
@@ -1,14 +0,0 @@
|
||||
syntax = "proto3";
|
||||
|
||||
// Enum for the different greeting types
|
||||
enum Greeting {
|
||||
HI = 0;
|
||||
HEY = 1;
|
||||
// Formal greeting
|
||||
HELLO = 2;
|
||||
}
|
||||
|
||||
message Test {
|
||||
// Greeting enum example
|
||||
Greeting greeting = 1;
|
||||
}
|
@@ -1,11 +0,0 @@
|
||||
syntax = "proto3";
|
||||
|
||||
import "root.proto";
|
||||
|
||||
package child;
|
||||
|
||||
// Tests generated imports when a message inside a child-package refers to a message defined in the root.
|
||||
|
||||
message Test {
|
||||
RootMessage message = 1;
|
||||
}
|
@@ -1,5 +0,0 @@
|
||||
{
|
||||
"for": 1,
|
||||
"with": 2,
|
||||
"as": 3
|
||||
}
|
@@ -1,11 +0,0 @@
|
||||
syntax = "proto3";
|
||||
|
||||
message Test {
|
||||
int32 for = 1;
|
||||
int32 with = 2;
|
||||
int32 as = 3;
|
||||
}
|
||||
|
||||
service TestService {
|
||||
rpc GetTest(Test) returns (Test) {}
|
||||
}
|
@@ -1,11 +0,0 @@
|
||||
{
|
||||
"root": {
|
||||
"name": "double-nested",
|
||||
"parent": {
|
||||
"child": [{"foo": "hello"}],
|
||||
"enumChild": ["A"],
|
||||
"rootParentChild": [{"a": "hello"}],
|
||||
"bar": true
|
||||
}
|
||||
}
|
||||
}
|
@@ -1,26 +0,0 @@
|
||||
syntax = "proto3";
|
||||
|
||||
message Test {
|
||||
message Root {
|
||||
message Parent {
|
||||
message RootParentChild {
|
||||
string a = 1;
|
||||
}
|
||||
enum EnumChild{
|
||||
A = 0;
|
||||
B = 1;
|
||||
}
|
||||
message Child {
|
||||
string foo = 1;
|
||||
}
|
||||
reserved 1;
|
||||
repeated Child child = 2;
|
||||
repeated EnumChild enumChild=3;
|
||||
repeated RootParentChild rootParentChild=4;
|
||||
bool bar = 5;
|
||||
}
|
||||
string name = 1;
|
||||
Parent parent = 2;
|
||||
}
|
||||
Root root = 1;
|
||||
}
|
@@ -1,15 +0,0 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package service;
|
||||
|
||||
message DoThingRequest {
|
||||
int32 iterations = 1;
|
||||
}
|
||||
|
||||
message DoThingResponse {
|
||||
int32 successfulIterations = 1;
|
||||
}
|
||||
|
||||
service Test {
|
||||
rpc DoThing (DoThingRequest) returns (DoThingResponse);
|
||||
}
|
@@ -1,132 +0,0 @@
|
||||
import betterproto
|
||||
import grpclib
|
||||
from grpclib.testing import ChannelFor
|
||||
import pytest
|
||||
from typing import Dict
|
||||
|
||||
from betterproto.tests.output_betterproto.service.service import (
|
||||
DoThingResponse,
|
||||
DoThingRequest,
|
||||
TestStub as ExampleServiceStub,
|
||||
)
|
||||
|
||||
|
||||
class ExampleService:
|
||||
def __init__(self, test_hook=None):
|
||||
# This lets us pass assertions to the servicer ;)
|
||||
self.test_hook = test_hook
|
||||
|
||||
async def DoThing(
|
||||
self, stream: "grpclib.server.Stream[DoThingRequest, DoThingResponse]"
|
||||
):
|
||||
request = await stream.recv_message()
|
||||
print("self.test_hook", self.test_hook)
|
||||
if self.test_hook is not None:
|
||||
self.test_hook(stream)
|
||||
for iteration in range(request.iterations):
|
||||
pass
|
||||
await stream.send_message(DoThingResponse(request.iterations))
|
||||
|
||||
def __mapping__(self) -> Dict[str, grpclib.const.Handler]:
|
||||
return {
|
||||
"/service.Test/DoThing": grpclib.const.Handler(
|
||||
self.DoThing,
|
||||
grpclib.const.Cardinality.UNARY_UNARY,
|
||||
DoThingRequest,
|
||||
DoThingResponse,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
async def _test_stub(stub, iterations=42, **kwargs):
|
||||
response = await stub.do_thing(iterations=iterations)
|
||||
assert response.successful_iterations == iterations
|
||||
|
||||
|
||||
def _get_server_side_test(deadline, metadata):
|
||||
def server_side_test(stream):
|
||||
assert stream.deadline._timestamp == pytest.approx(
|
||||
deadline._timestamp, 1
|
||||
), "The provided deadline should be recieved serverside"
|
||||
assert (
|
||||
stream.metadata["authorization"] == metadata["authorization"]
|
||||
), "The provided authorization metadata should be recieved serverside"
|
||||
|
||||
return server_side_test
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_simple_service_call():
|
||||
async with ChannelFor([ExampleService()]) as channel:
|
||||
await _test_stub(ExampleServiceStub(channel))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_service_call_with_upfront_request_params():
|
||||
# Setting deadline
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(22)
|
||||
metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ExampleService(test_hook=_get_server_side_test(deadline, metadata))]
|
||||
) as channel:
|
||||
await _test_stub(
|
||||
ExampleServiceStub(channel, deadline=deadline, metadata=metadata)
|
||||
)
|
||||
|
||||
# Setting timeout
|
||||
timeout = 99
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(timeout)
|
||||
metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ExampleService(test_hook=_get_server_side_test(deadline, metadata))]
|
||||
) as channel:
|
||||
await _test_stub(
|
||||
ExampleServiceStub(channel, timeout=timeout, metadata=metadata)
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_service_call_lower_level_with_overrides():
|
||||
ITERATIONS = 99
|
||||
|
||||
# Setting deadline
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(22)
|
||||
metadata = {"authorization": "12345"}
|
||||
kwarg_deadline = grpclib.metadata.Deadline.from_timeout(28)
|
||||
kwarg_metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ExampleService(test_hook=_get_server_side_test(deadline, metadata))]
|
||||
) as channel:
|
||||
stub = ExampleServiceStub(channel, deadline=deadline, metadata=metadata)
|
||||
response = await stub._unary_unary(
|
||||
"/service.Test/DoThing",
|
||||
DoThingRequest(ITERATIONS),
|
||||
DoThingResponse,
|
||||
deadline=kwarg_deadline,
|
||||
metadata=kwarg_metadata,
|
||||
)
|
||||
assert response.successful_iterations == ITERATIONS
|
||||
|
||||
# Setting timeout
|
||||
timeout = 99
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(timeout)
|
||||
metadata = {"authorization": "12345"}
|
||||
kwarg_timeout = 9000
|
||||
kwarg_deadline = grpclib.metadata.Deadline.from_timeout(kwarg_timeout)
|
||||
kwarg_metadata = {"authorization": "09876"}
|
||||
async with ChannelFor(
|
||||
[
|
||||
ExampleService(
|
||||
test_hook=_get_server_side_test(kwarg_deadline, kwarg_metadata)
|
||||
)
|
||||
]
|
||||
) as channel:
|
||||
stub = ExampleServiceStub(channel, deadline=deadline, metadata=metadata)
|
||||
response = await stub._unary_unary(
|
||||
"/service.Test/DoThing",
|
||||
DoThingRequest(ITERATIONS),
|
||||
DoThingResponse,
|
||||
timeout=kwarg_timeout,
|
||||
metadata=kwarg_metadata,
|
||||
)
|
||||
assert response.successful_iterations == ITERATIONS
|
@@ -1,10 +0,0 @@
|
||||
# Test cases that are expected to fail, e.g. unimplemented features or bug-fixes.
|
||||
# Remove from list when fixed.
|
||||
tests = {
|
||||
"import_root_sibling",
|
||||
"import_child_package_from_package",
|
||||
"import_root_package_from_child",
|
||||
"import_parent_package_from_child",
|
||||
"import_circular_dependency",
|
||||
"oneof_enum",
|
||||
}
|
@@ -1,61 +0,0 @@
|
||||
import os
|
||||
import subprocess
|
||||
from typing import Generator
|
||||
|
||||
os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
|
||||
|
||||
root_path = os.path.dirname(os.path.realpath(__file__))
|
||||
inputs_path = os.path.join(root_path, "inputs")
|
||||
output_path_reference = os.path.join(root_path, "output_reference")
|
||||
output_path_betterproto = os.path.join(root_path, "output_betterproto")
|
||||
|
||||
if os.name == "nt":
|
||||
plugin_path = os.path.join(root_path, "..", "plugin.bat")
|
||||
else:
|
||||
plugin_path = os.path.join(root_path, "..", "plugin.py")
|
||||
|
||||
|
||||
def get_files(path, end: str) -> Generator[str, None, None]:
|
||||
for r, dirs, files in os.walk(path):
|
||||
for filename in [f for f in files if f.endswith(end)]:
|
||||
yield os.path.join(r, filename)
|
||||
|
||||
|
||||
def get_directories(path):
|
||||
for root, directories, files in os.walk(path):
|
||||
for directory in directories:
|
||||
yield directory
|
||||
|
||||
|
||||
def relative(file: str, path: str):
|
||||
return os.path.join(os.path.dirname(file), path)
|
||||
|
||||
|
||||
def read_relative(file: str, path: str):
|
||||
with open(relative(file, path)) as fh:
|
||||
return fh.read()
|
||||
|
||||
|
||||
def protoc_plugin(path: str, output_dir: str):
|
||||
subprocess.run(
|
||||
f"protoc --plugin=protoc-gen-custom={plugin_path} --custom_out={output_dir} --proto_path={path} {path}/*.proto",
|
||||
shell=True,
|
||||
)
|
||||
|
||||
|
||||
def protoc_reference(path: str, output_dir: str):
|
||||
subprocess.run(
|
||||
f"protoc --python_out={output_dir} --proto_path={path} {path}/*.proto",
|
||||
shell=True,
|
||||
)
|
||||
|
||||
|
||||
def get_test_case_json_data(test_case_name, json_file_name=None):
|
||||
test_data_file_name = json_file_name if json_file_name else f"{test_case_name}.json"
|
||||
test_data_file_path = os.path.join(inputs_path, test_case_name, test_data_file_name)
|
||||
|
||||
if not os.path.exists(test_data_file_path):
|
||||
return None
|
||||
|
||||
with open(test_data_file_path) as fh:
|
||||
return fh.read()
|
31
docs/api.rst
Normal file
31
docs/api.rst
Normal file
@@ -0,0 +1,31 @@
|
||||
.. currentmodule:: betterproto
|
||||
|
||||
API reference
|
||||
=============
|
||||
|
||||
The following document outlines betterproto's api. **None** of these classes should be
|
||||
extended by the user manually.
|
||||
|
||||
|
||||
Message
|
||||
--------
|
||||
|
||||
.. autoclass:: betterproto.Message
|
||||
:members:
|
||||
:special-members: __bytes__, __bool__
|
||||
|
||||
|
||||
.. autofunction:: betterproto.serialized_on_wire
|
||||
|
||||
.. autofunction:: betterproto.which_one_of
|
||||
|
||||
|
||||
Enumerations
|
||||
-------------
|
||||
|
||||
.. autoclass:: betterproto.Enum()
|
||||
:members:
|
||||
|
||||
|
||||
.. autoclass:: betterproto.Casing()
|
||||
:members:
|
60
docs/conf.py
Normal file
60
docs/conf.py
Normal file
@@ -0,0 +1,60 @@
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file only contains a selection of the most common options. For a full
|
||||
# list see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
|
||||
import pathlib
|
||||
|
||||
import toml
|
||||
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = "betterproto"
|
||||
copyright = "2019 Daniel G. Taylor"
|
||||
author = "danielgtaylor"
|
||||
pyproject = toml.load(open(pathlib.Path(__file__).parent.parent / "pyproject.toml"))
|
||||
|
||||
|
||||
# The full version, including alpha/beta/rc tags.
|
||||
release = pyproject["tool"]["poetry"]["version"]
|
||||
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.intersphinx",
|
||||
"sphinx.ext.napoleon",
|
||||
]
|
||||
|
||||
autodoc_member_order = "bysource"
|
||||
autodoc_typehints = "none"
|
||||
|
||||
extlinks = {
|
||||
"issue": ("https://github.com/danielgtaylor/python-betterproto/issues/%s", "GH-"),
|
||||
}
|
||||
|
||||
# Links used for cross-referencing stuff in other documentation
|
||||
intersphinx_mapping = {
|
||||
"py": ("https://docs.python.org/3", None),
|
||||
}
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = "friendly"
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
|
||||
html_theme = "sphinx_rtd_theme"
|
33
docs/index.rst
Normal file
33
docs/index.rst
Normal file
@@ -0,0 +1,33 @@
|
||||
Welcome to betterproto's documentation!
|
||||
=======================================
|
||||
|
||||
betterproto is a protobuf compiler and interpreter. It improves the experience of using
|
||||
Protobuf and gRPC in Python, by generating readable, understandable, and idiomatic
|
||||
Python code, using modern language features.
|
||||
|
||||
|
||||
Features:
|
||||
~~~~~~~~~
|
||||
|
||||
- Generated messages are both binary & JSON serializable
|
||||
- Messages use relevant python types, e.g. ``Enum``, ``datetime`` and ``timedelta``
|
||||
objects
|
||||
- ``async``/``await`` support for gRPC Clients
|
||||
- Generates modern, readable, idiomatic python code
|
||||
|
||||
Contents:
|
||||
~~~~~~~~~
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
quick-start
|
||||
api
|
||||
migrating
|
||||
|
||||
|
||||
If you still can't find what you're looking for, try in one of the following pages:
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`modindex`
|
||||
* :ref:`search`
|
157
docs/migrating.rst
Normal file
157
docs/migrating.rst
Normal file
@@ -0,0 +1,157 @@
|
||||
Migrating Guide
|
||||
===============
|
||||
|
||||
Google's protocolbuffers
|
||||
------------------------
|
||||
|
||||
betterproto has a mostly 1 to 1 drop in replacement for Google's protocolbuffers (after
|
||||
regenerating your protobufs of course) although there are some minor differences.
|
||||
|
||||
.. note::
|
||||
|
||||
betterproto implements the same basic methods including:
|
||||
|
||||
- :meth:`betterproto.Message.FromString`
|
||||
- :meth:`betterproto.Message.SerializeToString`
|
||||
|
||||
for compatibility purposes, however it is important to note that these are
|
||||
effectively aliases for :meth:`betterproto.Message.parse` and
|
||||
:meth:`betterproto.Message.__bytes__` respectively.
|
||||
|
||||
|
||||
Determining if a message was sent
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Sometimes it is useful to be able to determine whether a message has been sent on
|
||||
the wire. This is how the Google wrapper types work to let you know whether a value is
|
||||
unset (set as the default/zero value), or set as something else, for example.
|
||||
|
||||
Use ``betterproto.serialized_on_wire(message)`` to determine if it was sent. This is
|
||||
a little bit different from the official Google generated Python code, and it lives
|
||||
outside the generated ``Message`` class to prevent name clashes. Note that it only
|
||||
supports Proto 3 and thus can only be used to check if ``Message`` fields are set.
|
||||
You cannot check if a scalar was sent on the wire.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Old way (official Google Protobuf package)
|
||||
>>> mymessage.HasField('myfield')
|
||||
True
|
||||
|
||||
# New way (this project)
|
||||
>>> betterproto.serialized_on_wire(mymessage.myfield)
|
||||
True
|
||||
|
||||
|
||||
One-of Support
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
Protobuf supports grouping fields in a oneof clause. Only one of the fields in the group
|
||||
may be set at a given time. For example, given the proto:
|
||||
|
||||
.. code-block:: proto
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
message Test {
|
||||
oneof foo {
|
||||
bool on = 1;
|
||||
int32 count = 2;
|
||||
string name = 3;
|
||||
}
|
||||
}
|
||||
|
||||
You can use ``betterproto.which_one_of(message, group_name)`` to determine which of the
|
||||
fields was set. It returns a tuple of the field name and value, or a blank string and
|
||||
``None`` if unset. Again this is a little different than the official Google code
|
||||
generator:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Old way (official Google protobuf package)
|
||||
>>> message.WhichOneof("group")
|
||||
"foo"
|
||||
|
||||
# New way (this project)
|
||||
>>> betterproto.which_one_of(message, "group")
|
||||
("foo", "foo's value")
|
||||
|
||||
|
||||
Well-Known Google Types
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Google provides several well-known message types like a timestamp, duration, and several
|
||||
wrappers used to provide optional zero value support. Each of these has a special JSON
|
||||
representation and is handled a little differently from normal messages. The Python
|
||||
mapping for these is as follows:
|
||||
|
||||
+-------------------------------+-----------------------------------------------+--------------------------+
|
||||
| ``Google Message`` | ``Python Type`` | ``Default`` |
|
||||
+===============================+===============================================+==========================+
|
||||
| ``google.protobuf.duration`` | :class:`datetime.timedelta` | ``0`` |
|
||||
+-------------------------------+-----------------------------------------------+--------------------------+
|
||||
| ``google.protobuf.timestamp`` | ``Timezone-aware`` :class:`datetime.datetime` | ``1970-01-01T00:00:00Z`` |
|
||||
+-------------------------------+-----------------------------------------------+--------------------------+
|
||||
| ``google.protobuf.*Value`` | ``Optional[...]``/``None`` | ``None`` |
|
||||
+-------------------------------+-----------------------------------------------+--------------------------+
|
||||
| ``google.protobuf.*`` | ``betterproto.lib.google.protobuf.*`` | ``None`` |
|
||||
+-------------------------------+-----------------------------------------------+--------------------------+
|
||||
|
||||
|
||||
For the wrapper types, the Python type corresponds to the wrapped type, e.g.
|
||||
``google.protobuf.BoolValue`` becomes ``Optional[bool]`` while
|
||||
``google.protobuf.Int32Value`` becomes ``Optional[int]``. All of the optional values
|
||||
default to None, so don't forget to check for that possible state.
|
||||
|
||||
Given:
|
||||
|
||||
.. code-block:: proto
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
import "google/protobuf/duration.proto";
|
||||
import "google/protobuf/timestamp.proto";
|
||||
import "google/protobuf/wrappers.proto";
|
||||
|
||||
message Test {
|
||||
google.protobuf.BoolValue maybe = 1;
|
||||
google.protobuf.Timestamp ts = 2;
|
||||
google.protobuf.Duration duration = 3;
|
||||
}
|
||||
|
||||
You can use it as such:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> t = Test().from_dict({"maybe": True, "ts": "2019-01-01T12:00:00Z", "duration": "1.200s"})
|
||||
>>> t
|
||||
Test(maybe=True, ts=datetime.datetime(2019, 1, 1, 12, 0, tzinfo=datetime.timezone.utc), duration=datetime.timedelta(seconds=1, microseconds=200000))
|
||||
|
||||
>>> t.ts - t.duration
|
||||
datetime.datetime(2019, 1, 1, 11, 59, 58, 800000, tzinfo=datetime.timezone.utc)
|
||||
|
||||
>>> t.ts.isoformat()
|
||||
'2019-01-01T12:00:00+00:00'
|
||||
|
||||
>>> t.maybe = None
|
||||
>>> t.to_dict()
|
||||
{'ts': '2019-01-01T12:00:00Z', 'duration': '1.200s'}
|
||||
|
||||
|
||||
[1.2.5] to [2.0.0b1]
|
||||
--------------------
|
||||
|
||||
Updated package structures
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Generated code now strictly follows the *package structure* of the ``.proto`` files.
|
||||
Consequently ``.proto`` files without a package will be combined in a single
|
||||
``__init__.py`` file. To avoid overwriting existing ``__init__.py`` files, its best
|
||||
to compile into a dedicated subdirectory.
|
||||
|
||||
Upgrading:
|
||||
|
||||
- Remove your previously compiled ``.py`` files.
|
||||
- Create a new *empty* directory, e.g. ``generated`` or ``lib/generated/proto`` etc.
|
||||
- Regenerate your python files into this directory
|
||||
- Update import statements, e.g. ``import ExampleMessage from generated``
|
192
docs/quick-start.rst
Normal file
192
docs/quick-start.rst
Normal file
@@ -0,0 +1,192 @@
|
||||
Getting Started
|
||||
===============
|
||||
|
||||
Installation
|
||||
++++++++++++
|
||||
|
||||
Installation from PyPI is as simple as running:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python3 -m pip install -U betterproto
|
||||
|
||||
If you are using Windows, then the following should be used instead:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
py -3 -m pip install -U betterproto
|
||||
|
||||
To include the protoc plugin, install betterproto[compiler] instead of betterproto,
|
||||
e.g.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python3 -m pip install -U "betterproto[compiler]"
|
||||
|
||||
Compiling proto files
|
||||
+++++++++++++++++++++
|
||||
|
||||
|
||||
Given you installed the compiler and have a proto file, e.g ``example.proto``:
|
||||
|
||||
.. code-block:: proto
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
package hello;
|
||||
|
||||
// Greeting represents a message you can tell a user.
|
||||
message Greeting {
|
||||
string message = 1;
|
||||
}
|
||||
|
||||
To compile the proto you would run the following:
|
||||
|
||||
You can run the following to invoke protoc directly:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mkdir hello
|
||||
protoc -I . --python_betterproto_out=lib example.proto
|
||||
|
||||
or run the following to invoke protoc via grpcio-tools:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
pip install grpcio-tools
|
||||
python -m grpc_tools.protoc -I . --python_betterproto_out=lib example.proto
|
||||
|
||||
|
||||
This will generate ``lib/__init__.py`` which looks like:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
||||
# sources: example.proto
|
||||
# plugin: python-betterproto
|
||||
from dataclasses import dataclass
|
||||
|
||||
import betterproto
|
||||
|
||||
|
||||
@dataclass
|
||||
class Greeting(betterproto.Message):
|
||||
"""Greeting represents a message you can tell a user."""
|
||||
|
||||
message: str = betterproto.string_field(1)
|
||||
|
||||
|
||||
Then to use it:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> from lib import Greeting
|
||||
|
||||
>>> test = Greeting()
|
||||
>>> test
|
||||
Greeting(message='')
|
||||
|
||||
>>> test.message = "Hey!"
|
||||
>>> test
|
||||
Greeting(message="Hey!")
|
||||
|
||||
>>> bytes(test)
|
||||
b'\n\x04Hey!'
|
||||
>>> Greeting().parse(serialized)
|
||||
Greeting(message="Hey!")
|
||||
|
||||
|
||||
Async gRPC Support
|
||||
++++++++++++++++++
|
||||
|
||||
The generated code includes `grpclib <https://grpclib.readthedocs.io/en/latest>`_ based
|
||||
stub (client) classes for rpc services declared in the input proto files.
|
||||
It is enabled by default.
|
||||
|
||||
|
||||
Given a service definition similar to the one below:
|
||||
|
||||
.. code-block:: proto
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
package echo;
|
||||
|
||||
message EchoRequest {
|
||||
string value = 1;
|
||||
// Number of extra times to echo
|
||||
uint32 extra_times = 2;
|
||||
}
|
||||
|
||||
message EchoResponse {
|
||||
repeated string values = 1;
|
||||
}
|
||||
|
||||
message EchoStreamResponse {
|
||||
string value = 1;
|
||||
}
|
||||
|
||||
service Echo {
|
||||
rpc Echo(EchoRequest) returns (EchoResponse);
|
||||
rpc EchoStream(EchoRequest) returns (stream EchoStreamResponse);
|
||||
}
|
||||
|
||||
The generated client can be used like so:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import asyncio
|
||||
from grpclib.client import Channel
|
||||
import echo
|
||||
|
||||
|
||||
async def main():
|
||||
channel = Channel(host="127.0.0.1", port=50051)
|
||||
service = echo.EchoStub(channel)
|
||||
response = await service.echo(value="hello", extra_times=1)
|
||||
print(response)
|
||||
|
||||
async for response in service.echo_stream(value="hello", extra_times=1):
|
||||
print(response)
|
||||
|
||||
# don't forget to close the channel when you're done!
|
||||
channel.close()
|
||||
|
||||
asyncio.run(main()) # python 3.7 only
|
||||
|
||||
# outputs
|
||||
EchoResponse(values=['hello', 'hello'])
|
||||
EchoStreamResponse(value='hello')
|
||||
EchoStreamResponse(value='hello')
|
||||
|
||||
|
||||
JSON
|
||||
++++
|
||||
Message objects include :meth:`betterproto.Message.to_json` and
|
||||
:meth:`betterproto.Message.from_json` methods for JSON (de)serialisation, and
|
||||
:meth:`betterproto.Message.to_dict`, :meth:`betterproto.Message.from_dict` for
|
||||
converting back and forth from JSON serializable dicts.
|
||||
|
||||
For compatibility the default is to convert field names to
|
||||
:attr:`betterproto.Casing.CAMEL`. You can control this behavior by passing a
|
||||
different casing value, e.g:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@dataclass
|
||||
class MyMessage(betterproto.Message):
|
||||
a_long_field_name: str = betterproto.string_field(1)
|
||||
|
||||
|
||||
>>> test = MyMessage(a_long_field_name="Hello World!")
|
||||
>>> test.to_dict(betterproto.Casing.SNAKE)
|
||||
{"a_long_field_name": "Hello World!"}
|
||||
>>> test.to_dict(betterproto.Casing.CAMEL)
|
||||
{"aLongFieldName": "Hello World!"}
|
||||
|
||||
>>> test.to_json(indent=2)
|
||||
'{\n "aLongFieldName": "Hello World!"\n}'
|
||||
|
||||
>>> test.from_dict({"aLongFieldName": "Goodbye World!"})
|
||||
>>> test.a_long_field_name
|
||||
"Goodbye World!"
|
1392
poetry.lock
generated
Normal file
1392
poetry.lock
generated
Normal file
File diff suppressed because it is too large
Load Diff
@@ -1,9 +1,81 @@
|
||||
[tool.poetry]
|
||||
name = "betterproto"
|
||||
version = "2.0.0b2"
|
||||
description = "A better Protobuf / gRPC generator & library"
|
||||
authors = ["Daniel G. Taylor <danielgtaylor@gmail.com>"]
|
||||
readme = "README.md"
|
||||
repository = "https://github.com/danielgtaylor/python-betterproto"
|
||||
keywords = ["protobuf", "gRPC"]
|
||||
license = "MIT"
|
||||
packages = [
|
||||
{ include = "betterproto", from = "src" }
|
||||
]
|
||||
|
||||
[tool.poetry.dependencies]
|
||||
python = "^3.6"
|
||||
backports-datetime-fromisoformat = { version = "^1.0.0", python = "<3.7" }
|
||||
black = { version = ">=19.3b0", optional = true }
|
||||
dataclasses = { version = "^0.7", python = ">=3.6, <3.7" }
|
||||
grpclib = "^0.4.1"
|
||||
jinja2 = { version = "^2.11.2", optional = true }
|
||||
protobuf = { version = "^3.12.2", optional = true }
|
||||
|
||||
[tool.poetry.dev-dependencies]
|
||||
black = "^20.8b1"
|
||||
bpython = "^0.19"
|
||||
grpcio-tools = "^1.30.0"
|
||||
jinja2 = "^2.11.2"
|
||||
mypy = "^0.770"
|
||||
poethepoet = "^0.5.0"
|
||||
protobuf = "^3.12.2"
|
||||
pytest = "^5.4.2"
|
||||
pytest-asyncio = "^0.12.0"
|
||||
pytest-cov = "^2.9.0"
|
||||
pytest-mock = "^3.1.1"
|
||||
tox = "^3.15.1"
|
||||
sphinx = "3.1.2"
|
||||
sphinx-rtd-theme = "0.5.0"
|
||||
asv = "^0.4.2"
|
||||
|
||||
[tool.poetry.scripts]
|
||||
protoc-gen-python_betterproto = "betterproto.plugin:main"
|
||||
|
||||
[tool.poetry.extras]
|
||||
compiler = ["black", "jinja2", "protobuf"]
|
||||
|
||||
[tool.poe.tasks]
|
||||
# Dev workflow tasks
|
||||
generate = { script = "tests.generate:main", help = "Generate test cases (do this once before running test)" }
|
||||
test = { cmd = "pytest --cov src", help = "Run tests" }
|
||||
types = { cmd = "mypy src --ignore-missing-imports", help = "Check types with mypy" }
|
||||
format = { cmd = "black . --exclude tests/output_", help = "Apply black formatting to source code" }
|
||||
clean = { cmd = "rm -rf .coverage .mypy_cache .pytest_cache dist betterproto.egg-info **/__pycache__ tests/output_*", help = "Clean out generated files from the workspace" }
|
||||
docs = { cmd = "sphinx-build docs docs/build", help = "Build the sphinx docs"}
|
||||
bench = { shell = "asv run master^! && asv run HEAD^! && asv compare master HEAD", help = "Benchmark current commit vs. master branch"}
|
||||
|
||||
# CI tasks
|
||||
full-test = { shell = "poe generate && tox", help = "Run tests with multiple pythons" }
|
||||
check-style = { cmd = "black . --check --diff --exclude tests/output_", help = "Check if code style is correct"}
|
||||
|
||||
[tool.black]
|
||||
target-version = ['py36']
|
||||
|
||||
[tool.isort]
|
||||
multi_line_output = 3
|
||||
include_trailing_comma = true
|
||||
force_grid_wrap = 0
|
||||
use_parentheses = true
|
||||
line_length = 88
|
||||
[tool.coverage.run]
|
||||
omit = ["betterproto/tests/*"]
|
||||
|
||||
[tool.tox]
|
||||
legacy_tox_ini = """
|
||||
[tox]
|
||||
isolated_build = true
|
||||
envlist = py36, py37, py38
|
||||
|
||||
[testenv]
|
||||
whitelist_externals = poetry
|
||||
commands =
|
||||
poetry install -v --extras compiler
|
||||
poetry run pytest --cov betterproto
|
||||
"""
|
||||
|
||||
[build-system]
|
||||
requires = ["poetry-core>=1.0.0,<2"]
|
||||
build-backend = "poetry.core.masonry.api"
|
||||
|
29
setup.py
29
setup.py
@@ -1,29 +0,0 @@
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
setup(
|
||||
name="betterproto",
|
||||
version="1.2.5",
|
||||
description="A better Protobuf / gRPC generator & library",
|
||||
long_description=open("README.md", "r", encoding="utf-8").read(),
|
||||
long_description_content_type="text/markdown",
|
||||
url="http://github.com/danielgtaylor/python-betterproto",
|
||||
author="Daniel G. Taylor",
|
||||
author_email="danielgtaylor@gmail.com",
|
||||
license="MIT",
|
||||
entry_points={
|
||||
"console_scripts": ["protoc-gen-python_betterproto=betterproto.plugin:main"]
|
||||
},
|
||||
packages=find_packages(
|
||||
exclude=["tests", "*.tests", "*.tests.*", "output", "output.*"]
|
||||
),
|
||||
package_data={"betterproto": ["py.typed", "templates/template.py.j2"]},
|
||||
python_requires=">=3.6",
|
||||
install_requires=[
|
||||
'dataclasses; python_version<"3.7"',
|
||||
'backports-datetime-fromisoformat; python_version<"3.7"',
|
||||
"grpclib",
|
||||
"stringcase",
|
||||
],
|
||||
extras_require={"compiler": ["black", "jinja2", "protobuf"]},
|
||||
zip_safe=False,
|
||||
)
|
File diff suppressed because it is too large
Load Diff
9
src/betterproto/_types.py
Normal file
9
src/betterproto/_types.py
Normal file
@@ -0,0 +1,9 @@
|
||||
from typing import TYPE_CHECKING, TypeVar
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from grpclib._typing import IProtoMessage
|
||||
from . import Message
|
||||
|
||||
# Bound type variable to allow methods to return `self` of subclasses
|
||||
T = TypeVar("T", bound="Message")
|
||||
ST = TypeVar("ST", bound="IProtoMessage")
|
138
src/betterproto/casing.py
Normal file
138
src/betterproto/casing.py
Normal file
@@ -0,0 +1,138 @@
|
||||
import keyword
|
||||
import re
|
||||
|
||||
# Word delimiters and symbols that will not be preserved when re-casing.
|
||||
# language=PythonRegExp
|
||||
SYMBOLS = "[^a-zA-Z0-9]*"
|
||||
|
||||
# Optionally capitalized word.
|
||||
# language=PythonRegExp
|
||||
WORD = "[A-Z]*[a-z]*[0-9]*"
|
||||
|
||||
# Uppercase word, not followed by lowercase letters.
|
||||
# language=PythonRegExp
|
||||
WORD_UPPER = "[A-Z]+(?![a-z])[0-9]*"
|
||||
|
||||
|
||||
def safe_snake_case(value: str) -> str:
|
||||
"""Snake case a value taking into account Python keywords."""
|
||||
value = snake_case(value)
|
||||
value = sanitize_name(value)
|
||||
return value
|
||||
|
||||
|
||||
def snake_case(value: str, strict: bool = True) -> str:
|
||||
"""
|
||||
Join words with an underscore into lowercase and remove symbols.
|
||||
|
||||
Parameters
|
||||
-----------
|
||||
value: :class:`str`
|
||||
The value to convert.
|
||||
strict: :class:`bool`
|
||||
Whether or not to force single underscores.
|
||||
|
||||
Returns
|
||||
--------
|
||||
:class:`str`
|
||||
The value in snake_case.
|
||||
"""
|
||||
|
||||
def substitute_word(symbols: str, word: str, is_start: bool) -> str:
|
||||
if not word:
|
||||
return ""
|
||||
if strict:
|
||||
delimiter_count = 0 if is_start else 1 # Single underscore if strict.
|
||||
elif is_start:
|
||||
delimiter_count = len(symbols)
|
||||
elif word.isupper() or word.islower():
|
||||
delimiter_count = max(
|
||||
1, len(symbols)
|
||||
) # Preserve all delimiters if not strict.
|
||||
else:
|
||||
delimiter_count = len(symbols) + 1 # Extra underscore for leading capital.
|
||||
|
||||
return ("_" * delimiter_count) + word.lower()
|
||||
|
||||
snake = re.sub(
|
||||
f"(^)?({SYMBOLS})({WORD_UPPER}|{WORD})",
|
||||
lambda groups: substitute_word(groups[2], groups[3], groups[1] is not None),
|
||||
value,
|
||||
)
|
||||
return snake
|
||||
|
||||
|
||||
def pascal_case(value: str, strict: bool = True) -> str:
|
||||
"""
|
||||
Capitalize each word and remove symbols.
|
||||
|
||||
Parameters
|
||||
-----------
|
||||
value: :class:`str`
|
||||
The value to convert.
|
||||
strict: :class:`bool`
|
||||
Whether or not to output only alphanumeric characters.
|
||||
|
||||
Returns
|
||||
--------
|
||||
:class:`str`
|
||||
The value in PascalCase.
|
||||
"""
|
||||
|
||||
def substitute_word(symbols, word):
|
||||
if strict:
|
||||
return word.capitalize() # Remove all delimiters
|
||||
|
||||
if word.islower():
|
||||
delimiter_length = len(symbols[:-1]) # Lose one delimiter
|
||||
else:
|
||||
delimiter_length = len(symbols) # Preserve all delimiters
|
||||
|
||||
return ("_" * delimiter_length) + word.capitalize()
|
||||
|
||||
return re.sub(
|
||||
f"({SYMBOLS})({WORD_UPPER}|{WORD})",
|
||||
lambda groups: substitute_word(groups[1], groups[2]),
|
||||
value,
|
||||
)
|
||||
|
||||
|
||||
def camel_case(value: str, strict: bool = True) -> str:
|
||||
"""
|
||||
Capitalize all words except first and remove symbols.
|
||||
|
||||
Parameters
|
||||
-----------
|
||||
value: :class:`str`
|
||||
The value to convert.
|
||||
strict: :class:`bool`
|
||||
Whether or not to output only alphanumeric characters.
|
||||
|
||||
Returns
|
||||
--------
|
||||
:class:`str`
|
||||
The value in camelCase.
|
||||
"""
|
||||
return lowercase_first(pascal_case(value, strict=strict))
|
||||
|
||||
|
||||
def lowercase_first(value: str) -> str:
|
||||
"""
|
||||
Lower cases the first character of the value.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
value: :class:`str`
|
||||
The value to lower case.
|
||||
|
||||
Returns
|
||||
-------
|
||||
:class:`str`
|
||||
The lower cased string.
|
||||
"""
|
||||
return value[0:1].lower() + value[1:]
|
||||
|
||||
|
||||
def sanitize_name(value: str) -> str:
|
||||
# https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles
|
||||
return f"{value}_" if keyword.iskeyword(value) else value
|
160
src/betterproto/compile/importing.py
Normal file
160
src/betterproto/compile/importing.py
Normal file
@@ -0,0 +1,160 @@
|
||||
import os
|
||||
import re
|
||||
from typing import Dict, List, Set, Tuple, Type
|
||||
|
||||
from ..casing import safe_snake_case
|
||||
from ..lib.google import protobuf as google_protobuf
|
||||
from .naming import pythonize_class_name
|
||||
|
||||
WRAPPER_TYPES: Dict[str, Type] = {
|
||||
".google.protobuf.DoubleValue": google_protobuf.DoubleValue,
|
||||
".google.protobuf.FloatValue": google_protobuf.FloatValue,
|
||||
".google.protobuf.Int32Value": google_protobuf.Int32Value,
|
||||
".google.protobuf.Int64Value": google_protobuf.Int64Value,
|
||||
".google.protobuf.UInt32Value": google_protobuf.UInt32Value,
|
||||
".google.protobuf.UInt64Value": google_protobuf.UInt64Value,
|
||||
".google.protobuf.BoolValue": google_protobuf.BoolValue,
|
||||
".google.protobuf.StringValue": google_protobuf.StringValue,
|
||||
".google.protobuf.BytesValue": google_protobuf.BytesValue,
|
||||
}
|
||||
|
||||
|
||||
def parse_source_type_name(field_type_name: str) -> Tuple[str, str]:
|
||||
"""
|
||||
Split full source type name into package and type name.
|
||||
E.g. 'root.package.Message' -> ('root.package', 'Message')
|
||||
'root.Message.SomeEnum' -> ('root', 'Message.SomeEnum')
|
||||
"""
|
||||
package_match = re.match(r"^\.?([^A-Z]+)\.(.+)", field_type_name)
|
||||
if package_match:
|
||||
package = package_match.group(1)
|
||||
name = package_match.group(2)
|
||||
else:
|
||||
package = ""
|
||||
name = field_type_name.lstrip(".")
|
||||
return package, name
|
||||
|
||||
|
||||
def get_type_reference(
|
||||
package: str, imports: set, source_type: str, unwrap: bool = True
|
||||
) -> str:
|
||||
"""
|
||||
Return a Python type name for a proto type reference. Adds the import if
|
||||
necessary. Unwraps well known type if required.
|
||||
"""
|
||||
if unwrap:
|
||||
if source_type in WRAPPER_TYPES:
|
||||
wrapped_type = type(WRAPPER_TYPES[source_type]().value)
|
||||
return f"Optional[{wrapped_type.__name__}]"
|
||||
|
||||
if source_type == ".google.protobuf.Duration":
|
||||
return "timedelta"
|
||||
|
||||
elif source_type == ".google.protobuf.Timestamp":
|
||||
return "datetime"
|
||||
|
||||
source_package, source_type = parse_source_type_name(source_type)
|
||||
|
||||
current_package: List[str] = package.split(".") if package else []
|
||||
py_package: List[str] = source_package.split(".") if source_package else []
|
||||
py_type: str = pythonize_class_name(source_type)
|
||||
|
||||
compiling_google_protobuf = current_package == ["google", "protobuf"]
|
||||
importing_google_protobuf = py_package == ["google", "protobuf"]
|
||||
if importing_google_protobuf and not compiling_google_protobuf:
|
||||
py_package = ["betterproto", "lib"] + py_package
|
||||
|
||||
if py_package[:1] == ["betterproto"]:
|
||||
return reference_absolute(imports, py_package, py_type)
|
||||
|
||||
if py_package == current_package:
|
||||
return reference_sibling(py_type)
|
||||
|
||||
if py_package[: len(current_package)] == current_package:
|
||||
return reference_descendent(current_package, imports, py_package, py_type)
|
||||
|
||||
if current_package[: len(py_package)] == py_package:
|
||||
return reference_ancestor(current_package, imports, py_package, py_type)
|
||||
|
||||
return reference_cousin(current_package, imports, py_package, py_type)
|
||||
|
||||
|
||||
def reference_absolute(imports: Set[str], py_package: List[str], py_type: str) -> str:
|
||||
"""
|
||||
Returns a reference to a python type located in the root, i.e. sys.path.
|
||||
"""
|
||||
string_import = ".".join(py_package)
|
||||
string_alias = safe_snake_case(string_import)
|
||||
imports.add(f"import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
|
||||
|
||||
def reference_sibling(py_type: str) -> str:
|
||||
"""
|
||||
Returns a reference to a python type within the same package as the current package.
|
||||
"""
|
||||
return f'"{py_type}"'
|
||||
|
||||
|
||||
def reference_descendent(
|
||||
current_package: List[str], imports: Set[str], py_package: List[str], py_type: str
|
||||
) -> str:
|
||||
"""
|
||||
Returns a reference to a python type in a package that is a descendent of the current package,
|
||||
and adds the required import that is aliased to avoid name conflicts.
|
||||
"""
|
||||
importing_descendent = py_package[len(current_package) :]
|
||||
string_from = ".".join(importing_descendent[:-1])
|
||||
string_import = importing_descendent[-1]
|
||||
if string_from:
|
||||
string_alias = "_".join(importing_descendent)
|
||||
imports.add(f"from .{string_from} import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
else:
|
||||
imports.add(f"from . import {string_import}")
|
||||
return f'"{string_import}.{py_type}"'
|
||||
|
||||
|
||||
def reference_ancestor(
|
||||
current_package: List[str], imports: Set[str], py_package: List[str], py_type: str
|
||||
) -> str:
|
||||
"""
|
||||
Returns a reference to a python type in a package which is an ancestor to the current package,
|
||||
and adds the required import that is aliased (if possible) to avoid name conflicts.
|
||||
|
||||
Adds trailing __ to avoid name mangling (python.org/dev/peps/pep-0008/#id34).
|
||||
"""
|
||||
distance_up = len(current_package) - len(py_package)
|
||||
if py_package:
|
||||
string_import = py_package[-1]
|
||||
string_alias = f"_{'_' * distance_up}{string_import}__"
|
||||
string_from = f"..{'.' * distance_up}"
|
||||
imports.add(f"from {string_from} import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
else:
|
||||
string_alias = f"{'_' * distance_up}{py_type}__"
|
||||
imports.add(f"from .{'.' * distance_up} import {py_type} as {string_alias}")
|
||||
return f'"{string_alias}"'
|
||||
|
||||
|
||||
def reference_cousin(
|
||||
current_package: List[str], imports: Set[str], py_package: List[str], py_type: str
|
||||
) -> str:
|
||||
"""
|
||||
Returns a reference to a python type in a package that is not descendent, ancestor or sibling,
|
||||
and adds the required import that is aliased to avoid name conflicts.
|
||||
"""
|
||||
shared_ancestry = os.path.commonprefix([current_package, py_package])
|
||||
distance_up = len(current_package) - len(shared_ancestry)
|
||||
string_from = f".{'.' * distance_up}" + ".".join(
|
||||
py_package[len(shared_ancestry) : -1]
|
||||
)
|
||||
string_import = py_package[-1]
|
||||
# Add trailing __ to avoid name mangling (python.org/dev/peps/pep-0008/#id34)
|
||||
string_alias = (
|
||||
f"{'_' * distance_up}"
|
||||
+ safe_snake_case(".".join(py_package[len(shared_ancestry) :]))
|
||||
+ "__"
|
||||
)
|
||||
imports.add(f"from {string_from} import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
13
src/betterproto/compile/naming.py
Normal file
13
src/betterproto/compile/naming.py
Normal file
@@ -0,0 +1,13 @@
|
||||
from betterproto import casing
|
||||
|
||||
|
||||
def pythonize_class_name(name: str) -> str:
|
||||
return casing.pascal_case(name)
|
||||
|
||||
|
||||
def pythonize_field_name(name: str) -> str:
|
||||
return casing.safe_snake_case(name)
|
||||
|
||||
|
||||
def pythonize_method_name(name: str) -> str:
|
||||
return casing.safe_snake_case(name)
|
0
src/betterproto/grpc/__init__.py
Normal file
0
src/betterproto/grpc/__init__.py
Normal file
171
src/betterproto/grpc/grpclib_client.py
Normal file
171
src/betterproto/grpc/grpclib_client.py
Normal file
@@ -0,0 +1,171 @@
|
||||
import asyncio
|
||||
from abc import ABC
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
AsyncIterable,
|
||||
AsyncIterator,
|
||||
Collection,
|
||||
Iterable,
|
||||
Mapping,
|
||||
Optional,
|
||||
Tuple,
|
||||
Type,
|
||||
Union,
|
||||
)
|
||||
|
||||
import grpclib.const
|
||||
|
||||
from .._types import ST, T
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from grpclib.client import Channel
|
||||
from grpclib.metadata import Deadline
|
||||
|
||||
|
||||
_Value = Union[str, bytes]
|
||||
_MetadataLike = Union[Mapping[str, _Value], Collection[Tuple[str, _Value]]]
|
||||
_MessageLike = Union[T, ST]
|
||||
_MessageSource = Union[Iterable[ST], AsyncIterable[ST]]
|
||||
|
||||
|
||||
class ServiceStub(ABC):
|
||||
"""
|
||||
Base class for async gRPC clients.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
channel: "Channel",
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
deadline: Optional["Deadline"] = None,
|
||||
metadata: Optional[_MetadataLike] = None,
|
||||
) -> None:
|
||||
self.channel = channel
|
||||
self.timeout = timeout
|
||||
self.deadline = deadline
|
||||
self.metadata = metadata
|
||||
|
||||
def __resolve_request_kwargs(
|
||||
self,
|
||||
timeout: Optional[float],
|
||||
deadline: Optional["Deadline"],
|
||||
metadata: Optional[_MetadataLike],
|
||||
):
|
||||
return {
|
||||
"timeout": self.timeout if timeout is None else timeout,
|
||||
"deadline": self.deadline if deadline is None else deadline,
|
||||
"metadata": self.metadata if metadata is None else metadata,
|
||||
}
|
||||
|
||||
async def _unary_unary(
|
||||
self,
|
||||
route: str,
|
||||
request: _MessageLike,
|
||||
response_type: Type[T],
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
deadline: Optional["Deadline"] = None,
|
||||
metadata: Optional[_MetadataLike] = None,
|
||||
) -> T:
|
||||
"""Make a unary request and return the response."""
|
||||
async with self.channel.request(
|
||||
route,
|
||||
grpclib.const.Cardinality.UNARY_UNARY,
|
||||
type(request),
|
||||
response_type,
|
||||
**self.__resolve_request_kwargs(timeout, deadline, metadata),
|
||||
) as stream:
|
||||
await stream.send_message(request, end=True)
|
||||
response = await stream.recv_message()
|
||||
assert response is not None
|
||||
return response
|
||||
|
||||
async def _unary_stream(
|
||||
self,
|
||||
route: str,
|
||||
request: _MessageLike,
|
||||
response_type: Type[T],
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
deadline: Optional["Deadline"] = None,
|
||||
metadata: Optional[_MetadataLike] = None,
|
||||
) -> AsyncIterator[T]:
|
||||
"""Make a unary request and return the stream response iterator."""
|
||||
async with self.channel.request(
|
||||
route,
|
||||
grpclib.const.Cardinality.UNARY_STREAM,
|
||||
type(request),
|
||||
response_type,
|
||||
**self.__resolve_request_kwargs(timeout, deadline, metadata),
|
||||
) as stream:
|
||||
await stream.send_message(request, end=True)
|
||||
async for message in stream:
|
||||
yield message
|
||||
|
||||
async def _stream_unary(
|
||||
self,
|
||||
route: str,
|
||||
request_iterator: _MessageSource,
|
||||
request_type: Type[ST],
|
||||
response_type: Type[T],
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
deadline: Optional["Deadline"] = None,
|
||||
metadata: Optional[_MetadataLike] = None,
|
||||
) -> T:
|
||||
"""Make a stream request and return the response."""
|
||||
async with self.channel.request(
|
||||
route,
|
||||
grpclib.const.Cardinality.STREAM_UNARY,
|
||||
request_type,
|
||||
response_type,
|
||||
**self.__resolve_request_kwargs(timeout, deadline, metadata),
|
||||
) as stream:
|
||||
await self._send_messages(stream, request_iterator)
|
||||
response = await stream.recv_message()
|
||||
assert response is not None
|
||||
return response
|
||||
|
||||
async def _stream_stream(
|
||||
self,
|
||||
route: str,
|
||||
request_iterator: _MessageSource,
|
||||
request_type: Type[ST],
|
||||
response_type: Type[T],
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
deadline: Optional["Deadline"] = None,
|
||||
metadata: Optional[_MetadataLike] = None,
|
||||
) -> AsyncIterator[T]:
|
||||
"""
|
||||
Make a stream request and return an AsyncIterator to iterate over response
|
||||
messages.
|
||||
"""
|
||||
async with self.channel.request(
|
||||
route,
|
||||
grpclib.const.Cardinality.STREAM_STREAM,
|
||||
request_type,
|
||||
response_type,
|
||||
**self.__resolve_request_kwargs(timeout, deadline, metadata),
|
||||
) as stream:
|
||||
await stream.send_request()
|
||||
sending_task = asyncio.ensure_future(
|
||||
self._send_messages(stream, request_iterator)
|
||||
)
|
||||
try:
|
||||
async for response in stream:
|
||||
yield response
|
||||
except:
|
||||
sending_task.cancel()
|
||||
raise
|
||||
|
||||
@staticmethod
|
||||
async def _send_messages(stream, messages: _MessageSource):
|
||||
if isinstance(messages, AsyncIterable):
|
||||
async for message in messages:
|
||||
await stream.send_message(message)
|
||||
else:
|
||||
for message in messages:
|
||||
await stream.send_message(message)
|
||||
await stream.end()
|
0
src/betterproto/grpc/util/__init__.py
Normal file
0
src/betterproto/grpc/util/__init__.py
Normal file
185
src/betterproto/grpc/util/async_channel.py
Normal file
185
src/betterproto/grpc/util/async_channel.py
Normal file
@@ -0,0 +1,185 @@
|
||||
import asyncio
|
||||
from typing import AsyncIterable, AsyncIterator, Iterable, Optional, TypeVar, Union
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
|
||||
class ChannelClosed(Exception):
|
||||
"""
|
||||
An exception raised on an attempt to send through a closed channel
|
||||
"""
|
||||
|
||||
|
||||
class ChannelDone(Exception):
|
||||
"""
|
||||
An exception raised on an attempt to send receive from a channel that is both closed
|
||||
and empty.
|
||||
"""
|
||||
|
||||
|
||||
class AsyncChannel(AsyncIterable[T]):
|
||||
"""
|
||||
A buffered async channel for sending items between coroutines with FIFO ordering.
|
||||
|
||||
This makes decoupled bidirectional steaming gRPC requests easy if used like:
|
||||
|
||||
.. code-block:: python
|
||||
client = GeneratedStub(grpclib_chan)
|
||||
request_channel = await AsyncChannel()
|
||||
# We can start be sending all the requests we already have
|
||||
await request_channel.send_from([RequestObject(...), RequestObject(...)])
|
||||
async for response in client.rpc_call(request_channel):
|
||||
# The response iterator will remain active until the connection is closed
|
||||
...
|
||||
# More items can be sent at any time
|
||||
await request_channel.send(RequestObject(...))
|
||||
...
|
||||
# The channel must be closed to complete the gRPC connection
|
||||
request_channel.close()
|
||||
|
||||
Items can be sent through the channel by either:
|
||||
- providing an iterable to the send_from method
|
||||
- passing them to the send method one at a time
|
||||
|
||||
Items can be received from the channel by either:
|
||||
- iterating over the channel with a for loop to get all items
|
||||
- calling the receive method to get one item at a time
|
||||
|
||||
If the channel is empty then receivers will wait until either an item appears or the
|
||||
channel is closed.
|
||||
|
||||
Once the channel is closed then subsequent attempt to send through the channel will
|
||||
fail with a ChannelClosed exception.
|
||||
|
||||
When th channel is closed and empty then it is done, and further attempts to receive
|
||||
from it will fail with a ChannelDone exception
|
||||
|
||||
If multiple coroutines receive from the channel concurrently, each item sent will be
|
||||
received by only one of the receivers.
|
||||
|
||||
:param source:
|
||||
An optional iterable will items that should be sent through the channel
|
||||
immediately.
|
||||
:param buffer_limit:
|
||||
Limit the number of items that can be buffered in the channel, A value less than
|
||||
1 implies no limit. If the channel is full then attempts to send more items will
|
||||
result in the sender waiting until an item is received from the channel.
|
||||
:param close:
|
||||
If set to True then the channel will automatically close after exhausting source
|
||||
or immediately if no source is provided.
|
||||
"""
|
||||
|
||||
def __init__(self, *, buffer_limit: int = 0, close: bool = False):
|
||||
self._queue: asyncio.Queue[Union[T, object]] = asyncio.Queue(buffer_limit)
|
||||
self._closed = False
|
||||
self._waiting_receivers: int = 0
|
||||
# Track whether flush has been invoked so it can only happen once
|
||||
self._flushed = False
|
||||
|
||||
def __aiter__(self) -> AsyncIterator[T]:
|
||||
return self
|
||||
|
||||
async def __anext__(self) -> T:
|
||||
if self.done():
|
||||
raise StopAsyncIteration
|
||||
self._waiting_receivers += 1
|
||||
try:
|
||||
result = await self._queue.get()
|
||||
if result is self.__flush:
|
||||
raise StopAsyncIteration
|
||||
return result
|
||||
finally:
|
||||
self._waiting_receivers -= 1
|
||||
self._queue.task_done()
|
||||
|
||||
def closed(self) -> bool:
|
||||
"""
|
||||
Returns True if this channel is closed and no-longer accepting new items
|
||||
"""
|
||||
return self._closed
|
||||
|
||||
def done(self) -> bool:
|
||||
"""
|
||||
Check if this channel is done.
|
||||
|
||||
:return: True if this channel is closed and and has been drained of items in
|
||||
which case any further attempts to receive an item from this channel will raise
|
||||
a ChannelDone exception.
|
||||
"""
|
||||
# After close the channel is not yet done until there is at least one waiting
|
||||
# receiver per enqueued item.
|
||||
return self._closed and self._queue.qsize() <= self._waiting_receivers
|
||||
|
||||
async def send_from(
|
||||
self, source: Union[Iterable[T], AsyncIterable[T]], close: bool = False
|
||||
) -> "AsyncChannel[T]":
|
||||
"""
|
||||
Iterates the given [Async]Iterable and sends all the resulting items.
|
||||
If close is set to True then subsequent send calls will be rejected with a
|
||||
ChannelClosed exception.
|
||||
:param source: an iterable of items to send
|
||||
:param close:
|
||||
if True then the channel will be closed after the source has been exhausted
|
||||
|
||||
"""
|
||||
if self._closed:
|
||||
raise ChannelClosed("Cannot send through a closed channel")
|
||||
if isinstance(source, AsyncIterable):
|
||||
async for item in source:
|
||||
await self._queue.put(item)
|
||||
else:
|
||||
for item in source:
|
||||
await self._queue.put(item)
|
||||
if close:
|
||||
# Complete the closing process
|
||||
self.close()
|
||||
return self
|
||||
|
||||
async def send(self, item: T) -> "AsyncChannel[T]":
|
||||
"""
|
||||
Send a single item over this channel.
|
||||
:param item: The item to send
|
||||
"""
|
||||
if self._closed:
|
||||
raise ChannelClosed("Cannot send through a closed channel")
|
||||
await self._queue.put(item)
|
||||
return self
|
||||
|
||||
async def receive(self) -> Optional[T]:
|
||||
"""
|
||||
Returns the next item from this channel when it becomes available,
|
||||
or None if the channel is closed before another item is sent.
|
||||
:return: An item from the channel
|
||||
"""
|
||||
if self.done():
|
||||
raise ChannelDone("Cannot receive from a closed channel")
|
||||
self._waiting_receivers += 1
|
||||
try:
|
||||
result = await self._queue.get()
|
||||
if result is self.__flush:
|
||||
return None
|
||||
return result
|
||||
finally:
|
||||
self._waiting_receivers -= 1
|
||||
self._queue.task_done()
|
||||
|
||||
def close(self):
|
||||
"""
|
||||
Close this channel to new items
|
||||
"""
|
||||
self._closed = True
|
||||
asyncio.ensure_future(self._flush_queue())
|
||||
|
||||
async def _flush_queue(self):
|
||||
"""
|
||||
To be called after the channel is closed. Pushes a number of self.__flush
|
||||
objects to the queue to ensure no waiting consumers get deadlocked.
|
||||
"""
|
||||
if not self._flushed:
|
||||
self._flushed = True
|
||||
deadlocked_receivers = max(0, self._waiting_receivers - self._queue.qsize())
|
||||
for _ in range(deadlocked_receivers):
|
||||
await self._queue.put(self.__flush)
|
||||
|
||||
# A special signal object for flushing the queue when the channel is closed
|
||||
__flush = object()
|
0
src/betterproto/lib/__init__.py
Normal file
0
src/betterproto/lib/__init__.py
Normal file
0
src/betterproto/lib/google/__init__.py
Normal file
0
src/betterproto/lib/google/__init__.py
Normal file
1312
src/betterproto/lib/google/protobuf/__init__.py
Normal file
1312
src/betterproto/lib/google/protobuf/__init__.py
Normal file
File diff suppressed because it is too large
Load Diff
1
src/betterproto/plugin/__init__.py
Normal file
1
src/betterproto/plugin/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
from .main import main
|
4
src/betterproto/plugin/__main__.py
Normal file
4
src/betterproto/plugin/__main__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
from .main import main
|
||||
|
||||
|
||||
main()
|
37
src/betterproto/plugin/compiler.py
Normal file
37
src/betterproto/plugin/compiler.py
Normal file
@@ -0,0 +1,37 @@
|
||||
import os.path
|
||||
|
||||
try:
|
||||
# betterproto[compiler] specific dependencies
|
||||
import black
|
||||
import jinja2
|
||||
except ImportError as err:
|
||||
print(
|
||||
"\033[31m"
|
||||
f"Unable to import `{err.name}` from betterproto plugin! "
|
||||
"Please ensure that you've installed betterproto as "
|
||||
'`pip install "betterproto[compiler]"` so that compiler dependencies '
|
||||
"are included."
|
||||
"\033[0m"
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
from .models import OutputTemplate
|
||||
|
||||
|
||||
def outputfile_compiler(output_file: OutputTemplate) -> str:
|
||||
|
||||
templates_folder = os.path.abspath(
|
||||
os.path.join(os.path.dirname(__file__), "..", "templates")
|
||||
)
|
||||
|
||||
env = jinja2.Environment(
|
||||
trim_blocks=True,
|
||||
lstrip_blocks=True,
|
||||
loader=jinja2.FileSystemLoader(templates_folder),
|
||||
)
|
||||
template = env.get_template("template.py.j2")
|
||||
|
||||
return black.format_str(
|
||||
template.render(output_file=output_file),
|
||||
mode=black.FileMode(target_versions={black.TargetVersion.PY37}),
|
||||
)
|
49
src/betterproto/plugin/main.py
Normal file
49
src/betterproto/plugin/main.py
Normal file
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
from google.protobuf.compiler import plugin_pb2 as plugin
|
||||
|
||||
from betterproto.plugin.parser import generate_code
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""The plugin's main entry point."""
|
||||
# Read request message from stdin
|
||||
data = sys.stdin.buffer.read()
|
||||
|
||||
# Parse request
|
||||
request = plugin.CodeGeneratorRequest()
|
||||
request.ParseFromString(data)
|
||||
|
||||
dump_file = os.getenv("BETTERPROTO_DUMP")
|
||||
if dump_file:
|
||||
dump_request(dump_file, request)
|
||||
|
||||
# Create response
|
||||
response = plugin.CodeGeneratorResponse()
|
||||
|
||||
# Generate code
|
||||
generate_code(request, response)
|
||||
|
||||
# Serialise response message
|
||||
output = response.SerializeToString()
|
||||
|
||||
# Write to stdout
|
||||
sys.stdout.buffer.write(output)
|
||||
|
||||
|
||||
def dump_request(dump_file: str, request: plugin.CodeGeneratorRequest) -> None:
|
||||
"""
|
||||
For developers: Supports running plugin.py standalone so its possible to debug it.
|
||||
Run protoc (or generate.py) with BETTERPROTO_DUMP="yourfile.bin" to write the request to a file.
|
||||
Then run plugin.py from your IDE in debugging mode, and redirect stdin to the file.
|
||||
"""
|
||||
with open(str(dump_file), "wb") as fh:
|
||||
sys.stderr.write(f"\033[31mWriting input from protoc to: {dump_file}\033[0m\n")
|
||||
fh.write(request.SerializeToString())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
706
src/betterproto/plugin/models.py
Normal file
706
src/betterproto/plugin/models.py
Normal file
@@ -0,0 +1,706 @@
|
||||
"""Plugin model dataclasses.
|
||||
|
||||
These classes are meant to be an intermediate representation
|
||||
of protobuf objects. They are used to organize the data collected during parsing.
|
||||
|
||||
The general intention is to create a doubly-linked tree-like structure
|
||||
with the following types of references:
|
||||
- Downwards references: from message -> fields, from output package -> messages
|
||||
or from service -> service methods
|
||||
- Upwards references: from field -> message, message -> package.
|
||||
- Input/output message references: from a service method to it's corresponding
|
||||
input/output messages, which may even be in another package.
|
||||
|
||||
There are convenience methods to allow climbing up and down this tree, for
|
||||
example to retrieve the list of all messages that are in the same package as
|
||||
the current message.
|
||||
|
||||
Most of these classes take as inputs:
|
||||
- proto_obj: A reference to it's corresponding protobuf object as
|
||||
presented by the protoc plugin.
|
||||
- parent: a reference to the parent object in the tree.
|
||||
|
||||
With this information, the class is able to expose attributes,
|
||||
such as a pythonized name, that will be calculated from proto_obj.
|
||||
|
||||
The instantiation should also attach a reference to the new object
|
||||
into the corresponding place within it's parent object. For example,
|
||||
instantiating field `A` with parent message `B` should add a
|
||||
reference to `A` to `B`'s `fields` attribute.
|
||||
"""
|
||||
|
||||
import re
|
||||
import textwrap
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Dict, Iterator, List, Optional, Set, Text, Type, Union
|
||||
|
||||
import betterproto
|
||||
|
||||
from ..casing import sanitize_name
|
||||
from ..compile.importing import get_type_reference, parse_source_type_name
|
||||
from ..compile.naming import (
|
||||
pythonize_class_name,
|
||||
pythonize_field_name,
|
||||
pythonize_method_name,
|
||||
)
|
||||
|
||||
try:
|
||||
# betterproto[compiler] specific dependencies
|
||||
from google.protobuf.compiler import plugin_pb2 as plugin
|
||||
from google.protobuf.descriptor_pb2 import (
|
||||
DescriptorProto,
|
||||
EnumDescriptorProto,
|
||||
FieldDescriptorProto,
|
||||
FileDescriptorProto,
|
||||
MethodDescriptorProto,
|
||||
)
|
||||
except ImportError as err:
|
||||
print(
|
||||
"\033[31m"
|
||||
f"Unable to import `{err.name}` from betterproto plugin! "
|
||||
"Please ensure that you've installed betterproto as "
|
||||
'`pip install "betterproto[compiler]"` so that compiler dependencies '
|
||||
"are included."
|
||||
"\033[0m"
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
# Create a unique placeholder to deal with
|
||||
# https://stackoverflow.com/questions/51575931/class-inheritance-in-python-3-7-dataclasses
|
||||
PLACEHOLDER = object()
|
||||
|
||||
# Organize proto types into categories
|
||||
PROTO_FLOAT_TYPES = (
|
||||
FieldDescriptorProto.TYPE_DOUBLE, # 1
|
||||
FieldDescriptorProto.TYPE_FLOAT, # 2
|
||||
)
|
||||
PROTO_INT_TYPES = (
|
||||
FieldDescriptorProto.TYPE_INT64, # 3
|
||||
FieldDescriptorProto.TYPE_UINT64, # 4
|
||||
FieldDescriptorProto.TYPE_INT32, # 5
|
||||
FieldDescriptorProto.TYPE_FIXED64, # 6
|
||||
FieldDescriptorProto.TYPE_FIXED32, # 7
|
||||
FieldDescriptorProto.TYPE_UINT32, # 13
|
||||
FieldDescriptorProto.TYPE_SFIXED32, # 15
|
||||
FieldDescriptorProto.TYPE_SFIXED64, # 16
|
||||
FieldDescriptorProto.TYPE_SINT32, # 17
|
||||
FieldDescriptorProto.TYPE_SINT64, # 18
|
||||
)
|
||||
PROTO_BOOL_TYPES = (FieldDescriptorProto.TYPE_BOOL,) # 8
|
||||
PROTO_STR_TYPES = (FieldDescriptorProto.TYPE_STRING,) # 9
|
||||
PROTO_BYTES_TYPES = (FieldDescriptorProto.TYPE_BYTES,) # 12
|
||||
PROTO_MESSAGE_TYPES = (
|
||||
FieldDescriptorProto.TYPE_MESSAGE, # 11
|
||||
FieldDescriptorProto.TYPE_ENUM, # 14
|
||||
)
|
||||
PROTO_MAP_TYPES = (FieldDescriptorProto.TYPE_MESSAGE,) # 11
|
||||
PROTO_PACKED_TYPES = (
|
||||
FieldDescriptorProto.TYPE_DOUBLE, # 1
|
||||
FieldDescriptorProto.TYPE_FLOAT, # 2
|
||||
FieldDescriptorProto.TYPE_INT64, # 3
|
||||
FieldDescriptorProto.TYPE_UINT64, # 4
|
||||
FieldDescriptorProto.TYPE_INT32, # 5
|
||||
FieldDescriptorProto.TYPE_FIXED64, # 6
|
||||
FieldDescriptorProto.TYPE_FIXED32, # 7
|
||||
FieldDescriptorProto.TYPE_BOOL, # 8
|
||||
FieldDescriptorProto.TYPE_UINT32, # 13
|
||||
FieldDescriptorProto.TYPE_SFIXED32, # 15
|
||||
FieldDescriptorProto.TYPE_SFIXED64, # 16
|
||||
FieldDescriptorProto.TYPE_SINT32, # 17
|
||||
FieldDescriptorProto.TYPE_SINT64, # 18
|
||||
)
|
||||
|
||||
|
||||
def get_comment(
|
||||
proto_file: "FileDescriptorProto", path: List[int], indent: int = 4
|
||||
) -> str:
|
||||
pad = " " * indent
|
||||
for sci in proto_file.source_code_info.location:
|
||||
if list(sci.path) == path and sci.leading_comments:
|
||||
lines = textwrap.wrap(
|
||||
sci.leading_comments.strip().replace("\n", ""), width=79 - indent
|
||||
)
|
||||
|
||||
if path[-2] == 2 and path[-4] != 6:
|
||||
# This is a field
|
||||
return f"{pad}# " + f"\n{pad}# ".join(lines)
|
||||
else:
|
||||
# This is a message, enum, service, or method
|
||||
if len(lines) == 1 and len(lines[0]) < 79 - indent - 6:
|
||||
lines[0] = lines[0].strip('"')
|
||||
return f'{pad}"""{lines[0]}"""'
|
||||
else:
|
||||
joined = f"\n{pad}".join(lines)
|
||||
return f'{pad}"""\n{pad}{joined}\n{pad}"""'
|
||||
|
||||
return ""
|
||||
|
||||
|
||||
class ProtoContentBase:
|
||||
"""Methods common to MessageCompiler, ServiceCompiler and ServiceMethodCompiler."""
|
||||
|
||||
path: List[int]
|
||||
comment_indent: int = 4
|
||||
parent: Union["betterproto.Message", "OutputTemplate"]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Checks that no fake default fields were left as placeholders."""
|
||||
for field_name, field_val in self.__dataclass_fields__.items():
|
||||
if field_val is PLACEHOLDER:
|
||||
raise ValueError(f"`{field_name}` is a required field.")
|
||||
|
||||
@property
|
||||
def output_file(self) -> "OutputTemplate":
|
||||
current = self
|
||||
while not isinstance(current, OutputTemplate):
|
||||
current = current.parent
|
||||
return current
|
||||
|
||||
@property
|
||||
def proto_file(self) -> FieldDescriptorProto:
|
||||
current = self
|
||||
while not isinstance(current, OutputTemplate):
|
||||
current = current.parent
|
||||
return current.package_proto_obj
|
||||
|
||||
@property
|
||||
def request(self) -> "PluginRequestCompiler":
|
||||
current = self
|
||||
while not isinstance(current, OutputTemplate):
|
||||
current = current.parent
|
||||
return current.parent_request
|
||||
|
||||
@property
|
||||
def comment(self) -> str:
|
||||
"""Crawl the proto source code and retrieve comments
|
||||
for this object.
|
||||
"""
|
||||
return get_comment(
|
||||
proto_file=self.proto_file, path=self.path, indent=self.comment_indent
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PluginRequestCompiler:
|
||||
|
||||
plugin_request_obj: plugin.CodeGeneratorRequest
|
||||
output_packages: Dict[str, "OutputTemplate"] = field(default_factory=dict)
|
||||
|
||||
@property
|
||||
def all_messages(self) -> List["MessageCompiler"]:
|
||||
"""All of the messages in this request.
|
||||
|
||||
Returns
|
||||
-------
|
||||
List[MessageCompiler]
|
||||
List of all of the messages in this request.
|
||||
"""
|
||||
return [
|
||||
msg for output in self.output_packages.values() for msg in output.messages
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class OutputTemplate:
|
||||
"""Representation of an output .py file.
|
||||
|
||||
Each output file corresponds to a .proto input file,
|
||||
but may need references to other .proto files to be
|
||||
built.
|
||||
"""
|
||||
|
||||
parent_request: PluginRequestCompiler
|
||||
package_proto_obj: FileDescriptorProto
|
||||
input_files: List[str] = field(default_factory=list)
|
||||
imports: Set[str] = field(default_factory=set)
|
||||
datetime_imports: Set[str] = field(default_factory=set)
|
||||
typing_imports: Set[str] = field(default_factory=set)
|
||||
messages: List["MessageCompiler"] = field(default_factory=list)
|
||||
enums: List["EnumDefinitionCompiler"] = field(default_factory=list)
|
||||
services: List["ServiceCompiler"] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def package(self) -> str:
|
||||
"""Name of input package.
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
Name of input package.
|
||||
"""
|
||||
return self.package_proto_obj.package
|
||||
|
||||
@property
|
||||
def input_filenames(self) -> List[str]:
|
||||
"""Names of the input files used to build this output.
|
||||
|
||||
Returns
|
||||
-------
|
||||
List[str]
|
||||
Names of the input files used to build this output.
|
||||
"""
|
||||
return [f.name for f in self.input_files]
|
||||
|
||||
@property
|
||||
def python_module_imports(self) -> Set[str]:
|
||||
imports = set()
|
||||
if any(x for x in self.messages if any(x.deprecated_fields)):
|
||||
imports.add("warnings")
|
||||
return imports
|
||||
|
||||
|
||||
@dataclass
|
||||
class MessageCompiler(ProtoContentBase):
|
||||
"""Representation of a protobuf message."""
|
||||
|
||||
parent: Union["MessageCompiler", OutputTemplate] = PLACEHOLDER
|
||||
proto_obj: DescriptorProto = PLACEHOLDER
|
||||
path: List[int] = PLACEHOLDER
|
||||
fields: List[Union["FieldCompiler", "MessageCompiler"]] = field(
|
||||
default_factory=list
|
||||
)
|
||||
deprecated: bool = field(default=False, init=False)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Add message to output file
|
||||
if isinstance(self.parent, OutputTemplate):
|
||||
if isinstance(self, EnumDefinitionCompiler):
|
||||
self.output_file.enums.append(self)
|
||||
else:
|
||||
self.output_file.messages.append(self)
|
||||
self.deprecated = self.proto_obj.options.deprecated
|
||||
super().__post_init__()
|
||||
|
||||
@property
|
||||
def proto_name(self) -> str:
|
||||
return self.proto_obj.name
|
||||
|
||||
@property
|
||||
def py_name(self) -> str:
|
||||
return pythonize_class_name(self.proto_name)
|
||||
|
||||
@property
|
||||
def annotation(self) -> str:
|
||||
if self.repeated:
|
||||
return f"List[{self.py_name}]"
|
||||
return self.py_name
|
||||
|
||||
@property
|
||||
def deprecated_fields(self) -> Iterator[str]:
|
||||
for f in self.fields:
|
||||
if f.deprecated:
|
||||
yield f.py_name
|
||||
|
||||
|
||||
def is_map(
|
||||
proto_field_obj: FieldDescriptorProto, parent_message: DescriptorProto
|
||||
) -> bool:
|
||||
"""True if proto_field_obj is a map, otherwise False."""
|
||||
if proto_field_obj.type == FieldDescriptorProto.TYPE_MESSAGE:
|
||||
# This might be a map...
|
||||
message_type = proto_field_obj.type_name.split(".").pop().lower()
|
||||
map_entry = f"{proto_field_obj.name.replace('_', '').lower()}entry"
|
||||
if message_type == map_entry:
|
||||
for nested in parent_message.nested_type: # parent message
|
||||
if (
|
||||
nested.name.replace("_", "").lower() == map_entry
|
||||
and nested.options.map_entry
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def is_oneof(proto_field_obj: FieldDescriptorProto) -> bool:
|
||||
"""True if proto_field_obj is a OneOf, otherwise False."""
|
||||
return proto_field_obj.HasField("oneof_index")
|
||||
|
||||
|
||||
@dataclass
|
||||
class FieldCompiler(MessageCompiler):
|
||||
parent: MessageCompiler = PLACEHOLDER
|
||||
proto_obj: FieldDescriptorProto = PLACEHOLDER
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Add field to message
|
||||
self.parent.fields.append(self)
|
||||
# Check for new imports
|
||||
annotation = self.annotation
|
||||
if "Optional[" in annotation:
|
||||
self.output_file.typing_imports.add("Optional")
|
||||
if "List[" in annotation:
|
||||
self.output_file.typing_imports.add("List")
|
||||
if "Dict[" in annotation:
|
||||
self.output_file.typing_imports.add("Dict")
|
||||
if "timedelta" in annotation:
|
||||
self.output_file.datetime_imports.add("timedelta")
|
||||
if "datetime" in annotation:
|
||||
self.output_file.datetime_imports.add("datetime")
|
||||
super().__post_init__() # call FieldCompiler-> MessageCompiler __post_init__
|
||||
|
||||
def get_field_string(self, indent: int = 4) -> str:
|
||||
"""Construct string representation of this field as a field."""
|
||||
name = f"{self.py_name}"
|
||||
annotations = f": {self.annotation}"
|
||||
field_args = ", ".join(
|
||||
([""] + self.betterproto_field_args) if self.betterproto_field_args else []
|
||||
)
|
||||
betterproto_field_type = (
|
||||
f"betterproto.{self.field_type}_field({self.proto_obj.number}{field_args})"
|
||||
)
|
||||
return f"{name}{annotations} = {betterproto_field_type}"
|
||||
|
||||
@property
|
||||
def betterproto_field_args(self) -> List[str]:
|
||||
args = []
|
||||
if self.field_wraps:
|
||||
args.append(f"wraps={self.field_wraps}")
|
||||
return args
|
||||
|
||||
@property
|
||||
def field_wraps(self) -> Optional[str]:
|
||||
"""Returns betterproto wrapped field type or None."""
|
||||
match_wrapper = re.match(
|
||||
r"\.google\.protobuf\.(.+)Value", self.proto_obj.type_name
|
||||
)
|
||||
if match_wrapper:
|
||||
wrapped_type = "TYPE_" + match_wrapper.group(1).upper()
|
||||
if hasattr(betterproto, wrapped_type):
|
||||
return f"betterproto.{wrapped_type}"
|
||||
return None
|
||||
|
||||
@property
|
||||
def repeated(self) -> bool:
|
||||
return (
|
||||
self.proto_obj.label == FieldDescriptorProto.LABEL_REPEATED
|
||||
and not is_map(self.proto_obj, self.parent)
|
||||
)
|
||||
|
||||
@property
|
||||
def mutable(self) -> bool:
|
||||
"""True if the field is a mutable type, otherwise False."""
|
||||
return self.annotation.startswith(("List[", "Dict["))
|
||||
|
||||
@property
|
||||
def field_type(self) -> str:
|
||||
"""String representation of proto field type."""
|
||||
return (
|
||||
self.proto_obj.Type.Name(self.proto_obj.type).lower().replace("type_", "")
|
||||
)
|
||||
|
||||
@property
|
||||
def default_value_string(self) -> Union[Text, None, float, int]:
|
||||
"""Python representation of the default proto value."""
|
||||
if self.repeated:
|
||||
return "[]"
|
||||
if self.py_type == "int":
|
||||
return "0"
|
||||
if self.py_type == "float":
|
||||
return "0.0"
|
||||
elif self.py_type == "bool":
|
||||
return "False"
|
||||
elif self.py_type == "str":
|
||||
return '""'
|
||||
elif self.py_type == "bytes":
|
||||
return 'b""'
|
||||
else:
|
||||
# Message type
|
||||
return "None"
|
||||
|
||||
@property
|
||||
def packed(self) -> bool:
|
||||
"""True if the wire representation is a packed format."""
|
||||
return self.repeated and self.proto_obj.type in PROTO_PACKED_TYPES
|
||||
|
||||
@property
|
||||
def py_name(self) -> str:
|
||||
"""Pythonized name."""
|
||||
return pythonize_field_name(self.proto_name)
|
||||
|
||||
@property
|
||||
def proto_name(self) -> str:
|
||||
"""Original protobuf name."""
|
||||
return self.proto_obj.name
|
||||
|
||||
@property
|
||||
def py_type(self) -> str:
|
||||
"""String representation of Python type."""
|
||||
if self.proto_obj.type in PROTO_FLOAT_TYPES:
|
||||
return "float"
|
||||
elif self.proto_obj.type in PROTO_INT_TYPES:
|
||||
return "int"
|
||||
elif self.proto_obj.type in PROTO_BOOL_TYPES:
|
||||
return "bool"
|
||||
elif self.proto_obj.type in PROTO_STR_TYPES:
|
||||
return "str"
|
||||
elif self.proto_obj.type in PROTO_BYTES_TYPES:
|
||||
return "bytes"
|
||||
elif self.proto_obj.type in PROTO_MESSAGE_TYPES:
|
||||
# Type referencing another defined Message or a named enum
|
||||
return get_type_reference(
|
||||
package=self.output_file.package,
|
||||
imports=self.output_file.imports,
|
||||
source_type=self.proto_obj.type_name,
|
||||
)
|
||||
else:
|
||||
raise NotImplementedError(f"Unknown type {field.type}")
|
||||
|
||||
@property
|
||||
def annotation(self) -> str:
|
||||
if self.repeated:
|
||||
return f"List[{self.py_type}]"
|
||||
return self.py_type
|
||||
|
||||
|
||||
@dataclass
|
||||
class OneOfFieldCompiler(FieldCompiler):
|
||||
@property
|
||||
def betterproto_field_args(self) -> List[str]:
|
||||
args = super().betterproto_field_args
|
||||
group = self.parent.proto_obj.oneof_decl[self.proto_obj.oneof_index].name
|
||||
args.append(f'group="{group}"')
|
||||
return args
|
||||
|
||||
|
||||
@dataclass
|
||||
class MapEntryCompiler(FieldCompiler):
|
||||
py_k_type: Type = PLACEHOLDER
|
||||
py_v_type: Type = PLACEHOLDER
|
||||
proto_k_type: str = PLACEHOLDER
|
||||
proto_v_type: str = PLACEHOLDER
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Explore nested types and set k_type and v_type if unset."""
|
||||
map_entry = f"{self.proto_obj.name.replace('_', '').lower()}entry"
|
||||
for nested in self.parent.proto_obj.nested_type:
|
||||
if (
|
||||
nested.name.replace("_", "").lower() == map_entry
|
||||
and nested.options.map_entry
|
||||
):
|
||||
# Get Python types
|
||||
self.py_k_type = FieldCompiler(
|
||||
parent=self, proto_obj=nested.field[0] # key
|
||||
).py_type
|
||||
self.py_v_type = FieldCompiler(
|
||||
parent=self, proto_obj=nested.field[1] # value
|
||||
).py_type
|
||||
# Get proto types
|
||||
self.proto_k_type = self.proto_obj.Type.Name(nested.field[0].type)
|
||||
self.proto_v_type = self.proto_obj.Type.Name(nested.field[1].type)
|
||||
super().__post_init__() # call FieldCompiler-> MessageCompiler __post_init__
|
||||
|
||||
@property
|
||||
def betterproto_field_args(self) -> List[str]:
|
||||
return [f"betterproto.{self.proto_k_type}", f"betterproto.{self.proto_v_type}"]
|
||||
|
||||
@property
|
||||
def field_type(self) -> str:
|
||||
return "map"
|
||||
|
||||
@property
|
||||
def annotation(self) -> str:
|
||||
return f"Dict[{self.py_k_type}, {self.py_v_type}]"
|
||||
|
||||
@property
|
||||
def repeated(self) -> bool:
|
||||
return False # maps cannot be repeated
|
||||
|
||||
|
||||
@dataclass
|
||||
class EnumDefinitionCompiler(MessageCompiler):
|
||||
"""Representation of a proto Enum definition."""
|
||||
|
||||
proto_obj: EnumDescriptorProto = PLACEHOLDER
|
||||
entries: List["EnumDefinitionCompiler.EnumEntry"] = PLACEHOLDER
|
||||
|
||||
@dataclass(unsafe_hash=True)
|
||||
class EnumEntry:
|
||||
"""Representation of an Enum entry."""
|
||||
|
||||
name: str
|
||||
value: int
|
||||
comment: str
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Get entries/allowed values for this Enum
|
||||
self.entries = [
|
||||
self.EnumEntry(
|
||||
name=sanitize_name(entry_proto_value.name),
|
||||
value=entry_proto_value.number,
|
||||
comment=get_comment(
|
||||
proto_file=self.proto_file, path=self.path + [2, entry_number]
|
||||
),
|
||||
)
|
||||
for entry_number, entry_proto_value in enumerate(self.proto_obj.value)
|
||||
]
|
||||
super().__post_init__() # call MessageCompiler __post_init__
|
||||
|
||||
@property
|
||||
def default_value_string(self) -> str:
|
||||
"""Python representation of the default value for Enums.
|
||||
|
||||
As per the spec, this is the first value of the Enum.
|
||||
"""
|
||||
return str(self.entries[0].value) # ideally, should ALWAYS be int(0)!
|
||||
|
||||
|
||||
@dataclass
|
||||
class ServiceCompiler(ProtoContentBase):
|
||||
parent: OutputTemplate = PLACEHOLDER
|
||||
proto_obj: DescriptorProto = PLACEHOLDER
|
||||
path: List[int] = PLACEHOLDER
|
||||
methods: List["ServiceMethodCompiler"] = field(default_factory=list)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Add service to output file
|
||||
self.output_file.services.append(self)
|
||||
super().__post_init__() # check for unset fields
|
||||
|
||||
@property
|
||||
def proto_name(self) -> str:
|
||||
return self.proto_obj.name
|
||||
|
||||
@property
|
||||
def py_name(self) -> str:
|
||||
return pythonize_class_name(self.proto_name)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ServiceMethodCompiler(ProtoContentBase):
|
||||
|
||||
parent: ServiceCompiler
|
||||
proto_obj: MethodDescriptorProto
|
||||
path: List[int] = PLACEHOLDER
|
||||
comment_indent: int = 8
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Add method to service
|
||||
self.parent.methods.append(self)
|
||||
|
||||
# Check for Optional import
|
||||
if self.py_input_message:
|
||||
for f in self.py_input_message.fields:
|
||||
if f.default_value_string == "None":
|
||||
self.output_file.typing_imports.add("Optional")
|
||||
if "Optional" in self.py_output_message_type:
|
||||
self.output_file.typing_imports.add("Optional")
|
||||
self.mutable_default_args # ensure this is called before rendering
|
||||
|
||||
# Check for Async imports
|
||||
if self.client_streaming:
|
||||
self.output_file.typing_imports.add("AsyncIterable")
|
||||
self.output_file.typing_imports.add("Iterable")
|
||||
self.output_file.typing_imports.add("Union")
|
||||
if self.server_streaming:
|
||||
self.output_file.typing_imports.add("AsyncIterator")
|
||||
|
||||
super().__post_init__() # check for unset fields
|
||||
|
||||
@property
|
||||
def mutable_default_args(self) -> Dict[str, str]:
|
||||
"""Handle mutable default arguments.
|
||||
|
||||
Returns a list of tuples containing the name and default value
|
||||
for arguments to this message who's default value is mutable.
|
||||
The defaults are swapped out for None and replaced back inside
|
||||
the method's body.
|
||||
Reference:
|
||||
https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments
|
||||
|
||||
Returns
|
||||
-------
|
||||
Dict[str, str]
|
||||
Name and actual default value (as a string)
|
||||
for each argument with mutable default values.
|
||||
"""
|
||||
mutable_default_args = {}
|
||||
|
||||
if self.py_input_message:
|
||||
for f in self.py_input_message.fields:
|
||||
if (
|
||||
not self.client_streaming
|
||||
and f.default_value_string != "None"
|
||||
and f.mutable
|
||||
):
|
||||
mutable_default_args[f.py_name] = f.default_value_string
|
||||
self.output_file.typing_imports.add("Optional")
|
||||
|
||||
return mutable_default_args
|
||||
|
||||
@property
|
||||
def py_name(self) -> str:
|
||||
"""Pythonized method name."""
|
||||
return pythonize_method_name(self.proto_obj.name)
|
||||
|
||||
@property
|
||||
def proto_name(self) -> str:
|
||||
"""Original protobuf name."""
|
||||
return self.proto_obj.name
|
||||
|
||||
@property
|
||||
def route(self) -> str:
|
||||
return f"/{self.output_file.package}.{self.parent.proto_name}/{self.proto_name}"
|
||||
|
||||
@property
|
||||
def py_input_message(self) -> Optional[MessageCompiler]:
|
||||
"""Find the input message object.
|
||||
|
||||
Returns
|
||||
-------
|
||||
Optional[MessageCompiler]
|
||||
Method instance representing the input message.
|
||||
If not input message could be found or there are no
|
||||
input messages, None is returned.
|
||||
"""
|
||||
package, name = parse_source_type_name(self.proto_obj.input_type)
|
||||
|
||||
# Nested types are currently flattened without dots.
|
||||
# Todo: keep a fully quantified name in types, that is
|
||||
# comparable with method.input_type
|
||||
for msg in self.request.all_messages:
|
||||
if (
|
||||
msg.py_name == name.replace(".", "")
|
||||
and msg.output_file.package == package
|
||||
):
|
||||
return msg
|
||||
return None
|
||||
|
||||
@property
|
||||
def py_input_message_type(self) -> str:
|
||||
"""String representation of the Python type corresponding to the
|
||||
input message.
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
String representation of the Python type corresponding to the input message.
|
||||
"""
|
||||
return get_type_reference(
|
||||
package=self.output_file.package,
|
||||
imports=self.output_file.imports,
|
||||
source_type=self.proto_obj.input_type,
|
||||
).strip('"')
|
||||
|
||||
@property
|
||||
def py_output_message_type(self) -> str:
|
||||
"""String representation of the Python type corresponding to the
|
||||
output message.
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
String representation of the Python type corresponding to the output message.
|
||||
"""
|
||||
return get_type_reference(
|
||||
package=self.output_file.package,
|
||||
imports=self.output_file.imports,
|
||||
source_type=self.proto_obj.output_type,
|
||||
unwrap=False,
|
||||
).strip('"')
|
||||
|
||||
@property
|
||||
def client_streaming(self) -> bool:
|
||||
return self.proto_obj.client_streaming
|
||||
|
||||
@property
|
||||
def server_streaming(self) -> bool:
|
||||
return self.proto_obj.server_streaming
|
176
src/betterproto/plugin/parser.py
Normal file
176
src/betterproto/plugin/parser.py
Normal file
@@ -0,0 +1,176 @@
|
||||
import itertools
|
||||
import pathlib
|
||||
import sys
|
||||
from typing import TYPE_CHECKING, Iterator, List, Tuple, Union, Set
|
||||
|
||||
try:
|
||||
# betterproto[compiler] specific dependencies
|
||||
from google.protobuf.compiler import plugin_pb2 as plugin
|
||||
from google.protobuf.descriptor_pb2 import (
|
||||
DescriptorProto,
|
||||
EnumDescriptorProto,
|
||||
FieldDescriptorProto,
|
||||
ServiceDescriptorProto,
|
||||
)
|
||||
except ImportError as err:
|
||||
print(
|
||||
"\033[31m"
|
||||
f"Unable to import `{err.name}` from betterproto plugin! "
|
||||
"Please ensure that you've installed betterproto as "
|
||||
'`pip install "betterproto[compiler]"` so that compiler dependencies '
|
||||
"are included."
|
||||
"\033[0m"
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
from .compiler import outputfile_compiler
|
||||
from .models import (
|
||||
EnumDefinitionCompiler,
|
||||
FieldCompiler,
|
||||
MapEntryCompiler,
|
||||
MessageCompiler,
|
||||
OneOfFieldCompiler,
|
||||
OutputTemplate,
|
||||
PluginRequestCompiler,
|
||||
ServiceCompiler,
|
||||
ServiceMethodCompiler,
|
||||
is_map,
|
||||
is_oneof,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from google.protobuf.descriptor import Descriptor
|
||||
|
||||
|
||||
def traverse(
|
||||
proto_file: FieldDescriptorProto,
|
||||
) -> "itertools.chain[Tuple[Union[str, EnumDescriptorProto], List[int]]]":
|
||||
# Todo: Keep information about nested hierarchy
|
||||
def _traverse(
|
||||
path: List[int], items: List["Descriptor"], prefix=""
|
||||
) -> Iterator[Tuple[Union[str, EnumDescriptorProto], List[int]]]:
|
||||
for i, item in enumerate(items):
|
||||
# Adjust the name since we flatten the hierarchy.
|
||||
# Todo: don't change the name, but include full name in returned tuple
|
||||
item.name = next_prefix = prefix + item.name
|
||||
yield item, path + [i]
|
||||
|
||||
if isinstance(item, DescriptorProto):
|
||||
for enum in item.enum_type:
|
||||
enum.name = next_prefix + enum.name
|
||||
yield enum, path + [i, 4]
|
||||
|
||||
if item.nested_type:
|
||||
for n, p in _traverse(path + [i, 3], item.nested_type, next_prefix):
|
||||
yield n, p
|
||||
|
||||
return itertools.chain(
|
||||
_traverse([5], proto_file.enum_type), _traverse([4], proto_file.message_type)
|
||||
)
|
||||
|
||||
|
||||
def generate_code(
|
||||
request: plugin.CodeGeneratorRequest, response: plugin.CodeGeneratorResponse
|
||||
) -> None:
|
||||
plugin_options = request.parameter.split(",") if request.parameter else []
|
||||
|
||||
request_data = PluginRequestCompiler(plugin_request_obj=request)
|
||||
# Gather output packages
|
||||
for proto_file in request.proto_file:
|
||||
if (
|
||||
proto_file.package == "google.protobuf"
|
||||
and "INCLUDE_GOOGLE" not in plugin_options
|
||||
):
|
||||
# If not INCLUDE_GOOGLE,
|
||||
# skip re-compiling Google's well-known types
|
||||
continue
|
||||
|
||||
output_package_name = proto_file.package
|
||||
if output_package_name not in request_data.output_packages:
|
||||
# Create a new output if there is no output for this package
|
||||
request_data.output_packages[output_package_name] = OutputTemplate(
|
||||
parent_request=request_data, package_proto_obj=proto_file
|
||||
)
|
||||
# Add this input file to the output corresponding to this package
|
||||
request_data.output_packages[output_package_name].input_files.append(proto_file)
|
||||
|
||||
# Read Messages and Enums
|
||||
# We need to read Messages before Services in so that we can
|
||||
# get the references to input/output messages for each service
|
||||
for output_package_name, output_package in request_data.output_packages.items():
|
||||
for proto_input_file in output_package.input_files:
|
||||
for item, path in traverse(proto_input_file):
|
||||
read_protobuf_type(item=item, path=path, output_package=output_package)
|
||||
|
||||
# Read Services
|
||||
for output_package_name, output_package in request_data.output_packages.items():
|
||||
for proto_input_file in output_package.input_files:
|
||||
for index, service in enumerate(proto_input_file.service):
|
||||
read_protobuf_service(service, index, output_package)
|
||||
|
||||
# Generate output files
|
||||
output_paths: Set[pathlib.Path] = set()
|
||||
for output_package_name, output_package in request_data.output_packages.items():
|
||||
|
||||
# Add files to the response object
|
||||
output_path = pathlib.Path(*output_package_name.split("."), "__init__.py")
|
||||
output_paths.add(output_path)
|
||||
|
||||
f: response.File = response.file.add()
|
||||
f.name = str(output_path)
|
||||
|
||||
# Render and then format the output file
|
||||
f.content = outputfile_compiler(output_file=output_package)
|
||||
|
||||
# Make each output directory a package with __init__ file
|
||||
init_files = {
|
||||
directory.joinpath("__init__.py")
|
||||
for path in output_paths
|
||||
for directory in path.parents
|
||||
} - output_paths
|
||||
|
||||
for init_file in init_files:
|
||||
init = response.file.add()
|
||||
init.name = str(init_file)
|
||||
|
||||
for output_package_name in sorted(output_paths.union(init_files)):
|
||||
print(f"Writing {output_package_name}", file=sys.stderr)
|
||||
|
||||
|
||||
def read_protobuf_type(
|
||||
item: DescriptorProto, path: List[int], output_package: OutputTemplate
|
||||
) -> None:
|
||||
if isinstance(item, DescriptorProto):
|
||||
if item.options.map_entry:
|
||||
# Skip generated map entry messages since we just use dicts
|
||||
return
|
||||
# Process Message
|
||||
message_data = MessageCompiler(parent=output_package, proto_obj=item, path=path)
|
||||
for index, field in enumerate(item.field):
|
||||
if is_map(field, item):
|
||||
MapEntryCompiler(
|
||||
parent=message_data, proto_obj=field, path=path + [2, index]
|
||||
)
|
||||
elif is_oneof(field):
|
||||
OneOfFieldCompiler(
|
||||
parent=message_data, proto_obj=field, path=path + [2, index]
|
||||
)
|
||||
else:
|
||||
FieldCompiler(
|
||||
parent=message_data, proto_obj=field, path=path + [2, index]
|
||||
)
|
||||
elif isinstance(item, EnumDescriptorProto):
|
||||
# Enum
|
||||
EnumDefinitionCompiler(parent=output_package, proto_obj=item, path=path)
|
||||
|
||||
|
||||
def read_protobuf_service(
|
||||
service: ServiceDescriptorProto, index: int, output_package: OutputTemplate
|
||||
) -> None:
|
||||
service_data = ServiceCompiler(
|
||||
parent=output_package, proto_obj=service, path=[6, index]
|
||||
)
|
||||
for j, method in enumerate(service.method):
|
||||
ServiceMethodCompiler(
|
||||
parent=service_data, proto_obj=method, path=[6, index, 2, j]
|
||||
)
|
2
src/betterproto/plugin/plugin.bat
Normal file
2
src/betterproto/plugin/plugin.bat
Normal file
@@ -0,0 +1,2 @@
|
||||
@SET plugin_dir=%~dp0
|
||||
@python -m %plugin_dir% %*
|
0
src/betterproto/py.typed
Normal file
0
src/betterproto/py.typed
Normal file
159
src/betterproto/templates/template.py.j2
Normal file
159
src/betterproto/templates/template.py.j2
Normal file
@@ -0,0 +1,159 @@
|
||||
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
||||
# sources: {{ ', '.join(output_file.input_filenames) }}
|
||||
# plugin: python-betterproto
|
||||
{% for i in output_file.python_module_imports|sort %}
|
||||
import {{ i }}
|
||||
{% endfor %}
|
||||
from dataclasses import dataclass
|
||||
{% if output_file.datetime_imports %}
|
||||
from datetime import {% for i in output_file.datetime_imports|sort %}{{ i }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||
|
||||
{% endif%}
|
||||
{% if output_file.typing_imports %}
|
||||
from typing import {% for i in output_file.typing_imports|sort %}{{ i }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||
|
||||
{% endif %}
|
||||
|
||||
import betterproto
|
||||
{% if output_file.services %}
|
||||
import grpclib
|
||||
{% endif %}
|
||||
|
||||
|
||||
{% if output_file.enums %}{% for enum in output_file.enums %}
|
||||
class {{ enum.py_name }}(betterproto.Enum):
|
||||
{% if enum.comment %}
|
||||
{{ enum.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for entry in enum.entries %}
|
||||
{% if entry.comment %}
|
||||
{{ entry.comment }}
|
||||
{% endif %}
|
||||
{{ entry.name }} = {{ entry.value }}
|
||||
{% endfor %}
|
||||
|
||||
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% for message in output_file.messages %}
|
||||
@dataclass(eq=False, repr=False)
|
||||
class {{ message.py_name }}(betterproto.Message):
|
||||
{% if message.comment %}
|
||||
{{ message.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for field in message.fields %}
|
||||
{% if field.comment %}
|
||||
{{ field.comment }}
|
||||
{% endif %}
|
||||
{{ field.get_field_string() }}
|
||||
{% endfor %}
|
||||
{% if not message.fields %}
|
||||
pass
|
||||
{% endif %}
|
||||
|
||||
{% if message.deprecated or message.deprecated_fields %}
|
||||
def __post_init__(self) -> None:
|
||||
{% if message.deprecated %}
|
||||
warnings.warn("{{ message.py_name }} is deprecated", DeprecationWarning)
|
||||
{% endif %}
|
||||
super().__post_init__()
|
||||
{% for field in message.deprecated_fields %}
|
||||
if self.{{ field }}:
|
||||
warnings.warn("{{ message.py_name }}.{{ field }} is deprecated", DeprecationWarning)
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
|
||||
{% endfor %}
|
||||
{% for service in output_file.services %}
|
||||
class {{ service.py_name }}Stub(betterproto.ServiceStub):
|
||||
{% if service.comment %}
|
||||
{{ service.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for method in service.methods %}
|
||||
async def {{ method.py_name }}(self
|
||||
{%- if not method.client_streaming -%}
|
||||
{%- if method.py_input_message and method.py_input_message.fields -%}, *,
|
||||
{%- for field in method.py_input_message.fields -%}
|
||||
{{ field.py_name }}: {% if field.py_name in method.mutable_default_args and not field.annotation.startswith("Optional[") -%}
|
||||
Optional[{{ field.annotation }}]
|
||||
{%- else -%}
|
||||
{{ field.annotation }}
|
||||
{%- endif -%} =
|
||||
{%- if field.py_name not in method.mutable_default_args -%}
|
||||
{{ field.default_value_string }}
|
||||
{%- else -%}
|
||||
None
|
||||
{% endif -%}
|
||||
{%- if not loop.last %}, {% endif -%}
|
||||
{%- endfor -%}
|
||||
{%- endif -%}
|
||||
{%- else -%}
|
||||
{# Client streaming: need a request iterator instead #}
|
||||
, request_iterator: Union[AsyncIterable["{{ method.py_input_message_type }}"], Iterable["{{ method.py_input_message_type }}"]]
|
||||
{%- endif -%}
|
||||
) -> {% if method.server_streaming %}AsyncIterator["{{ method.py_output_message_type }}"]{% else %}"{{ method.py_output_message_type }}"{% endif %}:
|
||||
{% if method.comment %}
|
||||
{{ method.comment }}
|
||||
|
||||
{% endif %}
|
||||
{%- for py_name, zero in method.mutable_default_args.items() %}
|
||||
{{ py_name }} = {{ py_name }} or {{ zero }}
|
||||
{% endfor %}
|
||||
|
||||
{% if not method.client_streaming %}
|
||||
request = {{ method.py_input_message_type }}()
|
||||
{% for field in method.py_input_message.fields %}
|
||||
{% if field.field_type == 'message' %}
|
||||
if {{ field.py_name }} is not None:
|
||||
request.{{ field.py_name }} = {{ field.py_name }}
|
||||
{% else %}
|
||||
request.{{ field.py_name }} = {{ field.py_name }}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
{% if method.server_streaming %}
|
||||
{% if method.client_streaming %}
|
||||
async for response in self._stream_stream(
|
||||
"{{ method.route }}",
|
||||
request_iterator,
|
||||
{{ method.py_input_message_type }},
|
||||
{{ method.py_output_message_type.strip('"') }},
|
||||
):
|
||||
yield response
|
||||
{% else %}{# i.e. not client streaming #}
|
||||
async for response in self._unary_stream(
|
||||
"{{ method.route }}",
|
||||
request,
|
||||
{{ method.py_output_message_type.strip('"') }},
|
||||
):
|
||||
yield response
|
||||
|
||||
{% endif %}{# if client streaming #}
|
||||
{% else %}{# i.e. not server streaming #}
|
||||
{% if method.client_streaming %}
|
||||
return await self._stream_unary(
|
||||
"{{ method.route }}",
|
||||
request_iterator,
|
||||
{{ method.py_input_message_type }},
|
||||
{{ method.py_output_message_type.strip('"') }}
|
||||
)
|
||||
{% else %}{# i.e. not client streaming #}
|
||||
return await self._unary_unary(
|
||||
"{{ method.route }}",
|
||||
request,
|
||||
{{ method.py_output_message_type.strip('"') }}
|
||||
)
|
||||
{% endif %}{# client streaming #}
|
||||
{% endif %}
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
{% for i in output_file.imports|sort %}
|
||||
{{ i }}
|
||||
{% endfor %}
|
@@ -12,12 +12,12 @@ inputs/
|
||||
|
||||
## Test case directory structure
|
||||
|
||||
Each testcase has a `<name>.proto` file with a message called `Test`, a matching `.json` file and optionally a custom test file called `test_*.py`.
|
||||
Each testcase has a `<name>.proto` file with a message called `Test`, and optionally a matching `.json` file and a custom test called `test_*.py`.
|
||||
|
||||
```bash
|
||||
bool/
|
||||
bool.proto
|
||||
bool.json
|
||||
bool.json # optional
|
||||
test_bool.py # optional
|
||||
```
|
||||
|
||||
@@ -50,7 +50,7 @@ You can add multiple `.proto` files to the test case, as long as one file matche
|
||||
`test_<name>.py` — *Custom test to validate specific aspects of the generated class*
|
||||
|
||||
```python
|
||||
from betterproto.tests.output_betterproto.bool.bool import Test
|
||||
from tests.output_betterproto.bool.bool import Test
|
||||
|
||||
def test_value():
|
||||
message = Test()
|
||||
@@ -61,21 +61,22 @@ def test_value():
|
||||
|
||||
The following tests are automatically executed for all cases:
|
||||
|
||||
- [x] Can the generated python code imported?
|
||||
- [x] Can the generated python code be imported?
|
||||
- [x] Can the generated message class be instantiated?
|
||||
- [x] Is the generated code compatible with the Google's `grpc_tools.protoc` implementation?
|
||||
- _when `.json` is present_
|
||||
|
||||
## Running the tests
|
||||
|
||||
- `pipenv run generate`
|
||||
This generates
|
||||
- `pipenv run generate`
|
||||
This generates:
|
||||
- `betterproto/tests/output_betterproto` — *the plugin generated python classes*
|
||||
- `betterproto/tests/output_reference` — *reference implementation classes*
|
||||
- `pipenv run test`
|
||||
|
||||
## Intentionally Failing tests
|
||||
|
||||
The standard test suite includes tests that fail by intention. These tests document known bugs and missing features that are intended to be corrented in the future.
|
||||
The standard test suite includes tests that fail by intention. These tests document known bugs and missing features that are intended to be corrected in the future.
|
||||
|
||||
When running `pytest`, they show up as `x` or `X` in the test results.
|
||||
|
||||
@@ -87,4 +88,4 @@ betterproto/tests/test_inputs.py ..x...x..x...x.X........xx........x.....x......
|
||||
- `x` — XFAIL: expected failure
|
||||
- `X` — XPASS: expected failure, but still passed
|
||||
|
||||
Test cases marked for expected failure are declared in [inputs/xfail.py](inputs.xfail.py)
|
||||
Test cases marked for expected failure are declared in [inputs/config.py](inputs/config.py)
|
0
tests/__init__.py
Normal file
0
tests/__init__.py
Normal file
146
tests/generate.py
Executable file
146
tests/generate.py
Executable file
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env python
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import shutil
|
||||
import sys
|
||||
from typing import Set
|
||||
|
||||
from tests.util import (
|
||||
get_directories,
|
||||
inputs_path,
|
||||
output_path_betterproto,
|
||||
output_path_reference,
|
||||
protoc,
|
||||
)
|
||||
|
||||
# Force pure-python implementation instead of C++, otherwise imports
|
||||
# break things because we can't properly reset the symbol database.
|
||||
os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
|
||||
|
||||
|
||||
def clear_directory(dir_path: Path):
|
||||
for file_or_directory in dir_path.glob("*"):
|
||||
if file_or_directory.is_dir():
|
||||
shutil.rmtree(file_or_directory)
|
||||
else:
|
||||
file_or_directory.unlink()
|
||||
|
||||
|
||||
async def generate(whitelist: Set[str], verbose: bool):
|
||||
test_case_names = set(get_directories(inputs_path)) - {"__pycache__"}
|
||||
|
||||
path_whitelist = set()
|
||||
name_whitelist = set()
|
||||
for item in whitelist:
|
||||
if item in test_case_names:
|
||||
name_whitelist.add(item)
|
||||
continue
|
||||
path_whitelist.add(item)
|
||||
|
||||
generation_tasks = []
|
||||
for test_case_name in sorted(test_case_names):
|
||||
test_case_input_path = inputs_path.joinpath(test_case_name).resolve()
|
||||
if (
|
||||
whitelist
|
||||
and str(test_case_input_path) not in path_whitelist
|
||||
and test_case_name not in name_whitelist
|
||||
):
|
||||
continue
|
||||
generation_tasks.append(
|
||||
generate_test_case_output(test_case_input_path, test_case_name, verbose)
|
||||
)
|
||||
|
||||
failed_test_cases = []
|
||||
# Wait for all subprocs and match any failures to names to report
|
||||
for test_case_name, result in zip(
|
||||
sorted(test_case_names), await asyncio.gather(*generation_tasks)
|
||||
):
|
||||
if result != 0:
|
||||
failed_test_cases.append(test_case_name)
|
||||
|
||||
if failed_test_cases:
|
||||
sys.stderr.write(
|
||||
"\n\033[31;1;4mFailed to generate the following test cases:\033[0m\n"
|
||||
)
|
||||
for failed_test_case in failed_test_cases:
|
||||
sys.stderr.write(f"- {failed_test_case}\n")
|
||||
|
||||
|
||||
async def generate_test_case_output(
|
||||
test_case_input_path: Path, test_case_name: str, verbose: bool
|
||||
) -> int:
|
||||
"""
|
||||
Returns the max of the subprocess return values
|
||||
"""
|
||||
|
||||
test_case_output_path_reference = output_path_reference.joinpath(test_case_name)
|
||||
test_case_output_path_betterproto = output_path_betterproto.joinpath(test_case_name)
|
||||
|
||||
os.makedirs(test_case_output_path_reference, exist_ok=True)
|
||||
os.makedirs(test_case_output_path_betterproto, exist_ok=True)
|
||||
|
||||
clear_directory(test_case_output_path_reference)
|
||||
clear_directory(test_case_output_path_betterproto)
|
||||
|
||||
(
|
||||
(ref_out, ref_err, ref_code),
|
||||
(plg_out, plg_err, plg_code),
|
||||
) = await asyncio.gather(
|
||||
protoc(test_case_input_path, test_case_output_path_reference, True),
|
||||
protoc(test_case_input_path, test_case_output_path_betterproto, False),
|
||||
)
|
||||
|
||||
message = f"Generated output for {test_case_name!r}"
|
||||
if verbose:
|
||||
print(f"\033[31;1;4m{message}\033[0m")
|
||||
if ref_out:
|
||||
sys.stdout.buffer.write(ref_out)
|
||||
if ref_err:
|
||||
sys.stderr.buffer.write(ref_err)
|
||||
if plg_out:
|
||||
sys.stdout.buffer.write(plg_out)
|
||||
if plg_err:
|
||||
sys.stderr.buffer.write(plg_err)
|
||||
sys.stdout.buffer.flush()
|
||||
sys.stderr.buffer.flush()
|
||||
else:
|
||||
print(message)
|
||||
|
||||
return max(ref_code, plg_code)
|
||||
|
||||
|
||||
HELP = "\n".join(
|
||||
(
|
||||
"Usage: python generate.py [-h] [-v] [DIRECTORIES or NAMES]",
|
||||
"Generate python classes for standard tests.",
|
||||
"",
|
||||
"DIRECTORIES One or more relative or absolute directories of test-cases to generate classes for.",
|
||||
" python generate.py inputs/bool inputs/double inputs/enum",
|
||||
"",
|
||||
"NAMES One or more test-case names to generate classes for.",
|
||||
" python generate.py bool double enums",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
if set(sys.argv).intersection({"-h", "--help"}):
|
||||
print(HELP)
|
||||
return
|
||||
if sys.argv[1:2] == ["-v"]:
|
||||
verbose = True
|
||||
whitelist = set(sys.argv[2:])
|
||||
else:
|
||||
verbose = False
|
||||
whitelist = set(sys.argv[1:])
|
||||
|
||||
if platform.system() == "Windows":
|
||||
asyncio.set_event_loop(asyncio.ProactorEventLoop())
|
||||
|
||||
asyncio.get_event_loop().run_until_complete(generate(whitelist, verbose))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
0
tests/grpc/__init__.py
Normal file
0
tests/grpc/__init__.py
Normal file
218
tests/grpc/test_grpclib_client.py
Normal file
218
tests/grpc/test_grpclib_client.py
Normal file
@@ -0,0 +1,218 @@
|
||||
import asyncio
|
||||
import sys
|
||||
|
||||
from tests.output_betterproto.service.service import (
|
||||
DoThingRequest,
|
||||
DoThingResponse,
|
||||
GetThingRequest,
|
||||
TestStub as ThingServiceClient,
|
||||
)
|
||||
import grpclib
|
||||
import grpclib.metadata
|
||||
import grpclib.server
|
||||
from grpclib.testing import ChannelFor
|
||||
import pytest
|
||||
from betterproto.grpc.util.async_channel import AsyncChannel
|
||||
from .thing_service import ThingService
|
||||
|
||||
|
||||
async def _test_client(client, name="clean room", **kwargs):
|
||||
response = await client.do_thing(name=name)
|
||||
assert response.names == [name]
|
||||
|
||||
|
||||
def _assert_request_meta_received(deadline, metadata):
|
||||
def server_side_test(stream):
|
||||
assert stream.deadline._timestamp == pytest.approx(
|
||||
deadline._timestamp, 1
|
||||
), "The provided deadline should be received serverside"
|
||||
assert (
|
||||
stream.metadata["authorization"] == metadata["authorization"]
|
||||
), "The provided authorization metadata should be received serverside"
|
||||
|
||||
return server_side_test
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def handler_trailer_only_unauthenticated():
|
||||
async def handler(stream: grpclib.server.Stream):
|
||||
await stream.recv_message()
|
||||
await stream.send_initial_metadata()
|
||||
await stream.send_trailing_metadata(status=grpclib.Status.UNAUTHENTICATED)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_simple_service_call():
|
||||
async with ChannelFor([ThingService()]) as channel:
|
||||
await _test_client(ThingServiceClient(channel))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trailer_only_error_unary_unary(
|
||||
mocker, handler_trailer_only_unauthenticated
|
||||
):
|
||||
service = ThingService()
|
||||
mocker.patch.object(
|
||||
service,
|
||||
"do_thing",
|
||||
side_effect=handler_trailer_only_unauthenticated,
|
||||
autospec=True,
|
||||
)
|
||||
async with ChannelFor([service]) as channel:
|
||||
with pytest.raises(grpclib.exceptions.GRPCError) as e:
|
||||
await ThingServiceClient(channel).do_thing(name="something")
|
||||
assert e.value.status == grpclib.Status.UNAUTHENTICATED
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trailer_only_error_stream_unary(
|
||||
mocker, handler_trailer_only_unauthenticated
|
||||
):
|
||||
service = ThingService()
|
||||
mocker.patch.object(
|
||||
service,
|
||||
"do_many_things",
|
||||
side_effect=handler_trailer_only_unauthenticated,
|
||||
autospec=True,
|
||||
)
|
||||
async with ChannelFor([service]) as channel:
|
||||
with pytest.raises(grpclib.exceptions.GRPCError) as e:
|
||||
await ThingServiceClient(channel).do_many_things(
|
||||
request_iterator=[DoThingRequest(name="something")]
|
||||
)
|
||||
await _test_client(ThingServiceClient(channel))
|
||||
assert e.value.status == grpclib.Status.UNAUTHENTICATED
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skipif(
|
||||
sys.version_info < (3, 8), reason="async mock spy does works for python3.8+"
|
||||
)
|
||||
async def test_service_call_mutable_defaults(mocker):
|
||||
async with ChannelFor([ThingService()]) as channel:
|
||||
client = ThingServiceClient(channel)
|
||||
spy = mocker.spy(client, "_unary_unary")
|
||||
await _test_client(client)
|
||||
comments = spy.call_args_list[-1].args[1].comments
|
||||
await _test_client(client)
|
||||
assert spy.call_args_list[-1].args[1].comments is not comments
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_service_call_with_upfront_request_params():
|
||||
# Setting deadline
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(22)
|
||||
metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ThingService(test_hook=_assert_request_meta_received(deadline, metadata))]
|
||||
) as channel:
|
||||
await _test_client(
|
||||
ThingServiceClient(channel, deadline=deadline, metadata=metadata)
|
||||
)
|
||||
|
||||
# Setting timeout
|
||||
timeout = 99
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(timeout)
|
||||
metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ThingService(test_hook=_assert_request_meta_received(deadline, metadata))]
|
||||
) as channel:
|
||||
await _test_client(
|
||||
ThingServiceClient(channel, timeout=timeout, metadata=metadata)
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_service_call_lower_level_with_overrides():
|
||||
THING_TO_DO = "get milk"
|
||||
|
||||
# Setting deadline
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(22)
|
||||
metadata = {"authorization": "12345"}
|
||||
kwarg_deadline = grpclib.metadata.Deadline.from_timeout(28)
|
||||
kwarg_metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ThingService(test_hook=_assert_request_meta_received(deadline, metadata))]
|
||||
) as channel:
|
||||
client = ThingServiceClient(channel, deadline=deadline, metadata=metadata)
|
||||
response = await client._unary_unary(
|
||||
"/service.Test/DoThing",
|
||||
DoThingRequest(THING_TO_DO),
|
||||
DoThingResponse,
|
||||
deadline=kwarg_deadline,
|
||||
metadata=kwarg_metadata,
|
||||
)
|
||||
assert response.names == [THING_TO_DO]
|
||||
|
||||
# Setting timeout
|
||||
timeout = 99
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(timeout)
|
||||
metadata = {"authorization": "12345"}
|
||||
kwarg_timeout = 9000
|
||||
kwarg_deadline = grpclib.metadata.Deadline.from_timeout(kwarg_timeout)
|
||||
kwarg_metadata = {"authorization": "09876"}
|
||||
async with ChannelFor(
|
||||
[
|
||||
ThingService(
|
||||
test_hook=_assert_request_meta_received(kwarg_deadline, kwarg_metadata),
|
||||
)
|
||||
]
|
||||
) as channel:
|
||||
client = ThingServiceClient(channel, deadline=deadline, metadata=metadata)
|
||||
response = await client._unary_unary(
|
||||
"/service.Test/DoThing",
|
||||
DoThingRequest(THING_TO_DO),
|
||||
DoThingResponse,
|
||||
timeout=kwarg_timeout,
|
||||
metadata=kwarg_metadata,
|
||||
)
|
||||
assert response.names == [THING_TO_DO]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_gen_for_unary_stream_request():
|
||||
thing_name = "my milkshakes"
|
||||
|
||||
async with ChannelFor([ThingService()]) as channel:
|
||||
client = ThingServiceClient(channel)
|
||||
expected_versions = [5, 4, 3, 2, 1]
|
||||
async for response in client.get_thing_versions(name=thing_name):
|
||||
assert response.name == thing_name
|
||||
assert response.version == expected_versions.pop()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_gen_for_stream_stream_request():
|
||||
some_things = ["cake", "cricket", "coral reef"]
|
||||
more_things = ["ball", "that", "56kmodem", "liberal humanism", "cheesesticks"]
|
||||
expected_things = (*some_things, *more_things)
|
||||
|
||||
async with ChannelFor([ThingService()]) as channel:
|
||||
client = ThingServiceClient(channel)
|
||||
# Use an AsyncChannel to decouple sending and recieving, it'll send some_things
|
||||
# immediately and we'll use it to send more_things later, after recieving some
|
||||
# results
|
||||
request_chan = AsyncChannel()
|
||||
send_initial_requests = asyncio.ensure_future(
|
||||
request_chan.send_from(GetThingRequest(name) for name in some_things)
|
||||
)
|
||||
response_index = 0
|
||||
async for response in client.get_different_things(request_chan):
|
||||
assert response.name == expected_things[response_index]
|
||||
assert response.version == response_index + 1
|
||||
response_index += 1
|
||||
if more_things:
|
||||
# Send some more requests as we receive responses to be sure coordination of
|
||||
# send/receive events doesn't matter
|
||||
await request_chan.send(GetThingRequest(more_things.pop(0)))
|
||||
elif not send_initial_requests.done():
|
||||
# Make sure the sending task it completed
|
||||
await send_initial_requests
|
||||
else:
|
||||
# No more things to send make sure channel is closed
|
||||
request_chan.close()
|
||||
assert response_index == len(
|
||||
expected_things
|
||||
), "Didn't receive all expected responses"
|
97
tests/grpc/test_stream_stream.py
Normal file
97
tests/grpc/test_stream_stream.py
Normal file
@@ -0,0 +1,97 @@
|
||||
import asyncio
|
||||
import betterproto
|
||||
from betterproto.grpc.util.async_channel import AsyncChannel
|
||||
from dataclasses import dataclass
|
||||
import pytest
|
||||
from typing import AsyncIterator
|
||||
|
||||
|
||||
@dataclass
|
||||
class Message(betterproto.Message):
|
||||
body: str = betterproto.string_field(1)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def expected_responses():
|
||||
return [Message("Hello world 1"), Message("Hello world 2"), Message("Done")]
|
||||
|
||||
|
||||
class ClientStub:
|
||||
async def connect(self, requests: AsyncIterator):
|
||||
await asyncio.sleep(0.1)
|
||||
async for request in requests:
|
||||
await asyncio.sleep(0.1)
|
||||
yield request
|
||||
await asyncio.sleep(0.1)
|
||||
yield Message("Done")
|
||||
|
||||
|
||||
async def to_list(generator: AsyncIterator):
|
||||
return [value async for value in generator]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
# channel = Channel(host='127.0.0.1', port=50051)
|
||||
# return ClientStub(channel)
|
||||
return ClientStub()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_from_before_connect_and_close_automatically(
|
||||
client, expected_responses
|
||||
):
|
||||
requests = AsyncChannel()
|
||||
await requests.send_from(
|
||||
[Message(body="Hello world 1"), Message(body="Hello world 2")], close=True
|
||||
)
|
||||
responses = client.connect(requests)
|
||||
|
||||
assert await to_list(responses) == expected_responses
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_from_after_connect_and_close_automatically(
|
||||
client, expected_responses
|
||||
):
|
||||
requests = AsyncChannel()
|
||||
responses = client.connect(requests)
|
||||
await requests.send_from(
|
||||
[Message(body="Hello world 1"), Message(body="Hello world 2")], close=True
|
||||
)
|
||||
|
||||
assert await to_list(responses) == expected_responses
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_from_close_manually_immediately(client, expected_responses):
|
||||
requests = AsyncChannel()
|
||||
responses = client.connect(requests)
|
||||
await requests.send_from(
|
||||
[Message(body="Hello world 1"), Message(body="Hello world 2")], close=False
|
||||
)
|
||||
requests.close()
|
||||
|
||||
assert await to_list(responses) == expected_responses
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_individually_and_close_before_connect(client, expected_responses):
|
||||
requests = AsyncChannel()
|
||||
await requests.send(Message(body="Hello world 1"))
|
||||
await requests.send(Message(body="Hello world 2"))
|
||||
requests.close()
|
||||
responses = client.connect(requests)
|
||||
|
||||
assert await to_list(responses) == expected_responses
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_individually_and_close_after_connect(client, expected_responses):
|
||||
requests = AsyncChannel()
|
||||
await requests.send(Message(body="Hello world 1"))
|
||||
await requests.send(Message(body="Hello world 2"))
|
||||
responses = client.connect(requests)
|
||||
requests.close()
|
||||
|
||||
assert await to_list(responses) == expected_responses
|
83
tests/grpc/thing_service.py
Normal file
83
tests/grpc/thing_service.py
Normal file
@@ -0,0 +1,83 @@
|
||||
from tests.output_betterproto.service.service import (
|
||||
DoThingResponse,
|
||||
DoThingRequest,
|
||||
GetThingRequest,
|
||||
GetThingResponse,
|
||||
)
|
||||
import grpclib
|
||||
import grpclib.server
|
||||
from typing import Dict
|
||||
|
||||
|
||||
class ThingService:
|
||||
def __init__(self, test_hook=None):
|
||||
# This lets us pass assertions to the servicer ;)
|
||||
self.test_hook = test_hook
|
||||
|
||||
async def do_thing(
|
||||
self, stream: "grpclib.server.Stream[DoThingRequest, DoThingResponse]"
|
||||
):
|
||||
request = await stream.recv_message()
|
||||
if self.test_hook is not None:
|
||||
self.test_hook(stream)
|
||||
await stream.send_message(DoThingResponse([request.name]))
|
||||
|
||||
async def do_many_things(
|
||||
self, stream: "grpclib.server.Stream[DoThingRequest, DoThingResponse]"
|
||||
):
|
||||
thing_names = [request.name for request in stream]
|
||||
if self.test_hook is not None:
|
||||
self.test_hook(stream)
|
||||
await stream.send_message(DoThingResponse(thing_names))
|
||||
|
||||
async def get_thing_versions(
|
||||
self, stream: "grpclib.server.Stream[GetThingRequest, GetThingResponse]"
|
||||
):
|
||||
request = await stream.recv_message()
|
||||
if self.test_hook is not None:
|
||||
self.test_hook(stream)
|
||||
for version_num in range(1, 6):
|
||||
await stream.send_message(
|
||||
GetThingResponse(name=request.name, version=version_num)
|
||||
)
|
||||
|
||||
async def get_different_things(
|
||||
self, stream: "grpclib.server.Stream[GetThingRequest, GetThingResponse]"
|
||||
):
|
||||
if self.test_hook is not None:
|
||||
self.test_hook(stream)
|
||||
# Respond to each input item immediately
|
||||
response_num = 0
|
||||
async for request in stream:
|
||||
response_num += 1
|
||||
await stream.send_message(
|
||||
GetThingResponse(name=request.name, version=response_num)
|
||||
)
|
||||
|
||||
def __mapping__(self) -> Dict[str, "grpclib.const.Handler"]:
|
||||
return {
|
||||
"/service.Test/DoThing": grpclib.const.Handler(
|
||||
self.do_thing,
|
||||
grpclib.const.Cardinality.UNARY_UNARY,
|
||||
DoThingRequest,
|
||||
DoThingResponse,
|
||||
),
|
||||
"/service.Test/DoManyThings": grpclib.const.Handler(
|
||||
self.do_many_things,
|
||||
grpclib.const.Cardinality.STREAM_UNARY,
|
||||
DoThingRequest,
|
||||
DoThingResponse,
|
||||
),
|
||||
"/service.Test/GetThingVersions": grpclib.const.Handler(
|
||||
self.get_thing_versions,
|
||||
grpclib.const.Cardinality.UNARY_STREAM,
|
||||
GetThingRequest,
|
||||
GetThingResponse,
|
||||
),
|
||||
"/service.Test/GetDifferentThings": grpclib.const.Handler(
|
||||
self.get_different_things,
|
||||
grpclib.const.Cardinality.STREAM_STREAM,
|
||||
GetThingRequest,
|
||||
GetThingResponse,
|
||||
),
|
||||
}
|
@@ -1,4 +1,4 @@
|
||||
from betterproto.tests.output_betterproto.bool.bool import Test
|
||||
from tests.output_betterproto.bool import Test
|
||||
|
||||
|
||||
def test_value():
|
@@ -10,6 +10,7 @@ message Test {
|
||||
int32 camelCase = 1;
|
||||
my_enum snake_case = 2;
|
||||
snake_case_message snake_case_message = 3;
|
||||
int32 UPPERCASE = 4;
|
||||
}
|
||||
|
||||
message snake_case_message {
|
@@ -1,5 +1,5 @@
|
||||
import betterproto.tests.output_betterproto.casing.casing as casing
|
||||
from betterproto.tests.output_betterproto.casing.casing import Test
|
||||
import tests.output_betterproto.casing as casing
|
||||
from tests.output_betterproto.casing import Test
|
||||
|
||||
|
||||
def test_message_attributes():
|
||||
@@ -8,6 +8,7 @@ def test_message_attributes():
|
||||
message, "snake_case_message"
|
||||
), "snake_case field name is same in python"
|
||||
assert hasattr(message, "camel_case"), "CamelCase field is snake_case in python"
|
||||
assert hasattr(message, "uppercase"), "UPPERCASE field is lowercase in python"
|
||||
|
||||
|
||||
def test_message_casing():
|
@@ -0,0 +1,7 @@
|
||||
syntax = "proto3";
|
||||
|
||||
message Test {
|
||||
int32 UPPERCASE = 1;
|
||||
int32 UPPERCASE_V2 = 2;
|
||||
int32 UPPER_CAMEL_CASE = 3;
|
||||
}
|
@@ -0,0 +1,14 @@
|
||||
from tests.output_betterproto.casing_message_field_uppercase import Test
|
||||
|
||||
|
||||
def test_message_casing():
|
||||
message = Test()
|
||||
assert hasattr(
|
||||
message, "uppercase"
|
||||
), "UPPERCASE attribute is converted to 'uppercase' in python"
|
||||
assert hasattr(
|
||||
message, "uppercase_v2"
|
||||
), "UPPERCASE_V2 attribute is converted to 'uppercase_v2' in python"
|
||||
assert hasattr(
|
||||
message, "upper_camel_case"
|
||||
), "UPPER_CAMEL_CASE attribute is converted to upper_camel_case in python"
|
20
tests/inputs/config.py
Normal file
20
tests/inputs/config.py
Normal file
@@ -0,0 +1,20 @@
|
||||
# Test cases that are expected to fail, e.g. unimplemented features or bug-fixes.
|
||||
# Remove from list when fixed.
|
||||
xfail = {
|
||||
"oneof_enum", # 63
|
||||
"namespace_keywords", # 70
|
||||
"namespace_builtin_types", # 53
|
||||
"googletypes_struct", # 9
|
||||
"googletypes_value", # 9
|
||||
"import_capitalized_package",
|
||||
"example", # This is the example in the readme. Not a test.
|
||||
}
|
||||
|
||||
services = {
|
||||
"googletypes_response",
|
||||
"googletypes_response_embedded",
|
||||
"service",
|
||||
"import_service_input_message",
|
||||
"googletypes_service_returns_empty",
|
||||
"googletypes_service_returns_googletype",
|
||||
}
|
4
tests/inputs/deprecated/deprecated.json
Normal file
4
tests/inputs/deprecated/deprecated.json
Normal file
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"v": 10,
|
||||
"value": 10
|
||||
}
|
9
tests/inputs/deprecated/deprecated.proto
Normal file
9
tests/inputs/deprecated/deprecated.proto
Normal file
@@ -0,0 +1,9 @@
|
||||
syntax = "proto3";
|
||||
|
||||
// Some documentation about the Test message.
|
||||
message Test {
|
||||
// Some documentation about the value.
|
||||
option deprecated = true;
|
||||
int32 v = 1 [deprecated=true];
|
||||
int32 value = 2;
|
||||
}
|
4
tests/inputs/deprecated_field/deprecated_field.json
Normal file
4
tests/inputs/deprecated_field/deprecated_field.json
Normal file
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"v": 10,
|
||||
"value": 10
|
||||
}
|
8
tests/inputs/deprecated_field/deprecated_field.proto
Normal file
8
tests/inputs/deprecated_field/deprecated_field.proto
Normal file
@@ -0,0 +1,8 @@
|
||||
syntax = "proto3";
|
||||
|
||||
// Some documentation about the Test message.
|
||||
message Test {
|
||||
// Some documentation about the value.
|
||||
int32 v = 1 [deprecated=true];
|
||||
int32 value = 2;
|
||||
}
|
9
tests/inputs/enum/enum.json
Normal file
9
tests/inputs/enum/enum.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"choice": "FOUR",
|
||||
"choices": [
|
||||
"ZERO",
|
||||
"ONE",
|
||||
"THREE",
|
||||
"FOUR"
|
||||
]
|
||||
}
|
15
tests/inputs/enum/enum.proto
Normal file
15
tests/inputs/enum/enum.proto
Normal file
@@ -0,0 +1,15 @@
|
||||
syntax = "proto3";
|
||||
|
||||
// Tests that enums are correctly serialized and that it correctly handles skipped and out-of-order enum values
|
||||
message Test {
|
||||
Choice choice = 1;
|
||||
repeated Choice choices = 2;
|
||||
}
|
||||
|
||||
enum Choice {
|
||||
ZERO = 0;
|
||||
ONE = 1;
|
||||
// TWO = 2;
|
||||
FOUR = 4;
|
||||
THREE = 3;
|
||||
}
|
84
tests/inputs/enum/test_enum.py
Normal file
84
tests/inputs/enum/test_enum.py
Normal file
@@ -0,0 +1,84 @@
|
||||
from tests.output_betterproto.enum import (
|
||||
Test,
|
||||
Choice,
|
||||
)
|
||||
|
||||
|
||||
def test_enum_set_and_get():
|
||||
assert Test(choice=Choice.ZERO).choice == Choice.ZERO
|
||||
assert Test(choice=Choice.ONE).choice == Choice.ONE
|
||||
assert Test(choice=Choice.THREE).choice == Choice.THREE
|
||||
assert Test(choice=Choice.FOUR).choice == Choice.FOUR
|
||||
|
||||
|
||||
def test_enum_set_with_int():
|
||||
assert Test(choice=0).choice == Choice.ZERO
|
||||
assert Test(choice=1).choice == Choice.ONE
|
||||
assert Test(choice=3).choice == Choice.THREE
|
||||
assert Test(choice=4).choice == Choice.FOUR
|
||||
|
||||
|
||||
def test_enum_is_comparable_with_int():
|
||||
assert Test(choice=Choice.ZERO).choice == 0
|
||||
assert Test(choice=Choice.ONE).choice == 1
|
||||
assert Test(choice=Choice.THREE).choice == 3
|
||||
assert Test(choice=Choice.FOUR).choice == 4
|
||||
|
||||
|
||||
def test_enum_to_dict():
|
||||
assert (
|
||||
"choice" not in Test(choice=Choice.ZERO).to_dict()
|
||||
), "Default enum value is not serialized"
|
||||
assert (
|
||||
Test(choice=Choice.ZERO).to_dict(include_default_values=True)["choice"]
|
||||
== "ZERO"
|
||||
)
|
||||
assert Test(choice=Choice.ONE).to_dict()["choice"] == "ONE"
|
||||
assert Test(choice=Choice.THREE).to_dict()["choice"] == "THREE"
|
||||
assert Test(choice=Choice.FOUR).to_dict()["choice"] == "FOUR"
|
||||
|
||||
|
||||
def test_repeated_enum_is_comparable_with_int():
|
||||
assert Test(choices=[Choice.ZERO]).choices == [0]
|
||||
assert Test(choices=[Choice.ONE]).choices == [1]
|
||||
assert Test(choices=[Choice.THREE]).choices == [3]
|
||||
assert Test(choices=[Choice.FOUR]).choices == [4]
|
||||
|
||||
|
||||
def test_repeated_enum_set_and_get():
|
||||
assert Test(choices=[Choice.ZERO]).choices == [Choice.ZERO]
|
||||
assert Test(choices=[Choice.ONE]).choices == [Choice.ONE]
|
||||
assert Test(choices=[Choice.THREE]).choices == [Choice.THREE]
|
||||
assert Test(choices=[Choice.FOUR]).choices == [Choice.FOUR]
|
||||
|
||||
|
||||
def test_repeated_enum_to_dict():
|
||||
assert Test(choices=[Choice.ZERO]).to_dict()["choices"] == ["ZERO"]
|
||||
assert Test(choices=[Choice.ONE]).to_dict()["choices"] == ["ONE"]
|
||||
assert Test(choices=[Choice.THREE]).to_dict()["choices"] == ["THREE"]
|
||||
assert Test(choices=[Choice.FOUR]).to_dict()["choices"] == ["FOUR"]
|
||||
|
||||
all_enums_dict = Test(
|
||||
choices=[Choice.ZERO, Choice.ONE, Choice.THREE, Choice.FOUR]
|
||||
).to_dict()
|
||||
assert (all_enums_dict["choices"]) == ["ZERO", "ONE", "THREE", "FOUR"]
|
||||
|
||||
|
||||
def test_repeated_enum_with_single_value_to_dict():
|
||||
assert Test(choices=Choice.ONE).to_dict()["choices"] == ["ONE"]
|
||||
assert Test(choices=1).to_dict()["choices"] == ["ONE"]
|
||||
|
||||
|
||||
def test_repeated_enum_with_non_list_iterables_to_dict():
|
||||
assert Test(choices=(1, 3)).to_dict()["choices"] == ["ONE", "THREE"]
|
||||
assert Test(choices=(1, 3)).to_dict()["choices"] == ["ONE", "THREE"]
|
||||
assert Test(choices=(Choice.ONE, Choice.THREE)).to_dict()["choices"] == [
|
||||
"ONE",
|
||||
"THREE",
|
||||
]
|
||||
|
||||
def enum_generator():
|
||||
yield Choice.ONE
|
||||
yield Choice.THREE
|
||||
|
||||
assert Test(choices=enum_generator()).to_dict()["choices"] == ["ONE", "THREE"]
|
8
tests/inputs/example/example.proto
Normal file
8
tests/inputs/example/example.proto
Normal file
@@ -0,0 +1,8 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package hello;
|
||||
|
||||
// Greeting represents a message you can tell a user.
|
||||
message Greeting {
|
||||
string message = 1;
|
||||
}
|
6
tests/inputs/fixed/fixed.json
Normal file
6
tests/inputs/fixed/fixed.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"foo": 4294967295,
|
||||
"bar": -2147483648,
|
||||
"baz": "18446744073709551615",
|
||||
"qux": "-9223372036854775808"
|
||||
}
|
8
tests/inputs/fixed/fixed.proto
Normal file
8
tests/inputs/fixed/fixed.proto
Normal file
@@ -0,0 +1,8 @@
|
||||
syntax = "proto3";
|
||||
|
||||
message Test {
|
||||
fixed32 foo = 1;
|
||||
sfixed32 bar = 2;
|
||||
fixed64 baz = 3;
|
||||
sfixed64 qux = 4;
|
||||
}
|
@@ -0,0 +1,13 @@
|
||||
syntax = "proto3";
|
||||
|
||||
message Foo{
|
||||
int64 bar = 1;
|
||||
}
|
||||
|
||||
message Test{
|
||||
oneof group{
|
||||
string string = 1;
|
||||
int64 integer = 2;
|
||||
Foo foo = 3;
|
||||
}
|
||||
}
|
@@ -0,0 +1,55 @@
|
||||
import pytest
|
||||
|
||||
from google.protobuf import json_format
|
||||
import betterproto
|
||||
from tests.output_betterproto.google_impl_behavior_equivalence import (
|
||||
Test,
|
||||
Foo,
|
||||
)
|
||||
from tests.output_reference.google_impl_behavior_equivalence.google_impl_behavior_equivalence_pb2 import (
|
||||
Test as ReferenceTest,
|
||||
Foo as ReferenceFoo,
|
||||
)
|
||||
|
||||
|
||||
def test_oneof_serializes_similar_to_google_oneof():
|
||||
|
||||
tests = [
|
||||
(Test(string="abc"), ReferenceTest(string="abc")),
|
||||
(Test(integer=2), ReferenceTest(integer=2)),
|
||||
(Test(foo=Foo(bar=1)), ReferenceTest(foo=ReferenceFoo(bar=1))),
|
||||
# Default values should also behave the same within oneofs
|
||||
(Test(string=""), ReferenceTest(string="")),
|
||||
(Test(integer=0), ReferenceTest(integer=0)),
|
||||
(Test(foo=Foo(bar=0)), ReferenceTest(foo=ReferenceFoo(bar=0))),
|
||||
]
|
||||
for message, message_reference in tests:
|
||||
# NOTE: As of July 2020, MessageToJson inserts newlines in the output string so,
|
||||
# just compare dicts
|
||||
assert message.to_dict() == json_format.MessageToDict(message_reference)
|
||||
|
||||
|
||||
def test_bytes_are_the_same_for_oneof():
|
||||
|
||||
message = Test(string="")
|
||||
message_reference = ReferenceTest(string="")
|
||||
|
||||
message_bytes = bytes(message)
|
||||
message_reference_bytes = message_reference.SerializeToString()
|
||||
|
||||
assert message_bytes == message_reference_bytes
|
||||
|
||||
message2 = Test().parse(message_reference_bytes)
|
||||
message_reference2 = ReferenceTest()
|
||||
message_reference2.ParseFromString(message_reference_bytes)
|
||||
|
||||
assert message == message2
|
||||
assert message_reference == message_reference2
|
||||
|
||||
# None of these fields were explicitly set BUT they should not actually be null
|
||||
# themselves
|
||||
assert isinstance(message.foo, Foo)
|
||||
assert isinstance(message2.foo, Foo)
|
||||
|
||||
assert isinstance(message_reference.foo, ReferenceFoo)
|
||||
assert isinstance(message_reference2.foo, ReferenceFoo)
|
@@ -1,5 +1,7 @@
|
||||
{
|
||||
"maybe": false,
|
||||
"ts": "1972-01-01T10:00:20.021Z",
|
||||
"duration": "1.200s"
|
||||
"duration": "1.200s",
|
||||
"important": 10,
|
||||
"empty": {}
|
||||
}
|
@@ -3,10 +3,12 @@ syntax = "proto3";
|
||||
import "google/protobuf/duration.proto";
|
||||
import "google/protobuf/timestamp.proto";
|
||||
import "google/protobuf/wrappers.proto";
|
||||
import "google/protobuf/empty.proto";
|
||||
|
||||
message Test {
|
||||
google.protobuf.BoolValue maybe = 1;
|
||||
google.protobuf.Timestamp ts = 2;
|
||||
google.protobuf.Duration duration = 3;
|
||||
google.protobuf.Int32Value important = 4;
|
||||
google.protobuf.Empty empty = 5;
|
||||
}
|
@@ -1,23 +1,21 @@
|
||||
from typing import Any, Callable, Optional
|
||||
|
||||
import google.protobuf.wrappers_pb2 as wrappers
|
||||
import betterproto.lib.google.protobuf as protobuf
|
||||
import pytest
|
||||
|
||||
from betterproto.tests.mocks import MockChannel
|
||||
from betterproto.tests.output_betterproto.googletypes_response.googletypes_response import (
|
||||
TestStub,
|
||||
)
|
||||
from tests.mocks import MockChannel
|
||||
from tests.output_betterproto.googletypes_response import TestStub
|
||||
|
||||
test_cases = [
|
||||
(TestStub.get_double, wrappers.DoubleValue, 2.5),
|
||||
(TestStub.get_float, wrappers.FloatValue, 2.5),
|
||||
(TestStub.get_int64, wrappers.Int64Value, -64),
|
||||
(TestStub.get_u_int64, wrappers.UInt64Value, 64),
|
||||
(TestStub.get_int32, wrappers.Int32Value, -32),
|
||||
(TestStub.get_u_int32, wrappers.UInt32Value, 32),
|
||||
(TestStub.get_bool, wrappers.BoolValue, True),
|
||||
(TestStub.get_string, wrappers.StringValue, "string"),
|
||||
(TestStub.get_bytes, wrappers.BytesValue, bytes(0xFF)[0:4]),
|
||||
(TestStub.get_double, protobuf.DoubleValue, 2.5),
|
||||
(TestStub.get_float, protobuf.FloatValue, 2.5),
|
||||
(TestStub.get_int64, protobuf.Int64Value, -64),
|
||||
(TestStub.get_u_int64, protobuf.UInt64Value, 64),
|
||||
(TestStub.get_int32, protobuf.Int32Value, -32),
|
||||
(TestStub.get_u_int32, protobuf.UInt32Value, 32),
|
||||
(TestStub.get_bool, protobuf.BoolValue, True),
|
||||
(TestStub.get_string, protobuf.StringValue, "string"),
|
||||
(TestStub.get_bytes, protobuf.BytesValue, bytes(0xFF)[0:4]),
|
||||
]
|
||||
|
||||
|
@@ -1,7 +1,7 @@
|
||||
import pytest
|
||||
|
||||
from betterproto.tests.mocks import MockChannel
|
||||
from betterproto.tests.output_betterproto.googletypes_response_embedded.googletypes_response_embedded import (
|
||||
from tests.mocks import MockChannel
|
||||
from tests.output_betterproto.googletypes_response_embedded import (
|
||||
Output,
|
||||
TestStub,
|
||||
)
|
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user