Compare commits
9 Commits
285-semant
...
michael-sa
| Author | SHA1 | Date | |
|---|---|---|---|
|
0c02d1b21a | ||
|
ac32bcd25a | ||
|
|
72855227bd | ||
|
|
47081617c2 | ||
|
|
d734206fe5 | ||
|
|
bbf40f9694 | ||
|
6671d87cef | ||
|
|
cd66b0511a | ||
|
|
c48ca2e386 |
23
.github/CONTRIBUTING.md
vendored
23
.github/CONTRIBUTING.md
vendored
@@ -1,23 +0,0 @@
|
||||
# Contributing
|
||||
|
||||
There's lots to do, and we're working hard, so any help is welcome!
|
||||
|
||||
- :speech_balloon: Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!
|
||||
|
||||
What can you do?
|
||||
|
||||
- :+1: Vote on [issues](https://github.com/danielgtaylor/python-betterproto/issues).
|
||||
- :speech_balloon: Give feedback on [Pull Requests](https://github.com/danielgtaylor/python-betterproto/pulls) and [Issues](https://github.com/danielgtaylor/python-betterproto/issues):
|
||||
- Suggestions
|
||||
- Express approval
|
||||
- Raise concerns
|
||||
- :small_red_triangle: Create an issue:
|
||||
- File a bug (please check its not a duplicate)
|
||||
- Propose an enhancement
|
||||
- :white_check_mark: Create a PR:
|
||||
- [Creating a failing test-case](https://github.com/danielgtaylor/python-betterproto/blob/master/betterproto/tests/README.md) to make bug-fixing easier
|
||||
- Fix any of the open issues
|
||||
- [Good first issues](https://github.com/danielgtaylor/python-betterproto/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
|
||||
- [Issues with tests](https://github.com/danielgtaylor/python-betterproto/issues?q=is%3Aissue+is%3Aopen+label%3A%22has+test%22)
|
||||
- New bugfix or idea
|
||||
- If you'd like to discuss your idea first, join us on Slack!
|
||||
9
.github/PULL_REQUEST_TEMPLATE.md
vendored
9
.github/PULL_REQUEST_TEMPLATE.md
vendored
@@ -1,9 +0,0 @@
|
||||
## Description
|
||||
|
||||
<!-- Thanks for contributing to betterproto! Add a thorough explanation of what your changes do below this line: -->
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] This PR targets the `rc` branch (**not** `master`).
|
||||
- [ ] [If this should release a new version to PyPI when merged] The title of the PR follows the [Angular Conventional Commit](https://www.conventionalcommits.org/) syntax (`feat:` or `fix:`, with `BREAKING CHANGE:` in the commit message body if appropriate), and clearly describes the fix or feature.
|
||||
- [ ] Documentation is updated (`README.md` and docstrings).
|
||||
113
.github/workflows/ci.yml
vendored
113
.github/workflows/ci.yml
vendored
@@ -1,69 +1,74 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
name: ${{ matrix.os }} / ${{ matrix.python-version }}
|
||||
runs-on: ${{ matrix.os }}-latest
|
||||
strategy:
|
||||
matrix:
|
||||
os: [Ubuntu, MacOS, Windows]
|
||||
python-version: [3.6, 3.7, 3.8, 3.9]
|
||||
exclude:
|
||||
- os: Windows
|
||||
python-version: 3.6
|
||||
|
||||
check-formatting:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
name: Consult black on python formatting
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.7
|
||||
- uses: Gr1N/setup-poetry@v2
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: ~/.cache/pypoetry/virtualenvs
|
||||
key: ${{ runner.os }}-poetry-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-poetry-
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
- name: Run black
|
||||
run: make check-style
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v2
|
||||
run-tests:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
name: Run tests with tox
|
||||
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: [ '3.6', '3.7', '3.8']
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Get full Python version
|
||||
id: full-python-version
|
||||
shell: bash
|
||||
run: echo ::set-output name=version::$(python -c "import sys; print('-'.join(str(v) for v in sys.version_info))")
|
||||
|
||||
- name: Install poetry
|
||||
shell: bash
|
||||
run: |
|
||||
python -m pip install poetry
|
||||
echo "$HOME/.poetry/bin" >> $GITHUB_PATH
|
||||
|
||||
- name: Configure poetry
|
||||
shell: bash
|
||||
run: poetry config virtualenvs.in-project true
|
||||
|
||||
- name: Set up cache
|
||||
uses: actions/cache@v2
|
||||
id: cache
|
||||
- uses: Gr1N/setup-poetry@v2
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
|
||||
|
||||
- name: Ensure cache is healthy
|
||||
if: steps.cache.outputs.cache-hit == 'true'
|
||||
shell: bash
|
||||
run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
|
||||
|
||||
path: ~/.cache/pypoetry/virtualenvs
|
||||
key: ${{ runner.os }}-poetry-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-poetry-
|
||||
- name: Install dependencies
|
||||
shell: bash
|
||||
run: |
|
||||
poetry run python -m pip install pip -U
|
||||
sudo apt install protobuf-compiler libprotobuf-dev
|
||||
poetry install
|
||||
- name: Run tests
|
||||
run: |
|
||||
make generate
|
||||
make test
|
||||
|
||||
- name: Generate code from proto files
|
||||
shell: bash
|
||||
run: poetry run python -m tests.generate -v
|
||||
build-release:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
- name: Execute test suite
|
||||
shell: bash
|
||||
run: poetry run pytest tests/
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.7
|
||||
- uses: Gr1N/setup-poetry@v2
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
- name: Publish package to PyPI
|
||||
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
|
||||
run: poetry publish -n
|
||||
env:
|
||||
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.pypi }}
|
||||
|
||||
26
.github/workflows/code-quality.yml
vendored
26
.github/workflows/code-quality.yml
vendored
@@ -1,26 +0,0 @@
|
||||
name: Code Quality
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
|
||||
jobs:
|
||||
check-formatting:
|
||||
name: Check code/doc formatting
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- name: Run Black
|
||||
uses: lgeiger/black-action@master
|
||||
with:
|
||||
args: --check src/ tests/ benchmarks/
|
||||
|
||||
- name: Install rST dependcies
|
||||
run: python -m pip install doc8
|
||||
- name: Lint documentation for errors
|
||||
run: python -m doc8 docs --max-line-length 88 --ignore-path-errors "docs/migrating.rst;D001"
|
||||
# it has a table which is longer than 88 characters long
|
||||
37
.github/workflows/release.yml
vendored
37
.github/workflows/release.yml
vendored
@@ -1,37 +0,0 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
tags:
|
||||
- '**'
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
|
||||
jobs:
|
||||
packaging:
|
||||
name: Distribution
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Set up Python 3.8
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
- name: Install poetry
|
||||
run: python -m pip install poetry
|
||||
- name: Semantic Release
|
||||
uses: cycjimmy/semantic-release-action@v2
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
- name: Publish package to PyPI
|
||||
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
|
||||
env:
|
||||
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.pypi }}
|
||||
run: poetry publish -n
|
||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -6,8 +6,7 @@
|
||||
.pytest_cache
|
||||
.python-version
|
||||
build/
|
||||
node_modules/
|
||||
tests/output_*
|
||||
betterproto/tests/output_*
|
||||
**/__pycache__
|
||||
dist
|
||||
**/*.egg-info
|
||||
@@ -15,6 +14,3 @@ output
|
||||
.idea
|
||||
.DS_Store
|
||||
.tox
|
||||
.venv
|
||||
.asv
|
||||
venv
|
||||
|
||||
@@ -1,17 +0,0 @@
|
||||
version: 2
|
||||
formats: []
|
||||
|
||||
build:
|
||||
image: latest
|
||||
|
||||
sphinx:
|
||||
configuration: docs/conf.py
|
||||
fail_on_warning: false
|
||||
|
||||
python:
|
||||
version: 3.7
|
||||
install:
|
||||
- method: pip
|
||||
path: .
|
||||
extra_requirements:
|
||||
- dev
|
||||
66
CHANGELOG.md
66
CHANGELOG.md
@@ -5,72 +5,6 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
- Versions suffixed with `b*` are in `beta` and can be installed with `pip install --pre betterproto`.
|
||||
|
||||
## [2.0.0b3] - 2021-04-07
|
||||
|
||||
- Generate grpclib service stubs [#170](https://github.com/danielgtaylor/python-betterproto/pull/170)
|
||||
- Add \_\_version\_\_ attribute to package [#134](https://github.com/danielgtaylor/python-betterproto/pull/134)
|
||||
- Use betterproto generated messages in the plugin [#161](https://github.com/danielgtaylor/python-betterproto/pull/161)
|
||||
- Sort the list of sources in generated file headers [#164](https://github.com/danielgtaylor/python-betterproto/pull/164)
|
||||
- Micro-optimization: use tuples instead of lists for conditions [#228](https://github.com/danielgtaylor/python-betterproto/pull/228)
|
||||
- Improve datestring parsing [#213](https://github.com/danielgtaylor/python-betterproto/pull/213)
|
||||
|
||||
- Fix serialization of repeated fields with empty messages [#180](https://github.com/danielgtaylor/python-betterproto/pull/180)
|
||||
- Fix compilation of fields named 'bytes' or 'str' [#226](https://github.com/danielgtaylor/python-betterproto/pull/226)
|
||||
- Fix json serialization of infinite and nan floats/doubles [#215](https://github.com/danielgtaylor/python-betterproto/pull/215)
|
||||
- Fix template bug resulting in empty \_\_post_init\_\_ methods [#162](https://github.com/danielgtaylor/python-betterproto/pull/162)
|
||||
- Fix serialization of zero-value messages in a oneof group [#176](https://github.com/danielgtaylor/python-betterproto/pull/176)
|
||||
- Fix missing typing and datetime imports [#183](https://github.com/danielgtaylor/python-betterproto/pull/183)
|
||||
- Fix code generation for empty services [#222](https://github.com/danielgtaylor/python-betterproto/pull/222)
|
||||
- Fix Message.to_dict and from_dict handling of repeated timestamps and durations [#211](https://github.com/danielgtaylor/python-betterproto/pull/211)
|
||||
- Fix incorrect routes in generated client when service is not in a package [#177](https://github.com/danielgtaylor/python-betterproto/pull/177)
|
||||
|
||||
## [2.0.0b2] - 2020-11-24
|
||||
|
||||
- Add support for deprecated message and fields [#126](https://github.com/danielgtaylor/python-betterproto/pull/126)
|
||||
- Add support for recursive messages [#130](https://github.com/danielgtaylor/python-betterproto/pull/130)
|
||||
- Add support for `bool(Message)` [#142](https://github.com/danielgtaylor/python-betterproto/pull/142)
|
||||
- Improve support for Python 3.9 [#140](https://github.com/danielgtaylor/python-betterproto/pull/140) [#173](https://github.com/danielgtaylor/python-betterproto/pull/173)
|
||||
- Improve keyword sanitisation for generated code [#137](https://github.com/danielgtaylor/python-betterproto/pull/137)
|
||||
|
||||
- Fix missing serialized_on_wire when message contains only lists [#81](https://github.com/danielgtaylor/python-betterproto/pull/81)
|
||||
- Fix circular dependencies [#100](https://github.com/danielgtaylor/python-betterproto/pull/100)
|
||||
- Fix to_dict enum fields when numbering is not consecutive [#102](https://github.com/danielgtaylor/python-betterproto/pull/102)
|
||||
- Fix argument generation for stub methods when using `import` with proto definition [#103](https://github.com/danielgtaylor/python-betterproto/pull/103)
|
||||
- Fix missing async/await keywords when casing [#104](https://github.com/danielgtaylor/python-betterproto/pull/104)
|
||||
- Fix mutable default arguments in generated code [#105](https://github.com/danielgtaylor/python-betterproto/pull/105)
|
||||
- Fix serialisation of default values in oneofs when calling to_dict() or to_json() [#110](https://github.com/danielgtaylor/python-betterproto/pull/110)
|
||||
- Fix static type checking for grpclib client [#124](https://github.com/danielgtaylor/python-betterproto/pull/124)
|
||||
- Fix python3.6 compatibility issue with dataclasses [#124](https://github.com/danielgtaylor/python-betterproto/pull/124)
|
||||
- Fix handling of trailer-only responses [#127](https://github.com/danielgtaylor/python-betterproto/pull/127)
|
||||
|
||||
- Refactor plugin.py to use modular dataclasses in tree-like structure to represent parsed data [#121](https://github.com/danielgtaylor/python-betterproto/pull/121)
|
||||
- Refactor template compilation logic [#136](https://github.com/danielgtaylor/python-betterproto/pull/136)
|
||||
|
||||
- Replace use of platform provided protoc with development dependency on grpcio-tools [#107](https://github.com/danielgtaylor/python-betterproto/pull/107)
|
||||
- Switch to using `poe` from `make` to manage project development tasks [#118](https://github.com/danielgtaylor/python-betterproto/pull/118)
|
||||
- Improve CI platform coverage [#128](https://github.com/danielgtaylor/python-betterproto/pull/128)
|
||||
|
||||
## [2.0.0b1] - 2020-07-04
|
||||
|
||||
[Upgrade Guide](./docs/upgrading.md)
|
||||
|
||||
> Several bugfixes and improvements required or will require small breaking changes, necessitating a new version.
|
||||
> `2.0.0` will be released once the interface is stable.
|
||||
|
||||
- Add support for gRPC and **stream-stream** [#83](https://github.com/danielgtaylor/python-betterproto/pull/83)
|
||||
- Switch from `pipenv` to `poetry` for development [#75](https://github.com/danielgtaylor/python-betterproto/pull/75)
|
||||
- Fix two packages with the same name suffix should not cause naming conflict [#25](https://github.com/danielgtaylor/python-betterproto/issues/25)
|
||||
|
||||
- Fix Import child package from root [#57](https://github.com/danielgtaylor/python-betterproto/issues/57)
|
||||
- Fix Import child package from package [#58](https://github.com/danielgtaylor/python-betterproto/issues/58)
|
||||
- Fix Import parent package from child package [#59](https://github.com/danielgtaylor/python-betterproto/issues/59)
|
||||
- Fix Import root package from child package [#60](https://github.com/danielgtaylor/python-betterproto/issues/60)
|
||||
- Fix Import root package from root [#61](https://github.com/danielgtaylor/python-betterproto/issues/61)
|
||||
|
||||
- Fix ALL_CAPS message fields are parsed incorrectly. [#11](https://github.com/danielgtaylor/python-betterproto/issues/11)
|
||||
|
||||
## [1.2.5] - 2020-04-27
|
||||
|
||||
- Add .j2 suffix to python template names to avoid confusing certain build tools [#72](https://github.com/danielgtaylor/python-betterproto/pull/72)
|
||||
|
||||
42
Makefile
Normal file
42
Makefile
Normal file
@@ -0,0 +1,42 @@
|
||||
.PHONY: help setup generate test types format clean plugin full-test check-style
|
||||
|
||||
help: ## - Show this help.
|
||||
@fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//'
|
||||
|
||||
# Dev workflow tasks
|
||||
|
||||
generate: ## - Generate test cases (do this once before running test)
|
||||
poetry run ./betterproto/tests/generate.py
|
||||
|
||||
test: ## - Run tests
|
||||
poetry run pytest --cov betterproto
|
||||
|
||||
types: ## - Check types with mypy
|
||||
poetry run mypy betterproto --ignore-missing-imports
|
||||
|
||||
format: ## - Apply black formatting to source code
|
||||
poetry run black . --exclude tests/output_
|
||||
|
||||
clean: ## - Clean out generated files from the workspace
|
||||
rm -rf .coverage \
|
||||
.mypy_cache \
|
||||
.pytest_cache \
|
||||
dist \
|
||||
**/__pycache__ \
|
||||
betterproto/tests/output_*
|
||||
|
||||
# Manual testing
|
||||
|
||||
# By default write plugin output to a directory called output
|
||||
o=output
|
||||
plugin: ## - Execute the protoc plugin, with output write to `output` or the value passed to `-o`
|
||||
mkdir -p $(o)
|
||||
protoc --plugin=protoc-gen-custom=betterproto/plugin.py $(i) --custom_out=$(o)
|
||||
|
||||
# CI tasks
|
||||
|
||||
full-test: generate ## - Run full testing sequence with multiple pythons
|
||||
poetry run tox
|
||||
|
||||
check-style: ## - Check if code style is correct
|
||||
poetry run black . --check --diff --exclude tests/output_
|
||||
145
README.md
145
README.md
@@ -1,7 +1,6 @@
|
||||
# Better Protobuf / gRPC Support for Python
|
||||
|
||||
|
||||
> :octocat: If you're reading this on github, please be aware that it might mention unreleased features! See the latest released README on [pypi](https://pypi.org/project/betterproto/).
|
||||
|
||||
This project aims to provide an improved experience when using Protobuf / gRPC in a modern Python environment by making use of modern language features and generating readable, understandable, idiomatic Python code. It will not support legacy features or environments (e.g. Protobuf 2). The following are supported:
|
||||
|
||||
@@ -38,9 +37,10 @@ This project exists because I am unhappy with the state of the official Google p
|
||||
- Uses `SerializeToString()` rather than the built-in `__bytes__()`
|
||||
- Special wrapped types don't use Python's `None`
|
||||
- Timestamp/duration types don't use Python's built-in `datetime` module
|
||||
|
||||
This project is a reimplementation from the ground up focused on idiomatic modern Python to help fix some of the above. While it may not be a 1:1 drop-in replacement due to changed method names and call patterns, the wire format is identical.
|
||||
|
||||
## Installation
|
||||
## Installation & Getting Started
|
||||
|
||||
First, install the package. Note that the `[compiler]` feature flag tells it to install extra dependencies only needed by the `protoc` plugin:
|
||||
|
||||
@@ -52,12 +52,6 @@ pip install "betterproto[compiler]"
|
||||
pip install betterproto
|
||||
```
|
||||
|
||||
*Betterproto* is under active development. To install the latest beta version, use `pip install --pre betterproto`.
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Compiling proto files
|
||||
|
||||
Now, given you installed the compiler and have a proto file, e.g `example.proto`:
|
||||
|
||||
```protobuf
|
||||
@@ -71,20 +65,13 @@ message Greeting {
|
||||
}
|
||||
```
|
||||
|
||||
You can run the following to invoke protoc directly:
|
||||
You can run the following:
|
||||
|
||||
```sh
|
||||
mkdir lib
|
||||
protoc -I . --python_betterproto_out=lib example.proto
|
||||
```
|
||||
|
||||
or run the following to invoke protoc via grpcio-tools:
|
||||
|
||||
```sh
|
||||
pip install grpcio-tools
|
||||
python -m grpc_tools.protoc -I . --python_betterproto_out=lib example.proto
|
||||
```
|
||||
|
||||
This will generate `lib/hello/__init__.py` which looks like:
|
||||
|
||||
```python
|
||||
@@ -133,7 +120,7 @@ Greeting(message="Hey!")
|
||||
|
||||
The generated Protobuf `Message` classes are compatible with [grpclib](https://github.com/vmagamedov/grpclib) so you are free to use it if you like. That said, this project also includes support for async gRPC stub generation with better static type checking and code completion support. It is enabled by default.
|
||||
|
||||
Given an example service definition:
|
||||
Given an example like:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
@@ -160,74 +147,22 @@ service Echo {
|
||||
}
|
||||
```
|
||||
|
||||
Generate echo proto file:
|
||||
You can use it like so (enable async in the interactive shell first):
|
||||
|
||||
```
|
||||
python -m grpc_tools.protoc -I . --python_betterproto_out=. echo.proto
|
||||
```
|
||||
```py
|
||||
>>> import echo
|
||||
>>> from grpclib.client import Channel
|
||||
|
||||
A client can be implemented as follows:
|
||||
```python
|
||||
import asyncio
|
||||
import echo
|
||||
>>> channel = Channel(host="127.0.0.1", port=1234)
|
||||
>>> service = echo.EchoStub(channel)
|
||||
>>> await service.echo(value="hello", extra_times=1)
|
||||
EchoResponse(values=["hello", "hello"])
|
||||
|
||||
from grpclib.client import Channel
|
||||
|
||||
|
||||
async def main():
|
||||
channel = Channel(host="127.0.0.1", port=50051)
|
||||
service = echo.EchoStub(channel)
|
||||
response = await service.echo(value="hello", extra_times=1)
|
||||
print(response)
|
||||
|
||||
async for response in service.echo_stream(value="hello", extra_times=1):
|
||||
>>> async for response in service.echo_stream(value="hello", extra_times=1)
|
||||
print(response)
|
||||
|
||||
# don't forget to close the channel when done!
|
||||
channel.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.run_until_complete(main())
|
||||
|
||||
```
|
||||
which would output
|
||||
```python
|
||||
EchoResponse(values=['hello', 'hello'])
|
||||
EchoStreamResponse(value='hello')
|
||||
EchoStreamResponse(value='hello')
|
||||
```
|
||||
|
||||
This project also produces server-facing stubs that can be used to implement a Python
|
||||
gRPC server.
|
||||
To use them, simply subclass the base class in the generated files and override the
|
||||
service methods:
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from echo import EchoBase, EchoResponse, EchoStreamResponse
|
||||
from grpclib.server import Server
|
||||
from typing import AsyncIterator
|
||||
|
||||
|
||||
class EchoService(EchoBase):
|
||||
async def echo(self, value: str, extra_times: int) -> "EchoResponse":
|
||||
return EchoResponse([value for _ in range(extra_times)])
|
||||
|
||||
async def echo_stream(self, value: str, extra_times: int) -> AsyncIterator["EchoStreamResponse"]:
|
||||
for _ in range(extra_times):
|
||||
yield EchoStreamResponse(value)
|
||||
|
||||
|
||||
async def main():
|
||||
server = Server([EchoService()])
|
||||
await server.start("127.0.0.1", 50051)
|
||||
await server.wait_closed()
|
||||
|
||||
if __name__ == '__main__':
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.run_until_complete(main())
|
||||
EchoStreamResponse(value="hello")
|
||||
EchoStreamResponse(value="hello")
|
||||
```
|
||||
|
||||
### JSON
|
||||
@@ -239,8 +174,8 @@ Both serializing and parsing are supported to/from JSON and Python dictionaries
|
||||
|
||||
For compatibility the default is to convert field names to `camelCase`. You can control this behavior by passing a casing value, e.g:
|
||||
|
||||
```python
|
||||
MyMessage().to_dict(casing=betterproto.Casing.SNAKE)
|
||||
```py
|
||||
>>> MyMessage().to_dict(casing=betterproto.Casing.SNAKE)
|
||||
```
|
||||
|
||||
### Determining if a message was sent
|
||||
@@ -363,31 +298,20 @@ datetime.datetime(2019, 1, 1, 11, 59, 58, 800000, tzinfo=datetime.timezone.utc)
|
||||
|
||||
## Development
|
||||
|
||||
- _Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!_
|
||||
- _See how you can help → [Contributing](.github/CONTRIBUTING.md)_
|
||||
Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python (3.6 or higher)
|
||||
|
||||
- [poetry](https://python-poetry.org/docs/#installation)
|
||||
*Needed to install dependencies in a virtual environment*
|
||||
|
||||
- [poethepoet](https://github.com/nat-n/poethepoet) for running development tasks as defined in pyproject.toml
|
||||
- Can be installed to your host environment via `pip install poethepoet` then executed as simple `poe`
|
||||
- or run from the poetry venv as `poetry run poe`
|
||||
|
||||
### Setup
|
||||
First, make sure you have Python 3.6+ and `poetry` installed, along with the official [Protobuf Compiler](https://github.com/protocolbuffers/protobuf/releases) for your platform. Then:
|
||||
|
||||
```sh
|
||||
# Get set up with the virtual env & dependencies
|
||||
poetry run pip install --upgrade pip
|
||||
poetry install
|
||||
|
||||
# Activate the poetry environment
|
||||
poetry shell
|
||||
```
|
||||
|
||||
To benefit from the collection of standard development tasks ensure you have make installed and run `make help` to see available tasks.
|
||||
|
||||
### Code style
|
||||
|
||||
This project enforces [black](https://github.com/psf/black) python code formatting.
|
||||
@@ -395,7 +319,7 @@ This project enforces [black](https://github.com/psf/black) python code formatti
|
||||
Before committing changes run:
|
||||
|
||||
```sh
|
||||
poe format
|
||||
make format
|
||||
```
|
||||
|
||||
To avoid merge conflicts later, non-black formatted python code will fail in CI.
|
||||
@@ -429,15 +353,15 @@ Here's how to run the tests.
|
||||
|
||||
```sh
|
||||
# Generate assets from sample .proto files required by the tests
|
||||
poe generate
|
||||
make generate
|
||||
# Run the tests
|
||||
poe test
|
||||
make test
|
||||
```
|
||||
|
||||
To run tests as they are run in CI (with tox) run:
|
||||
|
||||
```sh
|
||||
poe full-test
|
||||
make full-test
|
||||
```
|
||||
|
||||
### (Re)compiling Google Well-known Types
|
||||
@@ -451,13 +375,14 @@ Assuming your `google.protobuf` source files (included with all releases of `pro
|
||||
|
||||
```sh
|
||||
protoc \
|
||||
--plugin=protoc-gen-custom=src/betterproto/plugin/main.py \
|
||||
--plugin=protoc-gen-custom=betterproto/plugin.py \
|
||||
--custom_opt=INCLUDE_GOOGLE \
|
||||
--custom_out=src/betterproto/lib \
|
||||
--custom_out=betterproto/lib \
|
||||
-I /usr/local/include/ \
|
||||
/usr/local/include/google/protobuf/*.proto
|
||||
```
|
||||
|
||||
|
||||
### TODO
|
||||
|
||||
- [x] Fixed length fields
|
||||
@@ -488,26 +413,16 @@ protoc \
|
||||
- [x] Enum strings
|
||||
- [x] Well known types support (timestamp, duration, wrappers)
|
||||
- [x] Support different casing (orig vs. camel vs. others?)
|
||||
- [x] Async service stubs
|
||||
- [ ] Async service stubs
|
||||
- [x] Unary-unary
|
||||
- [x] Server streaming response
|
||||
- [x] Client streaming request
|
||||
- [ ] Client streaming request
|
||||
- [x] Renaming messages and fields to conform to Python name standards
|
||||
- [x] Renaming clashes with language keywords
|
||||
- [x] Python package
|
||||
- [x] Automate running tests
|
||||
- [ ] Cleanup!
|
||||
|
||||
|
||||
## Release
|
||||
|
||||
New versions are versioned and released using [Semantic Release](https://github.com/semantic-release/semantic-release). When new commits
|
||||
using the Angular Conventional Commits syntax land on `master` or `rc`, those commits are used to determine what new version to release.
|
||||
|
||||
All Pull Requests must target the `rc` branch; when merged into `rc` they will publish new release candidate (`rc`) versions to PyPI
|
||||
automatically. When maintainers want to publish a new full release, they simply merge `rc` into `master`. This flow ensures that features
|
||||
and fixes are published quickly and continuously rather than awaiting a manual release process.
|
||||
|
||||
## Community
|
||||
|
||||
Join us on [Slack](https://join.slack.com/t/betterproto/shared_invite/zt-f0n0uolx-iN8gBNrkPxtKHTLpG3o1OQ)!
|
||||
|
||||
157
asv.conf.json
157
asv.conf.json
@@ -1,157 +0,0 @@
|
||||
{
|
||||
// The version of the config file format. Do not change, unless
|
||||
// you know what you are doing.
|
||||
"version": 1,
|
||||
|
||||
// The name of the project being benchmarked
|
||||
"project": "python-betterproto",
|
||||
|
||||
// The project's homepage
|
||||
"project_url": "https://github.com/danielgtaylor/python-betterproto",
|
||||
|
||||
// The URL or local path of the source code repository for the
|
||||
// project being benchmarked
|
||||
"repo": ".",
|
||||
|
||||
// The Python project's subdirectory in your repo. If missing or
|
||||
// the empty string, the project is assumed to be located at the root
|
||||
// of the repository.
|
||||
// "repo_subdir": "",
|
||||
|
||||
// Customizable commands for building, installing, and
|
||||
// uninstalling the project. See asv.conf.json documentation.
|
||||
//
|
||||
"install_command": ["python -m pip install ."],
|
||||
"uninstall_command": ["return-code=any python -m pip uninstall -y {project}"],
|
||||
"build_command": ["python -m pip wheel -w {build_cache_dir} {build_dir}"],
|
||||
|
||||
// List of branches to benchmark. If not provided, defaults to "master"
|
||||
// (for git) or "default" (for mercurial).
|
||||
// "branches": ["master"], // for git
|
||||
// "branches": ["default"], // for mercurial
|
||||
|
||||
// The DVCS being used. If not set, it will be automatically
|
||||
// determined from "repo" by looking at the protocol in the URL
|
||||
// (if remote), or by looking for special directories, such as
|
||||
// ".git" (if local).
|
||||
// "dvcs": "git",
|
||||
|
||||
// The tool to use to create environments. May be "conda",
|
||||
// "virtualenv" or other value depending on the plugins in use.
|
||||
// If missing or the empty string, the tool will be automatically
|
||||
// determined by looking for tools on the PATH environment
|
||||
// variable.
|
||||
"environment_type": "virtualenv",
|
||||
|
||||
// timeout in seconds for installing any dependencies in environment
|
||||
// defaults to 10 min
|
||||
//"install_timeout": 600,
|
||||
|
||||
// the base URL to show a commit for the project.
|
||||
// "show_commit_url": "http://github.com/owner/project/commit/",
|
||||
|
||||
// The Pythons you'd like to test against. If not provided, defaults
|
||||
// to the current version of Python used to run `asv`.
|
||||
// "pythons": ["2.7", "3.6"],
|
||||
|
||||
// The list of conda channel names to be searched for benchmark
|
||||
// dependency packages in the specified order
|
||||
// "conda_channels": ["conda-forge", "defaults"],
|
||||
|
||||
// The matrix of dependencies to test. Each key is the name of a
|
||||
// package (in PyPI) and the values are version numbers. An empty
|
||||
// list or empty string indicates to just test against the default
|
||||
// (latest) version. null indicates that the package is to not be
|
||||
// installed. If the package to be tested is only available from
|
||||
// PyPi, and the 'environment_type' is conda, then you can preface
|
||||
// the package name by 'pip+', and the package will be installed via
|
||||
// pip (with all the conda available packages installed first,
|
||||
// followed by the pip installed packages).
|
||||
//
|
||||
// "matrix": {
|
||||
// "numpy": ["1.6", "1.7"],
|
||||
// "six": ["", null], // test with and without six installed
|
||||
// "pip+emcee": [""], // emcee is only available for install with pip.
|
||||
// },
|
||||
|
||||
// Combinations of libraries/python versions can be excluded/included
|
||||
// from the set to test. Each entry is a dictionary containing additional
|
||||
// key-value pairs to include/exclude.
|
||||
//
|
||||
// An exclude entry excludes entries where all values match. The
|
||||
// values are regexps that should match the whole string.
|
||||
//
|
||||
// An include entry adds an environment. Only the packages listed
|
||||
// are installed. The 'python' key is required. The exclude rules
|
||||
// do not apply to includes.
|
||||
//
|
||||
// In addition to package names, the following keys are available:
|
||||
//
|
||||
// - python
|
||||
// Python version, as in the *pythons* variable above.
|
||||
// - environment_type
|
||||
// Environment type, as above.
|
||||
// - sys_platform
|
||||
// Platform, as in sys.platform. Possible values for the common
|
||||
// cases: 'linux2', 'win32', 'cygwin', 'darwin'.
|
||||
//
|
||||
// "exclude": [
|
||||
// {"python": "3.2", "sys_platform": "win32"}, // skip py3.2 on windows
|
||||
// {"environment_type": "conda", "six": null}, // don't run without six on conda
|
||||
// ],
|
||||
//
|
||||
// "include": [
|
||||
// // additional env for python2.7
|
||||
// {"python": "2.7", "numpy": "1.8"},
|
||||
// // additional env if run on windows+conda
|
||||
// {"platform": "win32", "environment_type": "conda", "python": "2.7", "libpython": ""},
|
||||
// ],
|
||||
|
||||
// The directory (relative to the current directory) that benchmarks are
|
||||
// stored in. If not provided, defaults to "benchmarks"
|
||||
// "benchmark_dir": "benchmarks",
|
||||
|
||||
// The directory (relative to the current directory) to cache the Python
|
||||
// environments in. If not provided, defaults to "env"
|
||||
"env_dir": ".asv/env",
|
||||
|
||||
// The directory (relative to the current directory) that raw benchmark
|
||||
// results are stored in. If not provided, defaults to "results".
|
||||
"results_dir": ".asv/results",
|
||||
|
||||
// The directory (relative to the current directory) that the html tree
|
||||
// should be written to. If not provided, defaults to "html".
|
||||
"html_dir": ".asv/html",
|
||||
|
||||
// The number of characters to retain in the commit hashes.
|
||||
// "hash_length": 8,
|
||||
|
||||
// `asv` will cache results of the recent builds in each
|
||||
// environment, making them faster to install next time. This is
|
||||
// the number of builds to keep, per environment.
|
||||
// "build_cache_size": 2,
|
||||
|
||||
// The commits after which the regression search in `asv publish`
|
||||
// should start looking for regressions. Dictionary whose keys are
|
||||
// regexps matching to benchmark names, and values corresponding to
|
||||
// the commit (exclusive) after which to start looking for
|
||||
// regressions. The default is to start from the first commit
|
||||
// with results. If the commit is `null`, regression detection is
|
||||
// skipped for the matching benchmark.
|
||||
//
|
||||
// "regressions_first_commits": {
|
||||
// "some_benchmark": "352cdf", // Consider regressions only after this commit
|
||||
// "another_benchmark": null, // Skip regression detection altogether
|
||||
// },
|
||||
|
||||
// The thresholds for relative change in results, after which `asv
|
||||
// publish` starts reporting regressions. Dictionary of the same
|
||||
// form as in ``regressions_first_commits``, with values
|
||||
// indicating the thresholds. If multiple entries match, the
|
||||
// maximum is taken. If no entry matches, the default is 5%.
|
||||
//
|
||||
// "regressions_thresholds": {
|
||||
// "some_benchmark": 0.01, // Threshold of 1%
|
||||
// "another_benchmark": 0.5, // Threshold of 50%
|
||||
// },
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
|
||||
@@ -1,128 +0,0 @@
|
||||
import betterproto
|
||||
from dataclasses import dataclass
|
||||
|
||||
from typing import List
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestMessage(betterproto.Message):
|
||||
foo: int = betterproto.uint32_field(0)
|
||||
bar: str = betterproto.string_field(1)
|
||||
baz: float = betterproto.float_field(2)
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestNestedChildMessage(betterproto.Message):
|
||||
str_key: str = betterproto.string_field(0)
|
||||
bytes_key: bytes = betterproto.bytes_field(1)
|
||||
bool_key: bool = betterproto.bool_field(2)
|
||||
float_key: float = betterproto.float_field(3)
|
||||
int_key: int = betterproto.uint64_field(4)
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestNestedMessage(betterproto.Message):
|
||||
foo: TestNestedChildMessage = betterproto.message_field(0)
|
||||
bar: TestNestedChildMessage = betterproto.message_field(1)
|
||||
baz: TestNestedChildMessage = betterproto.message_field(2)
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestRepeatedMessage(betterproto.Message):
|
||||
foo_repeat: List[str] = betterproto.string_field(0)
|
||||
bar_repeat: List[int] = betterproto.int64_field(1)
|
||||
baz_repeat: List[bool] = betterproto.bool_field(2)
|
||||
|
||||
|
||||
class BenchMessage:
|
||||
"""Test creation and usage a proto message."""
|
||||
|
||||
def setup(self):
|
||||
self.cls = TestMessage
|
||||
self.instance = TestMessage()
|
||||
self.instance_filled = TestMessage(0, "test", 0.0)
|
||||
self.instance_filled_bytes = bytes(self.instance_filled)
|
||||
self.instance_filled_nested = TestNestedMessage(
|
||||
TestNestedChildMessage("foo", bytearray(b"test1"), True, 0.1234, 500),
|
||||
TestNestedChildMessage("bar", bytearray(b"test2"), True, 3.1415, -302),
|
||||
TestNestedChildMessage("baz", bytearray(b"test3"), False, 1e5, 300),
|
||||
)
|
||||
self.instance_filled_nested_bytes = bytes(self.instance_filled_nested)
|
||||
self.instance_filled_repeated = TestRepeatedMessage(
|
||||
[
|
||||
"test1",
|
||||
"test2",
|
||||
"test3",
|
||||
"test4",
|
||||
"test5",
|
||||
"test6",
|
||||
"test7",
|
||||
"test8",
|
||||
"test9",
|
||||
"test10",
|
||||
],
|
||||
[2, -100, 0, 500000, 600, -425678, 1000000000, -300, 1, -694214214466],
|
||||
[True, False, False, False, True, True, False, True, False, False],
|
||||
)
|
||||
self.instance_filled_repeated_bytes = bytes(self.instance_filled_repeated)
|
||||
|
||||
def time_overhead(self):
|
||||
"""Overhead in class definition."""
|
||||
|
||||
@dataclass
|
||||
class Message(betterproto.Message):
|
||||
foo: int = betterproto.uint32_field(0)
|
||||
bar: str = betterproto.string_field(1)
|
||||
baz: float = betterproto.float_field(2)
|
||||
|
||||
def time_instantiation(self):
|
||||
"""Time instantiation"""
|
||||
self.cls()
|
||||
|
||||
def time_attribute_access(self):
|
||||
"""Time to access an attribute"""
|
||||
self.instance.foo
|
||||
self.instance.bar
|
||||
self.instance.baz
|
||||
|
||||
def time_init_with_values(self):
|
||||
"""Time to set an attribute"""
|
||||
self.cls(0, "test", 0.0)
|
||||
|
||||
def time_attribute_setting(self):
|
||||
"""Time to set attributes"""
|
||||
self.instance.foo = 0
|
||||
self.instance.bar = "test"
|
||||
self.instance.baz = 0.0
|
||||
|
||||
def time_serialize(self):
|
||||
"""Time serializing a message to wire."""
|
||||
bytes(self.instance_filled)
|
||||
|
||||
def time_deserialize(self):
|
||||
"""Time deserialize a message."""
|
||||
TestMessage().parse(self.instance_filled_bytes)
|
||||
|
||||
def time_serialize_nested(self):
|
||||
"""Time serializing a nested message to wire."""
|
||||
bytes(self.instance_filled_nested)
|
||||
|
||||
def time_deserialize_nested(self):
|
||||
"""Time deserialize a nested message."""
|
||||
TestNestedMessage().parse(self.instance_filled_nested_bytes)
|
||||
|
||||
def time_serialize_repeated(self):
|
||||
"""Time serializing a repeated message to wire."""
|
||||
bytes(self.instance_filled_repeated)
|
||||
|
||||
def time_deserialize_repeated(self):
|
||||
"""Time deserialize a repeated message."""
|
||||
TestRepeatedMessage().parse(self.instance_filled_repeated_bytes)
|
||||
|
||||
|
||||
class MemSuite:
|
||||
def setup(self):
|
||||
self.cls = TestMessage
|
||||
|
||||
def mem_instance(self):
|
||||
return self.cls()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,8 +1,8 @@
|
||||
from typing import TYPE_CHECKING, TypeVar
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from grpclib._typing import IProtoMessage
|
||||
from . import Message
|
||||
from grpclib._protocols import IProtoMessage
|
||||
|
||||
# Bound type variable to allow methods to return `self` of subclasses
|
||||
T = TypeVar("T", bound="Message")
|
||||
@@ -1,4 +1,3 @@
|
||||
import keyword
|
||||
import re
|
||||
|
||||
# Word delimiters and symbols that will not be preserved when re-casing.
|
||||
@@ -17,28 +16,51 @@ WORD_UPPER = "[A-Z]+(?![a-z])[0-9]*"
|
||||
def safe_snake_case(value: str) -> str:
|
||||
"""Snake case a value taking into account Python keywords."""
|
||||
value = snake_case(value)
|
||||
value = sanitize_name(value)
|
||||
if value in [
|
||||
"and",
|
||||
"as",
|
||||
"assert",
|
||||
"break",
|
||||
"class",
|
||||
"continue",
|
||||
"def",
|
||||
"del",
|
||||
"elif",
|
||||
"else",
|
||||
"except",
|
||||
"finally",
|
||||
"for",
|
||||
"from",
|
||||
"global",
|
||||
"if",
|
||||
"import",
|
||||
"in",
|
||||
"is",
|
||||
"lambda",
|
||||
"nonlocal",
|
||||
"not",
|
||||
"or",
|
||||
"pass",
|
||||
"raise",
|
||||
"return",
|
||||
"try",
|
||||
"while",
|
||||
"with",
|
||||
"yield",
|
||||
]:
|
||||
# https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles
|
||||
value += "_"
|
||||
return value
|
||||
|
||||
|
||||
def snake_case(value: str, strict: bool = True) -> str:
|
||||
def snake_case(value: str, strict: bool = True):
|
||||
"""
|
||||
Join words with an underscore into lowercase and remove symbols.
|
||||
|
||||
Parameters
|
||||
-----------
|
||||
value: :class:`str`
|
||||
The value to convert.
|
||||
strict: :class:`bool`
|
||||
Whether or not to force single underscores.
|
||||
|
||||
Returns
|
||||
--------
|
||||
:class:`str`
|
||||
The value in snake_case.
|
||||
@param value: value to convert
|
||||
@param strict: force single underscores
|
||||
"""
|
||||
|
||||
def substitute_word(symbols: str, word: str, is_start: bool) -> str:
|
||||
def substitute_word(symbols, word, is_start):
|
||||
if not word:
|
||||
return ""
|
||||
if strict:
|
||||
@@ -62,21 +84,11 @@ def snake_case(value: str, strict: bool = True) -> str:
|
||||
return snake
|
||||
|
||||
|
||||
def pascal_case(value: str, strict: bool = True) -> str:
|
||||
def pascal_case(value: str, strict: bool = True):
|
||||
"""
|
||||
Capitalize each word and remove symbols.
|
||||
|
||||
Parameters
|
||||
-----------
|
||||
value: :class:`str`
|
||||
The value to convert.
|
||||
strict: :class:`bool`
|
||||
Whether or not to output only alphanumeric characters.
|
||||
|
||||
Returns
|
||||
--------
|
||||
:class:`str`
|
||||
The value in PascalCase.
|
||||
@param value: value to convert
|
||||
@param strict: output only alphanumeric characters
|
||||
"""
|
||||
|
||||
def substitute_word(symbols, word):
|
||||
@@ -97,52 +109,12 @@ def pascal_case(value: str, strict: bool = True) -> str:
|
||||
)
|
||||
|
||||
|
||||
def camel_case(value: str, strict: bool = True) -> str:
|
||||
def camel_case(value: str, strict: bool = True):
|
||||
"""
|
||||
Capitalize all words except first and remove symbols.
|
||||
|
||||
Parameters
|
||||
-----------
|
||||
value: :class:`str`
|
||||
The value to convert.
|
||||
strict: :class:`bool`
|
||||
Whether or not to output only alphanumeric characters.
|
||||
|
||||
Returns
|
||||
--------
|
||||
:class:`str`
|
||||
The value in camelCase.
|
||||
"""
|
||||
return lowercase_first(pascal_case(value, strict=strict))
|
||||
|
||||
|
||||
def lowercase_first(value: str) -> str:
|
||||
"""
|
||||
Lower cases the first character of the value.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
value: :class:`str`
|
||||
The value to lower case.
|
||||
|
||||
Returns
|
||||
-------
|
||||
:class:`str`
|
||||
The lower cased string.
|
||||
"""
|
||||
def lowercase_first(value: str):
|
||||
return value[0:1].lower() + value[1:]
|
||||
|
||||
|
||||
def is_reserved_name(value: str) -> bool:
|
||||
if keyword.iskeyword(value):
|
||||
return True
|
||||
|
||||
if value in ("bytes", "str"):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def sanitize_name(value: str) -> str:
|
||||
# https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles
|
||||
return f"{value}_" if is_reserved_name(value) else value
|
||||
@@ -1,10 +1,10 @@
|
||||
import os
|
||||
import re
|
||||
from typing import Dict, List, Set, Tuple, Type
|
||||
from typing import Dict, List, Set, Type
|
||||
|
||||
from ..casing import safe_snake_case
|
||||
from ..lib.google import protobuf as google_protobuf
|
||||
from .naming import pythonize_class_name
|
||||
from betterproto import safe_snake_case
|
||||
from betterproto.compile.naming import pythonize_class_name
|
||||
from betterproto.lib.google import protobuf as google_protobuf
|
||||
|
||||
WRAPPER_TYPES: Dict[str, Type] = {
|
||||
".google.protobuf.DoubleValue": google_protobuf.DoubleValue,
|
||||
@@ -19,7 +19,7 @@ WRAPPER_TYPES: Dict[str, Type] = {
|
||||
}
|
||||
|
||||
|
||||
def parse_source_type_name(field_type_name: str) -> Tuple[str, str]:
|
||||
def parse_source_type_name(field_type_name):
|
||||
"""
|
||||
Split full source type name into package and type name.
|
||||
E.g. 'root.package.Message' -> ('root.package', 'Message')
|
||||
@@ -36,7 +36,7 @@ def parse_source_type_name(field_type_name: str) -> Tuple[str, str]:
|
||||
|
||||
|
||||
def get_type_reference(
|
||||
package: str, imports: set, source_type: str, unwrap: bool = True
|
||||
package: str, imports: set, source_type: str, unwrap: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
Return a Python type name for a proto type reference. Adds the import if
|
||||
@@ -50,7 +50,7 @@ def get_type_reference(
|
||||
if source_type == ".google.protobuf.Duration":
|
||||
return "timedelta"
|
||||
|
||||
elif source_type == ".google.protobuf.Timestamp":
|
||||
if source_type == ".google.protobuf.Timestamp":
|
||||
return "datetime"
|
||||
|
||||
source_package, source_type = parse_source_type_name(source_type)
|
||||
@@ -79,14 +79,14 @@ def get_type_reference(
|
||||
return reference_cousin(current_package, imports, py_package, py_type)
|
||||
|
||||
|
||||
def reference_absolute(imports: Set[str], py_package: List[str], py_type: str) -> str:
|
||||
def reference_absolute(imports, py_package, py_type):
|
||||
"""
|
||||
Returns a reference to a python type located in the root, i.e. sys.path.
|
||||
"""
|
||||
string_import = ".".join(py_package)
|
||||
string_alias = safe_snake_case(string_import)
|
||||
imports.add(f"import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
return f"{string_alias}.{py_type}"
|
||||
|
||||
|
||||
def reference_sibling(py_type: str) -> str:
|
||||
@@ -100,9 +100,8 @@ def reference_descendent(
|
||||
current_package: List[str], imports: Set[str], py_package: List[str], py_type: str
|
||||
) -> str:
|
||||
"""
|
||||
Returns a reference to a python type in a package that is a descendent of the
|
||||
current package, and adds the required import that is aliased to avoid name
|
||||
conflicts.
|
||||
Returns a reference to a python type in a package that is a descendent of the current package,
|
||||
and adds the required import that is aliased to avoid name conflicts.
|
||||
"""
|
||||
importing_descendent = py_package[len(current_package) :]
|
||||
string_from = ".".join(importing_descendent[:-1])
|
||||
@@ -110,19 +109,18 @@ def reference_descendent(
|
||||
if string_from:
|
||||
string_alias = "_".join(importing_descendent)
|
||||
imports.add(f"from .{string_from} import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
return f"{string_alias}.{py_type}"
|
||||
else:
|
||||
imports.add(f"from . import {string_import}")
|
||||
return f'"{string_import}.{py_type}"'
|
||||
return f"{string_import}.{py_type}"
|
||||
|
||||
|
||||
def reference_ancestor(
|
||||
current_package: List[str], imports: Set[str], py_package: List[str], py_type: str
|
||||
) -> str:
|
||||
"""
|
||||
Returns a reference to a python type in a package which is an ancestor to the
|
||||
current package, and adds the required import that is aliased (if possible) to avoid
|
||||
name conflicts.
|
||||
Returns a reference to a python type in a package which is an ancestor to the current package,
|
||||
and adds the required import that is aliased (if possible) to avoid name conflicts.
|
||||
|
||||
Adds trailing __ to avoid name mangling (python.org/dev/peps/pep-0008/#id34).
|
||||
"""
|
||||
@@ -132,21 +130,21 @@ def reference_ancestor(
|
||||
string_alias = f"_{'_' * distance_up}{string_import}__"
|
||||
string_from = f"..{'.' * distance_up}"
|
||||
imports.add(f"from {string_from} import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
return f"{string_alias}.{py_type}"
|
||||
else:
|
||||
string_alias = f"{'_' * distance_up}{py_type}__"
|
||||
imports.add(f"from .{'.' * distance_up} import {py_type} as {string_alias}")
|
||||
return f'"{string_alias}"'
|
||||
return string_alias
|
||||
|
||||
|
||||
def reference_cousin(
|
||||
current_package: List[str], imports: Set[str], py_package: List[str], py_type: str
|
||||
) -> str:
|
||||
"""
|
||||
Returns a reference to a python type in a package that is not descendent, ancestor
|
||||
or sibling, and adds the required import that is aliased to avoid name conflicts.
|
||||
Returns a reference to a python type in a package that is not descendent, ancestor or sibling,
|
||||
and adds the required import that is aliased to avoid name conflicts.
|
||||
"""
|
||||
shared_ancestry = os.path.commonprefix([current_package, py_package]) # type: ignore
|
||||
shared_ancestry = os.path.commonprefix([current_package, py_package])
|
||||
distance_up = len(current_package) - len(shared_ancestry)
|
||||
string_from = f".{'.' * distance_up}" + ".".join(
|
||||
py_package[len(shared_ancestry) : -1]
|
||||
@@ -159,4 +157,4 @@ def reference_cousin(
|
||||
+ "__"
|
||||
)
|
||||
imports.add(f"from {string_from} import {string_import} as {string_alias}")
|
||||
return f'"{string_alias}.{py_type}"'
|
||||
return f"{string_alias}.{py_type}"
|
||||
@@ -1,13 +1,13 @@
|
||||
from betterproto import casing
|
||||
|
||||
|
||||
def pythonize_class_name(name: str) -> str:
|
||||
def pythonize_class_name(name):
|
||||
return casing.pascal_case(name)
|
||||
|
||||
|
||||
def pythonize_field_name(name: str) -> str:
|
||||
def pythonize_field_name(name: str):
|
||||
return casing.safe_snake_case(name)
|
||||
|
||||
|
||||
def pythonize_method_name(name: str) -> str:
|
||||
def pythonize_method_name(name: str):
|
||||
return casing.safe_snake_case(name)
|
||||
@@ -1,7 +1,8 @@
|
||||
import asyncio
|
||||
from abc import ABC
|
||||
import asyncio
|
||||
import grpclib.const
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
AsyncIterable,
|
||||
AsyncIterator,
|
||||
Collection,
|
||||
@@ -9,23 +10,21 @@ from typing import (
|
||||
Mapping,
|
||||
Optional,
|
||||
Tuple,
|
||||
TYPE_CHECKING,
|
||||
Type,
|
||||
Union,
|
||||
)
|
||||
|
||||
import grpclib.const
|
||||
|
||||
from .._types import ST, T
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from grpclib.client import Channel
|
||||
from grpclib._protocols import IProtoMessage
|
||||
from grpclib.client import Channel, Stream
|
||||
from grpclib.metadata import Deadline
|
||||
|
||||
|
||||
_Value = Union[str, bytes]
|
||||
_MetadataLike = Union[Mapping[str, _Value], Collection[Tuple[str, _Value]]]
|
||||
_MessageLike = Union[T, ST]
|
||||
_MessageSource = Union[Iterable[ST], AsyncIterable[ST]]
|
||||
_MessageSource = Union[Iterable["IProtoMessage"], AsyncIterable["IProtoMessage"]]
|
||||
|
||||
|
||||
class ServiceStub(ABC):
|
||||
@@ -61,7 +60,7 @@ class ServiceStub(ABC):
|
||||
async def _unary_unary(
|
||||
self,
|
||||
route: str,
|
||||
request: _MessageLike,
|
||||
request: "IProtoMessage",
|
||||
response_type: Type[T],
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
@@ -78,13 +77,13 @@ class ServiceStub(ABC):
|
||||
) as stream:
|
||||
await stream.send_message(request, end=True)
|
||||
response = await stream.recv_message()
|
||||
assert response is not None
|
||||
return response
|
||||
assert response is not None
|
||||
return response
|
||||
|
||||
async def _unary_stream(
|
||||
self,
|
||||
route: str,
|
||||
request: _MessageLike,
|
||||
request: "IProtoMessage",
|
||||
response_type: Type[T],
|
||||
*,
|
||||
timeout: Optional[float] = None,
|
||||
@@ -124,8 +123,8 @@ class ServiceStub(ABC):
|
||||
) as stream:
|
||||
await self._send_messages(stream, request_iterator)
|
||||
response = await stream.recv_message()
|
||||
assert response is not None
|
||||
return response
|
||||
assert response is not None
|
||||
return response
|
||||
|
||||
async def _stream_stream(
|
||||
self,
|
||||
@@ -1,5 +1,12 @@
|
||||
import asyncio
|
||||
from typing import AsyncIterable, AsyncIterator, Iterable, Optional, TypeVar, Union
|
||||
from typing import (
|
||||
AsyncIterable,
|
||||
AsyncIterator,
|
||||
Iterable,
|
||||
Optional,
|
||||
TypeVar,
|
||||
Union,
|
||||
)
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
@@ -9,53 +16,57 @@ class ChannelClosed(Exception):
|
||||
An exception raised on an attempt to send through a closed channel
|
||||
"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class ChannelDone(Exception):
|
||||
"""
|
||||
An exception raised on an attempt to send receive from a channel that is both closed
|
||||
An exception raised on an attempt to send recieve from a channel that is both closed
|
||||
and empty.
|
||||
"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class AsyncChannel(AsyncIterable[T]):
|
||||
"""
|
||||
A buffered async channel for sending items between coroutines with FIFO ordering.
|
||||
|
||||
This makes decoupled bidirectional steaming gRPC requests easy if used like:
|
||||
This makes decoupled bidirection steaming gRPC requests easy if used like:
|
||||
|
||||
.. code-block:: python
|
||||
client = GeneratedStub(grpclib_chan)
|
||||
request_channel = await AsyncChannel()
|
||||
request_chan = await AsyncChannel()
|
||||
# We can start be sending all the requests we already have
|
||||
await request_channel.send_from([RequestObject(...), RequestObject(...)])
|
||||
async for response in client.rpc_call(request_channel):
|
||||
await request_chan.send_from([ReqestObject(...), ReqestObject(...)])
|
||||
async for response in client.rpc_call(request_chan):
|
||||
# The response iterator will remain active until the connection is closed
|
||||
...
|
||||
# More items can be sent at any time
|
||||
await request_channel.send(RequestObject(...))
|
||||
await request_chan.send(ReqestObject(...))
|
||||
...
|
||||
# The channel must be closed to complete the gRPC connection
|
||||
request_channel.close()
|
||||
request_chan.close()
|
||||
|
||||
Items can be sent through the channel by either:
|
||||
- providing an iterable to the send_from method
|
||||
- passing them to the send method one at a time
|
||||
|
||||
Items can be received from the channel by either:
|
||||
Items can be recieved from the channel by either:
|
||||
- iterating over the channel with a for loop to get all items
|
||||
- calling the receive method to get one item at a time
|
||||
- calling the recieve method to get one item at a time
|
||||
|
||||
If the channel is empty then receivers will wait until either an item appears or the
|
||||
If the channel is empty then recievers will wait until either an item appears or the
|
||||
channel is closed.
|
||||
|
||||
Once the channel is closed then subsequent attempt to send through the channel will
|
||||
fail with a ChannelClosed exception.
|
||||
|
||||
When th channel is closed and empty then it is done, and further attempts to receive
|
||||
When th channel is closed and empty then it is done, and further attempts to recieve
|
||||
from it will fail with a ChannelDone exception
|
||||
|
||||
If multiple coroutines receive from the channel concurrently, each item sent will be
|
||||
received by only one of the receivers.
|
||||
If multiple coroutines recieve from the channel concurrently, each item sent will be
|
||||
recieved by only one of the recievers.
|
||||
|
||||
:param source:
|
||||
An optional iterable will items that should be sent through the channel
|
||||
@@ -63,16 +74,18 @@ class AsyncChannel(AsyncIterable[T]):
|
||||
:param buffer_limit:
|
||||
Limit the number of items that can be buffered in the channel, A value less than
|
||||
1 implies no limit. If the channel is full then attempts to send more items will
|
||||
result in the sender waiting until an item is received from the channel.
|
||||
result in the sender waiting until an item is recieved from the channel.
|
||||
:param close:
|
||||
If set to True then the channel will automatically close after exhausting source
|
||||
or immediately if no source is provided.
|
||||
"""
|
||||
|
||||
def __init__(self, *, buffer_limit: int = 0, close: bool = False):
|
||||
self._queue: asyncio.Queue[T] = asyncio.Queue(buffer_limit)
|
||||
def __init__(
|
||||
self, *, buffer_limit: int = 0, close: bool = False,
|
||||
):
|
||||
self._queue: asyncio.Queue[Union[T, object]] = asyncio.Queue(buffer_limit)
|
||||
self._closed = False
|
||||
self._waiting_receivers: int = 0
|
||||
self._waiting_recievers: int = 0
|
||||
# Track whether flush has been invoked so it can only happen once
|
||||
self._flushed = False
|
||||
|
||||
@@ -82,14 +95,14 @@ class AsyncChannel(AsyncIterable[T]):
|
||||
async def __anext__(self) -> T:
|
||||
if self.done():
|
||||
raise StopAsyncIteration
|
||||
self._waiting_receivers += 1
|
||||
self._waiting_recievers += 1
|
||||
try:
|
||||
result = await self._queue.get()
|
||||
if result is self.__flush:
|
||||
raise StopAsyncIteration
|
||||
return result
|
||||
finally:
|
||||
self._waiting_receivers -= 1
|
||||
self._waiting_recievers -= 1
|
||||
self._queue.task_done()
|
||||
|
||||
def closed(self) -> bool:
|
||||
@@ -103,12 +116,12 @@ class AsyncChannel(AsyncIterable[T]):
|
||||
Check if this channel is done.
|
||||
|
||||
:return: True if this channel is closed and and has been drained of items in
|
||||
which case any further attempts to receive an item from this channel will raise
|
||||
which case any further attempts to recieve an item from this channel will raise
|
||||
a ChannelDone exception.
|
||||
"""
|
||||
# After close the channel is not yet done until there is at least one waiting
|
||||
# receiver per enqueued item.
|
||||
return self._closed and self._queue.qsize() <= self._waiting_receivers
|
||||
# reciever per enqueued item.
|
||||
return self._closed and self._queue.qsize() <= self._waiting_recievers
|
||||
|
||||
async def send_from(
|
||||
self, source: Union[Iterable[T], AsyncIterable[T]], close: bool = False
|
||||
@@ -145,22 +158,22 @@ class AsyncChannel(AsyncIterable[T]):
|
||||
await self._queue.put(item)
|
||||
return self
|
||||
|
||||
async def receive(self) -> Optional[T]:
|
||||
async def recieve(self) -> Optional[T]:
|
||||
"""
|
||||
Returns the next item from this channel when it becomes available,
|
||||
or None if the channel is closed before another item is sent.
|
||||
:return: An item from the channel
|
||||
"""
|
||||
if self.done():
|
||||
raise ChannelDone("Cannot receive from a closed channel")
|
||||
self._waiting_receivers += 1
|
||||
raise ChannelDone("Cannot recieve from a closed channel")
|
||||
self._waiting_recievers += 1
|
||||
try:
|
||||
result = await self._queue.get()
|
||||
if result is self.__flush:
|
||||
return None
|
||||
return result
|
||||
finally:
|
||||
self._waiting_receivers -= 1
|
||||
self._waiting_recievers -= 1
|
||||
self._queue.task_done()
|
||||
|
||||
def close(self):
|
||||
@@ -177,8 +190,8 @@ class AsyncChannel(AsyncIterable[T]):
|
||||
"""
|
||||
if not self._flushed:
|
||||
self._flushed = True
|
||||
deadlocked_receivers = max(0, self._waiting_receivers - self._queue.qsize())
|
||||
for _ in range(deadlocked_receivers):
|
||||
deadlocked_recievers = max(0, self._waiting_recievers - self._queue.qsize())
|
||||
for _ in range(deadlocked_recievers):
|
||||
await self._queue.put(self.__flush)
|
||||
|
||||
# A special signal object for flushing the queue when the channel is closed
|
||||
File diff suppressed because it is too large
Load Diff
2
betterproto/plugin.bat
Normal file
2
betterproto/plugin.bat
Normal file
@@ -0,0 +1,2 @@
|
||||
@SET plugin_dir=%~dp0
|
||||
@python %plugin_dir%/plugin.py %*
|
||||
403
betterproto/plugin.py
Executable file
403
betterproto/plugin.py
Executable file
@@ -0,0 +1,403 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import itertools
|
||||
import os.path
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
import textwrap
|
||||
from typing import List, Union
|
||||
|
||||
import betterproto
|
||||
from betterproto.compile.importing import get_type_reference
|
||||
from betterproto.compile.naming import (
|
||||
pythonize_class_name,
|
||||
pythonize_field_name,
|
||||
pythonize_method_name,
|
||||
)
|
||||
|
||||
try:
|
||||
# betterproto[compiler] specific dependencies
|
||||
import black
|
||||
from google.protobuf.compiler import plugin_pb2 as plugin
|
||||
from google.protobuf.descriptor_pb2 import (
|
||||
DescriptorProto,
|
||||
EnumDescriptorProto,
|
||||
FieldDescriptorProto,
|
||||
)
|
||||
import google.protobuf.wrappers_pb2 as google_wrappers
|
||||
import jinja2
|
||||
except ImportError as err:
|
||||
missing_import = err.args[0][17:-1]
|
||||
print(
|
||||
"\033[31m"
|
||||
f"Unable to import `{missing_import}` from betterproto plugin! "
|
||||
"Please ensure that you've installed betterproto as "
|
||||
'`pip install "betterproto[compiler]"` so that compiler dependencies '
|
||||
"are included."
|
||||
"\033[0m"
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
def py_type(package: str, imports: set, field: FieldDescriptorProto) -> str:
|
||||
if field.type in [1, 2]:
|
||||
return "float"
|
||||
elif field.type in [3, 4, 5, 6, 7, 13, 15, 16, 17, 18]:
|
||||
return "int"
|
||||
elif field.type == 8:
|
||||
return "bool"
|
||||
elif field.type == 9:
|
||||
return "str"
|
||||
elif field.type in [11, 14]:
|
||||
# Type referencing another defined Message or a named enum
|
||||
return get_type_reference(package, imports, field.type_name)
|
||||
elif field.type == 12:
|
||||
return "bytes"
|
||||
else:
|
||||
raise NotImplementedError(f"Unknown type {field.type}")
|
||||
|
||||
|
||||
def get_py_zero(type_num: int) -> Union[str, float]:
|
||||
zero: Union[str, float] = 0
|
||||
if type_num in []:
|
||||
zero = 0.0
|
||||
elif type_num == 8:
|
||||
zero = "False"
|
||||
elif type_num == 9:
|
||||
zero = '""'
|
||||
elif type_num == 11:
|
||||
zero = "None"
|
||||
elif type_num == 12:
|
||||
zero = 'b""'
|
||||
|
||||
return zero
|
||||
|
||||
|
||||
def traverse(proto_file):
|
||||
def _traverse(path, items, prefix=""):
|
||||
for i, item in enumerate(items):
|
||||
# Adjust the name since we flatten the heirarchy.
|
||||
item.name = next_prefix = prefix + item.name
|
||||
yield item, path + [i]
|
||||
|
||||
if isinstance(item, DescriptorProto):
|
||||
for enum in item.enum_type:
|
||||
enum.name = next_prefix + enum.name
|
||||
yield enum, path + [i, 4]
|
||||
|
||||
if item.nested_type:
|
||||
for n, p in _traverse(path + [i, 3], item.nested_type, next_prefix):
|
||||
yield n, p
|
||||
|
||||
return itertools.chain(
|
||||
_traverse([5], proto_file.enum_type), _traverse([4], proto_file.message_type)
|
||||
)
|
||||
|
||||
|
||||
def get_comment(proto_file, path: List[int], indent: int = 4) -> str:
|
||||
pad = " " * indent
|
||||
for sci in proto_file.source_code_info.location:
|
||||
# print(list(sci.path), path, file=sys.stderr)
|
||||
if list(sci.path) == path and sci.leading_comments:
|
||||
lines = textwrap.wrap(
|
||||
sci.leading_comments.strip().replace("\n", ""), width=79 - indent
|
||||
)
|
||||
|
||||
if path[-2] == 2 and path[-4] != 6:
|
||||
# This is a field
|
||||
return f"{pad}# " + f"\n{pad}# ".join(lines)
|
||||
else:
|
||||
# This is a message, enum, service, or method
|
||||
if len(lines) == 1 and len(lines[0]) < 79 - indent - 6:
|
||||
lines[0] = lines[0].strip('"')
|
||||
return f'{pad}"""{lines[0]}"""'
|
||||
else:
|
||||
joined = f"\n{pad}".join(lines)
|
||||
return f'{pad}"""\n{pad}{joined}\n{pad}"""'
|
||||
|
||||
return ""
|
||||
|
||||
|
||||
def generate_code(request, response):
|
||||
plugin_options = request.parameter.split(",") if request.parameter else []
|
||||
|
||||
env = jinja2.Environment(
|
||||
trim_blocks=True,
|
||||
lstrip_blocks=True,
|
||||
loader=jinja2.FileSystemLoader("%s/templates/" % os.path.dirname(__file__)),
|
||||
)
|
||||
template = env.get_template("template.py.j2")
|
||||
|
||||
output_map = {}
|
||||
for proto_file in request.proto_file:
|
||||
if (
|
||||
proto_file.package == "google.protobuf"
|
||||
and "INCLUDE_GOOGLE" not in plugin_options
|
||||
):
|
||||
continue
|
||||
|
||||
output_file = str(pathlib.Path(*proto_file.package.split("."), "__init__.py"))
|
||||
|
||||
if output_file not in output_map:
|
||||
output_map[output_file] = {"package": proto_file.package, "files": []}
|
||||
output_map[output_file]["files"].append(proto_file)
|
||||
|
||||
# TODO: Figure out how to handle gRPC request/response messages and add
|
||||
# processing below for Service.
|
||||
|
||||
for filename, options in output_map.items():
|
||||
package = options["package"]
|
||||
# print(package, filename, file=sys.stderr)
|
||||
output = {
|
||||
"package": package,
|
||||
"files": [f.name for f in options["files"]],
|
||||
"imports": set(),
|
||||
"datetime_imports": set(),
|
||||
"typing_imports": set(),
|
||||
"messages": [],
|
||||
"enums": [],
|
||||
"services": [],
|
||||
}
|
||||
|
||||
for proto_file in options["files"]:
|
||||
item: DescriptorProto
|
||||
for item, path in traverse(proto_file):
|
||||
data = {"name": item.name, "py_name": pythonize_class_name(item.name)}
|
||||
|
||||
if isinstance(item, DescriptorProto):
|
||||
# print(item, file=sys.stderr)
|
||||
if item.options.map_entry:
|
||||
# Skip generated map entry messages since we just use dicts
|
||||
continue
|
||||
|
||||
data.update(
|
||||
{
|
||||
"type": "Message",
|
||||
"comment": get_comment(proto_file, path),
|
||||
"properties": [],
|
||||
}
|
||||
)
|
||||
|
||||
for i, f in enumerate(item.field):
|
||||
t = py_type(package, output["imports"], f)
|
||||
zero = get_py_zero(f.type)
|
||||
|
||||
repeated = False
|
||||
packed = False
|
||||
|
||||
field_type = f.Type.Name(f.type).lower()[5:]
|
||||
|
||||
field_wraps = ""
|
||||
match_wrapper = re.match(
|
||||
r"\.google\.protobuf\.(.+)Value", f.type_name
|
||||
)
|
||||
if match_wrapper:
|
||||
wrapped_type = "TYPE_" + match_wrapper.group(1).upper()
|
||||
if hasattr(betterproto, wrapped_type):
|
||||
field_wraps = f"betterproto.{wrapped_type}"
|
||||
|
||||
map_types = None
|
||||
if f.type == 11:
|
||||
# This might be a map...
|
||||
message_type = f.type_name.split(".").pop().lower()
|
||||
# message_type = py_type(package)
|
||||
map_entry = f"{f.name.replace('_', '').lower()}entry"
|
||||
|
||||
if message_type == map_entry:
|
||||
for nested in item.nested_type:
|
||||
if (
|
||||
nested.name.replace("_", "").lower()
|
||||
== map_entry
|
||||
):
|
||||
if nested.options.map_entry:
|
||||
# print("Found a map!", file=sys.stderr)
|
||||
k = py_type(
|
||||
package,
|
||||
output["imports"],
|
||||
nested.field[0],
|
||||
)
|
||||
v = py_type(
|
||||
package,
|
||||
output["imports"],
|
||||
nested.field[1],
|
||||
)
|
||||
t = f"Dict[{k}, {v}]"
|
||||
field_type = "map"
|
||||
map_types = (
|
||||
f.Type.Name(nested.field[0].type),
|
||||
f.Type.Name(nested.field[1].type),
|
||||
)
|
||||
output["typing_imports"].add("Dict")
|
||||
|
||||
if f.label == 3 and field_type != "map":
|
||||
# Repeated field
|
||||
repeated = True
|
||||
t = f"List[{t}]"
|
||||
zero = "[]"
|
||||
output["typing_imports"].add("List")
|
||||
|
||||
if f.type in [1, 2, 3, 4, 5, 6, 7, 8, 13, 15, 16, 17, 18]:
|
||||
packed = True
|
||||
|
||||
one_of = ""
|
||||
if f.HasField("oneof_index"):
|
||||
one_of = item.oneof_decl[f.oneof_index].name
|
||||
|
||||
if "Optional[" in t:
|
||||
output["typing_imports"].add("Optional")
|
||||
|
||||
if "timedelta" in t:
|
||||
output["datetime_imports"].add("timedelta")
|
||||
elif "datetime" in t:
|
||||
output["datetime_imports"].add("datetime")
|
||||
|
||||
data["properties"].append(
|
||||
{
|
||||
"name": f.name,
|
||||
"py_name": pythonize_field_name(f.name),
|
||||
"number": f.number,
|
||||
"comment": get_comment(proto_file, path + [2, i]),
|
||||
"proto_type": int(f.type),
|
||||
"field_type": field_type,
|
||||
"field_wraps": field_wraps,
|
||||
"map_types": map_types,
|
||||
"type": t,
|
||||
"zero": zero,
|
||||
"repeated": repeated,
|
||||
"packed": packed,
|
||||
"one_of": one_of,
|
||||
}
|
||||
)
|
||||
# print(f, file=sys.stderr)
|
||||
|
||||
output["messages"].append(data)
|
||||
elif isinstance(item, EnumDescriptorProto):
|
||||
# print(item.name, path, file=sys.stderr)
|
||||
data.update(
|
||||
{
|
||||
"type": "Enum",
|
||||
"comment": get_comment(proto_file, path),
|
||||
"entries": [
|
||||
{
|
||||
"name": v.name,
|
||||
"value": v.number,
|
||||
"comment": get_comment(proto_file, path + [2, i]),
|
||||
}
|
||||
for i, v in enumerate(item.value)
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
output["enums"].append(data)
|
||||
|
||||
for i, service in enumerate(proto_file.service):
|
||||
# print(service, file=sys.stderr)
|
||||
|
||||
data = {
|
||||
"name": service.name,
|
||||
"py_name": pythonize_class_name(service.name),
|
||||
"comment": get_comment(proto_file, [6, i]),
|
||||
"methods": [],
|
||||
}
|
||||
|
||||
for j, method in enumerate(service.method):
|
||||
input_message = None
|
||||
input_type = get_type_reference(
|
||||
package, output["imports"], method.input_type
|
||||
).strip('"')
|
||||
for msg in output["messages"]:
|
||||
if msg["name"] == input_type:
|
||||
input_message = msg
|
||||
for field in msg["properties"]:
|
||||
if field["zero"] == "None":
|
||||
output["typing_imports"].add("Optional")
|
||||
break
|
||||
|
||||
data["methods"].append(
|
||||
{
|
||||
"name": method.name,
|
||||
"py_name": pythonize_method_name(method.name),
|
||||
"comment": get_comment(proto_file, [6, i, 2, j], indent=8),
|
||||
"route": f"/{package}.{service.name}/{method.name}",
|
||||
"input": get_type_reference(
|
||||
package, output["imports"], method.input_type
|
||||
).strip('"'),
|
||||
"input_message": input_message,
|
||||
"output": get_type_reference(
|
||||
package,
|
||||
output["imports"],
|
||||
method.output_type,
|
||||
unwrap=False,
|
||||
).strip('"'),
|
||||
"client_streaming": method.client_streaming,
|
||||
"server_streaming": method.server_streaming,
|
||||
}
|
||||
)
|
||||
|
||||
if method.client_streaming:
|
||||
output["typing_imports"].add("AsyncIterable")
|
||||
output["typing_imports"].add("Iterable")
|
||||
output["typing_imports"].add("Union")
|
||||
if method.server_streaming:
|
||||
output["typing_imports"].add("AsyncIterator")
|
||||
|
||||
output["services"].append(data)
|
||||
|
||||
output["imports"] = sorted(output["imports"])
|
||||
output["datetime_imports"] = sorted(output["datetime_imports"])
|
||||
output["typing_imports"] = sorted(output["typing_imports"])
|
||||
|
||||
# Fill response
|
||||
f = response.file.add()
|
||||
f.name = filename
|
||||
|
||||
# Render and then format the output file.
|
||||
f.content = black.format_str(
|
||||
template.render(description=output),
|
||||
mode=black.FileMode(target_versions=set([black.TargetVersion.PY37])),
|
||||
)
|
||||
|
||||
# Make each output directory a package with __init__ file
|
||||
output_paths = set(pathlib.Path(path) for path in output_map.keys())
|
||||
init_files = (
|
||||
set(
|
||||
directory.joinpath("__init__.py")
|
||||
for path in output_paths
|
||||
for directory in path.parents
|
||||
)
|
||||
- output_paths
|
||||
)
|
||||
|
||||
for init_file in init_files:
|
||||
init = response.file.add()
|
||||
init.name = str(init_file)
|
||||
|
||||
for filename in sorted(output_paths.union(init_files)):
|
||||
print(f"Writing {filename}", file=sys.stderr)
|
||||
|
||||
|
||||
def main():
|
||||
"""The plugin's main entry point."""
|
||||
# Read request message from stdin
|
||||
data = sys.stdin.buffer.read()
|
||||
|
||||
# Parse request
|
||||
request = plugin.CodeGeneratorRequest()
|
||||
request.ParseFromString(data)
|
||||
|
||||
# Create response
|
||||
response = plugin.CodeGeneratorResponse()
|
||||
|
||||
# Generate code
|
||||
generate_code(request, response)
|
||||
|
||||
# Serialise response message
|
||||
output = response.SerializeToString()
|
||||
|
||||
# Write to stdout
|
||||
sys.stdout.buffer.write(output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
135
betterproto/templates/template.py.j2
Normal file
135
betterproto/templates/template.py.j2
Normal file
@@ -0,0 +1,135 @@
|
||||
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
||||
# sources: {{ ', '.join(description.files) }}
|
||||
# plugin: python-betterproto
|
||||
from dataclasses import dataclass
|
||||
{% if description.datetime_imports %}
|
||||
from datetime import {% for i in description.datetime_imports %}{{ i }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||
|
||||
{% endif%}
|
||||
{% if description.typing_imports %}
|
||||
from typing import {% for i in description.typing_imports %}{{ i }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||
|
||||
{% endif %}
|
||||
|
||||
import betterproto
|
||||
{% if description.services %}
|
||||
import grpclib
|
||||
{% endif %}
|
||||
|
||||
{% for i in description.imports %}
|
||||
{{ i }}
|
||||
{% endfor %}
|
||||
|
||||
|
||||
{% if description.enums %}{% for enum in description.enums %}
|
||||
class {{ enum.py_name }}(betterproto.Enum):
|
||||
{% if enum.comment %}
|
||||
{{ enum.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for entry in enum.entries %}
|
||||
{% if entry.comment %}
|
||||
{{ entry.comment }}
|
||||
{% endif %}
|
||||
{{ entry.name }} = {{ entry.value }}
|
||||
{% endfor %}
|
||||
|
||||
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% for message in description.messages %}
|
||||
@dataclass
|
||||
class {{ message.py_name }}(betterproto.Message):
|
||||
{% if message.comment %}
|
||||
{{ message.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for field in message.properties %}
|
||||
{% if field.comment %}
|
||||
{{ field.comment }}
|
||||
{% endif %}
|
||||
{{ field.py_name }}: {{ field.type }} = betterproto.{{ field.field_type }}_field({{ field.number }}{% if field.field_type == 'map'%}, betterproto.{{ field.map_types[0] }}, betterproto.{{ field.map_types[1] }}{% endif %}{% if field.one_of %}, group="{{ field.one_of }}"{% endif %}{% if field.field_wraps %}, wraps={{ field.field_wraps }}{% endif %})
|
||||
{% endfor %}
|
||||
{% if not message.properties %}
|
||||
pass
|
||||
{% endif %}
|
||||
|
||||
|
||||
{% endfor %}
|
||||
{% for service in description.services %}
|
||||
class {{ service.py_name }}Stub(betterproto.ServiceStub):
|
||||
{% if service.comment %}
|
||||
{{ service.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% for method in service.methods %}
|
||||
async def {{ method.py_name }}(self
|
||||
{%- if not method.client_streaming -%}
|
||||
{%- if method.input_message and method.input_message.properties -%}, *,
|
||||
{%- for field in method.input_message.properties -%}
|
||||
{{ field.py_name }}: {% if field.zero == "None" and not field.type.startswith("Optional[") -%}
|
||||
Optional[{{ field.type }}]
|
||||
{%- else -%}
|
||||
{{ field.type }}
|
||||
{%- endif -%} = {{ field.zero }}
|
||||
{%- if not loop.last %}, {% endif -%}
|
||||
{%- endfor -%}
|
||||
{%- endif -%}
|
||||
{%- else -%}
|
||||
{# Client streaming: need a request iterator instead #}
|
||||
, request_iterator: Union[AsyncIterable["{{ method.input }}"], Iterable["{{ method.input }}"]]
|
||||
{%- endif -%}
|
||||
) -> {% if method.server_streaming %}AsyncIterator[{{ method.output }}]{% else %}{{ method.output }}{% endif %}:
|
||||
{% if method.comment %}
|
||||
{{ method.comment }}
|
||||
|
||||
{% endif %}
|
||||
{% if not method.client_streaming %}
|
||||
request = {{ method.input }}()
|
||||
{% for field in method.input_message.properties %}
|
||||
{% if field.field_type == 'message' %}
|
||||
if {{ field.py_name }} is not None:
|
||||
request.{{ field.py_name }} = {{ field.py_name }}
|
||||
{% else %}
|
||||
request.{{ field.py_name }} = {{ field.py_name }}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
{% if method.server_streaming %}
|
||||
{% if method.client_streaming %}
|
||||
async for response in self._stream_stream(
|
||||
"{{ method.route }}",
|
||||
request_iterator,
|
||||
{{ method.input }},
|
||||
{{ method.output }},
|
||||
):
|
||||
yield response
|
||||
{% else %}{# i.e. not client streaming #}
|
||||
async for response in self._unary_stream(
|
||||
"{{ method.route }}",
|
||||
request,
|
||||
{{ method.output }},
|
||||
):
|
||||
yield response
|
||||
|
||||
{% endif %}{# if client streaming #}
|
||||
{% else %}{# i.e. not server streaming #}
|
||||
{% if method.client_streaming %}
|
||||
return await self._stream_unary(
|
||||
"{{ method.route }}",
|
||||
request_iterator,
|
||||
{{ method.input }},
|
||||
{{ method.output }}
|
||||
)
|
||||
{% else %}{# i.e. not client streaming #}
|
||||
return await self._unary_unary(
|
||||
"{{ method.route }}",
|
||||
request,
|
||||
{{ method.output }}
|
||||
)
|
||||
{% endif %}{# client streaming #}
|
||||
{% endif %}
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
@@ -50,7 +50,7 @@ You can add multiple `.proto` files to the test case, as long as one file matche
|
||||
`test_<name>.py` — *Custom test to validate specific aspects of the generated class*
|
||||
|
||||
```python
|
||||
from tests.output_betterproto.bool.bool import Test
|
||||
from betterproto.tests.output_betterproto.bool.bool import Test
|
||||
|
||||
def test_value():
|
||||
message = Test()
|
||||
@@ -2,17 +2,18 @@
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Set
|
||||
|
||||
from tests.util import (
|
||||
from betterproto.tests.util import (
|
||||
get_directories,
|
||||
inputs_path,
|
||||
output_path_betterproto,
|
||||
output_path_reference,
|
||||
protoc,
|
||||
protoc_plugin,
|
||||
protoc_reference,
|
||||
)
|
||||
|
||||
# Force pure-python implementation instead of C++, otherwise imports
|
||||
@@ -88,8 +89,8 @@ async def generate_test_case_output(
|
||||
(ref_out, ref_err, ref_code),
|
||||
(plg_out, plg_err, plg_code),
|
||||
) = await asyncio.gather(
|
||||
protoc(test_case_input_path, test_case_output_path_reference, True),
|
||||
protoc(test_case_input_path, test_case_output_path_betterproto, False),
|
||||
protoc_reference(test_case_input_path, test_case_output_path_reference),
|
||||
protoc_plugin(test_case_input_path, test_case_output_path_betterproto),
|
||||
)
|
||||
|
||||
message = f"Generated output for {test_case_name!r}"
|
||||
@@ -135,10 +136,6 @@ def main():
|
||||
else:
|
||||
verbose = False
|
||||
whitelist = set(sys.argv[1:])
|
||||
|
||||
if platform.system() == "Windows":
|
||||
asyncio.set_event_loop(asyncio.ProactorEventLoop())
|
||||
|
||||
asyncio.get_event_loop().run_until_complete(generate(whitelist, verbose))
|
||||
|
||||
|
||||
@@ -1,15 +1,12 @@
|
||||
import asyncio
|
||||
import sys
|
||||
|
||||
from tests.output_betterproto.service.service import (
|
||||
DoThingRequest,
|
||||
from betterproto.tests.output_betterproto.service.service import (
|
||||
DoThingResponse,
|
||||
DoThingRequest,
|
||||
GetThingRequest,
|
||||
GetThingResponse,
|
||||
TestStub as ThingServiceClient,
|
||||
)
|
||||
import grpclib
|
||||
import grpclib.metadata
|
||||
import grpclib.server
|
||||
from grpclib.testing import ChannelFor
|
||||
import pytest
|
||||
from betterproto.grpc.util.async_channel import AsyncChannel
|
||||
@@ -21,92 +18,31 @@ async def _test_client(client, name="clean room", **kwargs):
|
||||
assert response.names == [name]
|
||||
|
||||
|
||||
def _assert_request_meta_received(deadline, metadata):
|
||||
def _assert_request_meta_recieved(deadline, metadata):
|
||||
def server_side_test(stream):
|
||||
assert stream.deadline._timestamp == pytest.approx(
|
||||
deadline._timestamp, 1
|
||||
), "The provided deadline should be received serverside"
|
||||
), "The provided deadline should be recieved serverside"
|
||||
assert (
|
||||
stream.metadata["authorization"] == metadata["authorization"]
|
||||
), "The provided authorization metadata should be received serverside"
|
||||
), "The provided authorization metadata should be recieved serverside"
|
||||
|
||||
return server_side_test
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def handler_trailer_only_unauthenticated():
|
||||
async def handler(stream: grpclib.server.Stream):
|
||||
await stream.recv_message()
|
||||
await stream.send_initial_metadata()
|
||||
await stream.send_trailing_metadata(status=grpclib.Status.UNAUTHENTICATED)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_simple_service_call():
|
||||
async with ChannelFor([ThingService()]) as channel:
|
||||
await _test_client(ThingServiceClient(channel))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trailer_only_error_unary_unary(
|
||||
mocker, handler_trailer_only_unauthenticated
|
||||
):
|
||||
service = ThingService()
|
||||
mocker.patch.object(
|
||||
service,
|
||||
"do_thing",
|
||||
side_effect=handler_trailer_only_unauthenticated,
|
||||
autospec=True,
|
||||
)
|
||||
async with ChannelFor([service]) as channel:
|
||||
with pytest.raises(grpclib.exceptions.GRPCError) as e:
|
||||
await ThingServiceClient(channel).do_thing(name="something")
|
||||
assert e.value.status == grpclib.Status.UNAUTHENTICATED
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trailer_only_error_stream_unary(
|
||||
mocker, handler_trailer_only_unauthenticated
|
||||
):
|
||||
service = ThingService()
|
||||
mocker.patch.object(
|
||||
service,
|
||||
"do_many_things",
|
||||
side_effect=handler_trailer_only_unauthenticated,
|
||||
autospec=True,
|
||||
)
|
||||
async with ChannelFor([service]) as channel:
|
||||
with pytest.raises(grpclib.exceptions.GRPCError) as e:
|
||||
await ThingServiceClient(channel).do_many_things(
|
||||
request_iterator=[DoThingRequest(name="something")]
|
||||
)
|
||||
await _test_client(ThingServiceClient(channel))
|
||||
assert e.value.status == grpclib.Status.UNAUTHENTICATED
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skipif(
|
||||
sys.version_info < (3, 8), reason="async mock spy does works for python3.8+"
|
||||
)
|
||||
async def test_service_call_mutable_defaults(mocker):
|
||||
async with ChannelFor([ThingService()]) as channel:
|
||||
client = ThingServiceClient(channel)
|
||||
spy = mocker.spy(client, "_unary_unary")
|
||||
await _test_client(client)
|
||||
comments = spy.call_args_list[-1].args[1].comments
|
||||
await _test_client(client)
|
||||
assert spy.call_args_list[-1].args[1].comments is not comments
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_service_call_with_upfront_request_params():
|
||||
# Setting deadline
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(22)
|
||||
metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ThingService(test_hook=_assert_request_meta_received(deadline, metadata))]
|
||||
[ThingService(test_hook=_assert_request_meta_recieved(deadline, metadata),)]
|
||||
) as channel:
|
||||
await _test_client(
|
||||
ThingServiceClient(channel, deadline=deadline, metadata=metadata)
|
||||
@@ -117,7 +53,7 @@ async def test_service_call_with_upfront_request_params():
|
||||
deadline = grpclib.metadata.Deadline.from_timeout(timeout)
|
||||
metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ThingService(test_hook=_assert_request_meta_received(deadline, metadata))]
|
||||
[ThingService(test_hook=_assert_request_meta_recieved(deadline, metadata),)]
|
||||
) as channel:
|
||||
await _test_client(
|
||||
ThingServiceClient(channel, timeout=timeout, metadata=metadata)
|
||||
@@ -134,7 +70,7 @@ async def test_service_call_lower_level_with_overrides():
|
||||
kwarg_deadline = grpclib.metadata.Deadline.from_timeout(28)
|
||||
kwarg_metadata = {"authorization": "12345"}
|
||||
async with ChannelFor(
|
||||
[ThingService(test_hook=_assert_request_meta_received(deadline, metadata))]
|
||||
[ThingService(test_hook=_assert_request_meta_recieved(deadline, metadata),)]
|
||||
) as channel:
|
||||
client = ThingServiceClient(channel, deadline=deadline, metadata=metadata)
|
||||
response = await client._unary_unary(
|
||||
@@ -156,7 +92,7 @@ async def test_service_call_lower_level_with_overrides():
|
||||
async with ChannelFor(
|
||||
[
|
||||
ThingService(
|
||||
test_hook=_assert_request_meta_received(kwarg_deadline, kwarg_metadata),
|
||||
test_hook=_assert_request_meta_recieved(kwarg_deadline, kwarg_metadata),
|
||||
)
|
||||
]
|
||||
) as channel:
|
||||
@@ -204,8 +140,8 @@ async def test_async_gen_for_stream_stream_request():
|
||||
assert response.version == response_index + 1
|
||||
response_index += 1
|
||||
if more_things:
|
||||
# Send some more requests as we receive responses to be sure coordination of
|
||||
# send/receive events doesn't matter
|
||||
# Send some more requests as we recieve reponses to be sure coordination of
|
||||
# send/recieve events doesn't matter
|
||||
await request_chan.send(GetThingRequest(more_things.pop(0)))
|
||||
elif not send_initial_requests.done():
|
||||
# Make sure the sending task it completed
|
||||
@@ -215,4 +151,4 @@ async def test_async_gen_for_stream_stream_request():
|
||||
request_chan.close()
|
||||
assert response_index == len(
|
||||
expected_things
|
||||
), "Didn't receive all expected responses"
|
||||
), "Didn't recieve all exptected responses"
|
||||
@@ -27,7 +27,10 @@ class ClientStub:
|
||||
|
||||
|
||||
async def to_list(generator: AsyncIterator):
|
||||
return [value async for value in generator]
|
||||
result = []
|
||||
async for value in generator:
|
||||
result.append(value)
|
||||
return result
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -1,12 +1,12 @@
|
||||
from tests.output_betterproto.service.service import (
|
||||
from betterproto.tests.output_betterproto.service.service import (
|
||||
DoThingResponse,
|
||||
DoThingRequest,
|
||||
GetThingRequest,
|
||||
GetThingResponse,
|
||||
TestStub as ThingServiceClient,
|
||||
)
|
||||
import grpclib
|
||||
import grpclib.server
|
||||
from typing import Dict
|
||||
from typing import Any, Dict
|
||||
|
||||
|
||||
class ThingService:
|
||||
@@ -1,4 +1,4 @@
|
||||
from tests.output_betterproto.bool import Test
|
||||
from betterproto.tests.output_betterproto.bool import Test
|
||||
|
||||
|
||||
def test_value():
|
||||
@@ -1,5 +1,5 @@
|
||||
import tests.output_betterproto.casing as casing
|
||||
from tests.output_betterproto.casing import Test
|
||||
import betterproto.tests.output_betterproto.casing as casing
|
||||
from betterproto.tests.output_betterproto.casing import Test
|
||||
|
||||
|
||||
def test_message_attributes():
|
||||
@@ -1,4 +1,4 @@
|
||||
from tests.output_betterproto.casing_message_field_uppercase import Test
|
||||
from betterproto.tests.output_betterproto.casing_message_field_uppercase import Test
|
||||
|
||||
|
||||
def test_message_casing():
|
||||
@@ -1,9 +1,13 @@
|
||||
# Test cases that are expected to fail, e.g. unimplemented features or bug-fixes.
|
||||
# Remove from list when fixed.
|
||||
xfail = {
|
||||
"import_circular_dependency",
|
||||
"oneof_enum", # 63
|
||||
"namespace_keywords", # 70
|
||||
"namespace_builtin_types", # 53
|
||||
"googletypes_struct", # 9
|
||||
"googletypes_value", # 9
|
||||
"enum_skipped_value", # 93
|
||||
"import_capitalized_package",
|
||||
"example", # This is the example in the readme. Not a test.
|
||||
}
|
||||
@@ -12,17 +16,7 @@ services = {
|
||||
"googletypes_response",
|
||||
"googletypes_response_embedded",
|
||||
"service",
|
||||
"service_separate_packages",
|
||||
"import_service_input_message",
|
||||
"googletypes_service_returns_empty",
|
||||
"googletypes_service_returns_googletype",
|
||||
"example_service",
|
||||
"empty_service",
|
||||
}
|
||||
|
||||
|
||||
# Indicate json sample messages to skip when testing that json (de)serialization
|
||||
# is symmetrical becuase some cases legitimately are not symmetrical.
|
||||
# Each key references the name of the test scenario and the values in the tuple
|
||||
# Are the names of the json files.
|
||||
non_symmetrical_json = {"empty_repeated": ("empty_repeated",)}
|
||||
@@ -0,0 +1,12 @@
|
||||
syntax = "proto3";
|
||||
|
||||
message Test {
|
||||
enum MyEnum {
|
||||
ZERO = 0;
|
||||
ONE = 1;
|
||||
// TWO = 2;
|
||||
THREE = 3;
|
||||
FOUR = 4;
|
||||
}
|
||||
MyEnum x = 1;
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
from betterproto.tests.output_betterproto.enum_skipped_value import (
|
||||
Test,
|
||||
TestMyEnum,
|
||||
)
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.xfail(reason="#93")
|
||||
def test_message_attributes():
|
||||
assert (
|
||||
Test(x=TestMyEnum.ONE).to_dict()["x"] == "ONE"
|
||||
), "MyEnum.ONE is not serialized to 'ONE'"
|
||||
assert (
|
||||
Test(x=TestMyEnum.THREE).to_dict()["x"] == "THREE"
|
||||
), "MyEnum.THREE is not serialized to 'THREE'"
|
||||
assert (
|
||||
Test(x=TestMyEnum.FOUR).to_dict()["x"] == "FOUR"
|
||||
), "MyEnum.FOUR is not serialized to 'FOUR'"
|
||||
3
betterproto/tests/inputs/enums/enums.json
Normal file
3
betterproto/tests/inputs/enums/enums.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"greeting": "HEY"
|
||||
}
|
||||
14
betterproto/tests/inputs/enums/enums.proto
Normal file
14
betterproto/tests/inputs/enums/enums.proto
Normal file
@@ -0,0 +1,14 @@
|
||||
syntax = "proto3";
|
||||
|
||||
// Enum for the different greeting types
|
||||
enum Greeting {
|
||||
HI = 0;
|
||||
HEY = 1;
|
||||
// Formal greeting
|
||||
HELLO = 2;
|
||||
}
|
||||
|
||||
message Test {
|
||||
// Greeting enum example
|
||||
Greeting greeting = 1;
|
||||
}
|
||||
8
betterproto/tests/inputs/example/example.proto
Normal file
8
betterproto/tests/inputs/example/example.proto
Normal file
@@ -0,0 +1,8 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package hello;
|
||||
|
||||
// Greeting represents a message you can tell a user.
|
||||
message Greeting {
|
||||
string message = 1;
|
||||
}
|
||||
@@ -3,8 +3,8 @@ from typing import Any, Callable, Optional
|
||||
import betterproto.lib.google.protobuf as protobuf
|
||||
import pytest
|
||||
|
||||
from tests.mocks import MockChannel
|
||||
from tests.output_betterproto.googletypes_response import TestStub
|
||||
from betterproto.tests.mocks import MockChannel
|
||||
from betterproto.tests.output_betterproto.googletypes_response import TestStub
|
||||
|
||||
test_cases = [
|
||||
(TestStub.get_double, protobuf.DoubleValue, 2.5),
|
||||
@@ -21,7 +21,7 @@ test_cases = [
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(["service_method", "wrapper_class", "value"], test_cases)
|
||||
async def test_channel_receives_wrapped_type(
|
||||
async def test_channel_recieves_wrapped_type(
|
||||
service_method: Callable[[TestStub], Any], wrapper_class: Callable, value
|
||||
):
|
||||
wrapped_value = wrapper_class()
|
||||
@@ -1,7 +1,7 @@
|
||||
import pytest
|
||||
|
||||
from tests.mocks import MockChannel
|
||||
from tests.output_betterproto.googletypes_response_embedded import (
|
||||
from betterproto.tests.mocks import MockChannel
|
||||
from betterproto.tests.output_betterproto.googletypes_response_embedded import (
|
||||
Output,
|
||||
TestStub,
|
||||
)
|
||||
@@ -0,0 +1,15 @@
|
||||
syntax = "proto3";
|
||||
|
||||
import "request_message.proto";
|
||||
|
||||
// Tests generated service correctly imports the RequestMessage
|
||||
|
||||
service Test {
|
||||
rpc DoThing (RequestMessage) returns (RequestResponse);
|
||||
}
|
||||
|
||||
|
||||
message RequestResponse {
|
||||
int32 value = 1;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
import pytest
|
||||
|
||||
from betterproto.tests.mocks import MockChannel
|
||||
from betterproto.tests.output_betterproto.import_service_input_message import (
|
||||
RequestResponse,
|
||||
TestStub,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.xfail(reason="#68 Request Input Messages are not imported for service")
|
||||
@pytest.mark.asyncio
|
||||
async def test_service_correctly_imports_reference_message():
|
||||
mock_response = RequestResponse(value=10)
|
||||
service = TestStub(MockChannel([mock_response]))
|
||||
response = await service.do_thing()
|
||||
assert mock_response == response
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user