Update readme, add docs for standard tests

README.md

@@ -311,10 +311,26 @@ $ pip install -e .
 
 There are two types of tests:
 
-1. Manually-written tests for some behavior of the library
+1. Standard tests
-2. Proto files and JSON inputs for automated tests
+2. Custom tests
 
-For #2, you can add a new `*.proto` file into the `betterproto/tests` directory along with a sample `*.json` input and it will get automatically picked up.
+#### Standard tests
+
+Adding a standard test case is easy.
+
+- Create a new directory `betterproto/tests/inputs/<name>`
+  - add `<name>.proto`  with a message called `Test`
+  - add `<name>.json` with some test data
+
+It will be picked up automatically when you run `pipenv test`
+
+- See also: [Standard Tests Development Guide](betterproto\tests\standard-tests.md) 
+
+#### Custom tests
+
+Custom tests are found in `tests/test_*.py` and are run with pytest.
+
+#### Running
 
 Here's how to run the tests.
 
@@ -322,7 +338,7 @@ Here's how to run the tests.
 # Generate assets from sample .proto files
 $ pipenv run generate
 
-# Run the tests
+# Run all tests
 $ pipenv run test
 ```
 

betterproto/tests/inputs/bool/test.py

@@ -1,8 +0,0 @@
-from betterproto.tests.output_betterproto.bool.bool import Test
-from betterproto.tests.util import read_relative
-
-
-def test_value():
-    message = Test().from_json(read_relative(__file__, 'bool.json'))
-    assert message.value
-

betterproto/tests/inputs/bool/test_bool.py

@@ -0,0 +1,6 @@
+from betterproto.tests.output_betterproto.bool.bool import Test
+
+
+def test_value():
+    message = Test()
+    assert not message.value, "Boolean is False by default"

betterproto/tests/standard-tests.md

@@ -0,0 +1,75 @@
+# Standard Tests Development Guide
+
+Standard test cases are found in [betterproto/tests/inputs](inputs), where each subdirectory represents a testcase, that is verified in isolation.
+
+```
+inputs/
+   bool/
+   double/
+   int32/
+   ...
+```
+
+## Test case directory structure
+
+Each testcase has a `<name>.proto` file with a message called `Test`, a matching `.json` file and optionally a custom test file called `test_*.py`.
+
+```bash
+bool/
+  bool.proto
+  bool.json
+  test_bool.py  # optional
+```
+
+### proto
+
+`<name>.proto` &mdash; *The protobuf message to test*
+
+```protobuf
+syntax = "proto3";
+
+message Test {
+    bool value = 1;
+}
+```
+
+You can add multiple `.proto` files to the test case, as long as one file matches the directory name. 
+
+### json
+
+`<name>.json` &mdash; *Test-data to validate the message with*
+
+```json
+{
+  "value": true
+}
+```
+
+### pytest
+
+`test_<name>.py` &mdash; *Custom test to validate specific aspects of the generated class*
+
+```python
+from betterproto.tests.output_betterproto.bool.bool import Test
+
+def test_value():
+    message = Test()
+    assert not message.value, "Boolean is False by default"
+```
+
+## Standard tests
+
+The following tests are automatically executed for all cases:
+
+- [x] Can the generated python code imported?
+- [x] Can the generated message class be instantiated?
+- [x] Is the generated code compatible with the Google's `grpc_tools.protoc` implementation?
+
+## Running the tests
+
+- `pipenv run generate`
+  This generates
+  - `betterproto/tests/output_betterproto` &mdash; *the plugin generated python classes*
+  - `betterproto/tests/output_reference` &mdash; *reference implementation classes*
+- `pipenv run test`
+

pytest.ini

@@ -1,5 +1,5 @@
 [pytest]
-python_files = test*.py
+python_files = test_*.py
 python_classes =
 norecursedirs = **/output_*
 addopts = -p no:warnings