Merge branch 'master' of github.com:MongoEngine/mongoengine into add_validation_to_doc

2020-10-17 15:05:27 +02:00
parent d37a30e083 0bbc05995a
commit c218c8bb6c
119 changed files with 14293 additions and 12132 deletions
--- a/docs/changelog.rst
+++ b/docs/changelog.rst
@@ -6,208 +6,264 @@ Changelog
 Development
 ===========
 - (Fill this out as you fix issues and develop your features).
+- When using pymongo >= 3.7, make use of Collection.count_documents instead of Collection.count
+    and Cursor.count that got deprecated in pymongo >= 3.7.
+    This should have a negative impact on performance of count see Issue #2219
+- Fix a bug that made the queryset drop the read_preference after clone().
+- Fix the behavior of Doc.objects.limit(0) which should return all documents (similar to mongodb) #2311
+- Bug fix in ListField when updating the first item, it was saving the whole list, instead of
+    just replacing the first item (as it's usually done) #2392
+
+Changes in 0.20.0
+=================
+- ATTENTION: Drop support for Python2
+- Add Mongo 4.0 to Travis
+- Fix error when setting a string as a ComplexDateTimeField #2253
+- Bump development Status classifier to Production/Stable #2232
+- Improve Queryset.get to avoid confusing MultipleObjectsReturned message in case multiple match are found #630
+- Fixed a bug causing inaccurate query results, while combining ``__raw__`` and regular filters for the same field #2264
+- Add support for the `elemMatch` projection operator in .fields() (e.g BlogPost.objects.fields(elemMatch__comments="test")) #2267
+- DictField validate failed without default connection (bug introduced in 0.19.0) #2239
+- Remove methods that were deprecated years ago:
+    - name parameter in Field constructor e.g `StringField(name="...")`, was replaced by db_field
+    - Queryset.slave_okay() was deprecated since pymongo3
+    - dropDups was dropped with MongoDB3
+    - ``Queryset._ensure_indexes`` and ``Queryset.ensure_indexes``, the right method to use is ``Document.ensure_indexes``
+- Added pre-commit for development/CI #2212
+- Renamed requirements-lint.txt to requirements-dev.txt #2212
+- Support for setting ReadConcern #2255
+
+Changes in 0.19.1
+=================
+- Tests require Pillow < 7.0.0 as it dropped Python2 support
+- DEPRECATION: The interface of ``QuerySet.aggregate`` method was changed, it no longer takes an unpacked list of
+    pipeline steps (*pipeline) but simply takes the pipeline list just like ``pymongo.Collection.aggregate`` does. #2079
+
+Changes in 0.19.0
+=================
+- BREAKING CHANGE: ``class_check`` and ``read_preference`` keyword arguments are no longer available when filtering a ``QuerySet``. #2112
+    - Instead of ``Doc.objects(foo=bar, read_preference=...)`` use ``Doc.objects(foo=bar).read_preference(...)``.
+    - Instead of ``Doc.objects(foo=bar, class_check=False)`` use ``Doc.objects(foo=bar).clear_cls_query(...)``.
+    - This change also renames the private ``QuerySet._initial_query`` attribute to ``_cls_query``.
+- BREAKING CHANGE: Removed the deprecated ``format`` param from ``QuerySet.explain``. #2113
+- BREAKING CHANGE: Renamed ``MongoEngineConnectionError`` to ``ConnectionFailure``. #2111
+    - If you catch/use ``MongoEngineConnectionError`` in your code, you'll have to rename it.
+- BREAKING CHANGE: Positional arguments when instantiating a document are no longer supported. #2103
+    - From now on keyword arguments (e.g. ``Doc(field_name=value)``) are required.
+- BREAKING CHANGE: A ``LazyReferenceField`` is now stored in the ``_data`` field of its parent as a ``DBRef``, ``Document``, or ``EmbeddedDocument`` (``ObjectId`` is no longer allowed). #2182
+- DEPRECATION: ``Q.empty`` & ``QNode.empty`` are marked as deprecated and will be removed in a next version of MongoEngine. #2210
+    - Added ability to check if Q or QNode are empty by parsing them to bool.
+    - Instead of ``Q(name="John").empty`` use ``not Q(name="John")``.
+- Fix updating/modifying/deleting/reloading a document that's sharded by a field with ``db_field`` specified. #2125
+- Only set no_cursor_timeout when requested (fixes an incompatibility with MongoDB 4.2) #2148
+- ``ListField`` now accepts an optional ``max_length`` parameter. #2110
+- Improve error message related to InvalidDocumentError #2180
+- Added BulkWriteError to replace NotUniqueError which was misleading in bulk write insert #2152
+- Added ability to compare Q and Q operations #2204
+- Added ability to use a db alias on query_counter #2194
+- Added ability to specify collations for querysets with ``Doc.objects.collation`` #2024
+- Fix updates of a list field by negative index #2094
+- Switch from nosetest to pytest as test runner #2114
+- The codebase is now formatted using ``black``. #2109
+- Documentation improvements:
+    - Documented how `pymongo.monitoring` can be used to log all queries issued by MongoEngine to the driver.
+
+Changes in 0.18.2
+=================
+- Replace deprecated PyMongo v2.x methods with their v3.x equivalents in the ``SequenceField``. #2097
+- Various code clarity and documentation improvements.

 Changes in 0.18.1
 =================
- Fix a bug introduced in 0.18.0 which was causing `.save()` to update all the fields
-    instead of updating only the modified fields. This bug only occurs when using custom pk #2082
- Add Python 3.7 in travis #2058
+- Fix a bug introduced in 0.18.0 which was causing ``Document.save`` to update all the fields instead of updating only the modified fields. This bug only occurred when using a custom PK. #2082
+- Add Python 3.7 to Travis CI. #2058

 Changes in 0.18.0
 =================
 - Drop support for EOL'd MongoDB v2.6, v3.0, and v3.2.
- MongoEngine now requires PyMongo >= v3.4. Travis CI now tests against MongoDB v3.4 – v3.6 and PyMongo v3.4 – v3.6 (#2017 #2066).
- Improve performance by avoiding a call to `to_mongo` in `Document.save()` #2049
+- MongoEngine now requires PyMongo >= v3.4. Travis CI now tests against MongoDB v3.4 – v3.6 and PyMongo v3.4 – v3.6. #2017 #2066
+- Improve performance by avoiding a call to ``to_mongo`` in ``Document.save``. #2049
 - Connection/disconnection improvements:
-    - Expose `mongoengine.connection.disconnect` and `mongoengine.connection.disconnect_all`
-    - Fix disconnecting #566 #1599 #605 #607 #1213 #565
-    - Improve documentation of `connect`/`disconnect`
-    - Fix issue when using multiple connections to the same mongo with different credentials #2047
-    - `connect` fails immediately when db name contains invalid characters #2031 #1718
- Fix the default write concern of `Document.save` that was overwriting the connection write concern #568
- Fix querying on `List(EmbeddedDocument)` subclasses fields #1961 #1492
- Fix querying on `(Generic)EmbeddedDocument` subclasses fields #475
- Fix `QuerySet.aggregate` so that it takes limit and skip value into account #2029
- Generate unique indices for `SortedListField` and `EmbeddedDocumentListFields` #2020
- BREAKING CHANGE: Changed the behavior of a custom field validator (i.e `validation` parameter of a `Field`). It is now expected to raise a `ValidationError` instead of returning True/False #2050
- BREAKING CHANGES (associated with connect/disconnect fixes):
-    - Calling `connect` 2 times with the same alias and different parameter will raise an error (should call `disconnect` first).
-    - `disconnect` now clears `mongoengine.connection._connection_settings`.
-    - `disconnect` now clears the cached attribute `Document._collection`.
- BREAKING CHANGE: `EmbeddedDocument.save` & `.reload` is no longier exist #1552
+    - Expose ``mongoengine.connection.disconnect`` and ``mongoengine.connection.disconnect_all``.
+    - Fix disconnecting. #566 #1599 #605 #607 #1213 #565
+    - Improve documentation of ``connect``/``disconnect``.
+    - Fix issue when using multiple connections to the same mongo with different credentials. #2047
+    - ``connect`` fails immediately when db name contains invalid characters. #2031 #1718
+- Fix the default write concern of ``Document.save`` that was overwriting the connection write concern. #568
+- Fix querying on ``List(EmbeddedDocument)`` subclasses fields. #1961 #1492
+- Fix querying on ``(Generic)EmbeddedDocument`` subclasses fields. #475
+- Fix ``QuerySet.aggregate`` so that it takes limit and skip value into account. #2029
+- Generate unique indices for ``SortedListField`` and ``EmbeddedDocumentListFields``. #2020
+- BREAKING CHANGE: Changed the behavior of a custom field validator (i.e ``validation`` parameter of a ``Field``). It is now expected to raise a ``ValidationError`` instead of returning ``True``/``False``. #2050
+- BREAKING CHANGES (associated with connection/disconnection fixes):
+    - Calling ``connect`` 2 times with the same alias and different parameter will raise an error (should call ``disconnect`` first).
+    - ``disconnect`` now clears ``mongoengine.connection._connection_settings``.
+    - ``disconnect`` now clears the cached attribute ``Document._collection``.
+- BREAKING CHANGE: ``EmbeddedDocument.save`` & ``.reload`` no longer exist. #1552

 Changes in 0.17.0
 =================
- Fix .only() working improperly after using .count() of the same instance of QuerySet
- Fix batch_size that was not copied when cloning a queryset object #2011
- POTENTIAL BREAKING CHANGE: All result fields are now passed, including internal fields (_cls, _id) when using `QuerySet.as_pymongo` #1976
- Document a BREAKING CHANGE introduced in 0.15.3 and not reported at that time (#1995)
- Fix InvalidStringData error when using modify on a BinaryField #1127
- DEPRECATION: `EmbeddedDocument.save` & `.reload` are marked as deprecated and will be removed in a next version of mongoengine #1552
- Fix test suite and CI to support MongoDB 3.4 #1445
- Fix reference fields querying the database on each access if value contains orphan DBRefs
+- POTENTIAL BREAKING CHANGE: All result fields are now passed, including internal fields (``_cls``, ``_id``) when using ``QuerySet.as_pymongo``. #1976
+- Document a BREAKING CHANGE introduced in 0.15.3 and not reported at that time. #1995
+- DEPRECATION: ``EmbeddedDocument.save`` & ``.reload`` are marked as deprecated and will be removed in a next version of MongoEngine. #1552
+- Fix ``QuerySet.only`` working improperly after using ``QuerySet.count`` of the same instance of a ``QuerySet``.
+- Fix ``batch_size`` that was not copied when cloning a ``QuerySet`` object. #2011
+- Fix ``InvalidStringData`` error when using ``modify`` on a ``BinaryField``. #1127
+- Fix test suite and CI to support MongoDB v3.4. #1445
+- Fix reference fields querying the database on each access if value contains orphan DBRefs.

-=================
 Changes in 0.16.3
 =================
- Fix $push with $position operator not working with lists in embedded document #1965
+- Fix ``$push`` with the ``$position`` operator not working with lists in embedded documents. #1965

-=================
 Changes in 0.16.2
 =================
- Fix .save() that fails when called with write_concern=None (regression of 0.16.1) #1958
+- Fix ``Document.save`` that fails when called with ``write_concern=None`` (regression of 0.16.1). #1958

-=================
 Changes in 0.16.1
 =================
- Fix `_cls` that is not set properly in Document constructor (regression) #1950
- Fix bug in _delta method - Update of a ListField depends on an unrelated dynamic field update #1733
- Remove deprecated `save()` method and used `insert_one()` #1899
+- Fix ``_cls`` that is not set properly in the ``Document`` constructor (regression). #1950
+- Fix a bug in the ``_delta`` method - update of a ``ListField`` depends on an unrelated dynamic field update. #1733
+- Remove PyMongo's deprecated ``Collection.save`` method and use ``Collection.insert_one`` instead. #1899

-=================
 Changes in 0.16.0
 =================
- Various improvements to the doc
- Improvement to code quality
 - POTENTIAL BREAKING CHANGES:
-    - EmbeddedDocumentField will no longer accept references to Document classes in its constructor #1661
-    - Get rid of the `basecls` parameter from the DictField constructor (dead code) #1876
-    - default value of ComplexDateTime is now None (and no longer the current datetime) #1368
- Fix unhashable TypeError when referencing a Document with a compound key in an EmbeddedDocument #1685
- Fix bug where an EmbeddedDocument with the same id as its parent would not be tracked for changes #1768
- Fix the fact that bulk `insert()` was not setting primary keys of inserted documents instances #1919
- Fix bug when referencing the abstract class in a ReferenceField #1920
- Allow modification to the document made in pre_save_post_validation to be taken into account #1202
- Replaced MongoDB 2.4 tests in CI by MongoDB 3.2 #1903
- Fix side effects of using queryset.`no_dereference` on other documents #1677
- Fix TypeError when using lazy django translation objects as translated choices #1879
- Improve 2-3 codebase compatibility #1889
- Fix the support for changing the default value of ComplexDateTime #1368
- Improves error message in case an EmbeddedDocumentListField receives an EmbeddedDocument instance
-    instead of a list #1877
- Fix the Decimal operator inc/dec #1517 #1320
- Ignore killcursors queries in `query_counter` context manager #1869
- Fix the fact that `query_counter` was modifying the initial profiling_level in case it was != 0 #1870
- Repaired the `no_sub_classes` context manager + fix the fact that it was swallowing exceptions #1865
- Fix index creation error that was swallowed by hasattr under python2 #1688
- QuerySet limit function behaviour: Passing 0 as parameter will return all the documents in the cursor #1611
- bulk insert updates the ids of the input documents instances #1919
- Fix an harmless bug related to GenericReferenceField where modifications in the generic-referenced document
-    were tracked in the parent #1934
- Improve validator of BinaryField #273
- Implemented lazy regex compiling in Field classes to improve 'import mongoengine' performance #1806
- Updated GridFSProxy.__str__  so that it would always print both the filename and grid_id #710
- Add __repr__ to Q and QCombination #1843
- fix bug in BaseList.__iter__ operator (was occuring when modifying a BaseList while iterating over it) #1676
- Added field `DateField`#513
+    - ``EmbeddedDocumentField`` will no longer accept references to Document classes in its constructor. #1661
+    - Get rid of the ``basecls`` parameter from the ``DictField`` constructor (dead code). #1876
+    - Default value of the ``ComplexDateTime`` field is now ``None`` (and no longer the current datetime). #1368
+- Fix an unhashable ``TypeError`` when referencing a ``Document`` with a compound key in an ``EmbeddedDocument``. #1685
+- Fix a bug where an ``EmbeddedDocument`` with the same id as its parent would not be tracked for changes. #1768
+- Fix the fact that a bulk ``QuerySet.insert`` was not setting primary keys of inserted document instances. #1919
+- Fix a bug when referencing an abstract class in a ``ReferenceField``. #1920
+- Allow modifications to the document made in ``pre_save_post_validation`` to be taken into account. #1202
+- Replace MongoDB v2.4 tests in Travis CI with MongoDB v3.2. #1903
+- Fix side effects of using ``QuerySet.no_dereference`` on other documents. #1677
+- Fix ``TypeError`` when using lazy Django translation objects as translated choices. #1879
+- Improve Python 2-3 codebase compatibility. #1889
+- Fix support for changing the default value of the ``ComplexDateTime`` field. #1368
+- Improve error message in case an ``EmbeddedDocumentListField`` receives an ``EmbeddedDocument`` instance instead of a list. #1877
+- Fix the ``inc`` and ``dec`` operators for the ``DecimalField``. #1517 #1320
+- Ignore ``killcursors`` queries in ``query_counter`` context manager. #1869
+- Fix the fact that ``query_counter`` was modifying the initial profiling level in case it was != 0. #1870
+- Repair the ``no_sub_classes`` context manager + fix the fact that it was swallowing exceptions. #1865
+- Fix index creation error that was swallowed by ``hasattr`` under Python 2. #1688
+- ``QuerySet.limit`` function behaviour: Passing 0 as parameter will return all the documents in the cursor. #1611
+- Bulk insert updates the IDs of the input documents instances. #1919
+- Fix a harmless bug related to ``GenericReferenceField`` where modifications in the generic-referenced document were tracked in the parent. #1934
+- Improve validation of the ``BinaryField``. #273
+- Implement lazy regex compiling in Field classes to improve ``import mongoengine`` performance. #1806
+- Update ``GridFSProxy.__str__``  so that it would always print both the filename and grid_id. #710
+- Add ``__repr__`` to ``Q`` and ``QCombination`` classes. #1843
+- Fix bug in the ``BaseList.__iter__`` operator (was occuring when modifying a BaseList while iterating over it). #1676
+- Add a ``DateField``. #513
+- Various improvements to the documentation.
+- Various code quality improvements.

 Changes in 0.15.3
 =================
-  BREAKING CHANGES: `Queryset.update/update_one` methods now returns an UpdateResult when `full_result=True` is provided and no longer a dict (relates to #1491)
-  Subfield resolve error in generic_emdedded_document query #1651 #1652
-  use each modifier only with $position #1673 #1675
-  Improve LazyReferenceField and GenericLazyReferenceField with nested fields #1704
-  Fix validation error instance in GenericEmbeddedDocumentField #1067
-  Update cached fields when fields argument is given #1712
-  Add a db parameter to register_connection for compatibility with connect
-  Use insert_one, insert_many in Document.insert #1491
-  Use new update_one, update_many on document/queryset update #1491
-  Use insert_one, insert_many in Document.insert #1491
-  Fix reload(fields) affect changed fields #1371
-  Fix Read-only access to database fails when trying to create indexes #1338
+- ``Queryset.update/update_one`` methods now return an ``UpdateResult`` when ``full_result=True`` is provided and no longer a dict. #1491
+- Improve ``LazyReferenceField`` and ``GenericLazyReferenceField`` with nested fields. #1704
+- Fix the subfield resolve error in ``generic_emdedded_document`` query. #1651 #1652
+- Use each modifier only with ``$position``. #1673 #1675
+- Fix validation errors in the ``GenericEmbeddedDocumentField``. #1067
+- Update cached fields when a ``fields`` argument is given. #1712
+- Add a ``db`` parameter to ``register_connection`` for compatibility with ``connect``.
+- Use PyMongo v3.x's ``insert_one`` and ``insert_many`` in ``Document.insert``. #1491
+- Use PyMongo v3.x's ``update_one`` and ``update_many`` in ``Document.update`` and ``QuerySet.update``. #1491
+- Fix how ``reload(fields)`` affects changed fields. #1371
+- Fix a bug where the read-only access to the database fails when trying to create indexes. #1338

 Changes in 0.15.0
 =================
- Add LazyReferenceField and GenericLazyReferenceField to address #1230
+- Add ``LazyReferenceField`` and ``GenericLazyReferenceField``. #1230

 Changes in 0.14.1
 =================
- Removed SemiStrictDict and started using a regular dict for `BaseDocument._data` #1630
- Added support for the `$position` param in the `$push` operator #1566
- Fixed `DateTimeField` interpreting an empty string as today #1533
- Added a missing `__ne__` method to the `GridFSProxy` class #1632
- Fixed `BaseQuerySet._fields_to_db_fields` #1553
+- Remove ``SemiStrictDict`` and start using a regular dict for ``BaseDocument._data``. #1630
+- Add support for the ``$position`` param in the ``$push`` operator. #1566
+- Fix ``DateTimeField`` interpreting an empty string as today. #1533
+- Add a missing ``__ne__`` method to the ``GridFSProxy`` class. #1632
+- Fix ``BaseQuerySet._fields_to_db_fields``. #1553

 Changes in 0.14.0
 =================
- BREAKING CHANGE: Removed the `coerce_types` param from `QuerySet.as_pymongo` #1549
- POTENTIAL BREAKING CHANGE: Made EmbeddedDocument not hashable by default #1528
- Improved code quality #1531, #1540, #1541, #1547
+- BREAKING CHANGE: Remove the ``coerce_types`` param from ``QuerySet.as_pymongo``. #1549
+- POTENTIAL BREAKING CHANGE: Make ``EmbeddedDocument`` not hashable by default. #1528
+- Improve code quality. #1531, #1540, #1541, #1547

 Changes in 0.13.0
 =================
- POTENTIAL BREAKING CHANGE: Added Unicode support to the `EmailField`, see
-  docs/upgrade.rst for details.
+- POTENTIAL BREAKING CHANGE: Added Unicode support to the ``EmailField``, see docs/upgrade.rst for details.

 Changes in 0.12.0
 =================
- POTENTIAL BREAKING CHANGE: Fixed limit/skip/hint/batch_size chaining #1476
- POTENTIAL BREAKING CHANGE: Changed a public `QuerySet.clone_into` method to a private `QuerySet._clone_into` #1476
- Fixed the way `Document.objects.create` works with duplicate IDs #1485
- Fixed connecting to a replica set with PyMongo 2.x #1436
- Fixed using sets in field choices #1481
- Fixed deleting items from a `ListField` #1318
- Fixed an obscure error message when filtering by `field__in=non_iterable`. #1237
- Fixed behavior of a `dec` update operator #1450
- Added a `rename` update operator #1454
- Added validation for the `db_field` parameter #1448
- Fixed the error message displayed when querying an `EmbeddedDocumentField` by an invalid value #1440
- Fixed the error message displayed when validating unicode URLs #1486
- Raise an error when trying to save an abstract document #1449
+- POTENTIAL BREAKING CHANGE: Fix ``limit``/``skip``/``hint``/``batch_size`` chaining. #1476
+- POTENTIAL BREAKING CHANGE: Change a public ``QuerySet.clone_into`` method to a private ``QuerySet._clone_into``. #1476
+- Fix the way ``Document.objects.create`` works with duplicate IDs. #1485
+- Fix connecting to a replica set with PyMongo 2.x. #1436
+- Fix using sets in field choices. #1481
+- Fix deleting items from a ``ListField``. #1318
+- Fix an obscure error message when filtering by ``field__in=non_iterable``. #1237
+- Fix behavior of a ``dec`` update operator. #1450
+- Add a ``rename`` update operator. #1454
+- Add validation for the ``db_field`` parameter. #1448
+- Fix the error message displayed when querying an ``EmbeddedDocumentField`` by an invalid value. #1440
+- Fix the error message displayed when validating Unicode URLs. #1486
+- Raise an error when trying to save an abstract document. #1449

 Changes in 0.11.0
 =================
- BREAKING CHANGE: Renamed `ConnectionError` to `MongoEngineConnectionError` since the former is a built-in exception name in Python v3.x. #1428
- BREAKING CHANGE: Dropped Python 2.6 support. #1428
- BREAKING CHANGE: `from mongoengine.base import ErrorClass` won't work anymore for any error from `mongoengine.errors` (e.g. `ValidationError`). Use `from mongoengine.errors import ErrorClass instead`. #1428
- BREAKING CHANGE: Accessing a broken reference will raise a `DoesNotExist` error. In the past it used to return `None`. #1334
- Fixed absent rounding for DecimalField when `force_string` is set. #1103
+- BREAKING CHANGE: Rename ``ConnectionError`` to ``MongoEngineConnectionError`` since the former is a built-in exception name in Python v3.x. #1428
+- BREAKING CHANGE: Drop Python v2.6 support. #1428
+- BREAKING CHANGE: ``from mongoengine.base import ErrorClass`` won't work anymore for any error from ``mongoengine.errors`` (e.g. ``ValidationError``). Use ``from mongoengine.errors import ErrorClass instead``. #1428
+- BREAKING CHANGE: Accessing a broken reference will raise a ``DoesNotExist`` error. In the past it used to return ``None``. #1334
+- Fix absent rounding for the ``DecimalField`` when ``force_string`` is set. #1103

 Changes in 0.10.8
 =================
- Added support for QuerySet.batch_size (#1426)
- Fixed query set iteration within iteration #1427
- Fixed an issue where specifying a MongoDB URI host would override more information than it should #1421
- Added ability to filter the generic reference field by ObjectId and DBRef #1425
- Fixed delete cascade for models with a custom primary key field #1247
- Added ability to specify an authentication mechanism (e.g. X.509) #1333
- Added support for falsey primary keys (e.g. doc.pk = 0) #1354
- Fixed QuerySet#sum/average for fields w/ explicit db_field #1417
- Fixed filtering by embedded_doc=None #1422
- Added support for cursor.comment #1420
- Fixed doc.get_<field>_display #1419
- Fixed __repr__ method of the StrictDict #1424
- Added a deprecation warning for Python 2.6
+- Add support for ``QuerySet.batch_size``. (#1426)
+- Fix a query set iteration within an iteration. #1427
+- Fix an issue where specifying a MongoDB URI host would override more information than it should. #1421
+- Add an ability to filter the ``GenericReferenceField`` by an ``ObjectId`` and a ``DBRef``. #1425
+- Fix cascading deletes for models with a custom primary key field. #1247
+- Add ability to specify an authentication mechanism (e.g. X.509). #1333
+- Add support for falsy primary keys (e.g. ``doc.pk = 0``). #1354
+- Fix ``QuerySet.sum/average`` for fields w/ an explicit ``db_field``. #1417
+- Fix filtering by ``embedded_doc=None``. #1422
+- Add support for ``Cursor.comment``. #1420
+- Fix ``doc.get_<field>_display`` methods. #1419
+- Fix the ``__repr__`` method of the ``StrictDict`` #1424
+- Add a deprecation warning for Python v2.6.

 Changes in 0.10.7
 =================
- Dropped Python 3.2 support #1390
- Fixed the bug where dynamic doc has index inside a dict field #1278
- Fixed: ListField minus index assignment does not work #1128
- Fixed cascade delete mixing among collections #1224
- Add `signal_kwargs` argument to `Document.save`, `Document.delete` and `BaseQuerySet.insert` to be passed to signals calls #1206
- Raise `OperationError` when trying to do a `drop_collection` on document with no collection set.
- count on ListField of EmbeddedDocumentField fails. #1187
- Fixed long fields stored as int32 in Python 3. #1253
- MapField now handles unicodes keys correctly. #1267
- ListField now handles negative indicies correctly. #1270
- Fixed AttributeError when initializing EmbeddedDocument with positional args. #681
- Fixed no_cursor_timeout error with pymongo 3.0+ #1304
- Replaced map-reduce based QuerySet.sum/average with aggregation-based implementations #1336
- Fixed support for `__` to escape field names that match operators names in `update` #1351
- Fixed BaseDocument#_mark_as_changed #1369
- Added support for pickling QuerySet instances. #1397
- Fixed connecting to a list of hosts #1389
- Fixed a bug where accessing broken references wouldn't raise a DoesNotExist error #1334
- Fixed not being able to specify use_db_field=False on ListField(EmbeddedDocumentField) instances #1218
- Improvements to the dictionary fields docs #1383
+- Drop Python 3.2 support #1390
+- Fix a bug where a dynamic doc has an index inside a dict field. #1278
+- Fix: ``ListField`` minus index assignment does not work. #1128
+- Fix cascade delete mixing among collections. #1224
+- Add ``signal_kwargs`` argument to ``Document.save``, ``Document.delete`` and ``BaseQuerySet.insert`` to be passed to signals calls. #1206
+- Raise ``OperationError`` when trying to do a ``drop_collection`` on document with no collection set.
+- Fix a bug where a count on ``ListField`` of ``EmbeddedDocumentField`` fails. #1187
+- Fix ``LongField`` values stored as int32 in Python 3. #1253
+- ``MapField`` now handles unicode keys correctly. #1267
+- ``ListField`` now handles negative indicies correctly. #1270
+- Fix an ``AttributeError`` when initializing an ``EmbeddedDocument`` with positional args. #681
+- Fix a ``no_cursor_timeout`` error with PyMongo v3.x. #1304
+- Replace map-reduce based ``QuerySet.sum/average`` with aggregation-based implementations. #1336
+- Fix support for ``__`` to escape field names that match operators' names in ``update``. #1351
+- Fix ``BaseDocument._mark_as_changed``. #1369
+- Add support for pickling ``QuerySet`` instances. #1397
+- Fix connecting to a list of hosts. #1389
+- Fix a bug where accessing broken references wouldn't raise a ``DoesNotExist`` error. #1334
+- Fix not being able to specify ``use_db_field=False`` on ``ListField(EmbeddedDocumentField)`` instances. #1218
+- Improvements to the dictionary field's docs. #1383

 Changes in 0.10.6
 =================
 - Add support for mocking MongoEngine based on mongomock. #1151
- Fixed not being able to run tests on Windows. #1153
+- Fix not being able to run tests on Windows. #1153
 - Allow creation of sparse compound indexes. #1114
- count on ListField of EmbeddedDocumentField fails. #1187

 Changes in 0.10.5
 =================
@@ -215,12 +271,12 @@ Changes in 0.10.5

 Changes in 0.10.4
 =================
- SaveConditionError is now importable from the top level package. #1165
- upsert_one method added. #1157
+- ``SaveConditionError`` is now importable from the top level package. #1165
+- Add a ``QuerySet.upsert_one`` method. #1157

 Changes in 0.10.3
 =================
- Fix `read_preference` (it had chaining issues with PyMongo 2.x and it didn't work at all with PyMongo 3.x) #1042
+- Fix ``read_preference`` (it had chaining issues with PyMongo v2.x and it didn't work at all with PyMongo v3.x). #1042

 Changes in 0.10.2
 =================
@@ -230,16 +286,16 @@ Changes in 0.10.2

 Changes in 0.10.1
 =================
- Fix infinite recursion with CASCADE delete rules under specific conditions. #1046
- Fix CachedReferenceField bug when loading cached docs as DBRef but failing to save them. #1047
- Fix ignored chained options #842
- Document save's save_condition error raises `SaveConditionError` exception #1070
- Fix Document.reload for DynamicDocument. #1050
- StrictDict & SemiStrictDict are shadowed at init time. #1105
- Fix ListField minus index assignment does not work. #1119
- Remove code that marks field as changed when the field has default but not existed in database #1126
- Remove test dependencies (nose and rednose) from install dependencies list. #1079
- Recursively build query when using elemMatch operator. #1130
+- Fix infinite recursion with cascade delete rules under specific conditions. #1046
+- Fix ``CachedReferenceField`` bug when loading cached docs as ``DBRef`` but failing to save them. #1047
+- Fix ignored chained options. #842
+- ``Document.save``'s ``save_condition`` error raises a ``SaveConditionError`` exception. #1070
+- Fix ``Document.reload`` for the ``DynamicDocument``. #1050
+- ``StrictDict`` & ``SemiStrictDict`` are shadowed at init time. #1105
+- Fix ``ListField`` negative index assignment not working. #1119
+- Remove code that marks a field as changed when the field has a default value but does not exist in the database. #1126
+- Remove test dependencies (nose and rednose) from install dependencies. #1079
+- Recursively build a query when using the ``elemMatch`` operator. #1130
 - Fix instance back references for lists of embedded documents. #1131

 Changes in 0.10.0
@@ -250,7 +306,7 @@ Changes in 0.10.0
 - Removed get_or_create() deprecated since 0.8.0. #300
 - Improve Document._created status when switch collection and db #1020
 - Queryset update doesn't go through field validation #453
- Added support for specifying authentication source as option `authSource` in URI. #967
+- Added support for specifying authentication source as option ``authSource`` in URI. #967
 - Fixed mark_as_changed to handle higher/lower level fields changed. #927
 - ListField of embedded docs doesn't set the _instance attribute when iterating over it #914
 - Support += and *= for ListField #595
@@ -266,7 +322,7 @@ Changes in 0.10.0
 - Fixes some internal _id handling issue. #961
 - Updated URL and Email Field regex validators, added schemes argument to URLField validation. #652
 - Capped collection multiple of 256. #1011
- Added `BaseQuerySet.aggregate_sum` and `BaseQuerySet.aggregate_average` methods.
+- Added ``BaseQuerySet.aggregate_sum`` and ``BaseQuerySet.aggregate_average`` methods.
 - Fix for delete with write_concern {'w': 0}. #1008
 - Allow dynamic lookup for more than two parts. #882
 - Added support for min_distance on geo queries. #831
@@ -275,10 +331,10 @@ Changes in 0.10.0
 Changes in 0.9.0
 ================
 - Update FileField when creating a new file #714
- Added `EmbeddedDocumentListField` for Lists of Embedded Documents. #826
+- Added ``EmbeddedDocumentListField`` for Lists of Embedded Documents. #826
 - ComplexDateTimeField should fall back to None when null=True #864
 - Request Support for $min, $max Field update operators #863
- `BaseDict` does not follow `setdefault` #866
+- ``BaseDict`` does not follow ``setdefault`` #866
 - Add support for $type operator # 766
 - Fix tests for pymongo 2.8+ #877
 - No module named 'django.utils.importlib' (Django dev) #872
@@ -299,13 +355,13 @@ Changes in 0.9.0
 - Stop ensure_indexes running on a secondaries unless connection is through mongos #746
 - Not overriding default values when loading a subset of fields #399
 - Saving document doesn't create new fields in existing collection #620
- Added `Queryset.aggregate` wrapper to aggregation framework #703
+- Added ``Queryset.aggregate`` wrapper to aggregation framework #703
 - Added support to show original model fields on to_json calls instead of db_field #697
 - Added Queryset.search_text to Text indexes searchs #700
 - Fixed tests for Django 1.7 #696
 - Follow ReferenceFields in EmbeddedDocuments with select_related #690
 - Added preliminary support for text indexes #680
- Added `elemMatch` operator as well - `match` is too obscure #653
+- Added ``elemMatch`` operator as well - ``match`` is too obscure #653
 - Added support for progressive JPEG #486 #548
 - Allow strings to be used in index creation #675
 - Fixed EmbeddedDoc weakref proxy issue #592
@@ -341,11 +397,11 @@ Changes in 0.9.0
 - Increase email field length to accommodate new TLDs #726
 - index_cls is ignored when deciding to set _cls as index prefix #733
 - Make 'db' argument to connection optional #737
- Allow atomic update for the entire `DictField` #742
+- Allow atomic update for the entire ``DictField`` #742
 - Added MultiPointField, MultiLineField, MultiPolygonField
 - Fix multiple connections aliases being rewritten #748
 - Fixed a few instances where reverse_delete_rule was written as reverse_delete_rules. #791
- Make `in_bulk()` respect `no_dereference()` #775
+- Make ``in_bulk()`` respect ``no_dereference()`` #775
 - Handle None from model __str__; Fixes #753 #754
 - _get_changed_fields fix for embedded documents with id field. #925

@@ -399,18 +455,15 @@ Changes in 0.8.4

 Changes in 0.8.3
 ================
- Fixed EmbeddedDocuments with `id` also storing `_id` (#402)
+- Fixed EmbeddedDocuments with ``id`` also storing ``_id`` (#402)
 - Added get_proxy_object helper to filefields (#391)
 - Added QuerySetNoCache and QuerySet.no_cache() for lower memory consumption (#365)
 - Fixed sum and average mapreduce dot notation support (#375, #376, #393)
 - Fixed as_pymongo to return the id (#386)
- Document.select_related() now respects `db_alias` (#377)
+- Document.select_related() now respects ``db_alias`` (#377)
 - Reload uses shard_key if applicable (#384)
 - Dynamic fields are ordered based on creation and stored in _fields_ordered (#396)
-
-  **Potential breaking change:** http://docs.mongoengine.org/en/latest/upgrade.html#to-0-8-3
-
- Fixed pickling dynamic documents `_dynamic_fields` (#387)
+- Fixed pickling dynamic documents ``_dynamic_fields`` (#387)
 - Fixed ListField setslice and delslice dirty tracking (#390)
 - Added Django 1.5 PY3 support (#392)
 - Added match ($elemMatch) support for EmbeddedDocuments (#379)
@@ -451,7 +504,7 @@ Changes in 0.8.0
 ================
 - Fixed querying ReferenceField custom_id (#317)
 - Fixed pickle issues with collections (#316)
- Added `get_next_value` preview for SequenceFields (#319)
+- Added ``get_next_value`` preview for SequenceFields (#319)
 - Added no_sub_classes context manager and queryset helper (#312)
 - Querysets now utilises a local cache
 - Changed __len__ behaviour in the queryset (#247, #311)
@@ -480,7 +533,7 @@ Changes in 0.8.0
 - Updated connection to use MongoClient (#262, #274)
 - Fixed db_alias and inherited Documents (#143)
 - Documentation update for document errors (#124)
- Deprecated `get_or_create` (#35)
+- Deprecated ``get_or_create`` (#35)
 - Updated inheritable objects created by upsert now contain _cls (#118)
 - Added support for creating documents with embedded documents in a single operation (#6)
 - Added to_json and from_json to Document (#1)
@@ -601,7 +654,7 @@ Changes in 0.7.0
 - Fixed UnboundLocalError in composite index with pk field (#88)
 - Updated ReferenceField's to optionally store ObjectId strings
  this will become the default in 0.8 (#89)
- Added FutureWarning - save will default to `cascade=False` in 0.8
+- Added FutureWarning - save will default to ``cascade=False`` in 0.8
 - Added example of indexing embedded document fields (#75)
 - Fixed ImageField resizing when forcing size (#80)
 - Add flexibility for fields handling bad data (#78)
@@ -697,7 +750,7 @@ Changes in 0.6.8
 ================
 - Fixed FileField losing reference when no default set
 - Removed possible race condition from FileField (grid_file)
- Added assignment to save, can now do: `b = MyDoc(**kwargs).save()`
+- Added assignment to save, can now do: ``b = MyDoc(**kwargs).save()``
 - Added support for pull operations on nested EmbeddedDocuments
 - Added support for choices with GenericReferenceFields
 - Added support for choices with GenericEmbeddedDocumentFields
@@ -712,7 +765,7 @@ Changes in 0.6.7
 - Fixed indexing on '_id' or 'pk' or 'id'
 - Invalid data from the DB now raises a InvalidDocumentError
 - Cleaned up the Validation Error - docs and code
- Added meta `auto_create_index` so you can disable index creation
+- Added meta ``auto_create_index`` so you can disable index creation
 - Added write concern options to inserts
 - Fixed typo in meta for index options
 - Bug fix Read preference now passed correctly
@@ -753,7 +806,6 @@ Changes in 0.6.1

 Changes in 0.6
 ==============
-
 - Added FutureWarning to inherited classes not declaring 'allow_inheritance' as the default will change in 0.7
 - Added support for covered indexes when inheritance is off
 - No longer always upsert on save for items with a '_id'
@@ -978,7 +1030,6 @@ Changes in v0.1.3
  querying takes place
 - A few minor bugfixes

-
 Changes in v0.1.2
 =================
 - Query values may be processed before before being used in queries
@@ -987,7 +1038,6 @@ Changes in v0.1.2
 - Added ``BooleanField``
 - Added ``Document.reload()`` method

-
 Changes in v0.1.1
 =================
 - Documents may now use capped collections
--- a/docs/code/tumblelog.py
+++ b/docs/code/tumblelog.py
@@ -1,16 +1,19 @@
 from mongoengine import *

-connect('tumblelog')
+connect("tumblelog")
+

 class Comment(EmbeddedDocument):
    content = StringField()
    name = StringField(max_length=120)

+
 class User(Document):
    email = StringField(required=True)
    first_name = StringField(max_length=50)
    last_name = StringField(max_length=50)

+
 class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User)
@@ -18,54 +21,57 @@ class Post(Document):
    comments = ListField(EmbeddedDocumentField(Comment))

    # bugfix
-    meta = {'allow_inheritance': True}
+    meta = {"allow_inheritance": True}


 class TextPost(Post):
    content = StringField()

+
 class ImagePost(Post):
    image_path = StringField()

+
 class LinkPost(Post):
    link_url = StringField()

+
 Post.drop_collection()

-john = User(email='jdoe@example.com', first_name='John', last_name='Doe')
+john = User(email="jdoe@example.com", first_name="John", last_name="Doe")
 john.save()

-post1 = TextPost(title='Fun with MongoEngine', author=john)
-post1.content = 'Took a look at MongoEngine today, looks pretty cool.'
-post1.tags = ['mongodb', 'mongoengine']
+post1 = TextPost(title="Fun with MongoEngine", author=john)
+post1.content = "Took a look at MongoEngine today, looks pretty cool."
+post1.tags = ["mongodb", "mongoengine"]
 post1.save()

-post2 = LinkPost(title='MongoEngine Documentation', author=john)
-post2.link_url = 'http://tractiondigital.com/labs/mongoengine/docs'
-post2.tags = ['mongoengine']
+post2 = LinkPost(title="MongoEngine Documentation", author=john)
+post2.link_url = "http://tractiondigital.com/labs/mongoengine/docs"
+post2.tags = ["mongoengine"]
 post2.save()

-print('ALL POSTS')
+print("ALL POSTS")
 print()
 for post in Post.objects:
    print(post.title)
-    #print '=' * post.title.count()
+    # print '=' * post.title.count()
    print("=" * 20)

    if isinstance(post, TextPost):
        print(post.content)

    if isinstance(post, LinkPost):
-        print('Link:', post.link_url)
+        print("Link:", post.link_url)

    print()
 print()

-print('POSTS TAGGED \'MONGODB\'')
+print("POSTS TAGGED 'MONGODB'")
 print()
-for post in Post.objects(tags='mongodb'):
+for post in Post.objects(tags="mongodb"):
    print(post.title)
 print()

-num_posts = Post.objects(tags='mongodb').count()
+num_posts = Post.objects(tags="mongodb").count()
 print('Found %d posts with tag "mongodb"' % num_posts)
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -11,7 +11,8 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.

-import sys, os
+import os
+import sys

 import sphinx_rtd_theme

@@ -20,29 +21,29 @@ import mongoengine
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
-sys.path.insert(0, os.path.abspath('..'))
+sys.path.insert(0, os.path.abspath(".."))

 # -- General configuration -----------------------------------------------------

 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo']
+extensions = ["sphinx.ext.autodoc", "sphinx.ext.todo"]

 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
+templates_path = ["_templates"]

 # The suffix of source filenames.
-source_suffix = '.rst'
+source_suffix = ".rst"

 # The encoding of source files.
-#source_encoding = 'utf-8'
+# source_encoding = 'utf-8'

 # The master toctree document.
-master_doc = 'index'
+master_doc = "index"

 # General information about the project.
-project = u'MongoEngine'
-copyright = u'2009, MongoEngine Authors'
+project = u"MongoEngine"
+copyright = u"2009, MongoEngine Authors"

 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -55,68 +56,66 @@ release = mongoengine.get_version()

 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
-#language = None
+# language = None

 # There are two options for replacing |today|: either, you set today to some
 # non-false value, then it is used:
-#today = ''
+# today = ''
 # Else, today_fmt is used as the format for a strftime call.
-#today_fmt = '%B %d, %Y'
+# today_fmt = '%B %d, %Y'

 # List of documents that shouldn't be included in the build.
-#unused_docs = []
+# unused_docs = []

 # List of directories, relative to source directory, that shouldn't be searched
 # for source files.
-exclude_trees = ['_build']
+exclude_trees = ["_build"]

 # The reST default role (used for this markup: `text`) to use for all documents.
-#default_role = None
+# default_role = None

 # If true, '()' will be appended to :func: etc. cross-reference text.
-#add_function_parentheses = True
+# add_function_parentheses = True

 # If true, the current module name will be prepended to all description
 # unit titles (such as .. function::).
-#add_module_names = True
+# add_module_names = True

 # If true, sectionauthor and moduleauthor directives will be shown in the
 # output. They are ignored by default.
-#show_authors = False
+# show_authors = False

 # The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
+pygments_style = "sphinx"

 # A list of ignored prefixes for module index sorting.
-#modindex_common_prefix = []
+# modindex_common_prefix = []


 # -- Options for HTML output ---------------------------------------------------

 # The theme to use for HTML and HTML Help pages.  Major themes that come with
 # Sphinx are currently 'default' and 'sphinxdoc'.
-html_theme = 'sphinx_rtd_theme'
+html_theme = "sphinx_rtd_theme"

 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
 # documentation.
-html_theme_options = {
-    'canonical_url': 'http://docs.mongoengine.org/en/latest/'
-}
+html_theme_options = {"canonical_url": "http://docs.mongoengine.org/en/latest/"}

 # Add any paths that contain custom themes here, relative to this directory.
 html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

 # The name for this set of Sphinx documents.  If None, it defaults to
 # "<project> v<release> documentation".
-#html_title = None
+# html_title = None

 # A shorter title for the navigation bar.  Default is the same as html_title.
-#html_short_title = None
+# html_short_title = None

 # The name of an image file (relative to this directory) to place at the top
 # of the sidebar.
-#html_logo = None
+# html_logo = None

 # The name of an image file (within the static path) to use as favicon of the
 # docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
@@ -126,11 +125,11 @@ html_favicon = "favicon.ico"
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-#html_static_path = ['_static']
+# html_static_path = ['_static']

 # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 # using the given strftime format.
-#html_last_updated_fmt = '%b %d, %Y'
+# html_last_updated_fmt = '%b %d, %Y'

 # If true, SmartyPants will be used to convert quotes and dashes to
 # typographically correct entities.
@@ -138,69 +137,68 @@ html_use_smartypants = True

 # Custom sidebar templates, maps document names to template names.
 html_sidebars = {
-    'index': ['globaltoc.html', 'searchbox.html'],
-    '**': ['localtoc.html', 'relations.html', 'searchbox.html']
+    "index": ["globaltoc.html", "searchbox.html"],
+    "**": ["localtoc.html", "relations.html", "searchbox.html"],
 }


 # Additional templates that should be rendered to pages, maps page names to
 # template names.
-#html_additional_pages = {}
+# html_additional_pages = {}

 # If false, no module index is generated.
-#html_use_modindex = True
+# html_use_modindex = True

 # If false, no index is generated.
-#html_use_index = True
+# html_use_index = True

 # If true, the index is split into individual pages for each letter.
-#html_split_index = False
+# html_split_index = False

 # If true, links to the reST sources are added to the pages.
-#html_show_sourcelink = True
+# html_show_sourcelink = True

 # If true, an OpenSearch description file will be output, and all pages will
 # contain a <link> tag referring to it.  The value of this option must be the
 # base URL from which the finished HTML is served.
-#html_use_opensearch = ''
+# html_use_opensearch = ''

 # If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
-#html_file_suffix = ''
+# html_file_suffix = ''

 # Output file base name for HTML help builder.
-htmlhelp_basename = 'MongoEnginedoc'
+htmlhelp_basename = "MongoEnginedoc"


 # -- Options for LaTeX output --------------------------------------------------

 # The paper size ('letter' or 'a4').
-latex_paper_size = 'a4'
+latex_paper_size = "a4"

 # The font size ('10pt', '11pt' or '12pt').
-#latex_font_size = '10pt'
+# latex_font_size = '10pt'

 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title, author, documentclass [howto/manual]).
 latex_documents = [
-  ('index', 'MongoEngine.tex', 'MongoEngine Documentation',
-   'Ross Lawley', 'manual'),
+    ("index", "MongoEngine.tex", "MongoEngine Documentation", "Ross Lawley", "manual")
 ]

 # The name of an image file (relative to this directory) to place at the top of
 # the title page.
-#latex_logo = None
+# latex_logo = None

 # For "manual" documents, if this is true, then toplevel headings are parts,
 # not chapters.
-#latex_use_parts = False
+# latex_use_parts = False

 # Additional stuff for the LaTeX preamble.
-#latex_preamble = ''
+# latex_preamble = ''

 # Documents to append as an appendix to all manuals.
-#latex_appendices = []
+# latex_appendices = []

 # If false, no module index is generated.
-#latex_use_modindex = True
+# latex_use_modindex = True

-autoclass_content = 'both'
+autoclass_content = "both"
--- a/docs/django.rst
+++ b/docs/django.rst
@@ -13,7 +13,7 @@ Help Wanted!

 The MongoEngine team is looking for help contributing and maintaining a new
 Django extension for MongoEngine! If you have Django experience and would like
-to help contribute to the project, please get in touch on the 
-`mailing list <http://groups.google.com/group/mongoengine-users>`_ or by 
+to help contribute to the project, please get in touch on the
+`mailing list <http://groups.google.com/group/mongoengine-users>`_ or by
 simply contributing on
 `GitHub <https://github.com/MongoEngine/django-mongoengine>`_.
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@ -0,0 +1,12 @@
+==========================
+Frequently Asked Questions
+==========================
+
+Does MongoEngine support asynchronous drivers (Motor, TxMongo)?
+---------------------------------------------------------------
+
+No, MongoEngine is exclusively based on PyMongo and isn't designed to support other driver.
+If this is a requirement for your project, check the alternative:  `uMongo`_ and `MotorEngine`_.
+
+.. _uMongo: https://umongo.readthedocs.io/
+.. _MotorEngine: https://motorengine.readthedocs.io/
--- a/docs/guide/connecting.rst
+++ b/docs/guide/connecting.rst
@@ -86,7 +86,7 @@ using 3 different databases to store data::
        connect(alias='user-db-alias', db='user-db')
        connect(alias='book-db-alias', db='book-db')
        connect(alias='users-books-db-alias', db='users-books-db')
-        
+
        class User(Document):
            name = StringField()

--- a/docs/guide/defining-documents.rst
+++ b/docs/guide/defining-documents.rst
@@ -352,7 +352,7 @@ Its value can take any of the following constants:
  Deletion is denied if there still exist references to the object being
  deleted.
 :const:`mongoengine.NULLIFY`
-  Any object's fields still referring to the object being deleted are removed
+  Any object's fields still referring to the object being deleted are set to None
  (using MongoDB's "unset" operation), effectively nullifying the relationship.
 :const:`mongoengine.CASCADE`
  Any object containing fields that are referring to the object being deleted
@@ -555,7 +555,6 @@ There are a few top level defaults for all indexes that can be set::
            'index_background': True,
            'index_cls': False,
            'auto_create_index': True,
-            'index_drop_dups': True,
        }


@@ -574,11 +573,6 @@ There are a few top level defaults for all indexes that can be set::
    in systems where indexes are managed separately. Disabling this will improve
    performance.

-:attr:`index_drop_dups` (Optional)
-    Set the default value for if an index should drop duplicates
-    Since MongoDB 3.0 drop_dups is not supported anymore. Raises a Warning
-    and has no effect
-

 Compound Indexes and Indexing sub documents
 -------------------------------------------
@@ -714,11 +708,16 @@ subsequent calls to :meth:`~mongoengine.queryset.QuerySet.order_by`. ::
 Shard keys
 ==========

-If your collection is sharded, then you need to specify the shard key as a tuple,
-using the :attr:`shard_key` attribute of :attr:`~mongoengine.Document.meta`.
-This ensures that the shard key is sent with the query when calling the
-:meth:`~mongoengine.document.Document.save` or
-:meth:`~mongoengine.document.Document.update` method on an existing
+If your collection is sharded by multiple keys, then you can improve shard
+routing (and thus the performance of your application) by specifying the shard
+key, using the :attr:`shard_key` attribute of
+:attr:`~mongoengine.Document.meta`. The shard key should be defined as a tuple.
+
+This ensures that the full shard key is sent with the query when calling
+methods such as :meth:`~mongoengine.document.Document.save`,
+:meth:`~mongoengine.document.Document.update`,
+:meth:`~mongoengine.document.Document.modify`, or
+:meth:`~mongoengine.document.Document.delete` on an existing
 :class:`~mongoengine.Document` instance::

    class LogEntry(Document):
@@ -728,7 +727,8 @@ This ensures that the shard key is sent with the query when calling the
        data = StringField()

        meta = {
-            'shard_key': ('machine', 'timestamp',)
+            'shard_key': ('machine', 'timestamp'),
+            'indexes': ('machine', 'timestamp'),
        }

 .. _document-inheritance:
@@ -738,7 +738,7 @@ Document inheritance

 To create a specialised type of a :class:`~mongoengine.Document` you have
 defined, you may subclass it and add any extra fields or methods you may need.
-As this is new class is not a direct subclass of
+As this new class is not a direct subclass of
 :class:`~mongoengine.Document`, it will not be stored in its own collection; it
 will use the same collection as its superclass uses. This allows for more
 convenient and efficient retrieval of related documents -- all you need do is
@@ -761,6 +761,27 @@ document.::
          Setting :attr:`allow_inheritance` to True should also be used in
          :class:`~mongoengine.EmbeddedDocument` class in case you need to subclass it

+When it comes to querying using :attr:`.objects()`, querying `Page.objects()` will query
+both `Page` and `DatedPage` whereas querying `DatedPage` will only query the `DatedPage` documents.
+Behind the scenes, MongoEngine deals with inheritance by adding a :attr:`_cls` attribute that contains
+the class name in every documents. When a document is loaded, MongoEngine checks
+it's :attr:`_cls` attribute and use that class to construct the instance.::
+
+    Page(title='a funky title').save()
+    DatedPage(title='another title', date=datetime.utcnow()).save()
+
+    print(Page.objects().count())         # 2
+    print(DatedPage.objects().count())    # 1
+
+    # print documents in their native form
+    # we remove 'id' to avoid polluting the output with unnecessary detail
+    qs = Page.objects.exclude('id').as_pymongo()
+    print(list(qs))
+    # [
+    #   {'_cls': u 'Page', 'title': 'a funky title'},
+    #   {'_cls': u 'Page.DatedPage', 'title': u 'another title', 'date': datetime.datetime(2019, 12, 13, 20, 16, 59, 993000)}
+    # ]
+
 Working with existing data
 --------------------------
 As MongoEngine no longer defaults to needing :attr:`_cls`, you can quickly and
--- a/docs/guide/gridfs.rst
+++ b/docs/guide/gridfs.rst
@@ -10,8 +10,9 @@ Writing
 GridFS support comes in the form of the :class:`~mongoengine.fields.FileField` field
 object. This field acts as a file-like object and provides a couple of
 different ways of inserting and retrieving data. Arbitrary metadata such as
-content type can also be stored alongside the files. In the following example,
-a document is created to store details about animals, including a photo::
+content type can also be stored alongside the files. The object returned when accessing a
+FileField is a proxy to `Pymongo's GridFS <https://api.mongodb.com/python/current/examples/gridfs.html#gridfs-example>`_
+In the following example, a document is created to store details about animals, including a photo::

    class Animal(Document):
        genus = StringField()
@@ -20,8 +21,8 @@ a document is created to store details about animals, including a photo::

    marmot = Animal(genus='Marmota', family='Sciuridae')

-    marmot_photo = open('marmot.jpg', 'rb')
-    marmot.photo.put(marmot_photo, content_type = 'image/jpeg')
+    with open('marmot.jpg', 'rb') as fd:
+        marmot.photo.put(fd, content_type = 'image/jpeg')
    marmot.save()

 Retrieval
@@ -34,6 +35,20 @@ field. The file can also be retrieved just as easily::
    photo = marmot.photo.read()
    content_type = marmot.photo.content_type

+.. note:: If you need to read() the content of a file multiple times, you'll need to "rewind"
+    the file-like object using `seek`::
+
+        marmot = Animal.objects(genus='Marmota').first()
+        content1 = marmot.photo.read()
+        assert content1 != ""
+
+        content2 = marmot.photo.read()    # will be empty
+        assert content2 == ""
+
+        marmot.photo.seek(0)              # rewind the file by setting the current position of the cursor in the file to 0
+        content3 = marmot.photo.read()
+        assert content3 == content1
+
 Streaming
 ---------

--- a/docs/guide/index.rst
+++ b/docs/guide/index.rst
@@ -14,4 +14,5 @@ User Guide
   gridfs
   signals
   text-indexes
+   logging-monitoring
   mongomock
--- a/docs/guide/installing.rst
+++ b/docs/guide/installing.rst
@@ -12,7 +12,7 @@ MongoEngine is available on PyPI, so you can use :program:`pip`:

 .. code-block:: console

-    $ pip install mongoengine
+    $ python -m pip install mongoengine

 Alternatively, if you don't have setuptools installed, `download it from PyPi
 <http://pypi.python.org/pypi/mongoengine/>`_ and run
--- a/docs/guide/logging-monitoring.rst
+++ b/docs/guide/logging-monitoring.rst
@@ -0,0 +1,80 @@
+==================
+Logging/Monitoring
+==================
+
+It is possible to use `pymongo.monitoring <https://api.mongodb.com/python/current/api/pymongo/monitoring.html>`_ to monitor
+the driver events (e.g: queries, connections, etc). This can be handy if you want to monitor the queries issued by
+MongoEngine to the driver.
+
+To use `pymongo.monitoring` with MongoEngine, you need to make sure that you are registering the listeners
+**before** establishing the database connection (i.e calling `connect`):
+
+The following snippet provides a basic logging of all command events:
+
+.. code-block:: python
+
+    import logging
+    from pymongo import monitoring
+    from mongoengine import *
+
+    log = logging.getLogger()
+    log.setLevel(logging.DEBUG)
+    logging.basicConfig(level=logging.DEBUG)
+
+
+    class CommandLogger(monitoring.CommandListener):
+
+        def started(self, event):
+            log.debug("Command {0.command_name} with request id "
+                     "{0.request_id} started on server "
+                     "{0.connection_id}".format(event))
+
+        def succeeded(self, event):
+            log.debug("Command {0.command_name} with request id "
+                     "{0.request_id} on server {0.connection_id} "
+                     "succeeded in {0.duration_micros} "
+                     "microseconds".format(event))
+
+        def failed(self, event):
+            log.debug("Command {0.command_name} with request id "
+                     "{0.request_id} on server {0.connection_id} "
+                     "failed in {0.duration_micros} "
+                     "microseconds".format(event))
+
+    monitoring.register(CommandLogger())
+
+
+    class Jedi(Document):
+        name = StringField()
+
+
+    connect()
+
+
+    log.info('GO!')
+
+    log.info('Saving an item through MongoEngine...')
+    Jedi(name='Obi-Wan Kenobii').save()
+
+    log.info('Querying through MongoEngine...')
+    obiwan = Jedi.objects.first()
+
+    log.info('Updating through MongoEngine...')
+    obiwan.name = 'Obi-Wan Kenobi'
+    obiwan.save()
+
+
+Executing this prints the following output::
+
+    INFO:root:GO!
+    INFO:root:Saving an item through MongoEngine...
+    DEBUG:root:Command insert with request id 1681692777 started on server ('localhost', 27017)
+    DEBUG:root:Command insert with request id 1681692777 on server ('localhost', 27017) succeeded in 562 microseconds
+    INFO:root:Querying through MongoEngine...
+    DEBUG:root:Command find with request id 1714636915 started on server ('localhost', 27017)
+    DEBUG:root:Command find with request id 1714636915 on server ('localhost', 27017) succeeded in 341 microseconds
+    INFO:root:Updating through MongoEngine...
+    DEBUG:root:Command update with request id 1957747793 started on server ('localhost', 27017)
+    DEBUG:root:Command update with request id 1957747793 on server ('localhost', 27017) succeeded in 455 microseconds
+
+More details can of course be obtained by checking the `event` argument from the `CommandListener`.
--- a/docs/guide/mongomock.rst
+++ b/docs/guide/mongomock.rst
@@ -2,10 +2,10 @@
 Use mongomock for testing
 ==============================

-`mongomock <https://github.com/vmalloc/mongomock/>`_ is a package to do just 
+`mongomock <https://github.com/vmalloc/mongomock/>`_ is a package to do just
 what the name implies, mocking a mongo database.

-To use with mongoengine, simply specify mongomock when connecting with 
+To use with mongoengine, simply specify mongomock when connecting with
 mongoengine:

 .. code-block:: python
@@ -21,7 +21,7 @@ or with an alias:
    conn = get_connection('testdb')

 Example of test file:
--------
+---------------------
 .. code-block:: python

    import unittest
@@ -45,4 +45,4 @@ Example of test file:
            pers.save()

            fresh_pers = Person.objects().first()
-            self.assertEqual(fresh_pers.name, 'John')
+            assert fresh_pers.name ==  'John'
--- a/docs/guide/querying.rst
+++ b/docs/guide/querying.rst
@@ -222,6 +222,18 @@ keyword argument::

 .. versionadded:: 0.4

+Sorting/Ordering results
+========================
+It is possible to order the results by 1 or more keys using :meth:`~mongoengine.queryset.QuerySet.order_by`.
+The order may be specified by prepending each of the keys by "+" or "-". Ascending order is assumed if there's no prefix.::
+
+    # Order by ascending date
+    blogs = BlogPost.objects().order_by('date')    # equivalent to .order_by('+date')
+
+    # Order by ascending date first, then descending title
+    blogs = BlogPost.objects().order_by('+date', '-title')
+
+
 Limiting and skipping results
 =============================
 Just as with traditional ORMs, you may limit the number of results returned or
@@ -349,9 +361,9 @@ Just as with limiting and skipping results, there is a method on a
 You could technically use ``len(User.objects)`` to get the same result, but it
 would be significantly slower than :meth:`~mongoengine.queryset.QuerySet.count`.
 When you execute a server-side count query, you let MongoDB do the heavy
-lifting and you receive a single integer over the wire. Meanwhile, len()
+lifting and you receive a single integer over the wire. Meanwhile, ``len()``
 retrieves all the results, places them in a local cache, and finally counts
-them. If we compare the performance of the two operations, len() is much slower
+them. If we compare the performance of the two operations, ``len()`` is much slower
 than :meth:`~mongoengine.queryset.QuerySet.count`.

 Further aggregation
@@ -386,6 +398,25 @@ would be generating "tag-clouds"::
    top_tags = sorted(tag_freqs.items(), key=itemgetter(1), reverse=True)[:10]


+MongoDB aggregation API
+-----------------------
+If you need to run aggregation pipelines, MongoEngine provides an entry point to `Pymongo's aggregation framework <https://api.mongodb.com/python/current/examples/aggregation.html#aggregation-framework>`_
+through :meth:`~mongoengine.queryset.QuerySet.aggregate`. Check out Pymongo's documentation for the syntax and pipeline.
+An example of its use would be::
+
+        class Person(Document):
+            name = StringField()
+
+        Person(name='John').save()
+        Person(name='Bob').save()
+
+        pipeline = [
+            {"$sort" : {"name" : -1}},
+            {"$project": {"_id": 0, "name": {"$toUpper": "$name"}}}
+            ]
+        data = Person.objects().aggregate(pipeline)
+        assert data == [{'name': 'BOB'}, {'name': 'JOHN'}]
+
 Query efficiency and performance
 ================================

@@ -578,7 +609,7 @@ to push values with index::
 .. note::
    Currently only top level lists are handled, future versions of mongodb /
    pymongo plan to support nested positional operators.  See `The $ positional
-    operator <http://www.mongodb.org/display/DOCS/Updating#Updating-The%24positionaloperator>`_.
+    operator <https://docs.mongodb.com/manual/tutorial/update-documents/#Updating-The%24positionaloperator>`_.

 Server-side javascript execution
 ================================
--- a/docs/guide/signals.rst
+++ b/docs/guide/signals.rst
@@ -44,8 +44,8 @@ Available signals include:

 `post_save`
  Called within :meth:`~mongoengine.Document.save` after most actions
-  (validation, insert/update, and cascades, but not clearing dirty flags) have 
-  completed successfully.  Passed the additional boolean keyword argument 
+  (validation, insert/update, and cascades, but not clearing dirty flags) have
+  completed successfully.  Passed the additional boolean keyword argument
  `created` to indicate if the save was an insert or an update.

 `pre_delete`
--- a/docs/guide/text-indexes.rst
+++ b/docs/guide/text-indexes.rst
@@ -8,7 +8,7 @@ After MongoDB 2.4 version, supports search documents by text indexes.
 Defining a Document with text index
 ===================================
 Use the *$* prefix to set a text index, Look the declaration::
-  
+
  class News(Document):
      title = StringField()
      content = StringField()
@@ -35,10 +35,10 @@ Saving a document::
       content="Various improvements").save()

 Next, start a text search using :attr:`QuerySet.search_text` method::
-  
+
  document = News.objects.search_text('testing').first()
  document.title # may be: "Using mongodb text search"
-  
+
  document = News.objects.search_text('released').first()
  document.title # may be: "MongoEngine 0.9 released"

--- a/docs/index.rst
+++ b/docs/index.rst
@@ -7,7 +7,7 @@ MongoDB. To install it, simply run

 .. code-block:: console

-    $ pip install -U mongoengine
+    $ python -m pip install -U mongoengine

 :doc:`tutorial`
  A quick tutorial building a tumblelog to get you up and running with
@@ -23,9 +23,18 @@ MongoDB. To install it, simply run
 :doc:`upgrade`
  How to upgrade MongoEngine.

+:doc:`faq`
+  Frequently Asked Questions
+
 :doc:`django`
  Using MongoEngine and Django

+MongoDB and driver support
+--------------------------
+
+MongoEngine is based on the PyMongo driver and tested against multiple versions of MongoDB.
+For further details, please refer to the `readme <https://github.com/MongoEngine/mongoengine#mongoengine>`_.
+
 Community
 ---------

@@ -73,6 +82,7 @@ formats for offline reading.
    apireference
    changelog
    upgrade
+    faq
    django

 Indices and tables
@@ -81,4 +91,3 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
-
--- a/docs/tutorial.rst
+++ b/docs/tutorial.rst
@@ -18,7 +18,7 @@ location --- running it locally will be easier, but if that is not an option
 then it may be run on a remote server. If you haven't installed MongoEngine,
 simply use pip to install it like so::

-    $ pip install mongoengine
+    $ python -m pip install mongoengine

 Before we can start using MongoEngine, we need to tell it how to connect to our
 instance of :program:`mongod`. For this we use the :func:`~mongoengine.connect`
--- a/docs/upgrade.rst
+++ b/docs/upgrade.rst
@@ -52,7 +52,7 @@ rename its occurrences.
 This release includes a major rehaul of MongoEngine's code quality and
 introduces a few breaking changes. It also touches many different parts of
 the package and although all the changes have been tested and scrutinized,
-you're encouraged to thorougly test the upgrade.
+you're encouraged to thoroughly test the upgrade.

 First breaking change involves renaming `ConnectionError` to `MongoEngineConnectionError`.
 If you import or catch this exception, you'll need to rename it in your code.
@@ -85,10 +85,10 @@ by default from now on.

 The 0.8.7 package on pypi was corrupted.  If upgrading from 0.8.7 to 0.9.0 please follow: ::

-    pip uninstall pymongo
-    pip uninstall mongoengine
-    pip install pymongo==2.8
-    pip install mongoengine
+    python -m pip uninstall pymongo
+    python -m pip uninstall mongoengine
+    python -m pip install pymongo==2.8
+    python -m pip install mongoengine

 0.8.7
 *****
@@ -153,7 +153,7 @@ inherited classes like so: ::

    # 4. Remove indexes
    info = collection.index_information()
-    indexes_to_drop = [key for key, value in info.iteritems()
+    indexes_to_drop = [key for key, value in info.items()
                       if '_types' in dict(value['key'])]
    for index in indexes_to_drop:
        collection.drop_index(index)