Compare commits

...

458 Commits

Author SHA1 Message Date
Stefan Wojcik
34ba527e6d include a link to PyMongo docs for batch_size 2016-12-05 11:27:46 -05:00
Stefan Wojcik
ea9027755f drop redundant test 2016-12-05 00:17:03 -05:00
Stefan Wojcik
43668a93a2 implement BaseQuerySet.batch_size 2016-12-04 19:28:26 -05:00
Stefan Wójcik
15714ef855 Fix __repr__ method of the StrictDict (#1424) 2016-12-04 16:10:59 -05:00
Stefan Wójcik
eb743beaa3 fix doc.get_<field>_display + unit test inspired by #1279 (#1419) 2016-12-04 00:34:24 -05:00
Stefan Wójcik
0007535a46 Add support for cursor.comment (#1420) 2016-12-04 00:33:42 -05:00
Stefan Wójcik
8391af026c Fix filtering by embedded_doc=None (#1422) 2016-12-04 00:32:53 -05:00
Stefan Wójcik
800f656dcf remove unnecessary randomness in indexes tests (#1423) 2016-12-04 00:31:54 -05:00
Stefan Wojcik
088c5f49d9 update the changelog 2016-12-03 16:32:14 -05:00
Ollie Ford
d8d98b6143 Support Falsey primary_keys (#1354) 2016-12-03 16:10:05 -05:00
zeez
02fb3b9315 Support for authentication mechanism #905 (#1333) 2016-12-03 16:08:24 -05:00
Francesc Elies
4f87db784e Make the README example easier to replicate (#1382) 2016-12-02 22:05:20 -05:00
Jérôme Lafréchoux
7e6287b925 Merge pull request #1417 from MongoEngine/fix-db-field-in-sum-and-average
Fix BaseQuerySet#sum and BaseQuerySet#average for fields that specify a db_field
2016-12-02 20:53:48 +01:00
Stefan Wojcik
999cdfd997 Fix BaseQuerySet#sum and BaseQuerySet#average for fields that specify a db_field 2016-12-02 11:32:38 -05:00
Jérôme Lafréchoux
8d6cb087c6 Fix changelog 2016-11-29 09:28:13 +01:00
Stefan Wojcik
2b7417c728 add a missing entry to the changelog 2016-11-28 19:33:11 -05:00
Stefan Wójcik
3c455cf1c1 Improve health of this package (#1409)
* added flake8 and flake8-import-order to travis for py27

* fixed a test that fails from time to time depending on an order of a dict

* flake8 tweaks for the entire codebase excluding tests
2016-11-28 19:00:34 -05:00
Stefan Wójcik
5135185e31 Use SVG in README badges 2016-11-28 12:31:50 -05:00
Stefan Wojcik
b461f26e5d version bump 2016-11-28 10:42:05 -05:00
Stefan Wojcik
faef5b8570 finalize the v0.10.7 changelog 2016-11-28 10:40:20 -05:00
Omer Katz
0a20e04c10 Merge pull request #1383 from BenCoDev/patch-1
Dictionnary Field recommended use
2016-11-27 18:39:04 +02:00
Stefan Wojcik
d19bb2308d add #1389 to the changelog 2016-11-24 09:40:17 -05:00
BenCotte
d8dd07d9ef Updating Dict Fields use description
Update dict fields use misleading description to clarify use case.
2016-11-20 11:22:34 +01:00
Omer Katz
36c56243cd Merge pull request #1399 from sallyruthstruik/master
Add info in CHANGELOG
2016-11-20 10:28:14 +02:00
Stanivlav Kaledin
23d06b79a6 Add info in CHANGELOG 2016-11-17 19:05:46 +03:00
Omer Katz
e4c4e923ee Merge pull request #1397 from sallyruthstruik/master
Fixed issue https://github.com/MongoEngine/mongoengine/issues/442
2016-11-17 11:32:09 +02:00
Omer Katz
936d2f1f47 Merge pull request #1334 from touilleMan/bug-892
Raise DoesNotExist when dereferencing unknown document
2016-11-17 11:31:15 +02:00
Emmanuel Leblond
07018b5060 Raise DoesNotExist when dereferencing unknown document 2016-11-17 09:21:34 +01:00
Stanivlav Kaledin
ac90d6ae5c Don't force _cursor 2016-11-14 20:06:34 +03:00
Stanivlav Kaledin
2141f2c4c5 Fixed issue https://github.com/MongoEngine/mongoengine/issues/442
Added support for pickling BaseQueryset instances
Added BaseQueryset.__getstate__, BaseQuerySet.__setstate__ methods
2016-11-14 19:57:48 +03:00
Jérôme Lafréchoux
81870777a9 Update changelog 2016-10-24 12:00:01 +02:00
Jérôme Lafréchoux
845092dcad Merge pull request #1390 from closeio/dont-test-on-py-32
Don't run tests for python 3.2
2016-10-20 09:42:13 +02:00
Stefan Wojcik
dd473d1e1e remove v3.2 from .travis.yml 2016-10-19 18:15:15 -04:00
Stefan Wojcik
d2869bf4ed Merge branch 'master' of github.com:MongoEngine/mongoengine into dont-test-on-py-32 2016-10-19 18:13:41 -04:00
Jérôme Lafréchoux
891a3f4b29 Merge pull request #1391 from closeio/fix-py26
Fix Python 2.6 tests
2016-10-19 23:24:55 +02:00
Jérôme Lafréchoux
6767b50d75 Merge pull request #1389 from jtharpla/topic/fix-hosts-as-list
Fix connecting to a list of hosts
2016-10-19 23:01:21 +02:00
Stefan Wojcik
d9e4b562a9 for good measure, remove py32 from the commented-out envlist, too 2016-10-19 16:30:39 -04:00
Stefan Wojcik
fb3243f1bc readme fix 2016-10-19 16:23:48 -04:00
Stefan Wojcik
5fe1497c92 remove rednose from tox deps 2016-10-19 16:14:02 -04:00
Stefan Wojcik
5446592d44 remove rednose to see if it masks another issue 2016-10-19 16:05:59 -04:00
Stefan Wojcik
40ed9a53c9 dont run tests for python 3.2 2016-10-19 15:43:07 -04:00
Jeff Tharp
f7ac8cea90 Fix connecting to a list of hosts 2016-10-19 11:57:02 -07:00
Jérôme Lafréchoux
4ef5d1f0cd Merge pull request #1384 from closeio/fix-email-address-english
Use proper English spelling for "email address"
2016-10-12 17:40:52 +02:00
Thomas Steinacher
6992615c98 Use English spelling for "email address" 2016-10-12 16:59:54 +02:00
BenCotte
43dabb2825 Dictionnary Field recommended use
At Dictionary Fields description, it seems that the intent of the sentence make more sense by adding : "not".
2016-10-11 18:14:17 +02:00
Jérôme Lafréchoux
05e40e5681 Merge pull request #1128 from iici-gli/master
Fixed: ListField minus index assignment does not work #1119
2016-09-07 09:29:31 +02:00
Gang Li
2c4536e137 redo fix for ListField loses use_db_field when serializing #1217
The new fix reverted the change on BaseField to_mongo so that it will not force
new field class to add kwargs to to_mongo function. The new derived field class
to_mongo can support use_db_field and fields parameters as needed.
Basically all field classes derived from ComplexBaseField support those parameters.
2016-09-06 17:27:47 -04:00
Jérôme Lafréchoux
3dc81058a0 Merge pull request #1346 from anih/master
Speed up checking if we passed missing field
2016-09-06 09:51:33 +02:00
anih
bd84667a2b fixes 2016-09-06 09:27:41 +02:00
iici-gli
e5b6a12977 Merge pull request #1 from MongoEngine/master
pull new changes from original
2016-09-04 23:43:04 -04:00
Gang Li
ca415d5d62 Fix for:Base document _mark_as_changed bug #1369 2016-09-04 14:20:59 -04:00
Jérôme Lafréchoux
99b4fe7278 Merge pull request #1351 from mindojo-victor/1176
Fix for #1176 -- similar to https://github.com/MongoEngine/mongoengin…
2016-09-04 09:18:14 +02:00
Victor
327e164869 Fix for #1176 -- similar to https://github.com/MongoEngine/mongoengine/pull/982 but for update. 2016-09-04 08:12:17 +03:00
Jérôme Lafréchoux
25bc571f30 Merge pull request #1331 from bagerard/fix_unit_test
fixes in the test suite
2016-09-03 22:54:28 +02:00
Jérôme Lafréchoux
38c7e8a1d2 Merge pull request #1363 from skoval00/fix-misleading-comment
Fix misleading comment about the descriptor
2016-09-03 22:03:06 +02:00
Jérôme Lafréchoux
ca282e28e0 Merge pull request #1360 from Gallaecio/patch-1
Fix array-slicing documentation
2016-08-22 10:13:08 +02:00
Sergey Kovalev
5ef59c06df Fix misleading comment about the descriptor 2016-08-13 09:41:26 +03:00
Gallaecio
8f55d385d6 Fix array-slicing documentation
Fixes #1359.
2016-08-11 08:52:53 +02:00
Jérôme Lafréchoux
cd2fc25c19 Merge pull request #1353 from DionysusG/master
fix typo at docs/guide/defineing-documents.rst
2016-08-04 11:28:11 +02:00
DionysusG
709983eea6 fix typo at docs/guide/defineing-documents.rst 2016-08-04 16:21:52 +08:00
anih
40e99b1b80 Speed up checking if we passed missing field 2016-07-27 12:10:46 +02:00
Jérôme Lafréchoux
488684d960 Merge pull request #1340 from latteier/master
fix example for register_delete_rule. see issue #1339
2016-07-18 22:54:54 +02:00
Amos Latteier
f35034b989 fix example for register_delete_rule. see issue #1339 2016-07-18 13:23:01 -07:00
Omer Katz
9d6f9b1f26 Merge pull request #1336 from closeio/aggregate-sum-and-avg
Replace map-reduce based QuerySet.sum/average with aggregation-based implementations
2016-07-12 11:20:13 +03:00
Stefan Wojcik
6148a608fb update the changelog 2016-07-11 10:45:40 -07:00
Stefan Wojcik
3fa9e70383 prefer tuples over lists for immutable structures 2016-07-11 10:42:27 -07:00
Stefan Wojcik
16fea6f009 replace QuerySet.sum/average implementations with aggregate_sum/average + tweaks 2016-07-10 13:21:12 -07:00
Bastien Gérard
df9ed835ca fixes in unit tests 2016-07-02 23:01:36 +02:00
Jérôme Lafréchoux
e394c8f0f2 Merge pull request #1328 from anentropic/upsert-docs-fix
better description for upsert arg on some methods
2016-06-29 15:56:45 +02:00
Anentropic
21974f7288 better description for upsert arg on some methods 2016-06-29 14:24:33 +01:00
Jérôme Lafréchoux
5ef0170d77 Merge pull request #1324 from vahana/patch-1
Update changelog.rst
2016-06-24 19:58:31 +02:00
vahan
c21dcf14de Update changelog.rst 2016-06-24 13:45:42 -04:00
Jérôme Lafréchoux
a8d20d4e1e Merge pull request #1313 from roivision/master
Fix for issue # 1278
2016-06-24 17:46:04 +02:00
Jérôme Lafréchoux
8b307485b0 Merge pull request #1314 from adamchainz/readthedocs.io
Convert readthedocs links for their .org -> .io migration for hosted projects
2016-06-17 14:55:04 +02:00
Adam Chainz
4544afe422 Convert readthedocs links for their .org -> .io migration for hosted projects
As per [their blog post of the 27th April](https://blog.readthedocs.com/securing-subdomains/) ‘Securing subdomains’:

> Starting today, Read the Docs will start hosting projects from subdomains on the domain readthedocs.io, instead of on readthedocs.org. This change addresses some security concerns around site cookies while hosting user generated data on the same domain as our dashboard.

Test Plan: Manually visited all the links I’ve modified.
2016-06-16 21:21:10 +01:00
Jérôme Lafréchoux
9d7eba5f70 Merge pull request #1307 from xiaost/update-for-1304
Update changelog for #1304
2016-06-02 20:38:25 +02:00
xiaost
be0aee95f2 Update changelog for #1304 2016-06-03 01:27:39 +08:00
Omer Katz
3469ed7ab9 Merge pull request #1304 from xiaost/fix-no-cursor-timeout
Fix no_cursor_timeout with pymongo3
2016-05-29 10:15:20 +03:00
xiaost
1f223aa7e6 Fix no_cursor_timeout with pymongo3 2016-05-26 00:29:41 +08:00
Omer Katz
0a431ead5e Merge pull request #1289 from closeio/fix-typo
Fix typo in the docstring for __len__
2016-05-05 15:53:00 +03:00
Stefan Wojcik
f750796444 fix typo 2016-05-04 17:11:38 -07:00
vahan
c82bcd882a Merge pull request #1 from roivision/dynamic_document_dict_fix
* fixed the bug where dynamic doc has indx inside dict field
2016-05-01 23:07:24 -04:00
vahan
7d0ec33b54 * fixed the bug where dynamic doc has indx inside dict field 2016-05-01 22:59:39 -04:00
Omer Katz
43d48b3feb Merge pull request #1271 from maitbayev/master
Fixes unicode bug in EmbeddedDocumentListField
2016-04-17 09:15:23 +03:00
Omer Katz
2e406d2687 Merge pull request #1277 from shushen/Bug-681
Fix AttributeError when initializing EmbeddedDocuments
2016-04-11 12:57:08 +03:00
Shu Shen
3f30808104 Fix AttributeError when creating EmbeddedDocument
When an EmbeddedDocument is initialized with positional arguments, the
document attempts to read _auto_id_field attribute which may not exist
and would throw an AttributeError exception and fail the initialization.

This change and the test is based on the discussion in issue #681 and
PR #777 with a number of community members.
2016-04-07 15:18:33 -07:00
Omer Katz
ab10217c86 Merge pull request #1270 from Neurostack/master
Bug fixed accessing BaseList with negative indices
2016-03-31 22:27:56 +03:00
Neurostack
00430491ca Fixed bug accessing ListField (BaseList) with negative indices
If you __setitem__ in BaseList with a negative index and then try to save this, you will get an error like: OperationError: Could not save document (cannot use the part (shape of signal.shape.-1) to traverse the element ({shape: [ 0 ]})). To fix this I rectify negative list indices in BaseList _mark_as_changed as the appropriate positive index. This fixes the above error.
2016-03-31 08:04:19 -06:00
Madiyar Aitbayev
109202329f Handles unicode correctly EmbeddedDocumentListField 2016-03-31 02:33:13 +01:00
Omer Katz
3b1509f307 Added changelog entry for #1267. 2016-03-26 09:13:25 +03:00
Omer Katz
7ad7b08bed Merge pull request #1267 from wishtack/hotfix-map-field-unicode-key
Fix MapField in order to handle unicode keys.
2016-03-26 09:06:24 +03:00
Younes JAAIDI
4650e5e8fb Fix MapField in order to handle unicode keys. 2016-03-25 12:42:00 +01:00
Omer Katz
af59d4929e Merge pull request #1254 from gilbsgilbs/fix_long_fields_python3
Fix long fields python3
2016-03-23 15:17:06 +02:00
Gilb's
e34100bab4 Another attempt to fix random fails of test test_compound_key_dictfield. 2016-03-18 23:43:23 +01:00
Gilb's
d9b3a9fb60 Use six integer types instead of explicit types, since six is now a dependency of the project. 2016-03-18 19:51:09 +01:00
Gilb's
39eec59c90 Fix test failing randomly because of concurrency. 2016-03-18 19:45:34 +01:00
Gilb's
d651d0d472 Fix tests and imports. issue #1253 2016-03-18 19:45:34 +01:00
Gilbert Gilb's
87a2358a65 Fix unused variable. issue #1253 2016-03-18 19:45:34 +01:00
Gilbert Gilb's
cef4e313e1 Update changelog for #1253 2016-03-18 19:45:34 +01:00
Gilbert Gilb's
7cc1a4eba0 Fix long fields stored as int32 in Python 3. issue #1253 2016-03-18 19:45:34 +01:00
Omer Katz
c6cc0133b3 Merge pull request #1240 from gukoff/long_in_floatfield
Added support for long values in FloatFields
2016-03-18 10:24:53 +02:00
Omer Katz
7748e68440 Adjust changelog for #1188. 2016-03-10 12:19:11 +02:00
Omer Katz
6c2230a076 Merge pull request #1188 from DavidBord/fix-1187
fix-#1187: count on ListField of EmbeddedDocumentField fails
2016-03-10 12:18:20 +02:00
Konstantin Gukov
66b233eaea Added the six module to test int/long support 2016-03-06 23:01:49 +05:00
Konstantin Gukov
fed58f3920 Added support for long values in FloatFields 2016-02-24 14:07:22 +05:00
Omer Katz
815b2be7f7 Merge pull request #1183 from bitdivision/patch-1
Add EmbeddedDocumentListField to user guide
2016-02-24 09:01:54 +02:00
Omer Katz
f420c9fb7c Merge pull request #1235 from hhstore/master
fix a small bug - ReferenceField() comment give a wrong demo .
2016-02-24 08:58:40 +02:00
Omer Katz
01bdf10b94 Merge pull request #1241 from gukoff/broad_exceptions
Fixed too broad exception clauses in the project
2016-02-24 08:57:21 +02:00
Konstantin Gukov
ddedc1ee92 Fixed too broad exception clauses in the project 2016-02-23 23:50:45 +05:00
Emmanuel Leblond
9e9703183f Add test for nested list in EmbeddedDocument 2016-02-19 02:16:37 +01:00
Emmanuel Leblond
adce9e6220 Raise OperationError in drop_collection if no collection is set 2016-02-19 01:58:15 +01:00
Emmanuel Leblond
c499133bbe Add missing drop_collection in tests fields 2016-02-19 00:11:30 +01:00
hhstore
8f505c2dcc fix a small bug - ReferenceField() comment give a wrong demo . 2016-02-17 10:55:31 +08:00
Emmanuel Leblond
b320064418 Add signal_kwargs arg for save/delete/bulk insert 2016-02-09 14:28:55 +01:00
Emmanuel Leblond
a643933d16 Fix cascade delete mixing among collections 2016-01-30 11:59:55 +01:00
Omer Katz
2659ec5887 Merge pull request #1196 from nickptrvc/master
Fix pre_bulk_insert signal
2016-01-30 12:25:29 +02:00
Emmanuel Leblond
9f8327926d Improve a bit queryset's test_elem_match 2016-01-28 18:18:51 +01:00
Emmanuel Leblond
7a568dc118 Add version 0.10.7 - DEV in changelog.rst 2016-01-26 15:54:57 +01:00
Emmanuel Leblond
c946b06be5 Merge pull request #1218 from bbenne10/master
Curry **kwargs through to_mongo on fields
2016-01-26 15:53:21 +01:00
Bryan Bennett
c65fd0e477 Note changes for #1217 in Changelog 2016-01-26 08:34:52 -05:00
Bryan Bennett
8f8217e928 Add Bryan Bennett to AUTHORS 2016-01-26 08:34:52 -05:00
Bryan Bennett
6c9e1799c7 MongoEngine/mongoengine #1217: Curry **kwargs through to_mongo on fields 2016-01-26 08:34:52 -05:00
Emmanuel Leblond
decd70eb23 Merge pull request #1220 from bagerard/patch-1
fixed minor typo in docstring
(PR has been issued by mistake to dev branch insteed of master)
2016-01-26 00:28:04 +01:00
Emmanuel Leblond
a20d40618f Bump to v0.10.6 2016-01-25 01:42:19 +01:00
Emmanuel Leblond
b4af8ec751 Fix travis for python 3.2 2016-01-22 08:38:42 +01:00
Bastien
feb5eed8a5 fixed minor typo in docstring 2016-01-21 16:59:37 +01:00
Emmanuel Leblond
f4fa39c70e Revert "Force pip version to 7.1.2 in tox for py32 (support dropped for latter versions)"
This reverts commit 7b7165f5d8.
2016-01-20 13:07:06 +01:00
Emmanuel Leblond
7b7165f5d8 Force pip version to 7.1.2 in tox for py32 (support dropped for latter versions) 2016-01-20 11:48:31 +01:00
Emmanuel Leblond
13897db6d3 Fix mongomock url prefix error during connection 2016-01-20 11:06:45 +01:00
Emmanuel Leblond
c4afdb7198 Merge pull request #1123 from Cykooz/master
Fixed detection of shared connections
2016-01-19 18:38:03 +01:00
Emmanuel Leblond
0284975f3f Correct test_reload_of_non_strict_with_special_field_name for pymongo<2.9 2016-01-19 15:34:38 +01:00
Omer Katz
269e3d1303 Merge pull request #1205 from Zephor5/patch-1
add highlight for python code
2016-01-13 15:59:08 +02:00
Zephor
8c81f7ece9 add highlight for python code 2016-01-06 12:00:48 +08:00
Omer Katz
f6e0593774 Merge pull request #1198 from rusnassonov/patch-1
fix missing quote in /docs/guide/mongomock.rst
2015-12-27 13:54:58 +02:00
Ruslan Nassonov
3d80e549cb fix missing quote in /docs/guide/mongomock.rst 2015-12-25 15:52:35 +05:00
Nick Pjetrovic
acc7448dc5 Fix pre_bulk_insert signal 2015-12-24 18:30:46 -05:00
David Bordeynik
35d3d3de72 fix-#1187: count on ListField of EmbeddedDocumentField fails 2015-12-15 22:27:53 +02:00
Omer Katz
0372e07eb0 Merge pull request #1114 from gmacon/sparse-compound
Allow sparse compound indexes
2015-12-10 07:27:22 +02:00
George Macon
00221e3410 Allow sparse compound indexes 2015-12-09 18:38:28 -05:00
bitdivision
9c264611cf Add EmbeddedDocumentListField to user guide
The 'defining a document' section currently doesn't include EmbeddedDocumentListField. Only EmbeddedDocumentField
2015-12-09 11:40:22 +00:00
Omer Katz
31d7f70e27 Merge pull request #1153 from AWhetter/fixWindowsTest
Fixed not being able to run tests on Windows
2015-12-08 20:29:28 +02:00
Ashley Whetter
04e8b83d45 Fixed being unable to run tests on Windows 2015-12-08 18:08:10 +00:00
Omer Katz
e87bf71f20 Merge pull request #1170 from hhstore/master
fix for docs.code.tumblelog.py
2015-12-08 07:16:59 +02:00
Omer Katz
2dd70c8d62 Merge pull request #1180 from moonso/add-mongomock-docs
Added page for documenting mongomock. Updated docs/guide/index.rst
2015-12-06 13:23:04 +02:00
moonso
a3886702a3 Added page for documenting mongomock. Updated docs/guide/index.rst 2015-12-06 11:02:26 +01:00
Omer Katz
713af133a0 Moved #1151 changelog entry to the correct version. 2015-12-06 07:54:25 +02:00
Omer Katz
057ffffbf2 Merge pull request #1151 from RussellLuo/feature-support-mocking
Add support for mocking MongoEngine based on mongomock
2015-12-06 07:53:00 +02:00
RussellLuo
a81d6d124b Update AUTHORS and add changelog entry for #1151 2015-12-06 11:11:46 +08:00
RussellLuo
23f07fde5e Add support for mocking MongoEngine based on mongomock
Using `mongomock://` scheme in URI enables the mocking. Fix #1045.
2015-12-06 11:08:00 +08:00
Omer Katz
b42b760393 Merge branch 'fix-reloading-strict' of https://github.com/paularmand/mongoengine into fix-reloading-strict and bumped version.
# Conflicts:
#	AUTHORS
2015-11-30 12:13:47 +02:00
Omer Katz
bf6f4c48c0 Merge pull request #1167 from BeardedSteve/upsert_one
Upsert one
2015-11-30 12:08:04 +02:00
Paul-Armand Verhaegen
6133f04841 Manual merge conflicts in AUTHORS 2015-11-27 23:55:55 +01:00
Paul-Armand Verhaegen
3c18f79ea4 Added test for reloading of strict with special fields #1156 2015-11-27 23:45:25 +01:00
hhstore
2af8342fea bugfix - two small bugs. 2015-11-26 12:01:42 +08:00
srossiter
fc3db7942d updated changelog and version tuple 2015-11-24 12:56:59 +00:00
srossiter
164e2b2678 Docstring change and rename variable to avoid clash with kwargs 2015-11-24 12:53:09 +00:00
srossiter
b7b28390df Added upsert_one method on BaseQuerySet and modified test_upsert_one 2015-11-24 12:46:38 +00:00
Omer Katz
a6e996d921 Added #1165 to the changelog. 2015-11-24 07:06:54 +02:00
Omer Katz
07e666345d Merge pull request #1165 from touilleMan/bug-1164
Add SaveConditionError to __all__
2015-11-24 07:04:54 +02:00
Omer Katz
007f10d29d Merge pull request #1161 from AWhetter/docFix
Fixed a couple of documentation typos
2015-11-24 07:01:49 +02:00
Omer Katz
f9284d20ca Moved #1042 to the next version in the changelog. 2015-11-24 07:00:09 +02:00
Omer Katz
9050869781 Merge pull request #1042 from closeio/fix-read-preference
Fix read_preference
2015-11-24 06:58:51 +02:00
Stefan Wojcik
54975de0f3 fix read_preference for PyMongo 3+ 2015-11-23 10:46:52 -08:00
Stefan Wojcik
a7aead5138 re-create the cursor object whenever we apply read_preference 2015-11-23 10:46:52 -08:00
Omer Katz
6868f66f24 Merge pull request #1155 from AWhetter/fix837
ReferenceFields can now reference abstract Document types
2015-11-23 15:52:54 +02:00
Omer Katz
3c0b00e42d Added python 3.5 to the build. 2015-11-23 15:40:16 +02:00
Omer Katz
3327388f1f Merge pull request #1122 from larsbutler/improve-reverse_delete_rule-docs
fields.ReferenceField: add integer values to `reverse_delete_rule` docs
2015-11-23 15:29:20 +02:00
Ashley Whetter
04497aec36 Fixed setting dbref to True on abstract reference fields causing the reference to be stored incorrectly 2015-11-23 13:21:30 +00:00
Ashley Whetter
aa9d596930 Updated documentation for abstract reference changes 2015-11-23 13:21:30 +00:00
Ashley Whetter
f96e68cd11 Made type inheritance a validation check for abstract references 2015-11-23 13:20:35 +00:00
Ashley Whetter
013227323d ReferenceFields can now reference abstract Document types
A class that inherits from an abstract Document type is stored in the database
as a reference with a 'cls' field that is the class name of the document being
stored.

Fixes #837
2015-11-23 13:20:35 +00:00
Omer Katz
19cbb442ee Added #1129 to the changelog. 2015-11-23 13:57:15 +02:00
Omer Katz
c0e7f341cb Merge pull request #1129 from illico/feature/arbitrary-metadata
Indirection-free optimized field metadata.
2015-11-23 12:49:48 +02:00
Emmanuel Leblond
0a1ba7c434 Add SaveConditionError to __all__ 2015-11-21 10:25:11 +01:00
Omer Katz
b708dabf98 Merge pull request #1158 from gmacon/551-shard-key-embedded
Allow shard key to be in an embedded document (#551)
2015-11-20 07:50:52 +02:00
George Macon
899e56e5b8 Add gmacon to AUTHORS 2015-11-19 17:15:43 -05:00
George Macon
f6d3bd8ccb Update changelog for #551 2015-11-19 17:15:27 -05:00
George Macon
deb5677a57 Allow shard key to be in an embedded document (#551) 2015-11-19 17:14:45 -05:00
Omer Katz
5c464c3f5a Bumped version to 0.10.1 2015-11-18 14:07:39 +02:00
Ashley Whetter
cceef33fef Fixed a couple of documentation typos 2015-11-17 14:22:10 +00:00
Paul-Armand Verhaegen
ed8174fe36 Added Paul-Armand Verhaegen to contributor list 2015-11-15 15:32:26 +01:00
Paul-Armand Verhaegen
3c8906494f Added #1156 to changelog 2015-11-15 15:31:22 +01:00
Paul-Armand Verhaegen
6e745e9882 fixed wrong indentation style 2015-11-10 21:13:24 +01:00
Paul-Armand Verhaegen
fb4e9c3772 fix for reloading of strict with special fields 2015-11-10 20:43:49 +01:00
Omer Katz
2c282f9550 Added changelog entry for #1131. 2015-11-08 12:18:12 +02:00
Omer Katz
d92d41cb05 Merge pull request #1131 from noirbizarre/fix-instance-back-references
Fix instance back references
2015-11-08 12:14:37 +02:00
Omer Katz
82e7050561 Merge pull request #1134 from abonhomme/patch-1
docstring correction
2015-11-05 15:49:53 +02:00
abonhomme
44f92d4169 docstring correction
Corrected the docstring for `mongoengine.queryset.base.update_one()`
2015-10-23 11:07:26 -04:00
David Bordeynik
2f1fae38dd Merge pull request #1117 from reallistic/master
Enable ops in queries using elemMatch for EmbeddedDocuments
fixes #1130
2015-10-21 08:40:39 +03:00
Axel Haustant
9fe99979fe Fix tests on Python 2.6 (assertIsNotNone does not exists) 2015-10-19 18:04:15 +02:00
Axel Haustant
6399de0b51 Fix _instance on list of EmbeddedDocuments 2015-10-19 16:39:00 +02:00
Axel Haustant
959740a585 Fix false positive test on _instance 2015-10-19 16:33:40 +02:00
reallistic
159b082828 Recursively create mongo query for embeddeddocument elemMatch 2015-10-18 16:34:24 -07:00
Gang Li
8e7c5af16c Merge remote-tracking branch 'remotes/upstream/master'
Conflicts:
	AUTHORS
	docs/changelog.rst
2015-10-18 01:28:50 -04:00
Gang Li
c1645ab7a7 restored 2015-10-18 01:14:27 -04:00
Gang Li
2ae2bfdde9 updated changelog.rst for #1119 2015-10-18 00:31:40 -04:00
Gang Li
3fe93968a6 update test case for: Please recall fix on: Saving document doesn't create new fields in existing collection #620 #1126 2015-10-18 00:19:36 -04:00
Omer Katz
79a2d715b0 Merge pull request #1121 from larsbutler/simplify-install-deps
Move nose/rednose from install dependencies to test dependencies
2015-10-14 12:45:14 +03:00
Alice Bevan–McGregor
50b271c868 Arbitrary metadata documentation. 2015-10-13 22:51:03 -04:00
Alice Bevan–McGregor
a57f28ac83 Correction for local monkeypatch. 2015-10-13 22:41:58 -04:00
Alice Bevan–McGregor
3f3747a2fe Minor formatting tweaks and additional comments. 2015-10-13 21:59:46 -04:00
Alice Bevan–McGregor
d133913c3d Remove now superfluous special cases.
Removes `verbose_name`, `help_text`, and `custom_data`.  All three are
covered by the one metadata assignment and will continue working as
expected.
2015-10-13 21:59:29 -04:00
Alice Bevan–McGregor
e049cef00a Add arbitrary metadata capture to BaseField.
Includes ability to detect and report conflicts.
2015-10-13 21:54:58 -04:00
Gang Li
eb8176971c Removed "elif field.default" block to avoid silently, inconsistently changing database
This resolved issue Please recall fix on: Saving document doesn't create new fields in existing collection #620 #1126
2015-10-12 23:33:54 -04:00
Gang Li
5bbfca45fa Fixed: ListField minus index assignment does not work #1119
Add code to detect '-1' as a integer.
Normalize negative index to regular list index
Added list assignment test case
2015-10-12 10:34:26 -04:00
Lars Butler
9b500cd867 docs/changelog.rst: fix #1079 to version 0.10.1 - DEV 2015-10-12 10:13:41 +02:00
Lars Butler
b52cae6575 AUTHORS: Add Lars Butler (that's me!) 2015-10-12 10:13:11 +02:00
Lars Butler
35a0142f9b setup.py, tox.ini: move nose/rednose from install deps to test deps
Remove nose/rednose from `setup_requires` and instead declare them in
`tests_require`. Also explicitly add `nose` and `rednose` to
dependencies list in tox.ini (to avoid breaking test runs).

`python setup.py nosetests` is the preferred way for running tests, and
this works, except that placing test deps (nose/rednose) in the
`setup_requires` also means that these dependencies are pulled in for
installs of mongoengine. These deps are not actually be required just
to run mongoengine, so setup.py should not force users to install these
dependencies.

This refactoring should not change any test run semantics.
2015-10-12 10:13:11 +02:00
David Bordeynik
d4f6ef4f1b Merge pull request #1113 from DavidBord/fix-1105
fix-#1105: StrictDict & SemiStrictDict are shadowed at init time
2015-10-11 21:14:05 +03:00
Kirill Kuzminykh
11024deaae Fixed detection of shared connections 2015-10-05 22:40:44 +03:00
Lars Butler
5a038de1d5 fields.ReferenceField: add integer values to reverse_delete_rule docs
When I first tried to use the `reverse_delete_rule` feature of
`ReferenceField`, I had to dig through the source code to find what the
actual integer values were expected to be for DO_NOTHING, NULLIFY,
CASCADE, DENY, and PULL (or at least, where these constants were defined
so that I could import and use them). This patch adds the integer values
for those constants (which are defined in mongoengine.queryset.base) to
the docs so that users can easily choose the correct integer value.

Note: A possible improvement on this change would be to include
`mongoengine.queryset.base` module documentation in the generated docs,
and then update the `ReferenceField` docs to link to the documentation
of these constants (DO_NOTHING, NULLIFY, etc.).
2015-10-05 14:37:03 +02:00
Emmanuel Leblond
903982e896 Merge pull request #1088 from touilleMan/bug-1058
Fix DictField with '_cls' field is converted to Document on access
2015-09-21 12:10:23 +02:00
David Bordeynik
6355c404cc fix-#1105: StrictDict & SemiStrictDict are shadowed at init time 2015-09-16 20:27:52 +03:00
Emmanuel Leblond
92b9cb5d43 Add drop_collection for test_subclass_field_query 2015-09-08 17:35:35 +02:00
Emmanuel Leblond
7580383d26 Add #1050 fix to changelog 2015-09-02 19:00:18 +02:00
Catstyle
ba0934e41e added DynamicTest.test_reload_dynamic_field 2015-09-02 18:42:30 +02:00
Catstyle
a6a1021521 use obj._data instead of self._fields_ordered since DynamicDocument missing some attributes 2015-09-02 18:42:30 +02:00
Emmanuel Leblond
33b4d83c73 Merge pull request #1084 from optik/patch-1
Bad property name for text search index meta
2015-09-02 18:27:10 +02:00
Emmanuel Leblond
6cf630c74a Merge pull request #1096 from vasion/save-condition-for-2.4
save_condition uses "n" instead of "nModified"
2015-09-02 18:23:37 +02:00
Emmanuel Leblond
736fe5b84e Fix unwanted dereference in DictField (issue #1058) 2015-08-30 10:01:24 +02:00
Omer Katz
4241bde6ea Merge pull request #1055 from zxhuang/master
comply to pymongo MongoClient constructor host
2015-08-16 11:52:27 +03:00
Momchil Rogelov
b4ce14d744 use n instead of nModified in save_condition 2015-08-13 10:11:42 +01:00
Momchil Rogelov
10832a2ccc save_condition falls back to "n" if "nModified" is not found to support mongo 2.4 2015-08-12 10:57:20 +01:00
Emmanuel Leblond
91aca44f67 Merge pull request #1093 from touilleMan/bug-1069
Replace disconnect with close method in pymongo
2015-08-10 18:40:59 +02:00
Emmanuel Leblond
96cfbb201a Replace use close method in pymongo 2015-08-04 18:02:57 +02:00
David Bordeynik
b2bc155701 Merge pull request #1087 from marcoskv/patch-1
QuerySet count vs len #937
2015-08-01 11:22:35 +03:00
marcoskv
a70ef5594d QuerySet count vs len #937
Current documentation does not consider performance issues in using len instead of count.
https://github.com/MongoEngine/mongoengine/issues/937
2015-07-26 22:41:24 +02:00
optik
6d991586fd Bad property name for indices description in docs
The correct name for MongoDB index definition property is weights not weight. Using "weight" will cause "Index with name ... already exists with different options"
2015-07-24 15:26:10 +02:00
Emmanuel Leblond
f8890ca841 Merge pull request #1070 from touilleMan/save-condition-error
Use SaveConditionError instead of OperationError in save_condition
2015-07-19 11:18:33 +02:00
Emmanuel Leblond
0752c6b24f Update changelog for #1070 2015-07-19 10:33:54 +02:00
Emmanuel Leblond
3ffaf2c0e1 Correct SaveConditionError involved tests 2015-07-15 11:59:29 +02:00
Emmanuel Leblond
a3e0fbd606 Add SaveConditionError exception 2015-07-15 11:15:40 +02:00
Emmanuel Leblond
9c8ceb6b4e Merge pull request #1060 from touilleMan/GenericReferenceField-choices
Fix GenericReferenceField choices parameter
2015-07-09 11:38:22 +02:00
Emmanuel Leblond
bebce2c053 Clean ununsed variables in iterations 2015-07-09 10:51:04 +02:00
Emmanuel Leblond
34c6790762 Simplify implementation of choices in GenericReferenceField 2015-07-06 10:10:05 +02:00
Emmanuel Leblond
a5fb009b62 Fix GenericReferenceField choices with DBRef and let it possible to set Document choice as string 2015-07-06 02:33:43 +02:00
David Bordeynik
9671ca5ebf Merge pull request #1049 from DavidBord/fix-842
fix-#842: Fix ignored chained options
2015-07-03 08:23:15 +03:00
David Bordeynik
5334ea393e fix-#842: Fix ignored chained options 2015-07-02 23:08:09 +03:00
Zeke Huang
2aaacc02e3 comply to pymongo MongoClient constructor host
Only MongoReplicaSetClient use hosts_or_uri param and it will be deprecated soon.
2015-07-01 12:39:30 -07:00
Matthieu Rigal
222e929b2d Merge pull request #1048 from amitlicht/amitlicht/1047_cached_reference_field_bugfix
Suggested fix for #1047: CachedReferenceField DBRefs bug
2015-07-01 08:53:03 +02:00
amitlicht
6f16d35a92 Adding a changelog line & adding myself to AUTHORS. 2015-06-30 15:08:20 +03:00
amitlicht
d7a2ccf5ac Adding a test case for #1047. 2015-06-30 15:03:06 +03:00
amitlicht
9ce605221a Suggested fix for #1047: CachedReferenceField creates DBRef on to_python, but can't save them on to_mongo.
Dereferencing DBRef to document type before returning it from to_python.
2015-06-28 17:53:20 +03:00
Matthieu Rigal
1e930fe950 Merge branch 'emilecaron-master' 2015-06-26 17:59:25 +02:00
Matthieu Rigal
4dc158589c Moved change to right place and added fancier test 2015-06-26 17:58:53 +02:00
emilecaron
4525eb457b update changelog 2015-06-26 14:23:42 +02:00
emilecaron
56a2e07dc2 always store docs in cascade_refs 2015-06-26 10:45:07 +00:00
emilecaron
9b7fe9ac31 restore broken behavior 2015-06-26 09:31:07 +00:00
emilecaron
c3da07ccf7 Merge branch 'master' of https://github.com/emilecaron/mongoengine 2015-06-26 08:54:12 +00:00
emilecaron
b691a56d51 late set instanciation 2015-06-26 08:52:30 +00:00
Emile Caron
13e0a1b5bb Merge pull request #1 from emilecaron/fix_delete_rule_cascade_cycle
Fix delete rule cascade cycle
2015-06-25 21:53:04 +02:00
emilecaron
646baddce4 fix cascade delete cycle issuue 2015-06-25 18:27:22 +00:00
emilecaron
02f61c323d update test 2015-06-25 18:26:52 +00:00
emilecaron
1e3d2df9e7 fix illogicality 2015-06-25 15:40:12 +00:00
emilecaron
e43fae86f1 reproduce RuntimeError 2015-06-25 15:37:15 +00:00
Matthieu Rigal
c6151e34e0 Bumped version to 0.10.0 2015-06-24 12:39:21 +02:00
Matthieu Rigal
45cb991254 Merge pull request #980 from MRigal/fix/various-fixes
Pep8, code clean-up and 0.10.0 changelog finalisation
2015-06-24 10:20:42 +02:00
Matthieu Rigal
839bc99f94 Updated changelog to prepare 0.10.0 release 2015-06-24 01:16:32 +02:00
Matthieu Rigal
0aeb1ca408 Various fixes again 2015-06-24 00:50:36 +02:00
Matthieu Rigal
cd76a906f4 Set coverage to specific version as 4+ is not Python 3.2 compatible 2015-06-24 00:49:39 +02:00
mrigal
e438491938 typos 2015-06-23 23:16:08 +02:00
mrigal
307b35a5bf some more update, mainly docs 2015-06-23 23:16:08 +02:00
mrigal
217c9720ea added iterkeys method and optimized repr, still very ugly 2015-06-23 23:15:44 +02:00
mrigal
778c7dc5f2 general pep8 and more clean-up 2015-06-23 23:15:44 +02:00
Matthieu Rigal
4c80154437 Merge pull request #1014 from kivistein/fix-705
Allow to add custom metadata to fields
2015-06-23 23:06:05 +02:00
Vicky Donchenko
6bd9529a66 Allow to add custom metadata to fields 2015-06-23 16:25:56 +03:00
Matthieu Rigal
33ea2b4844 Merge pull request #1036 from MRigal/snario-min-distance
Added test, doc to implementation of min_distance query
2015-06-22 18:35:45 +02:00
Matthieu Rigal
5c807f3dc8 Various test adjustments to improve stability independantly of execution order 2015-06-22 16:41:36 +02:00
Matthieu Rigal
9063b559c4 Fix for PyMongo3+ 2015-06-22 16:40:50 +02:00
Matthieu Rigal
40f6df7160 Adapted one more test for MongoDB < 3 2015-06-22 14:57:59 +02:00
Matthieu Rigal
95165aa92f Logic and test adaptations for MongoDB < 3 2015-06-22 14:57:59 +02:00
Matthieu Rigal
d96fcdb35c Fixed problem of ordering when using near_sphere operator 2015-06-22 14:57:58 +02:00
Matthieu Rigal
5efabdcea3 Added tests, documentation and simplified code 2015-06-22 14:57:58 +02:00
Liam Horne
2d57dc0565 Fixed an indentation mistake 2015-06-22 14:57:30 +02:00
lihorne
576629f825 Added support for $minDistance query 2015-06-22 14:57:30 +02:00
Matthieu Rigal
5badb9d151 Merge pull request #1035 from MRigal/fix/882-dynamic-lookup-more-than-two-parts
Simplified lookup_field mechanic and allow dynamic lookup for more than two parts
2015-06-22 14:56:15 +02:00
Matthieu Rigal
45dc379d9a Added to changelog 2015-06-22 14:55:38 +02:00
Matthieu Rigal
49c0c9f44c Simplified lookup-field method, allowing dynamic lookup for more than two parts 2015-06-22 14:55:06 +02:00
Matthieu Rigal
ef5fa4d062 Merge pull request #1037 from MRigal/fix/1008-delete-returns-none-allowed
Added test and fix for delete with write_concern w:0
2015-06-22 14:50:04 +02:00
Matthieu Rigal
35b66d5d94 Merge pull request #1020 from nextoa/master
Improve _created status when switching collection and/or db
2015-06-21 13:32:02 +02:00
Matthieu Rigal
d0b749a43c Made test explicit with an assert 2015-06-21 13:02:59 +02:00
Matthieu Rigal
bcc4d4e8c6 Added test and fix for delete with write_concern w:0 2015-06-21 03:40:45 +02:00
Breeze.kay
41bff0b293 remove testcase:test_signals_with_switch_sharding_db() and fix code style error for pull#1020 2015-06-21 09:32:31 +08:00
Breeze.kay
dfc7f35ef1 add testcase and changelog for pull:#1020 'improve _created status when switch collection and db' 2015-06-19 15:40:05 +08:00
Breeze.kay
0bbbbdde80 Merge remote-tracking branch 'MongoEngine/master' 2015-06-19 11:14:51 +08:00
Matthieu Rigal
5fa5284b58 Merge pull request #1021 from elasticsales/aggregate-sum-and-avg
aggregate_sum/average + unit tests
2015-06-18 23:14:27 +02:00
Stefan Wojcik
b7ef82cb67 style tweaks + changelog entry 2015-06-18 11:02:11 -07:00
Stefan Wojcik
1233780265 make aggregate_sum/average compatible with pymongo 3.x 2015-06-18 11:01:37 -07:00
Stefan Wojcik
dd095279c8 aggregate_sum/average + unit tests 2015-06-18 11:01:37 -07:00
Matthieu Rigal
4d5200c50f Merge pull request #1031 from MRigal/fix/1011-capped-collection-size-multiple-of-256
CappedCollection max_size normalized to multiple of 256
2015-06-15 15:25:33 +02:00
Matthieu Rigal
1bcd675ead Python 3 fix, uses floor division 2015-06-15 13:44:11 +02:00
Matthieu Rigal
2a3d3de0b2 CappedCollection max_size normalized to multiple of 256 2015-06-15 00:22:07 +02:00
Matthieu Rigal
b124836f3a Merge pull request #936 from MRigal/fix/712-avoid-crash-looping-on-corrupted-obj-id
changed ObjectIdField to_python() method to avoid crash, issue 712
2015-06-14 23:31:22 +02:00
Matthieu Rigal
93ba95971b Merge pull request #1029 from MRigal/feature/300-remove-get-or-create
Removed get_or_create() method, deprecated since 0.8
2015-06-13 01:09:29 +02:00
Matthieu Rigal
7b193b3745 Merge pull request #1030 from MongoEngine/improved-doc-sequence-field
Improved doc for SequenceField
2015-06-12 22:17:01 +02:00
Matthieu Rigal
2b647d2405 Improved doc for SequenceField
Related to issue #497
2015-06-12 21:20:59 +02:00
Matthieu Rigal
7714cca599 Removed get_or_create() method, deprecated since 0.8 2015-06-12 20:51:59 +02:00
Matthieu Rigal
42511aa9cf Merge pull request #1028 from MRigal/fix/652-url-field-validation-too-restrictive-use-django-validation
Updated URL and Email regex validators, added schemes to url validator
2015-06-12 20:47:58 +02:00
Matthieu Rigal
ace2a2f3d1 Merge pull request #1027 from MRigal/fix/530-combining-only-and-save-deletes-embedded-fields-value-with-default
Added passing test to prove save and only problem was fixed
2015-06-12 20:40:51 +02:00
Matthieu Rigal
2062fe7a08 Merge pull request #1026 from MRigal/fix/497-sequence-field-with-abstract-classes
SequenceField for abstract classes now have a proper name
2015-06-12 15:17:08 +02:00
Matthieu Rigal
d4c02c3988 Added to changelog 2015-06-12 13:12:35 +02:00
Matthieu Rigal
4c1496b4a4 Updated URL and Email field regex validators, added schemes arg to urlfield 2015-06-12 13:10:36 +02:00
Matthieu Rigal
eec876295d Added passing test to prove save and only problem was fixed 2015-06-12 12:13:28 +02:00
Matthieu Rigal
3093175f54 SequenceField for abstract classes now have a proper name 2015-06-12 11:03:52 +02:00
Matthieu Rigal
dd05c4d34a Merge pull request #1024 from touilleMan/issue-1017
Fix #1017 (document clash between same ids but different collections)
2015-06-12 09:24:32 +02:00
Matthieu Rigal
57e3a40321 Merge pull request #1025 from MRigal/feature/259-improve-error-detection-for-invalid-query
Improve error message for invalid query
2015-06-12 09:13:18 +02:00
Matthieu Rigal
9e70152076 Merge pull request #961 from MRigal/id-meta-foo
Fixes and tests for default 'id' field creation in Document metaclass
2015-06-12 09:13:00 +02:00
Matthieu Rigal
e1da83a8f6 Cosmetic 2015-06-12 09:12:19 +02:00
Matthieu Rigal
8108198613 corrected formatting for Python 2.6 compatibility 2015-06-11 22:48:34 +02:00
Matthieu Rigal
915849b2ce Implemented method to auto-generate non-collisioning auto_id names 2015-06-11 22:48:34 +02:00
mrigal
2e96302336 not in fix 2015-06-11 22:47:10 +02:00
mrigal
051cd744ad added another test to proove we still do not handle all cases well 2015-06-11 22:47:10 +02:00
mrigal
53fbc165ba added content of PR #688 with a test to proove it is a bit right 2015-06-11 22:47:10 +02:00
mrigal
1862bcf867 added test for abstract document without pk creation and adapted behaviour 2015-06-11 22:47:10 +02:00
Omer Katz
8909d1d144 Merge pull request #1005 from touilleMan/master
Raise error if save_condition fails #991
2015-06-11 22:25:17 +03:00
Matthieu Rigal
a2f0f20284 Improve error message for invalid query 2015-06-11 17:48:34 +02:00
Emmanuel Leblond
1951b52aa5 Fix #1017 (document clash between same ids but different collections) 2015-06-11 14:55:04 +02:00
Emmanuel Leblond
cd7a9345ec Add issue related in changelog.rst 2015-06-11 14:45:19 +02:00
Matthieu Rigal
dba4c33c81 Merge pull request #1016 from bigblind/patch-2
Solution for documentation issue #1003
2015-06-11 14:40:41 +02:00
Emmanuel Leblond
153c239c9b Replace assertRaisesRegexp by assertRaises (python2.6 compatibility) 2015-06-11 14:36:51 +02:00
Emmanuel Leblond
4034ab4182 Clean save_condition exception implementation and related tests 2015-06-11 14:30:10 +02:00
Emmanuel Leblond
9c917c3bd3 Update changelog 2015-06-11 14:30:10 +02:00
Emmanuel Leblond
cca0222e1d Update AUTHORS 2015-06-11 14:29:42 +02:00
Emmanuel Leblond
682db9b81f Add versionchanged to document save_condition 2015-06-11 14:29:42 +02:00
Emmanuel Leblond
3e000f9be1 Raise error if save_condition fails #991 2015-06-11 14:29:42 +02:00
Matthieu Rigal
548a552638 Merge pull request #994 from MRigal/fix/cls-index-at-desired-position
Added hashed index, a bit more of geo-indexes, possibility to give _cls
2015-06-11 14:20:01 +02:00
Matthieu Rigal
1d5b5b7d15 Merge pull request #1018 from MRigal/fix/517-no_dereference-not-respected-on-embedded-docs
Respect no_dereference() on embedded docs containing Ref
2015-06-11 14:16:11 +02:00
Frederik Creemers
91aa4586e2 Fixes after code review 2015-06-04 22:38:11 +02:00
Matthieu Rigal
6d3bc43ef6 Merge pull request #1000 from ProgressivePlanning/test_update_related
added passing test for updates on related models
2015-06-04 19:18:11 +02:00
Marcel van den Elst
0f63e26641 use AssertEqual instead of AssertListEqual for py2.6 compatibility 2015-06-04 15:02:32 +02:00
Breeze.kay
ab2ef69c6a improve _created status when switch collection and db 2015-06-03 18:13:54 +08:00
Matthieu Rigal
621350515e Added test was still failing and implemented solution as described in #517 2015-06-03 01:02:19 +02:00
Matthieu Rigal
03ed5c398a Merge pull request #1007 from charanpald/master
GridFS files never deleted with Document deletion
2015-06-02 23:43:57 +02:00
Frederik Creemers
65d6f8c018 Solution for documentation issue #1003
Solution for documentation issue #1003. The explanation about reverse_delete_rule was a bit mixed up.
2015-06-02 12:35:25 +02:00
Charanpal
79d0673ae6 Merge remote-tracking branch 'origin/patch-2' 2015-06-02 10:49:25 +01:00
Charanpal
cbd488e19f Merge remote-tracking branch 'origin/patch-1' 2015-06-02 10:49:15 +01:00
Charanpal Dhanjal
380d869195 Add fix to FileField deletion 2015-06-02 10:23:37 +01:00
Charanpal Dhanjal
73893f2a33 Added charanpald 2015-06-02 10:20:37 +01:00
Charanpal Dhanjal
ad81470d35 Put space after hash 2015-06-02 10:17:17 +01:00
Charanpal Dhanjal
fc140d04ef Fix comment in delete 2015-06-02 10:15:27 +01:00
Matthieu Rigal
a0257ed7e7 Updated test to use new create_index method 2015-06-02 00:14:18 +02:00
Matthieu Rigal
4769487c3b Merge pull request #1012 from elin3t/patch-1
little object name fix in the readme
2015-06-01 23:46:36 +02:00
Matthieu Rigal
29def587ff Merge pull request #1004 from brunopgalvao/patch-1
spelling change definion to definition
2015-06-01 23:27:50 +02:00
Matthieu Rigal
f35d0b2b37 Added create_index method, warnings for drop_dups and a geohaystack test 2015-06-01 23:12:43 +02:00
Matthieu Rigal
283e92d55d Added hashed index, a bit more of geo-indexes, possibility to give _cls and docs 2015-06-01 22:11:21 +02:00
Eliecer Daza
c82b26d334 little object name fix
replace little object name HtmlPost with TextPost that is the one used on the example
2015-05-31 18:28:12 -05:00
Charanpal
2753e02cda Fix for case where Document is deleted and it's files (FieldFields) in GridFS remain. 2015-05-23 14:46:56 +01:00
Bruno Pierri Galvao
fde733c205 spelling change definion to definition 2015-05-21 10:22:02 -04:00
Marcel van den Elst
f730591f2c added passing test for updates on related models
ref #570: test would fail from v0.8.5 up, but fixed in master
2015-05-20 13:01:44 +02:00
David Bordeynik
94eac1e79d Merge pull request #946 from MRigal/fix/pymongo3-connection
fixes #946
2015-05-11 15:51:51 +03:00
Matthew Ellison
9f2b6d0ec6 Merge pull request #894 from MongoEngine/topic/not-in-style-issue
Code Cleanup
- Use not in instead of not (x in y)
2015-05-08 09:06:32 -04:00
Omer Katz
7d7d0ea001 Use not in instead of not (x in y). 2015-05-08 12:50:34 +03:00
Matthieu Rigal
794101691c removed wire_concern usage and cosmetics 2015-05-07 19:34:31 +02:00
Matthew Ellison
a443144a5c Merge pull request #995 from seglberg/PR/952-squash
Unit Test - Unique Multikey Index
2015-05-07 13:12:11 -04:00
Eli Boyarski
73f0867061 Unit Test - Unique Multikey Index
Adds a unit test to exhibit the behavior of MongoDB when using a unique
multikey index. MongoDB treats any missing unique multikey index value
as NULL, thus throwing a Duplicate Key Error when saving multiple
missing values.

See #930 for more information.

- Closes #930
- Closes #952
2015-05-07 11:16:47 -04:00
Matthieu Rigal
f97db93212 corrected test for MongoDB 2.X 2015-05-07 12:48:25 +02:00
Matthieu Rigal
d36708933c author and changelog 2015-05-07 12:48:25 +02:00
Matthieu Rigal
14f82ea0a9 enabled PYMONGO 3 and DEV for travis 2015-05-07 12:47:31 +02:00
Matthieu Rigal
c41dd6495d corrected connection test for PyMongo3+ 2015-05-07 12:47:31 +02:00
Matthieu Rigal
1005c99e9c corrected index test for MongoDB 3+ 2015-05-07 12:47:31 +02:00
Matthieu Rigal
f4478fc762 removed sleep thanks to @seglberg suggestion 2015-05-07 12:47:31 +02:00
mrigal
c5ed308ea5 comments update after having tested PyMongo 3.0.1 2015-05-07 12:47:31 +02:00
mrigal
3ab5ba6149 added explicit warnings when calling methods having no effect anymore with PyMongo3+ 2015-05-07 12:47:30 +02:00
mrigal
9b2fde962c added try except to geo test to catch random mongo internal errors 2015-05-07 12:47:30 +02:00
mrigal
571a7dc42d Fix last issue with binary field as primary key and skipped new test 2015-05-07 12:47:30 +02:00
mrigal
3421fffa9b reactivated unnecessarily skipped test 2015-05-07 12:47:30 +02:00
mrigal
c25619fd63 improved deprecation documentation and added warning when using snapshot with PyMongo3 2015-05-07 12:47:30 +02:00
mrigal
76adb13a64 Minor text and comments enhancements 2015-05-07 12:47:30 +02:00
mrigal
33b1eed361 corrected logical test for not Pymongo3 versions 2015-05-07 12:47:30 +02:00
mrigal
c44891a1a8 changed unittest to call for compatibility with Python 2.6 2015-05-07 12:47:30 +02:00
mrigal
f31f52ff1c corrected test condition, depending on mongodb and not pymongo version 2015-05-07 12:47:30 +02:00
mrigal
6ad9a56bd9 corrected bad import preventing to run on PyMongo 2.X versions 2015-05-07 12:47:30 +02:00
mrigal
a5c2fc4f9d reinforced test for BinaryField being a Primary Key 2015-05-07 12:47:30 +02:00
mrigal
0a65006bb4 replaced find_and_modify by PyMongo3 equivalents 2015-05-07 12:47:30 +02:00
mrigal
3db896c4e2 work-around for pymongo 3 bug 2015-05-07 12:47:30 +02:00
mrigal
e80322021a corrected and enhanced geo_index test 2015-05-07 12:47:29 +02:00
mrigal
48316ba60d implemented global IS_PYMONGO_3 2015-05-07 12:47:29 +02:00
mrigal
c0f1493473 fix revert situated at the wrong location 2015-05-07 12:47:29 +02:00
mrigal
ccbd128fa2 first adaptations after comments and find-outs 2015-05-07 12:47:29 +02:00
mrigal
46817caa68 various unused imports removed (I am allergic) 2015-05-07 12:47:29 +02:00
mrigal
775c8624d4 change to try to address issues due to new save() behaviour, not satisfying, some tests are still failing 2015-05-07 12:47:29 +02:00
mrigal
36eedc987c adapted index test to new explain output in pymongo3 and added comment to a possible pymongo3 bug 2015-05-07 12:47:29 +02:00
mrigal
3b8f31c888 fix problems with cursor arguments 2015-05-07 12:47:29 +02:00
mrigal
a34fa74eaa fix connection problems with pymongo3 and added tests 2015-05-07 12:47:29 +02:00
Matthieu Rigal
d6b2d8dcb5 Merge pull request #982 from elephanter/operator_name_in_field_name
Added __ support to escape field name in fields lookup keywords that match operators names
2015-05-07 11:47:12 +02:00
Eremeev Danil
aab0599280 test moved to another file, cosmetical fixes 2015-05-07 10:55:35 +05:00
Eremeev Danil
dfa8eaf24e Added changeset, updated documentation and tests, changed test condition 2015-05-07 10:55:35 +05:00
Eremeev Danil
63d55cb797 solution for #949 2015-05-07 10:54:16 +05:00
Matthieu Rigal
c642eee0d2 Merge pull request #992 from touilleMan/master
Add primary_key notice in defining-documents doc according to issue #985
2015-05-06 23:13:20 +02:00
Emmanuel Leblond
5f33d298d7 Fix typo in guide/defining-documents.rst 2015-05-06 21:32:36 +02:00
Emmanuel Leblond
fc39fd7519 Update defining-documents.rst
Add primary_key notice according to issue #985
2015-05-06 18:30:49 +02:00
Matthew Ellison
7f442f7485 Merge pull request #978 2015-05-06 09:41:00 -04:00
rma4ok
0ee3203a5a [docs] Adding SortedListField fix to changelog 2015-05-06 09:40:36 -04:00
rma4ok
43a5df8780 [dist] Adding rma4ok to contributors 2015-05-06 09:40:09 -04:00
rma4ok
0949df014b [fix] SortedListField: update whole list if order is changed 2015-05-06 09:40:08 -04:00
Matthew Ellison
01f4dd8f97 Merge pull request #989 2015-05-06 09:38:26 -04:00
Matthew Ellison
8b7599f5d9 Merge pull request #900 2015-05-06 09:37:56 -04:00
Stefan Wojcik
9bdc320cf8 dont send a "cls" option to ensureIndex (related to https://jira.mongodb.org/browse/SERVER-769) 2015-05-06 11:25:45 +02:00
Matthew Ellison
d9c8285806 Merge pull request #988 from olivierlefloch/master
Fix code formatting in upgrade doc
2015-05-05 09:08:43 -04:00
Gregor Kališnik
4b8344082f Testing if we can query embedded document's field inside MapField. Part of #912, which is fixed in 0.9. 2015-05-05 12:49:45 +02:00
Olivier Le Floch
e5cf76b460 Match other code blocks
This fixes rendering on the documentation website:
http://docs.mongoengine.org/upgrade.html
2015-05-04 15:02:01 -07:00
David Bordeynik
422ca87a12 Merge pull request #979 from DavidBord/fix-453
fix-#453: Queryset update doesn't go through field validation
2015-05-02 20:26:56 +03:00
David Bordeynik
a512ccca28 fix-#453: Queryset update doesn't go through field validation 2015-05-02 15:15:02 +03:00
Matthew Ellison
ba215be97c Merge pull request #984 from asmacdo/asmacdo-patch-1
Update docs with NotUniqueError
2015-05-01 07:22:50 -04:00
Austin
ca16050681 Update docs with NotUniqueError
This was changed in https://github.com/MongoEngine/mongoengine/pull/62. It is present in versions > 0.7 http://docs.mongoengine.org/changelog.html#changes-in-0-7-0. I can reopen against an older branch if preferred.
2015-04-30 16:02:50 -04:00
Matthew Ellison
06e4ed1bb4 Merge pull request #976 from seglberg/bugfix/#954-ref-field-subclass
Reflect Inheritance in Field's 'owner_document'
2015-04-30 09:22:15 -04:00
Matthieu Rigal
d4a8ae5743 Merge pull request #932 from elephanter/nested_map_fields_delta_fix
fix wrong _delta results on nested MapFields #931
2015-04-30 10:55:12 +02:00
Eremeev Danil
a4f2f811d3 removed forgotten print 2015-04-30 09:33:19 +05:00
Eremeev Danil
ebaba95eb3 fixed same bug for nested List inside MapField, little code refactoring, added test for nested list and nested reference fields 2015-04-30 09:33:19 +05:00
Eremeev Danil
31f7769199 percent string formatting changed to format method 2015-04-30 09:33:19 +05:00
elephant
7726be94be fixed wrong _delta results on nested MapFields #931 2015-04-30 09:33:18 +05:00
Matthew Ellison
f2cbcea6d7 Unit Tests for #954 Fail on Exception, not Error 2015-04-29 14:26:05 -04:00
Matthew Ellison
5d6a28954b Reflect Inheritance in Field's 'owner_document'
The 'owner_document' property of a Field now reflects the parent field
which first contained the Field when a Document in inherited.

Fixes #954
Closes #955
2015-04-29 14:23:57 -04:00
Sridhar Sundarraman
319f1deceb Unit Test to Demonstrate #954 2015-04-29 14:23:57 -04:00
Omer Katz
3f14958741 Merge pull request #957 from noirbizarre/metastrict
Allow to loads undeclared field with meta attribute (fix #934)
2015-04-29 19:32:26 +03:00
Matthew Ellison
42ba4a5c56 Merge pull request #960 from noirbizarre/tox
Tox support for cross-versions testing
2015-04-28 20:29:23 -04:00
Axel Haustant
c804c395ed Post rebase and Django removal tuning (and prepare for PyMongo 3) 2015-04-28 21:36:07 +02:00
Axel Haustant
58c8cf1a3a Split dependencies installation and test running avoing travis_retry on tests 2015-04-28 20:20:21 +02:00
Axel Haustant
76ea8c86b7 Use travis_retry on tox execution 2015-04-28 20:20:21 +02:00
Axel Haustant
050378fa72 Little README tuning 2015-04-28 20:20:21 +02:00
Axel Haustant
29d858d58c Removed the deprecated py3where parameter 2015-04-28 20:20:21 +02:00
Axel Haustant
dc45920afb Added missing coveralls install 2015-04-28 20:19:56 +02:00
Axel Haustant
15fcb57e2f Fix typo in travis config 2015-04-28 20:19:56 +02:00
Axel Haustant
91ee85152c Tests/Tox/TravisCI improvements 2015-04-28 20:19:56 +02:00
mrigal
aa7bf7af1e adapted setup.cfg to use nosetests standard and allow usage of --tests argument, documenting it in the readme 2015-04-28 20:08:54 +02:00
Axel Haustant
02c1ba39ad Added Django 1.8 to tox 2015-04-28 18:54:10 +02:00
Axel Haustant
8e8d9426df Document about tox testing 2015-04-28 18:54:10 +02:00
Axel Haustant
57f301815d Added a tox.ini file allowing to test with different versions 2015-04-28 18:54:10 +02:00
Matthew Ellison
dfc9dc713c Merge pull request #973 from seglberg/feature/#958-django-split
Removed Django Support from MongoEngine

+1 @thedrow @MRigal @DavidBord @rozza
2015-04-28 10:37:22 -04:00
Matthew Ellison
1a0cad7f5f Updated Django Support Documentation
Added "Call to Arms" for new Django Extension.
2015-04-28 10:02:39 -04:00
Omer Katz
3df436f0d8 Merge pull request #974 from eli-b/spelling
Spelling
2015-04-26 20:15:46 +03:00
Eli Boyarski
d737fca295 Spelling 2015-04-26 17:23:13 +03:00
Omer Katz
da5a3532d7 Merge pull request #967 from RussellLuo/master
Override `authentication_source` by "authSource" in URI
2015-04-25 17:08:14 +03:00
RussellLuo
27111e7b29 Update changelog for added authSource support 2015-04-25 20:57:26 +08:00
RussellLuo
b847bc0aba Make test_connect_uri_with_authsource to focus on the key point 2015-04-25 10:22:24 +08:00
RussellLuo
6eb0bc50e2 Add a test for "authSource" feature 2015-04-25 08:01:24 +08:00
Matthew Ellison
7530f03bf6 Removed Django Support from MongoEngine
Django support has now been split out of MongoEngine and will be
revisted as a new but separate module.

Closes #958
2015-04-24 13:50:26 -04:00
RussellLuo
24a9633edc Override authentication_source by "authSource" in URI 2015-04-20 16:05:34 +08:00
Omer Katz
7e1a5ce445 Merge pull request #911 from jimmyshen/complexdatetime-microsecond-bug
Fixed microsecond-level ordering/filtering bug with ComplexDateTimeField
2015-04-19 12:40:01 +03:00
Jimmy Shen
2ffdbc7fc0 fixed microsecond-level ordering/filtering bug with ComplexDateTimeField as well as unused separator option 2015-04-19 03:26:14 -04:00
Axel Haustant
52c7b68cc3 Restore Py26 compatibility on assertRaises 2015-04-13 22:28:58 +02:00
Axel Haustant
ddbcc8e84b Ensure meta.strict does not bypass constructor check 2015-04-13 18:48:42 +02:00
Axel Haustant
2bfb195ad6 Document FieldDoesNotExist and meta.strict 2015-04-13 18:07:48 +02:00
Axel Haustant
cd2d9517a0 Added 'strict' meta parameter 2015-04-13 17:49:08 +02:00
Omer Katz
19dc312128 Merge pull request #927 from Catstyle/feature/mark_as_changed_issue
mark_as_changed issue
2015-04-10 12:06:00 +03:00
Catstyle
175659628d fix mark_as_changed: handle higher/lower level changed fields correctly to avoid conflict update error 2015-04-10 11:31:31 +08:00
Omer Katz
8fea2b09be Merge pull request #925 from elephanter/fix__get_changed_fields
_get_changed_fields fix for embedded documents with id field.
2015-04-09 21:13:47 +03:00
Eremeev Danil
f77f45b70c _get_changed_fields fix for embedded documents with id field.
removed commented out piece of code

added author and record to changelog
2015-04-09 12:36:48 +05:00
Omer Katz
103a287f11 Merge pull request #941 from MongoEngine/yograterol-patch-1
Remove support to PIL.
2015-04-09 10:19:48 +03:00
Omer Katz
d600ade40c Merge pull request #947 from YoApp/optimization_issue888
Major dereferencing optimizations and fix for de-pickling outdated documents [migrated 921]
2015-04-09 10:06:12 +03:00
Michael Chase
a6a7cba121 Current class fields when unpickling. Fixes #888
Optimize dereferencing map by using sets.
2015-04-08 19:40:43 -07:00
Yohan Graterol
7fff635a3f Remove support to PIL. 2015-04-08 11:29:55 -05:00
mrigal
7a749b88c7 added new test like defined in issue #712 and changed ObjectIdField to_python() method to use a try except similar to other Field classes 2015-04-08 15:38:49 +02:00
Ross Lawley
1ce6a7f4be Update upgrade.rst
Add note to the upgrade guide about 0.8.7 bad package issue.  #919 #929
2015-04-08 10:49:23 +01:00
David Bordeynik
a092910fdd Merge pull request #920 from DavidBord/fix-914
ListField of embedded docs doesn't set the _instance attribute when iterating over it
2015-04-02 15:31:08 +03:00
David Bordeynik
bb77838b3e fix-#914: ListField of embedded docs doesn't set the _instance attribute when iterating over it 2015-04-02 08:59:24 +03:00
David Bordeynik
1001f1bd36 Merge pull request #917 from DavidBord/fix-595
Fix #595: Support += and *= for ListField
2015-04-02 01:08:30 +03:00
David Bordeynik
de0e5583a5 Fix #595: Support += and *= for ListField 2015-03-29 09:28:26 +03:00
72 changed files with 4453 additions and 2756 deletions

View File

@@ -1,67 +1,58 @@
language: python
python:
- '2.6'
- '2.7'
- '3.2'
- '3.3'
- '3.4'
- '3.5'
- pypy
- pypy3
env:
- PYMONGO=2.7.2 DJANGO=dev
- PYMONGO=2.7.2 DJANGO=1.7.1
- PYMONGO=2.7.2 DJANGO=1.6.8
- PYMONGO=2.7.2 DJANGO=1.5.11
- PYMONGO=2.8 DJANGO=dev
- PYMONGO=2.8 DJANGO=1.7.1
- PYMONGO=2.8 DJANGO=1.6.8
- PYMONGO=2.8 DJANGO=1.5.11
- PYMONGO=2.7
- PYMONGO=2.8
- PYMONGO=3.0
- PYMONGO=dev
matrix:
exclude:
- python: '2.6'
env: PYMONGO=2.7.2 DJANGO=dev
- python: '2.6'
env: PYMONGO=2.8 DJANGO=dev
- python: '2.6'
env: PYMONGO=2.7.2 DJANGO=1.7.1
- python: '2.6'
env: PYMONGO=2.8 DJANGO=1.7.1
allow_failures:
- python: pypy3
fast_finish: true
before_install:
- travis_retry sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
- echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' |
sudo tee /etc/apt/sources.list.d/mongodb.list
- travis_retry sudo apt-get update
- travis_retry sudo apt-get install mongodb-org-server
install:
- sudo apt-get install python-dev python3-dev libopenjpeg-dev zlib1g-dev libjpeg-turbo8-dev
libtiff4-dev libjpeg8-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.5-dev tk8.5-dev
python-tk
- if [[ $PYMONGO == 'dev' ]]; then travis_retry pip install https://github.com/mongodb/mongo-python-driver/tarball/master;
true; fi
- if [[ $PYMONGO != 'dev' ]]; then travis_retry pip install pymongo==$PYMONGO; true;
fi
- if [[ $DJANGO == 'dev' ]]; then travis_retry pip install git+https://github.com/django/django.git;
fi
- if [[ $DJANGO != 'dev' ]]; then travis_retry pip install Django==$DJANGO; fi
- travis_retry pip install https://pypi.python.org/packages/source/p/python-dateutil/python-dateutil-2.1.tar.gz#md5=1534bb15cf311f07afaa3aacba1c028b
- travis_retry pip install --upgrade pip
- travis_retry pip install coveralls
- travis_retry python setup.py install
- travis_retry pip install flake8
- travis_retry pip install tox>=1.9
- travis_retry pip install "virtualenv<14.0.0" # virtualenv>=14.0.0 has dropped Python 3.2 support (and pypy3 is based on py32)
- travis_retry tox -e $(echo py$TRAVIS_PYTHON_VERSION-mg$PYMONGO | tr -d . | sed -e 's/pypypy/pypy/') -- -e test
# Run flake8 for py27
before_script:
- if [[ $TRAVIS_PYTHON_VERSION == '2.7' ]]; then tox -e flake8; fi
script:
- travis_retry python setup.py test
- if [[ $TRAVIS_PYTHON_VERSION == '3.'* ]]; then 2to3 . -w; fi;
- coverage run --source=mongoengine setup.py test
- coverage report -m
- python benchmark.py
- tox -e $(echo py$TRAVIS_PYTHON_VERSION-mg$PYMONGO | tr -d . | sed -e 's/pypypy/pypy/') -- --with-coverage
after_script: coveralls --verbose
notifications:
irc: irc.freenode.org#mongoengine
branches:
only:
- master
- /^v.*$/
deploy:
provider: pypi
user: the_drow

30
AUTHORS
View File

@@ -12,7 +12,7 @@ Laine Herron https://github.com/LaineHerron
CONTRIBUTORS
Dervived from the git logs, inevitably incomplete but all of whom and others
Derived from the git logs, inevitably incomplete but all of whom and others
have submitted patches, reported bugs and generally helped make MongoEngine
that much better:
@@ -119,7 +119,7 @@ that much better:
* Anton Kolechkin
* Sergey Nikitin
* psychogenic
* Stefan Wójcik
* Stefan Wójcik (https://github.com/wojcikstefan)
* dimonb
* Garry Polley
* James Slagle
@@ -138,7 +138,6 @@ that much better:
* hellysmile
* Jaepil Jeong
* Daniil Sharou
* Stefan Wójcik
* Pete Campton
* Martyn Smith
* Marcelo Anton
@@ -218,3 +217,28 @@ that much better:
* Matthew Ellison (https://github.com/seglberg)
* Jimmy Shen (https://github.com/jimmyshen)
* J. Fernando Sánchez (https://github.com/balkian)
* Michael Chase (https://github.com/rxsegrxup)
* Eremeev Danil (https://github.com/elephanter)
* Catstyle Lee (https://github.com/Catstyle)
* Kiryl Yermakou (https://github.com/rma4ok)
* Matthieu Rigal (https://github.com/MRigal)
* Charanpal Dhanjal (https://github.com/charanpald)
* Emmanuel Leblond (https://github.com/touilleMan)
* Breeze.Kay (https://github.com/9nix00)
* Vicki Donchenko (https://github.com/kivistein)
* Emile Caron (https://github.com/emilecaron)
* Amit Lichtenberg (https://github.com/amitlicht)
* Gang Li (https://github.com/iici-gli)
* Lars Butler (https://github.com/larsbutler)
* George Macon (https://github.com/gmacon)
* Ashley Whetter (https://github.com/AWhetter)
* Paul-Armand Verhaegen (https://github.com/paularmand)
* Steven Rossiter (https://github.com/BeardedSteve)
* Luo Peng (https://github.com/RussellLuo)
* Bryan Bennett (https://github.com/bbenne10)
* Gilb's Gilb's (https://github.com/gilbsgilbs)
* Joshua Nedrud (https://github.com/Neurostack)
* Shu Shen (https://github.com/shushen)
* xiaost7 (https://github.com/xiaost7)
* Victor Varvaryuk
* Stanislav Kaledin (https://github.com/sallyruthstruik)

View File

@@ -29,7 +29,10 @@ Style Guide
-----------
MongoEngine aims to follow `PEP8 <http://www.python.org/dev/peps/pep-0008/>`_
including 4 space indents and 79 character line limits.
including 4 space indents. When possible we try to stick to 79 character line limits.
However, screens got bigger and an ORM has a strong focus on readability and
if it can help, we accept 119 as maximum line length, in a similar way as
`django does <https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/coding-style/#python-style>`_
Testing
-------
@@ -38,14 +41,21 @@ All tests are run on `Travis <http://travis-ci.org/MongoEngine/mongoengine>`_
and any pull requests are automatically tested by Travis. Any pull requests
without tests will take longer to be integrated and might be refused.
You may also submit a simple failing test as a PullRequest if you don't know
how to fix it, it will be easier for other people to work on it and it may get
fixed faster.
General Guidelines
------------------
- Avoid backward breaking changes if at all possible.
- Write inline documentation for new classes and methods.
- Write tests and make sure they pass (make sure you have a mongod
running on the default port, then execute ``python setup.py test``
running on the default port, then execute ``python setup.py nosetests``
from the cmd line to run the test suite).
- Ensure tests pass on every Python and PyMongo versions.
You can test on these versions locally by executing ``tox``
- Add enhancements or problematic bug fixes to docs/changelog.rst
- Add yourself to AUTHORS :)
Documentation

View File

@@ -6,27 +6,29 @@ MongoEngine
:Author: Harry Marr (http://github.com/hmarr)
:Maintainer: Ross Lawley (http://github.com/rozza)
.. image:: https://secure.travis-ci.org/MongoEngine/mongoengine.png?branch=master
:target: http://travis-ci.org/MongoEngine/mongoengine
.. image:: https://coveralls.io/repos/MongoEngine/mongoengine/badge.png?branch=master
:target: https://coveralls.io/r/MongoEngine/mongoengine?branch=master
.. image:: https://landscape.io/github/MongoEngine/mongoengine/master/landscape.png
:target: https://landscape.io/github/MongoEngine/mongoengine/master
:alt: Code Health
.. image:: https://travis-ci.org/MongoEngine/mongoengine.svg?branch=master
:target: https://travis-ci.org/MongoEngine/mongoengine
.. image:: https://coveralls.io/repos/github/MongoEngine/mongoengine/badge.svg?branch=master
:target: https://coveralls.io/github/MongoEngine/mongoengine?branch=master
.. image:: https://landscape.io/github/MongoEngine/mongoengine/master/landscape.svg?style=flat
:target: https://landscape.io/github/MongoEngine/mongoengine/master
:alt: Code Health
About
=====
MongoEngine is a Python Object-Document Mapper for working with MongoDB.
Documentation available at http://mongoengine-odm.rtfd.org - there is currently
a `tutorial <http://readthedocs.org/docs/mongoengine-odm/en/latest/tutorial.html>`_, a `user guide
<https://mongoengine-odm.readthedocs.org/en/latest/guide/index.html>`_ and an `API reference
<http://readthedocs.org/docs/mongoengine-odm/en/latest/apireference.html>`_.
Documentation available at https://mongoengine-odm.readthedocs.io - there is currently
a `tutorial <https://mongoengine-odm.readthedocs.io/tutorial.html>`_, a `user guide
<https://mongoengine-odm.readthedocs.io/guide/index.html>`_ and an `API reference
<https://mongoengine-odm.readthedocs.io/apireference.html>`_.
Installation
============
If you have `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_
We recommend the use of `virtualenv <https://virtualenv.pypa.io/>`_ and of
`pip <https://pip.pypa.io/>`_. You can then use ``pip install -U mongoengine``.
You may also have `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_ and thus
you can use ``easy_install -U mongoengine``. Otherwise, you can download the
source from `GitHub <http://github.com/MongoEngine/mongoengine>`_ and run ``python
setup.py install``.
@@ -38,21 +40,26 @@ Dependencies
Optional Dependencies
---------------------
- **Django Integration:** Django>=1.4.0 for Python 2.x or PyPy and Django>=1.5.0 for Python 3.x
- **Image Fields**: Pillow>=2.0.0 or PIL (not recommended since MongoEngine is tested with Pillow)
- **Image Fields**: Pillow>=2.0.0
- dateutil>=2.1.0
.. note
MongoEngine always runs it's test suite against the latest patch version of each dependecy. e.g.: Django 1.6.5
MongoEngine always runs it's test suite against the latest patch version of each dependecy. e.g.: PyMongo 3.0.1
Examples
========
Some simple examples of what MongoEngine code looks like::
Some simple examples of what MongoEngine code looks like:
.. code :: python
from mongoengine import *
connect('mydb')
class BlogPost(Document):
title = StringField(required=True, max_length=200)
posted = DateTimeField(default=datetime.datetime.now)
tags = ListField(StringField(max_length=50))
meta = {'allow_inheritance': True}
class TextPost(BlogPost):
content = StringField(required=True)
@@ -82,7 +89,7 @@ Some simple examples of what MongoEngine code looks like::
>>> len(BlogPost.objects)
2
>>> len(HtmlPost.objects)
>>> len(TextPost.objects)
1
>>> len(LinkPost.objects)
1
@@ -96,7 +103,26 @@ Some simple examples of what MongoEngine code looks like::
Tests
=====
To run the test suite, ensure you are running a local instance of MongoDB on
the standard port, and run: ``python setup.py test``.
the standard port and have ``nose`` installed. Then, run: ``python setup.py nosetests``.
To run the test suite on every supported Python version and every supported PyMongo version,
you can use ``tox``.
tox and each supported Python version should be installed in your environment:
.. code-block:: shell
# Install tox
$ pip install tox
# Run the test suites
$ tox
If you wish to run one single or selected tests, use the nosetest convention. It will find the folder,
eventually the file, go to the TestClass specified after the colon and eventually right to the single test.
Also use the -s argument if you want to print out whatever or access pdb while testing.
.. code-block:: shell
$ python setup.py nosetests --tests tests/fields/fields.py:FieldTest.test_cls_field -s
Community
=========

View File

@@ -2,7 +2,7 @@
{% if next or prev %}
<div class="rst-footer-buttons">
{% if next %}
<a href="{{ next.link|e }}" class="btn btn-neutral float-right" title="{{ next.title|striptags|e }}"/>Next <span class="icon icon-circle-arrow-right"></span></a>
<a href="{{ next.link|e }}" class="btn btn-neutral float-right" title="{{ next.title|striptags|e }}">Next <span class="icon icon-circle-arrow-right"></span></a>
{% endif %}
{% if prev %}
<a href="{{ prev.link|e }}" class="btn btn-neutral" title="{{ prev.title|striptags|e }}"><span class="icon icon-circle-arrow-left"></span> Previous</a>

View File

@@ -34,6 +34,9 @@ Documents
.. autoclass:: mongoengine.ValidationError
:members:
.. autoclass:: mongoengine.FieldDoesNotExist
Context Managers
================

View File

@@ -2,9 +2,107 @@
Changelog
=========
Changes in 0.10.8
=================
- Added ability to specify an authentication mechanism (e.g. X.509) #1333
- Added support for falsey primary keys (e.g. doc.pk = 0) #1354
- Fixed BaseQuerySet#sum/average for fields w/ explicit db_field #1417
Changes in 0.9.X - DEV
======================
Changes in 0.10.7
=================
- Dropped Python 3.2 support #1390
- Fixed the bug where dynamic doc has index inside a dict field #1278
- Fixed: ListField minus index assignment does not work #1128
- Fixed cascade delete mixing among collections #1224
- Add `signal_kwargs` argument to `Document.save`, `Document.delete` and `BaseQuerySet.insert` to be passed to signals calls #1206
- Raise `OperationError` when trying to do a `drop_collection` on document with no collection set.
- count on ListField of EmbeddedDocumentField fails. #1187
- Fixed long fields stored as int32 in Python 3. #1253
- MapField now handles unicodes keys correctly. #1267
- ListField now handles negative indicies correctly. #1270
- Fixed AttributeError when initializing EmbeddedDocument with positional args. #681
- Fixed no_cursor_timeout error with pymongo 3.0+ #1304
- Replaced map-reduce based QuerySet.sum/average with aggregation-based implementations #1336
- Fixed support for `__` to escape field names that match operators names in `update` #1351
- Fixed BaseDocument#_mark_as_changed #1369
- Added support for pickling QuerySet instances. #1397
- Fixed connecting to a list of hosts #1389
- Fixed a bug where accessing broken references wouldn't raise a DoesNotExist error #1334
- Fixed not being able to specify use_db_field=False on ListField(EmbeddedDocumentField) instances #1218
- Improvements to the dictionary fields docs #1383
Changes in 0.10.6
=================
- Add support for mocking MongoEngine based on mongomock. #1151
- Fixed not being able to run tests on Windows. #1153
- Allow creation of sparse compound indexes. #1114
- count on ListField of EmbeddedDocumentField fails. #1187
Changes in 0.10.5
=================
- Fix for reloading of strict with special fields. #1156
Changes in 0.10.4
=================
- SaveConditionError is now importable from the top level package. #1165
- upsert_one method added. #1157
Changes in 0.10.3
=================
- Fix `read_preference` (it had chaining issues with PyMongo 2.x and it didn't work at all with PyMongo 3.x) #1042
Changes in 0.10.2
=================
- Allow shard key to point to a field in an embedded document. #551
- Allow arbirary metadata in fields. #1129
- ReferenceFields now support abstract document types. #837
Changes in 0.10.1
=================
- Fix infinite recursion with CASCADE delete rules under specific conditions. #1046
- Fix CachedReferenceField bug when loading cached docs as DBRef but failing to save them. #1047
- Fix ignored chained options #842
- Document save's save_condition error raises `SaveConditionError` exception #1070
- Fix Document.reload for DynamicDocument. #1050
- StrictDict & SemiStrictDict are shadowed at init time. #1105
- Fix ListField minus index assignment does not work. #1119
- Remove code that marks field as changed when the field has default but not existed in database #1126
- Remove test dependencies (nose and rednose) from install dependencies list. #1079
- Recursively build query when using elemMatch operator. #1130
- Fix instance back references for lists of embedded documents. #1131
Changes in 0.10.0
=================
- Django support was removed and will be available as a separate extension. #958
- Allow to load undeclared field with meta attribute 'strict': False #957
- Support for PyMongo 3+ #946
- Removed get_or_create() deprecated since 0.8.0. #300
- Improve Document._created status when switch collection and db #1020
- Queryset update doesn't go through field validation #453
- Added support for specifying authentication source as option `authSource` in URI. #967
- Fixed mark_as_changed to handle higher/lower level fields changed. #927
- ListField of embedded docs doesn't set the _instance attribute when iterating over it #914
- Support += and *= for ListField #595
- Use sets for populating dbrefs to dereference
- Fixed unpickled documents replacing the global field's list. #888
- Fixed storage of microseconds in ComplexDateTimeField and unused separator option. #910
- Don't send a "cls" option to ensureIndex (related to https://jira.mongodb.org/browse/SERVER-769)
- Fix for updating sorting in SortedListField. #978
- Added __ support to escape field name in fields lookup keywords that match operators names #949
- Fix for issue where FileField deletion did not free space in GridFS.
- No_dereference() not respected on embedded docs containing reference. #517
- Document save raise an exception if save_condition fails #1005
- Fixes some internal _id handling issue. #961
- Updated URL and Email Field regex validators, added schemes argument to URLField validation. #652
- Capped collection multiple of 256. #1011
- Added `BaseQuerySet.aggregate_sum` and `BaseQuerySet.aggregate_average` methods.
- Fix for delete with write_concern {'w': 0}. #1008
- Allow dynamic lookup for more than two parts. #882
- Added support for min_distance on geo queries. #831
- Allow to add custom metadata to fields #705
Changes in 0.9.0
================
- Update FileField when creating a new file #714
- Added `EmbeddedDocumentListField` for Lists of Embedded Documents. #826
- ComplexDateTimeField should fall back to None when null=True #864
@@ -78,6 +176,7 @@ Changes in 0.9.X - DEV
- Fixed a few instances where reverse_delete_rule was written as reverse_delete_rules. #791
- Make `in_bulk()` respect `no_dereference()` #775
- Handle None from model __str__; Fixes #753 #754
- _get_changed_fields fix for embedded documents with id field. #925
Changes in 0.8.7
================

View File

@@ -17,6 +17,10 @@ class Post(Document):
tags = ListField(StringField(max_length=30))
comments = ListField(EmbeddedDocumentField(Comment))
# bugfix
meta = {'allow_inheritance': True}
class TextPost(Post):
content = StringField()
@@ -45,7 +49,8 @@ print 'ALL POSTS'
print
for post in Post.objects:
print post.title
print '=' * post.title.count()
#print '=' * post.title.count()
print "=" * 20
if isinstance(post, TextPost):
print post.content

View File

@@ -2,176 +2,18 @@
Django Support
==============
.. note:: Updated to support Django 1.5
Connecting
==========
In your **settings.py** file, ignore the standard database settings (unless you
also plan to use the ORM in your project), and instead call
:func:`~mongoengine.connect` somewhere in the settings module.
.. note::
If you are not using another Database backend you may need to add a dummy
database backend to ``settings.py`` eg::
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.dummy'
}
}
Authentication
==============
MongoEngine includes a Django authentication backend, which uses MongoDB. The
:class:`~mongoengine.django.auth.User` model is a MongoEngine
:class:`~mongoengine.Document`, but implements most of the methods and
attributes that the standard Django :class:`User` model does - so the two are
moderately compatible. Using this backend will allow you to store users in
MongoDB but still use many of the Django authentication infrastructure (such as
the :func:`login_required` decorator and the :func:`authenticate` function). To
enable the MongoEngine auth backend, add the following to your **settings.py**
file::
AUTHENTICATION_BACKENDS = (
'mongoengine.django.auth.MongoEngineBackend',
)
The :mod:`~mongoengine.django.auth` module also contains a
:func:`~mongoengine.django.auth.get_user` helper function, that takes a user's
:attr:`id` and returns a :class:`~mongoengine.django.auth.User` object.
.. versionadded:: 0.1.3
Custom User model
=================
Django 1.5 introduced `Custom user Models
<https://docs.djangoproject.com/en/dev/topics/auth/customizing/#auth-custom-user>`_
which can be used as an alternative to the MongoEngine authentication backend.
The main advantage of this option is that other components relying on
:mod:`django.contrib.auth` and supporting the new swappable user model are more
likely to work. For example, you can use the ``createsuperuser`` management
command as usual.
To enable the custom User model in Django, add ``'mongoengine.django.mongo_auth'``
in your ``INSTALLED_APPS`` and set ``'mongo_auth.MongoUser'`` as the custom user
user model to use. In your **settings.py** file you will have::
INSTALLED_APPS = (
...
'django.contrib.auth',
'mongoengine.django.mongo_auth',
...
)
AUTH_USER_MODEL = 'mongo_auth.MongoUser'
An additional ``MONGOENGINE_USER_DOCUMENT`` setting enables you to replace the
:class:`~mongoengine.django.auth.User` class with another class of your choice::
MONGOENGINE_USER_DOCUMENT = 'mongoengine.django.auth.User'
The custom :class:`User` must be a :class:`~mongoengine.Document` class, but
otherwise has the same requirements as a standard custom user model,
as specified in the `Django Documentation
<https://docs.djangoproject.com/en/dev/topics/auth/customizing/>`_.
In particular, the custom class must define :attr:`USERNAME_FIELD` and
:attr:`REQUIRED_FIELDS` attributes.
Sessions
========
Django allows the use of different backend stores for its sessions. MongoEngine
provides a MongoDB-based session backend for Django, which allows you to use
sessions in your Django application with just MongoDB. To enable the MongoEngine
session backend, ensure that your settings module has
``'django.contrib.sessions.middleware.SessionMiddleware'`` in the
``MIDDLEWARE_CLASSES`` field and ``'django.contrib.sessions'`` in your
``INSTALLED_APPS``. From there, all you need to do is add the following line
into your settings module::
SESSION_ENGINE = 'mongoengine.django.sessions'
SESSION_SERIALIZER = 'mongoengine.django.sessions.BSONSerializer'
Django provides session cookie, which expires after ```SESSION_COOKIE_AGE``` seconds, but doesn't delete cookie at sessions backend, so ``'mongoengine.django.sessions'`` supports `mongodb TTL
<http://docs.mongodb.org/manual/tutorial/expire-data/>`_.
.. note:: ``SESSION_SERIALIZER`` is only necessary in Django 1.6 as the default
serializer is based around JSON and doesn't know how to convert
``bson.objectid.ObjectId`` instances to strings.
.. versionadded:: 0.2.1
Storage
=======
With MongoEngine's support for GridFS via the :class:`~mongoengine.fields.FileField`,
it is useful to have a Django file storage backend that wraps this. The new
storage module is called :class:`~mongoengine.django.storage.GridFSStorage`.
Using it is very similar to using the default FileSystemStorage.::
from mongoengine.django.storage import GridFSStorage
fs = GridFSStorage()
filename = fs.save('hello.txt', 'Hello, World!')
All of the `Django Storage API methods
<http://docs.djangoproject.com/en/dev/ref/files/storage/>`_ have been
implemented except :func:`path`. If the filename provided already exists, an
underscore and a number (before # the file extension, if one exists) will be
appended to the filename until the generated filename doesn't exist. The
:func:`save` method will return the new filename.::
>>> fs.exists('hello.txt')
True
>>> fs.open('hello.txt').read()
'Hello, World!'
>>> fs.size('hello.txt')
13
>>> fs.url('hello.txt')
'http://your_media_url/hello.txt'
>>> fs.open('hello.txt').name
'hello.txt'
>>> fs.listdir()
([], [u'hello.txt'])
All files will be saved and retrieved in GridFS via the :class:`FileDocument`
document, allowing easy access to the files without the GridFSStorage
backend.::
>>> from mongoengine.django.storage import FileDocument
>>> FileDocument.objects()
[<FileDocument: FileDocument object>]
.. versionadded:: 0.4
Shortcuts
=========
Inspired by the `Django shortcut get_object_or_404
<https://docs.djangoproject.com/en/dev/topics/http/shortcuts/#get-object-or-404>`_,
the :func:`~mongoengine.django.shortcuts.get_document_or_404` method returns
a document or raises an Http404 exception if the document does not exist::
from mongoengine.django.shortcuts import get_document_or_404
admin_user = get_document_or_404(User, username='root')
The first argument may be a Document or QuerySet object. All other passed arguments
and keyword arguments are used in the query::
foo_email = get_document_or_404(User.objects.only('email'), username='foo', is_active=True).email
.. note:: Like with :func:`get`, a MultipleObjectsReturned will be raised if more than one
object is found.
.. note:: Django support has been split from the main MongoEngine
repository. The *legacy* Django extension may be found bundled with the
0.9 release of MongoEngine.
Also inspired by the `Django shortcut get_list_or_404
<https://docs.djangoproject.com/en/dev/topics/http/shortcuts/#get-list-or-404>`_,
the :func:`~mongoengine.django.shortcuts.get_list_or_404` method returns a list of
documents or raises an Http404 exception if the list is empty::
from mongoengine.django.shortcuts import get_list_or_404
active_users = get_list_or_404(User, is_active=True)
The first argument may be a Document or QuerySet object. All other passed
arguments and keyword arguments are used to filter the query.
Help Wanted!
------------
The MongoEngine team is looking for help contributing and maintaining a new
Django extension for MongoEngine! If you have Django experience and would like
to help contribute to the project, please get in touch on the
`mailing list <http://groups.google.com/group/mongoengine-users>`_ or by
simply contributing on
`GitHub <https://github.com/MongoEngine/django-mongoengine>`_.

View File

@@ -29,7 +29,7 @@ documents are serialized based on their field order.
Dynamic document schemas
========================
One of the benefits of MongoDb is dynamic schemas for a collection, whilst data
One of the benefits of MongoDB is dynamic schemas for a collection, whilst data
should be planned and organised (after all explicit is better than implicit!)
there are scenarios where having dynamic / expando style documents is desirable.
@@ -75,6 +75,7 @@ are as follows:
* :class:`~mongoengine.fields.DynamicField`
* :class:`~mongoengine.fields.EmailField`
* :class:`~mongoengine.fields.EmbeddedDocumentField`
* :class:`~mongoengine.fields.EmbeddedDocumentListField`
* :class:`~mongoengine.fields.FileField`
* :class:`~mongoengine.fields.FloatField`
* :class:`~mongoengine.fields.GenericEmbeddedDocumentField`
@@ -114,7 +115,7 @@ arguments can be set on all fields:
:attr:`default` (Default: None)
A value to use when no value is set for this field.
The definion of default parameters follow `the general rules on Python
The definition of default parameters follow `the general rules on Python
<http://docs.python.org/reference/compound_stmts.html#function-definitions>`__,
which means that some care should be taken when dealing with default mutable objects
(like in :class:`~mongoengine.fields.ListField` or :class:`~mongoengine.fields.DictField`)::
@@ -146,6 +147,8 @@ arguments can be set on all fields:
When True, use this field as a primary key for the collection. `DictField`
and `EmbeddedDocuments` both support being the primary key for a document.
.. note:: If set, this field is also accessible through the `pk` field.
:attr:`choices` (Default: None)
An iterable (e.g. a list or tuple) of choices to which the value of this
field should be limited.
@@ -170,11 +173,11 @@ arguments can be set on all fields:
class Shirt(Document):
size = StringField(max_length=3, choices=SIZE)
:attr:`help_text` (Default: None)
Optional help text to output with the field -- used by form libraries
:attr:`verbose_name` (Default: None)
Optional human-readable name for the field -- used by form libraries
:attr:`**kwargs` (Optional)
You can supply additional metadata as arbitrary additional keyword
arguments. You can not override existing attributes, however. Common
choices include `help_text` and `verbose_name`, commonly used by form and
widget libraries.
List fields
@@ -211,9 +214,9 @@ document class as the first argument::
Dictionary Fields
-----------------
Often, an embedded document may be used instead of a dictionary -- generally
this is recommended as dictionaries don't support validation or custom field
types. However, sometimes you will not know the structure of what you want to
Often, an embedded document may be used instead of a dictionary generally
embedded documents are recommended as dictionaries dont support validation
or custom field types. However, sometimes you will not know the structure of what you want to
store; in this situation a :class:`~mongoengine.fields.DictField` is appropriate::
class SurveyResponse(Document):
@@ -313,12 +316,12 @@ reference with a delete rule specification. A delete rule is specified by
supplying the :attr:`reverse_delete_rule` attributes on the
:class:`ReferenceField` definition, like this::
class Employee(Document):
class ProfilePage(Document):
...
profile_page = ReferenceField('ProfilePage', reverse_delete_rule=mongoengine.NULLIFY)
employee = ReferenceField('Employee', reverse_delete_rule=mongoengine.CASCADE)
The declaration in this example means that when an :class:`Employee` object is
removed, the :class:`ProfilePage` that belongs to that employee is removed as
removed, the :class:`ProfilePage` that references that employee is removed as
well. If a whole batch of employees is removed, all profile pages that are
linked are removed as well.
@@ -401,7 +404,7 @@ MongoEngine allows you to specify that a field should be unique across a
collection by providing ``unique=True`` to a :class:`~mongoengine.fields.Field`\ 's
constructor. If you try to save a document that has the same value for a unique
field as a document that is already in the database, a
:class:`~mongoengine.OperationError` will be raised. You may also specify
:class:`~mongoengine.NotUniqueError` will be raised. You may also specify
multi-field uniqueness constraints by using :attr:`unique_with`, which may be
either a single field name, or a list or tuple of field names::
@@ -445,8 +448,10 @@ A :class:`~mongoengine.Document` may use a **Capped Collection** by specifying
:attr:`max_documents` and :attr:`max_size` in the :attr:`meta` dictionary.
:attr:`max_documents` is the maximum number of documents that is allowed to be
stored in the collection, and :attr:`max_size` is the maximum size of the
collection in bytes. If :attr:`max_size` is not specified and
:attr:`max_documents` is, :attr:`max_size` defaults to 10000000 bytes (10MB).
collection in bytes. :attr:`max_size` is rounded up to the next multiple of 256
by MongoDB internally and mongoengine before. Use also a multiple of 256 to
avoid confusions. If :attr:`max_size` is not specified and
:attr:`max_documents` is, :attr:`max_size` defaults to 10485760 bytes (10MB).
The following example shows a :class:`Log` document that will be limited to
1000 entries and 2MB of disk space::
@@ -463,19 +468,26 @@ You can specify indexes on collections to make querying faster. This is done
by creating a list of index specifications called :attr:`indexes` in the
:attr:`~mongoengine.Document.meta` dictionary, where an index specification may
either be a single field name, a tuple containing multiple field names, or a
dictionary containing a full index definition. A direction may be specified on
fields by prefixing the field name with a **+** (for ascending) or a **-** sign
(for descending). Note that direction only matters on multi-field indexes.
Text indexes may be specified by prefixing the field name with a **$**. ::
dictionary containing a full index definition.
A direction may be specified on fields by prefixing the field name with a
**+** (for ascending) or a **-** sign (for descending). Note that direction
only matters on multi-field indexes. Text indexes may be specified by prefixing
the field name with a **$**. Hashed indexes may be specified by prefixing
the field name with a **#**::
class Page(Document):
category = IntField()
title = StringField()
rating = StringField()
created = DateTimeField()
meta = {
'indexes': [
'title',
'$title', # text index
'#title', # hashed index
('title', '-rating'),
('category', '_cls'),
{
'fields': ['created'],
'expireAfterSeconds': 3600
@@ -530,11 +542,14 @@ There are a few top level defaults for all indexes that can be set::
:attr:`index_background` (Optional)
Set the default value for if an index should be indexed in the background
:attr:`index_cls` (Optional)
A way to turn off a specific index for _cls.
:attr:`index_drop_dups` (Optional)
Set the default value for if an index should drop duplicates
:attr:`index_cls` (Optional)
A way to turn off a specific index for _cls.
.. note:: Since MongoDB 3.0 drop_dups is not supported anymore. Raises a Warning
and has no effect
Compound Indexes and Indexing sub documents

View File

@@ -13,3 +13,4 @@ User Guide
gridfs
signals
text-indexes
mongomock

21
docs/guide/mongomock.rst Normal file
View File

@@ -0,0 +1,21 @@
==============================
Use mongomock for testing
==============================
`mongomock <https://github.com/vmalloc/mongomock/>`_ is a package to do just
what the name implies, mocking a mongo database.
To use with mongoengine, simply specify mongomock when connecting with
mongoengine:
.. code-block:: python
connect('mongoenginetest', host='mongomock://localhost')
conn = get_connection()
or with an alias:
.. code-block:: python
connect('mongoenginetest', host='mongomock://localhost', alias='testdb')
conn = get_connection('testdb')

View File

@@ -39,6 +39,14 @@ syntax::
# been written by a user whose 'country' field is set to 'uk'
uk_pages = Page.objects(author__country='uk')
.. note::
(version **0.9.1+**) if your field name is like mongodb operator name (for example
type, lte, lt...) and you want to place it at the end of lookup keyword
mongoengine automatically prepend $ to it. To avoid this use __ at the end of
your lookup keyword. For example if your field name is ``type`` and you want to
query by this field you must use ``.objects(user__type__="admin")`` instead of
``.objects(user__type="admin")``
Query operators
===============
@@ -138,9 +146,10 @@ The following were added in MongoEngine 0.8 for
loc.objects(point__near=[40, 5])
loc.objects(point__near={"type": "Point", "coordinates": [40, 5]})
You can also set the maximum distance in meters as well::
You can also set the maximum and/or the minimum distance in meters as well::
loc.objects(point__near=[40, 5], point__max_distance=1000)
loc.objects(point__near=[40, 5], point__min_distance=100)
The older 2D indexes are still supported with the
:class:`~mongoengine.fields.GeoPointField`:
@@ -160,7 +169,8 @@ The older 2D indexes are still supported with the
* ``max_distance`` -- can be added to your location queries to set a maximum
distance.
* ``min_distance`` -- can be added to your location queries to set a minimum
distance.
Querying lists
--------------
@@ -227,7 +237,7 @@ is preferred for achieving this::
# All except for the first 5 people
users = User.objects[5:]
# 5 users, starting from the 10th user found
# 5 users, starting from the 11th user found
users = User.objects[10:15]
You may also index the query to retrieve a single result. If an item at that
@@ -255,21 +265,11 @@ no document matches the query, and
if more than one document matched the query. These exceptions are merged into
your document definitions eg: `MyDoc.DoesNotExist`
A variation of this method exists,
:meth:`~mongoengine.queryset.QuerySet.get_or_create`, that will create a new
document with the query arguments if no documents match the query. An
additional keyword argument, :attr:`defaults` may be provided, which will be
used as default values for the new document, in the case that it should need
to be created::
>>> a, created = User.objects.get_or_create(name='User A', defaults={'age': 30})
>>> b, created = User.objects.get_or_create(name='User A', defaults={'age': 40})
>>> a.name == b.name and a.age == b.age
True
.. warning::
:meth:`~mongoengine.queryset.QuerySet.get_or_create` method is deprecated
since :mod:`mongoengine` 0.8.
A variation of this method, get_or_create() existed, but it was unsafe. It
could not be made safe, because there are no transactions in mongoDB. Other
approaches should be investigated, to ensure you don't accidentally duplicate
data when using something similar to this method. Therefore it was deprecated
in 0.8 and removed in 0.10.
Default Document queries
========================
@@ -347,6 +347,8 @@ way of achieving this::
num_users = len(User.objects)
Even if len() is the Pythonic way of counting results, keep in mind that if you concerned about performance, :meth:`~mongoengine.queryset.QuerySet.count` is the way to go since it only execute a server side count query, while len() retrieves the results, places them in cache, and finally counts them. If we compare the performance of the two operations, len() is much slower than :meth:`~mongoengine.queryset.QuerySet.count`.
Further aggregation
-------------------
You may sum over the values of a specific field on documents using
@@ -598,7 +600,7 @@ Some variables are made available in the scope of the Javascript function:
The following example demonstrates the intended usage of
:meth:`~mongoengine.queryset.QuerySet.exec_js` by defining a function that sums
over a field on a document (this functionality is already available throught
over a field on a document (this functionality is already available through
:meth:`~mongoengine.queryset.QuerySet.sum` but is shown here for sake of
example)::
@@ -663,4 +665,3 @@ following example shows how the substitutions are made::
return comments;
}
""")

View File

@@ -17,7 +17,7 @@ Use the *$* prefix to set a text index, Look the declaration::
meta = {'indexes': [
{'fields': ['$title', "$content"],
'default_language': 'english',
'weight': {'title': 10, 'content': 2}
'weights': {'title': 10, 'content': 2}
}
]}

View File

@@ -2,6 +2,16 @@
Upgrading
#########
0.9.0
*****
The 0.8.7 package on pypi was corrupted. If upgrading from 0.8.7 to 0.9.0 please follow: ::
pip uninstall pymongo
pip uninstall mongoengine
pip install pymongo==2.8
pip install mongoengine
0.8.7
*****

View File

@@ -1,21 +1,20 @@
import document
from document import *
import fields
from fields import *
import connection
from connection import *
import document
from document import *
import errors
from errors import *
import fields
from fields import *
import queryset
from queryset import *
import signals
from signals import *
from errors import *
import errors
import django
__all__ = (list(document.__all__) + fields.__all__ + connection.__all__ +
list(queryset.__all__) + signals.__all__ + list(errors.__all__))
VERSION = (0, 9, 0)
VERSION = (0, 10, 7)
def get_version():
@@ -23,4 +22,5 @@ def get_version():
return '.'.join(map(str, VERSION[:-1])) + VERSION[-1]
return '.'.join(map(str, VERSION))
__version__ = get_version()

View File

@@ -1,6 +1,6 @@
import weakref
import functools
import itertools
import weakref
from mongoengine.common import _import_class
from mongoengine.errors import DoesNotExist, MultipleObjectsReturned
@@ -21,7 +21,7 @@ class BaseDict(dict):
if isinstance(instance, (Document, EmbeddedDocument)):
self._instance = weakref.proxy(instance)
self._name = name
return super(BaseDict, self).__init__(dict_items)
super(BaseDict, self).__init__(dict_items)
def __getitem__(self, key, *args, **kwargs):
value = super(BaseDict, self).__getitem__(key)
@@ -66,7 +66,7 @@ class BaseDict(dict):
def clear(self, *args, **kwargs):
self._mark_as_changed()
return super(BaseDict, self).clear(*args, **kwargs)
return super(BaseDict, self).clear()
def pop(self, *args, **kwargs):
self._mark_as_changed()
@@ -74,7 +74,7 @@ class BaseDict(dict):
def popitem(self, *args, **kwargs):
self._mark_as_changed()
return super(BaseDict, self).popitem(*args, **kwargs)
return super(BaseDict, self).popitem()
def setdefault(self, *args, **kwargs):
self._mark_as_changed()
@@ -125,6 +125,10 @@ class BaseList(list):
value._instance = self._instance
return value
def __iter__(self):
for i in xrange(self.__len__()):
yield self[i]
def __setitem__(self, key, value, *args, **kwargs):
if isinstance(key, slice):
self._mark_as_changed()
@@ -156,6 +160,14 @@ class BaseList(list):
self = state
return self
def __iadd__(self, other):
self._mark_as_changed()
return super(BaseList, self).__iadd__(other)
def __imul__(self, other):
self._mark_as_changed()
return super(BaseList, self).__imul__(other)
def append(self, *args, **kwargs):
self._mark_as_changed()
return super(BaseList, self).append(*args, **kwargs)
@@ -178,7 +190,7 @@ class BaseList(list):
def reverse(self, *args, **kwargs):
self._mark_as_changed()
return super(BaseList, self).reverse(*args, **kwargs)
return super(BaseList, self).reverse()
def sort(self, *args, **kwargs):
self._mark_as_changed()
@@ -187,7 +199,9 @@ class BaseList(list):
def _mark_as_changed(self, key=None):
if hasattr(self._instance, '_mark_as_changed'):
if key:
self._instance._mark_as_changed('%s.%s' % (self._name, key))
self._instance._mark_as_changed(
'%s.%s' % (self._name, key % len(self))
)
else:
self._instance._mark_as_changed(self._name)
@@ -198,7 +212,7 @@ class EmbeddedDocumentList(BaseList):
def __match_all(cls, i, kwargs):
items = kwargs.items()
return all([
getattr(i, k) == v or str(getattr(i, k)) == v for k, v in items
getattr(i, k) == v or unicode(getattr(i, k)) == v for k, v in items
])
@classmethod
@@ -357,25 +371,31 @@ class StrictDict(object):
__slots__ = ()
_special_fields = set(['get', 'pop', 'iteritems', 'items', 'keys', 'create'])
_classes = {}
def __init__(self, **kwargs):
for k,v in kwargs.iteritems():
for k, v in kwargs.iteritems():
setattr(self, k, v)
def __getitem__(self, key):
key = '_reserved_' + key if key in self._special_fields else key
try:
return getattr(self, key)
except AttributeError:
raise KeyError(key)
def __setitem__(self, key, value):
key = '_reserved_' + key if key in self._special_fields else key
return setattr(self, key, value)
def __contains__(self, key):
return hasattr(self, key)
def get(self, key, default=None):
try:
return self[key]
except KeyError:
return default
def pop(self, key, default=None):
v = self.get(key, default)
try:
@@ -383,19 +403,29 @@ class StrictDict(object):
except AttributeError:
pass
return v
def iteritems(self):
for key in self:
yield key, self[key]
def items(self):
return [(k, self[k]) for k in iter(self)]
def iterkeys(self):
return iter(self)
def keys(self):
return list(iter(self))
def __iter__(self):
return (key for key in self.__slots__ if hasattr(self, key))
def __len__(self):
return len(list(self.iteritems()))
def __eq__(self, other):
return self.items() == other.items()
def __neq__(self, other):
return self.items() != other.items()
@@ -406,15 +436,18 @@ class StrictDict(object):
if allowed_keys not in cls._classes:
class SpecificStrictDict(cls):
__slots__ = allowed_keys_tuple
def __repr__(self):
return "{%s}" % ', '.join('"{0!s}": {0!r}'.format(k,v) for (k,v) in self.iteritems())
return "{%s}" % ', '.join('"{0!s}": {1!r}'.format(k, v) for k, v in self.items())
cls._classes[allowed_keys] = SpecificStrictDict
return cls._classes[allowed_keys]
class SemiStrictDict(StrictDict):
__slots__ = ('_extras')
__slots__ = ('_extras', )
_classes = {}
def __getattr__(self, attr):
try:
super(SemiStrictDict, self).__getattr__(attr)
@@ -423,6 +456,7 @@ class SemiStrictDict(StrictDict):
return self.__getattribute__('_extras')[attr]
except KeyError as e:
raise AttributeError(e)
def __setattr__(self, attr, value):
try:
super(SemiStrictDict, self).__setattr__(attr, value)

View File

@@ -1,29 +1,28 @@
import copy
import operator
import numbers
import operator
from collections import Hashable
from functools import partial
import pymongo
from bson import json_util, ObjectId
from bson import ObjectId, json_util
from bson.dbref import DBRef
from bson.son import SON
import pymongo
from mongoengine import signals
from mongoengine.common import _import_class
from mongoengine.errors import (ValidationError, InvalidDocumentError,
LookUpError, FieldDoesNotExist)
from mongoengine.python_support import PY3, txt_type
from mongoengine.base.common import get_document, ALLOW_INHERITANCE
from mongoengine.base.common import ALLOW_INHERITANCE, get_document
from mongoengine.base.datastructures import (
BaseDict,
BaseList,
EmbeddedDocumentList,
StrictDict,
SemiStrictDict
SemiStrictDict,
StrictDict
)
from mongoengine.base.fields import ComplexBaseField
from mongoengine.common import _import_class
from mongoengine.errors import (FieldDoesNotExist, InvalidDocumentError,
LookUpError, ValidationError)
from mongoengine.python_support import PY3, txt_type
__all__ = ('BaseDocument', 'NON_FIELD_ERRORS')
@@ -52,7 +51,7 @@ class BaseDocument(object):
# We only want named arguments.
field = iter(self._fields_ordered)
# If its an automatic id field then skip to the first defined field
if self._auto_id_field:
if getattr(self, '_auto_id_field', False):
next(field)
for value in args:
name = next(field)
@@ -70,15 +69,16 @@ class BaseDocument(object):
signals.pre_init.send(self.__class__, document=self, values=values)
# Check if there are undefined fields supplied, if so raise an
# Exception.
if not self._dynamic:
for var in values.keys():
if var not in self._fields.keys() + ['id', 'pk', '_cls', '_text_score']:
msg = (
"The field '{0}' does not exist on the document '{1}'"
).format(var, self._class_name)
raise FieldDoesNotExist(msg)
# Check if there are undefined fields supplied to the constructor,
# if so raise an Exception.
if not self._dynamic and (self._meta.get('strict', True) or _created):
_undefined_fields = set(values.keys()) - set(
self._fields.keys() + ['id', 'pk', '_cls', '_text_score'])
if _undefined_fields:
msg = (
"The fields '{0}' do not exist on the document '{1}'"
).format(_undefined_fields, self._class_name)
raise FieldDoesNotExist(msg)
if self.STRICT and not self._dynamic:
self._data = StrictDict.create(allowed_keys=self._fields_ordered)()
@@ -86,7 +86,6 @@ class BaseDocument(object):
self._data = SemiStrictDict.create(
allowed_keys=self._fields_ordered)()
self._data = {}
self._dynamic_fields = SON()
# Assign default values to instance
@@ -122,7 +121,7 @@ class BaseDocument(object):
else:
self._data[key] = value
# Set any get_fieldname_display methods
# Set any get_<field>_display methods
self.__set_field_display()
if self._dynamic:
@@ -150,7 +149,6 @@ class BaseDocument(object):
# Handle dynamic data only if an initialised dynamic document
if self._dynamic and not self._dynamic_lock:
field = None
if not hasattr(self, name) and not name.startswith('_'):
DynamicField = _import_class("DynamicField")
field = DynamicField(db_field=name)
@@ -183,8 +181,8 @@ class BaseDocument(object):
except AttributeError:
self__initialised = False
# Check if the user has created a new instance of a class
if (self._is_document and self__initialised
and self__created and name == self._meta['id_field']):
if (self._is_document and self__initialised and
self__created and name == self._meta.get('id_field')):
super(BaseDocument, self).__setattr__('_created', False)
super(BaseDocument, self).__setattr__(name, value)
@@ -206,7 +204,12 @@ class BaseDocument(object):
if k in data:
setattr(self, k, data[k])
if '_fields_ordered' in data:
setattr(type(self), '_fields_ordered', data['_fields_ordered'])
if self._dynamic:
setattr(self, '_fields_ordered', data['_fields_ordered'])
else:
_super_fields_ordered = type(self)._fields_ordered
setattr(self, '_fields_ordered', _super_fields_ordered)
dynamic_fields = data.get('_dynamic_fields') or SON()
for k in dynamic_fields.keys():
setattr(self, k, data["_data"].get(k))
@@ -303,11 +306,11 @@ class BaseDocument(object):
"""
if not fields:
fields = []
data = SON()
data["_id"] = None
data['_cls'] = self._class_name
EmbeddedDocumentField = _import_class("EmbeddedDocumentField")
# only root fields ['test1.a', 'test2'] => ['test1', 'test2']
root_fields = set([f.split('.')[0] for f in fields])
@@ -322,21 +325,20 @@ class BaseDocument(object):
field = self._dynamic_fields.get(field_name)
if value is not None:
f_inputs = field.to_mongo.__code__.co_varnames
ex_vars = {}
if fields and 'fields' in f_inputs:
key = '%s.' % field_name
embedded_fields = [
i.replace(key, '') for i in fields
if i.startswith(key)]
if isinstance(field, (EmbeddedDocumentField)):
if fields:
key = '%s.' % field_name
embedded_fields = [
i.replace(key, '') for i in fields
if i.startswith(key)]
ex_vars['fields'] = embedded_fields
else:
embedded_fields = []
if 'use_db_field' in f_inputs:
ex_vars['use_db_field'] = use_db_field
value = field.to_mongo(value, use_db_field=use_db_field,
fields=embedded_fields)
else:
value = field.to_mongo(value)
value = field.to_mongo(value, **ex_vars)
# Handle self generating fields
if value is None and field._auto_gen:
@@ -412,10 +414,11 @@ class BaseDocument(object):
def to_json(self, *args, **kwargs):
"""Converts a document to JSON.
:param use_db_field: Set to True by default but enables the output of the json structure with the field names and not the mongodb store db_names in case of set to False
:param use_db_field: Set to True by default but enables the output of the json structure with the field names
and not the mongodb store db_names in case of set to False
"""
use_db_field = kwargs.pop('use_db_field', True)
return json_util.dumps(self.to_mongo(use_db_field), *args, **kwargs)
return json_util.dumps(self.to_mongo(use_db_field), *args, **kwargs)
@classmethod
def from_json(cls, json_data, created=False):
@@ -478,7 +481,19 @@ class BaseDocument(object):
key = self._db_field_map.get(key, key)
if key not in self._changed_fields:
self._changed_fields.append(key)
levels, idx = key.split('.'), 1
while idx <= len(levels):
if '.'.join(levels[:idx]) in self._changed_fields:
break
idx += 1
else:
self._changed_fields.append(key)
# remove lower level changed fields
level = '.'.join(levels[:idx]) + '.'
remove = self._changed_fields.remove
for field in self._changed_fields[:]:
if field.startswith(level):
remove(field)
def _clear_changed_fields(self):
"""Using get_changed_fields iterate and remove any fields that are
@@ -530,6 +545,7 @@ class BaseDocument(object):
EmbeddedDocument = _import_class("EmbeddedDocument")
DynamicEmbeddedDocument = _import_class("DynamicEmbeddedDocument")
ReferenceField = _import_class("ReferenceField")
SortedListField = _import_class("SortedListField")
changed_fields = []
changed_fields += getattr(self, '_changed_fields', [])
@@ -548,12 +564,13 @@ class BaseDocument(object):
if hasattr(data, 'id'):
if data.id in inspected:
continue
inspected.add(data.id)
if isinstance(field, ReferenceField):
continue
elif (isinstance(data, (EmbeddedDocument, DynamicEmbeddedDocument))
and db_field_name not in changed_fields):
# Find all embedded fields that have been changed
elif (
isinstance(data, (EmbeddedDocument, DynamicEmbeddedDocument)) and
db_field_name not in changed_fields
):
# Find all embedded fields that have been changed
changed = data._get_changed_fields(inspected)
changed_fields += ["%s%s" % (key, k) for k in changed if k]
elif (isinstance(data, (list, tuple, dict)) and
@@ -561,6 +578,12 @@ class BaseDocument(object):
if (hasattr(field, 'field') and
isinstance(field.field, ReferenceField)):
continue
elif isinstance(field, SortedListField) and field._ordering:
# if ordering is affected whole list is changed
if any(map(lambda d: field._ordering in d._changed_fields, data)):
changed_fields.append(db_field_name)
continue
self._nestable_types_changed_fields(
changed_fields, key, data, inspected)
return changed_fields
@@ -585,7 +608,9 @@ class BaseDocument(object):
for p in parts:
if isinstance(d, (ObjectId, DBRef)):
break
elif isinstance(d, list) and p.isdigit():
elif isinstance(d, list) and p.lstrip('-').isdigit():
if p[0] == '-':
p = str(len(d) + int(p))
try:
d = d[int(p)]
except IndexError:
@@ -598,18 +623,18 @@ class BaseDocument(object):
else:
set_data = doc
if '_id' in set_data:
del(set_data['_id'])
del set_data['_id']
# Determine if any changed items were actually unset.
for path, value in set_data.items():
if value or isinstance(value, (numbers.Number, bool)):
continue
# If we've set a value that ain't the default value dont unset it.
# If we've set a value that ain't the default value don't unset it.
default = None
if (self._dynamic and len(parts) and parts[0] in
self._dynamic_fields):
del(set_data[path])
del set_data[path]
unset_data[path] = 1
continue
elif path in self._fields:
@@ -619,7 +644,9 @@ class BaseDocument(object):
parts = path.split('.')
db_field_name = parts.pop()
for p in parts:
if isinstance(d, list) and p.isdigit():
if isinstance(d, list) and p.lstrip('-').isdigit():
if p[0] == '-':
p = str(len(d) + int(p))
d = d[int(p)]
elif (hasattr(d, '__getattribute__') and
not isinstance(d, dict)):
@@ -643,13 +670,13 @@ class BaseDocument(object):
if default != value:
continue
del(set_data[path])
del set_data[path]
unset_data[path] = 1
return set_data, unset_data
@classmethod
def _get_collection_name(cls):
"""Returns the collection name for this class.
"""Returns the collection name for this class. None for abstract class
"""
return cls._meta.get('collection', None)
@@ -687,14 +714,6 @@ class BaseDocument(object):
del data[field.db_field]
except (AttributeError, ValueError), e:
errors_dict[field_name] = e
elif field.default:
default = field.default
if callable(default):
default = default()
if isinstance(default, BaseDocument):
changed_fields.append(field_name)
elif not only_fields or field_name in only_fields:
changed_fields.append(field_name)
if errors_dict:
errors = "\n".join(["%s - %s" % (k, v)
@@ -758,8 +777,12 @@ class BaseDocument(object):
# Check to see if we need to include _cls
allow_inheritance = cls._meta.get('allow_inheritance',
ALLOW_INHERITANCE)
include_cls = (allow_inheritance and not spec.get('sparse', False) and
spec.get('cls', True))
include_cls = (
allow_inheritance and
not spec.get('sparse', False) and
spec.get('cls', True) and
'_cls' not in spec['fields']
)
# 733: don't include cls if index_cls is False unless there is an explicit cls with the index
include_cls = include_cls and (spec.get('cls', False) or cls._meta.get('index_cls', True))
@@ -772,16 +795,25 @@ class BaseDocument(object):
# ASCENDING from +
# DESCENDING from -
# GEO2D from *
# TEXT from $
# HASHED from #
# GEOSPHERE from (
# GEOHAYSTACK from )
# GEO2D from *
direction = pymongo.ASCENDING
if key.startswith("-"):
direction = pymongo.DESCENDING
elif key.startswith("*"):
direction = pymongo.GEO2D
elif key.startswith("$"):
direction = pymongo.TEXT
if key.startswith(("+", "-", "*", "$")):
elif key.startswith("#"):
direction = pymongo.HASHED
elif key.startswith("("):
direction = pymongo.GEOSPHERE
elif key.startswith(")"):
direction = pymongo.GEOHAYSTACK
elif key.startswith("*"):
direction = pymongo.GEO2D
if key.startswith(("+", "-", "*", "$", "#", "(", ")")):
key = key[1:]
# Use real field name, do it manually because we need field
@@ -789,7 +821,6 @@ class BaseDocument(object):
parts = key.split('.')
if parts in (['pk'], ['id'], ['_id']):
key = '_id'
fields = []
else:
fields = cls._lookup_field(parts)
parts = []
@@ -804,15 +835,12 @@ class BaseDocument(object):
index_list.append((key, direction))
# Don't add cls to a geo index
if include_cls and direction is not pymongo.GEO2D:
if include_cls and direction not in (
pymongo.GEO2D, pymongo.GEOHAYSTACK, pymongo.GEOSPHERE):
index_list.insert(0, ('_cls', 1))
if index_list:
spec['fields'] = index_list
if spec.get('sparse', False) and len(spec['fields']) > 1:
raise ValueError(
'Sparse indexes can only have one field in them. '
'See https://jira.mongodb.org/browse/SERVER-2193')
return spec
@@ -902,6 +930,7 @@ class BaseDocument(object):
"""
ListField = _import_class("ListField")
DynamicField = _import_class('DynamicField')
if not isinstance(parts, (list, tuple)):
parts = [parts]
@@ -911,7 +940,6 @@ class BaseDocument(object):
for field_name in parts:
# Handle ListField indexing:
if field_name.isdigit() and isinstance(field, ListField):
new_field = field.field
fields.append(field_name)
continue
@@ -923,11 +951,9 @@ class BaseDocument(object):
if field_name in cls._fields:
field = cls._fields[field_name]
elif cls._dynamic:
DynamicField = _import_class('DynamicField')
field = DynamicField(db_field=field_name)
elif cls._meta.get("allow_inheritance", False) or cls._meta.get("abstract", False):
# 744: in case the field is defined in a subclass
field = None
for subcls in cls.__subclasses__():
try:
field = subcls._lookup_field([field_name])[0]
@@ -949,21 +975,20 @@ class BaseDocument(object):
'__'.join(parts))
if hasattr(getattr(field, 'field', None), 'lookup_member'):
new_field = field.field.lookup_member(field_name)
else:
# Look up subfield on the previous field
new_field = field.lookup_member(field_name)
if not new_field and isinstance(field, ComplexBaseField):
if hasattr(field.field, 'document_type') and cls._dynamic \
and field.field.document_type._dynamic:
DynamicField = _import_class('DynamicField')
new_field = DynamicField(db_field=field_name)
else:
fields.append(field_name)
continue
elif not new_field and hasattr(field, 'document_type') and cls._dynamic \
and field.document_type._dynamic:
DynamicField = _import_class('DynamicField')
elif cls._dynamic and (isinstance(field, DynamicField) or
getattr(getattr(field, 'document_type', None), '_dynamic', None)):
new_field = DynamicField(db_field=field_name)
else:
# Look up subfield on the previous field or raise
try:
new_field = field.lookup_member(field_name)
except AttributeError:
raise LookUpError('Cannot resolve subfield or operator {} '
'on the field {}'.format(
field_name, field.name))
if not new_field and isinstance(field, ComplexBaseField):
fields.append(field_name)
continue
elif not new_field:
raise LookUpError('Cannot resolve field "%s"'
% field_name)
@@ -980,19 +1005,18 @@ class BaseDocument(object):
return '.'.join(parts)
def __set_field_display(self):
"""Dynamically set the display value for a field with choices"""
for attr_name, field in self._fields.items():
if field.choices:
if self._dynamic:
obj = self
else:
obj = type(self)
setattr(obj,
'get_%s_display' % attr_name,
partial(self.__get_field_display, field=field))
"""For each field that specifies choices, create a
get_<field>_display method.
"""
fields_with_choices = [(n, f) for n, f in self._fields.items()
if f.choices]
for attr_name, field in fields_with_choices:
setattr(self,
'get_%s_display' % attr_name,
partial(self.__get_field_display, field=field))
def __get_field_display(self, field):
"""Returns the display value for a choice field"""
"""Return the display value for a choice field"""
value = getattr(self, field.name)
if field.choices and isinstance(field.choices[0], (list, tuple)):
return dict(field.choices).get(value, value)

View File

@@ -5,20 +5,23 @@ import weakref
from bson import DBRef, ObjectId, SON
import pymongo
from mongoengine.common import _import_class
from mongoengine.errors import ValidationError
from mongoengine.base.common import ALLOW_INHERITANCE
from mongoengine.base.datastructures import (
BaseDict, BaseList, EmbeddedDocumentList
)
from mongoengine.common import _import_class
from mongoengine.errors import ValidationError
__all__ = ("BaseField", "ComplexBaseField",
"ObjectIdField", "GeoJsonBaseField")
class BaseField(object):
UPDATE_OPERATORS = set(['set', 'unset', 'inc', 'dec', 'pop', 'push',
'push_all', 'pull', 'pull_all', 'add_to_set',
'set_on_insert', 'min', 'max'])
class BaseField(object):
"""A base class for fields in a MongoDB document. Instances of this class
may be added to subclasses of `Document` to define a document's schema.
@@ -38,8 +41,8 @@ class BaseField(object):
def __init__(self, db_field=None, name=None, required=False, default=None,
unique=False, unique_with=None, primary_key=False,
validation=None, choices=None, verbose_name=None,
help_text=None, null=False, sparse=False):
validation=None, choices=None, null=False, sparse=False,
**kwargs):
"""
:param db_field: The database field to store this field in
(defaults to the name of the field)
@@ -57,15 +60,15 @@ class BaseField(object):
field. Generally this is deprecated in favour of the
`FIELD.validate` method
:param choices: (optional) The valid choices
:param verbose_name: (optional) The verbose name for the field.
Designed to be human readable and is often used when generating
model forms from the document model.
:param help_text: (optional) The help text for this field and is often
used when generating model forms from the document model.
:param null: (optional) Is the field value can be null. If no and there is a default value
then the default value is set
:param sparse: (optional) `sparse=True` combined with `unique=True` and `required=False`
means that uniqueness won't be enforced for `None` values
:param **kwargs: (optional) Arbitrary indirection-free metadata for
this field can be supplied as additional keyword arguments and
accessed as attributes of the field. Must not conflict with any
existing attributes. Common metadata includes `verbose_name` and
`help_text`.
"""
self.db_field = (db_field or name) if not primary_key else '_id'
@@ -79,10 +82,19 @@ class BaseField(object):
self.primary_key = primary_key
self.validation = validation
self.choices = choices
self.verbose_name = verbose_name
self.help_text = help_text
self.null = null
self.sparse = sparse
self._owner_document = None
# Detect and report conflicts between metadata and base properties.
conflicts = set(dir(self)) & set(kwargs)
if conflicts:
raise TypeError("%s already has attribute(s): %s" % (
self.__class__.__name__, ', '.join(conflicts)))
# Assign metadata to the instance
# This efficient method is available because no __slots__ are defined.
self.__dict__.update(kwargs)
# Adjust the appropriate creation counter, and save our local copy.
if self.db_field == '_id':
@@ -106,7 +118,7 @@ class BaseField(object):
"""Descriptor for assigning a value to a field in a document.
"""
# If setting to None and theres a default
# If setting to None and there is a default
# Then set the value to the default value
if value is None:
if self.null:
@@ -121,7 +133,7 @@ class BaseField(object):
if (self.name not in instance._data or
instance._data[self.name] != value):
instance._mark_as_changed(self.name)
except:
except Exception:
# Values cant be compared eg: naive and tz datetimes
# So mark it as changed
instance._mark_as_changed(self.name)
@@ -129,6 +141,10 @@ class BaseField(object):
EmbeddedDocument = _import_class('EmbeddedDocument')
if isinstance(value, EmbeddedDocument):
value._instance = weakref.proxy(instance)
elif isinstance(value, (list, tuple)):
for v in value:
if isinstance(v, EmbeddedDocument):
v._instance = weakref.proxy(instance)
instance._data[self.name] = value
def error(self, message="", errors=None, field_name=None):
@@ -147,9 +163,24 @@ class BaseField(object):
"""
return self.to_python(value)
def _to_mongo_safe_call(self, value, use_db_field=True, fields=None):
"""A helper method to call to_mongo with proper inputs
"""
f_inputs = self.to_mongo.__code__.co_varnames
ex_vars = {}
if 'fields' in f_inputs:
ex_vars['fields'] = fields
if 'use_db_field' in f_inputs:
ex_vars['use_db_field'] = use_db_field
return self.to_mongo(value, **ex_vars)
def prepare_query_value(self, op, value):
"""Prepare a value that is being used in a query for PyMongo.
"""
if op in UPDATE_OPERATORS:
self.validate(value)
return value
def validate(self, value, clean=True):
@@ -157,26 +188,28 @@ class BaseField(object):
"""
pass
def _validate(self, value, **kwargs):
def _validate_choices(self, value):
Document = _import_class('Document')
EmbeddedDocument = _import_class('EmbeddedDocument')
choice_list = self.choices
if isinstance(choice_list[0], (list, tuple)):
choice_list = [k for k, _ in choice_list]
# Choices which are other types of Documents
if isinstance(value, (Document, EmbeddedDocument)):
if not any(isinstance(value, c) for c in choice_list):
self.error(
'Value must be instance of %s' % unicode(choice_list)
)
# Choices which are types other than Documents
elif value not in choice_list:
self.error('Value must be one of %s' % unicode(choice_list))
def _validate(self, value, **kwargs):
# Check the Choices Constraint
if self.choices:
choice_list = self.choices
if isinstance(self.choices[0], (list, tuple)):
choice_list = [k for k, v in self.choices]
# Choices which are other types of Documents
if isinstance(value, (Document, EmbeddedDocument)):
if not any(isinstance(value, c) for c in choice_list):
self.error(
'Value must be instance of %s' % unicode(choice_list)
)
# Choices which are types other than Documents
elif value not in choice_list:
self.error('Value must be one of %s' % unicode(choice_list))
self._validate_choices(value)
# check validation argument
if self.validation is not None:
@@ -189,9 +222,19 @@ class BaseField(object):
self.validate(value, **kwargs)
@property
def owner_document(self):
return self._owner_document
def _set_owner_document(self, owner_document):
self._owner_document = owner_document
@owner_document.setter
def owner_document(self, owner_document):
self._set_owner_document(owner_document)
class ComplexBaseField(BaseField):
"""Handles complex fields, such as lists / dictionaries.
Allows for nesting of embedded documents inside complex types.
@@ -241,8 +284,8 @@ class ComplexBaseField(BaseField):
instance._data[self.name] = value
if (self._auto_dereference and instance._initialised and
isinstance(value, (BaseList, BaseDict))
and not value._dereferenced):
isinstance(value, (BaseList, BaseDict)) and
not value._dereferenced):
value = _dereference(
value, max_depth=1, instance=instance, name=self.name
)
@@ -254,8 +297,6 @@ class ComplexBaseField(BaseField):
def to_python(self, value):
"""Convert a MongoDB-compatible type to a Python type.
"""
Document = _import_class('Document')
if isinstance(value, basestring):
return value
@@ -271,9 +312,11 @@ class ComplexBaseField(BaseField):
return value
if self.field:
self.field._auto_dereference = self._auto_dereference
value_dict = dict([(key, self.field.to_python(item))
for key, item in value.items()])
else:
Document = _import_class('Document')
value_dict = {}
for k, v in value.items():
if isinstance(v, Document):
@@ -289,11 +332,11 @@ class ComplexBaseField(BaseField):
value_dict[k] = self.to_python(v)
if is_list: # Convert back to a list
return [v for k, v in sorted(value_dict.items(),
return [v for _, v in sorted(value_dict.items(),
key=operator.itemgetter(0))]
return value_dict
def to_mongo(self, value):
def to_mongo(self, value, use_db_field=True, fields=None):
"""Convert a Python type to a MongoDB-compatible type.
"""
Document = _import_class("Document")
@@ -307,9 +350,9 @@ class ComplexBaseField(BaseField):
if isinstance(value, Document):
return GenericReferenceField().to_mongo(value)
cls = value.__class__
val = value.to_mongo()
# If we its a document thats not inherited add _cls
if (isinstance(value, EmbeddedDocument)):
val = value.to_mongo(use_db_field, fields)
# If it's a document that is not inherited add _cls
if isinstance(value, EmbeddedDocument):
val['_cls'] = cls.__name__
return val
@@ -322,7 +365,7 @@ class ComplexBaseField(BaseField):
return value
if self.field:
value_dict = dict([(key, self.field.to_mongo(item))
value_dict = dict([(key, self.field._to_mongo_safe_call(item, use_db_field, fields))
for key, item in value.iteritems()])
else:
value_dict = {}
@@ -347,16 +390,16 @@ class ComplexBaseField(BaseField):
value_dict[k] = DBRef(collection, v.pk)
elif hasattr(v, 'to_mongo'):
cls = v.__class__
val = v.to_mongo()
# If we its a document thats not inherited add _cls
if (isinstance(v, (Document, EmbeddedDocument))):
val = v.to_mongo(use_db_field, fields)
# If it's a document that is not inherited add _cls
if isinstance(v, (Document, EmbeddedDocument)):
val['_cls'] = cls.__name__
value_dict[k] = val
else:
value_dict[k] = self.to_mongo(v)
value_dict[k] = self.to_mongo(v, use_db_field, fields)
if is_list: # Convert back to a list
return [v for k, v in sorted(value_dict.items(),
return [v for _, v in sorted(value_dict.items(),
key=operator.itemgetter(0))]
return value_dict
@@ -398,20 +441,17 @@ class ComplexBaseField(BaseField):
self.field.owner_document = owner_document
self._owner_document = owner_document
def _get_owner_document(self, owner_document):
self._owner_document = owner_document
owner_document = property(_get_owner_document, _set_owner_document)
class ObjectIdField(BaseField):
"""A field wrapper around MongoDB's ObjectIds.
"""
def to_python(self, value):
if not isinstance(value, ObjectId):
value = ObjectId(value)
try:
if not isinstance(value, ObjectId):
value = ObjectId(value)
except Exception:
pass
return value
def to_mongo(self, value):
@@ -429,12 +469,11 @@ class ObjectIdField(BaseField):
def validate(self, value):
try:
ObjectId(unicode(value))
except:
except Exception:
self.error('Invalid Object ID')
class GeoJsonBaseField(BaseField):
"""A geo json field storing a geojson style object.
.. versionadded:: 0.8
@@ -482,7 +521,7 @@ class GeoJsonBaseField(BaseField):
# Quick and dirty validator
try:
value[0][0][0]
except:
except (TypeError, IndexError):
return "Invalid Polygon must contain at least one valid linestring"
errors = []
@@ -506,7 +545,7 @@ class GeoJsonBaseField(BaseField):
# Quick and dirty validator
try:
value[0][0]
except:
except (TypeError, IndexError):
return "Invalid LineString must contain at least one valid point"
errors = []
@@ -537,7 +576,7 @@ class GeoJsonBaseField(BaseField):
# Quick and dirty validator
try:
value[0][0]
except:
except (TypeError, IndexError):
return "Invalid MultiPoint must contain at least one valid point"
errors = []
@@ -556,7 +595,7 @@ class GeoJsonBaseField(BaseField):
# Quick and dirty validator
try:
value[0][0][0]
except:
except (TypeError, IndexError):
return "Invalid MultiLineString must contain at least one valid linestring"
errors = []
@@ -578,7 +617,7 @@ class GeoJsonBaseField(BaseField):
# Quick and dirty validator
try:
value[0][0][0][0]
except:
except (TypeError, IndexError):
return "Invalid MultiPolygon must contain at least one valid Polygon"
errors = []

View File

@@ -1,25 +1,22 @@
import warnings
import pymongo
from mongoengine.base.common import ALLOW_INHERITANCE, _document_registry
from mongoengine.base.fields import BaseField, ComplexBaseField, ObjectIdField
from mongoengine.common import _import_class
from mongoengine.errors import InvalidDocumentError
from mongoengine.python_support import PY3
from mongoengine.queryset import (DO_NOTHING, DoesNotExist,
MultipleObjectsReturned,
QuerySet, QuerySetManager)
QuerySetManager)
from mongoengine.base.common import _document_registry, ALLOW_INHERITANCE
from mongoengine.base.fields import BaseField, ComplexBaseField, ObjectIdField
__all__ = ('DocumentMetaclass', 'TopLevelDocumentMetaclass')
class DocumentMetaclass(type):
"""Metaclass for all documents."""
"""Metaclass for all documents.
"""
# TODO lower complexity of this method
def __new__(cls, name, bases, attrs):
flattened_bases = cls._get_bases(bases)
super_new = super(DocumentMetaclass, cls).__new__
@@ -113,7 +110,7 @@ class DocumentMetaclass(type):
for base in flattened_bases:
if (not getattr(base, '_is_base_cls', True) and
not getattr(base, '_meta', {}).get('abstract', True)):
# Collate heirarchy for _cls and _subclasses
# Collate hierarchy for _cls and _subclasses
class_name.append(base.__name__)
if hasattr(base, '_meta'):
@@ -146,7 +143,7 @@ class DocumentMetaclass(type):
for base in document_bases:
if _cls not in base._subclasses:
base._subclasses += (_cls,)
base._types = base._subclasses # TODO depreciate _types
base._types = base._subclasses # TODO depreciate _types
(Document, EmbeddedDocument, DictField,
CachedReferenceField) = cls._import_classes()
@@ -165,7 +162,7 @@ class DocumentMetaclass(type):
# copies __func__ into im_func and __self__ into im_self for
# classmethod objects in Document derived classes.
if PY3:
for key, val in new_class.__dict__.items():
for val in new_class.__dict__.values():
if isinstance(val, classmethod):
f = val.__get__(new_class)
if hasattr(f, '__func__') and not hasattr(f, 'im_func'):
@@ -176,7 +173,8 @@ class DocumentMetaclass(type):
# Handle delete rules
for field in new_class._fields.itervalues():
f = field
f.owner_document = new_class
if f.owner_document is None:
f.owner_document = new_class
delete_rule = getattr(f, 'reverse_delete_rule', DO_NOTHING)
if isinstance(f, CachedReferenceField):
@@ -185,7 +183,7 @@ class DocumentMetaclass(type):
"CachedReferenceFields is not allowed in EmbeddedDocuments")
if not f.document_type:
raise InvalidDocumentError(
"Document is not avaiable to sync")
"Document is not available to sync")
if f.auto_sync:
f.start_listener()
@@ -247,11 +245,10 @@ class DocumentMetaclass(type):
EmbeddedDocument = _import_class('EmbeddedDocument')
DictField = _import_class('DictField')
CachedReferenceField = _import_class('CachedReferenceField')
return (Document, EmbeddedDocument, DictField, CachedReferenceField)
return Document, EmbeddedDocument, DictField, CachedReferenceField
class TopLevelDocumentMetaclass(DocumentMetaclass):
"""Metaclass for top-level documents (i.e. documents that have their own
collection in the database.
"""
@@ -261,7 +258,7 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
super_new = super(TopLevelDocumentMetaclass, cls).__new__
# Set default _meta data if base class, otherwise get user defined meta
if (attrs.get('my_metaclass') == TopLevelDocumentMetaclass):
if attrs.get('my_metaclass') == TopLevelDocumentMetaclass:
# defaults
attrs['_meta'] = {
'abstract': True,
@@ -280,7 +277,7 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
attrs['_meta'].update(attrs.get('meta', {}))
else:
attrs['_meta'] = attrs.get('meta', {})
# Explictly set abstract to false unless set
# Explicitly set abstract to false unless set
attrs['_meta']['abstract'] = attrs['_meta'].get('abstract', False)
attrs['_is_base_cls'] = False
@@ -295,7 +292,7 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
# Clean up top level meta
if 'meta' in attrs:
del(attrs['meta'])
del attrs['meta']
# Find the parent document class
parent_doc_cls = [b for b in flattened_bases
@@ -304,11 +301,11 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
# Prevent classes setting collection different to their parents
# If parent wasn't an abstract class
if (parent_doc_cls and 'collection' in attrs.get('_meta', {})
and not parent_doc_cls._meta.get('abstract', True)):
if (parent_doc_cls and 'collection' in attrs.get('_meta', {}) and
not parent_doc_cls._meta.get('abstract', True)):
msg = "Trying to set a collection on a subclass (%s)" % name
warnings.warn(msg, SyntaxWarning)
del(attrs['_meta']['collection'])
del attrs['_meta']['collection']
# Ensure abstract documents have abstract bases
if attrs.get('_is_base_cls') or attrs['_meta'].get('abstract'):
@@ -386,15 +383,17 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
new_class._auto_id_field = getattr(parent_doc_cls,
'_auto_id_field', False)
if not new_class._meta.get('id_field'):
# After 0.10, find not existing names, instead of overwriting
id_name, id_db_name = cls.get_auto_id_names(new_class)
new_class._auto_id_field = True
new_class._meta['id_field'] = 'id'
new_class._fields['id'] = ObjectIdField(db_field='_id')
new_class._fields['id'].name = 'id'
new_class.id = new_class._fields['id']
# Prepend id field to _fields_ordered
if 'id' in new_class._fields and 'id' not in new_class._fields_ordered:
new_class._fields_ordered = ('id', ) + new_class._fields_ordered
new_class._meta['id_field'] = id_name
new_class._fields[id_name] = ObjectIdField(db_field=id_db_name)
new_class._fields[id_name].name = id_name
new_class.id = new_class._fields[id_name]
new_class._db_field_map[id_name] = id_db_name
new_class._reverse_db_field_map[id_db_name] = id_name
# Prepend id field to _fields_ordered
new_class._fields_ordered = (id_name, ) + new_class._fields_ordered
# Merge in exceptions with parent hierarchy
exceptions_to_merge = (DoesNotExist, MultipleObjectsReturned)
@@ -409,9 +408,22 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
return new_class
@classmethod
def get_auto_id_names(cls, new_class):
id_name, id_db_name = ('id', '_id')
if id_name not in new_class._fields and \
id_db_name not in (v.db_field for v in new_class._fields.values()):
return id_name, id_db_name
id_basename, id_db_basename, i = 'auto_id', '_auto_id', 0
while id_name in new_class._fields or \
id_db_name in (v.db_field for v in new_class._fields.values()):
id_name = '{0}_{1}'.format(id_basename, i)
id_db_name = '{0}_{1}'.format(id_db_basename, i)
i += 1
return id_name, id_db_name
class MetaDict(dict):
"""Custom dictionary for meta classes.
Handles the merging of set indexes
"""
@@ -426,6 +438,5 @@ class MetaDict(dict):
class BasesTuple(tuple):
"""Special class to handle introspection of bases tuple in __new__"""
pass

View File

@@ -1,6 +1,5 @@
import pymongo
from pymongo import MongoClient, MongoReplicaSetClient, uri_parser
from pymongo import MongoClient, ReadPreference, uri_parser
from mongoengine.python_support import (IS_PYMONGO_3, str_types)
__all__ = ['ConnectionError', 'connect', 'register_connection',
'DEFAULT_CONNECTION_NAME']
@@ -8,6 +7,12 @@ __all__ = ['ConnectionError', 'connect', 'register_connection',
DEFAULT_CONNECTION_NAME = 'default'
if IS_PYMONGO_3:
READ_PREFERENCE = ReadPreference.PRIMARY
else:
from pymongo import MongoReplicaSetClient
READ_PREFERENCE = False
class ConnectionError(Exception):
pass
@@ -19,8 +24,9 @@ _dbs = {}
def register_connection(alias, name=None, host=None, port=None,
read_preference=False,
read_preference=READ_PREFERENCE,
username=None, password=None, authentication_source=None,
authentication_mechanism=None,
**kwargs):
"""Add a connection.
@@ -34,8 +40,14 @@ def register_connection(alias, name=None, host=None, port=None,
:param username: username to authenticate with
:param password: password to authenticate with
:param authentication_source: database to authenticate against
:param authentication_mechanism: database authentication mechanisms.
By default, use SCRAM-SHA-1 with MongoDB 3.0 and later,
MONGODB-CR (MongoDB Challenge Response protocol) for older servers.
:param is_mock: explicitly use mongomock for this connection
(can also be done by using `mongomock://` as db host prefix)
:param kwargs: allow ad-hoc parameters to be passed into the pymongo driver
.. versionchanged:: 0.10.6 - added mongomock support
"""
global _connection_settings
@@ -46,20 +58,42 @@ def register_connection(alias, name=None, host=None, port=None,
'read_preference': read_preference,
'username': username,
'password': password,
'authentication_source': authentication_source
'authentication_source': authentication_source,
'authentication_mechanism': authentication_mechanism
}
# Handle uri style connections
if "://" in conn_settings['host']:
uri_dict = uri_parser.parse_uri(conn_settings['host'])
conn_settings.update({
'name': uri_dict.get('database') or name,
'username': uri_dict.get('username'),
'password': uri_dict.get('password'),
'read_preference': read_preference,
})
if "replicaSet" in conn_settings['host']:
conn_settings['replicaSet'] = True
conn_host = conn_settings['host']
# host can be a list or a string, so if string, force to a list
if isinstance(conn_host, str_types):
conn_host = [conn_host]
resolved_hosts = []
for entity in conn_host:
# Handle uri style connections
if entity.startswith('mongomock://'):
conn_settings['is_mock'] = True
# `mongomock://` is not a valid url prefix and must be replaced by `mongodb://`
resolved_hosts.append(entity.replace('mongomock://', 'mongodb://', 1))
elif '://' in entity:
uri_dict = uri_parser.parse_uri(entity)
resolved_hosts.append(entity)
conn_settings.update({
'name': uri_dict.get('database') or name,
'username': uri_dict.get('username'),
'password': uri_dict.get('password'),
'read_preference': read_preference,
})
uri_options = uri_dict['options']
if 'replicaset' in uri_options:
conn_settings['replicaSet'] = True
if 'authsource' in uri_options:
conn_settings['authentication_source'] = uri_options['authsource']
if 'authmechanism' in uri_options:
conn_settings['authentication_mechanism'] = uri_options['authmechanism']
else:
resolved_hosts.append(entity)
conn_settings['host'] = resolved_hosts
# Deprecated parameters that should not be passed on
kwargs.pop('slaves', None)
@@ -74,7 +108,7 @@ def disconnect(alias=DEFAULT_CONNECTION_NAME):
global _dbs
if alias in _connections:
get_connection(alias=alias).disconnect()
get_connection(alias=alias).close()
del _connections[alias]
if alias in _dbs:
del _dbs[alias]
@@ -98,25 +132,42 @@ def get_connection(alias=DEFAULT_CONNECTION_NAME, reconnect=False):
conn_settings.pop('username', None)
conn_settings.pop('password', None)
conn_settings.pop('authentication_source', None)
conn_settings.pop('authentication_mechanism', None)
is_mock = conn_settings.pop('is_mock', None)
if is_mock:
# Use MongoClient from mongomock
try:
import mongomock
except ImportError:
raise RuntimeError('You need mongomock installed '
'to mock MongoEngine.')
connection_class = mongomock.MongoClient
else:
# Use MongoClient from pymongo
connection_class = MongoClient
connection_class = MongoClient
if 'replicaSet' in conn_settings:
conn_settings['hosts_or_uri'] = conn_settings.pop('host', None)
# Discard port since it can't be used on MongoReplicaSetClient
conn_settings.pop('port', None)
# Discard replicaSet if not base string
if not isinstance(conn_settings['replicaSet'], basestring):
conn_settings.pop('replicaSet', None)
connection_class = MongoReplicaSetClient
if not IS_PYMONGO_3:
connection_class = MongoReplicaSetClient
conn_settings['hosts_or_uri'] = conn_settings.pop('host', None)
try:
connection = None
# check for shared connections
connection_settings_iterator = ((db_alias, settings.copy()) for db_alias, settings in _connection_settings.iteritems())
connection_settings_iterator = (
(db_alias, settings.copy()) for db_alias, settings in _connection_settings.iteritems())
for db_alias, connection_settings in connection_settings_iterator:
connection_settings.pop('name', None)
connection_settings.pop('username', None)
connection_settings.pop('password', None)
connection_settings.pop('authentication_source', None)
connection_settings.pop('authentication_mechanism', None)
if conn_settings == connection_settings and _connections.get(db_alias, None):
connection = _connections[db_alias]
break
@@ -136,11 +187,13 @@ def get_db(alias=DEFAULT_CONNECTION_NAME, reconnect=False):
conn = get_connection(alias)
conn_settings = _connection_settings[alias]
db = conn[conn_settings['name']]
auth_kwargs = {'source': conn_settings['authentication_source']}
if conn_settings['authentication_mechanism'] is not None:
auth_kwargs['mechanism'] = conn_settings['authentication_mechanism']
# Authenticate if necessary
if conn_settings['username'] and conn_settings['password']:
db.authenticate(conn_settings['username'],
conn_settings['password'],
source=conn_settings['authentication_source'])
if conn_settings['username'] and (conn_settings['password'] or
conn_settings['authentication_mechanism'] == 'MONGODB-X509'):
db.authenticate(conn_settings['username'], conn_settings['password'], **auth_kwargs)
_dbs[alias] = db
return _dbs[alias]

View File

@@ -1,17 +1,17 @@
from bson import DBRef, SON
from base import (
from .base import (
BaseDict, BaseList, EmbeddedDocumentList,
TopLevelDocumentMetaclass, get_document
)
from fields import (ReferenceField, ListField, DictField, MapField)
from connection import get_db
from queryset import QuerySet
from document import Document, EmbeddedDocument
from .connection import get_db
from .document import Document, EmbeddedDocument
from .fields import DictField, ListField, MapField, ReferenceField
from .python_support import txt_type
from .queryset import QuerySet
class DeReference(object):
def __call__(self, items, max_depth=1, instance=None, name=None):
"""
Cheaply dereferences the items to a set depth.
@@ -49,8 +49,8 @@ class DeReference(object):
if is_list and all([i.__class__ == doc_type for i in items]):
return items
elif not is_list and all([i.__class__ == doc_type
for i in items.values()]):
elif not is_list and all(
[i.__class__ == doc_type for i in items.values()]):
return items
elif not field.dbref:
if not hasattr(items, 'items'):
@@ -101,25 +101,25 @@ class DeReference(object):
if isinstance(item, (Document, EmbeddedDocument)):
for field_name, field in item._fields.iteritems():
v = item._data.get(field_name, None)
if isinstance(v, (DBRef)):
reference_map.setdefault(field.document_type, []).append(v.id)
if isinstance(v, DBRef):
reference_map.setdefault(field.document_type, set()).add(v.id)
elif isinstance(v, (dict, SON)) and '_ref' in v:
reference_map.setdefault(get_document(v['_cls']), []).append(v['_ref'].id)
reference_map.setdefault(get_document(v['_cls']), set()).add(v['_ref'].id)
elif isinstance(v, (dict, list, tuple)) and depth <= self.max_depth:
field_cls = getattr(getattr(field, 'field', None), 'document_type', None)
references = self._find_references(v, depth)
for key, refs in references.iteritems():
if isinstance(field_cls, (Document, TopLevelDocumentMetaclass)):
key = field_cls
reference_map.setdefault(key, []).extend(refs)
elif isinstance(item, (DBRef)):
reference_map.setdefault(item.collection, []).append(item.id)
reference_map.setdefault(key, set()).update(refs)
elif isinstance(item, DBRef):
reference_map.setdefault(item.collection, set()).add(item.id)
elif isinstance(item, (dict, SON)) and '_ref' in item:
reference_map.setdefault(get_document(item['_cls']), []).append(item['_ref'].id)
reference_map.setdefault(get_document(item['_cls']), set()).add(item['_ref'].id)
elif isinstance(item, (dict, list, tuple)) and depth - 1 <= self.max_depth:
references = self._find_references(item, depth - 1)
for key, refs in references.iteritems():
reference_map.setdefault(key, []).extend(refs)
reference_map.setdefault(key, set()).update(refs)
return reference_map
@@ -128,21 +128,25 @@ class DeReference(object):
"""
object_map = {}
for collection, dbrefs in self.reference_map.iteritems():
keys = object_map.keys()
refs = list(set([dbref for dbref in dbrefs if unicode(dbref).encode('utf-8') not in keys]))
if hasattr(collection, 'objects'): # We have a document class for the refs
col_name = collection._get_collection_name()
refs = [dbref for dbref in dbrefs
if (col_name, dbref) not in object_map]
references = collection.objects.in_bulk(refs)
for key, doc in references.iteritems():
object_map[key] = doc
object_map[(col_name, key)] = doc
else: # Generic reference: use the refs data to convert to document
if isinstance(doc_type, (ListField, DictField, MapField,)):
continue
refs = [dbref for dbref in dbrefs
if (collection, dbref) not in object_map]
if doc_type:
references = doc_type._get_db()[collection].find({'_id': {'$in': refs}})
for ref in references:
doc = doc_type._from_son(ref)
object_map[doc.id] = doc
object_map[(collection, doc.id)] = doc
else:
references = get_db()[collection].find({'_id': {'$in': refs}})
for ref in references:
@@ -151,10 +155,10 @@ class DeReference(object):
elif doc_type is None:
doc = get_document(
''.join(x.capitalize()
for x in collection.split('_')))._from_son(ref)
for x in collection.split('_')))._from_son(ref)
else:
doc = doc_type._from_son(ref)
object_map[doc.id] = doc
object_map[(collection, doc.id)] = doc
return object_map
def _attach_objects(self, items, depth=0, instance=None, name=None):
@@ -180,7 +184,8 @@ class DeReference(object):
if isinstance(items, (dict, SON)):
if '_ref' in items:
return self.object_map.get(items['_ref'].id, items)
return self.object_map.get(
(items['_ref'].collection, items['_ref'].id), items)
elif '_cls' in items:
doc = get_document(items['_cls'])._from_son(items)
_cls = doc._data.pop('_cls', None)
@@ -215,19 +220,20 @@ class DeReference(object):
elif isinstance(v, (Document, EmbeddedDocument)):
for field_name, field in v._fields.iteritems():
v = data[k]._data.get(field_name, None)
if isinstance(v, (DBRef)):
data[k]._data[field_name] = self.object_map.get(v.id, v)
if isinstance(v, DBRef):
data[k]._data[field_name] = self.object_map.get(
(v.collection, v.id), v)
elif isinstance(v, (dict, SON)) and '_ref' in v:
data[k]._data[field_name] = self.object_map.get(v['_ref'].id, v)
elif isinstance(v, dict) and depth <= self.max_depth:
data[k]._data[field_name] = self._attach_objects(v, depth, instance=instance, name=name)
elif isinstance(v, (list, tuple)) and depth <= self.max_depth:
data[k]._data[field_name] = self._attach_objects(v, depth, instance=instance, name=name)
data[k]._data[field_name] = self.object_map.get(
(v['_ref'].collection, v['_ref'].id), v)
elif isinstance(v, (dict, list, tuple)) and depth <= self.max_depth:
item_name = txt_type("{0}.{1}.{2}").format(name, k, field_name)
data[k]._data[field_name] = self._attach_objects(v, depth, instance=instance, name=item_name)
elif isinstance(v, (dict, list, tuple)) and depth <= self.max_depth:
item_name = '%s.%s' % (name, k) if name else name
data[k] = self._attach_objects(v, depth - 1, instance=instance, name=item_name)
elif hasattr(v, 'id'):
data[k] = self.object_map.get(v.id, v)
data[k] = self.object_map.get((v.collection, v.id), v)
if instance and name:
if is_list:

View File

@@ -1,412 +0,0 @@
from mongoengine import *
from django.utils.encoding import smart_str
from django.contrib.auth.models import _user_has_perm, _user_get_all_permissions, _user_has_module_perms
from django.db import models
from django.contrib.contenttypes.models import ContentTypeManager
from django.contrib import auth
from django.contrib.auth.models import AnonymousUser
from django.utils.translation import ugettext_lazy as _
from .utils import datetime_now
REDIRECT_FIELD_NAME = 'next'
try:
from django.contrib.auth.hashers import check_password, make_password
except ImportError:
"""Handle older versions of Django"""
from django.utils.hashcompat import md5_constructor, sha_constructor
def get_hexdigest(algorithm, salt, raw_password):
raw_password, salt = smart_str(raw_password), smart_str(salt)
if algorithm == 'md5':
return md5_constructor(salt + raw_password).hexdigest()
elif algorithm == 'sha1':
return sha_constructor(salt + raw_password).hexdigest()
raise ValueError('Got unknown password algorithm type in password')
def check_password(raw_password, password):
algo, salt, hash = password.split('$')
return hash == get_hexdigest(algo, salt, raw_password)
def make_password(raw_password):
from random import random
algo = 'sha1'
salt = get_hexdigest(algo, str(random()), str(random()))[:5]
hash = get_hexdigest(algo, salt, raw_password)
return '%s$%s$%s' % (algo, salt, hash)
class ContentType(Document):
name = StringField(max_length=100)
app_label = StringField(max_length=100)
model = StringField(max_length=100, verbose_name=_('python model class name'),
unique_with='app_label')
objects = ContentTypeManager()
class Meta:
verbose_name = _('content type')
verbose_name_plural = _('content types')
# db_table = 'django_content_type'
# ordering = ('name',)
# unique_together = (('app_label', 'model'),)
def __unicode__(self):
return self.name
def model_class(self):
"Returns the Python model class for this type of content."
from django.db import models
return models.get_model(self.app_label, self.model)
def get_object_for_this_type(self, **kwargs):
"""
Returns an object of this type for the keyword arguments given.
Basically, this is a proxy around this object_type's get_object() model
method. The ObjectNotExist exception, if thrown, will not be caught,
so code that calls this method should catch it.
"""
return self.model_class()._default_manager.using(self._state.db).get(**kwargs)
def natural_key(self):
return (self.app_label, self.model)
class SiteProfileNotAvailable(Exception):
pass
class PermissionManager(models.Manager):
def get_by_natural_key(self, codename, app_label, model):
return self.get(
codename=codename,
content_type=ContentType.objects.get_by_natural_key(app_label, model)
)
class Permission(Document):
"""The permissions system provides a way to assign permissions to specific
users and groups of users.
The permission system is used by the Django admin site, but may also be
useful in your own code. The Django admin site uses permissions as follows:
- The "add" permission limits the user's ability to view the "add"
form and add an object.
- The "change" permission limits a user's ability to view the change
list, view the "change" form and change an object.
- The "delete" permission limits the ability to delete an object.
Permissions are set globally per type of object, not per specific object
instance. It is possible to say "Mary may change news stories," but it's
not currently possible to say "Mary may change news stories, but only the
ones she created herself" or "Mary may only change news stories that have
a certain status or publication date."
Three basic permissions -- add, change and delete -- are automatically
created for each Django model.
"""
name = StringField(max_length=50, verbose_name=_('username'))
content_type = ReferenceField(ContentType)
codename = StringField(max_length=100, verbose_name=_('codename'))
# FIXME: don't access field of the other class
# unique_with=['content_type__app_label', 'content_type__model'])
objects = PermissionManager()
class Meta:
verbose_name = _('permission')
verbose_name_plural = _('permissions')
# unique_together = (('content_type', 'codename'),)
# ordering = ('content_type__app_label', 'content_type__model', 'codename')
def __unicode__(self):
return u"%s | %s | %s" % (
unicode(self.content_type.app_label),
unicode(self.content_type),
unicode(self.name))
def natural_key(self):
return (self.codename,) + self.content_type.natural_key()
natural_key.dependencies = ['contenttypes.contenttype']
class Group(Document):
"""Groups are a generic way of categorizing users to apply permissions,
or some other label, to those users. A user can belong to any number of
groups.
A user in a group automatically has all the permissions granted to that
group. For example, if the group Site editors has the permission
can_edit_home_page, any user in that group will have that permission.
Beyond permissions, groups are a convenient way to categorize users to
apply some label, or extended functionality, to them. For example, you
could create a group 'Special users', and you could write code that would
do special things to those users -- such as giving them access to a
members-only portion of your site, or sending them members-only
e-mail messages.
"""
name = StringField(max_length=80, unique=True, verbose_name=_('name'))
permissions = ListField(ReferenceField(Permission, verbose_name=_('permissions'), required=False))
class Meta:
verbose_name = _('group')
verbose_name_plural = _('groups')
def __unicode__(self):
return self.name
class UserManager(models.Manager):
def create_user(self, username, email, password=None):
"""
Creates and saves a User with the given username, e-mail and password.
"""
now = datetime_now()
# Normalize the address by lowercasing the domain part of the email
# address.
try:
email_name, domain_part = email.strip().split('@', 1)
except ValueError:
pass
else:
email = '@'.join([email_name, domain_part.lower()])
user = self.model(username=username, email=email, is_staff=False,
is_active=True, is_superuser=False, last_login=now,
date_joined=now)
user.set_password(password)
user.save(using=self._db)
return user
def create_superuser(self, username, email, password):
u = self.create_user(username, email, password)
u.is_staff = True
u.is_active = True
u.is_superuser = True
u.save(using=self._db)
return u
def make_random_password(self, length=10, allowed_chars='abcdefghjkmnpqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ23456789'):
"Generates a random password with the given length and given allowed_chars"
# Note that default value of allowed_chars does not have "I" or letters
# that look like it -- just to avoid confusion.
from random import choice
return ''.join([choice(allowed_chars) for i in range(length)])
class User(Document):
"""A User document that aims to mirror most of the API specified by Django
at http://docs.djangoproject.com/en/dev/topics/auth/#users
"""
username = StringField(max_length=30, required=True,
verbose_name=_('username'),
help_text=_("Required. 30 characters or fewer. Letters, numbers and @/./+/-/_ characters"))
first_name = StringField(max_length=30,
verbose_name=_('first name'))
last_name = StringField(max_length=30,
verbose_name=_('last name'))
email = EmailField(verbose_name=_('e-mail address'))
password = StringField(max_length=128,
verbose_name=_('password'),
help_text=_("Use '[algo]$[iterations]$[salt]$[hexdigest]' or use the <a href=\"password/\">change password form</a>."))
is_staff = BooleanField(default=False,
verbose_name=_('staff status'),
help_text=_("Designates whether the user can log into this admin site."))
is_active = BooleanField(default=True,
verbose_name=_('active'),
help_text=_("Designates whether this user should be treated as active. Unselect this instead of deleting accounts."))
is_superuser = BooleanField(default=False,
verbose_name=_('superuser status'),
help_text=_("Designates that this user has all permissions without explicitly assigning them."))
last_login = DateTimeField(default=datetime_now,
verbose_name=_('last login'))
date_joined = DateTimeField(default=datetime_now,
verbose_name=_('date joined'))
user_permissions = ListField(ReferenceField(Permission), verbose_name=_('user permissions'),
help_text=_('Permissions for the user.'))
USERNAME_FIELD = 'username'
REQUIRED_FIELDS = ['email']
meta = {
'allow_inheritance': True,
'indexes': [
{'fields': ['username'], 'unique': True, 'sparse': True}
]
}
def __unicode__(self):
return self.username
def get_full_name(self):
"""Returns the users first and last names, separated by a space.
"""
full_name = u'%s %s' % (self.first_name or '', self.last_name or '')
return full_name.strip()
def is_anonymous(self):
return False
def is_authenticated(self):
return True
def set_password(self, raw_password):
"""Sets the user's password - always use this rather than directly
assigning to :attr:`~mongoengine.django.auth.User.password` as the
password is hashed before storage.
"""
self.password = make_password(raw_password)
self.save()
return self
def check_password(self, raw_password):
"""Checks the user's password against a provided password - always use
this rather than directly comparing to
:attr:`~mongoengine.django.auth.User.password` as the password is
hashed before storage.
"""
return check_password(raw_password, self.password)
@classmethod
def create_user(cls, username, password, email=None):
"""Create (and save) a new user with the given username, password and
email address.
"""
now = datetime_now()
# Normalize the address by lowercasing the domain part of the email
# address.
if email is not None:
try:
email_name, domain_part = email.strip().split('@', 1)
except ValueError:
pass
else:
email = '@'.join([email_name, domain_part.lower()])
user = cls(username=username, email=email, date_joined=now)
user.set_password(password)
user.save()
return user
def get_group_permissions(self, obj=None):
"""
Returns a list of permission strings that this user has through his/her
groups. This method queries all available auth backends. If an object
is passed in, only permissions matching this object are returned.
"""
permissions = set()
for backend in auth.get_backends():
if hasattr(backend, "get_group_permissions"):
permissions.update(backend.get_group_permissions(self, obj))
return permissions
def get_all_permissions(self, obj=None):
return _user_get_all_permissions(self, obj)
def has_perm(self, perm, obj=None):
"""
Returns True if the user has the specified permission. This method
queries all available auth backends, but returns immediately if any
backend returns True. Thus, a user who has permission from a single
auth backend is assumed to have permission in general. If an object is
provided, permissions for this specific object are checked.
"""
# Active superusers have all permissions.
if self.is_active and self.is_superuser:
return True
# Otherwise we need to check the backends.
return _user_has_perm(self, perm, obj)
def has_module_perms(self, app_label):
"""
Returns True if the user has any permissions in the given app label.
Uses pretty much the same logic as has_perm, above.
"""
# Active superusers have all permissions.
if self.is_active and self.is_superuser:
return True
return _user_has_module_perms(self, app_label)
def email_user(self, subject, message, from_email=None):
"Sends an e-mail to this User."
from django.core.mail import send_mail
send_mail(subject, message, from_email, [self.email])
def get_profile(self):
"""
Returns site-specific profile for this user. Raises
SiteProfileNotAvailable if this site does not allow profiles.
"""
if not hasattr(self, '_profile_cache'):
from django.conf import settings
if not getattr(settings, 'AUTH_PROFILE_MODULE', False):
raise SiteProfileNotAvailable('You need to set AUTH_PROFILE_MO'
'DULE in your project settings')
try:
app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')
except ValueError:
raise SiteProfileNotAvailable('app_label and model_name should'
' be separated by a dot in the AUTH_PROFILE_MODULE set'
'ting')
try:
model = models.get_model(app_label, model_name)
if model is None:
raise SiteProfileNotAvailable('Unable to load the profile '
'model, check AUTH_PROFILE_MODULE in your project sett'
'ings')
self._profile_cache = model._default_manager.using(self._state.db).get(user__id__exact=self.id)
self._profile_cache.user = self
except (ImportError, ImproperlyConfigured):
raise SiteProfileNotAvailable
return self._profile_cache
class MongoEngineBackend(object):
"""Authenticate using MongoEngine and mongoengine.django.auth.User.
"""
supports_object_permissions = False
supports_anonymous_user = False
supports_inactive_user = False
_user_doc = False
def authenticate(self, username=None, password=None):
user = self.user_document.objects(username=username).first()
if user:
if password and user.check_password(password):
backend = auth.get_backends()[0]
user.backend = "%s.%s" % (backend.__module__, backend.__class__.__name__)
return user
return None
def get_user(self, user_id):
return self.user_document.objects.with_id(user_id)
@property
def user_document(self):
if self._user_doc is False:
from .mongo_auth.models import get_user_document
self._user_doc = get_user_document()
return self._user_doc
def get_user(userid):
"""Returns a User object from an id (User.id). Django's equivalent takes
request, but taking an id instead leaves it up to the developer to store
the id in any way they want (session, signed cookie, etc.)
"""
if not userid:
return AnonymousUser()
return MongoEngineBackend().get_user(userid) or AnonymousUser()

View File

@@ -1,119 +0,0 @@
from django.conf import settings
from django.contrib.auth.hashers import make_password
from django.contrib.auth.models import UserManager
from django.core.exceptions import ImproperlyConfigured
from django.db import models
try:
from django.utils.module_loading import import_module
except ImportError:
"""Handle older versions of Django"""
from django.utils.importlib import import_module
from django.utils.translation import ugettext_lazy as _
__all__ = (
'get_user_document',
)
MONGOENGINE_USER_DOCUMENT = getattr(
settings, 'MONGOENGINE_USER_DOCUMENT', 'mongoengine.django.auth.User')
def get_user_document():
"""Get the user document class used for authentication.
This is the class defined in settings.MONGOENGINE_USER_DOCUMENT, which
defaults to `mongoengine.django.auth.User`.
"""
name = MONGOENGINE_USER_DOCUMENT
dot = name.rindex('.')
module = import_module(name[:dot])
return getattr(module, name[dot + 1:])
class MongoUserManager(UserManager):
"""A User manager wich allows the use of MongoEngine documents in Django.
To use the manager, you must tell django.contrib.auth to use MongoUser as
the user model. In you settings.py, you need:
INSTALLED_APPS = (
...
'django.contrib.auth',
'mongoengine.django.mongo_auth',
...
)
AUTH_USER_MODEL = 'mongo_auth.MongoUser'
Django will use the model object to access the custom Manager, which will
replace the original queryset with MongoEngine querysets.
By default, mongoengine.django.auth.User will be used to store users. You
can specify another document class in MONGOENGINE_USER_DOCUMENT in your
settings.py.
The User Document class has the same requirements as a standard custom user
model: https://docs.djangoproject.com/en/dev/topics/auth/customizing/
In particular, the User Document class must define USERNAME_FIELD and
REQUIRED_FIELDS.
`AUTH_USER_MODEL` has been added in Django 1.5.
"""
def contribute_to_class(self, model, name):
super(MongoUserManager, self).contribute_to_class(model, name)
self.dj_model = self.model
self.model = get_user_document()
self.dj_model.USERNAME_FIELD = self.model.USERNAME_FIELD
username = models.CharField(_('username'), max_length=30, unique=True)
username.contribute_to_class(self.dj_model, self.dj_model.USERNAME_FIELD)
self.dj_model.REQUIRED_FIELDS = self.model.REQUIRED_FIELDS
for name in self.dj_model.REQUIRED_FIELDS:
field = models.CharField(_(name), max_length=30)
field.contribute_to_class(self.dj_model, name)
def get(self, *args, **kwargs):
try:
return self.get_query_set().get(*args, **kwargs)
except self.model.DoesNotExist:
# ModelBackend expects this exception
raise self.dj_model.DoesNotExist
@property
def db(self):
raise NotImplementedError
def get_empty_query_set(self):
return self.model.objects.none()
def get_query_set(self):
return self.model.objects
class MongoUser(models.Model):
""""Dummy user model for Django.
MongoUser is used to replace Django's UserManager with MongoUserManager.
The actual user document class is mongoengine.django.auth.User or any
other document class specified in MONGOENGINE_USER_DOCUMENT.
To get the user document class, use `get_user_document()`.
"""
objects = MongoUserManager()
class Meta:
app_label = 'mongo_auth'
def set_password(self, password):
"""Doesn't do anything, but works around the issue with Django 1.6."""
make_password(password)

View File

@@ -1,124 +0,0 @@
from bson import json_util
from django.conf import settings
from django.contrib.sessions.backends.base import SessionBase, CreateError
from django.core.exceptions import SuspiciousOperation
try:
from django.utils.encoding import force_unicode
except ImportError:
from django.utils.encoding import force_text as force_unicode
from mongoengine.document import Document
from mongoengine import fields
from mongoengine.queryset import OperationError
from mongoengine.connection import DEFAULT_CONNECTION_NAME
from .utils import datetime_now
MONGOENGINE_SESSION_DB_ALIAS = getattr(
settings, 'MONGOENGINE_SESSION_DB_ALIAS',
DEFAULT_CONNECTION_NAME)
# a setting for the name of the collection used to store sessions
MONGOENGINE_SESSION_COLLECTION = getattr(
settings, 'MONGOENGINE_SESSION_COLLECTION',
'django_session')
# a setting for whether session data is stored encoded or not
MONGOENGINE_SESSION_DATA_ENCODE = getattr(
settings, 'MONGOENGINE_SESSION_DATA_ENCODE',
True)
class MongoSession(Document):
session_key = fields.StringField(primary_key=True, max_length=40)
session_data = fields.StringField() if MONGOENGINE_SESSION_DATA_ENCODE \
else fields.DictField()
expire_date = fields.DateTimeField()
meta = {
'collection': MONGOENGINE_SESSION_COLLECTION,
'db_alias': MONGOENGINE_SESSION_DB_ALIAS,
'allow_inheritance': False,
'indexes': [
{
'fields': ['expire_date'],
'expireAfterSeconds': 0
}
]
}
def get_decoded(self):
return SessionStore().decode(self.session_data)
class SessionStore(SessionBase):
"""A MongoEngine-based session store for Django.
"""
def _get_session(self, *args, **kwargs):
sess = super(SessionStore, self)._get_session(*args, **kwargs)
if sess.get('_auth_user_id', None):
sess['_auth_user_id'] = str(sess.get('_auth_user_id'))
return sess
def load(self):
try:
s = MongoSession.objects(session_key=self.session_key,
expire_date__gt=datetime_now)[0]
if MONGOENGINE_SESSION_DATA_ENCODE:
return self.decode(force_unicode(s.session_data))
else:
return s.session_data
except (IndexError, SuspiciousOperation):
self.create()
return {}
def exists(self, session_key):
return bool(MongoSession.objects(session_key=session_key).first())
def create(self):
while True:
self._session_key = self._get_new_session_key()
try:
self.save(must_create=True)
except CreateError:
continue
self.modified = True
self._session_cache = {}
return
def save(self, must_create=False):
if self.session_key is None:
self._session_key = self._get_new_session_key()
s = MongoSession(session_key=self.session_key)
if MONGOENGINE_SESSION_DATA_ENCODE:
s.session_data = self.encode(self._get_session(no_load=must_create))
else:
s.session_data = self._get_session(no_load=must_create)
s.expire_date = self.get_expiry_date()
try:
s.save(force_insert=must_create)
except OperationError:
if must_create:
raise CreateError
raise
def delete(self, session_key=None):
if session_key is None:
if self.session_key is None:
return
session_key = self.session_key
MongoSession.objects(session_key=session_key).delete()
class BSONSerializer(object):
"""
Serializer that can handle BSON types (eg ObjectId).
"""
def dumps(self, obj):
return json_util.dumps(obj, separators=(',', ':')).encode('ascii')
def loads(self, data):
return json_util.loads(data.decode('ascii'))

View File

@@ -1,47 +0,0 @@
from mongoengine.queryset import QuerySet
from mongoengine.base import BaseDocument
from mongoengine.errors import ValidationError
def _get_queryset(cls):
"""Inspired by django.shortcuts.*"""
if isinstance(cls, QuerySet):
return cls
else:
return cls.objects
def get_document_or_404(cls, *args, **kwargs):
"""
Uses get() to return an document, or raises a Http404 exception if the document
does not exist.
cls may be a Document or QuerySet object. All other passed
arguments and keyword arguments are used in the get() query.
Note: Like with get(), an MultipleObjectsReturned will be raised if more than one
object is found.
Inspired by django.shortcuts.*
"""
queryset = _get_queryset(cls)
try:
return queryset.get(*args, **kwargs)
except (queryset._document.DoesNotExist, ValidationError):
from django.http import Http404
raise Http404('No %s matches the given query.' % queryset._document._class_name)
def get_list_or_404(cls, *args, **kwargs):
"""
Uses filter() to return a list of documents, or raise a Http404 exception if
the list is empty.
cls may be a Document or QuerySet object. All other passed
arguments and keyword arguments are used in the filter() query.
Inspired by django.shortcuts.*
"""
queryset = _get_queryset(cls)
obj_list = list(queryset.filter(*args, **kwargs))
if not obj_list:
from django.http import Http404
raise Http404('No %s matches the given query.' % queryset._document._class_name)
return obj_list

View File

@@ -1,112 +0,0 @@
import os
import itertools
import urlparse
from mongoengine import *
from django.conf import settings
from django.core.files.storage import Storage
from django.core.exceptions import ImproperlyConfigured
class FileDocument(Document):
"""A document used to store a single file in GridFS.
"""
file = FileField()
class GridFSStorage(Storage):
"""A custom storage backend to store files in GridFS
"""
def __init__(self, base_url=None):
if base_url is None:
base_url = settings.MEDIA_URL
self.base_url = base_url
self.document = FileDocument
self.field = 'file'
def delete(self, name):
"""Deletes the specified file from the storage system.
"""
if self.exists(name):
doc = self.document.objects.first()
field = getattr(doc, self.field)
self._get_doc_with_name(name).delete() # Delete the FileField
field.delete() # Delete the FileDocument
def exists(self, name):
"""Returns True if a file referened by the given name already exists in the
storage system, or False if the name is available for a new file.
"""
doc = self._get_doc_with_name(name)
if doc:
field = getattr(doc, self.field)
return bool(field.name)
else:
return False
def listdir(self, path=None):
"""Lists the contents of the specified path, returning a 2-tuple of lists;
the first item being directories, the second item being files.
"""
def name(doc):
return getattr(doc, self.field).name
docs = self.document.objects
return [], [name(d) for d in docs if name(d)]
def size(self, name):
"""Returns the total size, in bytes, of the file specified by name.
"""
doc = self._get_doc_with_name(name)
if doc:
return getattr(doc, self.field).length
else:
raise ValueError("No such file or directory: '%s'" % name)
def url(self, name):
"""Returns an absolute URL where the file's contents can be accessed
directly by a web browser.
"""
if self.base_url is None:
raise ValueError("This file is not accessible via a URL.")
return urlparse.urljoin(self.base_url, name).replace('\\', '/')
def _get_doc_with_name(self, name):
"""Find the documents in the store with the given name
"""
docs = self.document.objects
doc = [d for d in docs if hasattr(getattr(d, self.field), 'name') and getattr(d, self.field).name == name]
if doc:
return doc[0]
else:
return None
def _open(self, name, mode='rb'):
doc = self._get_doc_with_name(name)
if doc:
return getattr(doc, self.field)
else:
raise ValueError("No file found with the name '%s'." % name)
def get_available_name(self, name):
"""Returns a filename that's free on the target storage system, and
available for new content to be written to.
"""
file_root, file_ext = os.path.splitext(name)
# If the filename already exists, add an underscore and a number (before
# the file extension, if one exists) to the filename until the generated
# filename doesn't exist.
count = itertools.count(1)
while self.exists(name):
# file_ext includes the dot.
name = os.path.join("%s_%s%s" % (file_root, count.next(), file_ext))
return name
def _save(self, name, content):
doc = self.document()
getattr(doc, self.field).put(content, filename=name)
doc.save()
return name

View File

@@ -1,31 +0,0 @@
#coding: utf-8
from unittest import TestCase
from mongoengine import connect
from mongoengine.connection import get_db
class MongoTestCase(TestCase):
"""
TestCase class that clear the collection between the tests
"""
@property
def db_name(self):
from django.conf import settings
return 'test_%s' % getattr(settings, 'MONGO_DATABASE_NAME', 'dummy')
def __init__(self, methodName='runtest'):
connect(self.db_name)
self.db = get_db()
super(MongoTestCase, self).__init__(methodName)
def dropCollections(self):
for collection in self.db.collection_names():
if collection.startswith('system.'):
continue
self.db.drop_collection(collection)
def tearDown(self):
self.dropCollections()

View File

@@ -1,6 +0,0 @@
try:
# django >= 1.4
from django.utils.timezone import now as datetime_now
except ImportError:
from datetime import datetime
datetime_now = datetime.now

View File

@@ -1,29 +1,29 @@
import re
import warnings
import hashlib
import pymongo
import re
from pymongo.read_preferences import ReadPreference
from bson import ObjectId
from bson.dbref import DBRef
import pymongo
from pymongo.read_preferences import ReadPreference
from mongoengine import signals
from mongoengine.common import _import_class
from mongoengine.base import (
DocumentMetaclass,
TopLevelDocumentMetaclass,
BaseDocument,
BaseDict,
BaseList,
EmbeddedDocumentList,
ALLOW_INHERITANCE,
BaseDict,
BaseDocument,
BaseList,
DocumentMetaclass,
EmbeddedDocumentList,
TopLevelDocumentMetaclass,
get_document
)
from mongoengine.errors import ValidationError, InvalidQueryError, InvalidDocumentError
from mongoengine.queryset import (OperationError, NotUniqueError,
from mongoengine.common import _import_class
from mongoengine.connection import DEFAULT_CONNECTION_NAME, get_db
from mongoengine.context_managers import switch_collection, switch_db
from mongoengine.errors import (InvalidDocumentError, InvalidQueryError,
SaveConditionError)
from mongoengine.python_support import IS_PYMONGO_3
from mongoengine.queryset import (NotUniqueError, OperationError,
QuerySet, transform)
from mongoengine.connection import get_db, DEFAULT_CONNECTION_NAME
from mongoengine.context_managers import switch_db, switch_collection
__all__ = ('Document', 'EmbeddedDocument', 'DynamicDocument',
'DynamicEmbeddedDocument', 'OperationError',
@@ -48,7 +48,6 @@ class InvalidCollectionError(Exception):
class EmbeddedDocument(BaseDocument):
"""A :class:`~mongoengine.Document` that isn't stored in its own
collection. :class:`~mongoengine.EmbeddedDocument`\ s should be used as
fields on :class:`~mongoengine.Document`\ s through the
@@ -63,7 +62,7 @@ class EmbeddedDocument(BaseDocument):
dictionary.
"""
__slots__ = ('_instance')
__slots__ = ('_instance', )
# The __metaclass__ attribute is removed by 2to3 when running with Python3
# my_metaclass is defined so that metaclass can be queried in Python 2 & 3
@@ -91,7 +90,6 @@ class EmbeddedDocument(BaseDocument):
class Document(BaseDocument):
"""The base class used for defining the structure and properties of
collections of documents stored in MongoDB. Inherit from this class, and
add fields as class attributes to define a document's structure.
@@ -116,9 +114,11 @@ class Document(BaseDocument):
specifying :attr:`max_documents` and :attr:`max_size` in the :attr:`meta`
dictionary. :attr:`max_documents` is the maximum number of documents that
is allowed to be stored in the collection, and :attr:`max_size` is the
maximum size of the collection in bytes. If :attr:`max_size` is not
maximum size of the collection in bytes. :attr:`max_size` is rounded up
to the next multiple of 256 by MongoDB internally and mongoengine before.
Use also a multiple of 256 to avoid confusions. If :attr:`max_size` is not
specified and :attr:`max_documents` is, :attr:`max_size` defaults to
10000000 bytes (10MB).
10485760 bytes (10MB).
Indexes may be created by specifying :attr:`indexes` in the :attr:`meta`
dictionary. The value should be a list of field names or tuples of field
@@ -135,6 +135,11 @@ class Document(BaseDocument):
doesn't contain a list) if allow_inheritance is True. This can be
disabled by either setting cls to False on the specific index or
by setting index_cls to False on the meta dictionary for the document.
By default, any extra attribute existing in stored data but not declared
in your model will raise a :class:`~mongoengine.FieldDoesNotExist` error.
This can be disabled by setting :attr:`strict` to ``False``
in the :attr:`meta` dictionary.
"""
# The __metaclass__ attribute is removed by 2to3 when running with Python3
@@ -142,18 +147,22 @@ class Document(BaseDocument):
my_metaclass = TopLevelDocumentMetaclass
__metaclass__ = TopLevelDocumentMetaclass
__slots__ = ('__objects')
__slots__ = ('__objects',)
def pk():
"""Primary key alias
"""
def fget(self):
if 'id_field' not in self._meta:
return None
return getattr(self, self._meta['id_field'])
def fset(self, value):
return setattr(self, self._meta['id_field'], value)
return property(fget, fset)
pk = pk()
@classmethod
@@ -164,14 +173,18 @@ class Document(BaseDocument):
@classmethod
def _get_collection(cls):
"""Returns the collection for the document."""
# TODO: use new get_collection() with PyMongo3 ?
if not hasattr(cls, '_collection') or cls._collection is None:
db = cls._get_db()
collection_name = cls._get_collection_name()
# Create collection as a capped collection if specified
if cls._meta['max_size'] or cls._meta['max_documents']:
if cls._meta.get('max_size') or cls._meta.get('max_documents'):
# Get max document limit and max byte size from meta
max_size = cls._meta['max_size'] or 10000000 # 10MB default
max_documents = cls._meta['max_documents']
max_size = cls._meta.get('max_size') or 10 * 2 ** 20 # 10MB default
max_documents = cls._meta.get('max_documents')
# Round up to next 256 bytes as MongoDB would do it to avoid exception
if max_size % 256:
max_size = (max_size // 256 + 1) * 256
if collection_name in db.collection_names():
cls._collection = db[collection_name]
@@ -179,7 +192,7 @@ class Document(BaseDocument):
# options match the specified capped options
options = cls._collection.options()
if options.get('max') != max_documents or \
options.get('size') != max_size:
options.get('size') != max_size:
msg = (('Cannot create collection "%s" as a capped '
'collection as it already exists')
% cls._collection)
@@ -205,7 +218,7 @@ class Document(BaseDocument):
Returns True if the document has been updated or False if the document
in the database doesn't match the query.
.. note:: All unsaved changes that has been made to the document are
.. note:: All unsaved changes that have been made to the document are
rejected if the method returns True.
:param query: the update will be performed only if the document in the
@@ -237,8 +250,8 @@ class Document(BaseDocument):
return True
def save(self, force_insert=False, validate=True, clean=True,
write_concern=None, cascade=None, cascade_kwargs=None,
_refs=None, save_condition=None, **kwargs):
write_concern=None, cascade=None, cascade_kwargs=None,
_refs=None, save_condition=None, signal_kwargs=None, **kwargs):
"""Save the :class:`~mongoengine.Document` to the database. If the
document already exists, it will be updated, otherwise it will be
created.
@@ -262,7 +275,10 @@ class Document(BaseDocument):
to cascading saves. Implies ``cascade=True``.
:param _refs: A list of processed references used in cascading saves
:param save_condition: only perform save if matching record in db
satisfies condition(s) (e.g., version number)
satisfies condition(s) (e.g. version number).
Raises :class:`OperationError` if the conditions are not satisfied
:parm signal_kwargs: (optional) kwargs dictionary to be passed to
the signal calls.
.. versionchanged:: 0.5
In existing documents it only saves changed fields using
@@ -280,8 +296,15 @@ class Document(BaseDocument):
.. versionchanged:: 0.8.5
Optional save_condition that only overwrites existing documents
if the condition is satisfied in the current db record.
.. versionchanged:: 0.10
:class:`OperationError` exception raised if save_condition fails.
.. versionchanged:: 0.10.1
:class: save_condition failure now raises a `SaveConditionError`
.. versionchanged:: 0.10.7
Add signal_kwargs argument
"""
signals.pre_save.send(self.__class__, document=self)
signal_kwargs = signal_kwargs or {}
signals.pre_save.send(self.__class__, document=self, **signal_kwargs)
if validate:
self.validate(clean=clean)
@@ -294,7 +317,7 @@ class Document(BaseDocument):
created = ('_id' not in doc or self._created or force_insert)
signals.pre_save_post_validation.send(self.__class__, document=self,
created=created)
created=created, **signal_kwargs)
try:
collection = self._get_collection()
@@ -305,6 +328,15 @@ class Document(BaseDocument):
object_id = collection.insert(doc, **write_concern)
else:
object_id = collection.save(doc, **write_concern)
# In PyMongo 3.0, the save() call calls internally the _update() call
# but they forget to return the _id value passed back, therefore getting it back here
# Correct behaviour in 2.X and in 3.0.1+ versions
if not object_id and pymongo.version_tuple == (3, 0):
pk_as_mongo_obj = self._fields.get(self._meta['id_field']).to_mongo(self.pk)
object_id = (
self._qs.filter(pk=pk_as_mongo_obj).first() and
self._qs.filter(pk=pk_as_mongo_obj).first().pk
) # TODO doesn't this make 2 queries?
else:
object_id = doc['_id']
updates, removals = self._delta()
@@ -317,8 +349,12 @@ class Document(BaseDocument):
select_dict['_id'] = object_id
shard_key = self.__class__._meta.get('shard_key', tuple())
for k in shard_key:
actual_key = self._db_field_map.get(k, k)
select_dict[actual_key] = doc[actual_key]
path = self._lookup_field(k.split('.'))
actual_key = [p.db_field for p in path]
val = doc
for ak in actual_key:
val = val[ak]
select_dict['.'.join(actual_key)] = val
def is_new_object(last_error):
if last_error is not None:
@@ -337,6 +373,9 @@ class Document(BaseDocument):
upsert = save_condition is None
last_error = collection.update(select_dict, update_query,
upsert=upsert, **write_concern)
if not upsert and last_error["n"] == 0:
raise SaveConditionError('Race condition preventing'
' document update detected')
created = is_new_object(last_error)
if cascade is None:
@@ -369,14 +408,15 @@ class Document(BaseDocument):
if created or id_field not in self._meta.get('shard_key', []):
self[id_field] = self._fields[id_field].to_python(object_id)
signals.post_save.send(self.__class__, document=self, created=created)
signals.post_save.send(self.__class__, document=self,
created=created, **signal_kwargs)
self._clear_changed_fields()
self._created = False
return self
def cascade_save(self, *args, **kwargs):
"""Recursively saves any references /
generic references on an objects"""
generic references on the document"""
_refs = kwargs.get('_refs', []) or []
ReferenceField = _import_class('ReferenceField')
@@ -417,7 +457,12 @@ class Document(BaseDocument):
select_dict = {'pk': self.pk}
shard_key = self.__class__._meta.get('shard_key', tuple())
for k in shard_key:
select_dict[k] = getattr(self, k)
path = self._lookup_field(k.split('.'))
actual_key = [p.db_field for p in path]
val = self
for ak in actual_key:
val = getattr(val, ak)
select_dict['__'.join(actual_key)] = val
return select_dict
def update(self, **kwargs):
@@ -427,11 +472,11 @@ class Document(BaseDocument):
Raises :class:`OperationError` if called on an object that has not yet
been saved.
"""
if not self.pk:
if self.pk is None:
if kwargs.get('upsert', False):
query = self.to_mongo()
if "_cls" in query:
del(query["_cls"])
del query["_cls"]
return self._qs.filter(**query).update_one(**kwargs)
else:
raise OperationError(
@@ -440,18 +485,30 @@ class Document(BaseDocument):
# Need to add shard key to query, or you get an error
return self._qs.filter(**self._object_key).update_one(**kwargs)
def delete(self, **write_concern):
def delete(self, signal_kwargs=None, **write_concern):
"""Delete the :class:`~mongoengine.Document` from the database. This
will only take effect if the document has been previously saved.
:parm signal_kwargs: (optional) kwargs dictionary to be passed to
the signal calls.
:param write_concern: Extra keyword arguments are passed down which
will be used as options for the resultant
``getLastError`` command. For example,
``save(..., write_concern={w: 2, fsync: True}, ...)`` will
wait until at least two servers have recorded the write and
will force an fsync on the primary server.
.. versionchanged:: 0.10.7
Add signal_kwargs argument
"""
signals.pre_delete.send(self.__class__, document=self)
signal_kwargs = signal_kwargs or {}
signals.pre_delete.send(self.__class__, document=self, **signal_kwargs)
# Delete FileFields separately
FileField = _import_class('FileField')
for name, field in self._fields.iteritems():
if isinstance(field, FileField):
getattr(self, name).delete()
try:
self._qs.filter(
@@ -459,9 +516,9 @@ class Document(BaseDocument):
except pymongo.errors.OperationFailure, err:
message = u'Could not delete document (%s)' % err.message
raise OperationError(message)
signals.post_delete.send(self.__class__, document=self)
signals.post_delete.send(self.__class__, document=self, **signal_kwargs)
def switch_db(self, db_alias):
def switch_db(self, db_alias, keep_created=True):
"""
Temporarily switch the database for a document instance.
@@ -473,6 +530,9 @@ class Document(BaseDocument):
:param str db_alias: The database alias to use for saving the document
:param bool keep_created: keep self._created value after switching db, else is reset to True
.. seealso::
Use :class:`~mongoengine.context_managers.switch_collection`
if you need to read from another collection
@@ -483,12 +543,12 @@ class Document(BaseDocument):
self._get_collection = lambda: collection
self._get_db = lambda: db
self._collection = collection
self._created = True
self._created = True if not keep_created else self._created
self.__objects = self._qs
self.__objects._collection_obj = collection
return self
def switch_collection(self, collection_name):
def switch_collection(self, collection_name, keep_created=True):
"""
Temporarily switch the collection for a document instance.
@@ -501,6 +561,9 @@ class Document(BaseDocument):
:param str collection_name: The database alias to use for saving the
document
:param bool keep_created: keep self._created value after switching collection, else is reset to True
.. seealso::
Use :class:`~mongoengine.context_managers.switch_db`
if you need to read from another database
@@ -509,7 +572,7 @@ class Document(BaseDocument):
collection = cls._get_collection()
self._get_collection = lambda: collection
self._collection = collection
self._created = True
self._created = True if not keep_created else self._created
self.__objects = self._qs
self.__objects._collection_obj = collection
return self
@@ -541,26 +604,31 @@ class Document(BaseDocument):
elif "max_depth" in kwargs:
max_depth = kwargs["max_depth"]
if not self.pk:
if self.pk is None:
raise self.DoesNotExist("Document does not exist")
obj = self._qs.read_preference(ReadPreference.PRIMARY).filter(
**self._object_key).only(*fields).limit(1
).select_related(max_depth=max_depth)
**self._object_key).only(*fields).limit(
1).select_related(max_depth=max_depth)
if obj:
obj = obj[0]
else:
raise self.DoesNotExist("Document does not exist")
for field in self._fields_ordered:
for field in obj._data:
if not fields or field in fields:
try:
setattr(self, field, self._reload(field, obj[field]))
except KeyError:
# If field is removed from the database while the object
# is in memory, a reload would cause a KeyError
# i.e. obj.update(unset__field=1) followed by obj.reload()
delattr(self, field)
except (KeyError, AttributeError):
try:
# If field is a special field, e.g. items is stored as _reserved_items,
# an KeyError is thrown. So try to retrieve the field from _data
setattr(self, field, self._reload(field, obj._data.get(field)))
except KeyError:
# If field is removed from the database while the object
# is in memory, a reload would cause a KeyError
# i.e. obj.update(unset__field=1) followed by obj.reload()
delattr(self, field)
self._changed_fields = obj._changed_fields
self._created = False
@@ -587,7 +655,7 @@ class Document(BaseDocument):
def to_dbref(self):
"""Returns an instance of :class:`~bson.dbref.DBRef` useful in
`__raw__` queries."""
if not self.pk:
if self.pk is None:
msg = "Only saved documents can have a valid dbref"
raise OperationError(msg)
return DBRef(self.__class__._get_collection_name(), self.pk)
@@ -604,38 +672,76 @@ class Document(BaseDocument):
for class_name in document_cls._subclasses
if class_name != document_cls.__name__] + [document_cls]
for cls in classes:
for klass in classes:
for document_cls in documents:
delete_rules = cls._meta.get('delete_rules') or {}
delete_rules = klass._meta.get('delete_rules') or {}
delete_rules[(document_cls, field_name)] = rule
cls._meta['delete_rules'] = delete_rules
klass._meta['delete_rules'] = delete_rules
@classmethod
def drop_collection(cls):
"""Drops the entire collection associated with this
:class:`~mongoengine.Document` type from the database.
Raises :class:`OperationError` if the document has no collection set
(i.g. if it is `abstract`)
.. versionchanged:: 0.10.7
:class:`OperationError` exception raised if no collection available
"""
col_name = cls._get_collection_name()
if not col_name:
raise OperationError('Document %s has no collection defined '
'(is it abstract ?)' % cls)
cls._collection = None
db = cls._get_db()
db.drop_collection(cls._get_collection_name())
db.drop_collection(col_name)
@classmethod
def create_index(cls, keys, background=False, **kwargs):
"""Creates the given indexes if required.
:param keys: a single index key or a list of index keys (to
construct a multi-field index); keys may be prefixed with a **+**
or a **-** to determine the index ordering
:param background: Allows index creation in the background
"""
index_spec = cls._build_index_spec(keys)
index_spec = index_spec.copy()
fields = index_spec.pop('fields')
drop_dups = kwargs.get('drop_dups', False)
if IS_PYMONGO_3 and drop_dups:
msg = "drop_dups is deprecated and is removed when using PyMongo 3+."
warnings.warn(msg, DeprecationWarning)
elif not IS_PYMONGO_3:
index_spec['drop_dups'] = drop_dups
index_spec['background'] = background
index_spec.update(kwargs)
if IS_PYMONGO_3:
return cls._get_collection().create_index(fields, **index_spec)
else:
return cls._get_collection().ensure_index(fields, **index_spec)
@classmethod
def ensure_index(cls, key_or_list, drop_dups=False, background=False,
**kwargs):
"""Ensure that the given indexes are in place.
"""Ensure that the given indexes are in place. Deprecated in favour
of create_index.
:param key_or_list: a single index key or a list of index keys (to
construct a multi-field index); keys may be prefixed with a **+**
or a **-** to determine the index ordering
:param background: Allows index creation in the background
:param drop_dups: Was removed/ignored with MongoDB >2.7.5. The value
will be removed if PyMongo3+ is used
"""
index_spec = cls._build_index_spec(key_or_list)
index_spec = index_spec.copy()
fields = index_spec.pop('fields')
index_spec['drop_dups'] = drop_dups
index_spec['background'] = background
index_spec.update(kwargs)
return cls._get_collection().ensure_index(fields, **index_spec)
if IS_PYMONGO_3 and drop_dups:
msg = "drop_dups is deprecated and is removed when using PyMongo 3+."
warnings.warn(msg, DeprecationWarning)
elif not IS_PYMONGO_3:
kwargs.update({'drop_dups': drop_dups})
return cls.create_index(key_or_list, background=background, **kwargs)
@classmethod
def ensure_indexes(cls):
@@ -650,6 +756,9 @@ class Document(BaseDocument):
drop_dups = cls._meta.get('index_drop_dups', False)
index_opts = cls._meta.get('index_opts') or {}
index_cls = cls._meta.get('index_cls', True)
if IS_PYMONGO_3 and drop_dups:
msg = "drop_dups is deprecated and is removed when using PyMongo 3+."
warnings.warn(msg, DeprecationWarning)
collection = cls._get_collection()
# 746: when connection is via mongos, the read preference is not necessarily an indication that
@@ -672,18 +781,37 @@ class Document(BaseDocument):
cls_indexed = cls_indexed or includes_cls(fields)
opts = index_opts.copy()
opts.update(spec)
collection.ensure_index(fields, background=background,
drop_dups=drop_dups, **opts)
# we shouldn't pass 'cls' to the collection.ensureIndex options
# because of https://jira.mongodb.org/browse/SERVER-769
if 'cls' in opts:
del opts['cls']
if IS_PYMONGO_3:
collection.create_index(fields, background=background, **opts)
else:
collection.ensure_index(fields, background=background,
drop_dups=drop_dups, **opts)
# If _cls is being used (for polymorphism), it needs an index,
# only if another index doesn't begin with _cls
if (index_cls and not cls_indexed and
cls._meta.get('allow_inheritance', ALLOW_INHERITANCE) is True):
collection.ensure_index('_cls', background=background,
**index_opts)
# we shouldn't pass 'cls' to the collection.ensureIndex options
# because of https://jira.mongodb.org/browse/SERVER-769
if 'cls' in index_opts:
del index_opts['cls']
if IS_PYMONGO_3:
collection.create_index('_cls', background=background,
**index_opts)
else:
collection.ensure_index('_cls', background=background,
**index_opts)
@classmethod
def list_indexes(cls, go_up=True, go_down=True):
def list_indexes(cls):
""" Lists all of the indexes that should be created for given
collection. It includes all the indexes from super- and sub-classes.
"""
@@ -730,8 +858,8 @@ class Document(BaseDocument):
return indexes
indexes = []
for cls in classes:
for index in get_indexes_spec(cls):
for klass in classes:
for index in get_indexes_spec(klass):
if index not in indexes:
indexes.append(index)
@@ -770,7 +898,6 @@ class Document(BaseDocument):
class DynamicDocument(Document):
"""A Dynamic Document class allowing flexible, expandable and uncontrolled
schemas. As a :class:`~mongoengine.Document` subclass, acts in the same
way as an ordinary document but has expando style properties. Any data
@@ -802,7 +929,6 @@ class DynamicDocument(Document):
class DynamicEmbeddedDocument(EmbeddedDocument):
"""A Dynamic Embedded Document class allowing flexible, expandable and
uncontrolled schemas. See :class:`~mongoengine.DynamicDocument` for more
information about dynamic documents.
@@ -829,7 +955,6 @@ class DynamicEmbeddedDocument(EmbeddedDocument):
class MapReduceDocument(object):
"""A document returned from a map/reduce query.
:param collection: An instance of :class:`~pymongo.Collection`
@@ -859,7 +984,7 @@ class MapReduceDocument(object):
if not isinstance(self.key, id_field_type):
try:
self.key = id_field_type(self.key)
except:
except Exception:
raise Exception("Could not cast key as %s" %
id_field_type.__name__)

View File

@@ -6,7 +6,7 @@ from mongoengine.python_support import txt_type
__all__ = ('NotRegistered', 'InvalidDocumentError', 'LookUpError',
'DoesNotExist', 'MultipleObjectsReturned', 'InvalidQueryError',
'OperationError', 'NotUniqueError', 'FieldDoesNotExist',
'ValidationError')
'ValidationError', 'SaveConditionError')
class NotRegistered(Exception):
@@ -41,10 +41,21 @@ class NotUniqueError(OperationError):
pass
class FieldDoesNotExist(Exception):
class SaveConditionError(OperationError):
pass
class FieldDoesNotExist(Exception):
"""Raised when trying to set a field
not declared in a :class:`~mongoengine.Document`
or an :class:`~mongoengine.EmbeddedDocument`.
To avoid this behavior on data loading,
you should the :attr:`strict` to ``False``
in the :attr:`meta` dictionnary.
"""
class ValidationError(AssertionError):
"""Validation exception.
@@ -108,6 +119,7 @@ class ValidationError(AssertionError):
else:
return unicode(source)
return errors_dict
if not self.errors:
return {}
return build_dict(self.errors)
@@ -118,9 +130,9 @@ class ValidationError(AssertionError):
def generate_key(value, prefix=''):
if isinstance(value, list):
value = ' '.join([generate_key(k) for k in value])
if isinstance(value, dict):
elif isinstance(value, dict):
value = ' '.join(
[generate_key(v, k) for k, v in value.iteritems()])
[generate_key(v, k) for k, v in value.iteritems()])
results = "%s.%s" % (prefix, value) if prefix else value
return results

View File

@@ -8,6 +8,11 @@ import uuid
import warnings
from operator import itemgetter
from bson import Binary, DBRef, ObjectId, SON
import gridfs
import pymongo
import six
try:
import dateutil
except ImportError:
@@ -15,18 +20,18 @@ except ImportError:
else:
import dateutil.parser
import pymongo
import gridfs
from bson import Binary, DBRef, SON, ObjectId
try:
from bson.int64 import Int64
except ImportError:
Int64 = long
from mongoengine.errors import ValidationError
from mongoengine.python_support import (PY3, bin_type, txt_type,
str_types, StringIO)
from base import (BaseField, ComplexBaseField, ObjectIdField, GeoJsonBaseField,
get_document, BaseDocument)
from queryset import DO_NOTHING, QuerySet
from document import Document, EmbeddedDocument
from connection import get_db, DEFAULT_CONNECTION_NAME
from .base import (BaseDocument, BaseField, ComplexBaseField, GeoJsonBaseField,
ObjectIdField, get_document)
from .connection import DEFAULT_CONNECTION_NAME, get_db
from .document import Document, EmbeddedDocument
from .errors import DoesNotExist, ValidationError
from .python_support import PY3, StringIO, bin_type, str_types, txt_type
from .queryset import DO_NOTHING, QuerySet
try:
from PIL import Image, ImageOps
@@ -47,12 +52,10 @@ __all__ = [
'SequenceField', 'UUIDField', 'MultiPointField', 'MultiLineStringField',
'MultiPolygonField', 'GeoJsonBaseField']
RECURSIVE_REFERENCE_CONSTANT = 'self'
class StringField(BaseField):
"""A unicode string field.
"""
@@ -67,7 +70,7 @@ class StringField(BaseField):
return value
try:
value = value.decode('utf-8')
except:
except Exception:
pass
return value
@@ -108,33 +111,41 @@ class StringField(BaseField):
# escape unsafe characters which could lead to a re.error
value = re.escape(value)
value = re.compile(regex % value, flags)
return value
return super(StringField, self).prepare_query_value(op, value)
class URLField(StringField):
"""A field that validates input as an URL.
.. versionadded:: 0.3
"""
_URL_REGEX = re.compile(
r'^(?:http|ftp)s?://' # http:// or https://
# domain...
r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|'
r'^(?:[a-z0-9\.\-]*)://' # scheme is validated separately
r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}(?<!-)\.?)|' # domain...
r'localhost|' # localhost...
r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})' # ...or ip
r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|' # ...or ipv4
r'\[?[A-F0-9]*:[A-F0-9:]+\]?)' # ...or ipv6
r'(?::\d+)?' # optional port
r'(?:/?|[/?]\S+)$', re.IGNORECASE)
_URL_SCHEMES = ['http', 'https', 'ftp', 'ftps']
def __init__(self, verify_exists=False, url_regex=None, **kwargs):
def __init__(self, verify_exists=False, url_regex=None, schemes=None, **kwargs):
self.verify_exists = verify_exists
self.url_regex = url_regex or self._URL_REGEX
self.schemes = schemes or self._URL_SCHEMES
super(URLField, self).__init__(**kwargs)
def validate(self, value):
# Check first if the scheme is valid
scheme = value.split('://')[0].lower()
if scheme not in self.schemes:
self.error('Invalid scheme {} in URL: {}'.format(scheme, value))
return
# Then check full URL
if not self.url_regex.match(value):
self.error('Invalid URL: %s' % value)
self.error('Invalid URL: {}'.format(value))
return
if self.verify_exists:
@@ -150,8 +161,7 @@ class URLField(StringField):
class EmailField(StringField):
"""A field that validates input as an E-Mail-Address.
"""A field that validates input as an email address.
.. versionadded:: 0.4
"""
@@ -162,17 +172,16 @@ class EmailField(StringField):
# quoted-string
r'|^"([\001-\010\013\014\016-\037!#-\[\]-\177]|\\[\001-011\013\014\016-\177])*"'
# domain (max length of an ICAAN TLD is 22 characters)
r')@(?:[A-Z0-9](?:[A-Z0-9-]{0,253}[A-Z0-9])?\.)+[A-Z]{2,22}$', re.IGNORECASE
r')@(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}|[A-Z0-9-]{2,}(?<!-))$', re.IGNORECASE
)
def validate(self, value):
if not EmailField.EMAIL_REGEX.match(value):
self.error('Invalid Mail-address: %s' % value)
self.error('Invalid email address: %s' % value)
super(EmailField, self).validate(value)
class IntField(BaseField):
"""An 32-bit integer field.
"""
@@ -190,7 +199,7 @@ class IntField(BaseField):
def validate(self, value):
try:
value = int(value)
except:
except Exception:
self.error('%s could not be converted to int' % value)
if self.min_value is not None and value < self.min_value:
@@ -203,11 +212,10 @@ class IntField(BaseField):
if value is None:
return value
return int(value)
return super(IntField, self).prepare_query_value(op, int(value))
class LongField(BaseField):
"""An 64-bit integer field.
"""
@@ -222,10 +230,13 @@ class LongField(BaseField):
pass
return value
def to_mongo(self, value):
return Int64(value)
def validate(self, value):
try:
value = long(value)
except:
except Exception:
self.error('%s could not be converted to long' % value)
if self.min_value is not None and value < self.min_value:
@@ -238,11 +249,10 @@ class LongField(BaseField):
if value is None:
return value
return long(value)
return super(LongField, self).prepare_query_value(op, long(value))
class FloatField(BaseField):
"""An floating point number field.
"""
@@ -258,10 +268,14 @@ class FloatField(BaseField):
return value
def validate(self, value):
if isinstance(value, int):
value = float(value)
if isinstance(value, six.integer_types):
try:
value = float(value)
except OverflowError:
self.error('The value is too large to be converted to float')
if not isinstance(value, float):
self.error('FloatField only accepts float values')
self.error('FloatField only accepts float and integer values')
if self.min_value is not None and value < self.min_value:
self.error('Float value is too small')
@@ -273,11 +287,10 @@ class FloatField(BaseField):
if value is None:
return value
return float(value)
return super(FloatField, self).prepare_query_value(op, float(value))
class DecimalField(BaseField):
"""A fixed-point decimal number field.
.. versionchanged:: 0.8
@@ -324,7 +337,7 @@ class DecimalField(BaseField):
return value
return value.quantize(decimal.Decimal(".%s" % ("0" * self.precision)), rounding=self.rounding)
def to_mongo(self, value, use_db_field=True):
def to_mongo(self, value):
if value is None:
return value
if self.force_string:
@@ -347,11 +360,10 @@ class DecimalField(BaseField):
self.error('Decimal value is too large')
def prepare_query_value(self, op, value):
return self.to_mongo(value)
return super(DecimalField, self).prepare_query_value(op, self.to_mongo(value))
class BooleanField(BaseField):
"""A boolean field type.
.. versionadded:: 0.1.2
@@ -370,7 +382,6 @@ class BooleanField(BaseField):
class DateTimeField(BaseField):
"""A datetime field.
Uses the python-dateutil library if available alternatively use time.strptime
@@ -434,11 +445,10 @@ class DateTimeField(BaseField):
return None
def prepare_query_value(self, op, value):
return self.to_mongo(value)
return super(DateTimeField, self).prepare_query_value(op, self.to_mongo(value))
class ComplexDateTimeField(StringField):
"""
ComplexDateTimeField handles microseconds exactly instead of rounding
like DateTimeField does.
@@ -458,37 +468,22 @@ class ComplexDateTimeField(StringField):
"""
def __init__(self, separator=',', **kwargs):
self.names = ['year', 'month', 'day', 'hour', 'minute', 'second',
'microsecond']
self.separtor = separator
self.names = ['year', 'month', 'day', 'hour', 'minute', 'second', 'microsecond']
self.separator = separator
self.format = separator.join(['%Y', '%m', '%d', '%H', '%M', '%S', '%f'])
super(ComplexDateTimeField, self).__init__(**kwargs)
def _leading_zero(self, number):
"""
Converts the given number to a string.
If it has only one digit, a leading zero so as it has always at least
two digits.
"""
if int(number) < 10:
return "0%s" % number
else:
return str(number)
def _convert_from_datetime(self, val):
"""
Convert a `datetime` object to a string representation (which will be
stored in MongoDB). This is the reverse function of
`_convert_from_string`.
>>> a = datetime(2011, 6, 8, 20, 26, 24, 192284)
>>> RealDateTimeField()._convert_from_datetime(a)
'2011,06,08,20,26,24,192284'
>>> a = datetime(2011, 6, 8, 20, 26, 24, 92284)
>>> ComplexDateTimeField()._convert_from_datetime(a)
'2011,06,08,20,26,24,092284'
"""
data = []
for name in self.names:
data.append(self._leading_zero(getattr(val, name)))
return ','.join(data)
return val.strftime(self.format)
def _convert_from_string(self, data):
"""
@@ -496,16 +491,12 @@ class ComplexDateTimeField(StringField):
will manipulate). This is the reverse function of
`_convert_from_datetime`.
>>> a = '2011,06,08,20,26,24,192284'
>>> a = '2011,06,08,20,26,24,092284'
>>> ComplexDateTimeField()._convert_from_string(a)
datetime.datetime(2011, 6, 8, 20, 26, 24, 192284)
datetime.datetime(2011, 6, 8, 20, 26, 24, 92284)
"""
data = data.split(',')
data = map(int, data)
values = {}
for i in range(7):
values[self.names[i]] = data[i]
return datetime.datetime(**values)
values = map(int, data.split(self.separator))
return datetime.datetime(*values)
def __get__(self, instance, owner):
data = super(ComplexDateTimeField, self).__get__(instance, owner)
@@ -529,7 +520,7 @@ class ComplexDateTimeField(StringField):
original_value = value
try:
return self._convert_from_string(value)
except:
except Exception:
return original_value
def to_mongo(self, value):
@@ -537,11 +528,10 @@ class ComplexDateTimeField(StringField):
return self._convert_from_datetime(value)
def prepare_query_value(self, op, value):
return self._convert_from_datetime(value)
return super(ComplexDateTimeField, self).prepare_query_value(op, self._convert_from_datetime(value))
class EmbeddedDocumentField(BaseField):
"""An embedded document field - with a declared document_type.
Only valid values are subclasses of :class:`~mongoengine.EmbeddedDocument`.
"""
@@ -565,14 +555,13 @@ class EmbeddedDocumentField(BaseField):
def to_python(self, value):
if not isinstance(value, self.document_type):
return self.document_type._from_son(value)
return self.document_type._from_son(value, _auto_dereference=self._auto_dereference)
return value
def to_mongo(self, value, use_db_field=True, fields=[]):
def to_mongo(self, value, use_db_field=True, fields=None):
if not isinstance(value, self.document_type):
return value
return self.document_type.to_mongo(value, use_db_field,
fields=fields)
return self.document_type.to_mongo(value, use_db_field, fields)
def validate(self, value, clean=True):
"""Make sure that the document instance is an instance of the
@@ -588,11 +577,13 @@ class EmbeddedDocumentField(BaseField):
return self.document_type._fields.get(member_name)
def prepare_query_value(self, op, value):
if value is not None and not isinstance(value, self.document_type):
value = self.document_type._from_son(value)
super(EmbeddedDocumentField, self).prepare_query_value(op, value)
return self.to_mongo(value)
class GenericEmbeddedDocumentField(BaseField):
"""A generic embedded document field - allows any
:class:`~mongoengine.EmbeddedDocument` to be stored.
@@ -604,7 +595,7 @@ class GenericEmbeddedDocumentField(BaseField):
"""
def prepare_query_value(self, op, value):
return self.to_mongo(value)
return super(GenericEmbeddedDocumentField, self).prepare_query_value(op, self.to_mongo(value))
def to_python(self, value):
if isinstance(value, dict):
@@ -620,24 +611,23 @@ class GenericEmbeddedDocumentField(BaseField):
value.validate(clean=clean)
def to_mongo(self, document, use_db_field=True):
def to_mongo(self, document, use_db_field=True, fields=None):
if document is None:
return None
data = document.to_mongo(use_db_field)
if not '_cls' in data:
data = document.to_mongo(use_db_field, fields)
if '_cls' not in data:
data['_cls'] = document._class_name
return data
class DynamicField(BaseField):
"""A truly dynamic field type capable of handling different and varying
types of data.
Used by :class:`~mongoengine.DynamicDocument` to handle dynamic data"""
def to_mongo(self, value):
def to_mongo(self, value, use_db_field=True, fields=None):
"""Convert a Python type to a MongoDB compatible type.
"""
@@ -646,11 +636,11 @@ class DynamicField(BaseField):
if hasattr(value, 'to_mongo'):
cls = value.__class__
val = value.to_mongo()
val = value.to_mongo(use_db_field, fields)
# If we its a document thats not inherited add _cls
if (isinstance(value, Document)):
if isinstance(value, Document):
val = {"_ref": value.to_dbref(), "_cls": cls.__name__}
if (isinstance(value, EmbeddedDocument)):
if isinstance(value, EmbeddedDocument):
val['_cls'] = cls.__name__
return val
@@ -664,7 +654,7 @@ class DynamicField(BaseField):
data = {}
for k, v in value.iteritems():
data[k] = self.to_mongo(v)
data[k] = self.to_mongo(v, use_db_field, fields)
value = data
if is_list: # Convert back to a list
@@ -685,9 +675,8 @@ class DynamicField(BaseField):
def prepare_query_value(self, op, value):
if isinstance(value, basestring):
from mongoengine.fields import StringField
return StringField().prepare_query_value(op, value)
return self.to_mongo(value)
return super(DynamicField, self).prepare_query_value(op, self.to_mongo(value))
def validate(self, value, clean=True):
if hasattr(value, "validate"):
@@ -695,7 +684,6 @@ class DynamicField(BaseField):
class ListField(ComplexBaseField):
"""A list field that wraps a standard field, allowing multiple instances
of the field to be used as a list in the database.
@@ -720,9 +708,10 @@ class ListField(ComplexBaseField):
def prepare_query_value(self, op, value):
if self.field:
if op in ('set', 'unset') and (not isinstance(value, basestring)
and not isinstance(value, BaseDocument)
and hasattr(value, '__iter__')):
if op in ('set', 'unset', None) and (
not isinstance(value, basestring) and
not isinstance(value, BaseDocument) and
hasattr(value, '__iter__')):
return [self.field.prepare_query_value(op, v) for v in value]
return self.field.prepare_query_value(op, value)
return super(ListField, self).prepare_query_value(op, value)
@@ -740,12 +729,10 @@ class EmbeddedDocumentListField(ListField):
"""
def __init__(self, document_type, *args, **kwargs):
def __init__(self, document_type, **kwargs):
"""
:param document_type: The type of
:class:`~mongoengine.EmbeddedDocument` the list will hold.
:param args: Arguments passed directly into the parent
:class:`~mongoengine.ListField`.
:param kwargs: Keyword arguments passed directly into the parent
:class:`~mongoengine.ListField`.
"""
@@ -755,7 +742,6 @@ class EmbeddedDocumentListField(ListField):
class SortedListField(ListField):
"""A ListField that sorts the contents of its list before writing to
the database in order to ensure that a sorted list is always
retrieved.
@@ -780,8 +766,8 @@ class SortedListField(ListField):
self._order_reverse = kwargs.pop('reverse')
super(SortedListField, self).__init__(field, **kwargs)
def to_mongo(self, value):
value = super(SortedListField, self).to_mongo(value)
def to_mongo(self, value, use_db_field=True, fields=None):
value = super(SortedListField, self).to_mongo(value, use_db_field, fields)
if self._ordering is not None:
return sorted(value, key=itemgetter(self._ordering),
reverse=self._order_reverse)
@@ -807,7 +793,6 @@ def key_has_dot_or_dollar(d):
class DictField(ComplexBaseField):
"""A dictionary field that wraps a standard Python dictionary. This is
similar to an embedded document, but the structure is not defined.
@@ -820,6 +805,7 @@ class DictField(ComplexBaseField):
def __init__(self, basecls=None, field=None, *args, **kwargs):
self.field = field
self._auto_dereference = False
self.basecls = basecls or BaseField
if not issubclass(self.basecls, BaseField):
self.error('DictField only accepts dict values')
@@ -863,7 +849,6 @@ class DictField(ComplexBaseField):
class MapField(DictField):
"""A field that maps a name to a specified field type. Similar to
a DictField, except the 'value' of each item must match the specified
field type.
@@ -879,7 +864,6 @@ class MapField(DictField):
class ReferenceField(BaseField):
"""A reference to a document that will be automatically dereferenced on
access (lazily).
@@ -890,12 +874,11 @@ class ReferenceField(BaseField):
The options are:
* DO_NOTHING - don't do anything (default).
* NULLIFY - Updates the reference to null.
* CASCADE - Deletes the documents associated with the reference.
* DENY - Prevent the deletion of the reference object.
* PULL - Pull the reference from a :class:`~mongoengine.fields.ListField`
of references
* DO_NOTHING (0) - don't do anything (default).
* NULLIFY (1) - Updates the reference to null.
* CASCADE (2) - Deletes the documents associated with the reference.
* DENY (3) - Prevent the deletion of the reference object.
* PULL (4) - Pull the reference from a :class:`~mongoengine.fields.ListField` of references
Alternative syntax for registering delete rules (useful when implementing
bi-directional delete rules)
@@ -906,7 +889,7 @@ class ReferenceField(BaseField):
content = StringField()
foo = ReferenceField('Foo')
Bar.register_delete_rule(Foo, 'bar', NULLIFY)
Foo.register_delete_rule(Bar, 'foo', NULLIFY)
.. note ::
`reverse_delete_rule` does not trigger pre / post delete signals to be
@@ -923,6 +906,10 @@ class ReferenceField(BaseField):
or as the :class:`~pymongo.objectid.ObjectId`.id .
:param reverse_delete_rule: Determines what to do when the referring
object is deleted
.. note ::
A reference to an abstract document type is always stored as a
:class:`~pymongo.dbref.DBRef`, regardless of the value of `dbref`.
"""
if not isinstance(document_type, basestring):
if not issubclass(document_type, (Document, basestring)):
@@ -955,9 +942,16 @@ class ReferenceField(BaseField):
self._auto_dereference = instance._fields[self.name]._auto_dereference
# Dereference DBRefs
if self._auto_dereference and isinstance(value, DBRef):
value = self.document_type._get_db().dereference(value)
if value is not None:
instance._data[self.name] = self.document_type._from_son(value)
if hasattr(value, 'cls'):
# Dereference using the class type specified in the reference
cls = get_document(value.cls)
else:
cls = self.document_type
dereferenced = cls._get_db().dereference(value)
if dereferenced is None:
raise DoesNotExist('Trying to dereference unknown document %s' % value)
else:
instance._data[self.name] = cls._from_son(dereferenced)
return super(ReferenceField, self).__get__(instance, owner)
@@ -967,21 +961,29 @@ class ReferenceField(BaseField):
return document.id
return document
id_field_name = self.document_type._meta['id_field']
id_field = self.document_type._fields[id_field_name]
if isinstance(document, Document):
# We need the id from the saved object to create the DBRef
id_ = document.pk
if id_ is None:
self.error('You can only reference documents once they have'
' been saved to the database')
# Use the attributes from the document instance, so that they
# override the attributes of this field's document type
cls = document
else:
id_ = document
cls = self.document_type
id_field_name = cls._meta['id_field']
id_field = cls._fields[id_field_name]
id_ = id_field.to_mongo(id_)
if self.dbref:
collection = self.document_type._get_collection_name()
if self.document_type._meta.get('abstract'):
collection = cls._get_collection_name()
return DBRef(collection, id_, cls=cls._class_name)
elif self.dbref:
collection = cls._get_collection_name()
return DBRef(collection, id_)
return id_
@@ -998,6 +1000,7 @@ class ReferenceField(BaseField):
def prepare_query_value(self, op, value):
if value is None:
return None
super(ReferenceField, self).prepare_query_value(op, value)
return self.to_mongo(value)
def validate(self, value):
@@ -1009,14 +1012,21 @@ class ReferenceField(BaseField):
self.error('You can only reference documents once they have been '
'saved to the database')
if self.document_type._meta.get('abstract') and \
not isinstance(value, self.document_type):
self.error(
'%s is not an instance of abstract reference type %s' % (
self.document_type._class_name)
)
def lookup_member(self, member_name):
return self.document_type._fields.get(member_name)
class CachedReferenceField(BaseField):
"""
A referencefield with cache fields to porpuse pseudo-joins
A referencefield with cache fields to purpose pseudo-joins
.. versionadded:: 0.9
"""
@@ -1028,7 +1038,6 @@ class CachedReferenceField(BaseField):
"""
if not isinstance(document_type, basestring) and \
not issubclass(document_type, (Document, basestring)):
self.error('Argument to CachedReferenceField constructor must be a'
' document class or a string')
@@ -1039,6 +1048,7 @@ class CachedReferenceField(BaseField):
def start_listener(self):
from mongoengine import signals
signals.post_save.connect(self.on_document_pre_save,
sender=self.document_type)
@@ -1061,6 +1071,7 @@ class CachedReferenceField(BaseField):
collection = self.document_type._get_collection_name()
value = DBRef(
collection, self.document_type.id.to_python(value['_id']))
return self.document_type._from_son(self.document_type._get_db().dereference(value))
return value
@@ -1083,16 +1094,17 @@ class CachedReferenceField(BaseField):
self._auto_dereference = instance._fields[self.name]._auto_dereference
# Dereference DBRefs
if self._auto_dereference and isinstance(value, DBRef):
value = self.document_type._get_db().dereference(value)
if value is not None:
instance._data[self.name] = self.document_type._from_son(value)
dereferenced = self.document_type._get_db().dereference(value)
if dereferenced is None:
raise DoesNotExist('Trying to dereference unknown document %s' % value)
else:
instance._data[self.name] = self.document_type._from_son(dereferenced)
return super(CachedReferenceField, self).__get__(instance, owner)
def to_mongo(self, document):
def to_mongo(self, document, use_db_field=True, fields=None):
id_field_name = self.document_type._meta['id_field']
id_field = self.document_type._fields[id_field_name]
doc_tipe = self.document_type
if isinstance(document, Document):
# We need the id from the saved object to create the DBRef
@@ -1102,12 +1114,18 @@ class CachedReferenceField(BaseField):
' been saved to the database')
else:
self.error('Only accept a document object')
# TODO: should raise here or will fail next statement
value = SON((
("_id", id_field.to_mongo(id_)),
))
value.update(dict(document.to_mongo(fields=self.fields)))
if fields:
new_fields = [f for f in self.fields if f in fields]
else:
new_fields = self.fields
value.update(dict(document.to_mongo(use_db_field, fields=new_fields)))
return value
def prepare_query_value(self, op, value):
@@ -1124,7 +1142,7 @@ class CachedReferenceField(BaseField):
def validate(self, value):
if not isinstance(value, (self.document_type)):
if not isinstance(value, self.document_type):
self.error("A CachedReferenceField only accepts documents")
if isinstance(value, Document) and value.id is None:
@@ -1153,7 +1171,6 @@ class CachedReferenceField(BaseField):
class GenericReferenceField(BaseField):
"""A reference to *any* :class:`~mongoengine.document.Document` subclass
that will be automatically dereferenced on access (lazily).
@@ -1167,6 +1184,30 @@ class GenericReferenceField(BaseField):
.. versionadded:: 0.3
"""
def __init__(self, *args, **kwargs):
choices = kwargs.pop('choices', None)
super(GenericReferenceField, self).__init__(*args, **kwargs)
self.choices = []
# Keep the choices as a list of allowed Document class names
if choices:
for choice in choices:
if isinstance(choice, basestring):
self.choices.append(choice)
elif isinstance(choice, type) and issubclass(choice, Document):
self.choices.append(choice._class_name)
else:
self.error('Invalid choices provided: must be a list of'
'Document subclasses and/or basestrings')
def _validate_choices(self, value):
if isinstance(value, dict):
# If the field has not been dereferenced, it is still a dict
# of class and DBRef
value = value.get('_cls')
elif isinstance(value, Document):
value = value._class_name
super(GenericReferenceField, self)._validate_choices(value)
def __get__(self, instance, owner):
if instance is None:
return self
@@ -1175,7 +1216,11 @@ class GenericReferenceField(BaseField):
self._auto_dereference = instance._fields[self.name]._auto_dereference
if self._auto_dereference and isinstance(value, (dict, SON)):
instance._data[self.name] = self.dereference(value)
dereferenced = self.dereference(value)
if dereferenced is None:
raise DoesNotExist('Trying to dereference unknown document %s' % value)
else:
instance._data[self.name] = dereferenced
return super(GenericReferenceField, self).__get__(instance, owner)
@@ -1200,7 +1245,7 @@ class GenericReferenceField(BaseField):
doc = doc_cls._from_son(doc)
return doc
def to_mongo(self, document, use_db_field=True):
def to_mongo(self, document):
if document is None:
return None
@@ -1235,7 +1280,6 @@ class GenericReferenceField(BaseField):
class BinaryField(BaseField):
"""A binary data field.
"""
@@ -1267,7 +1311,6 @@ class GridFSError(Exception):
class GridFSProxy(object):
"""Proxy object to handle writing and reading of files to and from GridFS
.. versionadded:: 0.4
@@ -1281,12 +1324,12 @@ class GridFSProxy(object):
instance=None,
db_alias=DEFAULT_CONNECTION_NAME,
collection_name='fs'):
self.grid_id = grid_id # Store GridFS id for file
self.grid_id = grid_id # Store GridFS id for file
self.key = key
self.instance = instance
self.db_alias = db_alias
self.collection_name = collection_name
self.newfile = None # Used for partial writes
self.newfile = None # Used for partial writes
self.gridout = None
def __getattr__(self, name):
@@ -1350,7 +1393,7 @@ class GridFSProxy(object):
if self.gridout is None:
self.gridout = self.fs.get(self.grid_id)
return self.gridout
except:
except Exception:
# File has been deleted
return None
@@ -1388,7 +1431,7 @@ class GridFSProxy(object):
else:
try:
return gridout.read(size)
except:
except Exception:
return ""
def delete(self):
@@ -1413,7 +1456,6 @@ class GridFSProxy(object):
class FileField(BaseField):
"""A GridFS storage field.
.. versionadded:: 0.4
@@ -1447,14 +1489,14 @@ class FileField(BaseField):
def __set__(self, instance, value):
key = self.name
if ((hasattr(value, 'read') and not
isinstance(value, GridFSProxy)) or isinstance(value, str_types)):
isinstance(value, GridFSProxy)) or isinstance(value, str_types)):
# using "FileField() = file/string" notation
grid_file = instance._data.get(self.name)
# If a file already exists, delete it
if grid_file:
try:
grid_file.delete()
except:
except Exception:
pass
# Create a new proxy object as we don't already have one
@@ -1497,7 +1539,6 @@ class FileField(BaseField):
class ImageGridFsProxy(GridFSProxy):
"""
Proxy for ImageField
@@ -1521,6 +1562,7 @@ class ImageGridFsProxy(GridFSProxy):
raise ValidationError('Invalid image: %s' % e)
# Progressive JPEG
# TODO: fixme, at least unused, at worst bad implementation
progressive = img.info.get('progressive') or False
if (kwargs.get('progressive') and
@@ -1581,7 +1623,7 @@ class ImageGridFsProxy(GridFSProxy):
if out and out.thumbnail_id:
self.fs.delete(out.thumbnail_id)
return super(ImageGridFsProxy, self).delete(*args, **kwargs)
return super(ImageGridFsProxy, self).delete()
def _put_thumbnail(self, thumbnail, format, progressive, **kwargs):
w, h = thumbnail.size
@@ -1636,7 +1678,6 @@ class ImproperlyConfigured(Exception):
class ImageField(FileField):
"""
A Image File storage field.
@@ -1675,7 +1716,6 @@ class ImageField(FileField):
class SequenceField(BaseField):
"""Provides a sequential counter see:
http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-SequenceNumbers
@@ -1687,12 +1727,21 @@ class SequenceField(BaseField):
cluster of machines, it is easier to create an object ID than have
global, uniformly increasing sequence numbers.
:param collection_name: Name of the counter collection (default 'mongoengine.counters')
:param sequence_name: Name of the sequence in the collection (default 'ClassName.counter')
:param value_decorator: Any callable to use as a counter (default int)
Use any callable as `value_decorator` to transform calculated counter into
any value suitable for your needs, e.g. string or hexadecimal
representation of the default integer counter value.
.. versionadded:: 0.5
.. note::
In case the counter is defined in the abstract document, it will be
common to all inherited documents and the default sequence name will
be the class name of the abstract document.
.. versionadded:: 0.5
.. versionchanged:: 0.8 added `value_decorator`
"""
@@ -1707,7 +1756,7 @@ class SequenceField(BaseField):
self.sequence_name = sequence_name
self.value_decorator = (callable(value_decorator) and
value_decorator or self.VALUE_DECORATOR)
return super(SequenceField, self).__init__(*args, **kwargs)
super(SequenceField, self).__init__(*args, **kwargs)
def generate(self):
"""
@@ -1753,7 +1802,7 @@ class SequenceField(BaseField):
if self.sequence_name:
return self.sequence_name
owner = self.owner_document
if issubclass(owner, Document):
if issubclass(owner, Document) and not owner._meta.get('abstract'):
return owner._get_collection_name()
else:
return ''.join('_%s' % c if c.isupper() else c
@@ -1790,7 +1839,6 @@ class SequenceField(BaseField):
class UUIDField(BaseField):
"""A UUID field.
.. versionadded:: 0.6
@@ -1816,7 +1864,7 @@ class UUIDField(BaseField):
if not isinstance(value, basestring):
value = unicode(value)
return uuid.UUID(value)
except:
except Exception:
return original_value
return value
@@ -1837,13 +1885,12 @@ class UUIDField(BaseField):
if not isinstance(value, basestring):
value = str(value)
try:
value = uuid.UUID(value)
uuid.UUID(value)
except Exception, exc:
self.error('Could not convert to UUID: %s' % exc)
class GeoPointField(BaseField):
"""A list storing a longitude and latitude coordinate.
.. note:: this represents a generic point in a 2D plane and a legacy way of
@@ -1873,7 +1920,6 @@ class GeoPointField(BaseField):
class PointField(GeoJsonBaseField):
"""A GeoJSON field storing a longitude and latitude coordinate.
The data is represented as:
@@ -1894,7 +1940,6 @@ class PointField(GeoJsonBaseField):
class LineStringField(GeoJsonBaseField):
"""A GeoJSON field storing a line of longitude and latitude coordinates.
The data is represented as:
@@ -1914,7 +1959,6 @@ class LineStringField(GeoJsonBaseField):
class PolygonField(GeoJsonBaseField):
"""A GeoJSON field storing a polygon of longitude and latitude coordinates.
The data is represented as:
@@ -1937,7 +1981,6 @@ class PolygonField(GeoJsonBaseField):
class MultiPointField(GeoJsonBaseField):
"""A GeoJSON field storing a list of Points.
The data is represented as:
@@ -1958,7 +2001,6 @@ class MultiPointField(GeoJsonBaseField):
class MultiLineStringField(GeoJsonBaseField):
"""A GeoJSON field storing a list of LineStrings.
The data is represented as:
@@ -1979,7 +2021,6 @@ class MultiLineStringField(GeoJsonBaseField):
class MultiPolygonField(GeoJsonBaseField):
"""A GeoJSON field storing list of Polygons.
The data is represented as:

View File

@@ -1,18 +1,26 @@
"""Helper functions and types to aid with Python 2.5 - 3 support."""
import sys
import pymongo
if pymongo.version_tuple[0] < 3:
IS_PYMONGO_3 = False
else:
IS_PYMONGO_3 = True
PY3 = sys.version_info[0] == 3
if PY3:
import codecs
from io import BytesIO as StringIO
# return s converted to binary. b('test') should be equivalent to b'test'
def b(s):
return codecs.latin_1_encode(s)[0]
bin_type = bytes
txt_type = str
txt_type = str
else:
try:
from cStringIO import StringIO

View File

@@ -1,6 +1,6 @@
from mongoengine.errors import (DoesNotExist, MultipleObjectsReturned,
InvalidQueryError, OperationError,
NotUniqueError)
from mongoengine.errors import (DoesNotExist, InvalidQueryError,
MultipleObjectsReturned, NotUniqueError,
OperationError)
from mongoengine.queryset.field_list import *
from mongoengine.queryset.manager import *
from mongoengine.queryset.queryset import *

View File

@@ -7,24 +7,27 @@ import pprint
import re
import warnings
from bson import SON
from bson import SON, json_util
from bson.code import Code
from bson import json_util
import pymongo
import pymongo.errors
from pymongo.common import validate_read_preference
from mongoengine import signals
from mongoengine.base.common import get_document
from mongoengine.common import _import_class
from mongoengine.connection import get_db
from mongoengine.context_managers import switch_db
from mongoengine.common import _import_class
from mongoengine.base.common import get_document
from mongoengine.errors import (OperationError, NotUniqueError,
InvalidQueryError, LookUpError)
from mongoengine.errors import (InvalidQueryError, LookUpError,
NotUniqueError, OperationError)
from mongoengine.python_support import IS_PYMONGO_3
from mongoengine.queryset import transform
from mongoengine.queryset.field_list import QueryFieldList
from mongoengine.queryset.visitor import Q, QNode
if IS_PYMONGO_3:
from pymongo.collection import ReturnDocument
__all__ = ('BaseQuerySet', 'DO_NOTHING', 'NULLIFY', 'CASCADE', 'DENY', 'PULL')
@@ -39,7 +42,6 @@ RE_TYPE = type(re.compile(''))
class BaseQuerySet(object):
"""A set of results returned from a query. Wraps a MongoDB cursor,
providing :class:`~mongoengine.Document` objects as the results.
"""
@@ -80,11 +82,12 @@ class BaseQuerySet(object):
self._limit = None
self._skip = None
self._hint = -1 # Using -1 as None is a valid value for hint
self._batch_size = None
self.only_fields = []
self._max_time_ms = None
def __call__(self, q_obj=None, class_check=True, slave_okay=False,
read_preference=None, **query):
def __call__(self, q_obj=None, class_check=True, read_preference=None,
**query):
"""Filter the selected documents by calling the
:class:`~mongoengine.queryset.QuerySet` with a query.
@@ -94,9 +97,7 @@ class BaseQuerySet(object):
objects, only the last one will be used
:param class_check: If set to False bypass class name check when
querying collection
:param slave_okay: if True, allows this query to be run against a
replica secondary.
:params read_preference: if set, overrides connection-level
:param read_preference: if set, overrides connection-level
read_preference from `ReplicaSetConnection`.
:param query: Django-style query keyword arguments
"""
@@ -122,9 +123,40 @@ class BaseQuerySet(object):
return queryset
def __getitem__(self, key):
"""Support skip and limit using getitem and slicing syntax.
def __getstate__(self):
"""
Need for pickling queryset
See https://github.com/MongoEngine/mongoengine/issues/442
"""
obj_dict = self.__dict__.copy()
# don't picke collection, instead pickle collection params
obj_dict.pop("_collection_obj")
# don't pickle cursor
obj_dict["_cursor_obj"] = None
return obj_dict
def __setstate__(self, obj_dict):
"""
Need for pickling queryset
See https://github.com/MongoEngine/mongoengine/issues/442
"""
obj_dict["_collection_obj"] = obj_dict["_document"]._get_collection()
# update attributes
self.__dict__.update(obj_dict)
# forse load cursor
# self._cursor
def __getitem__(self, key):
"""Support skip and limit using getitem and slicing syntax."""
queryset = self.clone()
# Slice provided
@@ -158,7 +190,8 @@ class BaseQuerySet(object):
if queryset._as_pymongo:
return queryset._get_as_pymongo(queryset._cursor[key])
return queryset._document._from_son(queryset._cursor[key],
_auto_dereference=self._auto_dereference, only_fields=self.only_fields)
_auto_dereference=self._auto_dereference,
only_fields=self.only_fields)
raise AttributeError
@@ -200,7 +233,8 @@ class BaseQuerySet(object):
:param language: The language that determines the list of stop words
for the search and the rules for the stemmer and tokenizer.
If not specified, the search uses the default language of the index.
For supported languages, see `Text Search Languages <http://docs.mongodb.org/manual/reference/text-search-languages/#text-search-languages>`.
For supported languages, see
`Text Search Languages <http://docs.mongodb.org/manual/reference/text-search-languages/#text-search-languages>`.
"""
queryset = self.clone()
if queryset._search_text:
@@ -253,54 +287,6 @@ class BaseQuerySet(object):
"""
return self._document(**kwargs).save()
def get_or_create(self, write_concern=None, auto_save=True,
*q_objs, **query):
"""Retrieve unique object or create, if it doesn't exist. Returns a
tuple of ``(object, created)``, where ``object`` is the retrieved or
created object and ``created`` is a boolean specifying whether a new
object was created. Raises
:class:`~mongoengine.queryset.MultipleObjectsReturned` or
`DocumentName.MultipleObjectsReturned` if multiple results are found.
A new document will be created if the document doesn't exists; a
dictionary of default values for the new document may be provided as a
keyword argument called :attr:`defaults`.
.. note:: This requires two separate operations and therefore a
race condition exists. Because there are no transactions in
mongoDB other approaches should be investigated, to ensure you
don't accidentally duplicate data when using this method. This is
now scheduled to be removed before 1.0
:param write_concern: optional extra keyword arguments used if we
have to create a new document.
Passes any write_concern onto :meth:`~mongoengine.Document.save`
:param auto_save: if the object is to be saved automatically if
not found.
.. deprecated:: 0.8
.. versionchanged:: 0.6 - added `auto_save`
.. versionadded:: 0.3
"""
msg = ("get_or_create is scheduled to be deprecated. The approach is "
"flawed without transactions. Upserts should be preferred.")
warnings.warn(msg, DeprecationWarning)
defaults = query.get('defaults', {})
if 'defaults' in query:
del query['defaults']
try:
doc = self.get(*q_objs, **query)
return doc, False
except self._document.DoesNotExist:
query.update(defaults)
doc = self._document(**query)
if auto_save:
doc.save(write_concern=write_concern)
return doc, True
def first(self):
"""Retrieve the first object matching the query.
"""
@@ -311,10 +297,11 @@ class BaseQuerySet(object):
result = None
return result
def insert(self, doc_or_docs, load_bulk=True, write_concern=None):
def insert(self, doc_or_docs, load_bulk=True,
write_concern=None, signal_kwargs=None):
"""bulk insert documents
:param docs_or_doc: a document or list of documents to be inserted
:param doc_or_docs: a document or list of documents to be inserted
:param load_bulk (optional): If True returns the list of document
instances
:param write_concern: Extra keyword arguments are passed down to
@@ -324,11 +311,15 @@ class BaseQuerySet(object):
``insert(..., {w: 2, fsync: True})`` will wait until at least
two servers have recorded the write and will force an fsync on
each server being written to.
:parm signal_kwargs: (optional) kwargs dictionary to be passed to
the signal calls.
By default returns document instances, set ``load_bulk`` to False to
return just ``ObjectIds``
.. versionadded:: 0.5
.. versionchanged:: 0.10.7
Add signal_kwargs argument
"""
Document = _import_class('Document')
@@ -341,7 +332,6 @@ class BaseQuerySet(object):
return_one = True
docs = [docs]
raw = []
for doc in docs:
if not isinstance(doc, self._document):
msg = ("Some documents inserted aren't instances of %s"
@@ -350,9 +340,12 @@ class BaseQuerySet(object):
if doc.pk and not doc._created:
msg = "Some documents have ObjectIds use doc.update() instead"
raise OperationError(msg)
raw.append(doc.to_mongo())
signals.pre_bulk_insert.send(self._document, documents=docs)
signal_kwargs = signal_kwargs or {}
signals.pre_bulk_insert.send(self._document,
documents=docs, **signal_kwargs)
raw = [doc.to_mongo() for doc in docs]
try:
ids = self._collection.insert(raw, **write_concern)
except pymongo.errors.DuplicateKeyError, err:
@@ -369,7 +362,7 @@ class BaseQuerySet(object):
if not load_bulk:
signals.post_bulk_insert.send(
self._document, documents=docs, loaded=False)
self._document, documents=docs, loaded=False, **signal_kwargs)
return return_one and ids[0] or ids
documents = self.in_bulk(ids)
@@ -377,7 +370,7 @@ class BaseQuerySet(object):
for obj_id in ids:
results.append(documents.get(obj_id))
signals.post_bulk_insert.send(
self._document, documents=results, loaded=True)
self._document, documents=results, loaded=True, **signal_kwargs)
return return_one and results[0] or results
def count(self, with_limit_and_skip=False):
@@ -391,7 +384,7 @@ class BaseQuerySet(object):
return 0
return self._cursor.count(with_limit_and_skip=with_limit_and_skip)
def delete(self, write_concern=None, _from_doc_delete=False):
def delete(self, write_concern=None, _from_doc_delete=False, cascade_refs=None):
"""Delete the documents matched by the query.
:param write_concern: Extra keyword arguments are passed down which
@@ -423,7 +416,7 @@ class BaseQuerySet(object):
if call_document_delete:
cnt = 0
for doc in queryset:
doc.delete(write_concern=write_concern)
doc.delete(**write_concern)
cnt += 1
return cnt
@@ -447,11 +440,15 @@ class BaseQuerySet(object):
continue
rule = doc._meta['delete_rules'][rule_entry]
if rule == CASCADE:
ref_q = document_cls.objects(**{field_name + '__in': self})
cascade_refs = set() if cascade_refs is None else cascade_refs
# Handle recursive reference
if doc._collection == document_cls._collection:
for ref in queryset:
cascade_refs.add(ref.id)
ref_q = document_cls.objects(**{field_name + '__in': self, 'id__nin': cascade_refs})
ref_q_count = ref_q.count()
if (doc != document_cls and ref_q_count > 0
or (doc == document_cls and ref_q_count > 0)):
ref_q.delete(write_concern=write_concern)
if ref_q_count > 0:
ref_q.delete(write_concern=write_concern, cascade_refs=cascade_refs)
elif rule == NULLIFY:
document_cls.objects(**{field_name + '__in': self}).update(
write_concern=write_concern, **{'unset__%s' % field_name: 1})
@@ -461,13 +458,14 @@ class BaseQuerySet(object):
**{'pull_all__%s' % field_name: self})
result = queryset._collection.remove(queryset._query, **write_concern)
return result["n"]
if result:
return result.get("n")
def update(self, upsert=False, multi=True, write_concern=None,
full_result=False, **update):
"""Perform an atomic update on the fields matched by the query.
:param upsert: Any existing document with that "_id" is overwritten.
:param upsert: insert if document doesn't exist (default ``False``)
:param multi: Update multiple documents.
:param write_concern: Extra keyword arguments are passed down which
will be used as options for the resultant
@@ -513,10 +511,37 @@ class BaseQuerySet(object):
raise OperationError(message)
raise OperationError(u'Update failed (%s)' % unicode(err))
def update_one(self, upsert=False, write_concern=None, **update):
"""Perform an atomic update on first field matched by the query.
def upsert_one(self, write_concern=None, **update):
"""Overwrite or add the first document matched by the query.
:param upsert: Any existing document with that "_id" is overwritten.
:param write_concern: Extra keyword arguments are passed down which
will be used as options for the resultant
``getLastError`` command. For example,
``save(..., write_concern={w: 2, fsync: True}, ...)`` will
wait until at least two servers have recorded the write and
will force an fsync on the primary server.
:param update: Django-style update keyword arguments
:returns the new or overwritten document
.. versionadded:: 0.10.2
"""
atomic_update = self.update(multi=False, upsert=True,
write_concern=write_concern,
full_result=True, **update)
if atomic_update['updatedExisting']:
document = self.get()
else:
document = self._document.objects.with_id(atomic_update['upserted'])
return document
def update_one(self, upsert=False, write_concern=None, **update):
"""Perform an atomic update on the fields of the first document
matched by the query.
:param upsert: insert if document doesn't exist (default ``False``)
:param write_concern: Extra keyword arguments are passed down which
will be used as options for the resultant
``getLastError`` command. For example,
@@ -545,7 +570,7 @@ class BaseQuerySet(object):
:param upsert: insert if document doesn't exist (default ``False``)
:param full_response: return the entire response object from the
server (default ``False``)
server (default ``False``, not available for PyMongo 3+)
:param remove: remove rather than updating (default ``False``)
:param new: return updated rather than original document
(default ``False``)
@@ -563,13 +588,31 @@ class BaseQuerySet(object):
queryset = self.clone()
query = queryset._query
update = transform.update(queryset._document, **update)
if not IS_PYMONGO_3 or not remove:
update = transform.update(queryset._document, **update)
sort = queryset._ordering
try:
result = queryset._collection.find_and_modify(
query, update, upsert=upsert, sort=sort, remove=remove, new=new,
full_response=full_response, **self._cursor_args)
if IS_PYMONGO_3:
if full_response:
msg = "With PyMongo 3+, it is not possible anymore to get the full response."
warnings.warn(msg, DeprecationWarning)
if remove:
result = queryset._collection.find_one_and_delete(
query, sort=sort, **self._cursor_args)
else:
if new:
return_doc = ReturnDocument.AFTER
else:
return_doc = ReturnDocument.BEFORE
result = queryset._collection.find_one_and_update(
query, update, upsert=upsert, sort=sort, return_document=return_doc,
**self._cursor_args)
else:
result = queryset._collection.find_and_modify(
query, update, upsert=upsert, sort=sort, remove=remove, new=new,
full_response=full_response, **self._cursor_args)
except pymongo.errors.DuplicateKeyError, err:
raise NotUniqueError(u"Update failed (%s)" % err)
except pymongo.errors.OperationFailure, err:
@@ -621,9 +664,10 @@ class BaseQuerySet(object):
doc_map[doc['_id']] = self._get_as_pymongo(doc)
else:
for doc in docs:
doc_map[doc['_id']] = self._document._from_son(doc,
only_fields=self.only_fields,
_auto_dereference=self._auto_dereference)
doc_map[doc['_id']] = self._document._from_son(
doc,
only_fields=self.only_fields,
_auto_dereference=self._auto_dereference)
return doc_map
@@ -643,7 +687,8 @@ class BaseQuerySet(object):
return self
def using(self, alias):
"""This method is for controlling which database the QuerySet will be evaluated against if you are using more than one database.
"""This method is for controlling which database the QuerySet will be
evaluated against if you are using more than one database.
:param alias: The database alias
@@ -706,11 +751,7 @@ class BaseQuerySet(object):
:param n: the maximum number of objects to return
"""
queryset = self.clone()
if n == 0:
queryset._cursor.limit(1)
else:
queryset._cursor.limit(n)
queryset._limit = n
queryset._limit = n if n != 0 else 1
# Return self to allow chaining
return queryset
@@ -721,7 +762,6 @@ class BaseQuerySet(object):
:param n: the number of objects to skip before returning results
"""
queryset = self.clone()
queryset._cursor.skip(n)
queryset._skip = n
return queryset
@@ -739,10 +779,22 @@ class BaseQuerySet(object):
.. versionadded:: 0.5
"""
queryset = self.clone()
queryset._cursor.hint(index)
queryset._hint = index
return queryset
def batch_size(self, size):
"""Limit the number of documents returned in a single batch (each
batch requires a round trip to the server).
See http://api.mongodb.com/python/current/api/pymongo/cursor.html#pymongo.cursor.Cursor.batch_size
for details.
:param size: desired size of each batch.
"""
queryset = self.clone()
queryset._batch_size = size
return queryset
def distinct(self, field):
"""Return a list of distinct values for a given field.
@@ -854,7 +906,6 @@ class BaseQuerySet(object):
cleaned_fields = []
for key, value in kwargs.items():
parts = key.split('__')
op = None
if parts[0] in operators:
op = parts.pop(0)
value = {'$' + op: value}
@@ -896,6 +947,14 @@ class BaseQuerySet(object):
queryset._ordering = queryset._get_order_by(keys)
return queryset
def comment(self, text):
"""Add a comment to the query.
See https://docs.mongodb.com/manual/reference/method/cursor.comment/#cursor.comment
for details.
"""
return self._chainable_method("comment", text)
def explain(self, format=False):
"""Return an explain plan record for the
:class:`~mongoengine.queryset.QuerySet`\ 's cursor.
@@ -907,13 +966,18 @@ class BaseQuerySet(object):
plan = pprint.pformat(plan)
return plan
# DEPRECATED. Has no more impact on PyMongo 3+
def snapshot(self, enabled):
"""Enable or disable snapshot mode when querying.
:param enabled: whether or not snapshot mode is enabled
..versionchanged:: 0.5 - made chainable
.. deprecated:: Ignored with PyMongo 3+
"""
if IS_PYMONGO_3:
msg = "snapshot is deprecated as it has no impact when using PyMongo 3+."
warnings.warn(msg, DeprecationWarning)
queryset = self.clone()
queryset._snapshot = enabled
return queryset
@@ -929,11 +993,17 @@ class BaseQuerySet(object):
queryset._timeout = enabled
return queryset
# DEPRECATED. Has no more impact on PyMongo 3+
def slave_okay(self, enabled):
"""Enable or disable the slave_okay when querying.
:param enabled: whether or not the slave_okay is enabled
.. deprecated:: Ignored with PyMongo 3+
"""
if IS_PYMONGO_3:
msg = "slave_okay is deprecated as it has no impact when using PyMongo 3+."
warnings.warn(msg, DeprecationWarning)
queryset = self.clone()
queryset._slave_okay = enabled
return queryset
@@ -947,6 +1017,7 @@ class BaseQuerySet(object):
validate_read_preference('read_preference', read_preference)
queryset = self.clone()
queryset._read_preference = read_preference
queryset._cursor_obj = None # we need to re-create the cursor object whenever we apply read_preference
return queryset
def scalar(self, *fields):
@@ -980,7 +1051,7 @@ class BaseQuerySet(object):
"""Instead of returning Document instances, return raw values from
pymongo.
:param coerce_type: Field types (if applicable) would be use to
:param coerce_types: Field types (if applicable) would be use to
coerce types.
"""
queryset = self.clone()
@@ -1219,103 +1290,62 @@ class BaseQuerySet(object):
def sum(self, field):
"""Sum over the values of the specified field.
:param field: the field to sum over; use dot-notation to refer to
:param field: the field to sum over; use dot notation to refer to
embedded document fields
.. versionchanged:: 0.5 - updated to map_reduce as db.eval doesnt work
with sharding.
"""
map_func = """
function() {
var path = '{{~%(field)s}}'.split('.'),
field = this;
db_field = self._fields_to_dbfields([field]).pop()
pipeline = [
{'$match': self._query},
{'$group': {'_id': 'sum', 'total': {'$sum': '$' + db_field}}}
]
for (p in path) {
if (typeof field != 'undefined')
field = field[path[p]];
else
break;
}
# if we're performing a sum over a list field, we sum up all the
# elements in the list, hence we need to $unwind the arrays first
ListField = _import_class('ListField')
field_parts = field.split('.')
field_instances = self._document._lookup_field(field_parts)
if isinstance(field_instances[-1], ListField):
pipeline.insert(1, {'$unwind': '$' + field})
if (field && field.constructor == Array) {
field.forEach(function(item) {
emit(1, item||0);
});
} else if (typeof field != 'undefined') {
emit(1, field||0);
}
}
""" % dict(field=field)
reduce_func = Code("""
function(key, values) {
var sum = 0;
for (var i in values) {
sum += values[i];
}
return sum;
}
""")
for result in self.map_reduce(map_func, reduce_func, output='inline'):
return result.value
result = self._document._get_collection().aggregate(pipeline)
if IS_PYMONGO_3:
result = tuple(result)
else:
return 0
result = result.get('result')
if result:
return result[0]['total']
return 0
def average(self, field):
"""Average over the values of the specified field.
:param field: the field to average over; use dot-notation to refer to
:param field: the field to average over; use dot notation to refer to
embedded document fields
.. versionchanged:: 0.5 - updated to map_reduce as db.eval doesnt work
with sharding.
"""
map_func = """
function() {
var path = '{{~%(field)s}}'.split('.'),
field = this;
db_field = self._fields_to_dbfields([field]).pop()
pipeline = [
{'$match': self._query},
{'$group': {'_id': 'avg', 'total': {'$avg': '$' + db_field}}}
]
for (p in path) {
if (typeof field != 'undefined')
field = field[path[p]];
else
break;
}
# if we're performing an average over a list field, we average out
# all the elements in the list, hence we need to $unwind the arrays
# first
ListField = _import_class('ListField')
field_parts = field.split('.')
field_instances = self._document._lookup_field(field_parts)
if isinstance(field_instances[-1], ListField):
pipeline.insert(1, {'$unwind': '$' + field})
if (field && field.constructor == Array) {
field.forEach(function(item) {
emit(1, {t: item||0, c: 1});
});
} else if (typeof field != 'undefined') {
emit(1, {t: field||0, c: 1});
}
}
""" % dict(field=field)
reduce_func = Code("""
function(key, values) {
var out = {t: 0, c: 0};
for (var i in values) {
var value = values[i];
out.t += value.t;
out.c += value.c;
}
return out;
}
""")
finalize_func = Code("""
function(key, value) {
return value.t / value.c;
}
""")
for result in self.map_reduce(map_func, reduce_func,
finalize_f=finalize_func, output='inline'):
return result.value
result = self._document._get_collection().aggregate(pipeline)
if IS_PYMONGO_3:
result = tuple(result)
else:
return 0
result = result.get('result')
if result:
return result[0]['total']
return 0
def item_frequencies(self, field, normalize=False, map_reduce=True):
"""Returns a dictionary of all items present in a field across
@@ -1327,7 +1357,7 @@ class BaseQuerySet(object):
Can only do direct simple mappings and cannot map across
:class:`~mongoengine.fields.ReferenceField` or
:class:`~mongoengine.fields.GenericReferenceField` for more complex
counting a manual map reduce call would is required.
counting a manual map reduce call is required.
If the field is a :class:`~mongoengine.fields.ListField`, the items within
each list will be counted individually.
@@ -1383,22 +1413,34 @@ class BaseQuerySet(object):
@property
def _cursor_args(self):
cursor_args = {
'snapshot': self._snapshot,
'timeout': self._timeout
}
if self._read_preference is not None:
cursor_args['read_preference'] = self._read_preference
if not IS_PYMONGO_3:
fields_name = 'fields'
cursor_args = {
'timeout': self._timeout,
'snapshot': self._snapshot
}
if self._read_preference is not None:
cursor_args['read_preference'] = self._read_preference
else:
cursor_args['slave_okay'] = self._slave_okay
else:
cursor_args['slave_okay'] = self._slave_okay
fields_name = 'projection'
# snapshot is not handled at all by PyMongo 3+
# TODO: evaluate similar possibilities using modifiers
if self._snapshot:
msg = "The snapshot option is not anymore available with PyMongo 3+"
warnings.warn(msg, DeprecationWarning)
cursor_args = {
'no_cursor_timeout': not self._timeout
}
if self._loaded_fields:
cursor_args['fields'] = self._loaded_fields.as_dict()
cursor_args[fields_name] = self._loaded_fields.as_dict()
if self._search_text:
if 'fields' not in cursor_args:
cursor_args['fields'] = {}
if fields_name not in cursor_args:
cursor_args[fields_name] = {}
cursor_args['fields']['_text_score'] = {'$meta': "textScore"}
cursor_args[fields_name]['_text_score'] = {'$meta': "textScore"}
return cursor_args
@@ -1406,8 +1448,16 @@ class BaseQuerySet(object):
def _cursor(self):
if self._cursor_obj is None:
self._cursor_obj = self._collection.find(self._query,
**self._cursor_args)
# In PyMongo 3+, we define the read preference on a collection
# level, not a cursor level. Thus, we need to get a cloned
# collection object using `with_options` first.
if IS_PYMONGO_3 and self._read_preference is not None:
self._cursor_obj = self._collection\
.with_options(read_preference=self._read_preference)\
.find(self._query, **self._cursor_args)
else:
self._cursor_obj = self._collection.find(self._query,
**self._cursor_args)
# Apply where clauses to cursor
if self._where_clause:
where_clause = self._sub_js_fields(self._where_clause)
@@ -1431,6 +1481,9 @@ class BaseQuerySet(object):
if self._hint != -1:
self._cursor_obj.hint(self._hint)
if self._batch_size is not None:
self._cursor_obj.batch_size(self._batch_size)
return self._cursor_obj
def __deepcopy__(self, memo):
@@ -1575,7 +1628,7 @@ class BaseQuerySet(object):
return frequencies
def _fields_to_dbfields(self, fields, subdoc=False):
def _fields_to_dbfields(self, fields):
"""Translate fields paths to its db equivalents"""
ret = []
subclasses = []
@@ -1597,7 +1650,7 @@ class BaseQuerySet(object):
ret.append(subfield)
found = True
break
except LookUpError, e:
except LookUpError:
pass
if not found:
@@ -1624,7 +1677,7 @@ class BaseQuerySet(object):
key = key.replace('__', '.')
try:
key = self._document._translate_field_name(key)
except:
except Exception:
pass
key_list.append((key, direction))

View File

@@ -1,4 +1,3 @@
__all__ = ('QueryFieldList',)

View File

@@ -29,7 +29,7 @@ class QuerySetManager(object):
Document.objects is accessed.
"""
if instance is not None:
# Document class being used rather than a document object
# Document object being used rather than a document class
return self
# owner is the document that contains the QuerySetManager

View File

@@ -1,6 +1,6 @@
from mongoengine.errors import OperationError
from mongoengine.queryset.base import (BaseQuerySet, DO_NOTHING, NULLIFY,
CASCADE, DENY, PULL)
from mongoengine.queryset.base import (BaseQuerySet, CASCADE, DENY, DO_NOTHING,
NULLIFY, PULL)
__all__ = ('QuerySet', 'QuerySetNoCache', 'DO_NOTHING', 'NULLIFY', 'CASCADE',
'DENY', 'PULL')
@@ -38,7 +38,7 @@ class QuerySet(BaseQuerySet):
def __len__(self):
"""Since __len__ is called quite frequently (for example, as part of
list(qs) we populate the result cache and cache the length.
list(qs)), we populate the result cache and cache the length.
"""
if self._len is not None:
return self._len
@@ -61,7 +61,6 @@ class QuerySet(BaseQuerySet):
data[-1] = "...(remaining elements truncated)..."
return repr(data)
def _iter_results(self):
"""A generator for iterating over the result cache.
@@ -74,7 +73,7 @@ class QuerySet(BaseQuerySet):
upper = len(self._result_cache)
while pos < upper:
yield self._result_cache[pos]
pos = pos + 1
pos += 1
if not self._has_more:
raise StopIteration
if len(self._result_cache) <= pos:
@@ -161,4 +160,4 @@ class QuerySetNoDeRef(QuerySet):
"""Special no_dereference QuerySet"""
def __dereference(items, max_depth=1, instance=None, name=None):
return items
return items

View File

@@ -1,20 +1,21 @@
from collections import defaultdict
import pymongo
from bson import SON
import pymongo
from mongoengine.connection import get_connection
from mongoengine.base.fields import UPDATE_OPERATORS
from mongoengine.common import _import_class
from mongoengine.errors import InvalidQueryError, LookUpError
from mongoengine.connection import get_connection
from mongoengine.errors import InvalidQueryError
from mongoengine.python_support import IS_PYMONGO_3
__all__ = ('query', 'update')
COMPARISON_OPERATORS = ('ne', 'gt', 'gte', 'lt', 'lte', 'in', 'nin', 'mod',
'all', 'size', 'exists', 'not', 'elemMatch', 'type')
GEO_OPERATORS = ('within_distance', 'within_spherical_distance',
'within_box', 'within_polygon', 'near', 'near_sphere',
'max_distance', 'geo_within', 'geo_within_box',
'max_distance', 'min_distance', 'geo_within', 'geo_within_box',
'geo_within_polygon', 'geo_within_center',
'geo_within_sphere', 'geo_intersects')
STRING_OPERATORS = ('contains', 'icontains', 'startswith',
@@ -24,17 +25,13 @@ CUSTOM_OPERATORS = ('match',)
MATCH_OPERATORS = (COMPARISON_OPERATORS + GEO_OPERATORS +
STRING_OPERATORS + CUSTOM_OPERATORS)
UPDATE_OPERATORS = ('set', 'unset', 'inc', 'dec', 'pop', 'push',
'push_all', 'pull', 'pull_all', 'add_to_set',
'set_on_insert', 'min', 'max')
def query(_doc_cls=None, _field_operation=False, **query):
def query(_doc_cls=None, **kwargs):
"""Transform a query from Django-style format to Mongo format.
"""
mongo_query = {}
merge_query = defaultdict(list)
for key, value in sorted(query.items()):
for key, value in sorted(kwargs.items()):
if key == "__raw__":
mongo_query.update(value)
continue
@@ -47,6 +44,10 @@ def query(_doc_cls=None, _field_operation=False, **query):
if len(parts) > 1 and parts[-1] in MATCH_OPERATORS:
op = parts.pop()
# Allow to escape operator-like field name by __
if len(parts) > 1 and parts[-1] == "":
parts.pop()
negate = False
if len(parts) > 1 and parts[-1] == 'not':
parts.pop()
@@ -104,13 +105,21 @@ def query(_doc_cls=None, _field_operation=False, **query):
if op:
if op in GEO_OPERATORS:
value = _geo_operator(field, op, value)
elif op in CUSTOM_OPERATORS:
if op in ('elem_match', 'match'):
value = field.prepare_query_value(op, value)
value = {"$elemMatch": value}
elif op in ('match', 'elemMatch'):
ListField = _import_class('ListField')
EmbeddedDocumentField = _import_class('EmbeddedDocumentField')
if (
isinstance(value, dict) and
isinstance(field, ListField) and
isinstance(field.field, EmbeddedDocumentField)
):
value = query(field.field.document_type, **value)
else:
NotImplementedError("Custom method '%s' has not "
"been implemented" % op)
value = field.prepare_query_value(op, value)
value = {"$elemMatch": value}
elif op in CUSTOM_OPERATORS:
NotImplementedError("Custom method '%s' has not "
"been implemented" % op)
elif op not in STRING_OPERATORS:
value = {'$' + op: value}
@@ -125,29 +134,34 @@ def query(_doc_cls=None, _field_operation=False, **query):
elif key in mongo_query:
if key in mongo_query and isinstance(mongo_query[key], dict):
mongo_query[key].update(value)
# $maxDistance needs to come last - convert to SON
# $max/minDistance needs to come last - convert to SON
value_dict = mongo_query[key]
if ('$maxDistance' in value_dict and '$near' in value_dict):
if ('$maxDistance' in value_dict or '$minDistance' in value_dict) and \
('$near' in value_dict or '$nearSphere' in value_dict):
value_son = SON()
if isinstance(value_dict['$near'], dict):
for k, v in value_dict.iteritems():
if k == '$maxDistance':
continue
value_son[k] = v
if (get_connection().max_wire_version <= 1):
value_son['$maxDistance'] = value_dict[
'$maxDistance']
else:
value_son['$near'] = SON(value_son['$near'])
value_son['$near'][
'$maxDistance'] = value_dict['$maxDistance']
else:
for k, v in value_dict.iteritems():
if k == '$maxDistance':
continue
value_son[k] = v
value_son['$maxDistance'] = value_dict['$maxDistance']
for k, v in value_dict.iteritems():
if k == '$maxDistance' or k == '$minDistance':
continue
value_son[k] = v
# Required for MongoDB >= 2.6, may fail when combining
# PyMongo 3+ and MongoDB < 2.6
near_embedded = False
for near_op in ('$near', '$nearSphere'):
if isinstance(value_dict.get(near_op), dict) and (
IS_PYMONGO_3 or get_connection().max_wire_version > 1):
value_son[near_op] = SON(value_son[near_op])
if '$maxDistance' in value_dict:
value_son[near_op][
'$maxDistance'] = value_dict['$maxDistance']
if '$minDistance' in value_dict:
value_son[near_op][
'$minDistance'] = value_dict['$minDistance']
near_embedded = True
if not near_embedded:
if '$maxDistance' in value_dict:
value_son['$maxDistance'] = value_dict['$maxDistance']
if '$minDistance' in value_dict:
value_son['$minDistance'] = value_dict['$minDistance']
mongo_query[key] = value_son
else:
# Store for manually merging later
@@ -201,6 +215,10 @@ def update(_doc_cls=None, **update):
if parts[-1] in COMPARISON_OPERATORS:
match = parts.pop()
# Allow to escape operator-like field name by __
if len(parts) > 1 and parts[-1] == "":
parts.pop()
if _doc_cls:
# Switch field names to proper names [set in Field(name='foo')]
try:
@@ -301,7 +319,11 @@ def update(_doc_cls=None, **update):
def _geo_operator(field, op, value):
"""Helper to return the query for a given geo query"""
if field._geo_index == pymongo.GEO2D:
if op == "max_distance":
value = {'$maxDistance': value}
elif op == "min_distance":
value = {'$minDistance': value}
elif field._geo_index == pymongo.GEO2D:
if op == "within_distance":
value = {'$within': {'$center': value}}
elif op == "within_spherical_distance":
@@ -314,8 +336,6 @@ def _geo_operator(field, op, value):
value = {'$nearSphere': value}
elif op == 'within_box':
value = {'$within': {'$box': value}}
elif op == "max_distance":
value = {'$maxDistance': value}
else:
raise NotImplementedError("Geo method '%s' has not "
"been implemented for a GeoPointField" % op)
@@ -334,8 +354,6 @@ def _geo_operator(field, op, value):
value = {"$geoIntersects": _infer_geometry(value)}
elif op == "near":
value = {'$near': _infer_geometry(value)}
elif op == "max_distance":
value = {'$maxDistance': value}
else:
raise NotImplementedError("Geo method '%s' has not "
"been implemented for a %s " % (op, field._name))
@@ -352,20 +370,25 @@ def _infer_geometry(value):
raise InvalidQueryError("Invalid $geometry dictionary should have "
"type and coordinates keys")
elif isinstance(value, (list, set)):
# TODO: shouldn't we test value[0][0][0][0] to see if it is MultiPolygon?
# TODO: should both TypeError and IndexError be alike interpreted?
try:
value[0][0][0]
return {"$geometry": {"type": "Polygon", "coordinates": value}}
except:
except (TypeError, IndexError):
pass
try:
value[0][0]
return {"$geometry": {"type": "LineString", "coordinates": value}}
except:
except (TypeError, IndexError):
pass
try:
value[0]
return {"$geometry": {"type": "Point", "coordinates": value}}
except:
except (TypeError, IndexError):
pass
raise InvalidQueryError("Invalid $geometry data. Can be either a dictionary "

View File

@@ -1,8 +1,5 @@
import copy
from itertools import product
from functools import reduce
from mongoengine.errors import InvalidQueryError
from mongoengine.queryset import transform

View File

@@ -6,6 +6,7 @@ __all__ = ['pre_init', 'post_init', 'pre_save', 'pre_save_post_validation',
signals_available = False
try:
from blinker import Namespace
signals_available = True
except ImportError:
class Namespace(object):
@@ -27,7 +28,8 @@ except ImportError:
raise RuntimeError('signalling support is unavailable '
'because the blinker library is '
'not installed.')
send = lambda *a, **kw: None
send = lambda *a, **kw: None # noqa
connect = disconnect = has_receivers_for = receivers_for = \
temporarily_connected_to = _fail
del _fail

View File

@@ -1,2 +1,5 @@
pymongo>=2.7.1
nose
pymongo>=2.7.1
six==1.10.0
flake8
flake8-import-order

View File

@@ -1,11 +1,13 @@
[nosetests]
verbosity = 3
verbosity = 2
detailed-errors = 1
#with-coverage = 1
#cover-erase = 1
#cover-html = 1
#cover-html-dir = ../htmlcov
#cover-package = mongoengine
py3where = build
where = tests
#tests = document/__init__.py
cover-erase = 1
cover-branches = 1
cover-package = mongoengine
tests = tests
[flake8]
ignore=E501,F401,F403,F405,I201
exclude=build,dist,docs,venv,.tox,.eggs,tests
max-complexity=42
application-import-names=mongoengine,tests

View File

@@ -1,6 +1,6 @@
import os
import sys
from setuptools import setup, find_packages
from setuptools import find_packages, setup
# Hack to silence atexit traceback in newer python versions
try:
@@ -8,13 +8,16 @@ try:
except ImportError:
pass
DESCRIPTION = 'MongoEngine is a Python Object-Document ' + \
'Mapper for working with MongoDB.'
LONG_DESCRIPTION = None
DESCRIPTION = (
'MongoEngine is a Python Object-Document '
'Mapper for working with MongoDB.'
)
try:
LONG_DESCRIPTION = open('README.rst').read()
except:
pass
with open('README.rst') as fin:
LONG_DESCRIPTION = fin.read()
except Exception:
LONG_DESCRIPTION = None
def get_version(version_tuple):
@@ -22,6 +25,7 @@ def get_version(version_tuple):
return '.'.join(map(str, version_tuple[:-1])) + version_tuple[-1]
return '.'.join(map(str, version_tuple))
# Dirty hack to get version number from monogengine/__init__.py - we can't
# import it as it depends on PyMongo and PyMongo isn't installed until this
# file is read
@@ -29,7 +33,6 @@ init = os.path.join(os.path.dirname(__file__), 'mongoengine', '__init__.py')
version_line = list(filter(lambda l: l.startswith('VERSION'), open(init)))[0]
VERSION = get_version(eval(version_line.split('=')[-1]))
print(VERSION)
CLASSIFIERS = [
'Development Status :: 4 - Beta',
@@ -53,31 +56,33 @@ CLASSIFIERS = [
extra_opts = {"packages": find_packages(exclude=["tests", "tests.*"])}
if sys.version_info[0] == 3:
extra_opts['use_2to3'] = True
extra_opts['tests_require'] = ['nose', 'coverage', 'blinker', 'jinja2==2.6', 'Pillow>=2.0.0', 'django>=1.5.1']
extra_opts['tests_require'] = ['nose', 'coverage==3.7.1', 'blinker', 'Pillow>=2.0.0']
if "test" in sys.argv or "nosetests" in sys.argv:
extra_opts['packages'] = find_packages()
extra_opts['package_data'] = {"tests": ["fields/mongoengine.png", "fields/mongodb_leaf.png"]}
else:
extra_opts['tests_require'] = ['nose', 'coverage', 'blinker', 'django>=1.4.2', 'Pillow>=2.0.0', 'jinja2>=2.6', 'python-dateutil']
# coverage 4 does not support Python 3.2 anymore
extra_opts['tests_require'] = ['nose', 'coverage==3.7.1', 'blinker', 'Pillow>=2.0.0', 'python-dateutil']
if sys.version_info[0] == 2 and sys.version_info[1] == 6:
extra_opts['tests_require'].append('unittest2')
setup(name='mongoengine',
version=VERSION,
author='Harry Marr',
author_email='harry.marr@{nospam}gmail.com',
maintainer="Ross Lawley",
maintainer_email="ross.lawley@{nospam}gmail.com",
url='http://mongoengine.org/',
download_url='https://github.com/MongoEngine/mongoengine/tarball/master',
license='MIT',
include_package_data=True,
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
platforms=['any'],
classifiers=CLASSIFIERS,
install_requires=['pymongo>=2.7.1'],
test_suite='nose.collector',
**extra_opts
setup(
name='mongoengine',
version=VERSION,
author='Harry Marr',
author_email='harry.marr@{nospam}gmail.com',
maintainer="Ross Lawley",
maintainer_email="ross.lawley@{nospam}gmail.com",
url='http://mongoengine.org/',
download_url='https://github.com/MongoEngine/mongoengine/tarball/master',
license='MIT',
include_package_data=True,
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
platforms=['any'],
classifiers=CLASSIFIERS,
install_requires=['pymongo>=2.7.1', 'six'],
test_suite='nose.collector',
**extra_opts
)

View File

@@ -93,6 +93,7 @@ class DeltaTest(unittest.TestCase):
def delta_recursive(self, DocClass, EmbeddedClass):
class Embedded(EmbeddedClass):
id = StringField()
string_field = StringField()
int_field = IntField()
dict_field = DictField()
@@ -114,6 +115,7 @@ class DeltaTest(unittest.TestCase):
self.assertEqual(doc._delta(), ({}, {}))
embedded_1 = Embedded()
embedded_1.id = "010101"
embedded_1.string_field = 'hello'
embedded_1.int_field = 1
embedded_1.dict_field = {'hello': 'world'}
@@ -123,6 +125,7 @@ class DeltaTest(unittest.TestCase):
self.assertEqual(doc._get_changed_fields(), ['embedded_field'])
embedded_delta = {
'id': "010101",
'string_field': 'hello',
'int_field': 1,
'dict_field': {'hello': 'world'},
@@ -250,13 +253,13 @@ class DeltaTest(unittest.TestCase):
self.assertEqual(doc.embedded_field.list_field[2].list_field,
[1, 2, {'hello': 'world'}])
del(doc.embedded_field.list_field[2].list_field[2]['hello'])
del doc.embedded_field.list_field[2].list_field[2]['hello']
self.assertEqual(doc._delta(),
({}, {'embedded_field.list_field.2.list_field.2.hello': 1}))
doc.save()
doc = doc.reload(10)
del(doc.embedded_field.list_field[2].list_field)
del doc.embedded_field.list_field[2].list_field
self.assertEqual(doc._delta(),
({}, {'embedded_field.list_field.2.list_field': 1}))
@@ -590,13 +593,13 @@ class DeltaTest(unittest.TestCase):
self.assertEqual(doc.embedded_field.list_field[2].list_field,
[1, 2, {'hello': 'world'}])
del(doc.embedded_field.list_field[2].list_field[2]['hello'])
del doc.embedded_field.list_field[2].list_field[2]['hello']
self.assertEqual(doc._delta(),
({}, {'db_embedded_field.db_list_field.2.db_list_field.2.hello': 1}))
doc.save()
doc = doc.reload(10)
del(doc.embedded_field.list_field[2].list_field)
del doc.embedded_field.list_field[2].list_field
self.assertEqual(doc._delta(), ({},
{'db_embedded_field.db_list_field.2.db_list_field': 1}))
@@ -612,7 +615,7 @@ class DeltaTest(unittest.TestCase):
SON([('_cls', 'Person'), ('name', 'James'), ('age', 34)]), {}))
p.doc = 123
del(p.doc)
del p.doc
self.assertEqual(p._delta(), (
SON([('_cls', 'Person'), ('name', 'James'), ('age', 34)]), {}))
@@ -732,6 +735,56 @@ class DeltaTest(unittest.TestCase):
mydoc._clear_changed_fields()
self.assertEqual([], mydoc._get_changed_fields())
def test_lower_level_mark_as_changed(self):
class EmbeddedDoc(EmbeddedDocument):
name = StringField()
class MyDoc(Document):
subs = MapField(EmbeddedDocumentField(EmbeddedDoc))
MyDoc.drop_collection()
MyDoc().save()
mydoc = MyDoc.objects.first()
mydoc.subs['a'] = EmbeddedDoc()
self.assertEqual(["subs.a"], mydoc._get_changed_fields())
subdoc = mydoc.subs['a']
subdoc.name = 'bar'
self.assertEqual(["name"], subdoc._get_changed_fields())
self.assertEqual(["subs.a"], mydoc._get_changed_fields())
mydoc.save()
mydoc._clear_changed_fields()
self.assertEqual([], mydoc._get_changed_fields())
def test_upper_level_mark_as_changed(self):
class EmbeddedDoc(EmbeddedDocument):
name = StringField()
class MyDoc(Document):
subs = MapField(EmbeddedDocumentField(EmbeddedDoc))
MyDoc.drop_collection()
MyDoc(subs={'a': EmbeddedDoc(name='foo')}).save()
mydoc = MyDoc.objects.first()
subdoc = mydoc.subs['a']
subdoc.name = 'bar'
self.assertEqual(["name"], subdoc._get_changed_fields())
self.assertEqual(["subs.a.name"], mydoc._get_changed_fields())
mydoc.subs['a'] = EmbeddedDoc()
self.assertEqual(["subs.a"], mydoc._get_changed_fields())
mydoc.save()
mydoc._clear_changed_fields()
self.assertEqual([], mydoc._get_changed_fields())
def test_referenced_object_changed_attributes(self):
"""Ensures that when you save a new reference to a field, the referenced object isn't altered"""
@@ -774,5 +827,43 @@ class DeltaTest(unittest.TestCase):
org2.reload()
self.assertEqual(org2.name, 'New Org 2')
def test_delta_for_nested_map_fields(self):
class UInfoDocument(Document):
phone = StringField()
class EmbeddedRole(EmbeddedDocument):
type = StringField()
class EmbeddedUser(EmbeddedDocument):
name = StringField()
roles = MapField(field=EmbeddedDocumentField(EmbeddedRole))
rolist = ListField(field=EmbeddedDocumentField(EmbeddedRole))
info = ReferenceField(UInfoDocument)
class Doc(Document):
users = MapField(field=EmbeddedDocumentField(EmbeddedUser))
num = IntField(default=-1)
Doc.drop_collection()
doc = Doc(num=1)
doc.users["007"] = EmbeddedUser(name="Agent007")
doc.save()
uinfo = UInfoDocument(phone="79089269066")
uinfo.save()
d = Doc.objects(num=1).first()
d.users["007"]["roles"]["666"] = EmbeddedRole(type="superadmin")
d.users["007"]["rolist"].append(EmbeddedRole(type="oops"))
d.users["007"]["info"] = uinfo
delta = d._delta()
self.assertEqual(True, "users.007.roles.666" in delta[0])
self.assertEqual(True, "users.007.rolist" in delta[0])
self.assertEqual(True, "users.007.info" in delta[0])
self.assertEqual('superadmin', delta[0]["users.007.roles.666"]["type"])
self.assertEqual('oops', delta[0]["users.007.rolist"][0]["type"])
self.assertEqual(uinfo.id, delta[0]["users.007.info"])
if __name__ == '__main__':
unittest.main()

View File

@@ -72,7 +72,7 @@ class DynamicTest(unittest.TestCase):
obj = collection.find_one()
self.assertEqual(sorted(obj.keys()), ['_cls', '_id', 'misc', 'name'])
del(p.misc)
del p.misc
p.save()
p = self.Person.objects.get()
@@ -88,6 +88,18 @@ class DynamicTest(unittest.TestCase):
p.update(unset__misc=1)
p.reload()
def test_reload_dynamic_field(self):
self.Person.objects.delete()
p = self.Person.objects.create()
p.update(age=1)
self.assertEqual(len(p._data), 3)
self.assertEqual(sorted(p._data.keys()), ['_cls', 'id', 'name'])
p.reload()
self.assertEqual(len(p._data), 4)
self.assertEqual(sorted(p._data.keys()), ['_cls', 'age', 'id', 'name'])
def test_dynamic_document_queries(self):
"""Ensure we can query dynamic fields"""
p = self.Person()
@@ -129,6 +141,15 @@ class DynamicTest(unittest.TestCase):
self.assertEqual(1, self.Person.objects(misc__hello='world').count())
def test_three_level_complex_data_lookups(self):
"""Ensure you can query three level document dynamic fields"""
p = self.Person()
p.misc = {'hello': {'hello2': 'world'}}
p.save()
# from pprint import pprint as pp; import pdb; pdb.set_trace();
print self.Person.objects(misc__hello__hello2='world')
self.assertEqual(1, self.Person.objects(misc__hello__hello2='world').count())
def test_complex_embedded_document_validation(self):
"""Ensure embedded dynamic documents may be validated"""
class Embedded(DynamicEmbeddedDocument):
@@ -331,7 +352,7 @@ class DynamicTest(unittest.TestCase):
person = Person.objects.first()
person.attrval = "This works"
person["phone"] = "555-1212" # but this should too
person["phone"] = "555-1212" # but this should too
# Same thing two levels deep
person["address"]["city"] = "Lundenne"
@@ -347,7 +368,6 @@ class DynamicTest(unittest.TestCase):
self.assertEqual(Person.objects.first().address.city, "Londinium")
person = Person.objects.first()
person["age"] = 35
person.save()

View File

@@ -1,9 +1,8 @@
# -*- coding: utf-8 -*-
import unittest
import sys
sys.path[0:0] = [""]
import os
import pymongo
from nose.plugins.skip import SkipTest
@@ -32,10 +31,7 @@ class IndexesTest(unittest.TestCase):
self.Person = Person
def tearDown(self):
for collection in self.db.collection_names():
if 'system.' in collection:
continue
self.db.drop_collection(collection)
self.connection.drop_database(self.db)
def test_indexes_document(self):
"""Ensure that indexes are used when meta[indexes] is specified for
@@ -143,7 +139,7 @@ class IndexesTest(unittest.TestCase):
meta = {
'indexes': [
{
'fields': ('title',),
'fields': ('title',),
},
],
'allow_inheritance': True,
@@ -275,6 +271,60 @@ class IndexesTest(unittest.TestCase):
info = [value['key'] for key, value in info.iteritems()]
self.assertTrue([('current.location.point', '2d')] in info)
def test_explicit_geosphere_index(self):
"""Ensure that geosphere indexes work when created via meta[indexes]
"""
class Place(Document):
location = DictField()
meta = {
'allow_inheritance': True,
'indexes': [
'(location.point',
]
}
self.assertEqual([{'fields': [('location.point', '2dsphere')]}],
Place._meta['index_specs'])
Place.ensure_indexes()
info = Place._get_collection().index_information()
info = [value['key'] for key, value in info.iteritems()]
self.assertTrue([('location.point', '2dsphere')] in info)
def test_explicit_geohaystack_index(self):
"""Ensure that geohaystack indexes work when created via meta[indexes]
"""
raise SkipTest('GeoHaystack index creation is not supported for now'
'from meta, as it requires a bucketSize parameter.')
class Place(Document):
location = DictField()
name = StringField()
meta = {
'indexes': [
(')location.point', 'name')
]
}
self.assertEqual([{'fields': [('location.point', 'geoHaystack'), ('name', 1)]}],
Place._meta['index_specs'])
Place.ensure_indexes()
info = Place._get_collection().index_information()
info = [value['key'] for key, value in info.iteritems()]
self.assertTrue([('location.point', 'geoHaystack')] in info)
def test_create_geohaystack_index(self):
"""Ensure that geohaystack indexes can be created
"""
class Place(Document):
location = DictField()
name = StringField()
Place.create_index({'fields': (')location.point', 'name')}, bucketSize=10)
info = Place._get_collection().index_information()
info = [value['key'] for key, value in info.iteritems()]
self.assertTrue([('location.point', 'geoHaystack'), ('name', 1)] in info)
def test_dictionary_indexes(self):
"""Ensure that indexes are used when meta[indexes] contains
dictionaries instead of lists.
@@ -432,6 +482,7 @@ class IndexesTest(unittest.TestCase):
class Test(Document):
a = IntField()
b = IntField()
meta = {
'indexes': ['a'],
@@ -443,16 +494,36 @@ class IndexesTest(unittest.TestCase):
obj = Test(a=1)
obj.save()
connection = get_connection()
IS_MONGODB_3 = connection.server_info()['versionArray'][0] >= 3
# Need to be explicit about covered indexes as mongoDB doesn't know if
# the documents returned might have more keys in that here.
query_plan = Test.objects(id=obj.id).exclude('a').explain()
self.assertFalse(query_plan['indexOnly'])
if not IS_MONGODB_3:
self.assertFalse(query_plan['indexOnly'])
else:
self.assertEqual(query_plan.get('queryPlanner').get('winningPlan').get('inputStage').get('stage'), 'IDHACK')
query_plan = Test.objects(id=obj.id).only('id').explain()
self.assertTrue(query_plan['indexOnly'])
if not IS_MONGODB_3:
self.assertTrue(query_plan['indexOnly'])
else:
self.assertEqual(query_plan.get('queryPlanner').get('winningPlan').get('inputStage').get('stage'), 'IDHACK')
query_plan = Test.objects(a=1).only('a').exclude('id').explain()
self.assertTrue(query_plan['indexOnly'])
if not IS_MONGODB_3:
self.assertTrue(query_plan['indexOnly'])
else:
self.assertEqual(query_plan.get('queryPlanner').get('winningPlan').get('inputStage').get('stage'), 'IXSCAN')
self.assertEqual(query_plan.get('queryPlanner').get('winningPlan').get('stage'), 'PROJECTION')
query_plan = Test.objects(a=1).explain()
if not IS_MONGODB_3:
self.assertFalse(query_plan['indexOnly'])
else:
self.assertEqual(query_plan.get('queryPlanner').get('winningPlan').get('inputStage').get('stage'), 'IXSCAN')
self.assertEqual(query_plan.get('queryPlanner').get('winningPlan').get('stage'), 'FETCH')
def test_index_on_id(self):
@@ -491,19 +562,22 @@ class IndexesTest(unittest.TestCase):
self.assertEqual(BlogPost.objects.count(), 10)
self.assertEqual(BlogPost.objects.hint().count(), 10)
self.assertEqual(BlogPost.objects.hint([('tags', 1)]).count(), 10)
self.assertEqual(BlogPost.objects.hint([('ZZ', 1)]).count(), 10)
# PyMongo 3.0 bug only, works correctly with 2.X and 3.0.1+ versions
if pymongo.version != '3.0':
self.assertEqual(BlogPost.objects.hint([('tags', 1)]).count(), 10)
self.assertEqual(BlogPost.objects.hint([('ZZ', 1)]).count(), 10)
if pymongo.version >= '2.8':
self.assertEqual(BlogPost.objects.hint('tags').count(), 10)
else:
def invalid_index():
BlogPost.objects.hint('tags')
BlogPost.objects.hint('tags').next()
self.assertRaises(TypeError, invalid_index)
def invalid_index_2():
return BlogPost.objects.hint(('tags', 1))
return BlogPost.objects.hint(('tags', 1)).next()
self.assertRaises(Exception, invalid_index_2)
def test_unique(self):
@@ -744,33 +818,34 @@ class IndexesTest(unittest.TestCase):
name = StringField(required=True)
term = StringField(required=True)
class Report(Document):
class ReportEmbedded(Document):
key = EmbeddedDocumentField(CompoundKey, primary_key=True)
text = StringField()
Report.drop_collection()
my_key = CompoundKey(name="n", term="ok")
report = Report(text="OK", key=my_key).save()
report = ReportEmbedded(text="OK", key=my_key).save()
self.assertEqual({'text': 'OK', '_id': {'term': 'ok', 'name': 'n'}},
report.to_mongo())
self.assertEqual(report, Report.objects.get(pk=my_key))
self.assertEqual(report, ReportEmbedded.objects.get(pk=my_key))
def test_compound_key_dictfield(self):
class Report(Document):
class ReportDictField(Document):
key = DictField(primary_key=True)
text = StringField()
Report.drop_collection()
my_key = {"name": "n", "term": "ok"}
report = Report(text="OK", key=my_key).save()
report = ReportDictField(text="OK", key=my_key).save()
self.assertEqual({'text': 'OK', '_id': {'term': 'ok', 'name': 'n'}},
report.to_mongo())
self.assertEqual(report, Report.objects.get(pk=my_key))
# We can't directly call ReportDictField.objects.get(pk=my_key),
# because dicts are unordered, and if the order in MongoDB is
# different than the one in `my_key`, this test will fail.
self.assertEqual(report, ReportDictField.objects.get(pk__name=my_key['name']))
self.assertEqual(report, ReportDictField.objects.get(pk__term=my_key['term']))
def test_string_indexes(self):
@@ -785,6 +860,20 @@ class IndexesTest(unittest.TestCase):
self.assertTrue([('provider_ids.foo', 1)] in info)
self.assertTrue([('provider_ids.bar', 1)] in info)
def test_sparse_compound_indexes(self):
class MyDoc(Document):
provider_ids = DictField()
meta = {
"indexes": [{'fields': ("provider_ids.foo", "provider_ids.bar"),
'sparse': True}],
}
info = MyDoc.objects._collection.index_information()
self.assertEqual([('provider_ids.foo', 1), ('provider_ids.bar', 1)],
info['provider_ids.foo_1_provider_ids.bar_1']['key'])
self.assertTrue(info['provider_ids.foo_1_provider_ids.bar_1']['sparse'])
def test_text_indexes(self):
class Book(Document):
@@ -798,6 +887,18 @@ class IndexesTest(unittest.TestCase):
key = indexes["title_text"]["key"]
self.assertTrue(('_fts', 'text') in key)
def test_hashed_indexes(self):
class Book(Document):
ref_id = StringField()
meta = {
"indexes": ["#ref_id"],
}
indexes = Book.objects._collection.index_information()
self.assertTrue("ref_id_hashed" in indexes)
self.assertTrue(('ref_id', 'hashed') in indexes["ref_id_hashed"]["key"])
def test_indexes_after_database_drop(self):
"""
Test to ensure that indexes are re-created on a collection even
@@ -805,26 +906,122 @@ class IndexesTest(unittest.TestCase):
Issue #812
"""
# Use a new connection and database since dropping the database could
# cause concurrent tests to fail.
connection = connect(db='tempdatabase',
alias='test_indexes_after_database_drop')
class BlogPost(Document):
title = StringField()
slug = StringField(unique=True)
BlogPost.drop_collection()
meta = {'db_alias': 'test_indexes_after_database_drop'}
# Create Post #1
post1 = BlogPost(title='test1', slug='test')
post1.save()
try:
BlogPost.drop_collection()
# Drop the Database
self.connection.drop_database(BlogPost._get_db().name)
# Create Post #1
post1 = BlogPost(title='test1', slug='test')
post1.save()
# Re-create Post #1
post1 = BlogPost(title='test1', slug='test')
post1.save()
# Drop the Database
connection.drop_database('tempdatabase')
# Re-create Post #1
post1 = BlogPost(title='test1', slug='test')
post1.save()
# Create Post #2
post2 = BlogPost(title='test2', slug='test')
self.assertRaises(NotUniqueError, post2.save)
finally:
# Drop the temporary database at the end
connection.drop_database('tempdatabase')
def test_index_dont_send_cls_option(self):
"""
Ensure that 'cls' option is not sent through ensureIndex. We shouldn't
send internal MongoEngine arguments that are not a part of the index
spec.
This is directly related to the fact that MongoDB doesn't validate the
options that are passed to ensureIndex. For more details, see:
https://jira.mongodb.org/browse/SERVER-769
"""
class TestDoc(Document):
txt = StringField()
meta = {
'allow_inheritance': True,
'indexes': [
{'fields': ('txt',), 'cls': False}
]
}
class TestChildDoc(TestDoc):
txt2 = StringField()
meta = {
'indexes': [
{'fields': ('txt2',), 'cls': False}
]
}
TestDoc.drop_collection()
TestDoc.ensure_indexes()
TestChildDoc.ensure_indexes()
index_info = TestDoc._get_collection().index_information()
for key in index_info:
del index_info[key]['v'] # drop the index version - we don't care about that here
if 'ns' in index_info[key]:
del index_info[key]['ns'] # drop the index namespace - we don't care about that here, MongoDB 3+
if 'dropDups' in index_info[key]:
del index_info[key]['dropDups'] # drop the index dropDups - it is deprecated in MongoDB 3+
self.assertEqual(index_info, {
'txt_1': {
'key': [('txt', 1)],
'background': False
},
'_id_': {
'key': [('_id', 1)],
},
'txt2_1': {
'key': [('txt2', 1)],
'background': False
},
'_cls_1': {
'key': [('_cls', 1)],
'background': False,
}
})
def test_compound_index_underscore_cls_not_overwritten(self):
"""
Test that the compound index doesn't get another _cls when it is specified
"""
class TestDoc(Document):
shard_1 = StringField()
txt_1 = StringField()
meta = {
'collection': 'test',
'allow_inheritance': True,
'sparse': True,
'shard_key': 'shard_1',
'indexes': [
('shard_1', '_cls', 'txt_1'),
]
}
TestDoc.drop_collection()
TestDoc.ensure_indexes()
index_info = TestDoc._get_collection().index_information()
self.assertTrue('shard_1_1__cls_1_txt_1_1' in index_info)
# Create Post #2
post2 = BlogPost(title='test2', slug='test')
self.assertRaises(NotUniqueError, post2.save)
if __name__ == '__main__':
unittest.main()

View File

@@ -307,6 +307,69 @@ class InheritanceTest(unittest.TestCase):
doc = Animal(name='dog')
self.assertFalse('_cls' in doc.to_mongo())
def test_abstract_handle_ids_in_metaclass_properly(self):
class City(Document):
continent = StringField()
meta = {'abstract': True,
'allow_inheritance': False}
class EuropeanCity(City):
name = StringField()
berlin = EuropeanCity(name='Berlin', continent='Europe')
self.assertEqual(len(berlin._db_field_map), len(berlin._fields_ordered))
self.assertEqual(len(berlin._reverse_db_field_map), len(berlin._fields_ordered))
self.assertEqual(len(berlin._fields_ordered), 3)
self.assertEqual(berlin._fields_ordered[0], 'id')
def test_auto_id_not_set_if_specific_in_parent_class(self):
class City(Document):
continent = StringField()
city_id = IntField(primary_key=True)
meta = {'abstract': True,
'allow_inheritance': False}
class EuropeanCity(City):
name = StringField()
berlin = EuropeanCity(name='Berlin', continent='Europe')
self.assertEqual(len(berlin._db_field_map), len(berlin._fields_ordered))
self.assertEqual(len(berlin._reverse_db_field_map), len(berlin._fields_ordered))
self.assertEqual(len(berlin._fields_ordered), 3)
self.assertEqual(berlin._fields_ordered[0], 'city_id')
def test_auto_id_vs_non_pk_id_field(self):
class City(Document):
continent = StringField()
id = IntField()
meta = {'abstract': True,
'allow_inheritance': False}
class EuropeanCity(City):
name = StringField()
berlin = EuropeanCity(name='Berlin', continent='Europe')
self.assertEqual(len(berlin._db_field_map), len(berlin._fields_ordered))
self.assertEqual(len(berlin._reverse_db_field_map), len(berlin._fields_ordered))
self.assertEqual(len(berlin._fields_ordered), 4)
self.assertEqual(berlin._fields_ordered[0], 'auto_id_0')
berlin.save()
self.assertEqual(berlin.pk, berlin.auto_id_0)
def test_abstract_document_creation_does_not_fail(self):
class City(Document):
continent = StringField()
meta = {'abstract': True,
'allow_inheritance': False}
bkk = City(continent='asia')
self.assertEqual(None, bkk.pk)
# TODO: expected error? Shouldn't we create a new error type?
self.assertRaises(KeyError, lambda: setattr(bkk, 'pk', 1))
def test_allow_inheritance_embedded_document(self):
"""Ensure embedded documents respect inheritance
"""
@@ -348,7 +411,7 @@ class InheritanceTest(unittest.TestCase):
try:
class MyDocument(DateCreatedDocument, DateUpdatedDocument):
pass
except:
except Exception:
self.assertTrue(False, "Couldn't create MyDocument class")
def test_abstract_documents(self):

View File

@@ -7,15 +7,18 @@ import os
import pickle
import unittest
import uuid
import weakref
from datetime import datetime
from bson import DBRef, ObjectId
from tests import fixtures
from tests.fixtures import (PickleEmbedded, PickleTest, PickleSignalsTest,
PickleDyanmicEmbedded, PickleDynamicTest)
PickleDynamicEmbedded, PickleDynamicTest)
from mongoengine import *
from mongoengine.errors import (NotRegistered, InvalidDocumentError,
InvalidQueryError, NotUniqueError)
InvalidQueryError, NotUniqueError,
FieldDoesNotExist, SaveConditionError)
from mongoengine.queryset import NULLIFY, Q
from mongoengine.connection import get_db
from mongoengine.base import get_document
@@ -28,6 +31,8 @@ TEST_IMAGE_PATH = os.path.join(os.path.dirname(__file__),
__all__ = ("InstanceTest",)
class InstanceTest(unittest.TestCase):
def setUp(self):
@@ -61,6 +66,14 @@ class InstanceTest(unittest.TestCase):
list(self.Person._get_collection().find().sort("id")),
sorted(docs, key=lambda doc: doc["_id"]))
def assertHasInstance(self, field, instance):
self.assertTrue(hasattr(field, "_instance"))
self.assertTrue(field._instance is not None)
if isinstance(field._instance, weakref.ProxyType):
self.assertTrue(field._instance.__eq__(instance))
else:
self.assertEqual(field._instance, instance)
def test_capped_collection(self):
"""Ensure that capped collections work properly.
"""
@@ -86,7 +99,7 @@ class InstanceTest(unittest.TestCase):
options = Log.objects._collection.options()
self.assertEqual(options['capped'], True)
self.assertEqual(options['max'], 10)
self.assertTrue(options['size'] >= 4096)
self.assertEqual(options['size'], 4096)
# Check that the document cannot be redefined with different options
def recreate_log_document():
@@ -101,6 +114,69 @@ class InstanceTest(unittest.TestCase):
Log.drop_collection()
def test_capped_collection_default(self):
"""Ensure that capped collections defaults work properly.
"""
class Log(Document):
date = DateTimeField(default=datetime.now)
meta = {
'max_documents': 10,
}
Log.drop_collection()
# Create a doc to create the collection
Log().save()
options = Log.objects._collection.options()
self.assertEqual(options['capped'], True)
self.assertEqual(options['max'], 10)
self.assertEqual(options['size'], 10 * 2**20)
# Check that the document with default value can be recreated
def recreate_log_document():
class Log(Document):
date = DateTimeField(default=datetime.now)
meta = {
'max_documents': 10,
}
# Create the collection by accessing Document.objects
Log.objects
recreate_log_document()
Log.drop_collection()
def test_capped_collection_no_max_size_problems(self):
"""Ensure that capped collections with odd max_size work properly.
MongoDB rounds up max_size to next multiple of 256, recreating a doc
with the same spec failed in mongoengine <0.10
"""
class Log(Document):
date = DateTimeField(default=datetime.now)
meta = {
'max_size': 10000,
}
Log.drop_collection()
# Create a doc to create the collection
Log().save()
options = Log.objects._collection.options()
self.assertEqual(options['capped'], True)
self.assertTrue(options['size'] >= 10000)
# Check that the document with odd max_size value can be recreated
def recreate_log_document():
class Log(Document):
date = DateTimeField(default=datetime.now)
meta = {
'max_size': 10000,
}
# Create the collection by accessing Document.objects
Log.objects
recreate_log_document()
Log.drop_collection()
def test_repr(self):
"""Ensure that unicode representation works
"""
@@ -408,6 +484,20 @@ class InstanceTest(unittest.TestCase):
doc.reload()
Animal.drop_collection()
def test_reload_sharded_nested(self):
class SuperPhylum(EmbeddedDocument):
name = StringField()
class Animal(Document):
superphylum = EmbeddedDocumentField(SuperPhylum)
meta = {'shard_key': ('superphylum.name',)}
Animal.drop_collection()
doc = Animal(superphylum=SuperPhylum(name='Deuterostomia'))
doc.save()
doc.reload()
Animal.drop_collection()
def test_reload_referencing(self):
"""Ensures reloading updates weakrefs correctly
"""
@@ -481,6 +571,28 @@ class InstanceTest(unittest.TestCase):
except Exception:
self.assertFalse("Threw wrong exception")
def test_reload_of_non_strict_with_special_field_name(self):
"""Ensures reloading works for documents with meta strict == False
"""
class Post(Document):
meta = {
'strict': False
}
title = StringField()
items = ListField()
Post.drop_collection()
Post._get_collection().insert({
"title": "Items eclipse",
"items": ["more lorem", "even more ipsum"]
})
post = Post.objects.first()
post.reload()
self.assertEqual(post.title, "Items eclipse")
self.assertEqual(post.items, ["more lorem", "even more ipsum"])
def test_dictionary_access(self):
"""Ensure that dictionary-style field access works properly.
"""
@@ -543,10 +655,12 @@ class InstanceTest(unittest.TestCase):
embedded_field = EmbeddedDocumentField(Embedded)
Doc.drop_collection()
Doc(embedded_field=Embedded(string="Hi")).save()
doc = Doc(embedded_field=Embedded(string="Hi"))
self.assertHasInstance(doc.embedded_field, doc)
doc.save()
doc = Doc.objects.get()
self.assertEqual(doc, doc.embedded_field._instance)
self.assertHasInstance(doc.embedded_field, doc)
def test_embedded_document_complex_instance(self):
"""Ensure that embedded documents in complex fields can reference
@@ -558,10 +672,25 @@ class InstanceTest(unittest.TestCase):
embedded_field = ListField(EmbeddedDocumentField(Embedded))
Doc.drop_collection()
Doc(embedded_field=[Embedded(string="Hi")]).save()
doc = Doc(embedded_field=[Embedded(string="Hi")])
self.assertHasInstance(doc.embedded_field[0], doc)
doc.save()
doc = Doc.objects.get()
self.assertEqual(doc, doc.embedded_field[0]._instance)
self.assertHasInstance(doc.embedded_field[0], doc)
def test_embedded_document_complex_instance_no_use_db_field(self):
"""Ensure that use_db_field is propagated to list of Emb Docs
"""
class Embedded(EmbeddedDocument):
string = StringField(db_field='s')
class Doc(Document):
embedded_field = ListField(EmbeddedDocumentField(Embedded))
d = Doc(embedded_field=[Embedded(string="Hi")]).to_mongo(
use_db_field=False).to_dict()
self.assertEqual(d['embedded_field'], [{'string': 'Hi'}])
def test_instance_is_set_on_setattr(self):
@@ -574,11 +703,28 @@ class InstanceTest(unittest.TestCase):
Account.drop_collection()
acc = Account()
acc.email = Email(email='test@example.com')
self.assertTrue(hasattr(acc._data["email"], "_instance"))
self.assertHasInstance(acc._data["email"], acc)
acc.save()
acc1 = Account.objects.first()
self.assertTrue(hasattr(acc1._data["email"], "_instance"))
self.assertHasInstance(acc1._data["email"], acc1)
def test_instance_is_set_on_setattr_on_embedded_document_list(self):
class Email(EmbeddedDocument):
email = EmailField()
class Account(Document):
emails = EmbeddedDocumentListField(Email)
Account.drop_collection()
acc = Account()
acc.emails = [Email(email='test@example.com')]
self.assertHasInstance(acc._data["emails"][0], acc)
acc.save()
acc1 = Account.objects.first()
self.assertHasInstance(acc1._data["emails"][0], acc1)
def test_document_clean(self):
class TestDocument(Document):
@@ -952,11 +1098,12 @@ class InstanceTest(unittest.TestCase):
self.assertEqual(w1.save_id, UUID(1))
self.assertEqual(w1.count, 0)
# mismatch in save_condition prevents save
# mismatch in save_condition prevents save and raise exception
flip(w1)
self.assertTrue(w1.toggle)
self.assertEqual(w1.count, 1)
w1.save(save_condition={'save_id': UUID(42)})
self.assertRaises(SaveConditionError,
w1.save, save_condition={'save_id': UUID(42)})
w1.reload()
self.assertFalse(w1.toggle)
self.assertEqual(w1.count, 0)
@@ -984,7 +1131,8 @@ class InstanceTest(unittest.TestCase):
self.assertEqual(w1.count, 2)
flip(w2)
flip(w2)
w2.save(save_condition={'save_id': old_id})
self.assertRaises(SaveConditionError,
w2.save, save_condition={'save_id': old_id})
w2.reload()
self.assertFalse(w2.toggle)
self.assertEqual(w2.count, 2)
@@ -996,7 +1144,8 @@ class InstanceTest(unittest.TestCase):
self.assertTrue(w1.toggle)
self.assertEqual(w1.count, 3)
flip(w1)
w1.save(save_condition={'count__gte': w1.count})
self.assertRaises(SaveConditionError,
w1.save, save_condition={'count__gte': w1.count})
w1.reload()
self.assertTrue(w1.toggle)
self.assertEqual(w1.count, 3)
@@ -1757,6 +1906,62 @@ class InstanceTest(unittest.TestCase):
author.delete()
self.assertEqual(BlogPost.objects.count(), 0)
def test_reverse_delete_rule_with_custom_id_field(self):
"""Ensure that a referenced document with custom primary key
is also deleted upon deletion.
"""
class User(Document):
name = StringField(primary_key=True)
class Book(Document):
author = ReferenceField(User, reverse_delete_rule=CASCADE)
reviewer = ReferenceField(User, reverse_delete_rule=NULLIFY)
User.drop_collection()
Book.drop_collection()
user = User(name='Mike').save()
reviewer = User(name='John').save()
book = Book(author=user, reviewer=reviewer).save()
reviewer.delete()
self.assertEqual(Book.objects.count(), 1)
self.assertEqual(Book.objects.get().reviewer, None)
user.delete()
self.assertEqual(Book.objects.count(), 0)
def test_reverse_delete_rule_with_shared_id_among_collections(self):
"""Ensure that cascade delete rule doesn't mix id among collections.
"""
class User(Document):
id = IntField(primary_key=True)
class Book(Document):
id = IntField(primary_key=True)
author = ReferenceField(User, reverse_delete_rule=CASCADE)
User.drop_collection()
Book.drop_collection()
user_1 = User(id=1).save()
user_2 = User(id=2).save()
book_1 = Book(id=1, author=user_2).save()
book_2 = Book(id=2, author=user_1).save()
user_2.delete()
# Deleting user_2 should also delete book_1 but not book_2
self.assertEqual(Book.objects.count(), 1)
self.assertEqual(Book.objects.get(), book_2)
user_3 = User(id=3).save()
book_3 = Book(id=3, author=user_3).save()
user_3.delete()
# Deleting user_3 should also delete book_3
self.assertEqual(Book.objects.count(), 1)
self.assertEqual(Book.objects.get(), book_2)
def test_reverse_delete_rule_with_document_inheritance(self):
"""Ensure that a referenced document is also deleted upon deletion
of a child document.
@@ -1829,11 +2034,11 @@ class InstanceTest(unittest.TestCase):
self.assertEqual(BlogPost.objects.count(), 0)
def test_reverse_delete_rule_cascade_triggers_pre_delete_signal(self):
''' ensure the pre_delete signal is triggered upon a cascading deletion
""" ensure the pre_delete signal is triggered upon a cascading deletion
setup a blog post with content, an author and editor
delete the author which triggers deletion of blogpost via cascade
blog post's pre_delete signal alters an editor attribute
'''
"""
class Editor(self.Person):
review_queue = IntField(default=0)
@@ -2085,11 +2290,34 @@ class InstanceTest(unittest.TestCase):
self.assertEqual(pickle_doc.string, "Two")
self.assertEqual(pickle_doc.lists, ["1", "2", "3"])
def test_regular_document_pickle(self):
pickle_doc = PickleTest(number=1, string="One", lists=['1', '2'])
pickled_doc = pickle.dumps(pickle_doc) # make sure pickling works even before the doc is saved
pickle_doc.save()
pickled_doc = pickle.dumps(pickle_doc)
# Test that when a document's definition changes the new
# definition is used
fixtures.PickleTest = fixtures.NewDocumentPickleTest
resurrected = pickle.loads(pickled_doc)
self.assertEqual(resurrected.__class__,
fixtures.NewDocumentPickleTest)
self.assertEqual(resurrected._fields_ordered,
fixtures.NewDocumentPickleTest._fields_ordered)
self.assertNotEqual(resurrected._fields_ordered,
pickle_doc._fields_ordered)
# The local PickleTest is still a ref to the original
fixtures.PickleTest = PickleTest
def test_dynamic_document_pickle(self):
pickle_doc = PickleDynamicTest(
name="test", number=1, string="One", lists=['1', '2'])
pickle_doc.embedded = PickleDyanmicEmbedded(foo="Bar")
pickle_doc.embedded = PickleDynamicEmbedded(foo="Bar")
pickled_doc = pickle.dumps(pickle_doc) # make sure pickling works even before the doc is saved
pickle_doc.save()
@@ -2443,6 +2671,114 @@ class InstanceTest(unittest.TestCase):
group = Group.objects.first()
self.assertEqual("hello - default", group.name)
def test_load_undefined_fields(self):
class User(Document):
name = StringField()
User.drop_collection()
User._get_collection().save({
'name': 'John',
'foo': 'Bar',
'data': [1, 2, 3]
})
self.assertRaises(FieldDoesNotExist, User.objects.first)
def test_load_undefined_fields_with_strict_false(self):
class User(Document):
name = StringField()
meta = {'strict': False}
User.drop_collection()
User._get_collection().save({
'name': 'John',
'foo': 'Bar',
'data': [1, 2, 3]
})
user = User.objects.first()
self.assertEqual(user.name, 'John')
self.assertFalse(hasattr(user, 'foo'))
self.assertEqual(user._data['foo'], 'Bar')
self.assertFalse(hasattr(user, 'data'))
self.assertEqual(user._data['data'], [1, 2, 3])
def test_load_undefined_fields_on_embedded_document(self):
class Thing(EmbeddedDocument):
name = StringField()
class User(Document):
name = StringField()
thing = EmbeddedDocumentField(Thing)
User.drop_collection()
User._get_collection().save({
'name': 'John',
'thing': {
'name': 'My thing',
'foo': 'Bar',
'data': [1, 2, 3]
}
})
self.assertRaises(FieldDoesNotExist, User.objects.first)
def test_load_undefined_fields_on_embedded_document_with_strict_false_on_doc(self):
class Thing(EmbeddedDocument):
name = StringField()
class User(Document):
name = StringField()
thing = EmbeddedDocumentField(Thing)
meta = {'strict': False}
User.drop_collection()
User._get_collection().save({
'name': 'John',
'thing': {
'name': 'My thing',
'foo': 'Bar',
'data': [1, 2, 3]
}
})
self.assertRaises(FieldDoesNotExist, User.objects.first)
def test_load_undefined_fields_on_embedded_document_with_strict_false(self):
class Thing(EmbeddedDocument):
name = StringField()
meta = {'strict': False}
class User(Document):
name = StringField()
thing = EmbeddedDocumentField(Thing)
User.drop_collection()
User._get_collection().save({
'name': 'John',
'thing': {
'name': 'My thing',
'foo': 'Bar',
'data': [1, 2, 3]
}
})
user = User.objects.first()
self.assertEqual(user.name, 'John')
self.assertEqual(user.thing.name, 'My thing')
self.assertFalse(hasattr(user.thing, 'foo'))
self.assertEqual(user.thing._data['foo'], 'Bar')
self.assertFalse(hasattr(user.thing, 'data'))
self.assertEqual(user.thing._data['data'], [1, 2, 3])
def test_spaces_in_keys(self):
class Embedded(DynamicEmbeddedDocument):
@@ -2484,6 +2820,32 @@ class InstanceTest(unittest.TestCase):
self.assertRaises(OperationError, change_shard_key)
def test_shard_key_in_embedded_document(self):
class Foo(EmbeddedDocument):
foo = StringField()
class Bar(Document):
meta = {
'shard_key': ('foo.foo',)
}
foo = EmbeddedDocumentField(Foo)
bar = StringField()
foo_doc = Foo(foo='hello')
bar_doc = Bar(foo=foo_doc, bar='world')
bar_doc.save()
self.assertTrue(bar_doc.id is not None)
bar_doc.bar = 'baz'
bar_doc.save()
def change_shard_key():
bar_doc.foo.foo = 'something'
bar_doc.save()
self.assertRaises(OperationError, change_shard_key)
def test_shard_key_primary(self):
class LogEntry(Document):
machine = StringField(primary_key=True)
@@ -2566,6 +2928,20 @@ class InstanceTest(unittest.TestCase):
self.assertEqual(person.name, "Test User")
self.assertEqual(person.age, 42)
def test_positional_creation_embedded(self):
"""Ensure that embedded document may be created using positional arguments.
"""
job = self.Job("Test Job", 4)
self.assertEqual(job.name, "Test Job")
self.assertEqual(job.years, 4)
def test_mixed_creation_embedded(self):
"""Ensure that embedded document may be created using mixed arguments.
"""
job = self.Job("Test Job", years=4)
self.assertEqual(job.name, "Test Job")
self.assertEqual(job.years, 4)
def test_mixed_creation_dynamic(self):
"""Ensure that document may be created using mixed arguments.
"""
@@ -2742,6 +3118,17 @@ class InstanceTest(unittest.TestCase):
p4 = Person.objects()[0]
p4.save()
self.assertEquals(p4.height, 189)
# However the default will not be fixed in DB
self.assertEquals(Person.objects(height=189).count(), 0)
# alter DB for the new default
coll = Person._get_collection()
for person in Person.objects.as_pymongo():
if 'height' not in person:
person['height'] = 189
coll.save(person)
self.assertEquals(Person.objects(height=189).count(), 1)
def test_from_son(self):
@@ -2799,5 +3186,36 @@ class InstanceTest(unittest.TestCase):
self.assertNotEqual(p, p1)
self.assertEqual(p, p)
def test_list_iter(self):
# 914
class B(EmbeddedDocument):
v = StringField()
class A(Document):
l = ListField(EmbeddedDocumentField(B))
A.objects.delete()
A(l=[B(v='1'), B(v='2'), B(v='3')]).save()
a = A.objects.get()
self.assertEqual(a.l._instance, a)
for idx, b in enumerate(a.l):
self.assertEqual(b._instance, a)
self.assertEqual(idx, 2)
def test_falsey_pk(self):
"""Ensure that we can create and update a document with Falsey PK.
"""
class Person(Document):
age = IntField(primary_key=True)
height = FloatField()
person = Person()
person.age = 0
person.height = 1.89
person.save()
person.update(set__height=2.0)
if __name__ == '__main__':
unittest.main()

View File

@@ -165,6 +165,53 @@ class ValidatorErrorTest(unittest.TestCase):
self.assertRaises(ValidationError, lambda: d2.validate())
def test_parent_reference_in_child_document(self):
"""
Test to ensure a ReferenceField can store a reference to a parent
class when inherited. Issue #954.
"""
class Parent(Document):
meta = {'allow_inheritance': True}
reference = ReferenceField('self')
class Child(Parent):
pass
parent = Parent()
parent.save()
child = Child(reference=parent)
# Saving child should not raise a ValidationError
try:
child.save()
except ValidationError as e:
self.fail("ValidationError raised: %s" % e.message)
def test_parent_reference_set_as_attribute_in_child_document(self):
"""
Test to ensure a ReferenceField can store a reference to a parent
class when inherited and when set via attribute. Issue #954.
"""
class Parent(Document):
meta = {'allow_inheritance': True}
reference = ReferenceField('self')
class Child(Parent):
pass
parent = Parent()
parent.save()
child = Child()
child.reference = parent
# Saving the child should not raise a ValidationError
try:
child.save()
except ValidationError as e:
self.fail("ValidationError raised: %s" % e.message)
if __name__ == '__main__':
unittest.main()

View File

@@ -1,3 +1,3 @@
from fields import *
from file_tests import *
from geo import *
from geo import *

View File

@@ -1,10 +1,18 @@
# -*- coding: utf-8 -*-
import sys
import six
from nose.plugins.skip import SkipTest
sys.path[0:0] = [""]
import datetime
import unittest
import uuid
import math
import itertools
import re
import six
try:
import dateutil
@@ -14,12 +22,16 @@ except ImportError:
from decimal import Decimal
from bson import Binary, DBRef, ObjectId
try:
from bson.int64 import Int64
except ImportError:
Int64 = long
from mongoengine import *
from mongoengine.connection import get_db
from mongoengine.base import _document_registry
from mongoengine.base.datastructures import BaseDict, EmbeddedDocumentList
from mongoengine.errors import NotRegistered
from mongoengine.errors import NotRegistered, DoesNotExist
from mongoengine.python_support import PY3, b, bin_type
__all__ = ("FieldTest", "EmbeddedDocumentListFieldTestCase")
@@ -34,6 +46,7 @@ class FieldTest(unittest.TestCase):
def tearDown(self):
self.db.drop_collection('fs.files')
self.db.drop_collection('fs.chunks')
self.db.drop_collection('mongoengine.counters')
def test_default_values_nothing_set(self):
"""Ensure that default field values are used when creating a document.
@@ -336,6 +349,23 @@ class FieldTest(unittest.TestCase):
link.url = 'http://www.google.com:8080'
link.validate()
def test_url_scheme_validation(self):
"""Ensure that URLFields validate urls with specific schemes properly.
"""
class Link(Document):
url = URLField()
class SchemeLink(Document):
url = URLField(schemes=['ws', 'irc'])
link = Link()
link.url = 'ws://google.com'
self.assertRaises(ValidationError, link.validate)
scheme_link = SchemeLink()
scheme_link.url = 'ws://google.com'
scheme_link.validate()
def test_int_validation(self):
"""Ensure that invalid values cannot be assigned to int fields.
"""
@@ -376,20 +406,37 @@ class FieldTest(unittest.TestCase):
class Person(Document):
height = FloatField(min_value=0.1, max_value=3.5)
class BigPerson(Document):
height = FloatField()
person = Person()
person.height = 1.89
person.validate()
person.height = '2.0'
self.assertRaises(ValidationError, person.validate)
person.height = 0.01
self.assertRaises(ValidationError, person.validate)
person.height = 4.0
self.assertRaises(ValidationError, person.validate)
person_2 = Person(height='something invalid')
self.assertRaises(ValidationError, person_2.validate)
big_person = BigPerson()
for value, value_type in enumerate(six.integer_types):
big_person.height = value_type(value)
big_person.validate()
big_person.height = 2 ** 500
big_person.validate()
big_person.height = 2 ** 100000 # Too big for a float value
self.assertRaises(ValidationError, big_person.validate)
def test_decimal_validation(self):
"""Ensure that invalid values cannot be assigned to decimal fields.
"""
@@ -689,6 +736,7 @@ class FieldTest(unittest.TestCase):
"""
class LogEntry(Document):
date = ComplexDateTimeField()
date_with_dots = ComplexDateTimeField(separator='.')
LogEntry.drop_collection()
@@ -729,6 +777,18 @@ class FieldTest(unittest.TestCase):
log1 = LogEntry.objects.get(date=d1)
self.assertEqual(log, log1)
# Test string padding
microsecond = map(int, [math.pow(10, x) for x in xrange(6)])
mm = dd = hh = ii = ss = [1, 10]
for values in itertools.product([2014], mm, dd, hh, ii, ss, microsecond):
stored = LogEntry(date=datetime.datetime(*values)).to_mongo()['date']
self.assertTrue(re.match('^\d{4},\d{2},\d{2},\d{2},\d{2},\d{2},\d{6}$', stored) is not None)
# Test separator
stored = LogEntry(date_with_dots=datetime.datetime(2014, 1, 1)).to_mongo()['date_with_dots']
self.assertTrue(re.match('^\d{4}.\d{2}.\d{2}.\d{2}.\d{2}.\d{2}.\d{6}$', stored) is not None)
LogEntry.drop_collection()
def test_complexdatetime_usage(self):
@@ -787,6 +847,25 @@ class FieldTest(unittest.TestCase):
LogEntry.drop_collection()
# Test microsecond-level ordering/filtering
for microsecond in (99, 999, 9999, 10000):
LogEntry(date=datetime.datetime(2015, 1, 1, 0, 0, 0, microsecond)).save()
logs = list(LogEntry.objects.order_by('date'))
for next_idx, log in enumerate(logs[:-1], start=1):
next_log = logs[next_idx]
self.assertTrue(log.date < next_log.date)
logs = list(LogEntry.objects.order_by('-date'))
for next_idx, log in enumerate(logs[:-1], start=1):
next_log = logs[next_idx]
self.assertTrue(log.date > next_log.date)
logs = LogEntry.objects.filter(date__lte=datetime.datetime(2015, 1, 1, 0, 0, 0, 10000))
self.assertEqual(logs.count(), 4)
LogEntry.drop_collection()
def test_list_validation(self):
"""Ensure that a list field only accepts lists with valid elements.
"""
@@ -881,10 +960,17 @@ class FieldTest(unittest.TestCase):
self.assertEqual(post.comments[0].content, comment2.content)
self.assertEqual(post.comments[1].content, comment1.content)
post.comments[0].order = 2
post.save()
post.reload()
self.assertEqual(post.comments[0].content, comment1.content)
self.assertEqual(post.comments[1].content, comment2.content)
BlogPost.drop_collection()
def test_reverse_list_sorting(self):
'''Ensure that a reverse sorted list field properly sorts values'''
"""Ensure that a reverse sorted list field properly sorts values"""
class Category(EmbeddedDocument):
count = IntField()
@@ -946,8 +1032,68 @@ class FieldTest(unittest.TestCase):
BlogPost.objects.filter(info__0__test__exact='5').count(), 0)
self.assertEqual(
BlogPost.objects.filter(info__100__test__exact='test').count(), 0)
post = BlogPost()
post.info = ['1', '2']
post.save()
post = BlogPost.objects(info=['1', '2']).get()
post.info += ['3', '4']
post.save()
self.assertEqual(BlogPost.objects(info=['1', '2', '3', '4']).count(), 1)
post = BlogPost.objects(info=['1', '2', '3', '4']).get()
post.info *= 2
post.save()
self.assertEqual(BlogPost.objects(info=['1', '2', '3', '4', '1', '2', '3', '4']).count(), 1)
BlogPost.drop_collection()
def test_list_assignment(self):
"""Ensure that list field element assignment and slicing work
"""
class BlogPost(Document):
info = ListField()
BlogPost.drop_collection()
post = BlogPost()
post.info = ['e1', 'e2', 3, '4', 5]
post.save()
post.info[0] = 1
post.save()
post.reload()
self.assertEqual(post.info[0], 1)
post.info[1:3] = ['n2', 'n3']
post.save()
post.reload()
self.assertEqual(post.info, [1, 'n2', 'n3', '4', 5])
post.info[-1] = 'n5'
post.save()
post.reload()
self.assertEqual(post.info, [1, 'n2', 'n3', '4', 'n5'])
post.info[-2] = 4
post.save()
post.reload()
self.assertEqual(post.info, [1, 'n2', 'n3', 4, 'n5'])
post.info[1:-1] = [2]
post.save()
post.reload()
self.assertEqual(post.info, [1, 2, 'n5'])
post.info[:-1] = [1, 'n2', 'n3', 4]
post.save()
post.reload()
self.assertEqual(post.info, [1, 'n2', 'n3', 4, 'n5'])
post.info[-4:3] = [2, 3]
post.save()
post.reload()
self.assertEqual(post.info, [1, 2, 3, 4, 'n5'])
def test_list_field_passed_in_value(self):
class Foo(Document):
bars = ListField(ReferenceField("Bar"))
@@ -1062,6 +1208,19 @@ class FieldTest(unittest.TestCase):
simple = simple.reload()
self.assertEqual(simple.widgets, [4])
def test_list_field_with_negative_indices(self):
class Simple(Document):
widgets = ListField()
simple = Simple(widgets=[1, 2, 3, 4]).save()
simple.widgets[-1] = 5
self.assertEqual(['widgets.3'], simple._changed_fields)
simple.save()
simple = simple.reload()
self.assertEqual(simple.widgets, [1, 2, 3, 5])
def test_list_field_complex(self):
"""Ensure that the list fields can handle the complex types."""
@@ -1183,6 +1342,44 @@ class FieldTest(unittest.TestCase):
BlogPost.drop_collection()
def test_dictfield_dump_document(self):
"""Ensure a DictField can handle another document's dump
"""
class Doc(Document):
field = DictField()
class ToEmbed(Document):
id = IntField(primary_key=True, default=1)
recursive = DictField()
class ToEmbedParent(Document):
id = IntField(primary_key=True, default=1)
recursive = DictField()
meta = {'allow_inheritance': True}
class ToEmbedChild(ToEmbedParent):
pass
to_embed_recursive = ToEmbed(id=1).save()
to_embed = ToEmbed(
id=2, recursive=to_embed_recursive.to_mongo().to_dict()).save()
doc = Doc(field=to_embed.to_mongo().to_dict())
doc.save()
assert isinstance(doc.field, dict)
assert doc.field == {'_id': 2, 'recursive': {'_id': 1, 'recursive': {}}}
# Same thing with a Document with a _cls field
to_embed_recursive = ToEmbedChild(id=1).save()
to_embed_child = ToEmbedChild(
id=2, recursive=to_embed_recursive.to_mongo().to_dict()).save()
doc = Doc(field=to_embed_child.to_mongo().to_dict())
doc.save()
assert isinstance(doc.field, dict)
assert doc.field == {
'_id': 2, '_cls': 'ToEmbedParent.ToEmbedChild',
'recursive': {'_id': 1, '_cls': 'ToEmbedParent.ToEmbedChild', 'recursive': {}}
}
def test_dictfield_strict(self):
"""Ensure that dict field handles validation if provided a strict field type."""
@@ -1260,7 +1457,6 @@ class FieldTest(unittest.TestCase):
def test_atomic_update_dict_field(self):
"""Ensure that the entire DictField can be atomically updated."""
class Simple(Document):
mapping = DictField(field=ListField(IntField(required=True)))
@@ -1275,7 +1471,7 @@ class FieldTest(unittest.TestCase):
self.assertEqual({"ints": [3, 4]}, e.mapping)
def create_invalid_mapping():
e.update(set__mapping={"somestrings": ["foo", "bar",]})
e.update(set__mapping={"somestrings": ["foo", "bar", ]})
self.assertRaises(ValueError, create_invalid_mapping)
@@ -1384,16 +1580,49 @@ class FieldTest(unittest.TestCase):
def test_map_field_lookup(self):
"""Ensure MapField lookups succeed on Fields without a lookup method"""
class Action(EmbeddedDocument):
operation = StringField()
object = StringField()
class Log(Document):
name = StringField()
visited = MapField(DateTimeField())
actions = MapField(EmbeddedDocumentField(Action))
Log.drop_collection()
Log(name="wilson", visited={'friends': datetime.datetime.now()}).save()
Log(name="wilson", visited={'friends': datetime.datetime.now()},
actions={'friends': Action(operation='drink', object='beer')}).save()
self.assertEqual(1, Log.objects(
visited__friends__exists=True).count())
self.assertEqual(1, Log.objects(
actions__friends__operation='drink',
actions__friends__object='beer').count())
def test_map_field_unicode(self):
class Info(EmbeddedDocument):
description = StringField()
value_list = ListField(field=StringField())
class BlogPost(Document):
info_dict = MapField(field=EmbeddedDocumentField(Info))
BlogPost.drop_collection()
tree = BlogPost(info_dict={
u"éééé": {
'description': u"VALUE: éééé"
}
})
tree.save()
self.assertEqual(BlogPost.objects.get(id=tree.id).info_dict[u"éééé"].description, u"VALUE: éééé")
BlogPost.drop_collection()
def test_embedded_db_field(self):
class Embedded(EmbeddedDocument):
@@ -1430,6 +1659,8 @@ class FieldTest(unittest.TestCase):
name = StringField()
preferences = EmbeddedDocumentField(PersonPreferences)
Person.drop_collection()
person = Person(name='Test User')
person.preferences = 'My Preferences'
self.assertRaises(ValidationError, person.validate)
@@ -1462,12 +1693,70 @@ class FieldTest(unittest.TestCase):
content = StringField()
author = EmbeddedDocumentField(User)
BlogPost.drop_collection()
post = BlogPost(content='What I did today...')
post.author = PowerUser(name='Test User', power=47)
post.save()
self.assertEqual(47, BlogPost.objects.first().author.power)
def test_embedded_document_inheritance_with_list(self):
"""Ensure that nested list of subclassed embedded documents is
handled correctly.
"""
class Group(EmbeddedDocument):
name = StringField()
content = ListField(StringField())
class Basedoc(Document):
groups = ListField(EmbeddedDocumentField(Group))
meta = {'abstract': True}
class User(Basedoc):
doctype = StringField(require=True, default='userdata')
User.drop_collection()
content = ['la', 'le', 'lu']
group = Group(name='foo', content=content)
foobar = User(groups=[group])
foobar.save()
self.assertEqual(content, User.objects.first().groups[0].content)
def test_reference_miss(self):
"""Ensure an exception is raised when dereferencing unknow document
"""
class Foo(Document):
pass
class Bar(Document):
ref = ReferenceField(Foo)
generic_ref = GenericReferenceField()
Foo.drop_collection()
Bar.drop_collection()
foo = Foo().save()
bar = Bar(ref=foo, generic_ref=foo).save()
# Reference is no longer valid
foo.delete()
bar = Bar.objects.get()
self.assertRaises(DoesNotExist, lambda: getattr(bar, 'ref'))
self.assertRaises(DoesNotExist, lambda: getattr(bar, 'generic_ref'))
# When auto_dereference is disabled, there is no trouble returning DBRef
bar = Bar.objects.get()
expected = foo.to_dbref()
bar._fields['ref']._auto_dereference = False
self.assertEqual(bar.ref, expected)
bar._fields['generic_ref']._auto_dereference = False
self.assertEqual(bar.generic_ref, {'_ref': expected, '_cls': 'Foo'})
def test_reference_validation(self):
"""Ensure that invalid docment objects cannot be assigned to reference
fields.
@@ -1535,6 +1824,27 @@ class FieldTest(unittest.TestCase):
mongoed = p1.to_mongo()
self.assertTrue(isinstance(mongoed['parent'], ObjectId))
def test_cached_reference_field_get_and_save(self):
"""
Tests #1047: CachedReferenceField creates DBRefs on to_python, but can't save them on to_mongo
"""
class Animal(Document):
name = StringField()
tag = StringField()
class Ocorrence(Document):
person = StringField()
animal = CachedReferenceField(Animal)
Animal.drop_collection()
Ocorrence.drop_collection()
Ocorrence(person="testte",
animal=Animal(name="Leopard", tag="heavy").save()).save()
p = Ocorrence.objects.get()
p.person = 'new_testte'
p.save()
def test_cached_reference_fields(self):
class Animal(Document):
name = StringField()
@@ -2068,9 +2378,7 @@ class FieldTest(unittest.TestCase):
obj = Product.objects(company=None).first()
self.assertEqual(obj, me)
obj, created = Product.objects.get_or_create(company=None)
self.assertEqual(created, False)
obj = Product.objects.get(company=None)
self.assertEqual(obj, me)
def test_reference_query_conversion(self):
@@ -2141,6 +2449,91 @@ class FieldTest(unittest.TestCase):
Member.drop_collection()
BlogPost.drop_collection()
def test_drop_abstract_document(self):
"""Ensure that an abstract document cannot be dropped given it
has no underlying collection.
"""
class AbstractDoc(Document):
name = StringField()
meta = {"abstract": True}
self.assertRaises(OperationError, AbstractDoc.drop_collection)
def test_reference_class_with_abstract_parent(self):
"""Ensure that a class with an abstract parent can be referenced.
"""
class Sibling(Document):
name = StringField()
meta = {"abstract": True}
class Sister(Sibling):
pass
class Brother(Sibling):
sibling = ReferenceField(Sibling)
Sister.drop_collection()
Brother.drop_collection()
sister = Sister(name="Alice")
sister.save()
brother = Brother(name="Bob", sibling=sister)
brother.save()
self.assertEquals(Brother.objects[0].sibling.name, sister.name)
Sister.drop_collection()
Brother.drop_collection()
def test_reference_abstract_class(self):
"""Ensure that an abstract class instance cannot be used in the
reference of that abstract class.
"""
class Sibling(Document):
name = StringField()
meta = {"abstract": True}
class Sister(Sibling):
pass
class Brother(Sibling):
sibling = ReferenceField(Sibling)
Sister.drop_collection()
Brother.drop_collection()
sister = Sibling(name="Alice")
brother = Brother(name="Bob", sibling=sister)
self.assertRaises(ValidationError, brother.save)
Sister.drop_collection()
Brother.drop_collection()
def test_abstract_reference_base_type(self):
"""Ensure that an an abstract reference fails validation when given a
Document that does not inherit from the abstract type.
"""
class Sibling(Document):
name = StringField()
meta = {"abstract": True}
class Brother(Sibling):
sibling = ReferenceField(Sibling)
class Mother(Document):
name = StringField()
Brother.drop_collection()
Mother.drop_collection()
mother = Mother(name="Carol")
mother.save()
brother = Brother(name="Bob", sibling=mother)
self.assertRaises(ValidationError, brother.save)
Brother.drop_collection()
Mother.drop_collection()
def test_generic_reference(self):
"""Ensure that a GenericReferenceField properly dereferences items.
"""
@@ -2294,6 +2687,62 @@ class FieldTest(unittest.TestCase):
bm = Bookmark.objects.first()
self.assertEqual(bm.bookmark_object, post_1)
def test_generic_reference_string_choices(self):
"""Ensure that a GenericReferenceField can handle choices as strings
"""
class Link(Document):
title = StringField()
class Post(Document):
title = StringField()
class Bookmark(Document):
bookmark_object = GenericReferenceField(choices=('Post', Link))
Link.drop_collection()
Post.drop_collection()
Bookmark.drop_collection()
link_1 = Link(title="Pitchfork")
link_1.save()
post_1 = Post(title="Behind the Scenes of the Pavement Reunion")
post_1.save()
bm = Bookmark(bookmark_object=link_1)
bm.save()
bm = Bookmark(bookmark_object=post_1)
bm.save()
bm = Bookmark(bookmark_object=bm)
self.assertRaises(ValidationError, bm.validate)
def test_generic_reference_choices_no_dereference(self):
"""Ensure that a GenericReferenceField can handle choices on
non-derefenreced (i.e. DBRef) elements
"""
class Post(Document):
title = StringField()
class Bookmark(Document):
bookmark_object = GenericReferenceField(choices=(Post, ))
other_field = StringField()
Post.drop_collection()
Bookmark.drop_collection()
post_1 = Post(title="Behind the Scenes of the Pavement Reunion")
post_1.save()
bm = Bookmark(bookmark_object=post_1)
bm.save()
bm = Bookmark.objects.get(id=bm.id)
# bookmark_object is now a DBRef
bm.other_field = 'dummy_change'
bm.save()
def test_generic_reference_list_choices(self):
"""Ensure that a ListField properly dereferences generic references and
respects choices.
@@ -2424,10 +2873,29 @@ class FieldTest(unittest.TestCase):
id = BinaryField(primary_key=True)
Attachment.drop_collection()
att = Attachment(id=uuid.uuid4().bytes).save()
binary_id = uuid.uuid4().bytes
att = Attachment(id=binary_id).save()
self.assertEqual(1, Attachment.objects.count())
self.assertEqual(1, Attachment.objects.filter(id=att.id).count())
# TODO use assertIsNotNone once Python 2.6 support is dropped
self.assertTrue(Attachment.objects.filter(id=att.id).first() is not None)
att.delete()
self.assertEqual(0, Attachment.objects.count())
def test_binary_field_primary_filter_by_binary_pk_as_str(self):
raise SkipTest("Querying by id as string is not currently supported")
class Attachment(Document):
id = BinaryField(primary_key=True)
Attachment.drop_collection()
binary_id = uuid.uuid4().bytes
att = Attachment(id=binary_id).save()
self.assertEqual(1, Attachment.objects.filter(id=binary_id).count())
# TODO use assertIsNotNone once Python 2.6 support is dropped
self.assertTrue(Attachment.objects.filter(id=binary_id).first() is not None)
att.delete()
self.assertEqual(0, Attachment.objects.count())
def test_choices_validation(self):
@@ -2533,28 +3001,32 @@ class FieldTest(unittest.TestCase):
('S', 'Small'), ('M', 'Medium'), ('L', 'Large'),
('XL', 'Extra Large'), ('XXL', 'Extra Extra Large')))
style = StringField(max_length=3, choices=(
('S', 'Small'), ('B', 'Baggy'), ('W', 'wide')), default='S')
('S', 'Small'), ('B', 'Baggy'), ('W', 'Wide')), default='W')
Shirt.drop_collection()
shirt = Shirt()
shirt1 = Shirt()
shirt2 = Shirt()
self.assertEqual(shirt.get_size_display(), None)
self.assertEqual(shirt.get_style_display(), 'Small')
# Make sure get_<field>_display returns the default value (or None)
self.assertEqual(shirt1.get_size_display(), None)
self.assertEqual(shirt1.get_style_display(), 'Wide')
shirt.size = "XXL"
shirt.style = "B"
self.assertEqual(shirt.get_size_display(), 'Extra Extra Large')
self.assertEqual(shirt.get_style_display(), 'Baggy')
shirt1.size = 'XXL'
shirt1.style = 'B'
shirt2.size = 'M'
shirt2.style = 'S'
self.assertEqual(shirt1.get_size_display(), 'Extra Extra Large')
self.assertEqual(shirt1.get_style_display(), 'Baggy')
self.assertEqual(shirt2.get_size_display(), 'Medium')
self.assertEqual(shirt2.get_style_display(), 'Small')
# Set as Z - an invalid choice
shirt.size = "Z"
shirt.style = "Z"
self.assertEqual(shirt.get_size_display(), 'Z')
self.assertEqual(shirt.get_style_display(), 'Z')
self.assertRaises(ValidationError, shirt.validate)
Shirt.drop_collection()
shirt1.size = 'Z'
shirt1.style = 'Z'
self.assertEqual(shirt1.get_size_display(), 'Z')
self.assertEqual(shirt1.get_style_display(), 'Z')
self.assertRaises(ValidationError, shirt1.validate)
def test_simple_choices_validation(self):
"""Ensure that value is in a container of allowed values.
@@ -2869,6 +3341,57 @@ class FieldTest(unittest.TestCase):
self.assertEqual(1, post.comments[0].id)
self.assertEqual(2, post.comments[1].id)
def test_inherited_sequencefield(self):
class Base(Document):
name = StringField()
counter = SequenceField()
meta = {'abstract': True}
class Foo(Base):
pass
class Bar(Base):
pass
bar = Bar(name='Bar')
bar.save()
foo = Foo(name='Foo')
foo.save()
self.assertTrue('base.counter' in
self.db['mongoengine.counters'].find().distinct('_id'))
self.assertFalse(('foo.counter' or 'bar.counter') in
self.db['mongoengine.counters'].find().distinct('_id'))
self.assertNotEqual(foo.counter, bar.counter)
self.assertEqual(foo._fields['counter'].owner_document, Base)
self.assertEqual(bar._fields['counter'].owner_document, Base)
def test_no_inherited_sequencefield(self):
class Base(Document):
name = StringField()
meta = {'abstract': True}
class Foo(Base):
counter = SequenceField()
class Bar(Base):
counter = SequenceField()
bar = Bar(name='Bar')
bar.save()
foo = Foo(name='Foo')
foo.save()
self.assertFalse('base.counter' in
self.db['mongoengine.counters'].find().distinct('_id'))
self.assertTrue(('foo.counter' and 'bar.counter') in
self.db['mongoengine.counters'].find().distinct('_id'))
self.assertEqual(foo.counter, bar.counter)
self.assertEqual(foo._fields['counter'].owner_document, Foo)
self.assertEqual(bar._fields['counter'].owner_document, Bar)
def test_generic_embedded_document(self):
class Car(EmbeddedDocument):
name = StringField()
@@ -3003,7 +3526,6 @@ class FieldTest(unittest.TestCase):
self.assertTrue(user.validate() is None)
user = User(email=("Kofq@rhom0e4klgauOhpbpNdogawnyIKvQS0wk2mjqrgGQ5S"
"ucictfqpdkK9iS1zeFw8sg7s7cwAF7suIfUfeyueLpfosjn3"
"aJIazqqWkm7.net"))
self.assertTrue(user.validate() is None)
@@ -3100,6 +3622,39 @@ class FieldTest(unittest.TestCase):
doc = Doc.objects.get()
self.assertEqual(doc.embed_me.field_1, "hello")
def test_dynamicfield_dump_document(self):
"""Ensure a DynamicField can handle another document's dump
"""
class Doc(Document):
field = DynamicField()
class ToEmbed(Document):
id = IntField(primary_key=True, default=1)
recursive = DynamicField()
class ToEmbedParent(Document):
id = IntField(primary_key=True, default=1)
recursive = DynamicField()
meta = {'allow_inheritance': True}
class ToEmbedChild(ToEmbedParent):
pass
to_embed_recursive = ToEmbed(id=1).save()
to_embed = ToEmbed(id=2, recursive=to_embed_recursive).save()
doc = Doc(field=to_embed)
doc.save()
assert isinstance(doc.field, ToEmbed)
assert doc.field == to_embed
# Same thing with a Document with a _cls field
to_embed_recursive = ToEmbedChild(id=1).save()
to_embed_child = ToEmbedChild(id=2, recursive=to_embed_recursive).save()
doc = Doc(field=to_embed_child)
doc.save()
assert isinstance(doc.field, ToEmbedChild)
assert doc.field == to_embed_child
def test_invalid_dict_value(self):
class DictFieldTest(Document):
dictionary = DictField(required=True)
@@ -3148,7 +3703,7 @@ class FieldTest(unittest.TestCase):
def test_undefined_field_exception(self):
"""Tests if a `FieldDoesNotExist` exception is raised when trying to
set a value to a field that's not defined.
instanciate a document with a field that's not defined.
"""
class Doc(Document):
@@ -3159,6 +3714,34 @@ class FieldTest(unittest.TestCase):
self.assertRaises(FieldDoesNotExist, test)
def test_undefined_field_exception_with_strict(self):
"""Tests if a `FieldDoesNotExist` exception is raised when trying to
instanciate a document with a field that's not defined,
even when strict is set to False.
"""
class Doc(Document):
foo = StringField(db_field='f')
meta = {'strict': False}
def test():
Doc(bar='test')
self.assertRaises(FieldDoesNotExist, test)
def test_long_field_is_considered_as_int64(self):
"""
Tests that long fields are stored as long in mongo, even if long value
is small enough to be an int.
"""
class TestLongFieldConsideredAsInt64(Document):
some_long = LongField()
doc = TestLongFieldConsideredAsInt64(some_long=42).save()
db = get_db()
self.assertTrue(isinstance(db.test_long_field_considered_as_int64.find()[0]['some_long'], Int64))
self.assertTrue(isinstance(doc.some_long, six.integer_types))
class EmbeddedDocumentListFieldTestCase(unittest.TestCase):
@@ -3546,6 +4129,17 @@ class EmbeddedDocumentListFieldTestCase(unittest.TestCase):
# modified
self.assertEqual(number, 2)
def test_unicode(self):
"""
Tests that unicode strings handled correctly
"""
post = self.BlogPost(comments=[
self.Comments(author='user1', message=u'сообщение'),
self.Comments(author='user2', message=u'хабарлама')
]).save()
self.assertEqual(post.comments.get(message=u'сообщение').author,
'user1')
def test_save(self):
"""
Tests the save method of a List of Embedded Documents.
@@ -3596,6 +4190,30 @@ class EmbeddedDocumentListFieldTestCase(unittest.TestCase):
# deleted from the database
self.assertEqual(number, 2)
def test_empty_list_embedded_documents_with_unique_field(self):
"""
Tests that only one document with an empty list of embedded documents
that have a unique field can be saved, but if the unique field is
also sparse than multiple documents with an empty list can be saved.
"""
class EmbeddedWithUnique(EmbeddedDocument):
number = IntField(unique=True)
class A(Document):
my_list = ListField(EmbeddedDocumentField(EmbeddedWithUnique))
A(my_list=[]).save()
self.assertRaises(NotUniqueError, lambda: A(my_list=[]).save())
class EmbeddedWithSparseUnique(EmbeddedDocument):
number = IntField(unique=True, sparse=True)
class B(Document):
my_list = ListField(EmbeddedDocumentField(EmbeddedWithSparseUnique))
B(my_list=[]).save()
B(my_list=[]).save()
def test_filtered_delete(self):
"""
Tests the delete method of a List of Embedded Documents
@@ -3627,5 +4245,23 @@ class EmbeddedDocumentListFieldTestCase(unittest.TestCase):
# deleted from the database
self.assertEqual(number, 1)
def test_custom_data(self):
"""
Tests that custom data is saved in the field object
and doesn't interfere with the rest of field functionalities.
"""
custom_data = {'a': 'a_value', 'b': [1, 2]}
class CustomData(Document):
a_field = IntField()
c_field = IntField(custom_data=custom_data)
a1 = CustomData(a_field=1, c_field=2).save()
self.assertEqual(2, a1.c_field)
self.assertFalse(hasattr(a1.c_field, 'custom_data'))
self.assertTrue(hasattr(CustomData.c_field, 'custom_data'))
self.assertEqual(custom_data['a'], CustomData.c_field.custom_data['a'])
if __name__ == '__main__':
unittest.main()

View File

@@ -12,7 +12,7 @@ import gridfs
from nose.plugins.skip import SkipTest
from mongoengine import *
from mongoengine.connection import get_db
from mongoengine.python_support import PY3, b, StringIO
from mongoengine.python_support import b, StringIO
try:
from PIL import Image
@@ -112,7 +112,7 @@ class FileTest(unittest.TestCase):
result.the_file.delete()
# Ensure deleted file returns None
self.assertTrue(result.the_file.read() == None)
self.assertTrue(result.the_file.read() is None)
def test_file_fields_stream_after_none(self):
"""Ensure that a file field can be written to after it has been saved as
@@ -138,7 +138,7 @@ class FileTest(unittest.TestCase):
result = StreamFile.objects.first()
self.assertTrue(streamfile == result)
self.assertEqual(result.the_file.read(), text + more_text)
#self.assertEqual(result.the_file.content_type, content_type)
# self.assertEqual(result.the_file.content_type, content_type)
result.the_file.seek(0)
self.assertEqual(result.the_file.tell(), 0)
self.assertEqual(result.the_file.read(len(text)), text)
@@ -148,7 +148,7 @@ class FileTest(unittest.TestCase):
result.the_file.delete()
# Ensure deleted file returns None
self.assertTrue(result.the_file.read() == None)
self.assertTrue(result.the_file.read() is None)
def test_file_fields_set(self):
@@ -297,6 +297,71 @@ class FileTest(unittest.TestCase):
test_file = TestFile()
self.assertFalse(test_file.the_file in [{"test": 1}])
def test_file_disk_space(self):
""" Test disk space usage when we delete/replace a file """
class TestFile(Document):
the_file = FileField()
text = b('Hello, World!')
content_type = 'text/plain'
testfile = TestFile()
testfile.the_file.put(text, content_type=content_type, filename="hello")
testfile.save()
# Now check fs.files and fs.chunks
db = TestFile._get_db()
files = db.fs.files.find()
chunks = db.fs.chunks.find()
self.assertEquals(len(list(files)), 1)
self.assertEquals(len(list(chunks)), 1)
# Deleting the docoument should delete the files
testfile.delete()
files = db.fs.files.find()
chunks = db.fs.chunks.find()
self.assertEquals(len(list(files)), 0)
self.assertEquals(len(list(chunks)), 0)
# Test case where we don't store a file in the first place
testfile = TestFile()
testfile.save()
files = db.fs.files.find()
chunks = db.fs.chunks.find()
self.assertEquals(len(list(files)), 0)
self.assertEquals(len(list(chunks)), 0)
testfile.delete()
files = db.fs.files.find()
chunks = db.fs.chunks.find()
self.assertEquals(len(list(files)), 0)
self.assertEquals(len(list(chunks)), 0)
# Test case where we overwrite the file
testfile = TestFile()
testfile.the_file.put(text, content_type=content_type, filename="hello")
testfile.save()
text = b('Bonjour, World!')
testfile.the_file.replace(text, content_type=content_type, filename="hello")
testfile.save()
files = db.fs.files.find()
chunks = db.fs.chunks.find()
self.assertEquals(len(list(files)), 1)
self.assertEquals(len(list(chunks)), 1)
testfile.delete()
files = db.fs.files.find()
chunks = db.fs.chunks.find()
self.assertEquals(len(list(files)), 0)
self.assertEquals(len(list(chunks)), 0)
def test_image_field(self):
if not HAS_PIL:
raise SkipTest('PIL not installed')

View File

@@ -115,7 +115,7 @@ class GeoFieldTest(unittest.TestCase):
expected = "Invalid LineString:\nBoth values (%s) in point must be float or int" % repr(coord[0])
self._test_for_expected_error(Location, coord, expected)
Location(loc=[[1, 2], [3, 4], [5, 6], [1,2]]).validate()
Location(loc=[[1, 2], [3, 4], [5, 6], [1, 2]]).validate()
def test_polygon_validation(self):
class Location(Document):
@@ -226,7 +226,7 @@ class GeoFieldTest(unittest.TestCase):
expected = "Invalid MultiLineString:\nBoth values (%s) in point must be float or int" % repr(coord[0][0])
self._test_for_expected_error(Location, coord, expected)
Location(loc=[[[1, 2], [3, 4], [5, 6], [1,2]]]).validate()
Location(loc=[[[1, 2], [3, 4], [5, 6], [1, 2]]]).validate()
def test_multipolygon_validation(self):
class Location(Document):
@@ -336,12 +336,11 @@ class GeoFieldTest(unittest.TestCase):
Location.drop_collection()
Parent.drop_collection()
list(Parent.objects)
collection = Parent._get_collection()
info = collection.index_information()
Parent(name='Berlin').save()
info = Parent._get_collection().index_information()
self.assertFalse('location_2d' in info)
info = Location._get_collection().index_information()
self.assertTrue('location_2d' in info)
self.assertEqual(len(Parent._geo_indices()), 0)
self.assertEqual(len(Location._geo_indices()), 1)

View File

@@ -17,7 +17,16 @@ class PickleTest(Document):
photo = FileField()
class PickleDyanmicEmbedded(DynamicEmbeddedDocument):
class NewDocumentPickleTest(Document):
number = IntField()
string = StringField(choices=(('One', '1'), ('Two', '2')))
embedded = EmbeddedDocumentField(PickleEmbedded)
lists = ListField(StringField())
photo = FileField()
new_field = StringField()
class PickleDynamicEmbedded(DynamicEmbeddedDocument):
date = DateTimeField(default=datetime.now)

View File

@@ -1,8 +1,11 @@
import unittest
from convert_to_new_inheritance_model import *
from decimalfield_as_float import *
from refrencefield_dbref_to_object_id import *
from referencefield_dbref_to_object_id import *
from turn_off_inheritance import *
from uuidfield_to_binary import *
if __name__ == '__main__':
unittest.main()

View File

@@ -1,12 +1,16 @@
import sys
sys.path[0:0] = [""]
import unittest
from datetime import datetime, timedelta
from mongoengine import *
from pymongo.errors import OperationFailure
from mongoengine import *
from mongoengine.connection import get_connection
from nose.plugins.skip import SkipTest
__all__ = ("GeoQueriesTest",)
@@ -66,6 +70,16 @@ class GeoQueriesTest(unittest.TestCase):
self.assertEqual(events.count(), 1)
self.assertEqual(events[0], event2)
# find events at least 10 degrees away of san francisco
point = [-122.415579, 37.7566023]
events = Event.objects(location__near=point, location__min_distance=10)
# The following real test passes on MongoDB 3 but minDistance seems
# buggy on older MongoDB versions
if get_connection().server_info()['versionArray'][0] > 2:
self.assertEqual(events.count(), 2)
else:
self.assertTrue(events.count() >= 2)
# find events within 10 degrees of san francisco
point_and_distance = [[-122.415579, 37.7566023], 10]
events = Event.objects(location__within_distance=point_and_distance)
@@ -141,7 +155,13 @@ class GeoQueriesTest(unittest.TestCase):
def test_spherical_geospatial_operators(self):
"""Ensure that spherical geospatial queries are working
"""
raise SkipTest("https://jira.mongodb.org/browse/SERVER-14039")
# Needs MongoDB > 2.6.4 https://jira.mongodb.org/browse/SERVER-14039
connection = get_connection()
info = connection.test.command('buildInfo')
mongodb_version = tuple([int(i) for i in info['version'].split('.')])
if mongodb_version < (2, 6, 4):
raise SkipTest("Need MongoDB version 2.6.4+")
class Point(Document):
location = GeoPointField()
@@ -161,7 +181,7 @@ class GeoQueriesTest(unittest.TestCase):
# Same behavior for _within_spherical_distance
points = Point.objects(
location__within_spherical_distance=[[-122, 37.5], 60/earth_radius]
location__within_spherical_distance=[[-122, 37.5], 60 / earth_radius]
)
self.assertEqual(points.count(), 2)
@@ -169,6 +189,24 @@ class GeoQueriesTest(unittest.TestCase):
location__max_distance=60 / earth_radius)
self.assertEqual(points.count(), 2)
# Test query works with max_distance, being farer from one point
points = Point.objects(location__near_sphere=[-122, 37.8],
location__max_distance=60 / earth_radius)
close_point = points.first()
self.assertEqual(points.count(), 1)
# Test query works with min_distance, being farer from one point
points = Point.objects(location__near_sphere=[-122, 37.8],
location__min_distance=60 / earth_radius)
# The following real test passes on MongoDB 3 but minDistance seems
# buggy on older MongoDB versions
if get_connection().server_info()['versionArray'][0] > 2:
self.assertEqual(points.count(), 1)
far_point = points.first()
self.assertNotEqual(close_point, far_point)
else:
self.assertTrue(points.count() >= 1)
# Finds both points, but orders the north point first because it's
# closer to the reference point to the north.
points = Point.objects(location__near_sphere=[-122, 38.5])
@@ -251,6 +289,20 @@ class GeoQueriesTest(unittest.TestCase):
self.assertEqual(events.count(), 2)
self.assertEqual(events[0], event3)
# ensure min_distance and max_distance combine well
events = Event.objects(location__near=[-87.67892, 41.9120459],
location__min_distance=1000,
location__max_distance=10000).order_by("-date")
self.assertEqual(events.count(), 1)
self.assertEqual(events[0], event3)
# ensure ordering is respected by "near"
events = Event.objects(location__near=[-87.67892, 41.9120459],
# location__min_distance=10000
location__min_distance=10000).order_by("-date")
self.assertEqual(events.count(), 1)
self.assertEqual(events[0], event2)
# check that within_box works
box = [(-125.0, 35.0), (-100.0, 40.0)]
events = Event.objects(location__geo_within_box=box)

View File

@@ -0,0 +1,78 @@
import pickle
import unittest
from pymongo.mongo_client import MongoClient
from mongoengine import Document, StringField, IntField
from mongoengine.connection import connect
__author__ = 'stas'
class Person(Document):
name = StringField()
age = IntField()
class TestQuerysetPickable(unittest.TestCase):
"""
Test for adding pickling support for QuerySet instances
See issue https://github.com/MongoEngine/mongoengine/issues/442
"""
def setUp(self):
super(TestQuerysetPickable, self).setUp()
connection = connect(db="test") #type: pymongo.mongo_client.MongoClient
connection.drop_database("test")
self.john = Person.objects.create(
name="John",
age=21
)
def test_picke_simple_qs(self):
qs = Person.objects.all()
pickle.dumps(qs)
def _get_loaded(self, qs):
s = pickle.dumps(qs)
return pickle.loads(s)
def test_unpickle(self):
qs = Person.objects.all()
loadedQs = self._get_loaded(qs)
self.assertEqual(qs.count(), loadedQs.count())
#can update loadedQs
loadedQs.update(age=23)
#check
self.assertEqual(Person.objects.first().age, 23)
def test_pickle_support_filtration(self):
Person.objects.create(
name="Alice",
age=22
)
Person.objects.create(
name="Bob",
age=23
)
qs = Person.objects.filter(age__gte=22)
self.assertEqual(qs.count(), 2)
loaded = self._get_loaded(qs)
self.assertEqual(loaded.count(), 2)
self.assertEqual(loaded.filter(name="Bob").first().age, 23)

File diff suppressed because it is too large Load Diff

View File

@@ -1,11 +1,7 @@
import sys
sys.path[0:0] = [""]
import unittest
from mongoengine import *
from mongoengine.queryset import Q
from mongoengine.queryset import transform
from mongoengine.queryset import Q, transform
__all__ = ("TransformTest",)
@@ -41,8 +37,8 @@ class TransformTest(unittest.TestCase):
DicDoc.drop_collection()
Doc.drop_collection()
DicDoc().save()
doc = Doc().save()
dic_doc = DicDoc().save()
for k, v in (("set", "$set"), ("set_on_insert", "$setOnInsert"), ("push", "$push")):
update = transform.update(DicDoc, **{"%s__dictField__test" % k: doc})
@@ -55,7 +51,6 @@ class TransformTest(unittest.TestCase):
update = transform.update(DicDoc, pull__dictField__test=doc)
self.assertTrue(isinstance(update["$pull"]["dictField"]["test"], dict))
def test_query_field_name(self):
"""Ensure that the correct field name is used when querying.
"""
@@ -156,26 +151,33 @@ class TransformTest(unittest.TestCase):
class Doc(Document):
meta = {'allow_inheritance': False}
raw_query = Doc.objects(__raw__={'deleted': False,
'scraped': 'yes',
'$nor': [{'views.extracted': 'no'},
{'attachments.views.extracted':'no'}]
})._query
raw_query = Doc.objects(__raw__={
'deleted': False,
'scraped': 'yes',
'$nor': [
{'views.extracted': 'no'},
{'attachments.views.extracted': 'no'}
]
})._query
expected = {'deleted': False, 'scraped': 'yes',
'$nor': [{'views.extracted': 'no'},
{'attachments.views.extracted': 'no'}]}
self.assertEqual(expected, raw_query)
self.assertEqual(raw_query, {
'deleted': False,
'scraped': 'yes',
'$nor': [
{'views.extracted': 'no'},
{'attachments.views.extracted': 'no'}
]
})
def test_geojson_PointField(self):
class Location(Document):
loc = PointField()
update = transform.update(Location, set__loc=[1, 2])
self.assertEqual(update, {'$set': {'loc': {"type": "Point", "coordinates": [1,2]}}})
self.assertEqual(update, {'$set': {'loc': {"type": "Point", "coordinates": [1, 2]}}})
update = transform.update(Location, set__loc={"type": "Point", "coordinates": [1,2]})
self.assertEqual(update, {'$set': {'loc': {"type": "Point", "coordinates": [1,2]}}})
update = transform.update(Location, set__loc={"type": "Point", "coordinates": [1, 2]})
self.assertEqual(update, {'$set': {'loc': {"type": "Point", "coordinates": [1, 2]}}})
def test_geojson_LineStringField(self):
class Location(Document):
@@ -208,6 +210,36 @@ class TransformTest(unittest.TestCase):
self.assertEqual(Doc.objects(df__type=2).count(), 1) # str
self.assertEqual(Doc.objects(df__type=16).count(), 1) # int
def test_last_field_name_like_operator(self):
class EmbeddedItem(EmbeddedDocument):
type = StringField()
name = StringField()
class Doc(Document):
item = EmbeddedDocumentField(EmbeddedItem)
Doc.drop_collection()
doc = Doc(item=EmbeddedItem(type="axe", name="Heroic axe"))
doc.save()
self.assertEqual(1, Doc.objects(item__type__="axe").count())
self.assertEqual(1, Doc.objects(item__name__="Heroic axe").count())
Doc.objects(id=doc.id).update(set__item__type__='sword')
self.assertEqual(1, Doc.objects(item__type__="sword").count())
self.assertEqual(0, Doc.objects(item__type__="axe").count())
def test_understandable_error_raised(self):
class Event(Document):
title = StringField()
location = GeoPointField()
box = [(35.0, -125.0), (40.0, -100.0)]
# I *meant* to execute location__within_box=box
events = Event.objects(location__within=box)
self.assertRaises(InvalidQueryError, lambda: events.count())
if __name__ == '__main__':
unittest.main()

View File

@@ -1,14 +1,12 @@
import sys
sys.path[0:0] = [""]
import datetime
import re
import unittest
from bson import ObjectId
from datetime import datetime
from mongoengine import *
from mongoengine.queryset import Q
from mongoengine.errors import InvalidQueryError
from mongoengine.queryset import Q
__all__ = ("QTest",)
@@ -132,12 +130,12 @@ class QTest(unittest.TestCase):
TestDoc(x=10).save()
TestDoc(y=True).save()
self.assertEqual(query,
{'$and': [
{'$or': [{'x': {'$gt': 0}}, {'x': {'$exists': False}}]},
{'$or': [{'x': {'$lt': 100}}, {'y': True}]}
]})
self.assertEqual(query, {
'$and': [
{'$or': [{'x': {'$gt': 0}}, {'x': {'$exists': False}}]},
{'$or': [{'x': {'$lt': 100}}, {'y': True}]}
]
})
self.assertEqual(2, TestDoc.objects(q1 & q2).count())
def test_or_and_or_combination(self):
@@ -157,15 +155,14 @@ class QTest(unittest.TestCase):
q2 = (Q(x__lt=100) & (Q(y=False) | Q(y__exists=False)))
query = (q1 | q2).to_query(TestDoc)
self.assertEqual(query,
{'$or': [
self.assertEqual(query, {
'$or': [
{'$and': [{'x': {'$gt': 0}},
{'$or': [{'y': True}, {'y': {'$exists': False}}]}]},
{'$and': [{'x': {'$lt': 100}},
{'$or': [{'y': False}, {'y': {'$exists': False}}]}]}
]}
)
]
})
self.assertEqual(2, TestDoc.objects(q1 | q2).count())
def test_multiple_occurence_in_field(self):
@@ -215,19 +212,19 @@ class QTest(unittest.TestCase):
BlogPost.drop_collection()
post1 = BlogPost(title='Test 1', publish_date=datetime(2010, 1, 8), published=False)
post1 = BlogPost(title='Test 1', publish_date=datetime.datetime(2010, 1, 8), published=False)
post1.save()
post2 = BlogPost(title='Test 2', publish_date=datetime(2010, 1, 15), published=True)
post2 = BlogPost(title='Test 2', publish_date=datetime.datetime(2010, 1, 15), published=True)
post2.save()
post3 = BlogPost(title='Test 3', published=True)
post3.save()
post4 = BlogPost(title='Test 4', publish_date=datetime(2010, 1, 8))
post4 = BlogPost(title='Test 4', publish_date=datetime.datetime(2010, 1, 8))
post4.save()
post5 = BlogPost(title='Test 1', publish_date=datetime(2010, 1, 15))
post5 = BlogPost(title='Test 1', publish_date=datetime.datetime(2010, 1, 15))
post5.save()
post6 = BlogPost(title='Test 1', published=False)
@@ -250,7 +247,7 @@ class QTest(unittest.TestCase):
self.assertTrue(all(obj.id in posts for obj in published_posts))
# Check Q object combination
date = datetime(2010, 1, 10)
date = datetime.datetime(2010, 1, 10)
q = BlogPost.objects(Q(publish_date__lte=date) | Q(published=True))
posts = [post.id for post in q]
@@ -273,8 +270,10 @@ class QTest(unittest.TestCase):
# Test invalid query objs
def wrong_query_objs():
self.Person.objects('user1')
def wrong_query_objs_filter():
self.Person.objects('user1')
self.assertRaises(InvalidQueryError, wrong_query_objs)
self.assertRaises(InvalidQueryError, wrong_query_objs_filter)
@@ -284,7 +283,6 @@ class QTest(unittest.TestCase):
person = self.Person(name='Guido van Rossum')
person.save()
import re
obj = self.Person.objects(Q(name=re.compile('^Gui'))).first()
self.assertEqual(obj, person)
obj = self.Person.objects(Q(name=re.compile('^gui'))).first()

View File

@@ -1,21 +1,34 @@
import sys
import datetime
from pymongo.errors import OperationFailure
sys.path[0:0] = [""]
try:
import unittest2 as unittest
except ImportError:
import unittest
import datetime
from nose.plugins.skip import SkipTest
import pymongo
from bson.tz_util import utc
from mongoengine import *
from mongoengine import (
connect, register_connection,
Document, DateTimeField
)
from mongoengine.python_support import IS_PYMONGO_3
import mongoengine.connection
from mongoengine.connection import get_db, get_connection, ConnectionError
def get_tz_awareness(connection):
if not IS_PYMONGO_3:
return connection.tz_aware
else:
return connection.codec_options.tz_aware
class ConnectionTest(unittest.TestCase):
def tearDown(self):
@@ -39,15 +52,99 @@ class ConnectionTest(unittest.TestCase):
conn = get_connection('testdb')
self.assertTrue(isinstance(conn, pymongo.mongo_client.MongoClient))
def test_connect_in_mocking(self):
"""Ensure that the connect() method works properly in mocking.
"""
try:
import mongomock
except ImportError:
raise SkipTest('you need mongomock installed to run this testcase')
connect('mongoenginetest', host='mongomock://localhost')
conn = get_connection()
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect('mongoenginetest2', host='mongomock://localhost', alias='testdb2')
conn = get_connection('testdb2')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect('mongoenginetest3', host='mongodb://localhost', is_mock=True, alias='testdb3')
conn = get_connection('testdb3')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect('mongoenginetest4', is_mock=True, alias='testdb4')
conn = get_connection('testdb4')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host='mongodb://localhost:27017/mongoenginetest5', is_mock=True, alias='testdb5')
conn = get_connection('testdb5')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host='mongomock://localhost:27017/mongoenginetest6', alias='testdb6')
conn = get_connection('testdb6')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host='mongomock://localhost:27017/mongoenginetest7', is_mock=True, alias='testdb7')
conn = get_connection('testdb7')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
def test_connect_with_host_list(self):
"""Ensure that the connect() method works when host is a list
Uses mongomock to test w/o needing multiple mongod/mongos processes
"""
try:
import mongomock
except ImportError:
raise SkipTest('you need mongomock installed to run this testcase')
connect(host=['mongomock://localhost'])
conn = get_connection()
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host=['mongodb://localhost'], is_mock=True, alias='testdb2')
conn = get_connection('testdb2')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host=['localhost'], is_mock=True, alias='testdb3')
conn = get_connection('testdb3')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host=['mongomock://localhost:27017', 'mongomock://localhost:27018'], alias='testdb4')
conn = get_connection('testdb4')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host=['mongodb://localhost:27017', 'mongodb://localhost:27018'], is_mock=True, alias='testdb5')
conn = get_connection('testdb5')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
connect(host=['localhost:27017', 'localhost:27018'], is_mock=True, alias='testdb6')
conn = get_connection('testdb6')
self.assertTrue(isinstance(conn, mongomock.MongoClient))
def test_disconnect(self):
"""Ensure that the disconnect() method works properly
"""
conn1 = connect('mongoenginetest')
mongoengine.connection.disconnect()
conn2 = connect('mongoenginetest')
self.assertTrue(conn1 is not conn2)
def test_sharing_connections(self):
"""Ensure that connections are shared when the connection settings are exactly the same
"""
connect('mongoenginetest', alias='testdb1')
connect('mongoenginetests', alias='testdb1')
expected_connection = get_connection('testdb1')
connect('mongoenginetest', alias='testdb2')
connect('mongoenginetests', alias='testdb2')
actual_connection = get_connection('testdb2')
# Handle PyMongo 3+ Async Connection
if IS_PYMONGO_3:
# Ensure we are connected, throws ServerSelectionTimeoutError otherwise.
# Purposely not catching exception to fail test if thrown.
expected_connection.server_info()
self.assertEqual(expected_connection, actual_connection)
def test_connect_uri(self):
@@ -61,7 +158,8 @@ class ConnectionTest(unittest.TestCase):
c.admin.authenticate("admin", "password")
c.mongoenginetest.add_user("username", "password")
self.assertRaises(ConnectionError, connect, "testdb_uri_bad", host='mongodb://test:password@localhost')
if not IS_PYMONGO_3:
self.assertRaises(ConnectionError, connect, "testdb_uri_bad", host='mongodb://test:password@localhost')
connect("testdb_uri", host='mongodb://username:password@localhost/mongoenginetest')
@@ -76,8 +174,7 @@ class ConnectionTest(unittest.TestCase):
c.mongoenginetest.system.users.remove({})
def test_connect_uri_without_db(self):
"""Ensure that the connect() method works properly with uri's
without database_name
"""Ensure connect() method works properly with uri's without database_name
"""
c = connect(db='mongoenginetest', alias='admin')
c.admin.system.users.remove({})
@@ -87,7 +184,8 @@ class ConnectionTest(unittest.TestCase):
c.admin.authenticate("admin", "password")
c.mongoenginetest.add_user("username", "password")
self.assertRaises(ConnectionError, connect, "testdb_uri_bad", host='mongodb://test:password@localhost')
if not IS_PYMONGO_3:
self.assertRaises(ConnectionError, connect, "testdb_uri_bad", host='mongodb://test:password@localhost')
connect("mongoenginetest", host='mongodb://localhost/')
@@ -101,6 +199,42 @@ class ConnectionTest(unittest.TestCase):
c.admin.system.users.remove({})
c.mongoenginetest.system.users.remove({})
def test_connect_uri_with_authsource(self):
"""Ensure that the connect() method works well with
the option `authSource` in URI.
This feature was introduced in MongoDB 2.4 and removed in 2.6
"""
# Create users
c = connect('mongoenginetest')
c.admin.system.users.remove({})
c.admin.add_user('username2', 'password')
# Authentication fails without "authSource"
if IS_PYMONGO_3:
test_conn = connect('mongoenginetest', alias='test1',
host='mongodb://username2:password@localhost/mongoenginetest')
self.assertRaises(OperationFailure, test_conn.server_info)
else:
self.assertRaises(
ConnectionError, connect, 'mongoenginetest', alias='test1',
host='mongodb://username2:password@localhost/mongoenginetest'
)
self.assertRaises(ConnectionError, get_db, 'test1')
# Authentication succeeds with "authSource"
connect(
'mongoenginetest', alias='test2',
host=('mongodb://username2:password@localhost/'
'mongoenginetest?authSource=admin')
)
# This will fail starting from MongoDB 2.6+
db = get_db('test2')
self.assertTrue(isinstance(db, pymongo.database.Database))
self.assertEqual(db.name, 'mongoenginetest')
# Clear all users
c.admin.system.users.remove({})
def test_register_connection(self):
"""Ensure that connections with different aliases may be registered.
"""
@@ -128,11 +262,11 @@ class ConnectionTest(unittest.TestCase):
connect('mongoenginetest', alias='t1', tz_aware=True)
conn = get_connection('t1')
self.assertTrue(conn.tz_aware)
self.assertTrue(get_tz_awareness(conn))
connect('mongoenginetest2', alias='t2')
conn = get_connection('t2')
self.assertFalse(conn.tz_aware)
self.assertFalse(get_tz_awareness(conn))
def test_datetime(self):
connect('mongoenginetest', tz_aware=True)
@@ -156,8 +290,17 @@ class ConnectionTest(unittest.TestCase):
self.assertEqual(len(mongo_connections.items()), 2)
self.assertTrue('t1' in mongo_connections.keys())
self.assertTrue('t2' in mongo_connections.keys())
self.assertEqual(mongo_connections['t1'].host, 'localhost')
self.assertEqual(mongo_connections['t2'].host, '127.0.0.1')
if not IS_PYMONGO_3:
self.assertEqual(mongo_connections['t1'].host, 'localhost')
self.assertEqual(mongo_connections['t2'].host, '127.0.0.1')
else:
# Handle PyMongo 3+ Async Connection
# Ensure we are connected, throws ServerSelectionTimeoutError otherwise.
# Purposely not catching exception to fail test if thrown.
mongo_connections['t1'].server_info()
mongo_connections['t2'].server_info()
self.assertEqual(mongo_connections['t1'].address[0], 'localhost')
self.assertEqual(mongo_connections['t2'].address[0], '127.0.0.1')
if __name__ == '__main__':

View File

@@ -1,18 +1,30 @@
import unittest
from mongoengine.base.datastructures import StrictDict, SemiStrictDict
from mongoengine.base.datastructures import StrictDict, SemiStrictDict
class TestStrictDict(unittest.TestCase):
def strict_dict_class(self, *args, **kwargs):
return StrictDict.create(*args, **kwargs)
def setUp(self):
self.dtype = self.strict_dict_class(("a", "b", "c"))
def test_init(self):
d = self.dtype(a=1, b=1, c=1)
self.assertEqual((d.a, d.b, d.c), (1, 1, 1))
def test_repr(self):
d = self.dtype(a=1, b=2, c=3)
self.assertEqual(repr(d), '{"a": 1, "b": 2, "c": 3}')
# make sure quotes are escaped properly
d = self.dtype(a='"', b="'", c="")
self.assertEqual(repr(d), '{"a": \'"\', "b": "\'", "c": \'\'}')
def test_init_fails_on_nonexisting_attrs(self):
self.assertRaises(AttributeError, lambda: self.dtype(a=1, b=2, d=3))
def test_eq(self):
d = self.dtype(a=1, b=1, c=1)
dd = self.dtype(a=1, b=1, c=1)
@@ -21,7 +33,7 @@ class TestStrictDict(unittest.TestCase):
g = self.strict_dict_class(("a", "b", "c", "d"))(a=1, b=1, c=1, d=1)
h = self.strict_dict_class(("a", "c", "b"))(a=1, b=1, c=1)
i = self.strict_dict_class(("a", "c", "b"))(a=1, b=1, c=2)
self.assertEqual(d, dd)
self.assertNotEqual(d, e)
self.assertNotEqual(d, f)
@@ -35,18 +47,19 @@ class TestStrictDict(unittest.TestCase):
d.a = 1
self.assertEqual(d.a, 1)
self.assertRaises(AttributeError, lambda: d.b)
def test_setattr_raises_on_nonexisting_attr(self):
d = self.dtype()
def _f():
d.x=1
d.x = 1
self.assertRaises(AttributeError, _f)
def test_setattr_getattr_special(self):
d = self.strict_dict_class(["items"])
d.items = 1
self.assertEqual(d.items, 1)
def test_get(self):
d = self.dtype(a=1)
self.assertEqual(d.get('a'), 1)
@@ -84,7 +97,7 @@ class TestSemiSrictDict(TestStrictDict):
def test_init_succeeds_with_nonexisting_attrs(self):
d = self.dtype(a=1, b=1, c=1, x=2)
self.assertEqual((d.a, d.b, d.c, d.x), (1, 1, 1, 2))
def test_iter_with_nonexisting_attrs(self):
d = self.dtype(a=1, b=1, c=1, x=2)
self.assertEqual(list(d), ['a', 'b', 'c', 'x'])

View File

@@ -12,9 +12,13 @@ from mongoengine.context_managers import query_counter
class FieldTest(unittest.TestCase):
def setUp(self):
connect(db='mongoenginetest')
self.db = get_db()
@classmethod
def setUpClass(cls):
cls.db = connect(db='mongoenginetest')
@classmethod
def tearDownClass(cls):
cls.db.drop_database('mongoenginetest')
def test_list_item_dereference(self):
"""Ensure that DBRef items in ListFields are dereferenced.
@@ -304,6 +308,7 @@ class FieldTest(unittest.TestCase):
User.drop_collection()
Post.drop_collection()
SimpleList.drop_collection()
u1 = User.objects.create(name='u1')
u2 = User.objects.create(name='u2')
@@ -1026,6 +1031,43 @@ class FieldTest(unittest.TestCase):
self.assertEqual(type(foo.bar), Bar)
self.assertEqual(type(foo.baz), Baz)
def test_document_reload_reference_integrity(self):
"""
Ensure reloading a document with multiple similar id
in different collections doesn't mix them.
"""
class Topic(Document):
id = IntField(primary_key=True)
class User(Document):
id = IntField(primary_key=True)
name = StringField()
class Message(Document):
id = IntField(primary_key=True)
topic = ReferenceField(Topic)
author = ReferenceField(User)
Topic.drop_collection()
User.drop_collection()
Message.drop_collection()
# All objects share the same id, but each in a different collection
topic = Topic(id=1).save()
user = User(id=1, name='user-name').save()
Message(id=1, topic=topic, author=user).save()
concurrent_change_user = User.objects.get(id=1)
concurrent_change_user.name = 'new-name'
concurrent_change_user.save()
self.assertNotEqual(user.name, 'new-name')
msg = Message.objects.get(id=1)
msg.reload()
self.assertEqual(msg.topic, topic)
self.assertEqual(msg.author, user)
self.assertEqual(msg.author.name, 'new-name')
def test_list_lookup_not_checked_in_map(self):
"""Ensure we dereference list data correctly
"""

View File

@@ -1,330 +0,0 @@
import sys
sys.path[0:0] = [""]
import unittest
from nose.plugins.skip import SkipTest
from mongoengine import *
from mongoengine.django.shortcuts import get_document_or_404
import django
from django.http import Http404
from django.template import Context, Template
from django.conf import settings
from django.core.paginator import Paginator
settings.configure(
USE_TZ=True,
INSTALLED_APPS=('django.contrib.auth', 'mongoengine.django.mongo_auth'),
AUTH_USER_MODEL=('mongo_auth.MongoUser'),
AUTHENTICATION_BACKENDS = ('mongoengine.django.auth.MongoEngineBackend',)
)
try:
# For Django >= 1.7
if hasattr(django, 'setup'):
django.setup()
except RuntimeError:
pass
try:
from django.contrib.auth import authenticate, get_user_model
from mongoengine.django.auth import User
from mongoengine.django.mongo_auth.models import (
MongoUser,
MongoUserManager,
get_user_document,
)
DJ15 = True
except Exception:
DJ15 = False
from mongoengine.django.sessions import SessionStore, MongoSession
from mongoengine.django.tests import MongoTestCase
from datetime import tzinfo, timedelta
ZERO = timedelta(0)
class FixedOffset(tzinfo):
"""Fixed offset in minutes east from UTC."""
def __init__(self, offset, name):
self.__offset = timedelta(minutes=offset)
self.__name = name
def utcoffset(self, dt):
return self.__offset
def tzname(self, dt):
return self.__name
def dst(self, dt):
return ZERO
def activate_timezone(tz):
"""Activate Django timezone support if it is available.
"""
try:
from django.utils import timezone
timezone.deactivate()
timezone.activate(tz)
except ImportError:
pass
class QuerySetTest(unittest.TestCase):
def setUp(self):
connect(db='mongoenginetest')
class Person(Document):
name = StringField()
age = IntField()
self.Person = Person
def test_order_by_in_django_template(self):
"""Ensure that QuerySets are properly ordered in Django template.
"""
self.Person.drop_collection()
self.Person(name="A", age=20).save()
self.Person(name="D", age=10).save()
self.Person(name="B", age=40).save()
self.Person(name="C", age=30).save()
t = Template("{% for o in ol %}{{ o.name }}-{{ o.age }}:{% endfor %}")
d = {"ol": self.Person.objects.order_by('-name')}
self.assertEqual(t.render(Context(d)), u'D-10:C-30:B-40:A-20:')
d = {"ol": self.Person.objects.order_by('+name')}
self.assertEqual(t.render(Context(d)), u'A-20:B-40:C-30:D-10:')
d = {"ol": self.Person.objects.order_by('-age')}
self.assertEqual(t.render(Context(d)), u'B-40:C-30:A-20:D-10:')
d = {"ol": self.Person.objects.order_by('+age')}
self.assertEqual(t.render(Context(d)), u'D-10:A-20:C-30:B-40:')
self.Person.drop_collection()
def test_q_object_filter_in_template(self):
self.Person.drop_collection()
self.Person(name="A", age=20).save()
self.Person(name="D", age=10).save()
self.Person(name="B", age=40).save()
self.Person(name="C", age=30).save()
t = Template("{% for o in ol %}{{ o.name }}-{{ o.age }}:{% endfor %}")
d = {"ol": self.Person.objects.filter(Q(age=10) | Q(name="C"))}
self.assertEqual(t.render(Context(d)), 'D-10:C-30:')
# Check double rendering doesn't throw an error
self.assertEqual(t.render(Context(d)), 'D-10:C-30:')
def test_get_document_or_404(self):
p = self.Person(name="G404")
p.save()
self.assertRaises(Http404, get_document_or_404, self.Person, pk='1234')
self.assertEqual(p, get_document_or_404(self.Person, pk=p.pk))
def test_pagination(self):
"""Ensure that Pagination works as expected
"""
class Page(Document):
name = StringField()
Page.drop_collection()
for i in xrange(1, 11):
Page(name=str(i)).save()
paginator = Paginator(Page.objects.all(), 2)
t = Template("{% for i in page.object_list %}{{ i.name }}:{% endfor %}")
for p in paginator.page_range:
d = {"page": paginator.page(p)}
end = p * 2
start = end - 1
self.assertEqual(t.render(Context(d)), u'%d:%d:' % (start, end))
def test_nested_queryset_template_iterator(self):
# Try iterating the same queryset twice, nested, in a Django template.
names = ['A', 'B', 'C', 'D']
class CustomUser(Document):
name = StringField()
def __unicode__(self):
return self.name
CustomUser.drop_collection()
for name in names:
CustomUser(name=name).save()
users = CustomUser.objects.all().order_by('name')
template = Template("{% for user in users %}{{ user.name }}{% ifequal forloop.counter 2 %} {% for inner_user in users %}{{ inner_user.name }}{% endfor %} {% endifequal %}{% endfor %}")
rendered = template.render(Context({'users': users}))
self.assertEqual(rendered, 'AB ABCD CD')
def test_filter(self):
"""Ensure that a queryset and filters work as expected
"""
class LimitCountQuerySet(QuerySet):
def count(self, with_limit_and_skip=True):
return super(LimitCountQuerySet, self).count(with_limit_and_skip)
class Note(Document):
meta = dict(queryset_class=LimitCountQuerySet)
name = StringField()
Note.drop_collection()
for i in xrange(1, 101):
Note(name="Note: %s" % i).save()
# Check the count
self.assertEqual(Note.objects.count(), 100)
# Get the first 10 and confirm
notes = Note.objects[:10]
self.assertEqual(notes.count(), 10)
# Test djangos template filters
# self.assertEqual(length(notes), 10)
t = Template("{{ notes.count }}")
c = Context({"notes": notes})
self.assertEqual(t.render(c), "10")
# Test with skip
notes = Note.objects.skip(90)
self.assertEqual(notes.count(), 10)
# Test djangos template filters
self.assertEqual(notes.count(), 10)
t = Template("{{ notes.count }}")
c = Context({"notes": notes})
self.assertEqual(t.render(c), "10")
# Test with limit
notes = Note.objects.skip(90)
self.assertEqual(notes.count(), 10)
# Test djangos template filters
self.assertEqual(notes.count(), 10)
t = Template("{{ notes.count }}")
c = Context({"notes": notes})
self.assertEqual(t.render(c), "10")
# Test with skip and limit
notes = Note.objects.skip(10).limit(10)
# Test djangos template filters
self.assertEqual(notes.count(), 10)
t = Template("{{ notes.count }}")
c = Context({"notes": notes})
self.assertEqual(t.render(c), "10")
class _BaseMongoDBSessionTest(unittest.TestCase):
backend = SessionStore
def setUp(self):
connect(db='mongoenginetest')
MongoSession.drop_collection()
super(_BaseMongoDBSessionTest, self).setUp()
def assertIn(self, first, second, msg=None):
self.assertTrue(first in second, msg)
def assertNotIn(self, first, second, msg=None):
self.assertFalse(first in second, msg)
def test_first_save(self):
session = SessionStore()
session['test'] = True
session.save()
self.assertTrue('test' in session)
def test_session_expiration_tz(self):
activate_timezone(FixedOffset(60, 'UTC+1'))
# create and save new session
session = SessionStore()
session.set_expiry(600) # expire in 600 seconds
session['test_expire'] = True
session.save()
# reload session with key
key = session.session_key
session = SessionStore(key)
self.assertTrue('test_expire' in session, 'Session has expired before it is expected')
try:
# SessionTestsMixin isn't available for import on django > 1.8a1
from django.contrib.sessions.tests import SessionTestsMixin
class _MongoDBSessionTest(SessionTestsMixin):
pass
class MongoDBSessionTest(_BaseMongoDBSessionTest):
pass
except ImportError:
class MongoDBSessionTest(_BaseMongoDBSessionTest):
pass
class MongoAuthTest(unittest.TestCase):
user_data = {
'username': 'user',
'email': 'user@example.com',
'password': 'test',
}
def setUp(self):
if not DJ15:
raise SkipTest('mongo_auth requires Django 1.5')
connect(db='mongoenginetest')
User.drop_collection()
super(MongoAuthTest, self).setUp()
def test_get_user_model(self):
self.assertEqual(get_user_model(), MongoUser)
def test_get_user_document(self):
self.assertEqual(get_user_document(), User)
def test_user_manager(self):
manager = get_user_model()._default_manager
self.assertTrue(isinstance(manager, MongoUserManager))
def test_user_manager_exception(self):
manager = get_user_model()._default_manager
self.assertRaises(MongoUser.DoesNotExist, manager.get,
username='not found')
def test_create_user(self):
manager = get_user_model()._default_manager
user = manager.create_user(**self.user_data)
self.assertTrue(isinstance(user, User))
db_user = User.objects.get(username='user')
self.assertEqual(user.id, db_user.id)
def test_authenticate(self):
get_user_model()._default_manager.create_user(**self.user_data)
user = authenticate(username='user', password='fail')
self.assertEqual(None, user)
user = authenticate(username='user', password='test')
db_user = User.objects.get(username='user')
self.assertEqual(user.id, db_user.id)
class MongoTestCaseTest(MongoTestCase):
def test_mongo_test_case(self):
self.db.dummy_collection.insert({'collection': 'will be dropped'})
if __name__ == '__main__':
unittest.main()

View File

@@ -1,47 +0,0 @@
import sys
sys.path[0:0] = [""]
import unittest
from mongoengine import *
import jinja2
class TemplateFilterTest(unittest.TestCase):
def setUp(self):
connect(db='mongoenginetest')
def test_jinja2(self):
env = jinja2.Environment()
class TestData(Document):
title = StringField()
description = StringField()
TestData.drop_collection()
examples = [('A', '1'),
('B', '2'),
('C', '3')]
for title, description in examples:
TestData(title=title, description=description).save()
tmpl = """
{%- for record in content -%}
{%- if loop.first -%}{ {%- endif -%}
"{{ record.title }}": "{{ record.description }}"
{%- if loop.last -%} }{%- else -%},{% endif -%}
{%- endfor -%}
"""
ctx = {'content': TestData.objects}
template = env.from_string(tmpl)
rendered = template.render(**ctx)
self.assertEqual('{"A": "1","B": "2","C": "3"}', rendered)
if __name__ == '__main__':
unittest.main()

View File

@@ -1,17 +1,33 @@
import sys
sys.path[0:0] = [""]
import unittest
import pymongo
from pymongo import ReadPreference, ReplicaSetConnection
from pymongo import ReadPreference
from mongoengine.python_support import IS_PYMONGO_3
if IS_PYMONGO_3:
from pymongo import MongoClient
CONN_CLASS = MongoClient
READ_PREF = ReadPreference.SECONDARY
else:
from pymongo import ReplicaSetConnection
CONN_CLASS = ReplicaSetConnection
READ_PREF = ReadPreference.SECONDARY_ONLY
import mongoengine
from mongoengine import *
from mongoengine.connection import get_db, get_connection, ConnectionError
from mongoengine.connection import ConnectionError
class ConnectionTest(unittest.TestCase):
def setUp(self):
mongoengine.connection._connection_settings = {}
mongoengine.connection._connections = {}
mongoengine.connection._dbs = {}
def tearDown(self):
mongoengine.connection._connection_settings = {}
mongoengine.connection._connections = {}
@@ -22,14 +38,17 @@ class ConnectionTest(unittest.TestCase):
"""
try:
conn = connect(db='mongoenginetest', host="mongodb://localhost/mongoenginetest?replicaSet=rs", read_preference=ReadPreference.SECONDARY_ONLY)
conn = connect(db='mongoenginetest',
host="mongodb://localhost/mongoenginetest?replicaSet=rs",
read_preference=READ_PREF)
except ConnectionError, e:
return
if not isinstance(conn, ReplicaSetConnection):
if not isinstance(conn, CONN_CLASS):
# really???
return
self.assertEqual(conn.read_preference, ReadPreference.SECONDARY_ONLY)
self.assertEqual(conn.read_preference, READ_PREF)
if __name__ == '__main__':
unittest.main()

View File

@@ -25,6 +25,8 @@ class SignalTests(unittest.TestCase):
connect(db='mongoenginetest')
class Author(Document):
# Make the id deterministic for easier testing
id = SequenceField(primary_key=True)
name = StringField()
def __unicode__(self):
@@ -33,7 +35,7 @@ class SignalTests(unittest.TestCase):
@classmethod
def pre_init(cls, sender, document, *args, **kwargs):
signal_output.append('pre_init signal, %s' % cls.__name__)
signal_output.append(str(kwargs['values']))
signal_output.append(kwargs['values'])
@classmethod
def post_init(cls, sender, document, **kwargs):
@@ -43,48 +45,55 @@ class SignalTests(unittest.TestCase):
@classmethod
def pre_save(cls, sender, document, **kwargs):
signal_output.append('pre_save signal, %s' % document)
signal_output.append(kwargs)
@classmethod
def pre_save_post_validation(cls, sender, document, **kwargs):
signal_output.append('pre_save_post_validation signal, %s' % document)
if 'created' in kwargs:
if kwargs['created']:
signal_output.append('Is created')
else:
signal_output.append('Is updated')
if kwargs.pop('created', False):
signal_output.append('Is created')
else:
signal_output.append('Is updated')
signal_output.append(kwargs)
@classmethod
def post_save(cls, sender, document, **kwargs):
dirty_keys = document._delta()[0].keys() + document._delta()[1].keys()
signal_output.append('post_save signal, %s' % document)
signal_output.append('post_save dirty keys, %s' % dirty_keys)
if 'created' in kwargs:
if kwargs['created']:
signal_output.append('Is created')
else:
signal_output.append('Is updated')
if kwargs.pop('created', False):
signal_output.append('Is created')
else:
signal_output.append('Is updated')
signal_output.append(kwargs)
@classmethod
def pre_delete(cls, sender, document, **kwargs):
signal_output.append('pre_delete signal, %s' % document)
signal_output.append(kwargs)
@classmethod
def post_delete(cls, sender, document, **kwargs):
signal_output.append('post_delete signal, %s' % document)
signal_output.append(kwargs)
@classmethod
def pre_bulk_insert(cls, sender, documents, **kwargs):
signal_output.append('pre_bulk_insert signal, %s' % documents)
signal_output.append(kwargs)
@classmethod
def post_bulk_insert(cls, sender, documents, **kwargs):
signal_output.append('post_bulk_insert signal, %s' % documents)
if kwargs.get('loaded', False):
if kwargs.pop('loaded', False):
signal_output.append('Is loaded')
else:
signal_output.append('Not loaded')
signal_output.append(kwargs)
self.Author = Author
Author.drop_collection()
Author.id.set_next_value(0)
class Another(Document):
@@ -96,10 +105,12 @@ class SignalTests(unittest.TestCase):
@classmethod
def pre_delete(cls, sender, document, **kwargs):
signal_output.append('pre_delete signal, %s' % document)
signal_output.append(kwargs)
@classmethod
def post_delete(cls, sender, document, **kwargs):
signal_output.append('post_delete signal, %s' % document)
signal_output.append(kwargs)
self.Another = Another
Another.drop_collection()
@@ -118,6 +129,41 @@ class SignalTests(unittest.TestCase):
self.ExplicitId = ExplicitId
ExplicitId.drop_collection()
class Post(Document):
title = StringField()
content = StringField()
active = BooleanField(default=False)
def __unicode__(self):
return self.title
@classmethod
def pre_bulk_insert(cls, sender, documents, **kwargs):
signal_output.append('pre_bulk_insert signal, %s' %
[(doc, {'active': documents[n].active})
for n, doc in enumerate(documents)])
# make changes here, this is just an example -
# it could be anything that needs pre-validation or looks-ups before bulk bulk inserting
for document in documents:
if not document.active:
document.active = True
signal_output.append(kwargs)
@classmethod
def post_bulk_insert(cls, sender, documents, **kwargs):
signal_output.append('post_bulk_insert signal, %s' %
[(doc, {'active': documents[n].active})
for n, doc in enumerate(documents)])
if kwargs.pop('loaded', False):
signal_output.append('Is loaded')
else:
signal_output.append('Not loaded')
signal_output.append(kwargs)
self.Post = Post
Post.drop_collection()
# Save up the number of connected signals so that we can check at the
# end that all the signals we register get properly unregistered
self.pre_signals = (
@@ -147,6 +193,9 @@ class SignalTests(unittest.TestCase):
signals.post_save.connect(ExplicitId.post_save, sender=ExplicitId)
signals.pre_bulk_insert.connect(Post.pre_bulk_insert, sender=Post)
signals.post_bulk_insert.connect(Post.post_bulk_insert, sender=Post)
def tearDown(self):
signals.pre_init.disconnect(self.Author.pre_init)
signals.post_init.disconnect(self.Author.post_init)
@@ -163,6 +212,9 @@ class SignalTests(unittest.TestCase):
signals.post_save.disconnect(self.ExplicitId.post_save)
signals.pre_bulk_insert.disconnect(self.Post.pre_bulk_insert)
signals.post_bulk_insert.disconnect(self.Post.post_bulk_insert)
# Check that all our signals got disconnected properly.
post_signals = (
len(signals.pre_init.receivers),
@@ -199,66 +251,121 @@ class SignalTests(unittest.TestCase):
a.save()
self.get_signal_output(lambda: None) # eliminate signal output
a1 = self.Author.objects(name='Bill Shakespeare')[0]
self.assertEqual(self.get_signal_output(create_author), [
"pre_init signal, Author",
"{'name': 'Bill Shakespeare'}",
{'name': 'Bill Shakespeare'},
"post_init signal, Bill Shakespeare, document._created = True",
])
a1 = self.Author(name='Bill Shakespeare')
self.assertEqual(self.get_signal_output(a1.save), [
"pre_save signal, Bill Shakespeare",
{},
"pre_save_post_validation signal, Bill Shakespeare",
"Is created",
{},
"post_save signal, Bill Shakespeare",
"post_save dirty keys, ['name']",
"Is created"
"Is created",
{}
])
a1.reload()
a1.name = 'William Shakespeare'
self.assertEqual(self.get_signal_output(a1.save), [
"pre_save signal, William Shakespeare",
{},
"pre_save_post_validation signal, William Shakespeare",
"Is updated",
{},
"post_save signal, William Shakespeare",
"post_save dirty keys, ['name']",
"Is updated"
"Is updated",
{}
])
self.assertEqual(self.get_signal_output(a1.delete), [
'pre_delete signal, William Shakespeare',
{},
'post_delete signal, William Shakespeare',
{}
])
signal_output = self.get_signal_output(load_existing_author)
# test signal_output lines separately, because of random ObjectID after object load
self.assertEqual(signal_output[0],
self.assertEqual(self.get_signal_output(load_existing_author), [
"pre_init signal, Author",
)
self.assertEqual(signal_output[2],
"post_init signal, Bill Shakespeare, document._created = False",
)
{'id': 2, 'name': 'Bill Shakespeare'},
"post_init signal, Bill Shakespeare, document._created = False"
])
signal_output = self.get_signal_output(bulk_create_author_with_load)
# The output of this signal is not entirely deterministic. The reloaded
# object will have an object ID. Hence, we only check part of the output
self.assertEqual(signal_output[3], "pre_bulk_insert signal, [<Author: Bill Shakespeare>]"
)
self.assertEqual(signal_output[-2:],
["post_bulk_insert signal, [<Author: Bill Shakespeare>]",
"Is loaded",])
self.assertEqual(self.get_signal_output(bulk_create_author_with_load), [
'pre_init signal, Author',
{'name': 'Bill Shakespeare'},
'post_init signal, Bill Shakespeare, document._created = True',
'pre_bulk_insert signal, [<Author: Bill Shakespeare>]',
{},
'pre_init signal, Author',
{'id': 3, 'name': 'Bill Shakespeare'},
'post_init signal, Bill Shakespeare, document._created = False',
'post_bulk_insert signal, [<Author: Bill Shakespeare>]',
'Is loaded',
{}
])
self.assertEqual(self.get_signal_output(bulk_create_author_without_load), [
"pre_init signal, Author",
"{'name': 'Bill Shakespeare'}",
{'name': 'Bill Shakespeare'},
"post_init signal, Bill Shakespeare, document._created = True",
"pre_bulk_insert signal, [<Author: Bill Shakespeare>]",
{},
"post_bulk_insert signal, [<Author: Bill Shakespeare>]",
"Not loaded",
{}
])
def test_signal_kwargs(self):
""" Make sure signal_kwargs is passed to signals calls. """
def live_and_let_die():
a = self.Author(name='Bill Shakespeare')
a.save(signal_kwargs={'live': True, 'die': False})
a.delete(signal_kwargs={'live': False, 'die': True})
self.assertEqual(self.get_signal_output(live_and_let_die), [
"pre_init signal, Author",
{'name': 'Bill Shakespeare'},
"post_init signal, Bill Shakespeare, document._created = True",
"pre_save signal, Bill Shakespeare",
{'die': False, 'live': True},
"pre_save_post_validation signal, Bill Shakespeare",
"Is created",
{'die': False, 'live': True},
"post_save signal, Bill Shakespeare",
"post_save dirty keys, ['name']",
"Is created",
{'die': False, 'live': True},
'pre_delete signal, Bill Shakespeare',
{'die': True, 'live': False},
'post_delete signal, Bill Shakespeare',
{'die': True, 'live': False}
])
def bulk_create_author():
a1 = self.Author(name='Bill Shakespeare')
self.Author.objects.insert([a1], signal_kwargs={'key': True})
self.assertEqual(self.get_signal_output(bulk_create_author), [
'pre_init signal, Author',
{'name': 'Bill Shakespeare'},
'post_init signal, Bill Shakespeare, document._created = True',
'pre_bulk_insert signal, [<Author: Bill Shakespeare>]',
{'key': True},
'pre_init signal, Author',
{'id': 2, 'name': 'Bill Shakespeare'},
'post_init signal, Bill Shakespeare, document._created = False',
'post_bulk_insert signal, [<Author: Bill Shakespeare>]',
'Is loaded',
{'key': True}
])
def test_queryset_delete_signals(self):
@@ -267,7 +374,9 @@ class SignalTests(unittest.TestCase):
self.Another(name='Bill Shakespeare').save()
self.assertEqual(self.get_signal_output(self.Another.objects.delete), [
'pre_delete signal, Bill Shakespeare',
{},
'post_delete signal, Bill Shakespeare',
{}
])
def test_signals_with_explicit_doc_ids(self):
@@ -279,5 +388,50 @@ class SignalTests(unittest.TestCase):
# second time, it must be an update
self.assertEqual(self.get_signal_output(ei.save), ['Is updated'])
def test_signals_with_switch_collection(self):
ei = self.ExplicitId(id=123)
ei.switch_collection("explicit__1")
self.assertEqual(self.get_signal_output(ei.save), ['Is created'])
ei.switch_collection("explicit__1")
self.assertEqual(self.get_signal_output(ei.save), ['Is updated'])
ei.switch_collection("explicit__1", keep_created=False)
self.assertEqual(self.get_signal_output(ei.save), ['Is created'])
ei.switch_collection("explicit__1", keep_created=False)
self.assertEqual(self.get_signal_output(ei.save), ['Is created'])
def test_signals_with_switch_db(self):
connect('mongoenginetest')
register_connection('testdb-1', 'mongoenginetest2')
ei = self.ExplicitId(id=123)
ei.switch_db("testdb-1")
self.assertEqual(self.get_signal_output(ei.save), ['Is created'])
ei.switch_db("testdb-1")
self.assertEqual(self.get_signal_output(ei.save), ['Is updated'])
ei.switch_db("testdb-1", keep_created=False)
self.assertEqual(self.get_signal_output(ei.save), ['Is created'])
ei.switch_db("testdb-1", keep_created=False)
self.assertEqual(self.get_signal_output(ei.save), ['Is created'])
def test_signals_bulk_insert(self):
def bulk_set_active_post():
posts = [
self.Post(title='Post 1'),
self.Post(title='Post 2'),
self.Post(title='Post 3')
]
self.Post.objects.insert(posts)
results = self.get_signal_output(bulk_set_active_post)
self.assertEqual(results, [
"pre_bulk_insert signal, [(<Post: Post 1>, {'active': False}), (<Post: Post 2>, {'active': False}), (<Post: Post 3>, {'active': False})]",
{},
"post_bulk_insert signal, [(<Post: Post 1>, {'active': True}), (<Post: Post 2>, {'active': True}), (<Post: Post 3>, {'active': True})]",
'Is loaded',
{}
])
if __name__ == '__main__':
unittest.main()

22
tox.ini Normal file
View File

@@ -0,0 +1,22 @@
[tox]
envlist = {py26,py27,py33,py34,py35,pypy,pypy3}-{mg27,mg28},flake8
[testenv]
commands =
python setup.py nosetests {posargs}
deps =
nose
mg27: PyMongo<2.8
mg28: PyMongo>=2.8,<3.0
mg30: PyMongo>=3.0
mgdev: https://github.com/mongodb/mongo-python-driver/tarball/master
setenv =
PYTHON_EGG_CACHE = {envdir}/python-eggs
passenv = windir
[testenv:flake8]
deps =
flake8
flake8-import-order
commands =
flake8