pub/mongoengine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'master' of git://github.com/hmarr/mongoengine

Browse Source
This commit is contained in:
blackbrrr 2010-01-05 12:00:07 -06:00
commit bb4444f54d
20 changed files with 1110 additions and 146 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
LICENSE
View File

@ -1,4 +1,4 @@
Copyright (c) 2009 Harry Marr
Copyright (c) 2009-2010 Harry Marr
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation

6
MANIFEST.in Normal file
View File

@ -0,0 +1,6 @@
include README.rst
include LICENSE
recursive-include docs *
prune docs/_build/*
recursive-include tests *
recursive-exclude * *.pyc *.swp

89
README.rst
View File

@ -1,8 +1,89 @@
===========
MongoEngine
===========
MongoEngine is an ORM-like layer on top of PyMongo.
:Info: MongoEngine is an ORM-like layer on top of PyMongo.
:Author: Harry Marr (http://github.com/hmarr)
Tutorial available at http://hmarr.com/mongoengine/
About
=====
MongoEngine is a Python Object-Document Mapper for working with MongoDB.
Documentation available at http://hmarr.com/mongoengine/ - there is currently
a `tutorial <http://hmarr.com/mongoengine/tutorial.html>`_, a `user guide
<http://hmarr.com/mongoengine/userguide.html>`_ and an `API reference
<http://hmarr.com/mongoengine/apireference.html>`_.
**Warning:** this software is still in development and should *not* be used
in production.
Installation
============
If you have `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_
you can use ``easy_install mongoengine``. Otherwise, you can download the
source from `GitHub <http://github.com/hmarr/mongoengine>`_ and run ``python
setup.py install``.
Dependencies
============
- pymongo 1.1+
- sphinx (optional - for documentation generation)
Examples
========
Some simple examples of what MongoEngine code looks like::
class BlogPost(Document):
title = StringField(required=True, max_length=200)
posted = DateTimeField(default=datetime.datetime.now)
tags = ListField(StringField(max_length=50))
class TextPost(BlogPost):
content = StringField(required=True)
class LinkPost(BlogPost):
url = StringField(required=True)
# Create a text-based post
>>> post1 = TextPost(title='Using MongoEngine', content='See the tutorial')
>>> post1.tags = ['mongodb', 'mongoengine']
>>> post1.save()
# Create a link-based post
>>> post2 = LinkPost(title='MongoEngine Docs', url='hmarr.com/mongoengine')
>>> post2.tags = ['mongoengine', 'documentation']
>>> post2.save()
# Iterate over all posts using the BlogPost superclass
>>> for post in BlogPost.objects:
... print '===', post.title, '==='
... if isinstance(post, TextPost):
... print post.content
... elif isinstance(post, LinkPost):
... print 'Link:', post.url
... print
...
=== Using MongoEngine ===
See the tutorial
=== MongoEngine Docs ===
Link: hmarr.com/mongoengine
>>> len(BlogPost.objects)
2
>>> len(HtmlPost.objects)
1
>>> len(LinkPost.objects)
1
# Find tagged posts
>>> len(BlogPost.objects(tags='mongoengine'))
2
>>> len(BlogPost.objects(tags='mongodb'))
1
Tests
=====
To run the test suite, ensure you are running a local instance of MongoDB on
the standard port, and run ``python setup.py test``.
Contributing
============
The source is available on `GitHub <http://github.com/hmarr/mongoengine>`_ - to
contribute to the project, fork it on GitHub and send a pull request, all
contributions and suggestions are welcome!

11
docs/apireference.rst
View File

@ -1,13 +1,14 @@
=============
API Reference
=============
Connecting
----------
==========
.. autofunction:: mongoengine.connect
Documents
---------
=========
.. autoclass:: mongoengine.Document
:members:
@ -21,13 +22,15 @@ Documents
:members:
Querying
--------
========
.. autoclass:: mongoengine.queryset.QuerySet
:members:
.. autofunction:: mongoengine.queryset.queryset_manager
Fields
------
======
.. autoclass:: mongoengine.StringField

7
docs/changelog.rst Normal file
View File

@ -0,0 +1,7 @@
=========
Changelog
=========
Changes in v0.1.1
=================
- Documents may now use capped collections

7
docs/conf.py
View File

@ -44,10 +44,11 @@ copyright = u'2009, Harry Marr'
# |version| and |release|, also used in various other places throughout the
# built documents.
#
import mongoengine
# The short X.Y version.
version = '0.1'
version = mongoengine.get_version()
# The full version, including alpha/beta/rc tags.
release = '0.1'
release = mongoengine.get_version()
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
@ -128,7 +129,7 @@ html_static_path = ['_static']
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}

21
docs/index.rst
View File

@ -1,23 +1,26 @@
.. MongoEngine documentation master file, created by
sphinx-quickstart on Sun Nov 22 18:14:13 2009.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
MongoEngine User Documentation
=======================================
Contents:
MongoEngine is an Object-Document Mapper, written in Python for working with
MongoDB. To install it, simply run
.. code-block:: console
# easy_install mongoengine
The source is available on `GitHub <http://github.com/hmarr/mongoengine>`_.
.. toctree::
:maxdepth: 2
tutorial.rst
apireference.rst
tutorial
userguide
apireference
changelog
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

63
docs/tutorial.rst
View File

@ -1,3 +1,4 @@
========
Tutorial
========
This tutorial introduces **MongoEngine** by means of example --- we will walk
@ -9,14 +10,14 @@ application. As the purpose of this tutorial is to introduce MongoEngine, we'll
focus on the data-modelling side of the application, leaving out a user
interface.
Connecting to MongoDB
---------------------
Getting started
===============
Before we start, make sure that a copy of MongoDB is running in an accessible
location --- running it locally will be easier, but if that is not an option
then it may be run on a remote server.
Before we can start using MongoEngine, we need to tell it how to connect to our
instance of **mongod**. For this we use the :func:`mongoengine.connect`
instance of :program:`mongod`. For this we use the :func:`~mongoengine.connect`
function. The only argument we need to provide is the name of the MongoDB
database to use::
@ -24,14 +25,10 @@ database to use::
connect('tumblelog')
This will connect to a mongod instance running locally on the default port. To
connect to a mongod instance running elsewhere, specify the host and port
explicitly::
connect('tumblelog', host='192.168.1.35', port=12345)
For more information about connecting to MongoDB see :ref:`guide-connecting`.
Defining our documents
----------------------
======================
MongoDB is *schemaless*, which means that no schema is enforced by the database
--- we may add and remove fields however we want and MongoDB won't complain.
This makes life a lot easier in many regards, especially when there is a change
@ -50,7 +47,7 @@ specified tag. Finally, it would be nice if **comments** could be added to
posts. We'll start with **users**, as the others are slightly more involved.
Users
^^^^^
-----
Just as if we were using a relational database with an ORM, we need to define
which fields a :class:`User` may have, and what their types will be::
@ -65,7 +62,7 @@ MongoDB --- this will only be enforced at the application level. Also, the User
documents will be stored in a MongoDB *collection* rather than a table.
Posts, Comments and Tags
^^^^^^^^^^^^^^^^^^^^^^^^
------------------------
Now we'll think about how to store the rest of the information. If we were
using a relational database, we would most likely have a table of **posts**, a
table of **comments** and a table of **tags**. To associate the comments with
@ -77,7 +74,7 @@ several ways we can achieve this, but each of them have their problems --- none
of them stand out as particularly intuitive solutions.
Posts
"""""
^^^^^
But MongoDB *isn't* a relational database, so we're not going to do it that
way. As it turns out, we can use MongoDB's schemaless nature to provide us with
a much nicer solution. We will store all of the posts in *one collection* ---
@ -103,12 +100,12 @@ this kind of modelling out of the box::
link_url = StringField()
We are storing a reference to the author of the posts using a
:class:`mongoengine.ReferenceField` object. These are similar to foreign key
:class:`~mongoengine.ReferenceField` object. These are similar to foreign key
fields in traditional ORMs, and are automatically translated into references
when they are saved, and dereferenced when they are loaded.
Tags
""""
^^^^
Now that we have our Post models figured out, how will we attach tags to them?
MongoDB allows us to store lists of items natively, so rather than having a
link table, we can just store a list of tags in each post. So, for both
@ -124,13 +121,13 @@ size of our database. So let's take a look that the code our modified
author = ReferenceField(User)
tags = ListField(StringField(max_length=30))
The :class:`mongoengine.ListField` object that is used to define a Post's tags
The :class:`~mongoengine.ListField` object that is used to define a Post's tags
takes a field object as its first argument --- this means that you can have
lists of any type of field (including lists). Note that we don't need to
modify the specialised post types as they all inherit from :class:`Post`.
Comments
""""""""
^^^^^^^^
A comment is typically associated with *one* post. In a relational database, to
display a post with its comments, we would have to retrieve the post from the
database, then query the database again for the comments associated with the
@ -156,7 +153,7 @@ We can then store a list of comment documents in our post document::
comments = ListField(EmbeddedDocumentField(Comment))
Adding data to our Tumblelog
----------------------------
============================
Now that we've defined how our documents will be structured, let's start adding
some documents to the database. Firstly, we'll need to create a :class:`User`
object::
@ -164,8 +161,7 @@ object::
john = User(email='jdoe@example.com', first_name='John', last_name='Doe')
john.save()
Note that only fields with ``required=True`` need to be specified in the
constructor, we could have also defined our user using attribute syntax::
Note that we could have also defined our user using attribute syntax::
john = User(email='jdoe@example.com')
john.first_name = 'John'
@ -188,10 +184,10 @@ Note that if you change a field on a object that has already been saved, then
call :meth:`save` again, the document will be updated.
Accessing our data
------------------
==================
So now we've got a couple of posts in our database, how do we display them?
Each document class (i.e. any class that inherits either directly or indirectly
from :class:`mongoengine.Document`) has an :attr:`objects` attribute, which is
from :class:`~mongoengine.Document`) has an :attr:`objects` attribute, which is
used to access the documents in the database collection associated with that
class. So let's see how we can get our posts' titles::
@ -199,7 +195,7 @@ class. So let's see how we can get our posts' titles::
print post.title
Retrieving type-specific information
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
------------------------------------
This will print the titles of our posts, one on each line. But What if we want
to access the type-specific data (link_url, content, etc.)? One way is simply
to use the :attr:`objects` attribute of a subclass of :class:`Post`::
@ -209,7 +205,7 @@ to use the :attr:`objects` attribute of a subclass of :class:`Post`::
Using TextPost's :attr:`objects` attribute only returns documents that were
created using :class:`TextPost`. Actually, there is a more general rule here:
the :attr:`objects` attribute of any subclass of :class:`mongoengine.Document`
the :attr:`objects` attribute of any subclass of :class:`~mongoengine.Document`
only looks for documents that were created using that subclass or one of its
subclasses.
@ -237,20 +233,21 @@ This would print the title of each post, followed by the content if it was a
text post, and "Link: <url>" if it was a link post.
Searching our posts by tag
^^^^^^^^^^^^^^^^^^^^^^^^^^
The :attr:`objects` attribute of a :class:`mongoengine.Document` is actually a
:class:`mongoengine.QuerySet` object. This lazily queries the database only
when you need the data. It may also be filtered to narrow down your query.
Let's adjust our query so that only posts with the tag "mongodb" are returned::
--------------------------
The :attr:`objects` attribute of a :class:`~mongoengine.Document` is actually a
:class:`~mongoengine.queryset.QuerySet` object. This lazily queries the
database only when you need the data. It may also be filtered to narrow down
your query. Let's adjust our query so that only posts with the tag "mongodb"
are returned::
for post in Post.objects(tags='mongodb'):
print post.title
There are also methods available on :class:`mongoengine.QuerySet` objects that
allow different results to be returned, for example, calling :meth:`first` on
the :attr:`objects` attribute will return a single document, the first matched
by the query you provide. Aggregation functions may also be used on
:class:`mongoengine.QuerySet` objects::
There are also methods available on :class:`~mongoengine.queryset.QuerySet`
objects that allow different results to be returned, for example, calling
:meth:`first` on the :attr:`objects` attribute will return a single document,
the first matched by the query you provide. Aggregation functions may also be
used on :class:`~mongoengine.queryset.QuerySet` objects::
num_posts = Post.objects(tags='mongodb').count()
print 'Found % posts with tag "mongodb"' % num_posts

385
docs/userguide.rst Normal file
View File

@ -0,0 +1,385 @@
==========
User Guide
==========
.. _guide-connecting:
Installing
==========
MongoEngine is available on PyPI, so to use it you can use
:program:`easy_install`
.. code-block:: console
# easy_install mongoengine
Alternatively, if you don't have setuptools installed, `download it from PyPi
<http://pypi.python.org/pypi/mongoengine/>`_ and run
.. code-block:: console
# python setup.py install
Connecting to MongoDB
=====================
To connect to a running instance of :program:`mongod`, use the
:func:`~mongoengine.connect` function. The first argument is the name of the
database to connect to. If the database does not exist, it will be created. If
the database requires authentication, :attr:`username` and :attr:`password`
arguments may be provided::
from mongoengine import connect
connect('project1', username='webapp', password='pwd123')
By default, MongoEngine assumes that the :program:`mongod` instance is running
on **localhost** on port **27017**. If MongoDB is running elsewhere, you may
provide :attr:`host` and :attr:`port` arguments to
:func:`~mongoengine.connect`::
connect('project1', host='192.168.1.35', port=12345)
Defining documents
==================
In MongoDB, a **document** is roughly equivalent to a **row** in an RDBMS. When
working with relational databases, rows are stored in **tables**, which have a
strict **schema** that the rows follow. MongoDB stores documents in
**collections** rather than tables - the principle difference is that no schema
is enforced at a database level.
Defining a document's schema
----------------------------
MongoEngine allows you to define schemata for documents as this helps to reduce
coding errors, and allows for utility methods to be defined on fields which may
be present.
To define a schema for a document, create a class that inherits from
:class:`~mongoengine.Document`. Fields are specified by adding **field
objects** as class attributes to the document class::
from mongoengine import *
import datetime
class Page(Document):
title = StringField(max_length=200, required=True)
date_modified = DateTimeField(default=datetime.now)
Fields
------
By default, fields are not required. To make a field mandatory, set the
:attr:`required` keyword argument of a field to ``True``. Fields also may have
validation constraints available (such as :attr:`max_length` in the example
above). Fields may also take default values, which will be used if a value is
not provided. Default values may optionally be a callable, which will be called
to retrieve the value (such as in the above example). The field types available
are as follows:
* :class:`~mongoengine.StringField`
* :class:`~mongoengine.IntField`
* :class:`~mongoengine.FloatField`
* :class:`~mongoengine.DateTimeField`
* :class:`~mongoengine.ListField`
* :class:`~mongoengine.ObjectIdField`
* :class:`~mongoengine.EmbeddedDocumentField`
* :class:`~mongoengine.ReferenceField`
List fields
^^^^^^^^^^^
MongoDB allows the storage of lists of items. To add a list of items to a
:class:`~mongoengine.Document`, use the :class:`~mongoengine.ListField` field
type. :class:`~mongoengine.ListField` takes another field object as its first
argument, which specifies which type elements may be stored within the list::
class Page(Document):
tags = ListField(StringField(max_length=50))
Embedded documents
^^^^^^^^^^^^^^^^^^
MongoDB has the ability to embed documents within other documents. Schemata may
be defined for these embedded documents, just as they may be for regular
documents. To create an embedded document, just define a document as usual, but
inherit from :class:`~mongoengine.EmbeddedDocument` rather than
:class:`~mongoengine.Document`::
class Comment(EmbeddedDocument):
content = StringField()
To embed the document within another document, use the
:class:`~mongoengine.EmbeddedDocumentField` field type, providing the embedded
document class as the first argument::
class Page(Document):
comments = ListField(EmbeddedDocumentField(Comment))
comment1 = Comment('Good work!')
comment2 = Comment('Nice article!')
page = Page(comments=[comment1, comment2])
Reference fields
^^^^^^^^^^^^^^^^
References may be stored to other documents in the database using the
:class:`~mongoengine.ReferenceField`. Pass in another document class as the
first argument to the constructor, then simply assign document objects to the
field::
class User(Document):
name = StringField()
class Page(Document):
content = StringField()
author = ReferenceField(User)
john = User(name="John Smith")
john.save()
post = Page(content="Test Page")
post.author = john
post.save()
The :class:`User` object is automatically turned into a reference behind the
scenes, and dereferenced when the :class:`Page` object is retrieved.
Document collections
--------------------
Document classes that inherit **directly** from :class:`~mongoengine.Document`
will have their own **collection** in the database. The name of the collection
is by default the name of the class, coverted to lowercase (so in the example
above, the collection would be called `page`). If you need to change the name
of the collection (e.g. to use MongoEngine with an existing database), then
create a class dictionary attribute called :attr:`meta` on your document, and
set :attr:`collection` to the name of the collection that you want your
document class to use::
class Page(Document):
title = StringField(max_length=200, required=True)
meta = {'collection': 'cmsPage'}
Capped collections
^^^^^^^^^^^^^^^^^^
A :class:`~mongoengine.Document` may use a **Capped Collection** by specifying
:attr:`max_documents` and :attr:`max_size` in the :attr:`meta` dictionary.
:attr:`max_documents` is the maximum number of documents that is allowed to be
stored in the collection, and :attr:`max_size` is the maximum size of the
collection in bytes. If :attr:`max_size` is not specified and
:attr:`max_documents` is, :attr:`max_size` defaults to 10000000 bytes (10MB).
The following example shows a :class:`Log` document that will be limited to
1000 entries and 2MB of disk space::
class Log(Document):
ip_address = StringField()
meta = {'max_documents': 1000, 'max_size': 2000000}
Document inheritance
--------------------
To create a specialised type of a :class:`~mongoengine.Document` you have
defined, you may subclass it and add any extra fields or methods you may need.
As this is new class is not a direct subclass of
:class:`~mongoengine.Document`, it will not be stored in its own collection; it
will use the same collection as its superclass uses. This allows for more
convenient and efficient retrieval of related documents::
# Stored in a collection named 'page'
class Page(Document):
title = StringField(max_length=200, required=True)
# Also stored in the collection named 'page'
class DatedPage(Page):
date = DateTimeField()
Working with existing data
^^^^^^^^^^^^^^^^^^^^^^^^^^
To enable correct retrieval of documents involved in this kind of heirarchy,
two extra attributes are stored on each document in the database: :attr:`_cls`
and :attr:`_types`. These are hidden from the user through the MongoEngine
interface, but may not be present if you are trying to use MongoEngine with
an existing database. For this reason, you may disable this inheritance
mechansim, removing the dependency of :attr:`_cls` and :attr:`_types`, enabling
you to work with existing databases. To disable inheritance on a document
class, set :attr:`allow_inheritance` to ``False`` in the :attr:`meta`
dictionary::
# Will work with data in an existing collection named 'cmsPage'
class Page(Document):
title = StringField(max_length=200, required=True)
meta = {
'collection': 'cmsPage',
'allow_inheritance': False,
}
Documents instances
===================
To create a new document object, create an instance of the relevant document
class, providing values for its fields as its constructor keyword arguments.
You may provide values for any of the fields on the document::
>>> page = Page(title="Test Page")
>>> page.title
'Test Page'
You may also assign values to the document's fields using standard object
attribute syntax::
>>> page.title = "Example Page"
>>> page.title
'Example Page'
Saving and deleting documents
-----------------------------
To save the document to the database, call the
:meth:`~mongoengine.Document.save` method. If the document does not exist in
the database, it will be created. If it does already exist, it will be
updated.
To delete a document, call the :meth:`~mongoengine.Document.delete` method.
Note that this will only work if the document exists in the database and has a
valide :attr:`id`.
Document IDs
------------
Each document in the database has a unique id. This may be accessed through the
:attr:`id` attribute on :class:`~mongoengine.Document` objects. Usually, the id
will be generated automatically by the database server when the object is save,
meaning that you may only access the :attr:`id` field once a document has been
saved::
>>> page = Page(title="Test Page")
>>> page.id
>>> page.save()
>>> page.id
ObjectId('123456789abcdef000000000')
Alternatively, you may explicitly set the :attr:`id` before you save the
document, but the id must be a valid PyMongo :class:`ObjectId`.
Querying the database
=====================
:class:`~mongoengine.Document` classes have an :attr:`objects` attribute, which
is used for accessing the objects in the database associated with the class.
The :attr:`objects` attribute is actually a
:class:`~mongoengine.queryset.QuerySetManager`, which creates and returns a new
a new :class:`~mongoengine.queryset.QuerySet` object on access. The
:class:`~mongoengine.queryset.QuerySet` object may may be iterated over to
fetch documents from the database::
# Prints out the names of all the users in the database
for user in User.objects:
print user.name
Filtering queries
-----------------
The query may be filtered by calling the
:class:`~mongoengine.queryset.QuerySet` object with field lookup keyword
arguments. The keys in the keyword arguments correspond to fields on the
:class:`~mongoengine.Document` you are querying::
# This will return a QuerySet that will only iterate over users whose
# 'country' field is set to 'uk'
uk_users = User.objects(country='uk')
Fields on embedded documents may also be referred to using field lookup syntax
by using a double-underscore in place of the dot in object attribute access
syntax::
# This will return a QuerySet that will only iterate over pages that have
# been written by a user whose 'country' field is set to 'uk'
uk_pages = Page.objects(author__country='uk')
Querying lists
^^^^^^^^^^^^^^
On most fields, this syntax will look up documents where the field specified
matches the given value exactly, but when the field refers to a
:class:`~mongoengine.ListField`, a single item may be provided, in which case
lists that contain that item will be matched::
class Page(Document):
tags = ListField(StringField())
# This will match all pages that have the word 'coding' as an item in the
# 'tags' list
Page.objects(tags='coding')
Query operators
---------------
Operators other than equality may also be used in queries; just attach the
operator name to a key with a double-underscore::
# Only find users whose age is 18 or less
young_users = Users.objects(age__lte=18)
Available operators are as follows:
* ``neq`` -- not equal to
* ``lt`` -- less than
* ``lte`` -- less than or equal to
* ``gt`` -- greater than
* ``gte`` -- greater than or equal to
* ``in`` -- value is in list (a list of values should be provided)
* ``nin`` -- value is not in list (a list of values should be provided)
* ``mod`` -- ``value % x == y``, where ``x`` and ``y`` are two provided values
* ``all`` -- every item in array is in list of values provided
* ``size`` -- the size of the array is
* ``exists`` -- value for field exists
Limiting and skipping results
-----------------------------
Just as with traditional ORMs, you may limit the number of results returned, or
skip a number or results in you query.
:meth:`~mongoengine.queryset.QuerySet.limit` and
:meth:`~mongoengine.queryset.QuerySet.skip` and methods are available on
:class:`~mongoengine.queryset.QuerySet` objects, but the prefered syntax for
achieving this is using array-slicing syntax::
# Only the first 5 people
users = User.objects[:5]
# All except for the first 5 people
users = User.objects[5:]
# 5 users, starting from the 10th user found
users = User.objects[10:15]
Aggregation
-----------
MongoDB provides some aggregation methods out of the box, but there are not as
many as you typically get with an RDBMS. MongoEngine provides a wrapper around
the built-in methods and provides some of its own, which are implemented as
Javascript code that is executed on the database server.
Counting results
^^^^^^^^^^^^^^^^
Just as with limiting and skipping results, there is a method on
:class:`~mongoengine.queryset.QuerySet` objects --
:meth:`~mongoengine.queryset.QuerySet.count`, but there is also a more Pythonic
way of achieving this::
num_users = len(User.objects)
Further aggregation
^^^^^^^^^^^^^^^^^^^
You may sum over the values of a specific field on documents using
:meth:`~mongoengine.queryset.QuerySet.sum`::
yearly_expense = Employee.objects.sum('salary')
.. note::
If the field isn't present on a document, that document will be ignored from
the sum.
To get the average (mean) of a field on a collection of documents, use
:meth:`~mongoengine.queryset.QuerySet.average`::
mean_age = User.objects.average('age')
As MongoDB provides native lists, MongoEngine provides a helper method to get a
dictionary of the frequencies of items in lists across an entire collection --
:meth:`~mongoengine.queryset.QuerySet.item_frequencies`. An example of its use
would be generating "tag-clouds"::
class Article(Document):
tag = ListField(StringField())
# After adding some tagged articles...
tag_freqs = Article.objects.item_frequencies('tag', normalize=True)
from operator import itemgetter
top_tags = sorted(tag_freqs.items(), key=itemgetter(1), reverse=True)[:10]

16
mongoengine/__init__.py
View File

@ -4,9 +4,21 @@ import fields
from fields import *
import connection
from connection import *
import queryset
from queryset import *
__all__ = document.__all__ + fields.__all__ + connection.__all__
__all__ = (document.__all__ + fields.__all__ + connection.__all__ +
queryset.__all__)
__author__ = 'Harry Marr'
__version__ = '0.1'
VERSION = (0, 1, 1)
def get_version():
version = '%s.%s' % (VERSION[0], VERSION[1])
if VERSION[2]:
version = '%s.%s' % (version, VERSION[2])
return version
__version__ = get_version()

25
mongoengine/base.py
View File

@ -28,26 +28,15 @@ class BaseField(object):
# Get value from document instance if available, if not use default
value = instance._data.get(self.name)
if value is None:
if self.default is not None:
value = self.default
# Allow callable default values
if callable(value):
value = value()
else:
raise AttributeError(self.name)
return value
def __set__(self, instance, value):
"""Descriptor for assigning a value to a field in a document. Do any
necessary conversion between Python and MongoDB types.
"""Descriptor for assigning a value to a field in a document.
"""
if value is not None:
try:
self.validate(value)
except (ValueError, AttributeError, AssertionError), e:
raise ValidationError('Invalid value for field of type "' +
self.__class__.__name__ + '"')
elif self.required:
raise ValidationError('Field "%s" is required' % self.name)
instance._data[self.name] = value
def to_python(self, value):
@ -156,6 +145,8 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
meta = {
'collection': collection,
'allow_inheritance': True,
'max_documents': None,
'max_size': None,
}
meta.update(attrs.get('meta', {}))
# Only simple classes - direct subclasses of Document - may set
@ -170,7 +161,7 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
# Set up collection manager, needs the class to have fields so use
# DocumentMetaclass before instantiating CollectionManager object
new_class = super_new(cls, name, bases, attrs)
new_class.objects = QuerySetManager(new_class)
new_class.objects = QuerySetManager()
return new_class
@ -186,8 +177,6 @@ class BaseDocument(object):
else:
# Use default value if present
value = getattr(self, attr_name, None)
if value is None and attr_value.required:
raise ValidationError('Field "%s" is required' % attr_name)
setattr(self, attr_name, value)
@classmethod
@ -228,8 +217,8 @@ class BaseDocument(object):
def __contains__(self, name):
try:
getattr(self, name)
return True
val = getattr(self, name)
return val is not None
except AttributeError:
return False

4
mongoengine/connection.py
View File

@ -29,15 +29,13 @@ def _get_db():
raise ConnectionError('Not connected to database')
return _db
def connect(db=None, username=None, password=None, **kwargs):
def connect(db, username=None, password=None, **kwargs):
"""Connect to the database specified by the 'db' argument. Connection
settings may be provided here as well if the database is not running on
the default port on localhost. If authentication is needed, provide
username and password arguments as well.
"""
global _db
if db is None:
raise TypeError('"db" argument must be provided to connect()')
_connection_settings.update(kwargs)
connection = _get_connection()

45
mongoengine/document.py
View File

@ -1,4 +1,5 @@
from base import DocumentMetaclass, TopLevelDocumentMetaclass, BaseDocument
from base import (DocumentMetaclass, TopLevelDocumentMetaclass, BaseDocument,
ValidationError)
from connection import _get_db
@ -35,6 +36,14 @@ class Document(BaseDocument):
though). To disable this behaviour and remove the dependence on the
presence of `_cls` and `_types`, set :attr:`allow_inheritance` to
``False`` in the :attr:`meta` dictionary.
A :class:`~mongoengine.Document` may use a **Capped Collection** by
specifying :attr:`max_documents` and :attr:`max_size` in the :attr:`meta`
dictionary. :attr:`max_documents` is the maximum number of documents that
is allowed to be stored in the collection, and :attr:`max_size` is the
maximum size of the collection in bytes. If :attr:`max_size` is not
specified and :attr:`max_documents` is, :attr:`max_size` defaults to
10000000 bytes (10MB).
"""
__metaclass__ = TopLevelDocumentMetaclass
@ -44,15 +53,43 @@ class Document(BaseDocument):
document already exists, it will be updated, otherwise it will be
created.
"""
object_id = self.objects._collection.save(self.to_mongo())
self.id = object_id
self.validate()
object_id = self.__class__.objects._collection.save(self.to_mongo())
self.id = self._fields['id'].to_python(object_id)
def delete(self):
"""Delete the :class:`~mongoengine.Document` from the database. This
will only take effect if the document has been previously saved.
"""
object_id = self._fields['id'].to_mongo(self.id)
self.__class__.objects(_id=object_id).delete()
self.__class__.objects(id=object_id).delete()
def reload(self):
"""Reloads all attributes from the database.
"""
object_id = self._fields['id'].to_mongo(self.id)
obj = self.__class__.objects(id=object_id).first()
for field in self._fields:
setattr(self, field, getattr(obj, field))
def validate(self):
"""Ensure that all fields' values are valid and that required fields
are present.
"""
# Get a list of tuples of field names and their current values
fields = [(field, getattr(self, name))
for name, field in self._fields.items()]
# Ensure that each field is matched to a valid value
for field, value in fields:
if value is not None:
try:
field.validate(value)
except (ValueError, AttributeError, AssertionError), e:
raise ValidationError('Invalid value for field of type "' +
field.__class__.__name__ + '"')
elif field.required:
raise ValidationError('Field "%s" is required' % field.name)
@classmethod
def drop_collection(cls):

15
mongoengine/fields.py
View File

@ -34,6 +34,9 @@ class StringField(BaseField):
message = 'String value did not match validation regex'
raise ValidationError(message)
def lookup_member(self, member_name):
return None
class IntField(BaseField):
"""An integer field.
@ -114,6 +117,9 @@ class EmbeddedDocumentField(BaseField):
raise ValidationError('Invalid embedded document instance '
'provided to an EmbeddedDocumentField')
def lookup_member(self, member_name):
return self.document._fields.get(member_name)
class ListField(BaseField):
"""A list field that wraps a standard field, allowing multiple instances
@ -146,6 +152,9 @@ class ListField(BaseField):
raise ValidationError('All items in a list field must be of the '
'specified type')
def lookup_member(self, member_name):
return self.field.lookup_member(member_name)
class ReferenceField(BaseField):
"""A reference to a document that will be automatically dereferenced on
@ -181,9 +190,8 @@ class ReferenceField(BaseField):
if isinstance(document, (str, unicode, pymongo.objectid.ObjectId)):
id_ = document
else:
try:
id_ = document.id
except:
if id_ is None:
raise ValidationError('You can only reference documents once '
'they have been saved to the database')
@ -195,3 +203,6 @@ class ReferenceField(BaseField):
def validate(self, value):
assert(isinstance(value, (self.document_type, pymongo.dbref.DBRef)))
def lookup_member(self, member_name):
return self.document_type._fields.get(member_name)

188
mongoengine/queryset.py
View File

@ -3,6 +3,13 @@ from connection import _get_db
import pymongo
__all__ = ['queryset_manager', 'InvalidQueryError', 'InvalidCollectionError']
class InvalidQueryError(Exception):
pass
class QuerySet(object):
"""A set of results returned from a query. Wraps a MongoDB cursor,
providing :class:`~mongoengine.Document` objects as the results.
@ -36,7 +43,8 @@ class QuerySet(object):
"""Filter the selected documents by calling the
:class:`~mongoengine.QuerySet` with a query.
"""
self._query.update(QuerySet._transform_query(**query))
query = QuerySet._transform_query(_doc_cls=self._document, **query)
self._query.update(query)
return self
@property
@ -46,7 +54,28 @@ class QuerySet(object):
return self._cursor_obj
@classmethod
def _transform_query(cls, **query):
def _translate_field_name(cls, document, parts):
"""Translate a field attribute name to a database field name.
"""
if not isinstance(parts, (list, tuple)):
parts = [parts]
field_names = []
field = None
for field_name in parts:
if field is None:
# Look up first field from the document
field = document._fields[field_name]
else:
# Look up subfield on the previous field
field = field.lookup_member(field_name)
if field is None:
raise InvalidQueryError('Cannot resolve field "%s"'
% field_name)
field_names.append(field.name)
return field_names
@classmethod
def _transform_query(cls, _doc_cls=None, **query):
"""Transform a query from Django-style format to Mongo format.
"""
operators = ['neq', 'gt', 'gte', 'lt', 'lte', 'in', 'nin', 'mod',
@ -61,6 +90,10 @@ class QuerySet(object):
op = parts.pop()
value = {'$' + op: value}
# Switch field names to proper names [set in Field(name='foo')]
if _doc_cls:
parts = QuerySet._translate_field_name(_doc_cls, parts)
key = '.'.join(parts)
if op is None or key not in mongo_query:
mongo_query[key] = value
@ -72,9 +105,10 @@ class QuerySet(object):
def first(self):
"""Retrieve the first object matching the query.
"""
result = self._collection.find_one(self._query)
if result is not None:
result = self._document._from_son(result)
try:
result = self[0]
except IndexError:
result = None
return result
def with_id(self, object_id):
@ -98,6 +132,9 @@ class QuerySet(object):
"""
return self._cursor.count()
def __len__(self):
return self.count()
def limit(self, n):
"""Limit the number of returned documents to `n`. This may also be
achieved using array-slicing syntax (e.g. ``User.objects[:5]``).
@ -161,15 +198,99 @@ class QuerySet(object):
def __iter__(self):
return self
def exec_js(self, code, *fields, **options):
"""Execute a Javascript function on the server. A list of fields may be
provided, which will be translated to their correct names and supplied
as the arguments to the function. A few extra variables are added to
the function's scope: ``collection``, which is the name of the
collection in use; ``query``, which is an object representing the
current query; and ``options``, which is an object containing any
options specified as keyword arguments.
"""
fields = [QuerySet._translate_field_name(self._document, f)
for f in fields]
collection = self._document._meta['collection']
scope = {
'collection': collection,
'query': self._query,
'options': options or {},
}
code = pymongo.code.Code(code, scope=scope)
db = _get_db()
return db.eval(code, *fields)
def sum(self, field):
"""Sum over the values of the specified field.
"""
sum_func = """
function(sumField) {
var total = 0.0;
db[collection].find(query).forEach(function(doc) {
total += (doc[sumField] || 0.0);
});
return total;
}
"""
return self.exec_js(sum_func, field)
def average(self, field):
"""Average over the values of the specified field.
"""
average_func = """
function(averageField) {
var total = 0.0;
var num = 0;
db[collection].find(query).forEach(function(doc) {
if (doc[averageField]) {
total += doc[averageField];
num += 1;
}
});
return total / num;
}
"""
return self.exec_js(average_func, field)
def item_frequencies(self, list_field, normalize=False):
"""Returns a dictionary of all items present in a list field across
the whole queried set of documents, and their corresponding frequency.
This is useful for generating tag clouds, or searching documents.
"""
freq_func = """
function(listField) {
if (options.normalize) {
var total = 0.0;
db[collection].find(query).forEach(function(doc) {
total += doc[listField].length;
});
}
var frequencies = {};
var inc = 1.0;
if (options.normalize) {
inc /= total;
}
db[collection].find(query).forEach(function(doc) {
doc[listField].forEach(function(item) {
frequencies[item] = inc + (frequencies[item] || 0);
});
});
return frequencies;
}
"""
return self.exec_js(freq_func, list_field, normalize=normalize)
class InvalidCollectionError(Exception):
pass
class QuerySetManager(object):
def __init__(self, document):
db = _get_db()
self._document = document
self._collection_name = document._meta['collection']
# This will create the collection if it doesn't exist
self._collection = db[self._collection_name]
def __init__(self, manager_func=None):
self._manager_func = manager_func
self._collection = None
def __get__(self, instance, owner):
"""Descriptor for instantiating a new QuerySet object when
@ -179,5 +300,46 @@ class QuerySetManager(object):
# Document class being used rather than a document object
return self
# self._document should be the same as owner
return QuerySet(self._document, self._collection)
if self._collection is None:
db = _get_db()
collection = owner._meta['collection']
# Create collection as a capped collection if specified
if owner._meta['max_size'] or owner._meta['max_documents']:
# Get max document limit and max byte size from meta
max_size = owner._meta['max_size'] or 10000000 # 10MB default
max_documents = owner._meta['max_documents']
if collection in db.collection_names():
self._collection = db[collection]
# The collection already exists, check if its capped
# options match the specified capped options
options = self._collection.options()
if options.get('max') != max_documents or \
options.get('size') != max_size:
msg = ('Cannot create collection "%s" as a capped '
'collection as it already exists') % collection
raise InvalidCollectionError(msg)
else:
# Create the collection as a capped collection
opts = {'capped': True, 'size': max_size}
if max_documents:
opts['max'] = max_documents
self._collection = db.create_collection(collection, opts)
else:
self._collection = db[collection]
# owner is the document that contains the QuerySetManager
queryset = QuerySet(owner, self._collection)
if self._manager_func:
queryset = self._manager_func(queryset)
return queryset
def queryset_manager(func):
"""Decorator that allows you to define custom QuerySet managers on
:class:`~mongoengine.Document` classes. The manager must be a function that
accepts a :class:`~mongoengine.queryset.QuerySet` as its only argument, and
returns a :class:`~mongoengine.queryset.QuerySet`, probably the same one
but modified in some way.
"""
return QuerySetManager(func)

37
setup.py Normal file
View File

@ -0,0 +1,37 @@
from setuptools import setup, find_packages
VERSION = '0.1.1'
DESCRIPTION = "A Python Document-Object Mapper for working with MongoDB"
LONG_DESCRIPTION = None
try:
LONG_DESCRIPTION = open('README.rst').read()
except:
pass
CLASSIFIERS = [
'Development Status :: 4 - Beta',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Topic :: Database',
'Topic :: Software Development :: Libraries :: Python Modules',
]
setup(name='mongoengine',
version=VERSION,
packages=find_packages(),
author='Harry Marr',
author_email='harry.marr@{nospam}gmail.com',
url='http://hmarr.com/mongoengine/',
license='MIT',
include_package_data=True,
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
platforms=['any'],
classifiers=CLASSIFIERS,
install_requires=['pymongo'],
test_suite='tests',
)

0
tests/__init__.py Normal file
View File

89
tests/document.py
View File

@ -1,4 +1,5 @@
import unittest
import datetime
import pymongo
from mongoengine import *
@ -156,6 +157,70 @@ class DocumentTest(unittest.TestCase):
meta = {'allow_inheritance': False}
self.assertRaises(ValueError, create_employee_class)
def test_collection_name(self):
"""Ensure that a collection with a specified name may be used.
"""
collection = 'personCollTest'
if collection in self.db.collection_names():
self.db.drop_collection(collection)
class Person(Document):
name = StringField()
meta = {'collection': collection}
user = Person(name="Test User")
user.save()
self.assertTrue(collection in self.db.collection_names())
user_obj = self.db[collection].find_one()
self.assertEqual(user_obj['name'], "Test User")
user_obj = Person.objects[0]
self.assertEqual(user_obj.name, "Test User")
Person.drop_collection()
self.assertFalse(collection in self.db.collection_names())
def test_capped_collection(self):
"""Ensure that capped collections work properly.
"""
class Log(Document):
date = DateTimeField(default=datetime.datetime.now)
meta = {
'max_documents': 10,
'max_size': 90000,
}
Log.drop_collection()
# Ensure that the collection handles up to its maximum
for i in range(10):
Log().save()
self.assertEqual(len(Log.objects), 10)
# Check that extra documents don't increase the size
Log().save()
self.assertEqual(len(Log.objects), 10)
options = Log.objects._collection.options()
self.assertEqual(options['capped'], True)
self.assertEqual(options['max'], 10)
self.assertEqual(options['size'], 90000)
# Check that the document cannot be redefined with different options
def recreate_log_document():
class Log(Document):
date = DateTimeField(default=datetime.datetime.now)
meta = {
'max_documents': 11,
}
# Create the collection by accessing Document.objects
Log.objects
self.assertRaises(InvalidCollectionError, recreate_log_document)
Log.drop_collection()
def test_creation(self):
"""Ensure that document may be created using keyword arguments.
"""
@ -163,6 +228,24 @@ class DocumentTest(unittest.TestCase):
self.assertEqual(person.name, "Test User")
self.assertEqual(person.age, 30)
def test_reload(self):
"""Ensure that attributes may be reloaded.
"""
person = self.Person(name="Test User", age=20)
person.save()
person_obj = self.Person.objects.first()
person_obj.name = "Mr Test User"
person_obj.age = 21
person_obj.save()
self.assertEqual(person.name, "Test User")
self.assertEqual(person.age, 20)
person.reload()
self.assertEqual(person.name, "Mr Test User")
self.assertEqual(person.age, 21)
def test_dictionary_access(self):
"""Ensure that dictionary-style field access works properly.
"""
@ -204,16 +287,16 @@ class DocumentTest(unittest.TestCase):
person_obj = collection.find_one({'name': 'Test User'})
self.assertEqual(person_obj['name'], 'Test User')
self.assertEqual(person_obj['age'], 30)
self.assertEqual(person_obj['_id'], person.id)
self.assertEqual(str(person_obj['_id']), person.id)
def test_delete(self):
"""Ensure that document may be deleted using the delete method.
"""
person = self.Person(name="Test User", age=30)
person.save()
self.assertEqual(self.Person.objects.count(), 1)
self.assertEqual(len(self.Person.objects), 1)
person.delete()
self.assertEqual(self.Person.objects.count(), 0)
self.assertEqual(len(self.Person.objects), 0)
def test_save_custom_id(self):
"""Ensure that a document may be saved with a custom _id.

113
tests/fields.py
View File

@ -31,13 +31,10 @@ class FieldTest(unittest.TestCase):
age = IntField(required=True)
userid = StringField()
self.assertRaises(ValidationError, Person, name="Test User")
self.assertRaises(ValidationError, Person, age=30)
person = Person(name="Test User", age=30, userid="testuser")
self.assertRaises(ValidationError, person.__setattr__, 'name', None)
self.assertRaises(ValidationError, person.__setattr__, 'age', None)
person.userid = None
person = Person(name="Test User")
self.assertRaises(ValidationError, person.validate)
person = Person(age=30)
self.assertRaises(ValidationError, person.validate)
def test_object_id_validation(self):
"""Ensure that invalid values cannot be assigned to string fields.
@ -46,10 +43,16 @@ class FieldTest(unittest.TestCase):
name = StringField()
person = Person(name='Test User')
self.assertRaises(AttributeError, getattr, person, 'id')
self.assertRaises(ValidationError, person.__setattr__, 'id', 47)
self.assertRaises(ValidationError, person.__setattr__, 'id', 'abc')
self.assertEqual(person.id, None)
person.id = 47
self.assertRaises(ValidationError, person.validate)
person.id = 'abc'
self.assertRaises(ValidationError, person.validate)
person.id = '497ce96f395f2f052a494fd4'
person.validate()
def test_string_validation(self):
"""Ensure that invalid values cannot be assigned to string fields.
@ -58,20 +61,23 @@ class FieldTest(unittest.TestCase):
name = StringField(max_length=20)
userid = StringField(r'[0-9a-z_]+$')
person = Person()
self.assertRaises(ValidationError, person.__setattr__, 'name', 34)
person = Person(name=34)
self.assertRaises(ValidationError, person.validate)
# Test regex validation on userid
self.assertRaises(ValidationError, person.__setattr__, 'userid',
'test.User')
person = Person(userid='test.User')
self.assertRaises(ValidationError, person.validate)
person.userid = 'test_user'
self.assertEqual(person.userid, 'test_user')
person.validate()
# Test max length validation on name
self.assertRaises(ValidationError, person.__setattr__, 'name',
'Name that is more than twenty characters')
person = Person(name='Name that is more than twenty characters')
self.assertRaises(ValidationError, person.validate)
person.name = 'Shorter name'
self.assertEqual(person.name, 'Shorter name')
person.validate()
def test_int_validation(self):
"""Ensure that invalid values cannot be assigned to int fields.
@ -81,9 +87,14 @@ class FieldTest(unittest.TestCase):
person = Person()
person.age = 50
self.assertRaises(ValidationError, person.__setattr__, 'age', -1)
self.assertRaises(ValidationError, person.__setattr__, 'age', 120)
self.assertRaises(ValidationError, person.__setattr__, 'age', 'ten')
person.validate()
person.age = -1
self.assertRaises(ValidationError, person.validate)
person.age = 120
self.assertRaises(ValidationError, person.validate)
person.age = 'ten'
self.assertRaises(ValidationError, person.validate)
def test_float_validation(self):
"""Ensure that invalid values cannot be assigned to float fields.
@ -93,9 +104,14 @@ class FieldTest(unittest.TestCase):
person = Person()
person.height = 1.89
self.assertRaises(ValidationError, person.__setattr__, 'height', 2)
self.assertRaises(ValidationError, person.__setattr__, 'height', 0.01)
self.assertRaises(ValidationError, person.__setattr__, 'height', 4.0)
person.validate()
person.height = 2
self.assertRaises(ValidationError, person.validate)
person.height = 0.01
self.assertRaises(ValidationError, person.validate)
person.height = 4.0
self.assertRaises(ValidationError, person.validate)
def test_datetime_validation(self):
"""Ensure that invalid values cannot be assigned to datetime fields.
@ -104,9 +120,13 @@ class FieldTest(unittest.TestCase):
time = DateTimeField()
log = LogEntry()
self.assertRaises(ValidationError, log.__setattr__, 'time', -1)
self.assertRaises(ValidationError, log.__setattr__, 'time', '1pm')
log.time = datetime.datetime.now()
log.validate()
log.time = -1
self.assertRaises(ValidationError, log.validate)
log.time = '1pm'
self.assertRaises(ValidationError, log.validate)
def test_list_validation(self):
"""Ensure that a list field only accepts lists with valid elements.
@ -120,16 +140,26 @@ class FieldTest(unittest.TestCase):
tags = ListField(StringField())
post = BlogPost(content='Went for a walk today...')
self.assertRaises(ValidationError, post.__setattr__, 'tags', 'fun')
self.assertRaises(ValidationError, post.__setattr__, 'tags', [1, 2])
post.validate()
post.tags = 'fun'
self.assertRaises(ValidationError, post.validate)
post.tags = [1, 2]
self.assertRaises(ValidationError, post.validate)
post.tags = ['fun', 'leisure']
post.validate()
post.tags = ('fun', 'leisure')
post.validate()
comments = [Comment(content='Good for you'), Comment(content='Yay.')]
self.assertRaises(ValidationError, post.__setattr__, 'comments', ['a'])
self.assertRaises(ValidationError, post.__setattr__, 'comments', 'Yay')
self.assertRaises(ValidationError, post.__setattr__, 'comments', 'Yay')
post.comments = comments
post.validate()
post.comments = ['a']
self.assertRaises(ValidationError, post.validate)
post.comments = 'yay'
self.assertRaises(ValidationError, post.validate)
def test_embedded_document_validation(self):
"""Ensure that invalid embedded documents cannot be assigned to
@ -147,12 +177,15 @@ class FieldTest(unittest.TestCase):
preferences = EmbeddedDocumentField(PersonPreferences)
person = Person(name='Test User')
self.assertRaises(ValidationError, person.__setattr__, 'preferences',
'My preferences')
self.assertRaises(ValidationError, person.__setattr__, 'preferences',
Comment(content='Nice blog post...'))
person.preferences = 'My Preferences'
self.assertRaises(ValidationError, person.validate)
person.preferences = Comment(content='Nice blog post...')
self.assertRaises(ValidationError, person.validate)
person.preferences = PersonPreferences(food='Cheese', number=47)
self.assertEqual(person.preferences.food, 'Cheese')
person.validate()
def test_embedded_document_inheritance(self):
"""Ensure that subclasses of embedded documents may be provided to
@ -173,8 +206,8 @@ class FieldTest(unittest.TestCase):
post.author = PowerUser(name='Test User', power=47)
def test_reference_validation(self):
"""Ensure that invalid embedded documents cannot be assigned to
embedded document fields.
"""Ensure that invalid docment objects cannot be assigned to reference
fields.
"""
class User(Document):
name = StringField()
@ -187,19 +220,23 @@ class FieldTest(unittest.TestCase):
user = User(name='Test User')
# Ensure that the referenced object must have been saved
post1 = BlogPost(content='Chips and gravy taste good.')
post1.author = user
self.assertRaises(ValidationError, post1.save)
# Check that an invalid object type cannot be used
post2 = BlogPost(content='Chips and chilli taste good.')
self.assertRaises(ValidationError, post1.__setattr__, 'author', post2)
post1.author = post2
self.assertRaises(ValidationError, post1.validate)
user.save()
post1.author = user
post1.save()
post2.save()
self.assertRaises(ValidationError, post1.__setattr__, 'author', post2)
post1.author = post2
self.assertRaises(ValidationError, post1.validate)
User.drop_collection()
BlogPost.drop_collection()

125
tests/queryset.py
View File

@ -50,7 +50,7 @@ class QuerySetTest(unittest.TestCase):
# Find all people in the collection
people = self.Person.objects
self.assertEqual(people.count(), 2)
self.assertEqual(len(people), 2)
results = list(people)
self.assertTrue(isinstance(results[0], self.Person))
self.assertTrue(isinstance(results[0].id, (pymongo.objectid.ObjectId,
@ -62,7 +62,7 @@ class QuerySetTest(unittest.TestCase):
# Use a query to filter the people found to just person1
people = self.Person.objects(age=20)
self.assertEqual(people.count(), 1)
self.assertEqual(len(people), 1)
person = people.next()
self.assertEqual(person.name, "User A")
self.assertEqual(person.age, 20)
@ -158,13 +158,13 @@ class QuerySetTest(unittest.TestCase):
self.Person(name="User B", age=30).save()
self.Person(name="User C", age=40).save()
self.assertEqual(self.Person.objects.count(), 3)
self.assertEqual(len(self.Person.objects), 3)
self.Person.objects(age__lt=30).delete()
self.assertEqual(self.Person.objects.count(), 2)
self.assertEqual(len(self.Person.objects), 2)
self.Person.objects.delete()
self.assertEqual(self.Person.objects.count(), 0)
self.assertEqual(len(self.Person.objects), 0)
def test_order_by(self):
"""Ensure that QuerySets may be ordered.
@ -185,6 +185,121 @@ class QuerySetTest(unittest.TestCase):
ages = [p.age for p in self.Person.objects.order_by('-name')]
self.assertEqual(ages, [30, 40, 20])
def test_item_frequencies(self):
"""Ensure that item frequencies are properly generated from lists.
"""
class BlogPost(Document):
hits = IntField()
tags = ListField(StringField(), name='blogTags')
BlogPost.drop_collection()
BlogPost(hits=1, tags=['music', 'film', 'actors']).save()
BlogPost(hits=2, tags=['music']).save()
BlogPost(hits=3, tags=['music', 'actors']).save()
f = BlogPost.objects.item_frequencies('tags')
f = dict((key, int(val)) for key, val in f.items())
self.assertEqual(set(['music', 'film', 'actors']), set(f.keys()))
self.assertEqual(f['music'], 3)
self.assertEqual(f['actors'], 2)
self.assertEqual(f['film'], 1)
# Ensure query is taken into account
f = BlogPost.objects(hits__gt=1).item_frequencies('tags')
f = dict((key, int(val)) for key, val in f.items())
self.assertEqual(set(['music', 'actors']), set(f.keys()))
self.assertEqual(f['music'], 2)
self.assertEqual(f['actors'], 1)
# Check that normalization works
f = BlogPost.objects.item_frequencies('tags', normalize=True)
self.assertAlmostEqual(f['music'], 3.0/6.0)
self.assertAlmostEqual(f['actors'], 2.0/6.0)
self.assertAlmostEqual(f['film'], 1.0/6.0)
BlogPost.drop_collection()
def test_average(self):
"""Ensure that field can be averaged correctly.
"""
ages = [23, 54, 12, 94, 27]
for i, age in enumerate(ages):
self.Person(name='test%s' % i, age=age).save()
avg = float(sum(ages)) / len(ages)
self.assertAlmostEqual(int(self.Person.objects.average('age')), avg)
self.Person(name='ageless person').save()
self.assertEqual(int(self.Person.objects.average('age')), avg)
def test_sum(self):
"""Ensure that field can be summed over correctly.
"""
ages = [23, 54, 12, 94, 27]
for i, age in enumerate(ages):
self.Person(name='test%s' % i, age=age).save()
self.assertEqual(int(self.Person.objects.sum('age')), sum(ages))
self.Person(name='ageless person').save()
self.assertEqual(int(self.Person.objects.sum('age')), sum(ages))
def test_custom_manager(self):
"""Ensure that custom QuerySetManager instances work as expected.
"""
class BlogPost(Document):
tags = ListField(StringField())
@queryset_manager
def music_posts(queryset):
return queryset(tags='music')
BlogPost.drop_collection()
post1 = BlogPost(tags=['music', 'film'])
post1.save()
post2 = BlogPost(tags=['music'])
post2.save()
post3 = BlogPost(tags=['film', 'actors'])
post3.save()
self.assertEqual([p.id for p in BlogPost.objects],
[post1.id, post2.id, post3.id])
self.assertEqual([p.id for p in BlogPost.music_posts],
[post1.id, post2.id])
BlogPost.drop_collection()
def test_query_field_name(self):
"""Ensure that the correct field name is used when querying.
"""
class Comment(EmbeddedDocument):
content = StringField(name='commentContent')
class BlogPost(Document):
title = StringField(name='postTitle')
comments = ListField(EmbeddedDocumentField(Comment),
name='postComments')
BlogPost.drop_collection()
data = {'title': 'Post 1', 'comments': [Comment(content='test')]}
BlogPost(**data).save()
self.assertTrue('postTitle' in
BlogPost.objects(title=data['title'])._query)
self.assertFalse('title' in
BlogPost.objects(title=data['title'])._query)
self.assertEqual(len(BlogPost.objects(title=data['title'])), 1)
self.assertTrue('postComments.commentContent' in
BlogPost.objects(comments__content='test')._query)
self.assertEqual(len(BlogPost.objects(comments__content='test')), 1)
BlogPost.drop_collection()
def tearDown(self):
self.Person.drop_collection()