merged master

docs/apireference.rst

@@ -46,6 +46,8 @@ Fields
 
 .. autoclass:: mongoengine.EmbeddedDocumentField
 
+.. autoclass:: mongoengine.DictField
+
 .. autoclass:: mongoengine.ListField
 
 .. autoclass:: mongoengine.ObjectIdField

docs/guide/defining-documents.rst

@@ -39,12 +39,13 @@ are as follows:
 * :class:`~mongoengine.FloatField`
 * :class:`~mongoengine.DateTimeField`
 * :class:`~mongoengine.ListField`
+* :class:`~mongoengine.DictField`
 * :class:`~mongoengine.ObjectIdField`
 * :class:`~mongoengine.EmbeddedDocumentField`
 * :class:`~mongoengine.ReferenceField`
 
 List fields
-^^^^^^^^^^^
+-----------
 MongoDB allows the storage of lists of items. To add a list of items to a
 :class:`~mongoengine.Document`, use the :class:`~mongoengine.ListField` field
 type. :class:`~mongoengine.ListField` takes another field object as its first
@@ -54,7 +55,7 @@ argument, which specifies which type elements may be stored within the list::
         tags = ListField(StringField(max_length=50))
 
 Embedded documents
-^^^^^^^^^^^^^^^^^^
+------------------
 MongoDB has the ability to embed documents within other documents. Schemata may
 be defined for these embedded documents, just as they may be for regular
 documents. To create an embedded document, just define a document as usual, but
@@ -75,8 +76,25 @@ document class as the first argument::
     comment2 = Comment('Nice article!')
     page = Page(comments=[comment1, comment2])
 
+Dictionary Fields
+-----------------
+Often, an embedded document may be used instead of a dictionary -- generally
+this is recommended as dictionaries don't support validation or custom field
+types. However, sometimes you will not know the structure of what you want to
+store; in this situation a :class:`~mongoengine.DictField` is appropriate::
+    
+    class SurveyResponse(Document):
+        date = DateTimeField()
+        user = ReferenceField(User)
+        answers = DictField()
+
+    survey_response = SurveyResponse(date=datetime.now(), user=request.user)
+    response_form = ResponseForm(request.POST)
+    survey_response.answers = response_form.cleaned_data()   
+    survey_response.save()
+
 Reference fields
-^^^^^^^^^^^^^^^^
+----------------
 References may be stored to other documents in the database using the
 :class:`~mongoengine.ReferenceField`. Pass in another document class as the
 first argument to the constructor, then simply assign document objects to the
@@ -100,7 +118,7 @@ The :class:`User` object is automatically turned into a reference behind the
 scenes, and dereferenced when the :class:`Page` object is retrieved.
 
 Uniqueness constraints
-^^^^^^^^^^^^^^^^^^^^^^
+----------------------
 MongoEngine allows you to specify that a field should be unique across a
 collection by providing ``unique=True`` to a :class:`~mongoengine.Field`\ 's
 constructor. If you try to save a document that has the same value for a unique
@@ -130,7 +148,7 @@ document class to use::
         meta = {'collection': 'cmsPage'}
 
 Capped collections
-^^^^^^^^^^^^^^^^^^
+------------------
 A :class:`~mongoengine.Document` may use a **Capped Collection** by specifying
 :attr:`max_documents` and :attr:`max_size` in the :attr:`meta` dictionary.
 :attr:`max_documents` is the maximum number of documents that is allowed to be
@@ -179,13 +197,13 @@ subsequent calls to :meth:`~mongoengine.queryset.QuerySet.order_by`. ::
         }
 
     blog_post_1 = BlogPost(title="Blog Post #1")
-    blog_post_1.published_date = datetime(2010, 1, 5, 0, 0 ,0))
+    blog_post_1.published_date = datetime(2010, 1, 5, 0, 0 ,0)
 
     blog_post_2 = BlogPost(title="Blog Post #2") 
-    blog_post_2.published_date = datetime(2010, 1, 6, 0, 0 ,0))
+    blog_post_2.published_date = datetime(2010, 1, 6, 0, 0 ,0)
 
     blog_post_3 = BlogPost(title="Blog Post #3")
-    blog_post_3.published_date = datetime(2010, 1, 7, 0, 0 ,0))
+    blog_post_3.published_date = datetime(2010, 1, 7, 0, 0 ,0)
 
     blog_post_1.save()
     blog_post_2.save()
@@ -194,11 +212,11 @@ subsequent calls to :meth:`~mongoengine.queryset.QuerySet.order_by`. ::
     # get the "first" BlogPost using default ordering
     # from BlogPost.meta.ordering
     latest_post = BlogPost.objects.first() 
-    self.assertEqual(latest_post.title, "Blog Post #3")
+    assert latest_post.title == "Blog Post #3"
 
     # override default ordering, order BlogPosts by "published_date"
     first_post = BlogPost.objects.order_by("+published_date").first()
-    self.assertEqual(first_post.title, "Blog Post #1")
+    assert first_post.title == "Blog Post #1"
 
 Document inheritance
 ====================
@@ -218,7 +236,7 @@ convenient and efficient retrieval of related documents::
         date = DateTimeField()
 
 Working with existing data
-^^^^^^^^^^^^^^^^^^^^^^^^^^
+--------------------------
 To enable correct retrieval of documents involved in this kind of heirarchy,
 two extra attributes are stored on each document in the database: :attr:`_cls`
 and :attr:`_types`. These are hidden from the user through the MongoEngine

docs/guide/document-instances.rst

@@ -17,7 +17,7 @@ attribute syntax::
     'Example Page'
 
 Saving and deleting documents
------------------------------
+=============================
 To save the document to the database, call the
 :meth:`~mongoengine.Document.save` method. If the document does not exist in
 the database, it will be created. If it does already exist, it will be
@@ -31,7 +31,7 @@ valide :attr:`id`.
     :ref:`guide-atomic-updates`
 
 Document IDs
-------------
+============
 Each document in the database has a unique id. This may be accessed through the
 :attr:`id` attribute on :class:`~mongoengine.Document` objects. Usually, the id
 will be generated automatically by the database server when the object is save,

docs/guide/querying.rst

@@ -14,7 +14,7 @@ fetch documents from the database::
         print user.name
 
 Filtering queries
------------------
+=================
 The query may be filtered by calling the
 :class:`~mongoengine.queryset.QuerySet` object with field lookup keyword 
 arguments. The keys in the keyword arguments correspond to fields on the
@@ -33,7 +33,7 @@ syntax::
     uk_pages = Page.objects(author__country='uk')
 
 Querying lists
-^^^^^^^^^^^^^^
+--------------
 On most fields, this syntax will look up documents where the field specified
 matches the given value exactly, but when the field refers to a
 :class:`~mongoengine.ListField`, a single item may be provided, in which case
@@ -47,7 +47,7 @@ lists that contain that item will be matched::
     Page.objects(tags='coding')
 
 Query operators
----------------
+===============
 Operators other than equality may also be used in queries; just attach the
 operator name to a key with a double-underscore::
     
@@ -69,7 +69,7 @@ Available operators are as follows:
 * ``exists`` -- value for field exists
 
 Limiting and skipping results
------------------------------
+=============================
 Just as with traditional ORMs, you may limit the number of results returned, or
 skip a number or results in you query.
 :meth:`~mongoengine.queryset.QuerySet.limit` and
@@ -86,15 +86,89 @@ achieving this is using array-slicing syntax::
     # 5 users, starting from the 10th user found
     users = User.objects[10:15]
 
+You may also index the query to retrieve a single result. If an item at that
+index does not exists, an :class:`IndexError` will be raised. A shortcut for
+retrieving the first result and returning :attr:`None` if no result exists is
+provided (:meth:`~mongoengine.queryset.QuerySet.first`)::
+    
+    >>> # Make sure there are no users
+    >>> User.drop_collection()
+    >>> User.objects[0]
+    IndexError: list index out of range
+    >>> User.objects.first() == None
+    True
+    >>> User(name='Test User').save()
+    >>> User.objects[0] == User.objects.first()
+    True
+
+Retrieving unique results
+-------------------------
+To retrieve a result that should be unique in the collection, use
+:meth:`~mongoengine.queryset.QuerySet.get`. This will raise
+:class:`~mongoengine.queryset.DoesNotExist` if no document matches the query,
+and :class:`~mongoengine.queryset.MultipleObjectsReturned` if more than one
+document matched the query.
+
+A variation of this method exists, 
+:meth:`~mongoengine.queryset.Queryset.get_or_create`, that will create a new
+document with the query arguments if no documents match the query. An 
+additional keyword argument, :attr:`defaults` may be provided, which will be
+used as default values for the new document, in the case that it should need
+to be created::
+
+    >>> a = User.objects.get_or_create(name='User A', defaults={'age': 30})
+    >>> b = User.objects.get_or_create(name='User A', defaults={'age': 40})
+    >>> a.name == b.name and a.age == b.age
+    True
+
+Default Document queries
+========================
+By default, the objects :attr:`~mongoengine.Document.objects` attribute on a
+document returns a :class:`~mongoengine.queryset.QuerySet` that doesn't filter
+the collection -- it returns all objects. This may be changed by defining a
+method on a document that modifies a queryset. The method should accept two
+arguments -- :attr:`doc_cls` and :attr:`queryset`. The first argument is the
+:class:`~mongoengine.Document` class that the method is defined on (in this
+sense, the method is more like a :func:`classmethod` than a regular method),
+and the second argument is the initial queryset. The method needs to be
+decorated with :func:`~mongoengine.queryset.queryset_manager` in order for it
+to be recognised. ::
+
+    class BlogPost(Document):
+        title = StringField()
+        date = DateTimeField()
+
+        @queryset_manager
+        def objects(doc_cls, queryset):
+            # This may actually also be done by defining a default ordering for
+            # the document, but this illustrates the use of manager methods
+            return queryset.order_by('-date')
+
+You don't need to call your method :attr:`objects` -- you may define as many
+custom manager methods as you like::
+
+    class BlogPost(Document):
+        title = StringField()
+        published = BooleanField()
+
+        @queryset_manager
+        def live_posts(doc_cls, queryset):
+            return queryset.order_by('-date')
+
+    BlogPost(title='test1', published=False).save()
+    BlogPost(title='test2', published=True).save()
+    assert len(BlogPost.objects) == 2
+    assert len(BlogPost.live_posts) == 1
+
 Aggregation
------------
+===========
 MongoDB provides some aggregation methods out of the box, but there are not as
 many as you typically get with an RDBMS. MongoEngine provides a wrapper around
 the built-in methods and provides some of its own, which are implemented as
 Javascript code that is executed on the database server.
 
 Counting results
-^^^^^^^^^^^^^^^^
+----------------
 Just as with limiting and skipping results, there is a method on
 :class:`~mongoengine.queryset.QuerySet` objects -- 
 :meth:`~mongoengine.queryset.QuerySet.count`, but there is also a more Pythonic
@@ -103,7 +177,7 @@ way of achieving this::
     num_users = len(User.objects)
 
 Further aggregation
-^^^^^^^^^^^^^^^^^^^
+-------------------
 You may sum over the values of a specific field on documents using
 :meth:`~mongoengine.queryset.QuerySet.sum`::
 
@@ -133,7 +207,7 @@ would be generating "tag-clouds"::
     top_tags = sorted(tag_freqs.items(), key=itemgetter(1), reverse=True)[:10]
 
 Advanced queries
-----------------
+================
 Sometimes calling a :class:`~mongoengine.queryset.QuerySet` object with keyword
 arguments can't fully express the query you want to use -- for example if you
 need to combine a number of constraints using *and* and *or*. This is made 
@@ -161,7 +235,7 @@ calling it with keyword arguments::
 .. _guide-atomic-updates:
 
 Atomic updates
---------------
+==============
 Documents may be updated atomically by using the
 :meth:`~mongoengine.queryset.QuerySet.update_one` and
 :meth:`~mongoengine.queryset.QuerySet.update` methods on a 

docs/index.rst

@@ -11,6 +11,11 @@ MongoDB. To install it, simply run
 
 The source is available on `GitHub <http://github.com/hmarr/mongoengine>`_.
 
+If you are interested in contributing, join the developers' `mailing list 
+<http://groups.google.com/group/mongoengine-dev>`_. Some of us also like to
+hang out at `#mongoengine IRC channel <irc://irc.freenode.net/mongoengine>`_.
+
+
 .. toctree::
    :maxdepth: 2
 

mongoengine/base.py

@@ -6,7 +6,6 @@ import pymongo
 class ValidationError(Exception):
     pass
 
-
 class BaseField(object):
     """A base class for fields in a MongoDB document. Instances of this class
     may be added to subclasses of `Document` to define a document's schema.
@@ -77,7 +76,10 @@ class ObjectIdField(BaseField):
 
     def to_mongo(self, value):
         if not isinstance(value, pymongo.objectid.ObjectId):
-            return pymongo.objectid.ObjectId(str(value))
+            try:
+                return pymongo.objectid.ObjectId(str(value))
+            except Exception, e:
+                raise ValidationError(e.message)
         return value
 
     def prepare_query_value(self, value):
@@ -103,6 +105,7 @@ class DocumentMetaclass(type):
         doc_fields = {}
         class_name = [name]
         superclasses = {}
+        simple_class = True
         for base in bases:
             # Include all fields present in superclasses
             if hasattr(base, '_fields'):
@@ -111,6 +114,29 @@ class DocumentMetaclass(type):
                 # Get superclasses from superclass
                 superclasses[base._class_name] = base
                 superclasses.update(base._superclasses)
+                
+            if hasattr(base, '_meta'):
+                # Ensure that the Document class may be subclassed - 
+                # inheritance may be disabled to remove dependency on 
+                # additional fields _cls and _types
+                if base._meta.get('allow_inheritance', True) == False:
+                    raise ValueError('Document %s may not be subclassed' %
+                                     base.__name__)
+                else:
+                    simple_class = False
+
+        meta = attrs.get('_meta', attrs.get('meta', {}))
+
+        if 'allow_inheritance' not in meta:
+            meta['allow_inheritance'] = True
+
+        # Only simple classes - direct subclasses of Document - may set
+        # allow_inheritance to False
+        if not simple_class and not meta['allow_inheritance']:
+            raise ValueError('Only direct subclasses of Document may set '
+                             '"allow_inheritance" to False')
+        attrs['_meta'] = meta
+
         attrs['_class_name'] = '.'.join(reversed(class_name))
         attrs['_superclasses'] = superclasses
 
@@ -143,21 +169,12 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
 
         collection = name.lower()
         
-        simple_class = True
         id_field = None
         base_indexes = []
 
         # Subclassed documents inherit collection from superclass
         for base in bases:
             if hasattr(base, '_meta') and 'collection' in base._meta:
-                # Ensure that the Document class may be subclassed - 
-                # inheritance may be disabled to remove dependency on 
-                # additional fields _cls and _types
-                if base._meta.get('allow_inheritance', True) == False:
-                    raise ValueError('Document %s may not be subclassed' %
-                                     base.__name__)
-                else:
-                    simple_class = False
                 collection = base._meta['collection']
 
                 id_field = id_field or base._meta.get('id_field')
@@ -165,7 +182,6 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
 
         meta = {
             'collection': collection,
-            'allow_inheritance': True,
             'max_documents': None,
             'max_size': None,
             'ordering': [], # default ordering applied at runtime
@@ -175,12 +191,6 @@ class TopLevelDocumentMetaclass(DocumentMetaclass):
 
         # Apply document-defined meta options
         meta.update(attrs.get('meta', {}))
-
-        # Only simple classes - direct subclasses of Document - may set
-        # allow_inheritance to False
-        if not simple_class and not meta['allow_inheritance']:
-            raise ValueError('Only direct subclasses of Document may set '
-                             '"allow_inheritance" to False')
         attrs['_meta'] = meta
 
         # Set up collection manager, needs the class to have fields so use
@@ -346,7 +356,7 @@ class BaseDocument(object):
     
     @classmethod
     def _from_son(cls, son):
-        """Create an instance of a Document (subclass) from a PyMongo SOM.
+        """Create an instance of a Document (subclass) from a PyMongo SON.
         """
         # get the class name from the document, falling back to the given
         # class if unavailable

mongoengine/connection.py

@@ -7,7 +7,6 @@ __all__ = ['ConnectionError', 'connect']
 _connection_settings = {
     'host': 'localhost',
     'port': 27017,
-    'pool_size': 1,
 }
 _connection = None
 

mongoengine/document.py

@@ -78,9 +78,9 @@ class Document(BaseDocument):
                 object_id = collection.save(doc, safe=safe)
         except pymongo.errors.OperationFailure, err:
             message = 'Could not save document (%s)'
-            if 'duplicate key' in str(err):
+            if u'duplicate key' in unicode(err):
-                message = 'Tried to save duplicate unique keys (%s)'
+                message = u'Tried to save duplicate unique keys (%s)'
-            raise OperationError(message % str(err))
+            raise OperationError(message % unicode(err))
         id_field = self._meta['id_field']
         self[id_field] = self._fields[id_field].to_python(object_id)
 
@@ -95,7 +95,8 @@ class Document(BaseDocument):
         try:
             self.__class__.objects(**{id_field: object_id}).delete(safe=safe)
         except pymongo.errors.OperationFailure, err:
-            raise OperationError('Could not delete document (%s)' % str(err))
+            message = u'Could not delete document (%s)' % err.message
+            raise OperationError(message)
 
     def reload(self):
         """Reloads all attributes from the database.

mongoengine/fields.py

@@ -9,18 +9,8 @@ import decimal
 
 
 __all__ = ['StringField', 'IntField', 'FloatField', 'BooleanField',
-           'DateTimeField', 'EmbeddedDocumentField', 'ListField',
+           'DateTimeField', 'EmbeddedDocumentField', 'ListField', 'DictField',
-           'ObjectIdField', 'ReferenceField', 'ValidationError',
+           'ObjectIdField', 'ReferenceField', 'ValidationError']
-           'URLField', 'DecimalField']
-
-
-URL_REGEX = re.compile(
-  r'^https?://'
-  r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+[A-Z]{2,6}\.?|'
-  r'localhost|'
-  r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
-  r'(?::\d+)?'
-  r'(?:/?|[/?]\S+)$', re.IGNORECASE)
 
 
 class StringField(BaseField):
@@ -104,6 +94,8 @@ class FloatField(BaseField):
         return float(value)
 
     def validate(self, value):
+        if isinstance(value, int):
+            value = float(value)
         assert isinstance(value, float)
 
         if self.min_value is not None and value < self.min_value:
@@ -191,6 +183,7 @@ class EmbeddedDocumentField(BaseField):
         if not isinstance(value, self.document):
             raise ValidationError('Invalid embedded document instance '
                                   'provided to an EmbeddedDocumentField')
+        self.document.validate(value)
 
     def lookup_member(self, member_name):
         return self.document._fields.get(member_name)
@@ -240,6 +233,28 @@ class ListField(BaseField):
         return self.field.lookup_member(member_name)
 
 
+class DictField(BaseField):
+    """A dictionary field that wraps a standard Python dictionary. This is
+    similar to an embedded document, but the structure is not defined.
+
+    .. versionadded:: 0.2.3
+    """
+
+    def validate(self, value):
+        """Make sure that a list of valid fields is being used.
+        """
+        if not isinstance(value, dict):
+            raise ValidationError('Only dictionaries may be used in a '
+                                  'DictField') 
+
+        if any(('.' in k or '$' in k) for k in value):
+            raise ValidationError('Invalid dictionary key name - keys may not ' 
+                                  'contain "." or "$" characters')
+
+    def lookup_member(self, member_name):
+        return BaseField(name=member_name)
+
+
 class ReferenceField(BaseField):
     """A reference to a document that will be automatically dereferenced on
     access (lazily).
@@ -271,20 +286,19 @@ class ReferenceField(BaseField):
         return super(ReferenceField, self).__get__(instance, owner)
 
     def to_mongo(self, document):
-        if isinstance(document, (str, unicode, pymongo.objectid.ObjectId)):
+        id_field_name = self.document_type._meta['id_field']
-            # document may already be an object id
+        id_field = self.document_type._fields[id_field_name]
-            id_ = document
+
-        else:
+        if isinstance(document, Document):
             # We need the id from the saved object to create the DBRef
             id_ = document.id
             if id_ is None:
                 raise ValidationError('You can only reference documents once '
                                       'they have been saved to the database')
+        else:
+            id_ = document
 
-        # id may be a string rather than an ObjectID object
+        id_ = id_field.to_mongo(id_)
-        if not isinstance(id_, pymongo.objectid.ObjectId):
-            id_ = pymongo.objectid.ObjectId(id_)
-
         collection = self.document_type._meta['collection']
         return pymongo.dbref.DBRef(collection, id_)
 

mongoengine/queryset.py

@@ -11,6 +11,14 @@ __all__ = ['queryset_manager', 'Q', 'InvalidQueryError',
 REPR_OUTPUT_SIZE = 20
 
 
+class DoesNotExist(Exception):
+	pass
+
+
+class MultipleObjectsReturned(Exception):
+    pass
+
+
 class InvalidQueryError(Exception):
     pass
 
@@ -25,14 +33,14 @@ class Q(object):
     AND = '&&'
     OPERATORS = {
         'eq': 'this.%(field)s == %(value)s',
-        'neq': 'this.%(field)s != %(value)s',
+        'ne': 'this.%(field)s != %(value)s',
         'gt': 'this.%(field)s > %(value)s',
         'gte': 'this.%(field)s >= %(value)s',
         'lt': 'this.%(field)s < %(value)s',
         'lte': 'this.%(field)s <= %(value)s',
         'lte': 'this.%(field)s <= %(value)s',
-        'in': 'this.%(field)s.indexOf(%(value)s) != -1',
+        'in': '%(value)s.indexOf(this.%(field)s) != -1',
-        'nin': 'this.%(field)s.indexOf(%(value)s) == -1',
+        'nin': '%(value)s.indexOf(this.%(field)s) == -1',
         'mod': '%(field)s %% %(value)s',
         'all': ('%(value)s.every(function(a){'
                 'return this.%(field)s.indexOf(a) != -1 })'),
@@ -257,7 +265,7 @@ class QuerySet(object):
     def _transform_query(cls, _doc_cls=None, **query):
         """Transform a query from Django-style format to Mongo format.
         """
-        operators = ['neq', 'gt', 'gte', 'lt', 'lte', 'in', 'nin', 'mod',
+        operators = ['ne', 'gt', 'gte', 'lt', 'lte', 'in', 'nin', 'mod',
                      'all', 'size', 'exists']
 
         mongo_query = {}
@@ -275,7 +283,7 @@ class QuerySet(object):
 
                 # Convert value to proper value
                 field = fields[-1]
-                if op in (None, 'neq', 'gt', 'gte', 'lt', 'lte'):
+                if op in (None, 'ne', 'gt', 'gte', 'lt', 'lte'):
                     value = field.prepare_query_value(value)
                 elif op in ('in', 'nin', 'all'):
                     # 'in', 'nin' and 'all' require a list of values
@@ -292,6 +300,46 @@ class QuerySet(object):
 
         return mongo_query
 
+    def get(self, *q_objs, **query):
+        """Retrieve the the matching object raising 
+        :class:`~mongoengine.queryset.MultipleObjectsReturned` or 
+        :class:`~mongoengine.queryset.DoesNotExist` exceptions if multiple or
+        no results are found.
+        """
+        self.__call__(*q_objs, **query)
+        count = self.count()
+        if count == 1:
+            return self[0]
+        elif count > 1:
+            message = u'%d items returned, instead of 1' % count
+            raise MultipleObjectsReturned(message)
+        else:
+            raise DoesNotExist('Document not found')
+
+    def get_or_create(self, *q_objs, **query):
+        """Retreive unique object or create, if it doesn't exist. Raises
+        :class:`~mongoengine.queryset.MultipleObjectsReturned` if multiple 
+        results are found. A new document will be created if the document 
+        doesn't exists; a dictionary of default values for the new document
+        may be provided as a keyword argument called :attr:`defaults`.
+        """
+        defaults = query.get('defaults', {})
+        if query.has_key('defaults'):
+            del query['defaults']
+        
+        self.__call__(*q_objs, **query)
+        count = self.count()
+        if count == 0:
+            query.update(defaults)
+            doc = self._document(**query)
+            doc.save()
+            return doc
+        elif count == 1:
+            return self.first()
+        else:
+            message = u'%d items returned, instead of 1' % count
+            raise MultipleObjectsReturned(message)
+
     def first(self):
         """Retrieve the first object matching the query.
         """
@@ -472,9 +520,10 @@ class QuerySet(object):
             self._collection.update(self._query, update, safe=safe_update, 
                                     multi=True)
         except pymongo.errors.OperationFailure, err:
-            if str(err) == 'multi not coded yet':
+            if unicode(err) == u'multi not coded yet':
-                raise OperationError('update() method requires MongoDB 1.1.3+')
+                message = u'update() method requires MongoDB 1.1.3+'
-            raise OperationError('Update failed (%s)' % str(err))
+                raise OperationError(message)
+            raise OperationError(u'Update failed (%s)' % unicode(err))
 
     def update_one(self, safe_update=True, **update):
         """Perform an atomic update on first field matched by the query.
@@ -660,14 +709,15 @@ class QuerySetManager(object):
         # owner is the document that contains the QuerySetManager
         queryset = QuerySet(owner, self._collection)
         if self._manager_func:
-            queryset = self._manager_func(queryset)
+            queryset = self._manager_func(owner, queryset)
         return queryset
 
 def queryset_manager(func):
     """Decorator that allows you to define custom QuerySet managers on
     :class:`~mongoengine.Document` classes. The manager must be a function that
-    accepts a :class:`~mongoengine.queryset.QuerySet` as its only argument, and
+    accepts a :class:`~mongoengine.Document` class as its first argument, and a
-    returns a :class:`~mongoengine.queryset.QuerySet`, probably the same one 
+    :class:`~mongoengine.queryset.QuerySet` as its second argument. The method
-    but modified in some way.
+    function should return a :class:`~mongoengine.queryset.QuerySet`, probably
+    the same one that was passed in, but modified in some way.
     """
     return QuerySetManager(func)

tests/document.py

@@ -157,6 +157,20 @@ class DocumentTest(unittest.TestCase):
                 meta = {'allow_inheritance': False}
         self.assertRaises(ValueError, create_employee_class)
         
+        # Test the same for embedded documents
+        class Comment(EmbeddedDocument):
+            content = StringField()
+            meta = {'allow_inheritance': False}
+
+        def create_special_comment():
+            class SpecialComment(Comment):
+                pass
+        self.assertRaises(ValueError, create_special_comment)
+
+        comment = Comment(content='test')
+        self.assertFalse('_cls' in comment.to_mongo())
+        self.assertFalse('_types' in comment.to_mongo())
+
     def test_collection_name(self):
         """Ensure that a collection with a specified name may be used.
         """
@@ -391,7 +405,7 @@ class DocumentTest(unittest.TestCase):
         
         self.assertTrue('content' in Comment._fields)
         self.assertFalse('id' in Comment._fields)
-        self.assertFalse(hasattr(Comment, '_meta'))
+        self.assertFalse('collection' in Comment._meta)
     
     def test_embedded_document_validation(self):
         """Ensure that embedded documents may be validated.

tests/fields.py

@@ -124,7 +124,7 @@ class FieldTest(unittest.TestCase):
         person.height = 1.89
         person.validate()
 
-        person.height = 2
+        person.height = '2.0'
         self.assertRaises(ValidationError, person.validate)
         person.height = 0.01
         self.assertRaises(ValidationError, person.validate)
@@ -212,6 +212,28 @@ class FieldTest(unittest.TestCase):
         post.comments = 'yay'
         self.assertRaises(ValidationError, post.validate)
 
+    def test_dict_validation(self):
+        """Ensure that dict types work as expected.
+        """
+        class BlogPost(Document):
+            info = DictField()
+
+        post = BlogPost()
+        post.info = 'my post'
+        self.assertRaises(ValidationError, post.validate)
+
+        post.info = ['test', 'test']
+        self.assertRaises(ValidationError, post.validate)
+
+        post.info = {'$title': 'test'}
+        self.assertRaises(ValidationError, post.validate)
+
+        post.info = {'the.title': 'test'}
+        self.assertRaises(ValidationError, post.validate)
+
+        post.info = {'title': 'test'}
+        post.validate()
+
     def test_embedded_document_validation(self):
         """Ensure that invalid embedded documents cannot be assigned to
         embedded document fields.
@@ -220,7 +242,7 @@ class FieldTest(unittest.TestCase):
             content = StringField()
 
         class PersonPreferences(EmbeddedDocument):
-            food = StringField()
+            food = StringField(required=True)
             number = IntField()
 
         class Person(Document):
@@ -231,9 +253,14 @@ class FieldTest(unittest.TestCase):
         person.preferences = 'My Preferences'
         self.assertRaises(ValidationError, person.validate)
 
+        # Check that only the right embedded doc works
         person.preferences = Comment(content='Nice blog post...')
         self.assertRaises(ValidationError, person.validate)
 
+        # Check that the embedded doc is valid
+        person.preferences = PersonPreferences()
+        self.assertRaises(ValidationError, person.validate)
+
         person.preferences = PersonPreferences(food='Cheese', number=47)
         self.assertEqual(person.preferences.food, 'Cheese')
         person.validate()
@@ -295,6 +322,40 @@ class FieldTest(unittest.TestCase):
         User.drop_collection()
         BlogPost.drop_collection()
 
+    def test_reference_query_conversion(self):
+        """Ensure that ReferenceFields can be queried using objects and values
+        of the type of the primary key of the referenced object.
+        """
+        class Member(Document):
+            user_num = IntField(primary_key=True)
+
+        class BlogPost(Document):
+            title = StringField()
+            author = ReferenceField(Member)
+
+        Member.drop_collection()
+        BlogPost.drop_collection()
+
+        m1 = Member(user_num=1)
+        m1.save()
+        m2 = Member(user_num=2)
+        m2.save()
+
+        post1 = BlogPost(title='post 1', author=m1)
+        post1.save()
+
+        post2 = BlogPost(title='post 2', author=m2)
+        post2.save()
+
+        post = BlogPost.objects(author=m1.id).first()
+        self.assertEqual(post.id, post1.id)
+
+        post = BlogPost.objects(author=m2.id).first()
+        self.assertEqual(post.id, post2.id)
+
+        Member.drop_collection()
+        BlogPost.drop_collection()
+
 
 if __name__ == '__main__':
     unittest.main()

tests/queryset.py

@@ -2,7 +2,8 @@ import unittest
 import pymongo
 from datetime import datetime
 
-from mongoengine.queryset import QuerySet
+from mongoengine.queryset import (QuerySet, MultipleObjectsReturned, 
+                                  DoesNotExist)
 from mongoengine import *
 
 
@@ -20,7 +21,7 @@ class QuerySetTest(unittest.TestCase):
         """Ensure that a QuerySet is correctly initialised by QuerySetManager.
         """
         self.assertTrue(isinstance(self.Person.objects, QuerySet))
-        self.assertEqual(self.Person.objects._collection.name(), 
+        self.assertEqual(self.Person.objects._collection.name, 
                          self.Person._meta['collection'])
         self.assertTrue(isinstance(self.Person.objects._collection,
                                    pymongo.collection.Collection))
@@ -135,6 +136,54 @@ class QuerySetTest(unittest.TestCase):
         person = self.Person.objects.with_id(person1.id)
         self.assertEqual(person.name, "User A")
 
+    def test_find_only_one(self):
+        """Ensure that a query using ``get`` returns at most one result.
+        """
+        # Try retrieving when no objects exists
+        self.assertRaises(DoesNotExist, self.Person.objects.get)
+
+        person1 = self.Person(name="User A", age=20)
+        person1.save()
+        person2 = self.Person(name="User B", age=30)
+        person2.save()
+
+        # Retrieve the first person from the database
+        self.assertRaises(MultipleObjectsReturned, self.Person.objects.get)
+
+        # Use a query to filter the people found to just person2
+        person = self.Person.objects.get(age=30)
+        self.assertEqual(person.name, "User B")
+
+        person = self.Person.objects.get(age__lt=30)
+        self.assertEqual(person.name, "User A")
+
+    def test_get_or_create(self):
+        """Ensure that ``get_or_create`` returns one result or creates a new
+        document.
+        """
+        person1 = self.Person(name="User A", age=20)
+        person1.save()
+        person2 = self.Person(name="User B", age=30)
+        person2.save()
+
+        # Retrieve the first person from the database
+        self.assertRaises(MultipleObjectsReturned, 
+                          self.Person.objects.get_or_create)
+
+        # Use a query to filter the people found to just person2
+        person = self.Person.objects.get_or_create(age=30)
+        self.assertEqual(person.name, "User B")
+
+        person = self.Person.objects.get_or_create(age__lt=30)
+        self.assertEqual(person.name, "User A")
+
+        # Try retrieving when no objects exists - new doc should be created
+        self.Person.objects.get_or_create(age=50, defaults={'name': 'User C'})
+
+        person = self.Person.objects.get(age=50)
+        self.assertEqual(person.name, "User C")
+
+
     def test_filter_chaining(self):
         """Ensure filters can be chained together.
         """
@@ -146,7 +195,7 @@ class QuerySetTest(unittest.TestCase):
             published_date = DateTimeField()
             
             @queryset_manager
-            def published(queryset):
+            def published(doc_cls, queryset):
                 return queryset(is_published=True)
                 
         blog_post_1 = BlogPost(title="Blog Post #1", 
@@ -252,7 +301,25 @@ class QuerySetTest(unittest.TestCase):
         
         BlogPost.drop_collection()
 
+    def test_find_dict_item(self):
+        """Ensure that DictField items may be found.
+        """
+        class BlogPost(Document):
+            info = DictField()
+
+        BlogPost.drop_collection()
+
+        post = BlogPost(info={'title': 'test'})
+        post.save()
+
+        post_obj = BlogPost.objects(info__title='test').first()
+        self.assertEqual(post_obj.id, post.id)
+
+        BlogPost.drop_collection()
+
     def test_q(self):
+        """Ensure that Q objects may be used to query for documents.
+        """
         class BlogPost(Document):
             publish_date = DateTimeField()
             published = BooleanField()
@@ -288,6 +355,15 @@ class QuerySetTest(unittest.TestCase):
 
         BlogPost.drop_collection()
 
+        # Check the 'in' operator
+        self.Person(name='user1', age=20).save()
+        self.Person(name='user2', age=20).save()
+        self.Person(name='user3', age=30).save()
+        self.Person(name='user4', age=40).save()
+        
+        self.assertEqual(len(self.Person.objects(Q(age__in=[20]))), 2)
+        self.assertEqual(len(self.Person.objects(Q(age__in=[20, 30]))), 3)
+
     def test_exec_js_query(self):
         """Ensure that queries are properly formed for use in exec_js.
         """
@@ -468,7 +544,7 @@ class QuerySetTest(unittest.TestCase):
             tags = ListField(StringField())
 
             @queryset_manager
-            def music_posts(queryset):
+            def music_posts(doc_cls, queryset):
                 return queryset(tags='music')
 
         BlogPost.drop_collection()
@@ -577,6 +653,8 @@ class QuerySetTest(unittest.TestCase):
 class QTest(unittest.TestCase):
     
     def test_or_and(self):
+        """Ensure that Q objects may be combined correctly.
+        """
         q1 = Q(name='test')
         q2 = Q(age__gte=18)