diff --git a/README.rst b/README.rst index 35816438..28b2dc67 100644 --- a/README.rst +++ b/README.rst @@ -1,8 +1,18 @@ +=========== MongoEngine =========== -MongoEngine is an ORM-like layer on top of PyMongo. +:Info: MongoEngine is an ORM-like layer on top of PyMongo. +:Author: Harry Marr (http://github.com/hmarr) -Tutorial available at http://hmarr.com/mongoengine/ +About +===== +MongoEngine is a Python Object-Document Mapper for working with MongoDB. +Documentation available at http://hmarr.com/mongoengine/ -- there is currently +a `tutorial `_, a `user guide +`_ and an `API reference +`_. -**Warning:** this software is still in development and should *not* be used -in production. +Dependencies +============ +pymongo 1.1+ +sphinx (optional -- for documentation generation) diff --git a/docs/apireference.rst b/docs/apireference.rst index 9ec4321a..ea8411e2 100644 --- a/docs/apireference.rst +++ b/docs/apireference.rst @@ -1,13 +1,14 @@ +============= API Reference ============= Connecting ----------- +========== .. autofunction:: mongoengine.connect Documents ---------- +========= .. autoclass:: mongoengine.Document :members: @@ -21,13 +22,13 @@ Documents :members: Querying --------- +======== .. autoclass:: mongoengine.queryset.QuerySet :members: Fields ------- +====== .. autoclass:: mongoengine.StringField diff --git a/docs/conf.py b/docs/conf.py index be7d3f5d..5abb2028 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -44,10 +44,11 @@ copyright = u'2009, Harry Marr' # |version| and |release|, also used in various other places throughout the # built documents. # +import mongoengine # The short X.Y version. -version = '0.1' +version = mongoengine.get_version() # The full version, including alpha/beta/rc tags. -release = '0.1' +release = mongoengine.get_release() # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. @@ -128,7 +129,7 @@ html_static_path = ['_static'] # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. -#html_use_smartypants = True +html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} diff --git a/docs/index.rst b/docs/index.rst index fbc5faa9..187f98f5 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -13,9 +13,9 @@ MongoDB. The source is available on .. toctree:: :maxdepth: 2 - tutorial.rst - userguide.rst - apireference.rst + tutorial + userguide + apireference Indices and tables ================== diff --git a/docs/tutorial.rst b/docs/tutorial.rst index 48069ef1..54a04c0b 100644 --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -1,3 +1,4 @@ +======== Tutorial ======== This tutorial introduces **MongoEngine** by means of example --- we will walk @@ -10,7 +11,7 @@ focus on the data-modelling side of the application, leaving out a user interface. Getting started ---------------- +=============== Before we start, make sure that a copy of MongoDB is running in an accessible location --- running it locally will be easier, but if that is not an option then it may be run on a remote server. @@ -27,7 +28,7 @@ database to use:: For more information about connecting to MongoDB see :ref:`guide-connecting`. Defining our documents ----------------------- +====================== MongoDB is *schemaless*, which means that no schema is enforced by the database --- we may add and remove fields however we want and MongoDB won't complain. This makes life a lot easier in many regards, especially when there is a change @@ -46,7 +47,7 @@ specified tag. Finally, it would be nice if **comments** could be added to posts. We'll start with **users**, as the others are slightly more involved. Users -^^^^^ +----- Just as if we were using a relational database with an ORM, we need to define which fields a :class:`User` may have, and what their types will be:: @@ -61,7 +62,7 @@ MongoDB --- this will only be enforced at the application level. Also, the User documents will be stored in a MongoDB *collection* rather than a table. Posts, Comments and Tags -^^^^^^^^^^^^^^^^^^^^^^^^ +------------------------ Now we'll think about how to store the rest of the information. If we were using a relational database, we would most likely have a table of **posts**, a table of **comments** and a table of **tags**. To associate the comments with @@ -73,7 +74,7 @@ several ways we can achieve this, but each of them have their problems --- none of them stand out as particularly intuitive solutions. Posts -""""" +^^^^^ But MongoDB *isn't* a relational database, so we're not going to do it that way. As it turns out, we can use MongoDB's schemaless nature to provide us with a much nicer solution. We will store all of the posts in *one collection* --- @@ -99,12 +100,12 @@ this kind of modelling out of the box:: link_url = StringField() We are storing a reference to the author of the posts using a -:class:`mongoengine.ReferenceField` object. These are similar to foreign key +:class:`~mongoengine.ReferenceField` object. These are similar to foreign key fields in traditional ORMs, and are automatically translated into references when they are saved, and dereferenced when they are loaded. Tags -"""" +^^^^ Now that we have our Post models figured out, how will we attach tags to them? MongoDB allows us to store lists of items natively, so rather than having a link table, we can just store a list of tags in each post. So, for both @@ -120,13 +121,13 @@ size of our database. So let's take a look that the code our modified author = ReferenceField(User) tags = ListField(StringField(max_length=30)) -The :class:`mongoengine.ListField` object that is used to define a Post's tags +The :class:`~mongoengine.ListField` object that is used to define a Post's tags takes a field object as its first argument --- this means that you can have lists of any type of field (including lists). Note that we don't need to modify the specialised post types as they all inherit from :class:`Post`. Comments -"""""""" +^^^^^^^^ A comment is typically associated with *one* post. In a relational database, to display a post with its comments, we would have to retrieve the post from the database, then query the database again for the comments associated with the @@ -152,7 +153,7 @@ We can then store a list of comment documents in our post document:: comments = ListField(EmbeddedDocumentField(Comment)) Adding data to our Tumblelog ----------------------------- +============================ Now that we've defined how our documents will be structured, let's start adding some documents to the database. Firstly, we'll need to create a :class:`User` object:: @@ -184,10 +185,10 @@ Note that if you change a field on a object that has already been saved, then call :meth:`save` again, the document will be updated. Accessing our data ------------------- +================== So now we've got a couple of posts in our database, how do we display them? Each document class (i.e. any class that inherits either directly or indirectly -from :class:`mongoengine.Document`) has an :attr:`objects` attribute, which is +from :class:`~mongoengine.Document`) has an :attr:`objects` attribute, which is used to access the documents in the database collection associated with that class. So let's see how we can get our posts' titles:: @@ -195,7 +196,7 @@ class. So let's see how we can get our posts' titles:: print post.title Retrieving type-specific information -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +------------------------------------ This will print the titles of our posts, one on each line. But What if we want to access the type-specific data (link_url, content, etc.)? One way is simply to use the :attr:`objects` attribute of a subclass of :class:`Post`:: @@ -205,7 +206,7 @@ to use the :attr:`objects` attribute of a subclass of :class:`Post`:: Using TextPost's :attr:`objects` attribute only returns documents that were created using :class:`TextPost`. Actually, there is a more general rule here: -the :attr:`objects` attribute of any subclass of :class:`mongoengine.Document` +the :attr:`objects` attribute of any subclass of :class:`~mongoengine.Document` only looks for documents that were created using that subclass or one of its subclasses. @@ -233,20 +234,21 @@ This would print the title of each post, followed by the content if it was a text post, and "Link: " if it was a link post. Searching our posts by tag -^^^^^^^^^^^^^^^^^^^^^^^^^^ -The :attr:`objects` attribute of a :class:`mongoengine.Document` is actually a -:class:`mongoengine.QuerySet` object. This lazily queries the database only -when you need the data. It may also be filtered to narrow down your query. -Let's adjust our query so that only posts with the tag "mongodb" are returned:: +-------------------------- +The :attr:`objects` attribute of a :class:`~mongoengine.Document` is actually a +:class:`~mongoengine.queryset.QuerySet` object. This lazily queries the +database only when you need the data. It may also be filtered to narrow down +your query. Let's adjust our query so that only posts with the tag "mongodb" +are returned:: for post in Post.objects(tags='mongodb'): print post.title -There are also methods available on :class:`mongoengine.QuerySet` objects that -allow different results to be returned, for example, calling :meth:`first` on -the :attr:`objects` attribute will return a single document, the first matched -by the query you provide. Aggregation functions may also be used on -:class:`mongoengine.QuerySet` objects:: +There are also methods available on :class:`~mongoengine.queryset.QuerySet` +objects that allow different results to be returned, for example, calling +:meth:`first` on the :attr:`objects` attribute will return a single document, +the first matched by the query you provide. Aggregation functions may also be +used on :class:`~mongoengine.queryset.QuerySet` objects:: num_posts = Post.objects(tags='mongodb').count() print 'Found % posts with tag "mongodb"' % num_posts diff --git a/docs/userguide.rst b/docs/userguide.rst index d4b8ecab..c196fd16 100644 --- a/docs/userguide.rst +++ b/docs/userguide.rst @@ -1,10 +1,11 @@ +========== User Guide ========== .. _guide-connecting: Connecting to MongoDB ---------------------- +===================== To connect to a running instance of :program:`mongod`, use the :func:`~mongoengine.connect` function. The first argument is the name of the database to connect to. If the database does not exist, it will be created. If @@ -22,7 +23,7 @@ provide :attr:`host` and :attr:`port` arguments to connect('project1', host='192.168.1.35', port=12345) Defining documents ------------------- +================== In MongoDB, a **document** is roughly equivalent to a **row** in an RDBMS. When working with relational databases, rows are stored in **tables**, which have a strict **schema** that the rows follow. MongoDB stores documents in @@ -30,7 +31,7 @@ strict **schema** that the rows follow. MongoDB stores documents in is enforced at a database level. Defining a document's schema -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +---------------------------- MongoEngine allows you to define schemata for documents as this helps to reduce coding errors, and allows for utility methods to be defined on fields which may be present. @@ -47,7 +48,7 @@ objects** as class attributes to the document class:: date_modified = DateTimeField(default=datetime.now) Fields -^^^^^^ +------ By default, fields are not required. To make a field mandatory, set the :attr:`required` keyword argument of a field to ``True``. Fields also may have validation constraints available (such as :attr:`max_length` in the example @@ -60,12 +61,69 @@ are as follows: * :class:`~mongoengine.IntField` * :class:`~mongoengine.FloatField` * :class:`~mongoengine.DateTimeField` +* :class:`~mongoengine.ListField` * :class:`~mongoengine.ObjectIdField` * :class:`~mongoengine.EmbeddedDocumentField` * :class:`~mongoengine.ReferenceField` +List fields +^^^^^^^^^^^ +MongoDB allows the storage of lists of items. To add a list of items to a +:class:`~mongoengine.Document`, use the :class:`~mongoengine.ListField` field +type. :class:`~mongoengine.ListField` takes another field object as its first +argument, which specifies which type elements may be stored within the list:: + + class Page(Document): + tags = ListField(StringField(max_length=50)) + +Embedded documents +^^^^^^^^^^^^^^^^^^ +MongoDB has the ability to embed documents within other documents. Schemata may +be defined for these embedded documents, just as they may be for regular +documents. To create an embedded document, just define a document as usual, but +inherit from :class:`~mongoengine.EmbeddedDocument` rather than +:class:`~mongoengine.Document`:: + + class Comment(EmbeddedDocument): + content = StringField() + +To embed the document within another document, use the +:class:`~mongoengine.EmbeddedDocumentField` field type, providing the embedded +document class as the first argument:: + + class Page(Document): + comments = ListField(EmbeddedDocumentField(Comment)) + + comment1 = Comment('Good work!') + comment2 = Comment('Nice article!') + page = Page(comments=[comment1, comment2]) + +Reference fields +^^^^^^^^^^^^^^^^ +References may be stored to other documents in the database using the +:class:`~mongoengine.ReferenceField`. Pass in another document class as the +first argument to the constructor, then simply assign document objects to the +field:: + + class User(Document): + name = StringField() + + class Page(Document): + content = StringField() + author = ReferenceField(User) + + john = User(name="John Smith") + john.save() + + post = Page(content="Test Page") + post.author = john + post.save() + +The :class:`User` object is automatically turned into a reference behind the +scenes, and dereferenced when the :class:`Page` object is retrieved. + Document collections -^^^^^^^^^^^^^^^^^^^^ +-------------------- Document classes that inherit **directly** from :class:`~mongoengine.Document` will have their own **collection** in the database. The name of the collection is by default the name of the class, coverted to lowercase (so in the example @@ -78,3 +136,175 @@ document class to use:: class Page(Document): title = StringField(max_length=200, required=True) meta = {'collection': 'cmsPage'} + +Document inheritance +-------------------- +To create a specialised type of a :class:`~mongoengine.Document` you have +defined, you may subclass it and add any extra fields or methods you may need. +As this is new class is not a direct subclass of +:class:`~mongoengine.Document`, it will not be stored in its own collection; it +will use the same collection as its superclass uses. This allows for more +convenient and efficient retrieval of related documents:: + + # Stored in a collection named 'page' + class Page(Document): + title = StringField(max_length=200, required=True) + + # Also stored in the collection named 'page' + class DatedPage(Page): + date = DateTimeField() + +Working with existing data +^^^^^^^^^^^^^^^^^^^^^^^^^^ +To enable correct retrieval of documents involved in this kind of heirarchy, +two extra attributes are stored on each document in the database: :attr:`_cls` +and :attr:`_types`. These are hidden from the user through the MongoEngine +interface, but may not be present if you are trying to use MongoEngine with +an existing database. For this reason, you may disable this inheritance +mechansim, removing the dependency of :attr:`_cls` and :attr:`_types`, enabling +you to work with existing databases. To disable inheritance on a document +class, set :attr:`allow_inheritance` to ``False`` in the :attr:`meta` +dictionary:: + + # Will work with data in an existing collection named 'cmsPage' + class Page(Document): + title = StringField(max_length=200, required=True) + meta = { + 'collection': 'cmsPage', + 'allow_inheritance': False, + } + +Documents instances +=================== +To create a new document object, create an instance of the relevant document +class, providing values for its fields as its constructor keyword arguments. +You may provide values for any of the fields on the document, but only +**required** fields are necessary at this stage:: + + >>> page = Page(title="Test Page") + >>> page.title + 'Test Page' + +You may also assign values to the document's fields using standard object +attribute syntax:: + + >>> page.title = "Example Page" + >>> page.title + 'Example Page' + +Saving and deleting documents +----------------------------- +To save the document to the database, call the +:meth:`~mongoengine.Document.save` method. If the document does not exist in +the database, it will be created. If it does already exist, it will be +updated. + +To delete a document, call the :meth:`~mongoengine.Document.delete` method. +Note that this will only work if the document exists in the database and has a +valide :attr:`id`. + +Document IDs +------------ +Each document in the database has a unique id. This may be accessed through the +:attr:`id` attribute on :class:`~mongoengine.Document` objects. Usually, the id +will be generated automatically by the database server when the object is save, +meaning that you may only access the :attr:`id` field once a document has been +saved:: + + >>> page = Page(title="Test Page") + >>> page.id + ... + AttributeError('_id') + >>> page.save() + >>> page.id + ObjectId('123456789abcdef000000000') + +Alternatively, you may explicitly set the :attr:`id` before you save the +document, but the id must be a valid PyMongo :class:`ObjectId`. + +Querying the database +===================== +:class:`~mongoengine.Document` classes have an :attr:`objects` attribute, which +is used for accessing the objects in the database associated with the class. +The :attr:`objects` attribute is actually a +:class:`~mongoengine.queryset.QuerySetManager`, which creates and returns a new +a new :class:`~mongoengine.queryset.QuerySet` object on access. The +:class:`~mongoengine.queryset.QuerySet` object may may be iterated over to +fetch documents from the database:: + + # Prints out the names of all the users in the database + for user in User.objects: + print user.name + +Filtering queries +----------------- +The query may be filtered by calling the +:class:`~mongoengine.queryset.QuerySet` object with field lookup keyword +arguments. The keys in the keyword arguments correspond to fields on the +:class:`~mongoengine.Document` you are querying:: + + # This will return a QuerySet that will only iterate over users whose + # 'country' field is set to 'uk' + uk_users = User.objects(country='uk') + +Fields on embedded documents may also be referred to using field lookup syntax +by using a double-underscore in place of the dot in object attribute access +syntax:: + + # This will return a QuerySet that will only iterate over pages that have + # been written by a user whose 'country' field is set to 'uk' + uk_pages = Page.objects(author__country='uk') + +Querying lists +^^^^^^^^^^^^^^ +On most fields, this syntax will look up documents where the field specified +matches the given value exactly, but when the field refers to a +:class:`~mongoengine.ListField`, a single item may be provided, in which case +lists that contain that item will be matched:: + + class Page(Document): + tags = ListField(StringField()) + + # This will match all pages that have the word 'coding' as an item in the + # 'tags' list + Page.objects(tags='coding') + +Query operators +--------------- +Operators other than equality may also be used in queries; just attach the +operator name to a key with a double-underscore:: + + # Only find users whose age is 18 or less + young_users = Users.objects(age__lte=18) + +Available operators are as follows: + +* ``neq`` -- not equal to +* ``lt`` -- less than +* ``lte`` -- less than or equal to +* ``gt`` -- greater than +* ``gte`` -- greater than or equal to +* ``in`` -- value is in list (a list of values should be provided) +* ``nin`` -- value is not in list (a list of values should be provided) +* ``mod`` -- ``value % x == y``, where ``x`` and ``y`` are two provided values +* ``all`` -- every item in array is in list of values provided +* ``size`` -- the size of the array is +* ``exists`` -- value for field exists + +Limiting and skipping results +----------------------------- +Just as with traditional ORMs, you may limit the number of results returned, or +skip a number or results in you query. +:meth:`mongoengine.queryset.QuerySet.limit` and +:meth:`mongoengine.queryset.QuerySet.skip` and methods are available on +:meth:`mongoengine.queryset.QuerySet` objects, but the prefered syntax for +achieving this is using array-slicing syntax:: + + # Only the first 5 people + users = User.objects[:5] + + # All except for the first 5 people + users = User.objects[5:] + + # 5 users, starting from the 10th user found + users = User.objects[10:15] diff --git a/mongoengine/__init__.py b/mongoengine/__init__.py index b91ca4b5..ae6b7c19 100644 --- a/mongoengine/__init__.py +++ b/mongoengine/__init__.py @@ -8,5 +8,20 @@ from connection import * __all__ = document.__all__ + fields.__all__ + connection.__all__ __author__ = 'Harry Marr' -__version__ = '0.1' + +VERSION = (0, 1, 0, 'alpha') + +def get_version(): + version = '%s.%s' % (VERSION[0], VERSION[1]) + if VERSION[2]: + version = '%s.%s' % (version, VERSION[2]) + return version + +def get_release(): + version = get_version() + if VERSION[3] != 'final': + version = '%s-%s' % (version, VERSION[3]) + return version + +__version__ = get_release()