Finished GridFS Documentation

* Also made GridFS replace test pass
This commit is contained in:
Steve Challis 2010-10-18 00:55:44 +01:00
parent 39e27735cc
commit 67736c849d
4 changed files with 88 additions and 28 deletions

View File

@ -47,9 +47,10 @@ into you settings module::
Storage Storage
======= =======
With MongoEngine's support for GridFS via the FileField, it is useful to have a With MongoEngine's support for GridFS via the :class:`~mongoengine.FileField`,
Django file storage backend that wraps this. The new storage module is called it is useful to have a Django file storage backend that wraps this. The new
GridFSStorage. Using it is very similar to using the default FileSystemStorage.:: storage module is called :class:`~mongoengine.django.GridFSStorage`. Using it
is very similar to using the default FileSystemStorage.::
fs = mongoengine.django.GridFSStorage() fs = mongoengine.django.GridFSStorage()
@ -57,29 +58,30 @@ GridFSStorage. Using it is very similar to using the default FileSystemStorage.:
All of the `Django Storage API methods All of the `Django Storage API methods
<http://docs.djangoproject.com/en/dev/ref/files/storage/>`_ have been <http://docs.djangoproject.com/en/dev/ref/files/storage/>`_ have been
implemented except ``path()``. If the filename provided already exists, an implemented except :func:`path`. If the filename provided already exists, an
underscore and a number (before # the file extension, if one exists) will be underscore and a number (before # the file extension, if one exists) will be
appended to the filename until the generated filename doesn't exist. The appended to the filename until the generated filename doesn't exist. The
``save()`` method will return the new filename.:: :func:`save` method will return the new filename.::
> fs.exists('hello.txt') >>> fs.exists('hello.txt')
True True
> fs.open('hello.txt').read() >>> fs.open('hello.txt').read()
'Hello, World!' 'Hello, World!'
> fs.size('hello.txt') >>> fs.size('hello.txt')
13 13
> fs.url('hello.txt') >>> fs.url('hello.txt')
'http://your_media_url/hello.txt' 'http://your_media_url/hello.txt'
> fs.open('hello.txt').name >>> fs.open('hello.txt').name
'hello.txt' 'hello.txt'
> fs.listdir() >>> fs.listdir()
([], [u'hello.txt']) ([], [u'hello.txt'])
All files will be saved and retrieved in GridFS via the ``FileDocument`` document, All files will be saved and retrieved in GridFS via the :class::`FileDocument`
allowing easy access to the files without the GridFSStorage backend.:: document, allowing easy access to the files without the GridFSStorage
backend.::
> from mongoengine.django.storage import FileDocument >>> from mongoengine.django.storage import FileDocument
> FileDocument.objects() >>> FileDocument.objects()
[<FileDocument: FileDocument object>] [<FileDocument: FileDocument object>]
.. versionadded:: 0.4 .. versionadded:: 0.4

View File

@ -1,11 +1,17 @@
====== ======
GridFS GridFS
====== ======
.. versionadded:: 0.4
Writing
-------
GridFS support comes in the form of the :class:`~mongoengine.FileField` field GridFS support comes in the form of the :class:`~mongoengine.FileField` field
object. This field acts as a file-like object and provides a couple of object. This field acts as a file-like object and provides a couple of
different ways of inserting and retrieving data. Metadata such as content-type different ways of inserting and retrieving data. Arbitrary metadata such as
can also be stored alongside the stored files. In the following example, an content type can also be stored alongside the files. In the following example,
document is created to store details about animals, including a photo: a document is created to store details about animals, including a photo::
class Animal(Document): class Animal(Document):
genus = StringField() genus = StringField()
@ -14,13 +20,64 @@ document is created to store details about animals, including a photo:
marmot = Animal('Marmota', 'Sciuridae') marmot = Animal('Marmota', 'Sciuridae')
marmot_photo = open('marmot.jpg') # Retrieve a photo from disk marmot_photo = open('marmot.jpg', 'r') # Retrieve a photo from disk
marmot.photo = marmot_photo # Store the photo in the document marmot.photo = marmot_photo # Store the photo in the document
marmot.photo.content_type = 'image/jpeg' # Store metadata
marmot.save() marmot.save()
So adding file data to a document is as easy as adding data to any other Another way of writing to a :class:`~mongoengine.FileField` is to use the
:func:`put` method. This allows for metadata to be stored in the same call as
the file::
.. versionadded:: 0.4 marmot.photo.put(marmot_photo, content_type='image/jpeg')
marmot.save()
Retrieval
---------
So using the :class:`~mongoengine.FileField` is just like using any other
field. The file can also be retrieved just as easily::
marmot = Animal.objects('Marmota').first()
photo = marmot.photo.read()
content_type = marmot.photo.content_type
Streaming
---------
Streaming data into a :class:`~mongoengine.FileField` is achieved in a
slightly different manner. First, a new file must be created by calling the
:func:`new_file` method. Data can then be written using :func:`write`::
marmot.photo.new_file()
marmot.photo.write('some_image_data')
marmot.photo.write('some_more_image_data')
marmot.photo.close()
marmot.photo.save()
Deletion
--------
Deleting stored files is achieved with the :func:`delete` method::
marmot.photo.delete()
.. note::
The FileField in a Document actually only stores the ID of a file in a
separate GridFS collection. This means that deleting a document
with a defined FileField does not actually delete the file. You must be
careful to delete any files in a Document as above before deleting the
Document itself.
Replacing files
---------------
Files can be replaced with the :func:`replace` method. This works just like
the :func:`put` method so even metadata can (and should) be replaced::
another_marmot = open('another_marmot.png', 'r')
marmot.photo.replace(another_marmot, content_type='image/png')

View File

@ -34,7 +34,7 @@ arguments. The keys in the keyword arguments correspond to fields on the
Fields on embedded documents may also be referred to using field lookup syntax Fields on embedded documents may also be referred to using field lookup syntax
by using a double-underscore in place of the dot in object attribute access by using a double-underscore in place of the dot in object attribute access
syntax:: syntax::
# This will return a QuerySet that will only iterate over pages that have # This will return a QuerySet that will only iterate over pages that have
# been written by a user whose 'country' field is set to 'uk' # been written by a user whose 'country' field is set to 'uk'
uk_pages = Page.objects(author__country='uk') uk_pages = Page.objects(author__country='uk')
@ -67,7 +67,7 @@ Query operators
=============== ===============
Operators other than equality may also be used in queries; just attach the Operators other than equality may also be used in queries; just attach the
operator name to a key with a double-underscore:: operator name to a key with a double-underscore::
# Only find users whose age is 18 or less # Only find users whose age is 18 or less
young_users = Users.objects(age__lte=18) young_users = Users.objects(age__lte=18)
@ -144,7 +144,7 @@ You may also index the query to retrieve a single result. If an item at that
index does not exists, an :class:`IndexError` will be raised. A shortcut for index does not exists, an :class:`IndexError` will be raised. A shortcut for
retrieving the first result and returning :attr:`None` if no result exists is retrieving the first result and returning :attr:`None` if no result exists is
provided (:meth:`~mongoengine.queryset.QuerySet.first`):: provided (:meth:`~mongoengine.queryset.QuerySet.first`)::
>>> # Make sure there are no users >>> # Make sure there are no users
>>> User.drop_collection() >>> User.drop_collection()
>>> User.objects[0] >>> User.objects[0]
@ -458,7 +458,7 @@ that you may use with these methods:
The syntax for atomic updates is similar to the querying syntax, but the The syntax for atomic updates is similar to the querying syntax, but the
modifier comes before the field, not after it:: modifier comes before the field, not after it::
>>> post = BlogPost(title='Test', page_views=0, tags=['database']) >>> post = BlogPost(title='Test', page_views=0, tags=['database'])
>>> post.save() >>> post.save()
>>> BlogPost.objects(id=post.id).update_one(inc__page_views=1) >>> BlogPost.objects(id=post.id).update_one(inc__page_views=1)

View File

@ -586,14 +586,14 @@ class GridFSProxy(object):
def put(self, file, **kwargs): def put(self, file, **kwargs):
if self.grid_id: if self.grid_id:
raise GridFSError('This document alreay has a file. Either delete ' raise GridFSError('This document already has a file. Either delete '
'it or call replace to overwrite it') 'it or call replace to overwrite it')
self.grid_id = self.fs.put(file, **kwargs) self.grid_id = self.fs.put(file, **kwargs)
def write(self, string): def write(self, string):
if self.grid_id: if self.grid_id:
if not self.newfile: if not self.newfile:
raise GridFSError('This document alreay has a file. Either ' raise GridFSError('This document already has a file. Either '
'delete it or call replace to overwrite it') 'delete it or call replace to overwrite it')
else: else:
self.new_file() self.new_file()
@ -622,6 +622,7 @@ class GridFSProxy(object):
def replace(self, file, **kwargs): def replace(self, file, **kwargs):
self.delete() self.delete()
self.grid_id = None
self.put(file, **kwargs) self.put(file, **kwargs)
def close(self): def close(self):