async-mongo: motor

.. currentmodule:: motor.motor_asyncio

Tutorial: Using Motor With :mod:asyncio

.. These setups are redundant because I can't figure out how to make doctest
run a common setup before the setup for the two groups. A "testsetup:: *"
is the obvious answer, but it's run after group-specific setup.

.. testsetup:: before-inserting-2000-docs

import pymongo
import motor.motor_asyncio
import asyncio
from asyncio import coroutine
db = motor.motor_asyncio.AsyncIOMotorClient().test_database

.. testsetup:: after-inserting-2000-docs

import pymongo
import motor.motor_asyncio
import asyncio
from asyncio import coroutine
db = motor.motor_asyncio.AsyncIOMotorClient().test_database
pymongo.MongoClient().test_database.test_collection.insert_many(
[{'i': i} for i in range(2000)])

.. testcleanup:: *

import pymongo
pymongo.MongoClient().test_database.test_collection.delete_many({})

A guide to using MongoDB and asyncio with Motor.

.. contents::

Tutorial Prerequisites

You can learn about MongoDB with the MongoDB Tutorial_ before you learn Motor.

Using Python 3.4 or later, do::

$ python3 -m pip install motor

This tutorial assumes that a MongoDB instance is running on the
default host and port. Assuming you have downloaded and installed <http://docs.mongodb.org/manual/installation/>_ MongoDB, you
can start it like so:

.. code-block:: bash

$ mongod

.. _pip: http://www.pip-installer.org/en/latest/installing.html

.. _MongoDB Tutorial: http://docs.mongodb.org/manual/tutorial/getting-started/

Object Hierarchy

Motor, like PyMongo, represents data with a 4-level object hierarchy:

  • :class:~motor.motor_asyncio.AsyncIOMotorClient
    represents a mongod process, or a cluster of them. You explicitly create one
    of these client objects, connect it to a running mongod or mongods, and
    use it for the lifetime of your application.
  • :class:~motor.motor_asyncio.AsyncIOMotorDatabase: Each mongod has a set of databases (distinct
    sets of data files on disk). You can get a reference to a database from a
    client.
  • :class:~motor.motor_asyncio.AsyncIOMotorCollection: A database has a set of collections, which
    contain documents; you get a reference to a collection from a database.
  • :class:~motor.motor_asyncio.AsyncIOMotorCursor: Executing :meth:~motor.motor_asyncio.AsyncIOMotorCollection.find on
    an :class:~motor.motor_asyncio.AsyncIOMotorCollection gets an :class:~motor.motor_asyncio.AsyncIOMotorCursor, which
    represents the set of documents matching a query.

Creating a Client

You typically create a single instance of :class:~motor.motor_asyncio.AsyncIOMotorClient at the time your
application starts up.

.. doctest:: before-inserting-2000-docs

import motor.motor_asyncio
client = motor.motor_asyncio.AsyncIOMotorClient()

This connects to a mongod listening on the default host and port. You can
specify the host and port like:

.. doctest:: before-inserting-2000-docs

client = motor.motor_asyncio.AsyncIOMotorClient('localhost', 27017)

Motor also supports connection URIs_:

.. doctest:: before-inserting-2000-docs

client = motor.motor_asyncio.AsyncIOMotorClient('mongodb://localhost:27017')

Connect to a replica set like:

client = motor.motor_asyncio.AsyncIOMotorClient('mongodb://host1,host2/?replicaSet=my-replicaset-name')

.. _connection URIs: http://docs.mongodb.org/manual/reference/connection-string/

Getting a Database

A single instance of MongoDB can support multiple independent
databases <http://docs.mongodb.org/manual/reference/glossary/#term-database>_.
From an open client, you can get a reference to a particular database with
dot-notation or bracket-notation:

.. doctest:: before-inserting-2000-docs

db = client.test_database
db = client['test_database']

Creating a reference to a database does no I/O and does not require an
await expression.

Getting a Collection

A collection <http://docs.mongodb.org/manual/reference/glossary/#term-collection>_
is a group of documents stored in MongoDB, and can be thought of as roughly
the equivalent of a table in a relational database. Getting a
collection in Motor works the same as getting a database:

.. doctest:: before-inserting-2000-docs

collection = db.test_collection
collection = db['test_collection']

Just like getting a reference to a database, getting a reference to a
collection does no I/O and doesn't require an await expression.

Inserting a Document

As in PyMongo, Motor represents MongoDB documents with Python dictionaries. To
store a document in MongoDB, call :meth:~AsyncIOMotorCollection.insert_one in an
await expression:

.. doctest:: before-inserting-2000-docs

async def do_insert():
... document = {'key': 'value'}
... result = await db.test_collection.insert_one(document)
... print('result %s' % repr(result.inserted_id))
...

loop = asyncio.get_event_loop()
loop.run_until_complete(do_insert())
result ObjectId('...')

.. mongodoc:: insert

.. doctest:: before-inserting-2000-docs
:hide:

Clean up from previous insert

pymongo.MongoClient().test_database.test_collection.delete_many({})
<pymongo.results.DeleteResult ...>

Using native coroutines

Starting in Python 3.5, you can define a native coroutine_ with async def
instead of the coroutine decorator. Within a native coroutine, wait
for an async operation with await instead of yield:

.. doctest:: before-inserting-2000-docs

async def do_insert():
... for i in range(2000):
... result = await db.test_collection.insert_one({'i': i})
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_insert())

Within a native coroutine, the syntax to use Motor with Tornado or asyncio
is often identical.

.. _native coroutine: https://www.python.org/dev/peps/pep-0492/

Getting a Single Document With find_one

Use :meth:~motor.motor_asyncio.AsyncIOMotorCollection.find_one to get the first document that
matches a query. For example, to get a document where the value for key "i" is
less than 1:

.. doctest:: after-inserting-2000-docs

async def do_find_one():
... document = await db.test_collection.find_one({'i': {'$lt': 1}})
... pprint.pprint(document)
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_find_one())
{'_id': ObjectId('...'), 'i': 0}

The result is a dictionary matching the one that we inserted previously.

.. note:: The returned document contains an "_id", which was
automatically added on insert.

.. mongodoc:: find

Querying for More Than One Document

Use :meth:~motor.motor_asyncio.AsyncIOMotorCollection.find to query for a set of documents.
:meth:~motor.motor_asyncio.AsyncIOMotorCollection.find does no I/O and does not require an await
expression. It merely creates an :class:~motor.motor_asyncio.AsyncIOMotorCursor instance. The query is
actually executed on the server when you call :meth:~motor.motor_asyncio.AsyncIOMotorCursor.to_list
or execute an async for loop.

To find all documents with "i" less than 5:

.. doctest:: after-inserting-2000-docs

async def do_find():
... cursor = db.test_collection.find({'i': {'$lt': 5}}).sort('i')
... for document in await cursor.to_list(length=100):
... pprint.pprint(document)
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_find())
{'_id': ObjectId('...'), 'i': 0}
{'_id': ObjectId('...'), 'i': 1}
{'_id': ObjectId('...'), 'i': 2}
{'_id': ObjectId('...'), 'i': 3}
{'_id': ObjectId('...'), 'i': 4}

A length argument is required when you call to_list to prevent Motor
from buffering an unlimited number of documents.

async for


You can handle one document at a time in an ``async for`` loop:

.. doctest:: after-inserting-2000-docs

  >>> async def do_find():
  ...     c = db.test_collection
  ...     async for document in c.find({'i': {'$lt': 2}}):
  ...         pprint.pprint(document)
  ...
  >>> loop = asyncio.get_event_loop()
  >>> loop.run_until_complete(do_find())
  {'_id': ObjectId('...'), 'i': 0}
  {'_id': ObjectId('...'), 'i': 1}

You can apply a sort, limit, or skip to a query before you begin iterating:

.. doctest:: after-inserting-2000-docs

  >>> async def do_find():
  ...     cursor = db.test_collection.find({'i': {'$lt': 5}})
  ...     # Modify the query before iterating
  ...     cursor.sort('i', -1).limit(2).skip(2)
  ...     async for document in cursor:
  ...         pprint.pprint(document)
  ...
  >>> loop = asyncio.get_event_loop()
  >>> loop.run_until_complete(do_find())
  {'_id': ObjectId('...'), 'i': 2}
  {'_id': ObjectId('...'), 'i': 1}

The cursor does not actually retrieve each document from the server
individually; it gets documents efficiently in `large batches`_.

.. _`large batches`: https://docs.mongodb.com/manual/tutorial/iterate-a-cursor/#cursor-batches

Iteration in Python 3.3 and 3.4

In Python versions without async for, handle one document at a time with
:attr:~motor.motor_asyncio.AsyncIOMotorCursor.fetch_next
and :meth:~motor.motor_asyncio.AsyncIOMotorCursor.next_object:

.. doctest:: after-inserting-2000-docs

@coroutine
... def do_find():
... cursor = db.test_collection.find({'i': {'$lt': 5}})
... while (yield from cursor.fetch_next):
... document = cursor.next_object()
... pprint.pprint(document)
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_find())
{'_id': ObjectId('...'), 'i': 0}
{'_id': ObjectId('...'), 'i': 1}
async def do_find():
... cursor = db.test_collection.find({'i': {'$lt': 5}})
... # Modify the query before iterating
... cursor.sort('i', -1).limit(2).skip(2)
... async for document in cursor:
... pprint.pprint(document)
...
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(do_find())
{'_id': ObjectId('...'), 'i': 2}
{'id': ObjectId('...'), 'i': 1}

The cursor does not actually retrieve each document from the server
individually; it gets documents efficiently in large batches.

.. _large batches: https://docs.mongodb.com/manual/tutorial/iterate-a-cursor/#cursor-batches

Iteration in Python 3.3 and 3.4
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In Python versions without async for, handle one document at a time with
:attr:~motor.motor_asyncio.AsyncIOMotorCursor.fetch_next
and :meth:~motor.motor_asyncio.AsyncIOMotorCursor.next_object:

.. doctest:: after-inserting-2000-docs

>>> @coroutine
... def do_find():
... cursor = db.test_collection.find({'i': {'$lt': 5}})
... while (yield from cursor.fetch_next):
... document = cursor.next_object()
... pprint.pprint(document)
...
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(do_find())
{'_id': ObjectId('...'), 'i': 0}
{'_id': ObjectId('...'), 'i': 1}
{'_id': ObjectId('...'), 'i': 2}
{'_id': ObjectId('...'), 'i': 3}
{'_id': ObjectId('...'), 'i': 4}

Counting Documents

Use :meth:~motor.motor_asyncio.AsyncIOMotorCursor.count to determine the number of documents in
a collection, or the number of documents that match a query:

.. doctest:: after-inserting-2000-docs

async def do_count():
... n = await db.test_collection.find().count()
... print('%s documents in collection' % n)
... n = await db.test_collection.find({'i': {'$gt': 1000}}).count()
... print('%s documents where i > 1000' % n)
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_count())
2000 documents in collection
999 documents where i > 1000

:meth:~motor.motor_asyncio.AsyncIOMotorCursor.count uses the count command internally; we'll
cover commands_ below.

.. seealso:: Count command <http://docs.mongodb.org/manual/reference/command/count/>_

Updating Documents

:meth:~motor.motor_asyncio.AsyncIOMotorCollection.replace_one changes a document. It requires two
parameters: a query that specifies which document to replace, and a
replacement document. The query follows the same syntax as for :meth:find or
:meth:find_one. To replace a document:

.. doctest:: after-inserting-2000-docs

async def do_replace():
... coll = db.test_collection
... old_document = await coll.find_one({'i': 50})
... print('found document: %s' % pprint.pformat(old_document))
... _id = old_document['_id']
... result = await coll.replace_one({'_id': _id}, {'key': 'value'})
... print('replaced %s document' % result.modified_count)
... new_document = await coll.find_one({'_id': id})
... print('document is now %s' % pprint.pformat(new_document))
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_replace())
found document: {'id': ObjectId('...'), 'i': 50}
replaced 1 document
loop.run_until_complete(do_count())
2000 documents in collection
999 documents where i > 1000

:meth:~motor.motor_asyncio.AsyncIOMotorCursor.count uses the count command internally; we'll
cover commands below.

.. seealso:: Count command <http://docs.mongodb.org/manual/reference/command/count/>

Updating Documents
------------------

:meth:~motor.motor_asyncio.AsyncIOMotorCollection.replace_one changes a document. It requires two
parameters: a query that specifies which document to replace, and a
replacement document. The query follows the same syntax as for :meth:find or
:meth:find_one. To replace a document:

.. doctest:: after-inserting-2000-docs

>>> async def do_replace():
... coll = db.test_collection
... old_document = await coll.find_one({'i': 50})
... print('found document: %s' % pprint.pformat(old_document))
... _id = old_document['_id']
... result = await coll.replace_one({'_id': _id}, {'key': 'value'})
... print('replaced %s document' % result.modified_count)
... new_document = await coll.find_one({'_id': _id})
... print('document is now %s' % pprint.pformat(new_document))
...
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(do_replace())
found document: {'_id': ObjectId('...'), 'i': 50}
replaced 1 document
document is now {'_id': ObjectId('...'), 'key': 'value'}

You can see that :meth:replace_one replaced everything in the old document
except its _id with the new document.

Use :meth:~motor.motor_asyncio.AsyncIOMotorCollection.update_one with MongoDB's modifier operators to
update part of a document and leave the
rest intact. We'll find the document whose "i" is 51 and use the $set
operator to set "key" to "value":

.. doctest:: after-inserting-2000-docs

async def do_update():
... coll = db.test_collection
... result = await coll.update_one({'i': 51}, {'$set': {'key': 'value'}})
... print('updated %s document' % result.modified_count)
... new_document = await coll.find_one({'i': 51})
... print('document is now %s' % pprint.pformat(new_document))
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_update())
updated 1 document
document is now {'_id': ObjectId('...'), 'i': 51, 'key': 'value'}

"key" is set to "value" and "i" is still 51.

:meth:update_one only affects the first document it finds, you can
update all of them with :meth:update_many::

await coll.update_many({'i': {'$gt': 100}},
                       {'$set': {'key': 'value'}})

.. mongodoc:: update

Deleting Documents

:meth:~motor.motor_asyncio.AsyncIOMotorCollection.delete_many takes a query with the same syntax as
:meth:~motor.motor_asyncio.AsyncIOMotorCollection.find.
:meth:delete_many immediately removes all matching documents.

.. doctest:: after-inserting-2000-docs

async def do_delete_many():
... coll = db.test_collection
... n = await coll.count()
... print('%s documents before calling delete_many()' % n)
... result = await db.test_collection.delete_many({'i': {'$gte': 1000}})
... print('%s documents after' % (await coll.count()))
...
loop = asyncio.get_event_loop()
loop.run_until_complete(do_delete_many())
2000 documents before calling delete_many()
1000 documents after

.. mongodoc:: remove

Commands

Besides the "CRUD" operations--insert, update, delete, and find--all other
operations on MongoDB are commands. Run them using
the :meth:~motor.motor_asyncio.AsyncIOMotorDatabase.command method on :class:~motor.motor_asyncio.AsyncIOMotorDatabase:

.. doctest:: after-inserting-2000-docs

from bson import SON
async def use_count_command():
... response = await db.command(SON([("count", "test_collection")]))
... print('response: %s' % pprint.pformat(response))
...
loop = asyncio.get_event_loop()
loop.run_until_complete(use_count_command())
response: {'n': 1000, 'ok': 1.0...}

Since the order of command parameters matters, don't use a Python dict to pass
the command's parameters. Instead, make a habit of using :class:bson.SON,
from the bson module included with PyMongo::

await db.command(SON([("distinct", "test_collection"), ("key", "my_key"]))

Many commands have special helper methods, such as
:meth:~motor.motor_asyncio.AsyncIOMotorDatabase.create_collection or
:meth:~motor.motor_asyncio.AsyncIOMotorCollection.aggregate, but these are just conveniences atop
the basic :meth:command method.

.. mongodoc:: commands

.. _example-web-application-aiohttp:

A Web Application With aiohttp_

Let us create a web application using aiohttp_, a popular HTTP package for
asyncio. Install it with::

python3 -m pip install aiohttp

We are going to make a trivial web site with two pages served from MongoDB.
To begin:

.. literalinclude:: examples/aiohttp_example.py
:language: python3
:start-after: setup-start
:end-before: setup-end

The AsyncIOMotorClient constructor does not actually connect to MongoDB.
The client connects on demand, when you attempt the first operation.
We create it and assign the "test" database's handle to db.

The setup_db coroutine drops the "pages" collection (plainly, this code is
for demonstration purposes), then inserts two documents. Each document's page
name is its unique id, and the "body" field is a simple HTML page. Finally,
setup_db returns the database handle.

We'll use the setup_db coroutine soon. First, we need a request handler
that serves pages from the data we stored in MongoDB.

.. literalinclude:: examples/aiohttp_example.py
:language: python3
:start-after: handler-start
:end-before: handler-end

We start the server by running setup_db and passing the database handle
to an :class:aiohttp.web.Application:

.. literalinclude:: examples/aiohttp_example.py
:language: python3
:start-after: main-start
:end-before: main-end

Note that it is a common mistake to create a new client object for every
request; this comes at a dire performance cost. Create the client
when your application starts and reuse that one client for the lifetime
of the process. You can maintain the client by storing a database handle
from the client on your application object, as shown in this example.

Visit localhost:8080/pages/page-one and the server responds "Hello!".
At localhost:8080/pages/page-two it responds "Goodbye." At other URLs it
returns a 404.

The complete code is in the Motor repository in examples/aiohttp_example.py.

.. _aiohttp: https://aiohttp.readthedocs.io/

See also the :doc:examples/aiohttp_gridfs_example.

Further Reading

The handful of classes and methods introduced here are sufficient for daily
tasks. The API documentation for :class:~motor.motor_asyncio.AsyncIOMotorClient, :class:~motor.motor_asyncio.AsyncIOMotorDatabase,
:class:~motor.motor_asyncio.AsyncIOMotorCollection, and :class:~motor.motor_asyncio.AsyncIOMotorCursor provides a
reference to Motor's complete feature set.

Learning to use the MongoDB driver is just the beginning, of course. For
in-depth instruction in MongoDB itself, see The MongoDB Manual_.

最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 197,368评论 5赞 462
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 82,941评论 2赞 374
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 144,369评论 0赞 326
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 52,848评论 1赞 267
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 61,719评论 5赞 358
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 46,505评论 1赞 275
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 36,904评论 3赞 388
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 35,528评论 0赞 254
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 39,819评论 1赞 293
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 34,848评论 2赞 314
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 36,652评论 1赞 328
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 32,468评论 3赞 316
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 37,912评论 3赞 300
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,095评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 30,389评论 1赞 255
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 41,906评论 2赞 343
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 41,120评论 2赞 339

推荐阅读更多精彩内容

  • 我想抽根烟
    小时候我想抽根烟。不为别的，只是想像那些烟民一样吞云吐雾的，感受一下那种飘飘欲仙的感觉，顺便也自己牛逼一回，不...
    A魅惑衣品田田阅读 457评论 0赞 0
  • 毕业鉴定
    高考报名，给同学们写毕业鉴定。毕业鉴定，通俗点说“评语”，以前也有老师开玩笑说写“水浒传”的，其实正式的名称是“...
    潘家文阅读 674评论 0赞 2
  • Frequency Number
    为什么使用Git: 能够对文件版本控制和多人协作开发拥有强大的分支特性，所以能够灵活地以不同的工作流协同开发分布式...
    TW_TOB_陈丹阳阅读 253评论 0赞 0