Features¶
Rather than trying to rewrite Pyramid, Aiopyramid
provides a set of features that will allow you to run existing code asynchronously
where possible.
Views¶
Aiopyramid
provides three view mappers for calling view callables:
CoroutineOrExecutorMapper
maps views to coroutines or separate threadsCoroutineMapper
maps views to coroutinesExecutorMapper
maps views to separate threads
When you include Aiopyramid
,
the default view mapper is replaced with the CoroutineOrExecutorMapper
which detects whether your view callable is a coroutine and does a yield from
to
call it asynchronously. If your view callable is not a coroutine, it will run it in a
separate thread to avoid blocking the thread with the main loop. asyncio
is not thread-safe,
so you will need to guarantee that either in memory resources are not shared between
view callables running in the executor or that such resources are synchronized.
This means that you should not necessarily have to change existing views. Also,
it is possible to restore the default view mapper, but note that this will mean that
coroutine views that do not specify CoroutineMapper
as their
view mapper will fail.
If most of your view needs to be a coroutine but you want to call out to code that blocks, you can
always use run_in_executor. Aiopyramid also provides a decorator, use_executor()
,
for specifying declaratively that a particular routine should run in a separate thread.
For example:
import asyncio
from aiopyramid.helpers import use_executor
class DatabaseUtilies:
@use_executor # query_it is now a coroutine
def query_it():
# some code that blocks
Authorization¶
If you are using the default authorization policy, then you will generally not need to make any modifications
to authorize users with Aiopyramid
. The exception is if you want to use a callable that performs
some io for your __acl__. In that case you will simply need to use a synchronized coroutine so
that the authorization policy can call your coroutine like a normal Python function during view lookup.
For example:
import asyncio
from aiopyramid.helpers import synchronize
class MyResource:
"""
This resource uses a callable for it's
__acl__ that accesses the db.
"""
# this
__acl__ = synchronize(my_coroutine)
# or this
@synchronize
@asyncio.coroutine
def __acl__(self):
...
# will work
If you are using a custom authorization policy, most likely it will work with Aiopyramid
in the same
fashion, but it is up to you to guarantee that it does.
Authentication¶
Authentication poses a problem because the interface for authentication policies uses normal Python methods that the framework expects to call noramlly but at the same time it is usually necessary to perform some io to retrieve relevant information. The built-in authentication policies generally accept a callback function that delegates retrieving principals to the application, but this callback function is also expected to be called in the regular fashion. So, it is necessary to use a synchronized coroutine as a callback function.
The final problem is that synchronized coroutines are expected
to be called from within a child greenlet, or in other words from within framework code (see Architecture).
However, it is often the case that we will want to access the policy through pyramid.request.Request.authenticated_userid
or by calling remember()
, etc. from within another coroutine such as a view callable.
To handle both situations, Aiopyramid
provides tools for wrapping a callback-based authentication policy to
work asynchronously. For example, the following code in your app constructor will allow you to use a coroutine as
a callback.
from pyramid.authentication import AuthTktAuthenticationPolicy
from aiopyramid.auth import authn_policy_factory
from .myauth import get_principals
...
# In the includeme or constructor
authentication = authn_policy_factory(
AuthTktAuthenticationPolicy,
get_principals,
'sosecret',
hashalg='sha512'
)
config.set_authentication_policy(authentication)
Relevant authentication tools will now return a coroutine when called from another coroutine, so you
would access the authentication policy using yield from
in your view callable since it performs io.
from pyramid.security import remember, forget
...
# in some coroutine
maybe = yield from request.unauthenticated_userid
checked = yield from request.authenticated_userid
principals = yield from request.effective_principals
headers = yield from remember(request, 'george')
fheaders = yield from forget(request)
Note
If you don’t perform asynchronous io or wrap the authentication policy as above,
then don’t use yield from
in your view. This approach only works for coroutine
views. If you have both coroutine views and legacy views running in an executor,
you will probably need to write a custom authentication policy.
Tweens¶
Pyramid allows you to write tweens which wrap the request/response chain. Most
existing tweens expect those tweens above and below them to run synchronously. Therefore,
if you have a tween that needs to run asynchronously (e.g. it looks up some data from a
database for each request), then you will need to write that tween so that it can wait
without other tweens needing to explicitly yield from
it. For example:
import asyncio
from aiopyramid.helpers import synchronize
def coroutine_logger_tween_factory(handler, registry):
"""
Example of an asynchronous tween that delegates
a synchronous function to a child thread.
This tween asynchronously logs all requests and responses.
"""
# We use the synchronize decorator because we will call this
# coroutine from a normal python context
@synchronize
# this is a coroutine
@asyncio.coroutine
def _async_print(content):
# print doesn't really need to be run in a separate thread
# but it works for demonstration purposes
yield from asyncio.get_event_loop().run_in_executor(
None,
print,
content
)
def coroutine_logger_tween(request):
# The following calls are guaranteed to happen in order
# but they do not block the event loop
# print the request on the aio event loop
# without needing to say yield
# at this point,
# other coroutines and requests can be handled
_async_print(request)
# get response, this should be done in this greenlet
# and not as a coroutine because this will call
# the next tween and subsequently yield if necessary
response = handler(request)
# print the response on the aio event loop
_async_print(request)
# return response after logging is done
return response
return coroutine_logger_tween
Traversal¶
When using Pyramid’s traversal view lookup, it is often the case that you will want to make some io calls to a database or storage when traversing via __getitem__. When using the default traverser, Pyramid will call __getitem__ as a normal Python function. Therefore, it is necessary to synchronize __getitem__ on any asynchronous resources like so:
import asyncio
from aiopyramid.helpers import synchronize
class MyResource:
""" This resource performs some asynchronous io. """
__name__ = "example"
__parent__ = None
@synchronize
@asyncio.coroutine
def __getitem__(self, key):
yield from self.example_coroutine()
return self # no matter the path, this is the context
@asyncio.coroutine
def example_coroutine(self):
yield from asyncio.sleep(0.1)
print('I am some async task.')
Servers¶
Aiopyramid
supports both asynchronous gunicorn and the uWSGI asyncio plugin.
Example gunicorn config:
[server:main]
use = egg:gunicorn#main
host = 0.0.0.0
port = 6543
worker_class = aiopyramid.gunicorn.worker.AsyncGunicornWorker
Example uWSGI config:
[uwsgi]
http-socket = 0.0.0.0:6543
workers = 1
plugins =
asyncio = 50
greenlet
For those setting up Aiopyramid
on a Mac, Ander Ustarroz’s tutorial may prove useful.
Rickert Mulder has also provided a fork of uWSGI that allows for quick installation by running
pip install git+git://github.com/circlingthesun/uwsgi.git in a virtualenv.
Websockets¶
Aiopyramid
provides additional view mappers for handling websocket connections with either
gunicorn or uWSGI. Websockets with gunicorn use the websockets library whereas
uWSGI has native websocket support. In either case, the interface is the same.
A function view callable for a websocket connection follows this pattern:
@view_config(mapper=<WebsocketMapper>)
def websocket_callable(ws):
# do stuff with ws
The ws
argument passed to the callable has three methods for communicating with the websocket
recv()
, send()
, and close()
methods, which correspond to similar methods in the websockets library.
A websocket connection that echoes all messages using gunicorn would be:
from pyramid.view import view_config
from aiopyramid.websocket.config import WebsocketMapper
@view_config(route_name="ws", mapper=WebsocketMapper)
def echo(ws):
while True:
message = yield from ws.recv()
if message is None:
break
yield from ws.send(message)
Aiopyramid
also provides a view callable class WebsocketConnectionView
that has on_message()
,
on_open()
,
and on_close()
callbacks.
Class-based websocket views also have a send()
convenience method,
otherwise the underyling ws
may be accessed as self.ws
.
Simply extend WebsocketConnectionView
specifying the correct view mapper for your server either via the __view_mapper__
attribute or the
view_config
decorator. The above example could be rewritten in a larger project, this time using uWSGI,
as follows:
from pyramid.view import view_config
from aiopyramid.websocket.view import WebsocketConnectionView
from aiopyramid.websocket.config import UWSGIWebsocketMapper
from myproject.resources import MyWebsocketContext
class MyWebsocket(WebsocketConnectionView):
__view_mapper__ = UWSGIWebsocketMapper
@view_config(context=MyWebsocketContext)
class EchoWebsocket(MyWebsocket):
def on_message(self, message):
yield from self.send(message)
The underyling websocket implementations of uWSGI and websockets differ in how they pass on
the WebSocket message. uWSGI always sends bytes even when the WebSocket frame indicates that
the message is text, whereas websockets decodes text messages to str.
Aiopyramid attempts to match the behavior of websockets by default, which means
that it coerces messages from uWSGI to str where possible. To adjust this behavior, you can set the
use_str
flag to False, or alternatively to coerce
websockets messages back to bytes, set the use_bytes
flag to True:
# In your app constructor
from aiopyramid.websocket.config import WebsocketMapper
WebsocketMapper.use_bytes = True
uWSGI Special Note¶
Aiopyramid
uses a special WebsocketClosed
exception
to disconnect a greenlet after a websocket
has been closed. This exception will be visible in log ouput when using uWSGI. In order to squelch this
message, wrap the wsgi application in the ignore_websocket_closed()
middleware
in your application’s constructor like so:
from aiopyramid.websocket.helpers import ignore_websocket_closed
...
app = config.make_wsgi_app()
return ignore_websocket_closed(app)