As a growing company with a growing customer base, we’re always thinking about ways to boost the performance of our app (but who isn’t?). We’ve got some long-running queries that occasionally result in page timeouts, and if not, some general slowness for the user. These are things to be avoided. This week, we connected our follower database to the app to pull some of these read-only queries off of the main database to decrease the load.
Initial setup of an additional (already existing) database is fairly basic, and can be done by following the docs. We have something like this in our settings:
DATABASES = {
'default':
,
'read-only':
}
All database queries, both reads and writes, will run off of ‘default’ unless you specify otherwise. You can make a query off of the ‘read-only’ database like so:
Object.objects.using('read-only').get(id=1)
Seems great. And is great, if your scope is limited. But I don’t want .using('read-only’) calls sprinkled all throughout my app, and I don’t want to remember to write them. We wanted to be able to specify a different database than the default for an entire view method (controller action, for the Rails folks out there), that would apply to every call within it. Decorators to the rescue!
Coming from a Rails background, the closest equivalent I can come up with is a before_action, though Django’s decorators are actually a little more versatile in that. Basically, they are functions that take another function as an argument, and execute their own code around it. Whereas before_actions in Rails happen, definitionally, before the action (function) is called, decorators wrap around a function and can execute code both before and after. Here’s a common use case:
def login_required_or_401(view_func):
def _decorator(request, *ars, **kwargs):
if not request.user.is_authenticated():
return HttpResponse(status=401)
return view_func(request, *ars, **kwargs)
return wraps(view_func)(_decorator)
You’d call this like so:
@login_required_or_401
def some_view_function(self):
...
A lot of what comes next was inspired by this blog, with some tweaks to fit our use case. You can read more about threading here.
Where you put your decorator depends on where you think you’ll use it — you know best! We may use ours project-wide, so they’re in an app called ‘common’ we have for just that purpose, in a file called decorators.py. Class based decorators aren’t particularly common, but it proves useful for reasons we’ll come back to.
import threading
threadlocal = threading.local()
class use_db_for_reads(object):
def __init__(self, database_name):
self.database_name = database_name
def __enter__(self):
setattr(threadlocal, ‘DB_FOR_READ_ONLY’, self.database_name)
def __exit__(self, exc_type, exc_value, traceback):
setattr(threadlocal, ‘DB_FOR_READ_ONLY’, None)
def __call__(self, test_func):
@wraps(test_func)
def inner(*args, **kwargs):
return test_func(*args, **kwargs)
return inner
def get_thread_local(attr, default=None):
return getattr(threadlocal, attr, default)
This is what I mean by being able to wrap your code, not just execute something before it, using the __enter__ and __exit__ functions.
You then need a router. More info on routers can be found back in the docs, but basically, they tell the code which database to execute a request on. Our writes will always go to default, while our reads will come from the ‘read-only’ database, if we use the decorator from above:
from common.decorators import get_thread_local
class AnalyticsRouter(object):
def db_for_read(self, model, **hints):
return get_thread_local('DB_FOR_READ_ONLY', 'default')
def db_for_write(self, model, **hints):
return 'default'
def allow_relation(self, obj1, obj2, **hints):
return True
Back in settings.py, don’t forget to add your new router, keeping in mind that they’re run through sequentially, and stop once a match has been found — order matters.
DATABASE_ROUTERS = ['common.routers.AnalyticsRouter']
Using the decorator The benefit of the class based decorator is that it can be used as a decorator:
@use_db_for_reads('read-only')
def view_function(request, *args):
...
But it can also be used for a block, if you only want queries in part of a view to be read off the read-only database:
def view_function(request, *args):
with use_db_for_reads('read_only'):
...
#Below will use the default db
...