For testability reasons, we’ve begun using a lot of dependency injection in our Python projects. Imagine that I have a function that does some work and then ultimately calls an API over-the-wire:


import simplejson
from google.appengine.api import urlfetch # App Engine example
def my_method(user_id):
  url = 'http://www.example.com/lookup-user/%d.json' % user_id
  result = urlfetch.fetch(url)
  if result.status_code == 200:
    return simplejson.loads(result.content)
  return {}


This is bad for testability because when my test code executes my_method(), it will either go out over the wire (super bad!) unless I monkey-patch urlfetch to be some other class (really inconvenient).

So, instead, we can accept an alternate urlfetch mechanism optionally in the method:


from google.appengine.api import urlfetch as urlfetch_builtin
def my_method(user_id, urlfetch=urlfetch_builtin):
  url = 'http://www.example.com/lookup-user/%d.json' % user_id
  result = urlfetch.fetch(url)
  if result.status_code == 200:
    return simplejson.loads(result.content)
  return {}


So now the test client can pass in an alternate implementation of urlfetch that can return instrumented results and exceptions. This proves to be extremely handy.

One problem with this particular implementation is that it forces the client to do some odd things. Imagine that I have a piece of client code that itself is dependency injected:


def client_function(urlfetch=None):
  # here, I must check for an injection so that I know to provide it to my_method
  if urlfetch:
    return my_method(123, urlfetch=urlfetch)
  else:
    return my_method(123)


This is unfortunate. A better way to implement dependency injection and prevent this client if statement is to adjust my_method:


from google.appengine.api import urlfetch as urlfetch_builtin
def my_method(user_id, urlfetch=None):
  urlfetch = urlfetch or urlfetch_builtin
  # ...


Since urlfetch is required by the method, we can take None to mean “use the default implementation”. Now the client code becomes simply:


def client_function(urlfetch=None):
  return my_method(123, urlfetch=urlfetch)


Make sure you setup your dependency injection params in this way and save your clients some headaches!

One of the great things about templating systems is the ability to encapsulate repeatable chunks of code. One of the tough things about templating systems is management of a global “state” to ensure the the overall document that is emitted is as efficient as possible – especially in the HTML world were bandwidth is a performance consideration.

An example of this is where the templating chunks require a particular external resource, like JavaScript. You don’t want each invocation of a repeated sub-template to cause a tag to be emitted; you only need it once for the entire document.

Asp.Net has some handling for JavaScript in their UserControl mechanism (RegisterClientScriptBlock()) that uses a server-side registration mechanism to emit an appropriate document.

Dave Mosher and Brett Zabos, here at VendAsta, have used a pure JavaScript approach to get the JavaScript resources loaded efficiently, built atop the YUI loader. Make sure you check out their escapades.

Google App Engine doesn’t have a unique constraint in the classical sense of relational databases. This is a favourite construct of application developers and it’s unfortunate that it’s not present. At the same time, a basic understanding of the underlying datastore points to why it’s tough, or at least inefficient.

There are places where you’re willing to sacrifice some performance in order to guarantee a unique value. Datastore does guarantee uniqueness on its keys, so we can use a secondary helper model to guarantee uniqueness.


class Unique(db.Model):
  @classmethod
  def check(cls, scope, value):
    def tx(scope, value):
      key_name = "U%s:%s" % (scope, value,)
      ue = Unique.get_by_key_name(key_name)
      if ue:
        raise UniqueConstraintViolation(scope, value)
      ue = Unique(key_name=key_name)
      ue.put()
    db.run_in_transaction(tx, scope, value)
class UniqueConstraintViolation(Exception):
  def __init__(self, scope, value):
    super(UniqueConstraintViolation, self).__init__("Value '%s' is not unique within scope '%s'." % (value, scope, ))


This class simply leverages the key uniqueness aspect of datastore to ensure that the value doesn’t exist. A call to check() where a matching scope and value already exists will raise a UniqueConstraintViolation.

To use this, you need to build out a common create method on your models. Below, I’ve used a classmethod to achieve this:


class Account(db.Model):
  name = db.StringProperty()
  email = db.StringProperty() # unique=True - wouldn't that be great?
  @classmethod
  def create(cls, name, email):
    Unique.check("email", email)
    a = Account(name=name, email=email)
    a.put()
    return a


I need to make a call to Unique.check() first to ensure uniqueness (which causes at least a lookup, but likely a lookup and a put – both are a performance hit relative to doing nothing at all), and then I create my own account. Unique.check() will throw if the value is not unique, preventing the Account from being created.

Note that this technique lugs around an additional dictionary in datastore (costing some $$, though realistically not much). Also note that if you jam too many scopes into this class, you’ll get degrading performance (though it’s still just a key lookup which is very efficient).

Recently, I discovered something surprising about Google App Engine’s datastore and ReferenceProperty. Imagine I have a class like the following:


  from google.appengine.ext import db
  class Home(db.Model):
    address = db.StringProperty()
    room = db.ReferenceProperty(Room)


where Room is also a db.Model.

Datastore uses a proxying technique such that db.Model objects are created lightly (i.e., with only their key) and any hit to an attribute causes the entire object to be inflated by looking it up in the datastore.

I thought this was achieved with a lazy load technique, meaning that Room in the above example contained the logic to load itself (or more accurately Home would populate room with a proxy to Room that would know how to inflate). To do this, the proxy class would hold only its key and use that key to look itself up. From this, it follows that I could access that key without inflating the object.

It doesn’t work this way!

Instead, the referencing class (Home in the above example) holds the key (in a protected attribute, which you shouldn’t touch) and is responsible for inflating reference types.

I’ll say that again: the referenced object does not inflate itself lazily, but the referencing object does the inflation.

So, I previously thought code like this would not cause a datastore lookup:


  # assume home is an instance of Home
  room_key = home.room.key()


but I was very wrong. Simply hitting home.room causes home to lookup the entire room.

But what if you only want to get at the key (which the home has, but is protecting)?

This post suggests that the safe way to get this key is to access it via the class attribute, not the instance attribute. Here is the resulting code for my example:


  room_key = Home.room.get_value_for_datastore(home)


Note carefully the capitalization (indicating the classes versus objects).

This gives me the room key without causing a room lookup. Over and out.