Dynamically Altering Classes in Python
The Very Basics
Everything in Python is an object. Don't believe me?
In : type(42) Out: <class 'int'>
Suffice to say that more complicated things are also classes. Derive from
object to declare your own classes, and then create an instance:
class Cat(object): # Add a property @property def legs(self): return 4 # Add a method def meow(self): print('meow') cat_1 = Cat() cat_1.meow() # meow cat_1.legs # 4
dir() on will show you what methods and properties are on the object:
In : dir(cat_1) Out: ['legs', 'meow']
Adding methods dynamically
One of the best things about Python is its delightfully adult attitude towards runtime mutability: if you have ever found yourself asking "can I add a method to an object after initializing it?", you should know not only that the answer is "yes", but that Python also makes it easy (and relatively safe) to do so by using the
MethodType method from the
import types def pounce(*args): print("pounce!") setattr(cat_1, 'pounce', types.MethodType(pounce, cat_1)) In : dir(cat_1) Out: [ 'legs', 'meow', 'pounce']
It's important to note that the new method only applies to the current instance of the class:
# cat_2 is not affected by the changes made to cat_1 cat_2 = Cat() In : dir(cat_2) Out: ['legs', 'meow']
But what if you want to add a method dynamically to all instances of the class?
Remember how we said that everything in Python is an object? Take a look at this:
In : type(Cat) Out: <class 'type'>
If you are like me, it'll will take a while to internalize the message being conveyed here, which is that the
Cat class is an instance of the
type metaclass. And as we saw in the last section, anything that is an instance can be manipulated at runtime:
setattr(Cat, 'pounce', pounce)
Checking in on
cat_1, we see an unsurprising result -- we did after all previously bind the
pounce method to it:
In : dir(cat_1) Out: [ 'legs', 'meow', 'pounce']
cat_2 on the other hand never had a
pounce method attached to it, but now shows one inherited from the class definition:
In : dir(cat_2) Out: [ 'legs', 'meow', 'pounce'] In : cat_2.pounce() pounce!
A few warnings for those of you who, like me, are occasionally taken unawares by things that others perhaps wouldn't be surprised by.
Firstly, class implementations of methods can be overridden on a per-instance basis:
def alternate_pounce(*args): print('alternate pounce!') setattr(cat_2, 'pounce', alternate_pounce) In : cat_2.pounce() alternate_pounce! In : cat_1.pounce() pounce!
Secondly, any changes made at the class level do not override local changes at the instance level:
def alternate_pounce_2(*args): print('alternate pounce 2!') setattr(Cat, 'pounce', alternate_pounce_2) # We overrode cat_2's 'pounce' method with alternate_pounce and # changing the class definition of 'pounce' isn't visible from cat_2 In : cat_2.pounce() alternate pounce! # A newly instantiated object shows the new 'pounce' output, however cat_3 = Cat() ln : cat_3.pounce() alternate pounce 2!
Finally, changes to the class last as long as the process is alive. This particular footgun can cause unintuitive behavior when using (and mutating) the class in unit tests that are not run in separate processes. You may think that the class is going to start out fresh, but really the definition of the class at the beginning of a test is what it was at the end of the last test that was run. Spare yourself hours of debugging test order and
unittest internals, and reset the class in your test class'
Why does this exist?
Every Python metaprogramming article seems to include this quote from Tim Peters:
Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why)
Under most circumstances, altering code while it is running increases its complexity and makes reasoning about it more difficult. There are, however, certain circumstances where it is warranted: while developing the database persistence portion of OpenARC, for example, it was necessary to map dynamically changing relationships between database objects. Metaprogramming fit the bill in that very limited use case, and I happily used it.
Knowing your options is good. Don't let black magic scare you.