There is a small mistake in the post: decorators cannot be arbitrary expressions. Something like @Foo(spam).bar fails with an unhelpful SyntaxError, something that everyone who designs complex APIs will probably encounter at some point. This is a restriction which is in place because Guido didn't like arbitrary expressions as decorators.
What I really want to know is how a framework such as Flask uses a decorator for the route. How is the correct function picked for a particular route that is defined against the decorator? (Maybe I'm completely misunderstanding this...)
Some might say it's a dangerous abuse of decorators.
Since decorators (generally) run at module load time, any stateful decorator (usually) implies the use of global mutable state, which is (considered by many to be) the root cause of much bad design, convoluted flow, limited reusability and untestability. This is perhaps why most decorators in the standard library are pure (off the top of my head).
This is well beyond the scope of the post but an important and often overlooked point in my opinion.
While you're right it is an easy to make mistake with decorators, I don't think the issue is global mutable state as much as uncontrolled mutable state, especially if used against singleton.
Typically, doing this:
@register
def foo():
….
is bad, but this is much better:
@registry.register
def foo():
…
if registry is a global object that is not a singleton. In that case, you can easily use distinct registries (for testing, etc…) and this is not much of an issue in practice. Another way of doing this kind of things is to post_pone the registration, with the decorator just labelling things:
and then build an explicit list of modules being imported, and look for every instance of WrappedFunc in that module locals(). I use this in my own packaging project where people can define multiple python scripts that hook into different stages of the packaging.
In flask, the app.route decorator mutates the app object. There are no globals necessary. Use of globals to maintain state is an orthogonal issue to use of decorators to update registries.
Well either the app object is global, in which case you've got global mutable state, or you're defining your handler functions later than module load time, which is pretty uncommon practice.
You can use the return value of some function as a decorator, as a way to avoid global state and tie the decorator to a given instance of your routing object. (I don't know flask, but this isn't a limitation of python)
When you call `route` with a URL pattern, it returns an inner function which is used as the decorator. That decorator just records your route and function in the Flask URL map, and returns your function unchanged.
So, Flask is arguably perpetuating a slight abuse of decorators, since it doesn't decorate or wrap your function at all, but merely saves a reference to it somewhere. But it's a fairly clean way to make up for the lack of code blocks or multi-statement anonymous functions in Python.
For some reason, decorators didn't click for me until now. I've been parroting the standard decorator logic (e.g. returning a function) without knowing why.
To authors: I would avoid try except in this code snippet, a simple if else is more explicit. I would also avoid a = b = c statement. One line per statement is better most of the time.
The Python community generally advocates an "it's easier to ask for forgiveness than permission" coding style. When faced with a condition of the form "if condition a holds, do b, else c", it's very often a better idea to do "let's try b, and do c in case b fails because condition a didn't hold".
In this case it's better because you can avoid computing an extra hash of the object in cases where it's already a key of the dictionary. This may seem like a silly optimization, but it can very easily add up if you're accessing existing elements most of the time--I once had a bit of code that went 60x faster when I replaced an if-else with a try-except.
In other cases it can be even more beneficial. Say you're opening a file. One approach to avoid errors would be to check if a file exists first. This is error-prone because the file might cease to exist in between the 'if' and the 'open' statements, and now you have no code written to handle the error. Using try-except will ensure that you actually handle the error intelligently.
This isn't to say there's never a good reason to use an 'if' to check things, just that if you can do it in one step instead of two, one is usually better.
Agree with that "try except" is better than "if else" for file handling ("with" is even better).
I am not sure if "try except" is really faster than "if else" in some edge cases in a memoization context, as you claim.
What I am sure is that in a didactic context, where you want people to understand code, something like:
if args not in stored_results:
stored_results[args] = fn(*args)
return stored_results[args]
is much better than that (from the OP):
try:
# try to get the cached result
return stored_results[args]
except KeyError:
# nothing was cached for those args. let's fix that.
result = stored_results[args] = fn(*args)
return result
For a memoize function, try/except is perfect. You gain a slight speed benefit from function calls that are memoized as compared to the "if x not in y", and since memoize is all about speeding up repetitive function calls, it's the most optimal choice. Using the "if x not in y" you are giving up speed of the "x is actually in the y" in exchange for a cheaper than "except", however if you are memoizing a function, chances are that the function is going to eat up processing power making your optimization of the "exceptional case" at the expense of the "normal case" not worth it.
As far as people understanding code - this is a common pattern in python code, try / except should be easy for anyone to understand.
Out of curiosity, in a situation where you are doing negligible condition testing, is try–except still considered "more performant" than if–else flow control? For some reason (and I'm going to do so reading to get clarity on this point), I have had the silly notion that throwing and handling exceptions can be costly.
I don't think the try-except is 'more performant'. At least from the below benchmark test it doesn't seem to be so.
>>> from timeit import timeit
>>> timeit(setup='x=dict([(i,i*2) for i in range(10)])',stmt=
"""
if 20 in x:
pass""")
0.07420943164572691
>>> timeit(setup='x=dict([(i,i*2) for i in range(10)])',
stmt="""
try:
x[20]
except KeyError:
pass""")
1.1514457843105674
I am on my phone and cannot test this, but the try:except: construct is optimised for the non-exceptional path. The latter is probably faster for x[0] than x[20].
I agree. The author uses exception for something that is not at all exceptional. In fact, it will occur at least once. This is misuse of an exception. No one expects Spanish Exception!
Earlier, he uses the map function for no apparent reason. I mean - list comprehension would perfectly fit here.
I still think I have something to learn from the article, though.
The try/except is the preferred python way of doing things. In Python, try/except is cheap, and in the case where the try actually succeeds, you can gain a slight performance benefit. In this case, since performance is the goal, it is perfect for memoize.
I think subtle optimization should be left aside in a tutorial. Try except is a good way to handle many cases that are not exceptions, like when reaching the end of an iteration, but I never heard that it was "the preferred way" for checking the presence of a key in a dict. I believe the "if in" construct is cleaner, clearer and "pythoniker", if you don't mind the Housism.
"if in" is not the preferred way to check for the presence of a key in a dict. You should use setdefault. The thing is, you aren't searching for a key in a dict only. You're memoizing. And try/except is the most efficient way to do that. This article isn't teaching you how to program, it's teaching decorators.
Nice description. I'd suggest explaining how decorators that accept arguments (i.e. @memcached('some-arg')) work lest it befuddle some beginner. It is not straightforward (the first argument of a decorator is the function being decorated).
Nice description of method decorators. Didn't touch on class decorators, or decorators that can decorate both classes and methods (arguably very ugly wrapper functions that return decorators that decorate).
Great primer, reading it gives a good sense for why the decorator syntax was made and the potential uses. Is this part of a wider series you are running?
You could say decorators are for the code that you want to put inside a function but that don't really belong to its logic. The memoization example shows that: if you have memoization logic inside the function it works but you should feel that there are two disjoint logics at works.
