outside of __init__, I thought it wasn't the pythonic way to directly invoke double underscore functions like closure? I recall reading that you should implement a class that overrides/implements such functions instead? I get why the author might be doing it this way to describe the topic of the post, but would calling __closure__() be acceptable in production code?
It's absolutely baffling that an article called demystifying decorators does not actually describe decorators at all, preferring to keep them a mystery. Decorators are nothing more than a special syntax for calling a function, the meat of which can easily be explained in a single paragraph including a supporting example.
I'm not sure who this article is for, but I think it doesn't strike at the heart of the topic.
Half of the article is devoted to closures, but closures aren't essential for decorators. And the __closure__ attribute is an implementation detail that is really irrelevant. (For comparison, JavaScript has closures just like Python, but it doesn't expose the closed-over variables explicitly the way Python does.)
Decorators are simply higher order functions that are used to wrap functions. The syntax is a little funky, but all you need to know is that code like:
@foo
@bar
def baz(args):
return quux(args)
Is essentially equivalent to:
baz = foo(bar(lambda args: quux(args))
...i.e. decorators are functions that take a callable argument and return a new callable (which typically does something and then calls the argument function--or not, as the case may be).
Then there are seemingly more complex expressions like:
@foo(42, 'blub')
def bar(..): ...
This looks like a special kind of decorator that takes arguments, but it looks more complex than it really is. Just like `foo` was an expression referencing a function `foo(42, 'blub')` is just a regular Python function call expression. That function call should then itself return a function, which takes a function argument to wrap the function being decorated. Okay, I admit that sounds pretty complex when I write it out like that, but if you implement it, it's again pretty simple:
This is an extra level of indirection but fundamentally still the same principle as without any arguments.
And yes, these examples use closures, which are very convenient when implementing decorators. But they aren't essential. It's perfectly possible to declare a decorator this way:
class Invoker:
def __init__(self, f):
self.f = f
def __call__(self):
print('before call')
self.f()
print('after call')
def decorator(f):
return Invoker(f)
@decorator
def hello():
print('Hello, world!')
hello()
# prints:
# before call
# Hello, world!
# after call
It's the same thing but now there are no closures whatsoever involved.
The key point in all these examples is that functions in Python are first-class objects that can be referenced by value, invoked dynamically, passed as arguments to functions, and returned from functions. Once you understand that, it's pretty clear that a decorator is simply a wrapper that takes a function argument and returns a new function to replace it, usually adding some behavior around the original function.
> decorators are functions that take a callable argument and return a new callable
there's nothing forcing a decorator to return a callable. A decorator _could_ return anything it wants. I don't know why you would want that, but Python won't stop you.
the thing that bothered me most reading through it was using decorators to mutate some global state with the `data` list variable.
like… it… just… it felt wrong reading that in the examples. felt very `def func(kw=[])` adjacent. i can see some rare uses for it, but eh. i dunno.
(also didn’t find the closure stuff that insightful, ended up skipping past that, but then i know decorators, so… maybe useful for someone else. i dunno.).
My "proudest" use of decorators has been in adding metrics gathering for functions in python, so you'd get automated statsd (and later prometheus) metrics just by having @tracked (or whatever I had its name be - it's been like 7 years) on the function header.
In a sense, that was mutating a global variable by including and tracking the metrics gathering. I imagine this person's early professional exposures to it and need to create their own also came from a similar situation, so "mutating global state" and closures sorta clicked for them.
People learn things by coming to those things from many different entry points and for many different reasons. This is another one of those instances :)
I'm getting old. Something so obvious that I thought everybody knew is getting reiterated in a blogpost by a younger generation encountering it for the first time.
You're certainly getting older :). You're assuming that since the author writes about something obvious, they must belong to the "younger generation" and encountered it for the first time.
Meanwhile, the author finished their PhD in 2004, and wrote 3 books about Python.
Which 3 books are you talking about? I can only find 1 on Amazon (and it has a grand total of 12 reviews, so... you know, not exactly a best seller).
Frankly it's a bit suspicious how defensive you are of this author, and combined with the blatant downvoting of my toplevel comment, makes me think there is some astro-turfing going on in this thread.
outside of __init__, I thought it wasn't the pythonic way to directly invoke double underscore functions like closure? I recall reading that you should implement a class that overrides/implements such functions instead? I get why the author might be doing it this way to describe the topic of the post, but would calling __closure__() be acceptable in production code?
It's absolutely baffling that an article called demystifying decorators does not actually describe decorators at all, preferring to keep them a mystery. Decorators are nothing more than a special syntax for calling a function, the meat of which can easily be explained in a single paragraph including a supporting example.
I'm not sure who this article is for, but I think it doesn't strike at the heart of the topic.
Half of the article is devoted to closures, but closures aren't essential for decorators. And the __closure__ attribute is an implementation detail that is really irrelevant. (For comparison, JavaScript has closures just like Python, but it doesn't expose the closed-over variables explicitly the way Python does.)
Decorators are simply higher order functions that are used to wrap functions. The syntax is a little funky, but all you need to know is that code like:
Is essentially equivalent to: ...i.e. decorators are functions that take a callable argument and return a new callable (which typically does something and then calls the argument function--or not, as the case may be).Then there are seemingly more complex expressions like:
This looks like a special kind of decorator that takes arguments, but it looks more complex than it really is. Just like `foo` was an expression referencing a function `foo(42, 'blub')` is just a regular Python function call expression. That function call should then itself return a function, which takes a function argument to wrap the function being decorated. Okay, I admit that sounds pretty complex when I write it out like that, but if you implement it, it's again pretty simple: This is an extra level of indirection but fundamentally still the same principle as without any arguments.And yes, these examples use closures, which are very convenient when implementing decorators. But they aren't essential. It's perfectly possible to declare a decorator this way:
It's the same thing but now there are no closures whatsoever involved.The key point in all these examples is that functions in Python are first-class objects that can be referenced by value, invoked dynamically, passed as arguments to functions, and returned from functions. Once you understand that, it's pretty clear that a decorator is simply a wrapper that takes a function argument and returns a new function to replace it, usually adding some behavior around the original function.
One tiny correction:
> decorators are functions that take a callable argument and return a new callable
there's nothing forcing a decorator to return a callable. A decorator _could_ return anything it wants. I don't know why you would want that, but Python won't stop you.
They also don’t have to act on callables, see @dataclass for instance.
Classes are callable.
the thing that bothered me most reading through it was using decorators to mutate some global state with the `data` list variable.
like… it… just… it felt wrong reading that in the examples. felt very `def func(kw=[])` adjacent. i can see some rare uses for it, but eh. i dunno.
(also didn’t find the closure stuff that insightful, ended up skipping past that, but then i know decorators, so… maybe useful for someone else. i dunno.).
My "proudest" use of decorators has been in adding metrics gathering for functions in python, so you'd get automated statsd (and later prometheus) metrics just by having @tracked (or whatever I had its name be - it's been like 7 years) on the function header.
In a sense, that was mutating a global variable by including and tracking the metrics gathering. I imagine this person's early professional exposures to it and need to create their own also came from a similar situation, so "mutating global state" and closures sorta clicked for them.
People learn things by coming to those things from many different entry points and for many different reasons. This is another one of those instances :)
I'm getting old. Something so obvious that I thought everybody knew is getting reiterated in a blogpost by a younger generation encountering it for the first time.
You're certainly getting older :). You're assuming that since the author writes about something obvious, they must belong to the "younger generation" and encountered it for the first time.
Meanwhile, the author finished their PhD in 2004, and wrote 3 books about Python.
Maybe not "by", but "for" a younger generation.
Which 3 books are you talking about? I can only find 1 on Amazon (and it has a grand total of 12 reviews, so... you know, not exactly a best seller).
Frankly it's a bit suspicious how defensive you are of this author, and combined with the blatant downvoting of my toplevel comment, makes me think there is some astro-turfing going on in this thread.