Python: wrap all functions in a library

python wrapper
python wrap function call
python wrap print function
python function wrapper with arguments
what is wrapping in python
python function wrapper in class
python iswrapper
method-wrapper python

We use a library provided by another internal team. (Shaky analogy starts now)

from externalTeam import dataCreator
datacreator.createPizza()
datacreator.createBurger()
datacreator.createHotDog()

Recently we found a single method of theirs was taking over a minute to execute in certain situations. To debug this, I had to go into our code and add timeouts around every call of this method.

import time
from externalTeam import dataCreator
start = time.clock()
datacreator.createPizza()
stop = time.clock()
print "It took %s seconds to perform createPizza" % (str(stop-start))

In hindsight, that's because we're calling createPizza all over the place, and we don't control createPizza itself (the analogy is starting to break down a little here). I'd rather just call createPizza in one place, and be able to add a timer around that. My first thought to accomplish this would be to create a wrap all their methods in my own wrapper class. That's the opposite of DRY though, and anytime they add another method I'd have to update our library to wrap that as well:

import time
from externalTeam import dataCreator
def createPizza(self):
    start = time.clock()
    datacreator.createPizza()
    stop = time.clock()
    print "It took %s seconds to perform createPizza" % (str(stop-start))

def createBurger(self):
    start = time.clock()
    datacreator.createPizza()
    stop = time.clock()
    print "It took %s seconds to perform createBurger" % (str(stop-start))

def createHotDog(self):
    start = time.clock()
    datacreator.createPizza()
    stop = time.clock()
    print "It took %s seconds to perform createHotDog" % (str(stop-start))    

What I want is a way to always execute a few lines of code around every function that's being called from dataCreator. There must be some way to do that through an intermediate class whose methods can be dynamically defined - or rather left undefined, right?

I would create a dataCreator adapter class that would work like this:

  1. Have a methods2wrap list of the methods from dataCreator that needs to be wrapped into the debugging/timing functionality.
  2. Have an overridden __getattribute__() that would map 1:1 onto the dataCreator methods, wrapping the methods in methods2wrap into a timing debug message.

Proof-of-concept code (the example wrap the class list and insert a debugging timestamp around its method append).

import time

class wrapper(list):

    def __getattribute__(self, name):
        TO_OVERRIDE = ['append']
        if name in TO_OVERRIDE:
            start = time.clock()
        ret = super(list, self).__getattribute__(name)
        if name in TO_OVERRIDE:
            stop = time.clock()
            print "It took %s seconds to perform %s" % (str(stop-start), name)
        return ret

profiled_list = wrapper('abc')
print profiled_list
profiled_list.append('d')
print profiled_list
profiled_list.pop()
print profiled_list

Of course you could build on this example and make it parametric, so that at initialisation time you can set what class to wrap and what methods should be timed...

EDIT: Note that TO_OVERRIDE is reassigned at each __getattribute__ call. This is by design. If you you would make it as a class attribute, __getattribute__ would recursively loop (you should use an explicit call to the parent __getattribute__ method to retrieve it, but this would probably be slower than simply rebuild the list from scratch.

HTH

9.8. functools, The functools module is for higher-order functions: functions that act on or return being converted to Python 3 where comparison functions are no longer supported. the effort involved in specifying all of the possible rich comparison operations: The main intended use for this function is in decorator functions which wrap  TextWrapper also provides some public methods, analogous to the module-level convenience functions: wrap (text) ¶ Wraps the single paragraph in text (a string) so every line is at most width characters long. All wrapping options are taken from instance attributes of the TextWrapper instance. Returns a list of output lines, without final newlines.

If you're trying to profile Python code, you should use Python's built-in profiling libraries instead of trying to do it manually.

functools, The functools module is for higher-order functions: functions that act on or return Transform a method of a class into a property whose value is computed once and converted from Python 2 which supported the use of comparison functions. Decorator to wrap a function with a memoizing callable that saves up to the  The textwrap module provides two convenience functions, wrap() and fill(), as well as TextWrapper, the class that does all the work, and a utility function dedent().If you’re just wrapping or filling one or two text strings, the convenience functions should be good enough; otherwise, you should use an instance of TextWrapper for efficiency.

Why not a single wrapper function which just calls its argument?

def wrapper(func, *args, **kwargs):
    ... timing logic ...
    response = func(*args, **kwargs)
    ... more timing logic
    return response

and call it:

wrapper(datacreator.createPizza, arg1, arg2, kwarg1=kwarg)

note you pass the function itself, but without calling it.

Is there a neat way to apply a decorator to all functions in a class or , We can use the built-in [code python]dir[/code] to get all the attributes of a Is there a neat way to apply a decorator to all functions in a class or in a module? textwrap.wrap(text, width=70, **kwargs) : This function wraps the input paragraph such that each line in the paragraph is at most width characters long. The wrap method returns a list of output lines. The returned list is empty if the wrapped output has no content. Default width is taken as 70.

The following template could help:

class MeteredClient(Client):
  def __init__(self, *args, **kwargs):
    super(MeteredClient, self).__init__(*args, **kwargs)

  def __getattribute__(self, method_name):
    attribute = super(Client, self).__getattribute__(method_name)

    if not inspect.ismethod(attribute):
      return attribute

    metric = TIMINGS.labels(method_name)

    def decorator(*args, **kw):
      start_time = get_time()
      rv = attribute(*args, **kw)
      metric.observe(get_time() - start_time)
      return rv

    return decorator

Decorators and Wrappers in Python - Hangar, A lot of libraries and frameworks offer useful decorators - one of my favorites is the What Python does when it sees @decorator is to call the decorator function and pass on all arguments - use (*args, **kwargs) when defining the wrapper  So in Python, there is a dir() method which can list all functions and attributes of a module. Inside of this dir() function, we specify the module that we would like to see all functions and attributes of. For example, in the following code below, we show all of the functions and attributes of the os module.

Apply decorators to all functions in a module « Python recipes , Apply decorators to all functions in a module (Python recipe) by Eric Snow. ActiveState Code (http://code.activestate.com/recipes/577742/). Use  The all () method returns True when all elements in the given iterable are true. If not, it returns False. The syntax of all () method is: The all () method takes a single parameter: The all () method returns: When you run the program, the output will be: The any () method works in similar way for tuples and sets like lists.

Primer on Python Decorators – Real Python, Recall that a decorator is just a regular Python function. All the usual tools for easy reusability are available. Let's move the decorator to its own module that can  See itertools.filterfalse() for the complementary function that returns elements of iterable for which function returns false. class float ( [ x ] ) ¶ Return a floating point number constructed from a number or string x .

[Python] Wrapping function calls? - Coding, I wanna pass the function call to an object that will handle some pre- and post- operations (logging mostly). class and thus use the call method instead of returning a decorated function. __module__ def __call__(self, *args, **kwargs): # . The function argument must be a callable object (a user-defined or built-in function or method, or a class object) and the args argument must be a sequence. The function is called with args as the argument list; the number of arguments is the length of the tuple.

Comments
  • are we talking about functions or classes with methods?
  • I know about decorators. What do I decorate though? createPizza and createBurger are functions on their side.
  • @jollybox.de - we're talking about classes with methods.
  • "That's the opposite of DRY though". Not really. You're wrapping, not repeating. And yes, your Facade will grow as the underlying library goes. That's okay.
  • This doesn't seem to handle the execution time inside the method that is returned. In this solution, profiled_list.append('d') will only include the time it took to retrieve the append method, not the time to actually append. I think it might be necessary to return a wrapped version of the method as well.
  • I mostly want to profile their one library's calls, and output to our log on the debug level. Seems like a profiling library might be able to do that, so I'll look into that link, thanks!
  • Like a decorator, right? This would work, but I'd have to write one line of code like wrapper(datacreator.createPizza, arg1, arg2, kwarg1=kwarg) for every function from their library that I call. Whenever they introduce new functionality I have to add another line... I'm wondering if there's a way to avoid that.
  • @caribou - not quite - a Python decorator returns a function object. But it's trivial to turn this into a decorator.
  • @caribou - You did not comment on my answer, but that's the problem my solution tries to go around. Of course you still have to specify which methods you want to time, but their name will be all you have to insert.... or did I miss anything?
  • How could it help? Can you give some extra explanation?