Override __new__() to replace class method with default arguments

python metaclass
python magic methods
python override __new__
python metaclass inheritance
python __new__ arguments
python super
typeerror: object.__new__() takes exactly one argument (the type to instantiate)
python singleton

Below is a slightly lengthy example for a class ProgressPrinter with two different implementations, depending on whether the file descriptor it is used with is connected to a console or something else. Users are create instances by calling ProgressPrinter.create(), which has a handy default argument:

import sys

class ProgressPrinter:
    def __init__(self, file):
        self._file = file

    def print_progress(self, message):
        raise NotImplementedError

    @classmethod
    def create(cls, file=sys.stderr):
        if file.isatty():
            return TTYProgressPrinter(file)
        else:
            return NoTTYProgressPrinter(file)

class TTYProgressPrinter(ProgressPrinter):
    def print_progress(self, message):
        # Image some fancy stuff with ANSI escape codes.
        pass

class NoTTYProgressPrinter(ProgressPrinter):
    def print_progress(self, message):
        print(message, file=self._file)

What is the best way to rewrite this so that, instead of calling ProgressPrinter.create([file]), users could directly call ProgressPrinter([file]) with same result?

I tried around with overwriting __new__() and even writing a metaclass but none of it worked (Lots of TypeError: object() takes no parameters and TypeError: __init__() missing 1 required positional argument: 'file' and even a RecursionError: maximum recursion depth exceeded while calling a Python object).

Update: Removed redundant code in example. Update: Make implementations actually subclass the base class.

What is the best way to rewrite this so that, instead of calling ProgressPrinter.create([file]), users could directly call ProgressPrinter([file]) with same result?

Just refrain. What you really want is that users feel comfortable with your code. When I see ProgressPrinter.create([file]), I can assume a factory which will create objects related to ProgressPrinter but of possibly different classes. You want them to be subclasses which is fine.

But when I see ProgressPrinter([file]), I expect an object from exactly ProgressPrinter class and not member of a subclass. That means that this code will be harder to read and understand, what you certainly do not want. I know that Python allows it, and I even know how to do it. But it is not natural because programmers are not expected to do it.

The rule is follow well established patterns. You can either use delegation to sub-object (not sub-classes) as suggested by @Barmar, or stick to the factory pattern. But please do not use anti-patterns...


If you really want to do it, the trick is to build and configure an object of the proper subclass in new. But you must use Object.__new__ to build an object of a subclass. If you just use the standard creation, you would recurse into ProgressPrinter.__new__ leading to a stack overflow.

The minimum change would then be to replace your __init__ method with this __new__ one:

def __new__(cls, file):
    if (cls == ProgressPrinter):                   # delegate sub object creation
        return ProgressPrinter.create(file)
    pp = super(ProgressPrinter, cls).__new__(cls)  # trick to avoid recursion
    pp.file = file
    return pp

You could also remove the create method and copy its code directly into __new__

But this is only to show you the possibilities of the Python language and I still strongly urge you not to use that in production code and stick to a nicer factory pattern. Future maintainers would certainly blame you for using this anti-pattern!

Python: __new__ magic method explained, Method __new__ will take class reference as the first argument followed by A class implementation with __new__ method overridden. Recall that you create class instances by calling the class. When the class is a new-style class, the following happens when it is called. First, the class's __new__ method is called, passing the class itself as first argument, followed by any (positional as well as keyword) arguments received by the original call. This returns a new instance.

Here's a working example to get you off the ground.

class IceCream:
    def __new__(ignored_cls, flavor):
        # Here we actually return a new instance,
        # not update one created for us as in __init__.
        # We ignore the class because we're not returning
        # instances of it directly; we could merge this class
        # and IceCreamBase, but then we'd have to override __new__
        # in both Vanilla and Chocolate back to the default impl.
        if flavor == 'vanilla':
            return Vanilla()
        else:
            return Chocolate()

class IceCreamBase:
    def eat(self):
        return "Yum! It was %s" % self.flavor

class Vanilla(IceCreamBase):
    @property
    def flavor(self):
        return "vanilla"

class Chocolate(IceCreamBase):
    @property
    def flavor(self):
        return "chocolate"

Now we can do this:

>>> IceCream('vanilla').eat()
'Yum! It was vanilla'
>>> IceCream('other').eat()
'Yum! It was chocolate'
>>> 

At this point you must have noticed, that IceCream is just a function in disguise, called in a convoluted way.

I would suggest that you used a factory function (not method) in your case, if returning a right value for the arguments (and not some other internal state) is what you need.

3. Data model — Python 2.7.18 documentation, Objects whose value can change are said to be mutable; objects whose value is A tuple containing default argument values for those arguments that have A user-defined method object combines a class, a class instance (or None ) and any The arguments of the call are passed to __new__() and, in the typical case ,� Subclass using Basic version 2 should not define __new__. Instead Basic.__new__ calls two class methods _eval_args and _eval_doit. The _eval_args method is responsible for handling any kwargs and default arguments and for validating the arguments and raising when they do not create a coherent object.

Do it by delegation, not subclassing, choosing the implementation class in the __init__ method.

import sys

class ProgressPrinter:
    def __init__(self, file=sys.stderr):
        self._file = file
        self.printer = ProgressPrinter.create(file)

    def print_progress(self, message):
        self.printer.print_progress(message)

    @classmethod
    def create(cls, file=sys.stderr):
        if file.isatty():
            return TTYProgressPrinter(file)
        else:
            return NoTTYProgressPrinter(file)

class TTYProgressPrinter:
    def __init__(self, file):
        self._file = file

    def print_progress(self, message):
        # Image some fancy stuff with ANSI escape codes.
        pass

class NoTTYProgressPrinter:
    def __init__(self, file):
        self._file = file

    def print_progress(self, message):
        print(message, file=self._file)

__new__() in python, We will see how to override method __new__ in a class. method. So, it is expected that we pass an instance of A as the first argument to met. For a member function of a non-template class, the default arguments are allowed on the out-of-class definition, and are combined with the default arguments provided by the declaration inside the class body. If these out-of-class defaults would turn a member function into a default, copy, or move constructor the program is ill-formed.

It's fine to define subclasses of a parent class (if such a design makes sense), but the parent class doesn't have to be responsible for creating the instances of the children.

import sys

class ProgressPrinter:
    def __init__(self, file):
        self._file = file

    def print_progress(self, message):
        raise NotImplementedError


class TTYProgressPrinter(ProgressPrinter):
    def print_progress(self, message):
        # Image some fancy stuff with ANSI escape codes.
        pass


class NoTTYProgressPrinter(ProgressPrinter):
    def print_progress(self, message):
        print(message, file=self._file)


def create_printer(file=sys.stderr):
    return (TTYProgressPrinter if file.isatty() else NoTTYProgressPrinter)(file)

Metaprogramming - Python 3 Patterns, Recipes and Idioms, The difference is that any change you make to a class affects all the objects of that Normally when we write a class, the default metaclass type is automatically rather than self as the first argument to all methods except __new__() (which Singleton using metaclasses, by overriding the __call__() metamethod, which is � An overriding method in a derived class may in fact want to extend rather than simply replace the base class method of the same name. There is a simple way to call the base class method directly: just call BaseClassName.methodname(self, arguments). This is occasionally useful to clients as well.

This is an excellent[0] exercise to demonstrate what metaclasses can do:

import sys

class ProgressPrinterMeta(type):
    def __call__(cls, file=sys.stderr):
        if cls is ProgressPrinter:
            if file.isatty():
                return TTYProgressPrinter(file)
            else:
                return NoTTYProgressPrinter(file)
        else:
            return super().__call__(file)

class ProgressPrinter(metaclass=ProgressPrinterMeta):
    def __init__(self, file):
        self._file = file

    def print_progress(self, message):
        raise NotImplementedError

class TTYProgressPrinter(ProgressPrinter):
    def print_progress(self, message):
        # Image some fancy stuff with ANSI escape codes.
        pass

class NoTTYProgressPrinter(ProgressPrinter):
    def print_progress(self, message):
        print(message, file=self._file)

[0]: Excellent if you're not going to use this in a project others are working on too.

Python: __init__() is not the only constructor - DEV, __new__() method is called before __init__() acts. It is because First argument is the class itself which is passed implicitly. Always return a� Method overriding in action¶ In Python method overriding occurs simply defining in the child class a method with the same name of a method in the parent class. When you define a method in the object you make the latter able to satisfy that method call, so the implementations of its ancestors do not come in play.

Overriding Default Arguments in Python, How to override the default arguments of a function. Sometimes you want to change the behavior of a function call in a Python test. Let's assume Every Python function with default arguments has a __defaults__ attribute. Python is Object oriented language, every thing is an object in python. Python is having special type of methods called magic methods named with preceded and trailing double underscores. When we talk about magic method __new__ we also need to talk about __init__ These methods will be called when you instantiate(The process of creating instance … Continue reading Python: __new__ magic method

Python Metaclasses – Real Python, __class__ designates the class, but type(obj) is always instance . In this first example, the <bases> and <dct> arguments passed to type() are If Foo does not define __new__() and __init__() , default methods are inherited from Foo 's ancestry. But if Foo does define these methods, they override those from the ancestry,� Notice: Undefined index: HTTP_REFERER in /home/u294630715/domains/poineerpolymers.co.uk/public_html/veddr/ojneyhzxlytbgvz.php on line 76 Notice: Undefined index: HTTP

self in Python, Demystified, In this case all the methods, including __init__ , have the first parameter as self . We know When __new__() is called, the class itself is passed as the first argument This is because most of the time you don't need to override it. Click on the different category headings to find out more and change our default settings. An overriding method in a derived class may in fact want to extend rather than simply replace the base class method of the same name. There is a simple way to call the base class method directly: just call BaseClassName.methodname(self, arguments). This is occasionally useful to clients as well.

Comments
  • Have you considered making ProgressPrinter a function instead of a class?
  • Search SO for [python] factory pattern you'll find lots of examples.
  • @GreenCloakGuy Actually, yes. But I would like to use ProgressPrinter as a type annotation (it's passed around a lot). Actually, I haven't tried, but I think that would't work if it were a function. But it would definitely be a simple and effective solution to my problem!
  • @Barmar Uuuhh, maybe link an answer you think particularly fits my question? I have no problem closing this question with already answered.
  • ProgressPrinter doesn't need to be a function, but ProgressPrinter.create certainly can be. Note that you aren't using cls in the definition, and it's slightly odd to make any part of a class require knowledge of its own children.
  • Thanks for the question. My question arose from the urge to understand how I could do this, even if it was agains Python conventions. I don't think I agree that returning a subclass is confusing. It's an implementation detail and it doesn't violate any rules of OO programming.
  • @Feuermurmel: it is indeed an implementation detail when you use the factory pattern. But in any OO language, new Clazz(..) or Clazz(...) is expected to build a Clazz object. It is not really confusing for the language itself (Python does allow it) but for future readers.
  • Oh, no thank you for belittling me. I'm trying to understand what the interactions between the metaclasses __call__() and the classes __new__() and __init__() are and thought this was a good exercise to get a better understanding. In my project, I'm currently using the variant I posted in my question I'm only going to change that if I see good reasons to do so.
  • @Feuermurmel: that is the reason why I posted that code using __new__ :-). But I still wanted to warn future readers that is was an anti-pattern. BTW metaclasses is an even more advanced tool which is not required here. They are intended to provide a consistent but not standard behaviour among a collection of classes that are not a direct hierarchy.
  • Thanks for the answer! What speaks agains a factory method? I would definitely prefer a plain function over this. :) Partly because it breaks the assumption that IceCream can be used as a type annotation.
  • Thanks for the answer. See my updated code example in the question. I forgot yo make TTYProgressPrinter and NoTTYProgressPrinter subclasses of ProgressPrinter.
  • I'm suggesting that you NOT make them subclasses.
  • In my real application, I have a lot of methods common to both implementation, so I'd have to add another base class.