Python: when to use __new__, when to use __init__

Question

I've dome some reading on the difference between __new__ and __init__, and never really groked it. I just followed the advice that you should almost always use __init__.

I recently came across a task that required using __new__ and not __init__. I was a bit intimidated at first, but it was quick and easy. This simple programming exercise really cleared a lot of things up for me.

Not to be immodest, but I think something like this ought to be the canonical example for explaining when/how to override __new__.

The task? I want to make a class that behaves exactly like a tuple, except changing the constructor argument signature and adding some extra methods. An example should clarify what I needed.

> x = ParetoTuple(1, 2, 0)
> x[1]
>> 2
> len(x)
>> 3
> 2 in x
>> True
> -1 in x
>> False
> x.dominates(ParetoTuple(1, 3, 0))
>> True
> x.equivalent(ParetoTuple(1, 2 + 1e-5, 0))
>> True

etc.

Since I want the constructor to take an (almost) arbitrary number of arguments, each of which will be elements of the resulting ParetoTuple, I need to override __new__. I don't need to overwrite __init__, because the tuple.__new__ will populate it's data when the arguments are properly formatted.

Also, since the world of Pareto comparisons makes sense only with 2 or more goals, I want my specialized constructor to take at least 2 arguments in a natural way.

Here is the code

class ParetoTuple(tuple) :
 def __new__ (cls, obj1, obj2, *rest):
 return super(ParetoTuple, cls).__new__(cls, (obj1, obj2) + rest)
 # nothing special about the dominates, equivalents methods...
 # no __init__ needed

I understand some people argue in favor of using a factory pattern for this sort of situation, but I disagree. I think the cognitive overhead of factories requires a more complicated task than re-signaturing the constructor method.

Answer 1

Object creation in Python is a two step process:

• create the object (aka __new__, and make sure you return the new object! ;)
• configure the object (aka __init__)

If the object is immutable, everything has to be done in __new__.

If the object is mutable, then you should split your code along the creation/configuration guidelines of __new__ and __init__, even though you could do it all in __new__. Why? To make subclassing easier.

As an example, consider the new Enum[1] data type: my personal preference is to not specify the numbers, and to have docstrings on the Enum members. In order to achieve this I have to override __new__ as that is when the class structures are created, but I set the docstring in __init__:

class AutoEnum(Enum):
 "
 Automatically numbers enum members starting from 1.
 Includes support for a custom docstring per member.
 "
 __last_number__ = 0
 def __new__(cls, *args):
 "Ignores arguments (will be handled in __init__."
 value = cls.__last_number__ + 1
 cls.__last_number__ = value
 obj = object.__new__(cls)
 obj._value_ = value
 return obj
 def __init__(self, *args):
 "Can handle 0 or 1 argument; more requires a custom __init__.
 0 = auto-number w/o docstring
 1 = auto-number w/ docstring
 2+ = needs custom __init__ (don't call this __init__)
 "
 if len(args) == 1 and isinstance(args[0], (str, unicode)):
 self.__doc__ = args[0]
 elif args:
 raise TypeError('%s not dealt with -- need custom __init__' % (args,))

Now, if I need some other Enum class with auto-numbering, but different arguments I can easily subclass AutoEnum:

class Rounds(AutoEnum):
 def __init__(self, x_length, y_length):
 self.x_length = x_length
 self.y_length = y_length
 SMALL_CICRLE = 100, 100
 LARGE_ELLIPSE = 5000, 3000

[1] enum34 is available on PyPI if you aren't able to move to Python3.4.