[Translation] Implementing Function Overloading in Python

Translated from: https://arpitbhayani.me/blogs/function-overloading
Author: @arpit_bhayani
Translation authorized by the original author

Function overloading refers to the ability to create multiple functions with the same name, but each function can have different parameters and implementations. When an overloaded function fn is called, the runtime determines and calls the corresponding implementation function based on the passed parameters/objects.

int area(int length, int breadth) {
  return length * breadth;
}

float area(int radius) {
  return 3.14 * radius * radius;
}

In the above example (C++ implementation), the function area is overloaded with two implementations. The first function allows two parameters (both of integer type) to be passed, returning the area of a rectangle based on the given length and breadth; the second function requires one integer parameter: the radius of a circle. When we call the function area, for example, calling area(7) will use the second function, while calling area(3, 4) will use the first.

Why is there no function overloading in Python?#

Python does not support function overloading by default. When we define multiple functions with the same name, the latter always overrides the former, so within the same namespace, each function name always has only one entry point. We can use the functions locals() and globals() to see what is in the namespace; these two functions return the local and global namespaces, respectively.

def area(radius):
    return 3.14 * radius ** 2

>>> locals()
{
  ...
  'area': <function __main__.area(radius)>,
  ...
}

After defining the function, executing the locals() function will return a dictionary containing all the variables defined in the current namespace. The keys of this dictionary are the variable names, and the values are references or values of the variables. When the runtime encounters another function with the same name, it updates the function entry in the local namespace, thus eliminating the possibility of two functions coexisting. Therefore, Python does not support function overloading. This was a design decision made when creating the language, but it does not prevent us from implementing it. Let's overload some functions.

Implementing function overloading in Python#

We already know how Python manages namespaces. If we want to implement function overloading, we need to:

Maintain a virtual namespace to manage function definitions
Find the method to call the corresponding function based on the passed parameters

For simplicity, the function overloading we implement will be distinguished by the number of parameters accepted by the function.

Wrapper function#

We create a class named Function, which can take any function as input. By overriding the class's __call__ method, we make this class callable. Finally, we implement a key method that returns a tuple that uniquely identifies the passed function globally.

from inspect import getfullargspec


class Function(object):
    """Wrap standard Python functions.
    """

    def __init__(self, fn):
        self.fn = fn

    def __call__(self, *args, **kwargs):
        """This method allows the entire class to be called like a function,
        then returns the result of the passed function call.
        """
        return self.fn(*args, **kwargs)

    def key(self, args=None):
        """Returns a unique key identifying the function.
        """
        # If no parameters are manually passed, get parameters from function definition
        if args is None:
            args = getfullargspec(self.fn).args

        return tuple(
            [self.fn.__module__, self.fn.__class__, self.fn.__name__, len(args or [])]
        )

In the above code snippet, the key method of the Function class returns a tuple that can uniquely identify the wrapped function throughout the codebase. The tuple also contains the following information:

The module to which the function belongs
The class to which the function belongs
The function name
The number of parameters the function accepts

The overridden __call__ method calls the wrapped function and returns the computed value (currently just a regular operation, returning the value as is). This allows instances of this class to be called like functions, and their behavior is identical to that of the wrapped function.

def area(l, b):
  return l * b

>>> func = Function(area)
>>> func.key()
('__main__', <class 'function'>, 'area', 2)
>>> func(3, 4)
12

In the above example, we passed the area function to the Function class and assigned the instance to func. Calling func.key() returned a tuple, where the first element is the current module name __main__, the second element is the type <class 'function'>, the third is the function name area, and the fourth is the number of parameters the area function can accept: 2.

This example also illustrates how to call the func instance just like we normally call the area function, passing in two parameters: 3 and 4, and then obtaining the result 12. This behaves exactly the same as calling area(3, 4).

This behavior will be very useful when we use decorators in later stages.

Creating a virtual namespace#

At this step, we will build a virtual namespace (function registry) that will store all the functions to be overloaded during the function definition phase.

Since only one namespace is needed globally (to keep all the functions to be overloaded together), we create a singleton class that internally uses a dictionary to store different functions. The keys of the dictionary are the tuples we obtain from the key method, which can uniquely identify functions throughout the code. This way, even if the function names are the same (but the parameters differ), we can store them in the dictionary, thus achieving function overloading.

class Namespace(object):
  """The Namespace singleton class stores all functions to be overloaded.
  """
  __instance = None

  def __init__(self):
    if self.__instance is None:
      self.function_map = dict()
      Namespace.__instance = self
    else:
      raise Exception("An instance already exists; cannot instantiate again.")

  @staticmethod
  def get_instance():
    if Namespace.__instance is None:
      Namespace()
    return Namespace.__instance

  def register(self, fn):
    """Register a function in the virtual namespace and return a wrapped
    `Function` instance.
    """
    func = Function(fn)
    self.function_map[func.key()] = fn
    return func

Namespace has a register method that takes a fn parameter, creates a unique key based on fn, and saves it in the dictionary, finally returning a wrapped Function instance.

This means that the return value of the register method is also callable and (so far) behaves exactly like the function fn to be wrapped.

def area(l, b):
  return l * b

>>> namespace = Namespace.get_instance()
>>> func = namespace.register(area)
>>> func(3, 4)
12

Using decorators#

Now that we have defined a virtual namespace that can register functions, we need a hook that will be called during function definition (note: the original text uses "hook"). Here, we use decorators to achieve this.

In Python, we can wrap functions with decorators, allowing us to add new functionality to functions without modifying their original structure. The decorator takes the function fn to be wrapped as a parameter and returns a new callable function. The new function can also accept the args and kwargs you pass during the function call and return a value.

Below is an example of a decorator that can measure the execution time of a function.

import time


def my_decorator(fn):
  """my_decorator is a custom decorator
  that wraps the passed function and outputs its execution time.
  """
  def wrapper_function(*args, **kwargs):
    start_time = time.time()

    # Call the original function to get its return value
    value = fn(*args, **kwargs)
    print("Function execution took:", time.time() - start_time, "seconds")

    # Return the original function's return value
    return value

  return wrapper_function


@my_decorator
def area(l, b):
  return l * b


>>> area(3, 4)
Function execution took: 9.5367431640625e-07 seconds
12

In the above example, we defined a decorator named my_decorator, which wraps the defined area function. The wrapped function can print the time taken for execution after it runs.

When the Python interpreter encounters a decorated function definition, it executes the decorator function my_decorator (thus completing the wrapping of the decorated function and storing the function returned by the decorator in Python's local or global namespace). For us, this is an ideal hook to register functions in the virtual namespace. Therefore, we create a decorator named overload, which will register the decorated function in the virtual namespace and return a callable Function instance.

def overload(fn):
  """overload is our decorator for wrapping functions, which returns a
  callable `Function` instance.
  """
  return Namespace.get_instance().register(fn)

The overload decorator returns an instance of Function, which is obtained by calling the .register() method in the Namespace singleton class. Now, whenever a function (decorated with overload) is called, it is actually the __call__ method of this Function instance that is invoked, and this method will also receive the corresponding parameters passed to the function.

Now, all we need to do is improve the __call__ method of the Function class so that when the __call__ method is invoked, it finds the correct function definition based on the passed parameters.

Finding the correct function from the namespace#

The conditions for function disambiguation, in addition to judging the module's class and name, also need to consider the number of parameters the function accepts. Therefore, we define a method get in the virtual namespace that receives a function from the Python namespace (unlike the virtual namespace we created, we have not changed the default behavior of the Python namespace). Based on the number of parameters of the function (the disambiguation factor we set), it returns a callable disambiguated function.

The purpose of this get method is to determine which implementation of the function (if overloaded) will be called. The process of finding the correct function is actually quite simple: create a globally unique key based on the passed function and its parameters, then check if this key has been registered in the namespace; if so, read the implementation corresponding to the key.

def get(self, fn, *args):
  """get returns the function matched in the namespace

  If no function is matched, it returns None.
  """
  func = Function(fn)
  return self.function_map.get(func.key(args=args))

The get method internally creates an instance of Function, then uses this instance's key method to obtain the unique key for the function. It then uses this key to retrieve the corresponding function from the function registry.

Function call#

As mentioned above, every time a function decorated with overload is called, the __call__ method in the Function class is invoked. In this method, we need to find and call the correct overloaded function implementation using the get method of the virtual namespace. The implementation of the __call__ method is as follows:

def __call__(self, *args, **kwargs):
  """Overriding the class's `__call__` method allows an instance to be callable.
  """
  # Get the actual function to be called from the virtual namespace
  fn = Namespace.get_instance().get(self.fn, *args)
  if not fn:
    raise Exception("no matching function found.")

  # Return the function's return value
  return fn(*args, **kwargs)

This method finds the correct function from the virtual namespace. If no function is found, it raises an Exception. If the function exists, it calls the function and returns the value.

Practical application#

Putting all the code together, we define two functions named area: one function calculates the area of a rectangle, and the other calculates the area of a circle.

We will decorate both functions with the overload decorator.

@overload
def area(l, b):
  return l * b

@overload
def area(r):
  import math
  return math.pi * r ** 2


>>> area(3, 4)
12
>>> area(7)
153.93804002589985

When we call area with only one parameter, we can see that it returns the area of the circle; when we pass in two parameters, we find that it returns the area of the rectangle.

See the complete code at: https://repl.it/@arpitbbhayani/Python-Function-Overloading

Conclusion#

Python does not support function overloading by default, but we have devised a solution using simple language structures. We use decorators and a user-maintained virtual namespace, taking the number of function parameters as the disambiguation factor to overload functions. We can also use the data types of function parameters for disambiguation, allowing for function overloading with the same number of parameters but different types. The granularity of overloading is only limited by the getfullargspec function and our imagination.

You can also create a cleaner, more efficient implementation based on the above content, so feel free to implement one and tweet it to me @arpit_bhayani, and I will be happy to learn from your implementation.

Starting from Python 3.4, you can use functools.singledispatch to implement function overloading.
Starting from Python 3.8, you can use functools.singledispatchmethod to implement method overloading in instances.

Thanks to Harry Percival for the correction.