Types are great, and so are stub files. However there's a major issue at the moment which means your implementation doesn't have to behave the same as what your stub file say. Don't use stub files yet.
When I write python, I insist on using types. There are 2 main reasons I do this:
- Autocomplete is fantastic when you are in the middle of writing out a component
- It forces me to use sensible yet minimal interfaces
Both of these amount to being able to write a library, then pick up where I left off 6 months later, something that is normally a common issue with dynamic languages.
While it's easy enough to do type hinting for the above points, I actually like relying upon the type checker to catch mistakes. This often results in some pretty gnarly looking code.
Lets take a look at a decorator:
def some_decorator(func=None, *, param=False):
def inner(func):
nonlocal param
func.my_param = param
return func
if func is None:
return inner
return inner(func)
@some_decorator
def foo(bar: int) -> int: ...
reveal_type(foo) # Revealed type is 'Any'
This decorator can be used on a function. It can be used in a few different ways:
@some_decorator
def _(): ...
@some_decorator()
def _(): ...
@some_decorator(param=None)
def _(): ...
Here is how you can do type hinting in a basic fashion:
from typing import Callable, Any
def some_decorator(
func: Callable = None,
*,
param: Any = False,
) -> Callable:
def inner(func: Callable) -> Callable:
nonlocal param
func.my_param = param
return func
if func is None:
return inner
return inner(func)
@some_decorator
def foo(bar: int) -> int: ...
reveal_type(foo) # Revealed type is 'def (*Any, **Any) -> Any'
While this makes it pass type checking, it doesn't really add any valuable information.
Let's add some context to the types:
from typing import Callable, Any, TypeVar, Optional, Union
# TypeVar allows the input type to be passed around, essentially a Generic.
# typescript has a good guide on generics: https://www.typescriptlang.org/docs/handbook/generics.html
T_Func = TypeVar('T_Func', bound=Callable)
def some_decorator(
func: Optional[T_Func] = None,
*,
param: Any = False,
) -> Union[
T_Func,
Callable[[T_Func], T_Func],
]:
def inner(func: T_Func) -> T_Func:
nonlocal param
func.my_param = param
return func
if func is None:
return inner
return inner(func)
@some_decorator
def foo(bar: int) -> int: ...
reveal_type(foo) # Revealed type is 'Union[def (bar: builtins.int) -> builtins.int, def (def (bar: builtins.int) -> builtins.int) -> def (bar: builtins.int) -> builtins.int]'
This works, and gives us more context when dealing with static checking. However the checks are pretty loose at the moment. The decorator can return either a function wrapping your function, or your function.
Reading the code, it's obvious that when you don't pass a function, you get a wrapper. but when you do pass a function, you'll get your function back.
Let's tell the type checker this:
from typing import Callable, Any, TypeVar, Optional, Union, overload
T_Func = TypeVar('T_Func', bound=Callable)
@overload
def some_decorator(
func: None = None,
*,
param: Any = False,
) -> Callable[[T_Func], T_Func]: ...
@overload
def some_decorator(
func: T_Func,
*,
param: Any = False,
) -> T_Func: ...
def some_decorator(
func: Optional[T_Func] = None,
*,
param: Any = False,
) -> Union[
T_Func,
Callable[[T_Func], T_Func],
]:
def inner(func: T_Func) -> T_Func:
nonlocal param
func.my_param = param
return func
if func is None:
return inner
return inner(func)
@some_decorator
def foo(bar: int) -> int: ...
reveal_type(foo) # Revealed type is 'def (bar: builtins.int) -> builtins.int'
This is a lot of code for a fairly simple decorator, and it heavily pollutes the logic.
Sure, I get it. it's not pretty, and it makes the actual logic hard to grok.
So let's split it up:
# some_decorator.py
def some_decorator(func=None, *, param=False):
def inner(func):
nonlocal param
func.my_param = param
return func
if func is None:
return inner
return inner(func)
# some_decorator.pyi
from typing import Callable, Any, TypeVar, overload
T_Func = TypeVar('T_Func', bound=Callable)
@overload
def some_decorator(
func: T_Func,
*,
param: Any = False,
) -> T_Func: ...
@overload
def some_decorator(
func: None = None,
*,
param: Any = False,
) -> Callable[[T_Func], T_Func]: ...
# foo.py
## Must be a different file for the .pyi files to pickup
from .some_decorator import some_decorator
@some_decorator
def foo(bar: int) -> int: ...
reveal_type(foo) # Revealed type is 'def (bar: builtins.int) -> builtins.int'
Note that now we have a .pyi
file, we don't need to use type hints in the main .py
file.
There's now a clean divide between types and logic.
There's currently a bug (even though it's classed as a feature) that makes it so .pyi
files are
not cross-checked against their counterpart .py
file.
python/mypy#5028
This means that the function defined in .py
doesn't have to align with .pyi
.
This is a massive hole, and means you can't actually do type checking using just stub files. While you can promise an implementation will behave, it doesn't mean the implementation will behave.