Skip to content

We are the Mods: Modules & Package (and Scripts)#

Basic Module and Package Layout#

A typical Python code layout structures the code in package directories and module files.

A very simple package structure could look like this:

mypackage/
mypackage/__init__.py
mypackage/module1.py
mypackage/module2.py

Suppose these files have the following contents:

# mypackage Python package
print('mypackage')

# module1
print("I'm module 1")

def f(*args, **kwargs):
    print('Module 1 is great')

# module2
print("I'm module 2")

def f(*args, **kwargs):
    print('Module 2 is also great')

This package can now be used as follows: 

>>> import mypackage
mypackage
>>> mypackage.module1.f()  # this will fail: mypackage.module1 not yet imported
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'mypackage' has no attribute 'module1'
>>>
>>> import mypackage.module1
I'm module 1
>>> mypackage.module1.f()
Module 1 is great
>>> import mypackage.module2
I'm module 2
>>> mypackage.module2.f()
Module 2 is also great
>>>
>> import mypackage.module1 # 2nd import 
>>>

Some things to note:

  • code in an __init__.py-file in a package dir gets executed when the package is imported
  • __init__.py code is just regular Python code
  • statements in a module get executed when the module is imported
  • modules are imported only once per interpreter session (you can enforce reload in interactive sessions)

Regular and Namespace Packages#

Module Search Path#

Lookup of modules involves a search path. The search order for mod.py is

  1. look for a built-in module with that name
  2. look in the directories available in sys.path for the mod.py file

sys.path basically contains

  • the directory containing the importing file, or the current working directory if no file, i.e. in an interactive interpreter session
  • the directories set in the (optional) PYTHONPATH environment variable

  • the default directories of the Python installation, e.g. for a Python 3.6 linux installation:

    .../lib/python3.6/                # stdlib
.../lib/python3.6/lib-dynload/    # stdlib shared libraries
.../lib/python3.6/site-packages/  # site-wide installed 3rd party packages

Scripts#

Scripts are Python modules that are intended to be run as executables. More often that not such scripts will want to parse command line options, receive stdin input or user input and write stdout output or output file(s).

Often, the duality of a script also being a normal module makes it desirable to have it act both as an importable module as well as an executable.

In such situations it makes sense to guard the executable operations with:

if __name__ == "__main__":
    # everything here will only get executed when run as the main module
    # i.e. the special name __name__ contains the string "__main__"
    ...

The special name __name__ is set to "__main__" for the main file that gets executed by the Python interpreter, e.g.

python myprogram.py

or

./myprogram.py

if this file has been made executable.

Note: The main file run by the Python interpreter doesn't need to have the .py file extension, so you can make such a file look like an executable or command with regard to its file name ("myprogram"). However, it is not importable itself, then.

Here's a template for a basic command line script:

import sys


def parse_args(args=None):
    """Parse arguments from sys.argv if args is None (the default) or from args
    sequence otherwise.
    """
    import argparse
    parser = argparse.ArgumentParser()
    # Add arguments here

    args = parser.parse_args(args)
    return args


def main(args=None):
    """Main module function.

    Exposes this modules' executable functionality for use as a module
    function. 
    Parses arguments from sys.argv if args is None (the default) or from args
    sequence otherwise.
    """
    args = parse_args(args)
    # Add main code here


if __name__ == "__main__":
    sys.exit(main())

Hint: More elaborate Python libraries that contain many packages and modules may want use the console_scripts entry point functionality of setuptools. This is allows you to expose functions of your library (the "entry points") as command line scripts, automatically generated by setuptools when building the installable library package from the source code files.

You can find more information on this here or in the Python Packaging User Guide.