How to write a Python module/package?

python modules list
create python package
python packages list
python import module from directory
python module vs package
built-in packages in python
python import package
python package structure

I've been making Python scripts for simple tasks at work and never really bothered packaging them for others to use. Now I have been assigned to make a Python wrapper for a REST API. I have absolutely no idea on how to start and I need help.

What I have:

(Just want to be specific as possible) I have the virtualenv ready, it's also up in github, the .gitignore file for python is there as well, plus, the requests library for interacting with the REST API. That's it.

Here's the current directory tree

.
├── bin
│   └── /the usual stuff/
├── include
│   └── /the usual stuff/
├── lib
│   └── python2.7
│       └── /the usual stuff/
├── local
│   └── /the usual stuff/
└── README.md

27 directories, 280 files

I don't even know where to put the .py files, if I ever make one.

What I wanted to do:

Make a python module install-able with "pip install ..."

If possible, I want a general step by step process on writing Python modules.

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py

create hello.py then write the following function as its content:

def helloworld():
   print "hello"

Then you can import hello:

>>> import hello
>>> hello.helloworld()
'hello'
>>>

To group many .py files put them in a folder. Any folder with an __init__.py is considered a module by python and you can call them a package

|-HelloModule
  |_ __init__.py
  |_ hellomodule.py

You can go about with the import statement on your module the usual way.

For more information, see 6.4. Packages.

6. Modules, Fibonacci numbers module def fib(n): # write Fibonacci series up to n a, b = 0, Note that in general the practice of importing * from a module or package is  Package: Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name A.B designates a submodule named B in a package named A. Just like the use of modules saves the authors of different modules from having to worry about each other’s global variable names, the use of dotted

Python 3 - UPDATED 18th November 2015

Found the accepted answer useful, yet wished to expand on several points for the benefit of others based on my own experiences.

Module: A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended.

Module Example: Assume we have a single python script in the current directory, here I am calling it mymodule.py

The file mymodule.py contains the following code:

def myfunc():
    print("Hello!")

If we run the python3 interpreter from the current directory, we can import and run the function myfunc in the following different ways (you would typically just choose one of the following):

>>> import mymodule
>>> mymodule.myfunc()
Hello!
>>> from mymodule import myfunc
>>> myfunc()
Hello!
>>> from mymodule import *
>>> myfunc()
Hello!

Ok, so that was easy enough.

Now assume you have the need to put this module into its own dedicated folder to provide a module namespace, instead of just running it ad-hoc from the current working directory. This is where it is worth explaining the concept of a package.

Package: Packages are a way of structuring Python’s module namespace by using "dotted module names". For example, the module name A.B designates a submodule named B in a package named A. Just like the use of modules saves the authors of different modules from having to worry about each other’s global variable names, the use of dotted module names saves the authors of multi-module packages like NumPy or the Python Imaging Library from having to worry about each other’s module names.

Package Example: Let's now assume we have the following folder and files. Here, mymodule.py is identical to before, and __init__.py is an empty file:

.
└── mypackage
    ├── __init__.py
    └── mymodule.py

The __init__.py files are required to make Python treat the directories as containing packages. For further information, please see the Modules documentation link provided later on.

Our current working directory is one level above the ordinary folder called mypackage

$ ls
mypackage

If we run the python3 interpreter now, we can import and run the module mymodule.py containing the required function myfunc in the following different ways (you would typically just choose one of the following):

>>> import mypackage
>>> from mypackage import mymodule
>>> mymodule.myfunc()
Hello!
>>> import mypackage.mymodule
>>> mypackage.mymodule.myfunc()
Hello!
>>> from mypackage import mymodule
>>> mymodule.myfunc()
Hello!
>>> from mypackage.mymodule import myfunc
>>> myfunc()
Hello!
>>> from mypackage.mymodule import *
>>> myfunc()
Hello!

Assuming Python 3, there is excellent documentation at: Modules

In terms of naming conventions for packages and modules, the general guidelines are given in PEP-0008 - please see Package and Module Names

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

Python Modules and Packages – An Introduction – Real Python, The Module Search Path. Continuing with the above example, let's take a look at what happens when Python executes the statement: import  Writing packages. Packages are namespaces which contain multiple packages and modules themselves. They are simply directories, but with a twist. Each package in Python is a directory which MUST contain a special file called __init__.py. This file can be empty, and it indicates that the directory it contains is a Python package, so it can be imported the same way a module can be imported.

Since nobody did cover this question of the OP yet:

What I wanted to do:

Make a python module install-able with "pip install ..."

Here is an absolute minimal example, showing the basic steps of preparing and uploading your package to PyPI using setuptools and twine.

This is by no means a substitute for reading at least the tutorial, there is much more to it than covered in this very basic example.

Creating the package itself is already covered by other answers here, so let us assume we have that step covered and our project structure like this:

.
└── hellostackoverflow/
    ├── __init__.py
    └── hellostackoverflow.py

In order to use setuptools for packaging, we need to add a file setup.py, this goes into the root folder of our project:

.
├── setup.py
└── hellostackoverflow/
    ├── __init__.py
    └── hellostackoverflow.py

At the minimum, we specify the metadata for our package, our setup.py would look like this:

from setuptools import setup

setup(
    name='hellostackoverflow',
    version='0.0.1',
    description='a pip-installable package example',
    license='MIT',
    packages=['hellostackoverflow'],
    author='Benjamin Gerfelder',
    author_email='benjamin.gerfelder@gmail.com',
    keywords=['example'],
    url='https://github.com/bgse/hellostackoverflow'
)

Since we have set license='MIT', we include a copy in our project as LICENCE.txt, alongside a readme file in reStructuredText as README.rst:

.
├── LICENCE.txt
├── README.rst
├── setup.py
└── hellostackoverflow/
    ├── __init__.py
    └── hellostackoverflow.py

At this point, we are ready to go to start packaging using setuptools, if we do not have it already installed, we can install it with pip:

pip install setuptools

In order to do that and create a source distribution, at our project root folder we call our setup.py from the command line, specifying we want sdist:

python setup.py sdist

This will create our distribution package and egg-info, and result in a folder structure like this, with our package in dist:

.
├── dist/
├── hellostackoverflow.egg-info/
├── LICENCE.txt
├── README.rst
├── setup.py
└── hellostackoverflow/
    ├── __init__.py
    └── hellostackoverflow.py

At this point, we have a package we can install using pip, so from our project root (assuming you have all the naming like in this example):

pip install ./dist/hellostackoverflow-0.0.1.tar.gz

If all goes well, we can now open a Python interpreter, I would say somewhere outside our project directory to avoid any confusion, and try to use our shiny new package:

Python 3.5.2 (default, Sep 14 2017, 22:51:06) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from hellostackoverflow import hellostackoverflow
>>> hellostackoverflow.greeting()
'Hello Stack Overflow!'

Now that we have confirmed the package installs and works, we can upload it to PyPI.

Since we do not want to pollute the live repository with our experiments, we create an account for the testing repository, and install twine for the upload process:

pip install twine

Now we're almost there, with our account created we simply tell twine to upload our package, it will ask for our credentials and upload our package to the specified repository:

twine upload --repository-url https://test.pypi.org/legacy/ dist/*

We can now log into our account on the PyPI test repository and marvel at our freshly uploaded package for a while, and then grab it using pip:

pip install --index-url https://test.pypi.org/simple/ hellostackoverflow

As we can see, the basic process is not very complicated. As I said earlier, there is a lot more to it than covered here, so go ahead and read the tutorial for more in-depth explanation.

How to write a Python module/package?, Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged. Writing a module is just like writing any other Python file. Modules can contain definitions of functions, classes, and variables that can then be utilized in other Python programs. From our Python 3 local programming environment or server-based programming environment , let’s start by creating a file hello.py that we’ll later import into another file.

Once you have defined your chosen commands, you can simply drag and drop the saved file into the Lib folder in your python program files.

>>> import mymodule 
>>> mymodule.myfunc()

[PDF] Writing Python Libraries, Python packages are collections of modules. In a directory structure, in order for a folder containing Python files to be recognized as a package, an __init__.py. A Python library is a coherent collection of Python modules that is organized as a Python package. In general, that means that all modules live under the same directory and that this directory is on the Python search path. Let's quickly write a little Python 3 package and illustrate all these concepts. The Pathology Package

Make a file named "hello.py"

If you are using Python 2.x

def func():
    print "Hello"

If you are using Python 3.x

def func():
    print("Hello")

Run the file. Then, you can try the following:

>>> import hello
>>> hello.func()
Hello

If you want a little bit hard, you can use the following:

If you are using Python 2.x

def say(text):
    print text

If you are using Python 3.x

def say(text):
    print(text)

See the one on the parenthesis beside the define? That is important. It is the one that you can use within the define.

Text - You can use it when you want the program to say what you want. According to its name, it is text. I hope you know what text means. It means "words" or "sentences".

Run the file. Then, you can try the following if you are using Python 3.x:

>>> import hello
>>> hello.say("hi")
hi
>>> from hello import say
>>> say("test")
test

For Python 2.x - I guess same thing with Python 3? No idea. Correct me if I made a mistake on Python 2.x (I know Python 2 but I am used with Python 3)

Creating Modules in Python 3, Some modules are available through the Python Standard Library and are therefore installed with your Python installation. Others can be installed with Python's package manager pip . Additionally, you can create your own Python modules since modules are comprised of Python . py files. You need to add the directory of the python modules to sys path. If you have something like this. Root here_using_my_module.py my_module __init__.py --> leave it empty a.py b.py c.py You need to add you module directory to sys_path

How to Write, Package and Distribute a Library in Python, In this tutorial, you'll learn everything you need to know about writing, packaging and distributing your own packages. How to Write a Python  How To Package Your Python Code¶ This tutorial aims to put forth an opinionated and specific pattern to make trouble-free packages for community use. It doesn’t describe the only way of doing things, merely one specific approach that works well.

How to Create a Python Package, Example On How to Create a Python Package. In this tutorial, we will create an Animals package – which simply contains two module files  Example On How to Create a Python Package. In this tutorial, we will create an Animals package – which simply contains two module files named Mammals and Birds, containing the Mammals and Birds classes, respectively. Step 1: Create the Package Directory. So, first we create a directory named Animals. Step 2: Add Classes. Now, we create the two classes for our package.

Python Modules, Python Modules Consider a module to be the same as a code library. Example. Save this code in a file named mymodule.py. def greeting(name): A Python package is simply an organized collection of python modules. A python module is simply a single python file. Creating a package with __init__.py is all about making it easier to develop larger Python projects. It provides a mechanism for you to group separate python scripts into a single importable module.

Comments
  • I would start with chapter 6 of the tutorial (2.7), or here for 3.x Search the internet for python module tutorial and you'll find plenty of others.
  • No one answered the pip part
  • github.com/MacHu-GWU/pygitrepo-project this library helps you to create project skeleton from scratch, and the feature you need is out of box.
  • would that last one be: from HellowModule import hellomodule? Could that be hello in the module folder, so it would be from HelloModule import hello
  • Nice simple explanation. What if you want to keep another folder inside mypackage ?
  • The include totally depends on what you did wrote. In the case of you put stuff outside a function on your module, you'll fire it when calling like import mypackage. In the case you wanna import just a function from a module (even a file) is better to use from module import function. In the case a subfolder from subfolder.module import function so you can simply call function() witthout fire other code parts. Also, do not use from module import * if you don't really need.
  • The only question left is how can I get the package to import everything on import mypackage? Adding import mymodule to __init__.py does not work..
  • Neat explanation! however, I have a question if numpy is a package how can I execute numpy.cos(1) in my interpreter because that seems to be a module name in between go missing. No?
  • How about pip ?
  • Will my package be published right after setuptools?
  • @U9-Forward No, the publishing is done with twine, but you can test your package locally before publishing after you created it with setuptools.