================== Creating a Package ================== Basics: Creating and Distributing Distributions =============================================== If you have some useful Python :term:`modules ` that you think others might benefit from, but aren't sure how to go about packaging them up and distributing them, then this short document is for you. By the end of it, you'll be a contributor to the :ref:`pypi_info`. For a more detailed look at packaging a larger project, see this :doc:`example `. Let's begin. Background ---------- Suppose you've written a couple modules to help you keep track of your towel (``location.py`` and ``utils.py``), and you'd like to share them. First thing to do is come up with a CamelCase project name for them. Let's go with "TowelStuff" since it seems appropriate and also it has not yet been used on the :ref:`pypi_info`. .. _directory_layout: Arranging your file and directory structure ------------------------------------------- "TowelStuff" will be the name of our project as well as the name of our :term:`distribution`. We should also come up with a :term:`package` name within which our modules will reside (to avoid naming conflicts with other modules). For this example, there's only one package, so let's reuse the project name and go with "towelstuff". Make the layout of your project directory (described below) look like this:: TowelStuff/ bin/ CHANGES.txt docs/ LICENSE.txt MANIFEST.in README.txt setup.py towelstuff/ __init__.py location.py utils.py test/ __init__.py test_location.py test_utils.py Here's what you should do for each of those listed above: .. _bin_description: * Put into ``bin`` any scripts you've written that use your ``towelstuff`` package and which you think would be useful for your users. If you don't have any, then remove the ``bin`` directory. .. _changes_txt_description: * For now, the ``CHANGES.txt`` file should only contain:: v, -- Initial release. since this is your very first version (version number will be described below) and there are no changes to report. .. _docs_description: * The ``docs`` dir should contain any design docs, implementation notes, a FAQ, or any other docs you've written. For now, stick to plain text files ending in ".txt". This author (JohnMG) likes to use `Pandoc's Markdown `_, but many Pythoneers use :term:`reStructuredText `. .. _license_txt_description: * The ``LICENSE.txt`` file is often just a copy/paste of your license of choice. We recommend going with a commonly-used license, such as the GPL, BSD, or MIT. .. _manifest_in_description: * The ``MANIFEST.in`` file should contain this:: include *.txt recursive-include docs *.txt .. _readme_txt_description: * The ``README.txt`` file should be written in :term:`reST ` so that the PyPI can use it to generate your project's PyPI page. Here's a 10-second intro to reST that you might use to start with:: =========== Towel Stuff =========== Towel Stuff provides such and such and so and so. You might find it most useful for tasks involving and also . Typical usage often looks like this:: #!/usr/bin/env python from towelstuff import location from towelstuff import utils if utils.has_towel(): print "Your towel is located:", location.where_is_my_towel() (Note the double-colon and 4-space indent formatting above.) Paragraphs are separated by blank lines. *Italics*, **bold**, and ``monospace`` look like this. A Section ========= Lists look like this: * First * Second. Can be multiple lines but must be indented properly. A Sub-Section ------------- Numbered lists look like you'd expect: 1. hi there 2. must be going Urls are http://like.this and links can be written `like this `_. You might also consider adding a "Contributors" section and/or a "Thanks also to" section to list the names of people who've helped. By the way, to see how the above ``README.txt`` looks rendered in html, see the `TowelStuff project `_ at the PyPI. .. _setup_py_description: * ``setup.py`` -- Create this file and make it look like this:: from distutils.core import setup setup( name='TowelStuff', version='0.1.0', author='J. Random Hacker', author_email='jrh@example.com', packages=['towelstuff', 'towelstuff.test'], scripts=['bin/stowe-towels.py','bin/wash-towels.py'], url='http://pypi.python.org/pypi/TowelStuff/', license='LICENSE.txt', description='Useful towel-related stuff.', long_description=open('README.txt').read(), install_requires=[ "Django >= 1.1.1", "caldav == 0.1.4", ], ) but, of course, replace the towel stuff with your own project and package names. For more details about picking version numbers, see :doc:`versioning`, but '0.1.0' will work just fine for a first release (this is using the common "major.minor.micro" numbering convention). Use the install_requires argument to automatically install dependencies when your package will be installed and include information about dependencies (so that package management tools like Pip can use the information). It takes a string or list of strings containing requirement specifiers. The syntax consists of a project's PyPI name, optionally followed by a comma-separated list of version specifiers. Modern packaging tools implement version specifiers syntax described in `PEP 345`_ and resolve version comparison in compliance with `PEP 386`_. .. _`PEP 345`: http://www.python.org/dev/peps/pep-0345/#version-specifiers .. _`PEP 386`: http://www.python.org/dev/peps/pep-0386/ If you have no scripts to distribute (and thus no ``bin`` dir), you can remove the above line which begins with "scripts". .. _towelstuff_description: * Inside the ``towelstuff`` directory, ``__init__.py`` can be empty. Likewise, inside ``towelstuff/test``, *that* ``__init__.py`` can be empty as well. If you have no tests written yet, you can leave the two other module files in ``towelstuff/test`` empty for now too. When writing your tests, use the standard ``unittest`` module. For our example, TowelStuff does not depend upon any other distributions (it only depends upon what's already in the Python standard library). To specify dependencies upon other distributions, see the more detailed :doc:`example`. Creating your distribution file ------------------------------- Create your distribution file like so:: $ cd path/to/TowelStuff $ python setup.py sdist Running that last command will create a ``MANIFEST`` file in your project directory, and also a ``dist`` and ``build`` directory. Inside that ``dist`` directory is the distribution that you'll be uploading to the PyPI. In our case, the distribution file will be named ``TowelStuff-0.1.0.tgz``. Feel free to poke around in the ``dist`` directory to look at your distribution. Uploading your distribution file -------------------------------- Before uploading you first need to create an account at http://pypi.python.org/pypi . Once that's complete, register your distribution at the PyPI like so:: $ cd path/to/TowelStuff $ python setup.py register Use your existing login (choice #1). It will prompt you to save the login info for future use (to which I agree). Then upload:: $ python setup.py sdist upload This builds the distribution one last time and then uploads it. Thanks for your contribution! Updating your distribution -------------------------- Down the road, after you've made updates to your distribution and wish to make a new release: 1. increment the version number in your ``setup.py`` file, 2. update your ``CHANGES.txt`` file, 3. if necessary, update the "Contributors" and "Thanks also to" sections of your ``README.txt`` file. 4. run ``python setup.py sdist upload`` again. Entry points ------------ Entry points are a Setuptools/Distribute feature that’s really handy in one specific case: register something under a specific key in package A that package B can query for. Distribute itself uses it. If you’re packaging your project up properly, you've probably used the ``console_scripts`` entry point:: setup(name='zest.releaser', ... entry_points={ 'console_scripts': ['release = zest.releaser.release:main', 'prerelease = zest.releaser.prerelease:main', ]} ) ``console_scripts`` is an entry point that Setuptools looks up. It looks up all entry points registered under the name console_scripts and uses that information to generate scripts. In the above example that’d be a bin/release script that runs the main() method in zest/releaser/release.py. You can use that for your own extension mechanism. For ``zest.releaser`` I needed some extension mechanism. I wanted to be able to do extra things on prerelease/release/postrelease time. - Downloading an external javascript library into a package that cannot be stored in (zope's) svn repository directly due to licensing issues. Before packaging and releasing it, that is. Automatically so you don’t forget it. - Uploading a version.cfg to ``scp://somewhere/kgs/ourmainproduct-version.cfg`` after making a release to use it as a so-called “known good set” (KGS). - Possibly modifying values (like a commit message) inside zest.releaser itself while doing a release. (I do get modification requests from time to time “hey, can you make x and y configurable”). So now every zest.releaser step (prerelease, release, postrelease) is splitted in two: a calculation phase and a “doing” phase. The results of the first phase are stored in a dict that gets used in the second phase. And you can register an entry point that gets passed that dict so you can modify it. See the entry point documentation of zest.releaser for details. An entry point for zest.releaser is configured like this in your setup.py:: entry_points={ 'console_scripts': ['myscript = my.package.scripts:main'], 'zest.releaser.prereleaser.middle': ['dosomething = my.package.some:some_entrypoint, ] } Replace prereleaser and middle in zest.releaser.prereleaser.middle with prerelease/release/postrelease and before/middle/after where needed. (For this specific zest.releaser example). Now, how to use this in your program? The best way is to show a quick example from zest.releaser where we query and use one of our entry points:: import pkg_resources ... def run_entry_point(data): # Note: data is zest.releaser specific: we want to pass # something to the plugin group = 'zest.releaser.prerelease.middle' for entrypoint in pkg_resources.iter_entry_points(group=group): # Grab the function that is the actual plugin. plugin = entrypoint.load() # Call the plugin plugin(data) So: pretty easy and simple way to allow other packages to register something that you want to know. Extra plugins, extra render methods, extra functionality you want to register in your web application, etcetera. Packaging for a Particular Operating System (OS) ================================================ General Packaging Guidelines for Unix ------------------------------------- General Packaging Guidelines for Windows ----------------------------------------