ÿØÿà JFIF    ÿÛ „ ( %!1!%*+...983,7(-.- PK\d+)', url='{package_url}/issues/{issue}', ), dict( pattern=r'BB Pull Request ?#(?P\d+)', url='{BB}/pypa/setuptools/pull-request/{bb_pull_request}', ), dict( pattern=r'Distribute #(?P\d+)', url='{BB}/tarek/distribute/issue/{distribute}', ), dict( pattern=r'Buildout #(?P\d+)', url='{GH}/buildout/buildout/issues/{buildout}', ), dict( pattern=r'Old Setuptools #(?P\d+)', url='http://bugs.python.org/setuptools/issue{old_setuptools}', ), dict( pattern=r'Jython #(?P\d+)', url='http://bugs.jython.org/issue{jython}', ), dict( pattern=r'(Python #|bpo-)(?P\d+)', url='http://bugs.python.org/issue{python}', ), dict( pattern=r'Interop #(?P\d+)', url='{GH}/pypa/interoperability-peps/issues/{interop}', ), dict( pattern=r'Pip #(?P\d+)', url='{GH}/pypa/pip/issues/{pip}', ), dict( pattern=r'Packaging #(?P\d+)', url='{GH}/pypa/packaging/issues/{packaging}', ), dict( pattern=r'[Pp]ackaging (?P\d+(\.\d+)+)', url='{GH}/pypa/packaging/blob/{packaging_ver}/CHANGELOG.rst', ), dict( pattern=r'PEP[- ](?P\d+)', url='https://www.python.org/dev/peps/pep-{pep_number:0>4}/', ), dict( pattern=r'setuptools_svn #(?P\d+)', url='{GH}/jaraco/setuptools_svn/issues/{setuptools_svn}', ), dict( pattern=r'pypa/distutils#(?P\d+)', url='{GH}/pypa/distutils/issues/{distutils}', ), dict( pattern=r'^(?m)((?Pv?\d+(\.\d+){1,2}))\n[-=]+\n', with_scm='{text}\n{rev[timestamp]:%d %b %Y}\n', ), ], ), } # Be strict about any broken references: nitpicky = True intersphinx_mapping = { 'pypa-build': ('https://pypa-build.readthedocs.io/en/latest/', None) } # Add support for linking usernames github_url = 'https://github.com' github_sponsors_url = f'{github_url}/sponsors' extlinks = { 'user': (f'{github_sponsors_url}/%s', '@'), # noqa: WPS323 } extensions += ['sphinx.ext.extlinks', 'sphinx.ext.intersphinx'] # Ref: https://github.com/python-attrs/attrs/pull/571/files\ # #diff-85987f48f1258d9ee486e3191495582dR82 default_role = 'any' # HTML theme html_theme = 'furo' # Add support for inline tabs extensions += ['sphinx_inline_tabs'] # Support for distutils # Ref: https://stackoverflow.com/a/30624034/595220 nitpick_ignore = [ ('c:func', 'SHGetSpecialFolderPath'), # ref to MS docs ('envvar', 'DISTUTILS_DEBUG'), # undocumented ('envvar', 'HOME'), # undocumented ('envvar', 'PLAT'), # undocumented ('py:attr', 'CCompiler.language_map'), # undocumented ('py:attr', 'CCompiler.language_order'), # undocumented ('py:class', 'distutils.dist.Distribution'), # undocumented ('py:class', 'distutils.extension.Extension'), # undocumented ('py:class', 'BorlandCCompiler'), # undocumented ('py:class', 'CCompiler'), # undocumented ('py:class', 'CygwinCCompiler'), # undocumented ('py:class', 'distutils.dist.DistributionMetadata'), # undocumented ('py:class', 'FileList'), # undocumented ('py:class', 'IShellLink'), # ref to MS docs ('py:class', 'MSVCCompiler'), # undocumented ('py:class', 'OptionDummy'), # undocumented ('py:class', 'UnixCCompiler'), # undocumented ('py:exc', 'CompileError'), # undocumented ('py:exc', 'DistutilsExecError'), # undocumented ('py:exc', 'DistutilsFileError'), # undocumented ('py:exc', 'LibError'), # undocumented ('py:exc', 'LinkError'), # undocumented ('py:exc', 'PreprocessError'), # undocumented ('py:func', 'distutils.CCompiler.new_compiler'), # undocumented # undocumented: ('py:func', 'distutils.dist.DistributionMetadata.read_pkg_file'), ('py:func', 'distutils.file_util._copy_file_contents'), # undocumented ('py:func', 'distutils.log.debug'), # undocumented ('py:func', 'distutils.spawn.find_executable'), # undocumented ('py:func', 'distutils.spawn.spawn'), # undocumented # TODO: check https://docutils.rtfd.io in the future ('py:mod', 'docutils'), # there's no Sphinx site documenting this ] # Allow linking objects on other Sphinx sites seamlessly: intersphinx_mapping.update( python=('https://docs.python.org/3', None), python2=('https://docs.python.org/2', None), ) # Add support for the unreleased "next-version" change notes extensions += ['sphinxcontrib.towncrier'] # Extension needs a path from here to the towncrier config. towncrier_draft_working_directory = '..' # Avoid an empty section for unpublished changes. towncrier_draft_include_empty = False extensions += ['jaraco.tidelift'] PK`_ to track a roadmap of large-scale goals. PK` command below for more details. Note that you can also apply setuptools commands to non-setuptools projects, using commands like this:: python -c "import setuptools; with open('setup.py') as f: exec(compile(f.read(), 'setup.py', 'exec'))" develop That is, you can simply list the normal setup commands and options following the quoted part. PK`_ Build system requirement ======================== Package requirement ------------------- After organizing all the scripts and files and getting ready for packaging, there needs to be a way to tell Python what programs it needs to actually do the packaging (in our case, ``setuptools`` of course). Usually, you also need the ``wheel`` package as well since it is recommended that you upload a ``.whl`` file to PyPI alongside your ``.tar.gz`` file. Unlike the other two types of dependency keyword, this one is specified in your ``pyproject.toml`` file (if you have forgot what this is, go to :doc:`quickstart` or (WIP)): .. code-block:: ini [build-system] requires = ["setuptools", "wheel"] #... .. note:: This used to be accomplished with the ``setup_requires`` keyword but is now considered deprecated in favor of the PEP 517 style described above. To peek into how this legacy keyword is used, consult our :doc:`guide on deprecated practice (WIP) <../deprecated/index>` .. _Declaring Dependencies: Declaring required dependency ============================= This is where a package declares its core dependencies, without which it won't be able to run. ``setuptools`` support automatically download and install these dependencies when the package is installed. Although there is more finesse to it, let's start with a simple example. .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = docutils BazSpam ==1.1 .. tab:: setup.py .. code-block:: python setup( ..., install_requires=[ 'docutils', 'BazSpam ==1.1', ], ) When your project is installed (e.g. using pip), all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed and 2) Any scripts in your project will be installed with wrappers that verify the availability of the specified dependencies at runtime. Platform specific dependencies ------------------------------ Setuptools offer the capability to evaluate certain conditions before blindly installing everything listed in ``install_requires``. This is great for platform specific dependencies. For example, the ``enum`` package was added in Python 3.4, therefore, package that depends on it can elect to install it only when the Python version is older than 3.4. To accomplish this .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = enum34;python_version<'3.4' .. tab:: setup.py .. code-block:: python setup( ..., install_requires=[ "enum34;python_version<'3.4'", ], ) Similarly, if you also wish to declare ``pywin32`` with a minimal version of 1.0 and only install it if the user is using a Windows operating system: .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = enum34;python_version<'3.4' pywin32 >= 1.0;platform_system=='Windows' .. tab:: setup.py .. code-block:: python setup( ..., install_requires=[ "enum34;python_version<'3.4'", "pywin32 >= 1.0;platform_system=='Windows'", ], ) The environmental markers that may be used for testing platform types are detailed in `PEP 508 `_. Dependencies that aren't in PyPI -------------------------------- .. warning:: Dependency links support has been dropped by pip starting with version 19.0 (released 2019-01-22). If your project depends on packages that don't exist on PyPI, you may still be able to depend on them, as long as they are available for download as: - an egg, in the standard distutils ``sdist`` format, - a single ``.py`` file, or - a VCS repository (Subversion, Mercurial, or Git). You just need to add some URLs to the ``dependency_links`` argument to ``setup()``. The URLs must be either: 1. direct download URLs, 2. the URLs of web pages that contain direct download links, or 3. the repository's URL In general, it's better to link to web pages, because it is usually less complex to update a web page than to release a new version of your project. You can also use a SourceForge ``showfiles.php`` link in the case where a package you depend on is distributed via SourceForge. If you depend on a package that's distributed as a single ``.py`` file, you must include an ``"#egg=project-version"`` suffix to the URL, to give a project name and version number. (Be sure to escape any dashes in the name or version by replacing them with underscores.) EasyInstall will recognize this suffix and automatically create a trivial ``setup.py`` to wrap the single ``.py`` file as an egg. In the case of a VCS checkout, you should also append ``#egg=project-version`` in order to identify for what package that checkout should be used. You can append ``@REV`` to the URL's path (before the fragment) to specify a revision. Additionally, you can also force the VCS being used by prepending the URL with a certain prefix. Currently available are: - ``svn+URL`` for Subversion, - ``git+URL`` for Git, and - ``hg+URL`` for Mercurial A more complete example would be: ``vcs+proto://host/path@revision#egg=project-version`` Be careful with the version. It should match the one inside the project files. If you want to disregard the version, you have to omit it both in the ``requires`` and in the URL's fragment. This will do a checkout (or a clone, in Git and Mercurial parlance) to a temporary folder and run ``setup.py bdist_egg``. The ``dependency_links`` option takes the form of a list of URL strings. For example, this will cause a search of the specified page for eggs or source distributions, if the package's dependencies aren't already installed: .. tab:: setup.cfg .. code-block:: ini [options] #... dependency_links = http://peak.telecommunity.com/snapshots/ .. tab:: setup.py .. code-block:: python setup( ..., dependency_links=[ "http://peak.telecommunity.com/snapshots/", ], ) Optional dependencies ===================== Setuptools allows you to declare dependencies that only get installed under specific circumstances. These dependencies are specified with ``extras_require`` keyword and are only installed if another package depends on it (either directly or indirectly) This makes it convenient to declare dependencies for ancillary functions such as "tests" and "docs". .. note:: ``tests_require`` is now deprecated For example, Package-A offers optional PDF support and requires two other dependencies for it to work: .. tab:: setup.cfg .. code-block:: ini [metadata] name = Package-A [options.extras_require] PDF = ReportLab>=1.2; RXP .. tab:: setup.py .. code-block:: python setup( name="Project-A", ..., extras_require={ "PDF": ["ReportLab>=1.2", "RXP"], }, ) The name ``PDF`` is an arbitrary identifier of such a list of dependencies, to which other components can refer and have them installed. There are two common use cases. First is the console_scripts entry point: .. tab:: setup.cfg .. code-block:: ini [metadata] name = Project A #... [options] #... entry_points= [console_scripts] rst2pdf = project_a.tools.pdfgen [PDF] rst2html = project_a.tools.htmlgen .. tab:: setup.py .. code-block:: python setup( name="Project-A", ..., entry_points={ "console_scripts": [ "rst2pdf = project_a.tools.pdfgen [PDF]", "rst2html = project_a.tools.htmlgen", ], }, ) This syntax indicates that the entry point (in this case a console script) is only valid when the PDF extra is installed. It is up to the installer to determine how to handle the situation where PDF was not indicated (e.g. omit the console script, provide a warning when attempting to load the entry point, assume the extras are present and let the implementation fail later). The second use case is that other package can use this "extra" for their own dependencies. For example, if "Project-B" needs "project A" with PDF support installed, it might declare the dependency like this: .. tab:: setup.cfg .. code-block:: ini [metadata] name = Project-B #... [options] #... install_requires = Project-A[PDF] .. tab:: setup.py .. code-block:: python setup( name="Project-B", install_requires=["Project-A[PDF]"], ..., ) This will cause ReportLab to be installed along with project A, if project B is installed -- even if project A was already installed. In this way, a project can encapsulate groups of optional "downstream dependencies" under a feature name, so that packages that depend on it don't have to know what the downstream dependencies are. If a later version of Project A builds in PDF support and no longer needs ReportLab, or if it ends up needing other dependencies besides ReportLab in order to provide PDF support, Project B's setup information does not need to change, but the right packages will still be installed if needed. .. note:: Best practice: if a project ends up not needing any other packages to support a feature, it should keep an empty requirements list for that feature in its ``extras_require`` argument, so that packages depending on that feature don't break (due to an invalid feature name). Python requirement ================== In some cases, you might need to specify the minimum required python version. This is handled with the ``python_requires`` keyword supplied to ``setup.cfg`` or ``setup.py``. .. tab:: setup.cfg .. code-block:: ini [metadata] name = Project-B #... [options] #... python_requires = >=3.6 .. tab:: setup.py .. code-block:: python setup( name="Project-B", python_requires=[">=3.6"], ..., ) PK`. To learn more about Python packaging in general, navigate to the :ref:`bottom ` of this page. Basic Use ========= For basic use of setuptools, you will need a ``pyproject.toml`` with the exact following info, which declares you want to use ``setuptools`` to package your project: .. code-block:: toml [build-system] requires = ["setuptools", "wheel"] build-backend = "setuptools.build_meta" Then, you will need a ``setup.cfg`` or ``setup.py`` to specify your package information, such as metadata, contents, dependencies, etc. Here we demonstrate the minimum .. tab:: setup.cfg .. code-block:: ini [metadata] name = mypackage version = 0.0.1 [options] packages = mypackage install_requires = requests importlib; python_version == "2.6" .. tab:: setup.py .. code-block:: python from setuptools import setup setup( name='mypackage', version='0.0.1', packages=['mypackage'], install_requires=[ 'requests', 'importlib; python_version == "2.6"', ], ) This is what your project would look like:: ~/mypackage/ pyproject.toml setup.cfg # or setup.py mypackage/__init__.py Then, you need an builder, such as :std:doc:`PyPA build ` which you can obtain via ``pip install build``. After downloading it, invoke the builder:: python -m build You now have your distribution ready (e.g. a ``tar.gz`` file and a ``.whl`` file in the ``dist`` directory), which you can upload to PyPI! Of course, before you release your project to PyPI, you'll want to add a bit more information to your setup script to help people find or learn about your project. And maybe your project will have grown by then to include a few dependencies, and perhaps some data files and scripts. In the next few sections, we will walk through those additional but essential information you need to specify to properly package your project. Automatic package discovery =========================== For simple projects, it's usually easy enough to manually add packages to the ``packages`` keyword in ``setup.cfg``. However, for very large projects , it can be a big burden to keep the package list updated. ``setuptools`` therefore provides two convenient tools to ease the burden: :literal:`find:\ ` and :literal:`find_namespace:\ `. To use it in your project: .. code-block:: ini [options] packages = find: [options.packages.find] #optional include=pkg1, pkg2 exclude=pk3, pk4 When you pass the above information, alongside other necessary ones, ``setuptools`` walks through the directory specified in ``where`` (omitted here as the package reside in current directory) and filters the packages it can find following the ``include`` (default to none), then remove those that match the ``exclude`` and return a list of Python packages. Note that each entry in the ``[options.packages.find]`` is optional. The above setup also allows you to adopt a ``src/`` layout. For more details and advanced use, go to :ref:`package_discovery` Entry points and automatic script creation =========================================== Setuptools support automatic creation of scripts upon installation, that runs code within your package if you specify them with the ``entry_points`` keyword. This is what allows you to run commands like ``pip install`` instead of having to type ``python -m pip install``. To accomplish this, add the entry_points keyword in your ``setup.cfg``: .. code-block:: ini [options.entry_points] console_scripts = main = mypkg:some_func When this project is installed, a ``main`` script will be installed and will invoke the ``some_func`` in the ``__init__.py`` file when called by the user. For detailed usage, including managing the additional or optional dependencies, go to :doc:`entry_point`. Dependency management ===================== ``setuptools`` supports automatically installing dependencies when a package is installed. The simplest way to include requirement specifiers is to use the ``install_requires`` argument to ``setup.cfg``. It takes a string or list of strings containing requirement specifiers (A version specifier is one of the operators <, >, <=, >=, == or !=, followed by a version identifier): .. code-block:: ini [options] install_requires = docutils >= 0.3 requests <= 0.4 When your project is installed, all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed. This, of course, is a simplified scenarios. ``setuptools`` also provide additional keywords such as ``setup_requires`` that allows you to install dependencies before running the script, and ``extras_requires`` that take care of those needed by automatically generated scripts. It also provides mechanisms to handle dependencies that are not in PyPI. For more advanced use, see :doc:`dependency_management` .. _Including Data Files: Including Data Files ==================== The distutils have traditionally allowed installation of "data files", which are placed in a platform-specific location. Setuptools offers three ways to specify data files to be included in your packages. For the simplest use, you can simply use the ``include_package_data`` keyword: .. code-block:: ini [options] include_package_data = True This tells setuptools to install any data files it finds in your packages. The data files must be specified via the distutils' ``MANIFEST.in`` file. For more details, see :doc:`datafiles` Development mode ================ ``setuptools`` allows you to install a package without copying any files to your interpreter directory (e.g. the ``site-packages`` directory). This allows you to modify your source code and have the changes take effect without you having to rebuild and reinstall. This is currently incompatible with PEP 517 and therefore it requires a ``setup.py`` script with the following content:: import setuptools setuptools.setup() Then:: pip install --editable . This creates a link file in your interpreter site package directory which associate with your source code. For more information, see :doc:`development_mode`. Uploading your package to PyPI ============================== After generating the distribution files, next step would be to upload your distribution so others can use it. This functionality is provided by `twine `_ and we will only demonstrate the basic use here. Transitioning from ``setup.py`` to ``setup.cfg`` ================================================ To avoid executing arbitrary scripts and boilerplate code, we are transitioning into a full-fledged ``setup.cfg`` to declare your package information instead of running ``setup()``. This inevitably brings challenges due to a different syntax. Here we provide a quick guide to understanding how ``setup.cfg`` is parsed by ``setuptool`` to ease the pain of transition. .. _packaging-resources: Resources on Python packaging ============================= Packaging in Python is hard. Here we provide a list of links for those that want to learn more. PK`_. When doing test-driven development, or running automated builds that need testing before they are deployed for downloading or use, it's often useful to be able to run a project's unit tests without actually deploying the project anywhere, even using the ``develop`` command. The ``test`` command runs a project's unit tests without actually deploying it, by temporarily putting the project's source on ``sys.path``, after first running ``build_ext -i`` and ``egg_info`` to ensure that any C extensions and project metadata are up-to-date. To use this command, your project's tests must be wrapped in a ``unittest`` test suite by either a function, a ``TestCase`` class or method, or a module or package containing ``TestCase`` classes. If the named suite is a module, and the module has an ``additional_tests()`` function, it is called and the result (which must be a ``unittest.TestSuite``) is added to the tests to be run. If the named suite is a package, any submodules and subpackages are recursively added to the overall test suite. (Note: if your project specifies a ``test_loader``, the rules for processing the chosen ``test_suite`` may differ; see the :ref:`test_loader ` documentation for more details.) Note that many test systems including ``doctest`` support wrapping their non-``unittest`` tests in ``TestSuite`` objects. So, if you are using a test package that does not support this, we suggest you encourage its developers to implement test suite support, as this is a convenient and standard way to aggregate a collection of tests to be run under a common test harness. By default, tests will be run in the "verbose" mode of the ``unittest`` package's text test runner, but you can get the "quiet" mode (just dots) if you supply the ``-q`` or ``--quiet`` option, either as a global option to the setup script (e.g. ``setup.py -q test``) or as an option for the ``test`` command itself (e.g. ``setup.py test -q``). There is one other option available: ``--test-suite=NAME, -s NAME`` Specify the test suite (or module, class, or method) to be run (e.g. ``some_module.test_suite``). The default for this option can be set by giving a ``test_suite`` argument to the ``setup()`` function, e.g.:: setup( # ... test_suite="my_package.tests.test_all" ) If you did not set a ``test_suite`` in your ``setup()`` call, and do not provide a ``--test-suite`` option, an error will occur. New in 41.5.0: Deprecated the test command. .. _upload: ``upload`` - Upload source and/or egg distributions to PyPI =========================================================== The ``upload`` command was deprecated in version 40.0 and removed in version 42.0. Use `twine `_ instead. For more information on the current best practices in uploading your packages to PyPI, see the Python Packaging User Guide's "Packaging Python Projects" tutorial specifically the section on `uploading the distribution archives `_. PK`_ module: .. code-block:: bash python -m timmins Adding a console script entry point allows the package to define a user-friendly name for installers of the package to execute. Installers like pip will create wrapper scripts to execute a function. In the above example, to create a command ``hello-world`` that invokes ``timmins.hello_world``, add a console script entry point to ``setup.cfg``: .. code-block:: ini [options.entry_points] console_scripts = hello-world = timmins:hello_world After installing the package, a user may invoke that function by simply calling ``hello-world`` on the command line. The syntax for entry points is specified as follows: .. code-block:: ini = [.[.]][:.] where ``name`` is the name for the script you want to create, the left hand side of ``:`` is the module that contains your function and the right hand side is the object you want to invoke (e.g. a function). In addition to ``console_scripts``, Setuptools supports ``gui_scripts``, which will launch a GUI application without running in a terminal window. .. _dynamic discovery of services and plugins: Advertising Behavior ==================== Console scripts are one use of the more general concept of entry points. Entry points more generally allow a packager to advertise behavior for discovery by other libraries and applications. This feature enables "plug-in"-like functionality, where one library solicits entry points and any number of other libraries provide those entry points. A good example of this plug-in behavior can be seen in `pytest plugins `_, where pytest is a test framework that allows other libraries to extend or modify its functionality through the ``pytest11`` entry point. The console scripts work similarly, where libraries advertise their commands and tools like ``pip`` create wrapper scripts that invoke those commands. For a project wishing to solicit entry points, Setuptools recommends the `importlib.metadata `_ module (part of stdlib since Python 3.8) or its backport, `importlib_metadata `_. For example, to find the console script entry points from the example above: .. code-block:: pycon >>> from importlib import metadata >>> eps = metadata.entry_points()['console_scripts'] ``eps`` is now a list of ``EntryPoint`` objects, one of which corresponds to the ``hello-world = timmins:hello_world`` defined above. Each ``EntryPoint`` contains the ``name``, ``group``, and ``value``. It also supplies a ``.load()`` method to import and load that entry point (module or object). .. code-block:: ini [options.entry_points] my.plugins = hello-world = timmins:hello_world Then, a different project wishing to load 'my.plugins' plugins could run the following routine to load (and invoke) such plugins: .. code-block:: pycon >>> from importlib import metadata >>> eps = metadata.entry_points()['my.plugins'] >>> for ep in eps: ... plugin = ep.load() ... plugin() ... The project soliciting the entry points needs not to have any dependency or prior knowledge about the libraries implementing the entry points, and downstream users are able to compose functionality by pulling together libraries implementing the entry points. Dependency Management ===================== Some entry points may require additional dependencies to properly function. For such an entry point, declare in square brackets any number of dependency ``extras`` following the entry point definition. Such entry points will only be viable if their extras were declared and installed. See the :doc:`guide on dependencies management ` for more information on defining extra requirements. Consider from the above example: .. code-block:: ini [options.entry_points] console_scripts = hello-world = timmins:hello_world [pretty-printer] In this case, the ``hello-world`` script is only viable if the ``pretty-printer`` extra is indicated, and so a plugin host might exclude that entry point (i.e. not install a console script) if the relevant extra dependencies are not installed. PK` command for more details. (Also, before you release your project, be sure to see the section on :ref:`Specifying Your Project's Version` for more information about how pre- and post-release tags affect how version numbers are interpreted. This is important in order to make sure that dependency processing tools will know which versions of your project are newer than others.) Finally, if you are creating builds frequently, and either building them in a downloadable location or are copying them to a distribution server, you should probably also check out the :ref:`rotate ` command, which lets you automatically delete all but the N most-recently-modified distributions matching a glob pattern. So, you can use a command line like:: setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3 to build an egg whose version info includes "DEV-rNNNN" (where NNNN is the most recent Subversion revision that affected the source tree), and then delete any egg files from the distribution directory except for the three that were built most recently. If you have to manage automated builds for multiple packages, each with different tagging and rotation policies, you may also want to check out the :ref:`alias ` command, which would let each package define an alias like ``daily`` that would perform the necessary tag, build, and rotate commands. Then, a simpler script or cron job could just run ``setup.py daily`` in each project directory. (And, you could also define sitewide or per-user default versions of the ``daily`` alias, so that projects that didn't define their own would use the appropriate defaults.) Generating Source Distributions ------------------------------- ``setuptools`` enhances the distutils' default algorithm for source file selection with pluggable endpoints for looking up files to include. If you are using a revision control system, and your source distributions only need to include files that you're tracking in revision control, use a corresponding plugin instead of writing a ``MANIFEST.in`` file. See the section below on :ref:`Adding Support for Revision Control Systems` for information on plugins. If you need to include automatically generated files, or files that are kept in an unsupported revision control system, you'll need to create a ``MANIFEST.in`` file to specify any files that the default file location algorithm doesn't catch. See the distutils documentation for more information on the format of the ``MANIFEST.in`` file. But, be sure to ignore any part of the distutils documentation that deals with ``MANIFEST`` or how it's generated from ``MANIFEST.in``; setuptools shields you from these issues and doesn't work the same way in any case. Unlike the distutils, setuptools regenerates the source distribution manifest file every time you build a source distribution, and it builds it inside the project's ``.egg-info`` directory, out of the way of your main project directory. You therefore need not worry about whether it is up-to-date or not. Indeed, because setuptools' approach to determining the contents of a source distribution is so much simpler, its ``sdist`` command omits nearly all of the options that the distutils' more complex ``sdist`` process requires. For all practical purposes, you'll probably use only the ``--formats`` option, if you use any option at all. Making "Official" (Non-Snapshot) Releases ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When you make an official release, creating source or binary distributions, you will need to override the tag settings from ``setup.cfg``, so that you don't end up registering versions like ``foobar-0.7a1.dev-r34832``. This is easy to do if you are developing on the trunk and using tags or branches for your releases - just make the change to ``setup.cfg`` after branching or tagging the release, so the trunk will still produce development snapshots. Alternately, if you are not branching for releases, you can override the default version options on the command line, using something like:: setup.py egg_info -Db "" sdist bdist_egg The first part of this command (``egg_info -Db ""``) will override the configured tag information, before creating source and binary eggs. Thus, these commands will use the plain version from your ``setup.py``, without adding the build designation string. Of course, if you will be doing this a lot, you may wish to create a personal alias for this operation, e.g.:: setup.py alias -u release egg_info -Db "" You can then use it like this:: setup.py release sdist bdist_egg Or of course you can create more elaborate aliases that do all of the above. See the sections below on the :ref:`egg_info ` and :ref:`alias ` commands for more ideas. Distributing Extensions compiled with Cython -------------------------------------------- ``setuptools`` will detect at build time whether Cython is installed or not. If Cython is not found ``setuptools`` will ignore pyx files. To ensure Cython is available, include Cython in the build-requires section of your pyproject.toml:: [build-system] requires=[..., "cython"] Built with pip 10 or later, that declaration is sufficient to include Cython in the build. For broader compatibility, declare the dependency in your setup-requires of setup.cfg:: [options] setup_requires = ... cython As long as Cython is present in the build environment, ``setuptools`` includes transparent support for building Cython extensions, as long as extensions are defined using ``setuptools.Extension``. If you follow these rules, you can safely list ``.pyx`` files as the source of your ``Extension`` objects in the setup script. If it is, then ``setuptools`` will use it. Of course, for this to work, your source distributions must include the C code generated by Cython, as well as your original ``.pyx`` files. This means that you will probably want to include current ``.c`` files in your revision control system, rebuilding them whenever you check changes in for the ``.pyx`` source files. This will ensure that people tracking your project in a revision control system will be able to build it even if they don't have Cython installed, and that your source releases will be similarly usable with or without Cython. .. _Specifying Your Project's Version: Specifying Your Project's Version --------------------------------- Setuptools can work well with most versioning schemes. Over the years, setuptools has tried to closely follow the `PEP 440 `_ scheme, but it also supports legacy versions. There are, however, a few special things to watch out for, in order to ensure that setuptools and other tools can always tell what version of your package is newer than another version. Knowing these things will also help you correctly specify what versions of other projects your project depends on. A version consists of an alternating series of release numbers and pre-release or post-release tags. A release number is a series of digits punctuated by dots, such as ``2.4`` or ``0.5``. Each series of digits is treated numerically, so releases ``2.1`` and ``2.1.0`` are different ways to spell the same release number, denoting the first subrelease of release 2. But ``2.10`` is the *tenth* subrelease of release 2, and so is a different and newer release from ``2.1`` or ``2.1.0``. Leading zeros within a series of digits are also ignored, so ``2.01`` is the same as ``2.1``, and different from ``2.0.1``. Following a release number, you can have either a pre-release or post-release tag. Pre-release tags make a version be considered *older* than the version they are appended to. So, revision ``2.4`` is *newer* than revision ``2.4c1``, which in turn is newer than ``2.4b1`` or ``2.4a1``. Postrelease tags make a version be considered *newer* than the version they are appended to. So, revisions like ``2.4-1`` are newer than ``2.4``, but *older* than ``2.4.1`` (which has a higher release number). In the case of legacy versions (for example, ``2.4pl1``), they are considered older than non-legacy versions. Taking that in count, a revision ``2.4pl1`` is *older* than ``2.4`` A pre-release tag is a series of letters that are alphabetically before "final". Some examples of prerelease tags would include ``alpha``, ``beta``, ``a``, ``c``, ``dev``, and so on. You do not have to place a dot or dash before the prerelease tag if it's immediately after a number, but it's okay to do so if you prefer. Thus, ``2.4c1`` and ``2.4.c1`` and ``2.4-c1`` all represent release candidate 1 of version ``2.4``, and are treated as identical by setuptools. In addition, there are three special prerelease tags that are treated as if they were the letter ``c``: ``pre``, ``preview``, and ``rc``. So, version ``2.4rc1``, ``2.4pre1`` and ``2.4preview1`` are all the exact same version as ``2.4c1``, and are treated as identical by setuptools. A post-release tag is either a series of letters that are alphabetically greater than or equal to "final", or a dash (``-``). Post-release tags are generally used to separate patch numbers, port numbers, build numbers, revision numbers, or date stamps from the release number. For example, the version ``2.4-r1263`` might denote Subversion revision 1263 of a post-release patch of version ``2.4``. Or you might use ``2.4-20051127`` to denote a date-stamped post-release. Notice that after each pre or post-release tag, you are free to place another release number, followed again by more pre- or post-release tags. For example, ``0.6a9.dev-r41475`` could denote Subversion revision 41475 of the in- development version of the ninth alpha of release 0.6. Notice that ``dev`` is a pre-release tag, so this version is a *lower* version number than ``0.6a9``, which would be the actual ninth alpha of release 0.6. But the ``-r41475`` is a post-release tag, so this version is *newer* than ``0.6a9.dev``. For the most part, setuptools' interpretation of version numbers is intuitive, but here are a few tips that will keep you out of trouble in the corner cases: * Don't stick adjoining pre-release tags together without a dot or number between them. Version ``1.9adev`` is the ``adev`` prerelease of ``1.9``, *not* a development pre-release of ``1.9a``. Use ``.dev`` instead, as in ``1.9a.dev``, or separate the prerelease tags with a number, as in ``1.9a0dev``. ``1.9a.dev``, ``1.9a0dev``, and even ``1.9.a.dev`` are identical versions from setuptools' point of view, so you can use whatever scheme you prefer. * If you want to be certain that your chosen numbering scheme works the way you think it will, you can use the ``pkg_resources.parse_version()`` function to compare different version numbers:: >>> from pkg_resources import parse_version >>> parse_version("1.9.a.dev") == parse_version("1.9a0dev") True >>> parse_version("2.1-rc2") < parse_version("2.1") True >>> parse_version("0.6a9dev-r41475") < parse_version("0.6a9") True Once you've decided on a version numbering scheme for your project, you can have setuptools automatically tag your in-development releases with various pre- or post-release tags. See the following sections for more details: * `Tagging and "Daily Build" or "Snapshot" Releases`_ * The :ref:`egg_info ` command PK=1.2; RXP rest = docutils>=0.3; pack ==1.1, ==1.3 [options.packages.find] exclude = src.subpackage1 src.subpackage2 [options.data_files] /etc/my_package = site.d/00_default.conf host.d/00_default.conf data = data/img/logo.png, data/svg/icon.svg fonts = data/fonts/*.ttf, data/fonts/*.otf Metadata and options are set in the config sections of the same name. * Keys are the same as the keyword arguments one provides to the ``setup()`` function. * Complex values can be written comma-separated or placed one per line in *dangling* config values. The following are equivalent: .. code-block:: ini [metadata] keywords = one, two [metadata] keywords = one two * In some cases, complex values can be provided in dedicated subsections for clarity. * Some keys allow ``file:``, ``attr:``, ``find:``, and ``find_namespace:`` directives in order to cover common usecases. * Unknown keys are ignored. Using a ``src/`` layout ======================= One commonly used package configuration has all the module source code in a subdirectory (often called the ``src/`` layout), like this:: ├── src │   └── mypackage │   ├── __init__.py │   └── mod1.py ├── setup.py └── setup.cfg You can set up your ``setup.cfg`` to automatically find all your packages in the subdirectory like this: .. code-block:: ini # This example contains just the necessary options for a src-layout, set up # the rest of the file as described above. [options] package_dir= =src packages=find: [options.packages.find] where=src Specifying values ================= Some values are treated as simple strings, some allow more logic. Type names used below: * ``str`` - simple string * ``list-comma`` - dangling list or string of comma-separated values * ``list-semi`` - dangling list or string of semicolon-separated values * ``bool`` - ``True`` is 1, yes, true * ``dict`` - list-comma where keys are separated from values by ``=`` * ``section`` - values are read from a dedicated (sub)section Special directives: * ``attr:`` - Value is read from a module attribute. ``attr:`` supports callables and iterables; unsupported types are cast using ``str()``. In order to support the common case of a literal value assigned to a variable in a module containing (directly or indirectly) third-party imports, ``attr:`` first tries to read the value from the module by examining the module's AST. If that fails, ``attr:`` falls back to importing the module. * ``file:`` - Value is read from a list of files and then concatenated .. note:: The ``file:`` directive is sandboxed and won't reach anything outside the directory containing ``setup.py``. Metadata -------- .. note:: The aliases given below are supported for compatibility reasons, but their use is not advised. ============================== ================= ================= =============== ========== Key Aliases Type Minimum Version Notes ============================== ================= ================= =============== ========== name str version attr:, file:, str 39.2.0 [#meta-1]_ url home-page str download_url download-url str project_urls dict 38.3.0 author str author_email author-email str maintainer str maintainer_email maintainer-email str classifiers classifier file:, list-comma license str license_files license_file list-comma 42.0.0 description summary file:, str long_description long-description file:, str long_description_content_type str 38.6.0 keywords list-comma platforms platform list-comma provides list-comma requires list-comma obsoletes list-comma ============================== ================= ================= =============== ========== **Notes**: .. [#meta-1] The ``version`` file attribute has only been supported since 39.2.0. A version loaded using the ``file:`` directive must comply with PEP 440. It is easy to accidentally put something other than a valid version string in such a file, so validation is stricter in this case. Options ------- ======================= =================================== =============== ========= Key Type Minimum Version Notes ======================= =================================== =============== ========= zip_safe bool setup_requires list-semi 36.7.0 install_requires list-semi extras_require section [#opt-2]_ python_requires str 34.4.0 entry_points file:, section 51.0.0 scripts list-comma eager_resources list-comma dependency_links list-comma tests_require list-semi include_package_data bool packages find:, find_namespace:, list-comma [#opt-3]_ package_dir dict package_data section [#opt-1]_ exclude_package_data section namespace_packages list-comma py_modules list-comma 34.4.0 data_files dict 40.6.0 ======================= =================================== =============== ========= **Notes**: .. [#opt-1] In the ``package_data`` section, a key named with a single asterisk (``*``) refers to all packages, in lieu of the empty string used in ``setup.py``. .. [#opt-2] In the ``extras_require`` section, values are parsed as ``list-semi``. This implies that in order to include markers, they **must** be *dangling*: .. code-block:: ini [options.extras_require] rest = docutils>=0.3; pack ==1.1, ==1.3 pdf = ReportLab>=1.2 RXP importlib-metadata; python_version < "3.8" .. [#opt-3] The ``find:`` and ``find_namespace:`` directive can be further configured in a dedicated subsection ``options.packages.find``. This subsection accepts the same keys as the ``setuptools.find_packages`` and the ``setuptools.find_namespace_packages`` function: ``where``, ``include``, and ``exclude``. The ``find_namespace:`` directive is supported since Python >=3.3. Compatibility with other tools ============================== Historically, several tools explored declarative package configuration in parallel. And several of them chose to place the packaging configuration within the project's :file:`setup.cfg` file. One of the first was ``distutils2``, which development has stopped in 2013. Other include ``pbr`` which is still under active development or ``d2to1``, which was a plug-in that backports declarative configuration to ``distutils``, but has had no release since Oct. 2015. As a way to harmonize packaging tools, ``setuptools``, having held the position of *de facto* standard, has gradually integrated those features as part of its core features. Still this has lead to some confusion and feature incompatibilities: - some tools support features others don't; - some have similar features but the declarative syntax differs; The table below tries to summarize the differences. But, please, refer to each tool documentation for up-to-date information. =========================== ========== ========== ===== === feature setuptools distutils2 d2to1 pbr =========================== ========== ========== ===== === [metadata] description-file S Y Y Y [files] S Y Y Y entry_points Y Y Y S [backwards_compat] N Y Y Y =========================== ========== ========== ===== === Y: supported, N: unsupported, S: syntax differs (see :ref:`above example`). Also note that some features were only recently added to ``setuptools``. Please refer to the previous sections to find out when. PK`_). As eggs are deprecated and pip-based installs fall back to the platform-specific location for installing data files, there is no supported facility to reliably retrieve these resources. Instead, the PyPA recommends that any data files you wish to be accessible at run time be included in the package. PK`. ``extras_require`` A dictionary mapping names of "extras" (optional features of your project) to strings or lists of strings specifying what other distributions must be installed to support those features. See the section on :ref:`Declaring Dependencies` for details and examples of the format of this argument. ``python_requires`` A string corresponding to a version specifier (as defined in PEP 440) for the Python version, used to specify the Requires-Python defined in PEP 345. ``setup_requires`` A string or list of strings specifying what other distributions need to be present in order for the *setup script* to run. ``setuptools`` will attempt to obtain these (using pip if available) before processing the rest of the setup script or commands. This argument is needed if you are using distutils extensions as part of your build process; for example, extensions that process setup() arguments and turn them into EGG-INFO metadata files. (Note: projects listed in ``setup_requires`` will NOT be automatically installed on the system where the setup script is being run. They are simply downloaded to the ./.eggs directory if they're not locally available already. If you want them to be installed, as well as being available when the setup script is run, you should add them to ``install_requires`` **and** ``setup_requires``.) ``dependency_links`` A list of strings naming URLs to be searched when satisfying dependencies. These links will be used if needed to install packages specified by ``setup_requires`` or ``tests_require``. They will also be written into the egg's metadata for use during install by tools that support them. ``namespace_packages`` A list of strings naming the project's "namespace packages". A namespace package is a package that may be split across multiple project distributions. For example, Zope 3's ``zope`` package is a namespace package, because subpackages like ``zope.interface`` and ``zope.publisher`` may be distributed separately. The egg runtime system can automatically merge such subpackages into a single parent package at runtime, as long as you declare them in each project that contains any subpackages of the namespace package, and as long as the namespace package's ``__init__.py`` does not contain any code other than a namespace declaration. See the section below on :ref:`Namespace Packages` for more information. ``test_suite`` A string naming a ``unittest.TestCase`` subclass (or a package or module containing one or more of them, or a method of such a subclass), or naming a function that can be called with no arguments and returns a ``unittest.TestSuite``. If the named suite is a module, and the module has an ``additional_tests()`` function, it is called and the results are added to the tests to be run. If the named suite is a package, any submodules and subpackages are recursively added to the overall test suite. Specifying this argument enables use of the :ref:`test ` command to run the specified test suite, e.g. via ``setup.py test``. See the section on the :ref:`test ` command below for more details. New in 41.5.0: Deprecated the test command. ``tests_require`` If your project's tests need one or more additional packages besides those needed to install it, you can use this option to specify them. It should be a string or list of strings specifying what other distributions need to be present for the package's tests to run. When you run the ``test`` command, ``setuptools`` will attempt to obtain these (using pip if available). Note that these required projects will *not* be installed on the system where the tests are run, but only downloaded to the project's setup directory if they're not already installed locally. New in 41.5.0: Deprecated the test command. .. _test_loader: ``test_loader`` If you would like to use a different way of finding tests to run than what setuptools normally uses, you can specify a module name and class name in this argument. The named class must be instantiable with no arguments, and its instances must support the ``loadTestsFromNames()`` method as defined in the Python ``unittest`` module's ``TestLoader`` class. Setuptools will pass only one test "name" in the ``names`` argument: the value supplied for the ``test_suite`` argument. The loader you specify may interpret this string in any way it likes, as there are no restrictions on what may be contained in a ``test_suite`` string. The module name and class name must be separated by a ``:``. The default value of this argument is ``"setuptools.command.test:ScanningLoader"``. If you want to use the default ``unittest`` behavior, you can specify ``"unittest:TestLoader"`` as your ``test_loader`` argument instead. This will prevent automatic scanning of submodules and subpackages. The module and class you specify here may be contained in another package, as long as you use the ``tests_require`` option to ensure that the package containing the loader class is available when the ``test`` command is run. New in 41.5.0: Deprecated the test command. ``eager_resources`` A list of strings naming resources that should be extracted together, if any of them is needed, or if any C extensions included in the project are imported. This argument is only useful if the project will be installed as a zipfile, and there is a need to have all of the listed resources be extracted to the filesystem *as a unit*. Resources listed here should be "/"-separated paths, relative to the source root, so to list a resource ``foo.png`` in package ``bar.baz``, you would include the string ``bar/baz/foo.png`` in this argument. If you only need to obtain resources one at a time, or you don't have any C extensions that access other files in the project (such as data files or shared libraries), you probably do NOT need this argument and shouldn't mess with it. For more details on how this argument works, see the section below on :ref:`Automatic Resource Extraction`. ``project_urls`` An arbitrary map of URL names to hyperlinks, allowing more extensible documentation of where various resources can be found than the simple ``url`` and ``download_url`` options provide. PK`_ - SVN: `setuptools_svn `_ If you would like to create a plugin for ``setuptools`` to find files tracked by another revision control system, you can do so by adding an entry point to the ``setuptools.file_finders`` group. The entry point should be a function accepting a single directory name, and should yield all the filenames within that directory (and any subdirectories thereof) that are under revision control. For example, if you were going to create a plugin for a revision control system called "foobar", you would write a function something like this: .. code-block:: python def find_files_for_foobar(dirname): ... # loop to yield paths that start with `dirname` And you would register it in a setup script using something like this:: entry_points={ "setuptools.file_finders": [ "foobar = my_foobar_module:find_files_for_foobar", ] } Then, anyone who wants to use your plugin can simply install it, and their local setuptools installation will be able to find the necessary files. It is not necessary to distribute source control plugins with projects that simply use the other source control system, or to specify the plugins in ``setup_requires``. When you create a source distribution with the ``sdist`` command, setuptools automatically records what files were found in the ``SOURCES.txt`` file. That way, recipients of source distributions don't need to have revision control at all. However, if someone is working on a package by checking out with that system, they will need the same plugin(s) that the original author is using. A few important points for writing revision control file finders: * Your finder function MUST return relative paths, created by appending to the passed-in directory name. Absolute paths are NOT allowed, nor are relative paths that reference a parent directory of the passed-in directory. * Your finder function MUST accept an empty string as the directory name, meaning the current directory. You MUST NOT convert this to a dot; just yield relative paths. So, yielding a subdirectory named ``some/dir`` under the current directory should NOT be rendered as ``./some/dir`` or ``/somewhere/some/dir``, but *always* as simply ``some/dir`` * Your finder function SHOULD NOT raise any errors, and SHOULD deal gracefully with the absence of needed programs (i.e., ones belonging to the revision control system itself. It *may*, however, use ``distutils.log.warn()`` to inform the user of the missing program(s). PK` .. note:: the examples provided here are only to demonstrate the functionality introduced. More metadata and options arguments need to be supplied if you want to replicate them on your system. If you are completely new to setuptools, the :doc:`quickstart section ` is a good place to start. ``Setuptools`` provide powerful tools to handle package discovery, including support for namespace package. Normally, you would specify the package to be included manually in the following manner: .. tab:: setup.cfg .. code-block:: ini [options] #... packages = mypkg1 mypkg2 .. tab:: setup.py .. code-block:: python setup( # ... packages=['mypkg1', 'mypkg2'] ) This can get tiresome really quickly. To speed things up, we introduce two functions provided by setuptools: .. tab:: setup.cfg .. code-block:: ini [options] packages = find: #or packages = find_namespace: .. tab:: setup.py .. code-block:: python from setuptools import find_packages # or from setuptools import find_namespace_packages Using ``find:`` or ``find_packages`` ==================================== Let's start with the first tool. ``find:`` (``find_packages``) takes a source directory and two lists of package name patterns to exclude and include, and then return a list of ``str`` representing the packages it could find. To use it, consider the following directory .. code-block:: bash mypkg/ src/ pkg1/__init__.py pkg2/__init__.py additional/__init__.py setup.cfg #or setup.py To have your setup.cfg or setup.py to automatically include packages found in ``src`` that starts with the name ``pkg`` and not ``additional``: .. tab:: setup.cfg .. code-block:: ini [options] packages = find: package_dir = =src [options.packages.find] where = src include = pkg* exclude = additional .. tab:: setup.py .. code-block:: python setup( # ... packages=find_packages( where='src', include=['pkg*'], exclude=['additional'], ), package_dir={"": "src"} # ... ) .. _Namespace Packages: Using ``find_namespace:`` or ``find_namespace_packages`` ======================================================== ``setuptools`` provides the ``find_namespace:`` (``find_namespace_packages``) which behaves similarly to ``find:`` but works with namespace package. Before diving in, it is important to have a good understanding of what namespace packages are. Here is a quick recap: Suppose you have two packages named as follows: .. code-block:: bash /Users/Desktop/timmins/foo/__init__.py /Library/timmins/bar/__init__.py If both ``Desktop`` and ``Library`` are on your ``PYTHONPATH``, then a namespace package called ``timmins`` will be created automatically for you when you invoke the import mechanism, allowing you to accomplish the following .. code-block:: pycon >>> import timmins.foo >>> import timmins.bar as if there is only one ``timmins`` on your system. The two packages can then be distributed separately and installed individually without affecting the other one. Suppose you are packaging the ``foo`` part: .. code-block:: bash foo/ src/ timmins/foo/__init__.py setup.cfg # or setup.py and you want the ``foo`` to be automatically included, ``find:`` won't work because timmins doesn't contain ``__init__.py`` directly, instead, you have to use ``find_namespace:``: .. code-block:: ini [options] package_dir = =src packages = find_namespace: [options.packages.find] where = src When you install the zipped distribution, ``timmins.foo`` would become available to your interpreter. You can think of ``find_namespace:`` as identical to ``find:`` except it would count a directory as a package even if it doesn't contain ``__init__.py`` file directly. As a result, this creates an interesting side effect. If you organize your package like this: .. code-block:: bash foo/ timmins/ foo/__init__.py setup.cfg # or setup.py tests/ test_foo/__init__.py a naive ``find_namespace:`` would include tests as part of your package to be installed. A simple way to fix it is to adopt the aforementioned ``src`` layout. Legacy Namespace Packages ========================= The fact you can create namespace package so effortlessly above is credited to `PEP 420 `_. It use to be more cumbersome to accomplish the same result. Historically, there were two methods to create namespace packages. One is the ``pkg_resources`` style supported by ``setuptools`` and the other one being ``pkgutils`` style offered by ``pkgutils`` module in Python. Both are now considered deprecated despite the fact they still linger in many existing packages. These two differ in many subtle yet significant aspects and you can find out more on `Python packaging user guide `_ ``pkg_resource`` style namespace package ---------------------------------------- This is the method ``setuptools`` directly supports. Starting with the same layout, there are two pieces you need to add to it. First, an ``__init__.py`` file directly under your namespace package directory that contains the following: .. code-block:: python __import__("pkg_resources").declare_namespace(__name__) And the ``namespace_packages`` keyword in your ``setup.cfg`` or ``setup.py``: .. tab:: setup.cfg .. code-block:: ini [options] namespace_packages = timmins .. tab:: setup.py .. code-block:: python setup( # ... namespace_packages=['timmins'] ) And your directory should look like this .. code-block:: bash /foo/ src/ timmins/ __init__.py foo/__init__.py setup.cfg #or setup.py Repeat the same for other packages and you can achieve the same result as the previous section. ``pkgutil`` style namespace package ----------------------------------- This method is almost identical to the ``pkg_resource`` except that the ``namespace_packages`` declaration is omitted and the ``__init__.py`` file contains the following: .. code-block:: python __path__ = __import__('pkgutil').extend_path(__path__, __name__) The project layout remains the same and ``setup.cfg`` remains the same. PK`. The quickstart provides an overview of the new workflow. .. toctree:: :maxdepth: 1 quickstart package_discovery entry_point dependency_management datafiles development_mode distribution extension declarative_config keywords commands functionalities_rewrite miscellaneous PK`_ for some tips on contributing to open source projects. Although the article is not authoritative, it was authored by the maintainer of Setuptools, so reflects his opinions and will improve the likelihood of acceptance and quality of contribution. ------------------ Project Management ------------------ Setuptools is maintained primarily in GitHub at `this home `_. Setuptools is maintained under the Python Packaging Authority (PyPA) with several core contributors. All bugs for Setuptools are filed and the canonical source is maintained in GitHub. User support and discussions are done through the issue tracker (for specific) issues, through the `distutils-sig mailing list `_, or on IRC (Freenode) at #pypa. Discussions about development happen on the distutils-sig mailing list or on `Gitter `_. ----------------- Authoring Tickets ----------------- Before authoring any source code, it's often prudent to file a ticket describing the motivation behind making changes. First search to see if a ticket already exists for your issue. If not, create one. Try to think from the perspective of the reader. Explain what behavior you expected, what you got instead, and what factors might have contributed to the unexpected behavior. In GitHub, surround a block of code or traceback with the triple backtick "\`\`\`" so that it is formatted nicely. Filing a ticket provides a forum for justification, discussion, and clarification. The ticket provides a record of the purpose for the change and any hard decisions that were made. It provides a single place for others to reference when trying to understand why the software operates the way it does or why certain changes were made. Setuptools makes extensive use of hyperlinks to tickets in the changelog so that system integrators and other users can get a quick summary, but then jump to the in-depth discussion about any subject referenced. --------------------- Making a pull request --------------------- When making a pull request, please :ref:`include a short summary of the changes ` and a reference to any issue tickets that the PR is intended to solve. All PRs with code changes should include tests. All changes should include a changelog entry. .. include:: ../../changelog.d/README.rst ------------------- Auto-Merge Requests ------------------- To support running all code through CI, even lightweight contributions, the project employs Mergify to auto-merge pull requests tagged as auto-merge. Use ``hub pull-request -l auto-merge`` to create such a pull request from the command line after pushing a new branch. ------- Testing ------- The primary tests are run using tox. Make sure you have tox installed, and invoke it:: $ tox Under continuous integration, additional tests may be run. See the ``.travis.yml`` file for full details on the tests run under Travis-CI. ------------------- Semantic Versioning ------------------- Setuptools follows ``semver``. .. explain value of reflecting meaning in versions. ---------------------- Building Documentation ---------------------- Setuptools relies on the `Sphinx`_ system for building documentation. The `published documentation`_ is hosted on Read the Docs. To build the docs locally, use tox:: $ tox -e docs .. _Sphinx: http://www.sphinx-doc.org/en/master/ .. _published documentation: https://setuptools.readthedocs.io/en/latest/ --------------------- Vendored Dependencies --------------------- Setuptools has some dependencies, but due to `bootstrapping issues `_, those dependencies cannot be declared as they won't be resolved soon enough to build setuptools from source. Eventually, this limitation may be lifted as PEP 517/518 reach ubiquitous adoption, but for now, Setuptools cannot declare dependencies other than through ``setuptools/_vendor/vendored.txt`` and ``pkg_resources/_vendor/vendored.txt`` and refreshed by way of ``paver update_vendored`` (pavement.py). PK`_. The traditional ``setuptools`` way of packaging Python modules uses a ``setup()`` function within the ``setup.py`` script. Commands such as ``python setup.py bdist`` or ``python setup.py bdist_wheel`` generate a distribution bundle and ``python setup.py install`` installs the distribution. This interface makes it difficult to choose other packaging tools without an overhaul. Because ``setup.py`` scripts allowed for arbitrary execution, it proved difficult to provide a reliable user experience across environments and history. `PEP 517 `_ therefore came to rescue and specified a new standard to package and distribute Python modules. Under PEP 517: a ``pyproject.toml`` file is used to specify what program to use for generating distribution. Then, two functions provided by the program, ``build_wheel(directory: str)`` and ``build_sdist(directory: str)`` create the distribution bundle at the specified ``directory``. The program is free to use its own configuration script or extend the ``.toml`` file. Lastly, ``pip install *.whl`` or ``pip install *.tar.gz`` does the actual installation. If ``*.whl`` is available, ``pip`` will go ahead and copy the files into ``site-packages`` directory. If not, ``pip`` will look at ``pyproject.toml`` and decide what program to use to 'build from source' (the default is ``setuptools``) With this standard, switching between packaging tools becomes a lot easier. ``build_meta`` implements ``setuptools``' build system support. How to use it? -------------- Starting with a package that you want to distribute. You will need your source scripts, a ``pyproject.toml`` file and a ``setup.cfg`` file:: ~/meowpkg/ pyproject.toml setup.cfg meowpkg/__init__.py The pyproject.toml file is required to specify the build system (i.e. what is being used to package your scripts and install from source). To use it with setuptools, the content would be:: [build-system] requires = ["setuptools", "wheel"] build-backend = "setuptools.build_meta" The ``setuptools`` package implements the ``build_sdist`` command and the ``wheel`` package implements the ``build_wheel`` command; both are required to be compliant with PEP 517. Use ``setuptools``' :ref:`declarative config ` to specify the package information:: [metadata] name = meowpkg version = 0.0.1 description = a package that meows [options] packages = find: Now generate the distribution. To build the package, use `PyPA build `_:: $ pip install -q build $ python -m build And now it's done! The ``.whl`` file and ``.tar.gz`` can then be distributed and installed:: dist/ meowpkg-0.0.1.whl meowpkg-0.0.1.tar.gz $ pip install dist/meowpkg-0.0.1.whl or:: $ pip install dist/meowpkg-0.0.1.tar.gz PK`_, `importlib.metadata `_, and their backports (`resources `_, `metadata `_). Please consider using those libraries instead of pkg_resources. -------- Overview -------- The ``pkg_resources`` module provides runtime facilities for finding, introspecting, activating and using installed Python distributions. Some of the more advanced features (notably the support for parallel installation of multiple versions) rely specifically on the "egg" format (either as a zip archive or subdirectory), while others (such as plugin discovery) will work correctly so long as "egg-info" metadata directories are available for relevant distributions. Eggs are a distribution format for Python modules, similar in concept to Java's "jars" or Ruby's "gems", or the "wheel" format defined in PEP 427. However, unlike a pure distribution format, eggs can also be installed and added directly to ``sys.path`` as an import location. When installed in this way, eggs are *discoverable*, meaning that they carry metadata that unambiguously identifies their contents and dependencies. This means that an installed egg can be *automatically* found and added to ``sys.path`` in response to simple requests of the form, "get me everything I need to use docutils' PDF support". This feature allows mutually conflicting versions of a distribution to co-exist in the same Python installation, with individual applications activating the desired version at runtime by manipulating the contents of ``sys.path`` (this differs from the virtual environment approach, which involves creating isolated environments for each application). The following terms are needed in order to explain the capabilities offered by this module: project A library, framework, script, plugin, application, or collection of data or other resources, or some combination thereof. Projects are assumed to have "relatively unique" names, e.g. names registered with PyPI. release A snapshot of a project at a particular point in time, denoted by a version identifier. distribution A file or files that represent a particular release. importable distribution A file or directory that, if placed on ``sys.path``, allows Python to import any modules contained within it. pluggable distribution An importable distribution whose filename unambiguously identifies its release (i.e. project and version), and whose contents unambiguously specify what releases of other projects will satisfy its runtime requirements. extra An "extra" is an optional feature of a release, that may impose additional runtime requirements. For example, if docutils PDF support required a PDF support library to be present, docutils could define its PDF support as an "extra", and list what other project releases need to be available in order to provide it. environment A collection of distributions potentially available for importing, but not necessarily active. More than one distribution (i.e. release version) for a given project may be present in an environment. working set A collection of distributions actually available for importing, as on ``sys.path``. At most one distribution (release version) of a given project may be present in a working set, as otherwise there would be ambiguity as to what to import. eggs Eggs are pluggable distributions in one of the three formats currently supported by ``pkg_resources``. There are built eggs, development eggs, and egg links. Built eggs are directories or zipfiles whose name ends with ``.egg`` and follows the egg naming conventions, and contain an ``EGG-INFO`` subdirectory (zipped or otherwise). Development eggs are normal directories of Python code with one or more ``ProjectName.egg-info`` subdirectories. The development egg format is also used to provide a default version of a distribution that is available to software that doesn't use ``pkg_resources`` to request specific versions. Egg links are ``*.egg-link`` files that contain the name of a built or development egg, to support symbolic linking on platforms that do not have native symbolic links (or where the symbolic link support is limited). (For more information about these terms and concepts, see also this `architectural overview`_ of ``pkg_resources`` and Python Eggs in general.) .. _architectural overview: http://mail.python.org/pipermail/distutils-sig/2005-June/004652.html .. ----------------- .. Developer's Guide .. ----------------- .. This section isn't written yet. Currently planned topics include Accessing Resources Finding and Activating Package Distributions get_provider() require() WorkingSet iter_distributions Running Scripts Configuration Namespace Packages Extensible Applications and Frameworks Locating entry points Activation listeners Metadata access Extended Discovery and Installation Supporting Custom PEP 302 Implementations .. For now, please check out the extensive `API Reference`_ below. ------------- API Reference ------------- Namespace Package Support ========================= A namespace package is a package that only contains other packages and modules, with no direct contents of its own. Such packages can be split across multiple, separately-packaged distributions. They are normally used to split up large packages produced by a single organization, such as in the ``zope`` namespace package for Zope Corporation packages, and the ``peak`` namespace package for the Python Enterprise Application Kit. To create a namespace package, you list it in the ``namespace_packages`` argument to ``setup()``, in your project's ``setup.py``. (See the :ref:`setuptools documentation on namespace packages ` for more information on this.) Also, you must add a ``declare_namespace()`` call in the package's ``__init__.py`` file(s): ``declare_namespace(name)`` Declare that the dotted package name ``name`` is a "namespace package" whose contained packages and modules may be spread across multiple distributions. The named package's ``__path__`` will be extended to include the corresponding package in all distributions on ``sys.path`` that contain a package of that name. (More precisely, if an importer's ``find_module(name)`` returns a loader, then it will also be searched for the package's contents.) Whenever a Distribution's ``activate()`` method is invoked, it checks for the presence of namespace packages and updates their ``__path__`` contents accordingly. Applications that manipulate namespace packages or directly alter ``sys.path`` at runtime may also need to use this API function: ``fixup_namespace_packages(path_item)`` Declare that ``path_item`` is a newly added item on ``sys.path`` that may need to be used to update existing namespace packages. Ordinarily, this is called for you when an egg is automatically added to ``sys.path``, but if your application modifies ``sys.path`` to include locations that may contain portions of a namespace package, you will need to call this function to ensure they are added to the existing namespace packages. Although by default ``pkg_resources`` only supports namespace packages for filesystem and zip importers, you can extend its support to other "importers" compatible with PEP 302 using the ``register_namespace_handler()`` function. See the section below on `Supporting Custom Importers`_ for details. ``WorkingSet`` Objects ====================== The ``WorkingSet`` class provides access to a collection of "active" distributions. In general, there is only one meaningful ``WorkingSet`` instance: the one that represents the distributions that are currently active on ``sys.path``. This global instance is available under the name ``working_set`` in the ``pkg_resources`` module. However, specialized tools may wish to manipulate working sets that don't correspond to ``sys.path``, and therefore may wish to create other ``WorkingSet`` instances. It's important to note that the global ``working_set`` object is initialized from ``sys.path`` when ``pkg_resources`` is first imported, but is only updated if you do all future ``sys.path`` manipulation via ``pkg_resources`` APIs. If you manually modify ``sys.path``, you must invoke the appropriate methods on the ``working_set`` instance to keep it in sync. Unfortunately, Python does not provide any way to detect arbitrary changes to a list object like ``sys.path``, so ``pkg_resources`` cannot automatically update the ``working_set`` based on changes to ``sys.path``. ``WorkingSet(entries=None)`` Create a ``WorkingSet`` from an iterable of path entries. If ``entries`` is not supplied, it defaults to the value of ``sys.path`` at the time the constructor is called. Note that you will not normally construct ``WorkingSet`` instances yourself, but instead you will implicitly or explicitly use the global ``working_set`` instance. For the most part, the ``pkg_resources`` API is designed so that the ``working_set`` is used by default, such that you don't have to explicitly refer to it most of the time. All distributions available directly on ``sys.path`` will be activated automatically when ``pkg_resources`` is imported. This behaviour can cause version conflicts for applications which require non-default versions of those distributions. To handle this situation, ``pkg_resources`` checks for a ``__requires__`` attribute in the ``__main__`` module when initializing the default working set, and uses this to ensure a suitable version of each affected distribution is activated. For example:: __requires__ = ["CherryPy < 3"] # Must be set before pkg_resources import import pkg_resources Basic ``WorkingSet`` Methods ---------------------------- The following methods of ``WorkingSet`` objects are also available as module- level functions in ``pkg_resources`` that apply to the default ``working_set`` instance. Thus, you can use e.g. ``pkg_resources.require()`` as an abbreviation for ``pkg_resources.working_set.require()``: ``require(*requirements)`` Ensure that distributions matching ``requirements`` are activated ``requirements`` must be a string or a (possibly-nested) sequence thereof, specifying the distributions and versions required. The return value is a sequence of the distributions that needed to be activated to fulfill the requirements; all relevant distributions are included, even if they were already activated in this working set. For the syntax of requirement specifiers, see the section below on `Requirements Parsing`_. In general, it should not be necessary for you to call this method directly. It's intended more for use in quick-and-dirty scripting and interactive interpreter hacking than for production use. If you're creating an actual library or application, it's strongly recommended that you create a "setup.py" script using ``setuptools``, and declare all your requirements there. That way, tools like pip can automatically detect what requirements your package has, and deal with them accordingly. Note that calling ``require('SomePackage')`` will not install ``SomePackage`` if it isn't already present. If you need to do this, you should use the ``resolve()`` method instead, which allows you to pass an ``installer`` callback that will be invoked when a needed distribution can't be found on the local machine. You can then have this callback display a dialog, automatically download the needed distribution, or whatever else is appropriate for your application. See the documentation below on the ``resolve()`` method for more information, and also on the ``obtain()`` method of ``Environment`` objects. ``run_script(requires, script_name)`` Locate distribution specified by ``requires`` and run its ``script_name`` script. ``requires`` must be a string containing a requirement specifier. (See `Requirements Parsing`_ below for the syntax.) The script, if found, will be executed in *the caller's globals*. That's because this method is intended to be called from wrapper scripts that act as a proxy for the "real" scripts in a distribution. A wrapper script usually doesn't need to do anything but invoke this function with the correct arguments. If you need more control over the script execution environment, you probably want to use the ``run_script()`` method of a ``Distribution`` object's `Metadata API`_ instead. ``iter_entry_points(group, name=None)`` Yield entry point objects from ``group`` matching ``name`` If ``name`` is None, yields all entry points in ``group`` from all distributions in the working set, otherwise only ones matching both ``group`` and ``name`` are yielded. Entry points are yielded from the active distributions in the order that the distributions appear in the working set. (For the global ``working_set``, this should be the same as the order that they are listed in ``sys.path``.) Note that within the entry points advertised by an individual distribution, there is no particular ordering. Please see the section below on `Entry Points`_ for more information. ``WorkingSet`` Methods and Attributes ------------------------------------- These methods are used to query or manipulate the contents of a specific working set, so they must be explicitly invoked on a particular ``WorkingSet`` instance: ``add_entry(entry)`` Add a path item to the ``entries``, finding any distributions on it. You should use this when you add additional items to ``sys.path`` and you want the global ``working_set`` to reflect the change. This method is also called by the ``WorkingSet()`` constructor during initialization. This method uses ``find_distributions(entry,True)`` to find distributions corresponding to the path entry, and then ``add()`` them. ``entry`` is always appended to the ``entries`` attribute, even if it is already present, however. (This is because ``sys.path`` can contain the same value more than once, and the ``entries`` attribute should be able to reflect this.) ``__contains__(dist)`` True if ``dist`` is active in this ``WorkingSet``. Note that only one distribution for a given project can be active in a given ``WorkingSet``. ``__iter__()`` Yield distributions for non-duplicate projects in the working set. The yield order is the order in which the items' path entries were added to the working set. ``find(req)`` Find a distribution matching ``req`` (a ``Requirement`` instance). If there is an active distribution for the requested project, this returns it, as long as it meets the version requirement specified by ``req``. But, if there is an active distribution for the project and it does *not* meet the ``req`` requirement, ``VersionConflict`` is raised. If there is no active distribution for the requested project, ``None`` is returned. ``resolve(requirements, env=None, installer=None)`` List all distributions needed to (recursively) meet ``requirements`` ``requirements`` must be a sequence of ``Requirement`` objects. ``env``, if supplied, should be an ``Environment`` instance. If not supplied, an ``Environment`` is created from the working set's ``entries``. ``installer``, if supplied, will be invoked with each requirement that cannot be met by an already-installed distribution; it should return a ``Distribution`` or ``None``. (See the ``obtain()`` method of `Environment Objects`_, below, for more information on the ``installer`` argument.) ``add(dist, entry=None)`` Add ``dist`` to working set, associated with ``entry`` If ``entry`` is unspecified, it defaults to ``dist.location``. On exit from this routine, ``entry`` is added to the end of the working set's ``.entries`` (if it wasn't already present). ``dist`` is only added to the working set if it's for a project that doesn't already have a distribution active in the set. If it's successfully added, any callbacks registered with the ``subscribe()`` method will be called. (See `Receiving Change Notifications`_, below.) Note: ``add()`` is automatically called for you by the ``require()`` method, so you don't normally need to use this method directly. ``entries`` This attribute represents a "shadow" ``sys.path``, primarily useful for debugging. If you are experiencing import problems, you should check the global ``working_set`` object's ``entries`` against ``sys.path``, to ensure that they match. If they do not, then some part of your program is manipulating ``sys.path`` without updating the ``working_set`` accordingly. IMPORTANT NOTE: do not directly manipulate this attribute! Setting it equal to ``sys.path`` will not fix your problem, any more than putting black tape over an "engine warning" light will fix your car! If this attribute is out of sync with ``sys.path``, it's merely an *indicator* of the problem, not the cause of it. Receiving Change Notifications ------------------------------ Extensible applications and frameworks may need to receive notification when a new distribution (such as a plug-in component) has been added to a working set. This is what the ``subscribe()`` method and ``add_activation_listener()`` function are for. ``subscribe(callback)`` Invoke ``callback(distribution)`` once for each active distribution that is in the set now, or gets added later. Because the callback is invoked for already-active distributions, you do not need to loop over the working set yourself to deal with the existing items; just register the callback and be prepared for the fact that it will be called immediately by this method. Note that callbacks *must not* allow exceptions to propagate, or they will interfere with the operation of other callbacks and possibly result in an inconsistent working set state. Callbacks should use a try/except block to ignore, log, or otherwise process any errors, especially since the code that caused the callback to be invoked is unlikely to be able to handle the errors any better than the callback itself. ``pkg_resources.add_activation_listener()`` is an alternate spelling of ``pkg_resources.working_set.subscribe()``. Locating Plugins ---------------- Extensible applications will sometimes have a "plugin directory" or a set of plugin directories, from which they want to load entry points or other metadata. The ``find_plugins()`` method allows you to do this, by scanning an environment for the newest version of each project that can be safely loaded without conflicts or missing requirements. ``find_plugins(plugin_env, full_env=None, fallback=True)`` Scan ``plugin_env`` and identify which distributions could be added to this working set without version conflicts or missing requirements. Example usage:: distributions, errors = working_set.find_plugins( Environment(plugin_dirlist) ) map(working_set.add, distributions) # add plugins+libs to sys.path print "Couldn't load", errors # display errors The ``plugin_env`` should be an ``Environment`` instance that contains only distributions that are in the project's "plugin directory" or directories. The ``full_env``, if supplied, should be an ``Environment`` instance that contains all currently-available distributions. If ``full_env`` is not supplied, one is created automatically from the ``WorkingSet`` this method is called on, which will typically mean that every directory on ``sys.path`` will be scanned for distributions. This method returns a 2-tuple: (``distributions``, ``error_info``), where ``distributions`` is a list of the distributions found in ``plugin_env`` that were loadable, along with any other distributions that are needed to resolve their dependencies. ``error_info`` is a dictionary mapping unloadable plugin distributions to an exception instance describing the error that occurred. Usually this will be a ``DistributionNotFound`` or ``VersionConflict`` instance. Most applications will use this method mainly on the master ``working_set`` instance in ``pkg_resources``, and then immediately add the returned distributions to the working set so that they are available on sys.path. This will make it possible to find any entry points, and allow any other metadata tracking and hooks to be activated. The resolution algorithm used by ``find_plugins()`` is as follows. First, the project names of the distributions present in ``plugin_env`` are sorted. Then, each project's eggs are tried in descending version order (i.e., newest version first). An attempt is made to resolve each egg's dependencies. If the attempt is successful, the egg and its dependencies are added to the output list and to a temporary copy of the working set. The resolution process continues with the next project name, and no older eggs for that project are tried. If the resolution attempt fails, however, the error is added to the error dictionary. If the ``fallback`` flag is true, the next older version of the plugin is tried, until a working version is found. If false, the resolution process continues with the next plugin project name. Some applications may have stricter fallback requirements than others. For example, an application that has a database schema or persistent objects may not be able to safely downgrade a version of a package. Others may want to ensure that a new plugin configuration is either 100% good or else revert to a known-good configuration. (That is, they may wish to revert to a known configuration if the ``error_info`` return value is non-empty.) Note that this algorithm gives precedence to satisfying the dependencies of alphabetically prior project names in case of version conflicts. If two projects named "AaronsPlugin" and "ZekesPlugin" both need different versions of "TomsLibrary", then "AaronsPlugin" will win and "ZekesPlugin" will be disabled due to version conflict. ``Environment`` Objects ======================= An "environment" is a collection of ``Distribution`` objects, usually ones that are present and potentially importable on the current platform. ``Environment`` objects are used by ``pkg_resources`` to index available distributions during dependency resolution. ``Environment(search_path=None, platform=get_supported_platform(), python=PY_MAJOR)`` Create an environment snapshot by scanning ``search_path`` for distributions compatible with ``platform`` and ``python``. ``search_path`` should be a sequence of strings such as might be used on ``sys.path``. If a ``search_path`` isn't supplied, ``sys.path`` is used. ``platform`` is an optional string specifying the name of the platform that platform-specific distributions must be compatible with. If unspecified, it defaults to the current platform. ``python`` is an optional string naming the desired version of Python (e.g. ``'2.4'``); it defaults to the currently-running version. You may explicitly set ``platform`` (and/or ``python``) to ``None`` if you wish to include *all* distributions, not just those compatible with the running platform or Python version. Note that ``search_path`` is scanned immediately for distributions, and the resulting ``Environment`` is a snapshot of the found distributions. It is not automatically updated if the system's state changes due to e.g. installation or removal of distributions. ``__getitem__(project_name)`` Returns a list of distributions for the given project name, ordered from newest to oldest version. (And highest to lowest format precedence for distributions that contain the same version of the project.) If there are no distributions for the project, returns an empty list. ``__iter__()`` Yield the unique project names of the distributions in this environment. The yielded names are always in lower case. ``add(dist)`` Add ``dist`` to the environment if it matches the platform and python version specified at creation time, and only if the distribution hasn't already been added. (i.e., adding the same distribution more than once is a no-op.) ``remove(dist)`` Remove ``dist`` from the environment. ``can_add(dist)`` Is distribution ``dist`` acceptable for this environment? If it's not compatible with the ``platform`` and ``python`` version values specified when the environment was created, a false value is returned. ``__add__(dist_or_env)`` (``+`` operator) Add a distribution or environment to an ``Environment`` instance, returning a *new* environment object that contains all the distributions previously contained by both. The new environment will have a ``platform`` and ``python`` of ``None``, meaning that it will not reject any distributions from being added to it; it will simply accept whatever is added. If you want the added items to be filtered for platform and Python version, or you want to add them to the *same* environment instance, you should use in-place addition (``+=``) instead. ``__iadd__(dist_or_env)`` (``+=`` operator) Add a distribution or environment to an ``Environment`` instance *in-place*, updating the existing instance and returning it. The ``platform`` and ``python`` filter attributes take effect, so distributions in the source that do not have a suitable platform string or Python version are silently ignored. ``best_match(req, working_set, installer=None)`` Find distribution best matching ``req`` and usable on ``working_set`` This calls the ``find(req)`` method of the ``working_set`` to see if a suitable distribution is already active. (This may raise ``VersionConflict`` if an unsuitable version of the project is already active in the specified ``working_set``.) If a suitable distribution isn't active, this method returns the newest distribution in the environment that meets the ``Requirement`` in ``req``. If no suitable distribution is found, and ``installer`` is supplied, then the result of calling the environment's ``obtain(req, installer)`` method will be returned. ``obtain(requirement, installer=None)`` Obtain a distro that matches requirement (e.g. via download). In the base ``Environment`` class, this routine just returns ``installer(requirement)``, unless ``installer`` is None, in which case None is returned instead. This method is a hook that allows subclasses to attempt other ways of obtaining a distribution before falling back to the ``installer`` argument. ``scan(search_path=None)`` Scan ``search_path`` for distributions usable on ``platform`` Any distributions found are added to the environment. ``search_path`` should be a sequence of strings such as might be used on ``sys.path``. If not supplied, ``sys.path`` is used. Only distributions conforming to the platform/python version defined at initialization are added. This method is a shortcut for using the ``find_distributions()`` function to find the distributions from each item in ``search_path``, and then calling ``add()`` to add each one to the environment. ``Requirement`` Objects ======================= ``Requirement`` objects express what versions of a project are suitable for some purpose. These objects (or their string form) are used by various ``pkg_resources`` APIs in order to find distributions that a script or distribution needs. Requirements Parsing -------------------- ``parse_requirements(s)`` Yield ``Requirement`` objects for a string or iterable of lines. Each requirement must start on a new line. See below for syntax. ``Requirement.parse(s)`` Create a ``Requirement`` object from a string or iterable of lines. A ``ValueError`` is raised if the string or lines do not contain a valid requirement specifier, or if they contain more than one specifier. (To parse multiple specifiers from a string or iterable of strings, use ``parse_requirements()`` instead.) The syntax of a requirement specifier is defined in full in PEP 508. Some examples of valid requirement specifiers:: FooProject >= 1.2 Fizzy [foo, bar] PickyThing>1.6,<=1.9,!=1.8.6 SomethingWhoseVersionIDontCareAbout SomethingWithMarker[foo]>1.0;python_version<"2.7" The project name is the only required portion of a requirement string, and if it's the only thing supplied, the requirement will accept any version of that project. The "extras" in a requirement are used to request optional features of a project, that may require additional project distributions in order to function. For example, if the hypothetical "Report-O-Rama" project offered optional PDF support, it might require an additional library in order to provide that support. Thus, a project needing Report-O-Rama's PDF features could use a requirement of ``Report-O-Rama[PDF]`` to request installation or activation of both Report-O-Rama and any libraries it needs in order to provide PDF support. For example, you could use:: pip install Report-O-Rama[PDF] To install the necessary packages using pip, or call ``pkg_resources.require('Report-O-Rama[PDF]')`` to add the necessary distributions to sys.path at runtime. The "markers" in a requirement are used to specify when a requirement should be installed -- the requirement will be installed if the marker evaluates as true in the current environment. For example, specifying ``argparse;python_version<"3.0"`` will not install in an Python 3 environment, but will in a Python 2 environment. ``Requirement`` Methods and Attributes -------------------------------------- ``__contains__(dist_or_version)`` Return true if ``dist_or_version`` fits the criteria for this requirement. If ``dist_or_version`` is a ``Distribution`` object, its project name must match the requirement's project name, and its version must meet the requirement's version criteria. If ``dist_or_version`` is a string, it is parsed using the ``parse_version()`` utility function. Otherwise, it is assumed to be an already-parsed version. The ``Requirement`` object's version specifiers (``.specs``) are internally sorted into ascending version order, and used to establish what ranges of versions are acceptable. Adjacent redundant conditions are effectively consolidated (e.g. ``">1, >2"`` produces the same results as ``">2"``, and ``"<2,<3"`` produces the same results as ``"<2"``). ``"!="`` versions are excised from the ranges they fall within. The version being tested for acceptability is then checked for membership in the resulting ranges. ``__eq__(other_requirement)`` A requirement compares equal to another requirement if they have case-insensitively equal project names, version specifiers, and "extras". (The order that extras and version specifiers are in is also ignored.) Equal requirements also have equal hashes, so that requirements can be used in sets or as dictionary keys. ``__str__()`` The string form of a ``Requirement`` is a string that, if passed to ``Requirement.parse()``, would return an equal ``Requirement`` object. ``project_name`` The name of the required project ``key`` An all-lowercase version of the ``project_name``, useful for comparison or indexing. ``extras`` A tuple of names of "extras" that this requirement calls for. (These will be all-lowercase and normalized using the ``safe_extra()`` parsing utility function, so they may not exactly equal the extras the requirement was created with.) ``specs`` A list of ``(op,version)`` tuples, sorted in ascending parsed-version order. The ``op`` in each tuple is a comparison operator, represented as a string. The ``version`` is the (unparsed) version number. ``marker`` An instance of ``packaging.markers.Marker`` that allows evaluation against the current environment. May be None if no marker specified. ``url`` The location to download the requirement from if specified. Entry Points ============ Entry points are a simple way for distributions to "advertise" Python objects (such as functions or classes) for use by other distributions. Extensible applications and frameworks can search for entry points with a particular name or group, either from a specific distribution or from all active distributions on sys.path, and then inspect or load the advertised objects at will. Entry points belong to "groups" which are named with a dotted name similar to a Python package or module name. For example, the ``setuptools`` package uses an entry point named ``distutils.commands`` in order to find commands defined by distutils extensions. ``setuptools`` treats the names of entry points defined in that group as the acceptable commands for a setup script. In a similar way, other packages can define their own entry point groups, either using dynamic names within the group (like ``distutils.commands``), or possibly using predefined names within the group. For example, a blogging framework that offers various pre- or post-publishing hooks might define an entry point group and look for entry points named "pre_process" and "post_process" within that group. To advertise an entry point, a project needs to use ``setuptools`` and provide an ``entry_points`` argument to ``setup()`` in its setup script, so that the entry points will be included in the distribution's metadata. For more details, see :ref:`Advertising Behavior`. Each project distribution can advertise at most one entry point of a given name within the same entry point group. For example, a distutils extension could advertise two different ``distutils.commands`` entry points, as long as they had different names. However, there is nothing that prevents *different* projects from advertising entry points of the same name in the same group. In some cases, this is a desirable thing, since the application or framework that uses the entry points may be calling them as hooks, or in some other way combining them. It is up to the application or framework to decide what to do if multiple distributions advertise an entry point; some possibilities include using both entry points, displaying an error message, using the first one found in sys.path order, etc. Convenience API --------------- In the following functions, the ``dist`` argument can be a ``Distribution`` instance, a ``Requirement`` instance, or a string specifying a requirement (i.e. project name, version, etc.). If the argument is a string or ``Requirement``, the specified distribution is located (and added to sys.path if not already present). An error will be raised if a matching distribution is not available. The ``group`` argument should be a string containing a dotted identifier, identifying an entry point group. If you are defining an entry point group, you should include some portion of your package's name in the group name so as to avoid collision with other packages' entry point groups. ``load_entry_point(dist, group, name)`` Load the named entry point from the specified distribution, or raise ``ImportError``. ``get_entry_info(dist, group, name)`` Return an ``EntryPoint`` object for the given ``group`` and ``name`` from the specified distribution. Returns ``None`` if the distribution has not advertised a matching entry point. ``get_entry_map(dist, group=None)`` Return the distribution's entry point map for ``group``, or the full entry map for the distribution. This function always returns a dictionary, even if the distribution advertises no entry points. If ``group`` is given, the dictionary maps entry point names to the corresponding ``EntryPoint`` object. If ``group`` is None, the dictionary maps group names to dictionaries that then map entry point names to the corresponding ``EntryPoint`` instance in that group. ``iter_entry_points(group, name=None)`` Yield entry point objects from ``group`` matching ``name``. If ``name`` is None, yields all entry points in ``group`` from all distributions in the working set on sys.path, otherwise only ones matching both ``group`` and ``name`` are yielded. Entry points are yielded from the active distributions in the order that the distributions appear on sys.path. (Within entry points for a particular distribution, however, there is no particular ordering.) (This API is actually a method of the global ``working_set`` object; see the section above on `Basic WorkingSet Methods`_ for more information.) Creating and Parsing -------------------- ``EntryPoint(name, module_name, attrs=(), extras=(), dist=None)`` Create an ``EntryPoint`` instance. ``name`` is the entry point name. The ``module_name`` is the (dotted) name of the module containing the advertised object. ``attrs`` is an optional tuple of names to look up from the module to obtain the advertised object. For example, an ``attrs`` of ``("foo","bar")`` and a ``module_name`` of ``"baz"`` would mean that the advertised object could be obtained by the following code:: import baz advertised_object = baz.foo.bar The ``extras`` are an optional tuple of "extra feature" names that the distribution needs in order to provide this entry point. When the entry point is loaded, these extra features are looked up in the ``dist`` argument to find out what other distributions may need to be activated on sys.path; see the ``load()`` method for more details. The ``extras`` argument is only meaningful if ``dist`` is specified. ``dist`` must be a ``Distribution`` instance. ``EntryPoint.parse(src, dist=None)`` (classmethod) Parse a single entry point from string ``src`` Entry point syntax follows the form:: name = some.module:some.attr [extra1,extra2] The entry name and module name are required, but the ``:attrs`` and ``[extras]`` parts are optional, as is the whitespace shown between some of the items. The ``dist`` argument is passed through to the ``EntryPoint()`` constructor, along with the other values parsed from ``src``. ``EntryPoint.parse_group(group, lines, dist=None)`` (classmethod) Parse ``lines`` (a string or sequence of lines) to create a dictionary mapping entry point names to ``EntryPoint`` objects. ``ValueError`` is raised if entry point names are duplicated, if ``group`` is not a valid entry point group name, or if there are any syntax errors. (Note: the ``group`` parameter is used only for validation and to create more informative error messages.) If ``dist`` is provided, it will be used to set the ``dist`` attribute of the created ``EntryPoint`` objects. ``EntryPoint.parse_map(data, dist=None)`` (classmethod) Parse ``data`` into a dictionary mapping group names to dictionaries mapping entry point names to ``EntryPoint`` objects. If ``data`` is a dictionary, then the keys are used as group names and the values are passed to ``parse_group()`` as the ``lines`` argument. If ``data`` is a string or sequence of lines, it is first split into .ini-style sections (using the ``split_sections()`` utility function) and the section names are used as group names. In either case, the ``dist`` argument is passed through to ``parse_group()`` so that the entry points will be linked to the specified distribution. ``EntryPoint`` Objects ---------------------- For simple introspection, ``EntryPoint`` objects have attributes that correspond exactly to the constructor argument names: ``name``, ``module_name``, ``attrs``, ``extras``, and ``dist`` are all available. In addition, the following methods are provided: ``load()`` Load the entry point, returning the advertised Python object. Effectively calls ``self.require()`` then returns ``self.resolve()``. ``require(env=None, installer=None)`` Ensure that any "extras" needed by the entry point are available on sys.path. ``UnknownExtra`` is raised if the ``EntryPoint`` has ``extras``, but no ``dist``, or if the named extras are not defined by the distribution. If ``env`` is supplied, it must be an ``Environment``, and it will be used to search for needed distributions if they are not already present on sys.path. If ``installer`` is supplied, it must be a callable taking a ``Requirement`` instance and returning a matching importable ``Distribution`` instance or None. ``resolve()`` Resolve the entry point from its module and attrs, returning the advertised Python object. Raises ``ImportError`` if it cannot be obtained. ``__str__()`` The string form of an ``EntryPoint`` is a string that could be passed to ``EntryPoint.parse()`` to produce an equivalent ``EntryPoint``. ``Distribution`` Objects ======================== ``Distribution`` objects represent collections of Python code that may or may not be importable, and may or may not have metadata and resources associated with them. Their metadata may include information such as what other projects the distribution depends on, what entry points the distribution advertises, and so on. Getting or Creating Distributions --------------------------------- Most commonly, you'll obtain ``Distribution`` objects from a ``WorkingSet`` or an ``Environment``. (See the sections above on `WorkingSet Objects`_ and `Environment Objects`_, which are containers for active distributions and available distributions, respectively.) You can also obtain ``Distribution`` objects from one of these high-level APIs: ``find_distributions(path_item, only=False)`` Yield distributions accessible via ``path_item``. If ``only`` is true, yield only distributions whose ``location`` is equal to ``path_item``. In other words, if ``only`` is true, this yields any distributions that would be importable if ``path_item`` were on ``sys.path``. If ``only`` is false, this also yields distributions that are "in" or "under" ``path_item``, but would not be importable unless their locations were also added to ``sys.path``. ``get_distribution(dist_spec)`` Return a ``Distribution`` object for a given ``Requirement`` or string. If ``dist_spec`` is already a ``Distribution`` instance, it is returned. If it is a ``Requirement`` object or a string that can be parsed into one, it is used to locate and activate a matching distribution, which is then returned. However, if you're creating specialized tools for working with distributions, or creating a new distribution format, you may also need to create ``Distribution`` objects directly, using one of the three constructors below. These constructors all take an optional ``metadata`` argument, which is used to access any resources or metadata associated with the distribution. ``metadata`` must be an object that implements the ``IResourceProvider`` interface, or None. If it is None, an ``EmptyProvider`` is used instead. ``Distribution`` objects implement both the `IResourceProvider`_ and `IMetadataProvider Methods`_ by delegating them to the ``metadata`` object. ``Distribution.from_location(location, basename, metadata=None, **kw)`` (classmethod) Create a distribution for ``location``, which must be a string such as a URL, filename, or other string that might be used on ``sys.path``. ``basename`` is a string naming the distribution, like ``Foo-1.2-py2.4.egg``. If ``basename`` ends with ``.egg``, then the project's name, version, python version and platform are extracted from the filename and used to set those properties of the created distribution. Any additional keyword arguments are forwarded to the ``Distribution()`` constructor. ``Distribution.from_filename(filename, metadata=None**kw)`` (classmethod) Create a distribution by parsing a local filename. This is a shorter way of saying ``Distribution.from_location(normalize_path(filename), os.path.basename(filename), metadata)``. In other words, it creates a distribution whose location is the normalize form of the filename, parsing name and version information from the base portion of the filename. Any additional keyword arguments are forwarded to the ``Distribution()`` constructor. ``Distribution(location,metadata,project_name,version,py_version,platform,precedence)`` Create a distribution by setting its properties. All arguments are optional and default to None, except for ``py_version`` (which defaults to the current Python version) and ``precedence`` (which defaults to ``EGG_DIST``; for more details see ``precedence`` under `Distribution Attributes`_ below). Note that it's usually easier to use the ``from_filename()`` or ``from_location()`` constructors than to specify all these arguments individually. ``Distribution`` Attributes --------------------------- location A string indicating the distribution's location. For an importable distribution, this is the string that would be added to ``sys.path`` to make it actively importable. For non-importable distributions, this is simply a filename, URL, or other way of locating the distribution. project_name A string, naming the project that this distribution is for. Project names are defined by a project's setup script, and they are used to identify projects on PyPI. When a ``Distribution`` is constructed, the ``project_name`` argument is passed through the ``safe_name()`` utility function to filter out any unacceptable characters. key ``dist.key`` is short for ``dist.project_name.lower()``. It's used for case-insensitive comparison and indexing of distributions by project name. extras A list of strings, giving the names of extra features defined by the project's dependency list (the ``extras_require`` argument specified in the project's setup script). version A string denoting what release of the project this distribution contains. When a ``Distribution`` is constructed, the ``version`` argument is passed through the ``safe_version()`` utility function to filter out any unacceptable characters. If no ``version`` is specified at construction time, then attempting to access this attribute later will cause the ``Distribution`` to try to discover its version by reading its ``PKG-INFO`` metadata file. If ``PKG-INFO`` is unavailable or can't be parsed, ``ValueError`` is raised. parsed_version The ``parsed_version`` is an object representing a "parsed" form of the distribution's ``version``. ``dist.parsed_version`` is a shortcut for calling ``parse_version(dist.version)``. It is used to compare or sort distributions by version. (See the `Parsing Utilities`_ section below for more information on the ``parse_version()`` function.) Note that accessing ``parsed_version`` may result in a ``ValueError`` if the ``Distribution`` was constructed without a ``version`` and without ``metadata`` capable of supplying the missing version info. py_version The major/minor Python version the distribution supports, as a string. For example, "2.7" or "3.4". The default is the current version of Python. platform A string representing the platform the distribution is intended for, or ``None`` if the distribution is "pure Python" and therefore cross-platform. See `Platform Utilities`_ below for more information on platform strings. precedence A distribution's ``precedence`` is used to determine the relative order of two distributions that have the same ``project_name`` and ``parsed_version``. The default precedence is ``pkg_resources.EGG_DIST``, which is the highest (i.e. most preferred) precedence. The full list of predefined precedences, from most preferred to least preferred, is: ``EGG_DIST``, ``BINARY_DIST``, ``SOURCE_DIST``, ``CHECKOUT_DIST``, and ``DEVELOP_DIST``. Normally, precedences other than ``EGG_DIST`` are used only by the ``setuptools.package_index`` module, when sorting distributions found in a package index to determine their suitability for installation. "System" and "Development" eggs (i.e., ones that use the ``.egg-info`` format), however, are automatically given a precedence of ``DEVELOP_DIST``. ``Distribution`` Methods ------------------------ ``activate(path=None)`` Ensure distribution is importable on ``path``. If ``path`` is None, ``sys.path`` is used instead. This ensures that the distribution's ``location`` is in the ``path`` list, and it also performs any necessary namespace package fixups or declarations. (That is, if the distribution contains namespace packages, this method ensures that they are declared, and that the distribution's contents for those namespace packages are merged with the contents provided by any other active distributions. See the section above on `Namespace Package Support`_ for more information.) ``pkg_resources`` adds a notification callback to the global ``working_set`` that ensures this method is called whenever a distribution is added to it. Therefore, you should not normally need to explicitly call this method. (Note that this means that namespace packages on ``sys.path`` are always imported as soon as ``pkg_resources`` is, which is another reason why namespace packages should not contain any code or import statements.) ``as_requirement()`` Return a ``Requirement`` instance that matches this distribution's project name and version. ``requires(extras=())`` List the ``Requirement`` objects that specify this distribution's dependencies. If ``extras`` is specified, it should be a sequence of names of "extras" defined by the distribution, and the list returned will then include any dependencies needed to support the named "extras". ``clone(**kw)`` Create a copy of the distribution. Any supplied keyword arguments override the corresponding argument to the ``Distribution()`` constructor, allowing you to change some of the copied distribution's attributes. ``egg_name()`` Return what this distribution's standard filename should be, not including the ".egg" extension. For example, a distribution for project "Foo" version 1.2 that runs on Python 2.3 for Windows would have an ``egg_name()`` of ``Foo-1.2-py2.3-win32``. Any dashes in the name or version are converted to underscores. (``Distribution.from_location()`` will convert them back when parsing a ".egg" file name.) ``__cmp__(other)``, ``__hash__()`` Distribution objects are hashed and compared on the basis of their parsed version and precedence, followed by their key (lowercase project name), location, Python version, and platform. The following methods are used to access ``EntryPoint`` objects advertised by the distribution. See the section above on `Entry Points`_ for more detailed information about these operations: ``get_entry_info(group, name)`` Return the ``EntryPoint`` object for ``group`` and ``name``, or None if no such point is advertised by this distribution. ``get_entry_map(group=None)`` Return the entry point map for ``group``. If ``group`` is None, return a dictionary mapping group names to entry point maps for all groups. (An entry point map is a dictionary of entry point names to ``EntryPoint`` objects.) ``load_entry_point(group, name)`` Short for ``get_entry_info(group, name).load()``. Returns the object advertised by the named entry point, or raises ``ImportError`` if the entry point isn't advertised by this distribution, or there is some other import problem. In addition to the above methods, ``Distribution`` objects also implement all of the `IResourceProvider`_ and `IMetadataProvider Methods`_ (which are documented in later sections): * ``has_metadata(name)`` * ``metadata_isdir(name)`` * ``metadata_listdir(name)`` * ``get_metadata(name)`` * ``get_metadata_lines(name)`` * ``run_script(script_name, namespace)`` * ``get_resource_filename(manager, resource_name)`` * ``get_resource_stream(manager, resource_name)`` * ``get_resource_string(manager, resource_name)`` * ``has_resource(resource_name)`` * ``resource_isdir(resource_name)`` * ``resource_listdir(resource_name)`` If the distribution was created with a ``metadata`` argument, these resource and metadata access methods are all delegated to that ``metadata`` provider. Otherwise, they are delegated to an ``EmptyProvider``, so that the distribution will appear to have no resources or metadata. This delegation approach is used so that supporting custom importers or new distribution formats can be done simply by creating an appropriate `IResourceProvider`_ implementation; see the section below on `Supporting Custom Importers`_ for more details. .. _ResourceManager API: ``ResourceManager`` API ======================= The ``ResourceManager`` class provides uniform access to package resources, whether those resources exist as files and directories or are compressed in an archive of some kind. Normally, you do not need to create or explicitly manage ``ResourceManager`` instances, as the ``pkg_resources`` module creates a global instance for you, and makes most of its methods available as top-level names in the ``pkg_resources`` module namespace. So, for example, this code actually calls the ``resource_string()`` method of the global ``ResourceManager``:: import pkg_resources my_data = pkg_resources.resource_string(__name__, "foo.dat") Thus, you can use the APIs below without needing an explicit ``ResourceManager`` instance; just import and use them as needed. Basic Resource Access --------------------- In the following methods, the ``package_or_requirement`` argument may be either a Python package/module name (e.g. ``foo.bar``) or a ``Requirement`` instance. If it is a package or module name, the named module or package must be importable (i.e., be in a distribution or directory on ``sys.path``), and the ``resource_name`` argument is interpreted relative to the named package. (Note that if a module name is used, then the resource name is relative to the package immediately containing the named module. Also, you should not use use a namespace package name, because a namespace package can be spread across multiple distributions, and is therefore ambiguous as to which distribution should be searched for the resource.) If it is a ``Requirement``, then the requirement is automatically resolved (searching the current ``Environment`` if necessary) and a matching distribution is added to the ``WorkingSet`` and ``sys.path`` if one was not already present. (Unless the ``Requirement`` can't be satisfied, in which case an exception is raised.) The ``resource_name`` argument is then interpreted relative to the root of the identified distribution; i.e. its first path segment will be treated as a peer of the top-level modules or packages in the distribution. Note that resource names must be ``/``-separated paths rooted at the package, cannot contain relative names like ``".."``, and cannot be absolute. Do *not* use ``os.path`` routines to manipulate resource paths, as they are *not* filesystem paths. ``resource_exists(package_or_requirement, resource_name)`` Does the named resource exist? Return ``True`` or ``False`` accordingly. ``resource_stream(package_or_requirement, resource_name)`` Return a readable file-like object for the specified resource; it may be an actual file, a ``StringIO``, or some similar object. The stream is in "binary mode", in the sense that whatever bytes are in the resource will be read as-is. ``resource_string(package_or_requirement, resource_name)`` Return the specified resource as a string. The resource is read in binary fashion, such that the returned string contains exactly the bytes that are stored in the resource. ``resource_isdir(package_or_requirement, resource_name)`` Is the named resource a directory? Return ``True`` or ``False`` accordingly. ``resource_listdir(package_or_requirement, resource_name)`` List the contents of the named resource directory, just like ``os.listdir`` except that it works even if the resource is in a zipfile. Note that only ``resource_exists()`` and ``resource_isdir()`` are insensitive as to the resource type. You cannot use ``resource_listdir()`` on a file resource, and you can't use ``resource_string()`` or ``resource_stream()`` on directory resources. Using an inappropriate method for the resource type may result in an exception or undefined behavior, depending on the platform and distribution format involved. Resource Extraction ------------------- ``resource_filename(package_or_requirement, resource_name)`` Sometimes, it is not sufficient to access a resource in string or stream form, and a true filesystem filename is needed. In such cases, you can use this method (or module-level function) to obtain a filename for a resource. If the resource is in an archive distribution (such as a zipped egg), it will be extracted to a cache directory, and the filename within the cache will be returned. If the named resource is a directory, then all resources within that directory (including subdirectories) are also extracted. If the named resource is a C extension or "eager resource" (see the ``setuptools`` documentation for details), then all C extensions and eager resources are extracted at the same time. Archived resources are extracted to a cache location that can be managed by the following two methods: ``set_extraction_path(path)`` Set the base path where resources will be extracted to, if needed. If you do not call this routine before any extractions take place, the path defaults to the return value of ``get_default_cache()``. (Which is based on the ``PYTHON_EGG_CACHE`` environment variable, with various platform-specific fallbacks. See that routine's documentation for more details.) Resources are extracted to subdirectories of this path based upon information given by the resource provider. You may set this to a temporary directory, but then you must call ``cleanup_resources()`` to delete the extracted files when done. There is no guarantee that ``cleanup_resources()`` will be able to remove all extracted files. (On Windows, for example, you can't unlink .pyd or .dll files that are still in use.) Note that you may not change the extraction path for a given resource manager once resources have been extracted, unless you first call ``cleanup_resources()``. ``cleanup_resources(force=False)`` Delete all extracted resource files and directories, returning a list of the file and directory names that could not be successfully removed. This function does not have any concurrency protection, so it should generally only be called when the extraction path is a temporary directory exclusive to a single process. This method is not automatically called; you must call it explicitly or register it as an ``atexit`` function if you wish to ensure cleanup of a temporary directory used for extractions. "Provider" Interface -------------------- If you are implementing an ``IResourceProvider`` and/or ``IMetadataProvider`` for a new distribution archive format, you may need to use the following ``IResourceManager`` methods to coordinate extraction of resources to the filesystem. If you're not implementing an archive format, however, you have no need to use these methods. Unlike the other methods listed above, they are *not* available as top-level functions tied to the global ``ResourceManager``; you must therefore have an explicit ``ResourceManager`` instance to use them. ``get_cache_path(archive_name, names=())`` Return absolute location in cache for ``archive_name`` and ``names`` The parent directory of the resulting path will be created if it does not already exist. ``archive_name`` should be the base filename of the enclosing egg (which may not be the name of the enclosing zipfile!), including its ".egg" extension. ``names``, if provided, should be a sequence of path name parts "under" the egg's extraction location. This method should only be called by resource providers that need to obtain an extraction location, and only for names they intend to extract, as it tracks the generated names for possible cleanup later. ``extraction_error()`` Raise an ``ExtractionError`` describing the active exception as interfering with the extraction process. You should call this if you encounter any OS errors extracting the file to the cache path; it will format the operating system exception for you, and add other information to the ``ExtractionError`` instance that may be needed by programs that want to wrap or handle extraction errors themselves. ``postprocess(tempname, filename)`` Perform any platform-specific postprocessing of ``tempname``. Resource providers should call this method ONLY after successfully extracting a compressed resource. They must NOT call it on resources that are already in the filesystem. ``tempname`` is the current (temporary) name of the file, and ``filename`` is the name it will be renamed to by the caller after this routine returns. Metadata API ============ The metadata API is used to access metadata resources bundled in a pluggable distribution. Metadata resources are virtual files or directories containing information about the distribution, such as might be used by an extensible application or framework to connect "plugins". Like other kinds of resources, metadata resource names are ``/``-separated and should not contain ``..`` or begin with a ``/``. You should not use ``os.path`` routines to manipulate resource paths. The metadata API is provided by objects implementing the ``IMetadataProvider`` or ``IResourceProvider`` interfaces. ``Distribution`` objects implement this interface, as do objects returned by the ``get_provider()`` function: ``get_provider(package_or_requirement)`` If a package name is supplied, return an ``IResourceProvider`` for the package. If a ``Requirement`` is supplied, resolve it by returning a ``Distribution`` from the current working set (searching the current ``Environment`` if necessary and adding the newly found ``Distribution`` to the working set). If the named package can't be imported, or the ``Requirement`` can't be satisfied, an exception is raised. NOTE: if you use a package name rather than a ``Requirement``, the object you get back may not be a pluggable distribution, depending on the method by which the package was installed. In particular, "development" packages and "single-version externally-managed" packages do not have any way to map from a package name to the corresponding project's metadata. Do not write code that passes a package name to ``get_provider()`` and then tries to retrieve project metadata from the returned object. It may appear to work when the named package is in an ``.egg`` file or directory, but it will fail in other installation scenarios. If you want project metadata, you need to ask for a *project*, not a package. ``IMetadataProvider`` Methods ----------------------------- The methods provided by objects (such as ``Distribution`` instances) that implement the ``IMetadataProvider`` or ``IResourceProvider`` interfaces are: ``has_metadata(name)`` Does the named metadata resource exist? ``metadata_isdir(name)`` Is the named metadata resource a directory? ``metadata_listdir(name)`` List of metadata names in the directory (like ``os.listdir()``) ``get_metadata(name)`` Return the named metadata resource as a string. The data is read in binary mode; i.e., the exact bytes of the resource file are returned. ``get_metadata_lines(name)`` Yield named metadata resource as list of non-blank non-comment lines. This is short for calling ``yield_lines(provider.get_metadata(name))``. See the section on `yield_lines()`_ below for more information on the syntax it recognizes. ``run_script(script_name, namespace)`` Execute the named script in the supplied namespace dictionary. Raises ``ResolutionError`` if there is no script by that name in the ``scripts`` metadata directory. ``namespace`` should be a Python dictionary, usually a module dictionary if the script is being run as a module. Exceptions ========== ``pkg_resources`` provides a simple exception hierarchy for problems that may occur when processing requests to locate and activate packages:: ResolutionError DistributionNotFound VersionConflict UnknownExtra ExtractionError ``ResolutionError`` This class is used as a base class for the other three exceptions, so that you can catch all of them with a single "except" clause. It is also raised directly for miscellaneous requirement-resolution problems like trying to run a script that doesn't exist in the distribution it was requested from. ``DistributionNotFound`` A distribution needed to fulfill a requirement could not be found. ``VersionConflict`` The requested version of a project conflicts with an already-activated version of the same project. ``UnknownExtra`` One of the "extras" requested was not recognized by the distribution it was requested from. ``ExtractionError`` A problem occurred extracting a resource to the Python Egg cache. The following attributes are available on instances of this exception: manager The resource manager that raised this exception cache_path The base directory for resource extraction original_error The exception instance that caused extraction to fail Supporting Custom Importers =========================== By default, ``pkg_resources`` supports normal filesystem imports, and ``zipimport`` importers. If you wish to use the ``pkg_resources`` features with other (PEP 302-compatible) importers or module loaders, you may need to register various handlers and support functions using these APIs: ``register_finder(importer_type, distribution_finder)`` Register ``distribution_finder`` to find distributions in ``sys.path`` items. ``importer_type`` is the type or class of a PEP 302 "Importer" (``sys.path`` item handler), and ``distribution_finder`` is a callable that, when passed a path item, the importer instance, and an ``only`` flag, yields ``Distribution`` instances found under that path item. (The ``only`` flag, if true, means the finder should yield only ``Distribution`` objects whose ``location`` is equal to the path item provided.) See the source of the ``pkg_resources.find_on_path`` function for an example finder function. ``register_loader_type(loader_type, provider_factory)`` Register ``provider_factory`` to make ``IResourceProvider`` objects for ``loader_type``. ``loader_type`` is the type or class of a PEP 302 ``module.__loader__``, and ``provider_factory`` is a function that, when passed a module object, returns an `IResourceProvider`_ for that module, allowing it to be used with the `ResourceManager API`_. ``register_namespace_handler(importer_type, namespace_handler)`` Register ``namespace_handler`` to declare namespace packages for the given ``importer_type``. ``importer_type`` is the type or class of a PEP 302 "importer" (sys.path item handler), and ``namespace_handler`` is a callable with a signature like this:: def namespace_handler(importer, path_entry, moduleName, module): # return a path_entry to use for child packages Namespace handlers are only called if the relevant importer object has already agreed that it can handle the relevant path item. The handler should only return a subpath if the module ``__path__`` does not already contain an equivalent subpath. Otherwise, it should return None. For an example namespace handler, see the source of the ``pkg_resources.file_ns_handler`` function, which is used for both zipfile importing and regular importing. IResourceProvider ----------------- ``IResourceProvider`` is an abstract class that documents what methods are required of objects returned by a ``provider_factory`` registered with ``register_loader_type()``. ``IResourceProvider`` is a subclass of ``IMetadataProvider``, so objects that implement this interface must also implement all of the `IMetadataProvider Methods`_ as well as the methods shown here. The ``manager`` argument to the methods below must be an object that supports the full `ResourceManager API`_ documented above. ``get_resource_filename(manager, resource_name)`` Return a true filesystem path for ``resource_name``, coordinating the extraction with ``manager``, if the resource must be unpacked to the filesystem. ``get_resource_stream(manager, resource_name)`` Return a readable file-like object for ``resource_name``. ``get_resource_string(manager, resource_name)`` Return a string containing the contents of ``resource_name``. ``has_resource(resource_name)`` Does the package contain the named resource? ``resource_isdir(resource_name)`` Is the named resource a directory? Return a false value if the resource does not exist or is not a directory. ``resource_listdir(resource_name)`` Return a list of the contents of the resource directory, ala ``os.listdir()``. Requesting the contents of a non-existent directory may raise an exception. Note, by the way, that your provider classes need not (and should not) subclass ``IResourceProvider`` or ``IMetadataProvider``! These classes exist solely for documentation purposes and do not provide any useful implementation code. You may instead wish to subclass one of the `built-in resource providers`_. Built-in Resource Providers --------------------------- ``pkg_resources`` includes several provider classes that are automatically used where appropriate. Their inheritance tree looks like this:: NullProvider EggProvider DefaultProvider PathMetadata ZipProvider EggMetadata EmptyProvider FileMetadata ``NullProvider`` This provider class is just an abstract base that provides for common provider behaviors (such as running scripts), given a definition for just a few abstract methods. ``EggProvider`` This provider class adds in some egg-specific features that are common to zipped and unzipped eggs. ``DefaultProvider`` This provider class is used for unpacked eggs and "plain old Python" filesystem modules. ``ZipProvider`` This provider class is used for all zipped modules, whether they are eggs or not. ``EmptyProvider`` This provider class always returns answers consistent with a provider that has no metadata or resources. ``Distribution`` objects created without a ``metadata`` argument use an instance of this provider class instead. Since all ``EmptyProvider`` instances are equivalent, there is no need to have more than one instance. ``pkg_resources`` therefore creates a global instance of this class under the name ``empty_provider``, and you may use it if you have need of an ``EmptyProvider`` instance. ``PathMetadata(path, egg_info)`` Create an ``IResourceProvider`` for a filesystem-based distribution, where ``path`` is the filesystem location of the importable modules, and ``egg_info`` is the filesystem location of the distribution's metadata directory. ``egg_info`` should usually be the ``EGG-INFO`` subdirectory of ``path`` for an "unpacked egg", and a ``ProjectName.egg-info`` subdirectory of ``path`` for a "development egg". However, other uses are possible for custom purposes. ``EggMetadata(zipimporter)`` Create an ``IResourceProvider`` for a zipfile-based distribution. The ``zipimporter`` should be a ``zipimport.zipimporter`` instance, and may represent a "basket" (a zipfile containing multiple ".egg" subdirectories) a specific egg *within* a basket, or a zipfile egg (where the zipfile itself is a ".egg"). It can also be a combination, such as a zipfile egg that also contains other eggs. ``FileMetadata(path_to_pkg_info)`` Create an ``IResourceProvider`` that provides exactly one metadata resource: ``PKG-INFO``. The supplied path should be a distutils PKG-INFO file. This is basically the same as an ``EmptyProvider``, except that requests for ``PKG-INFO`` will be answered using the contents of the designated file. (This provider is used to wrap ``.egg-info`` files installed by vendor-supplied system packages.) Utility Functions ================= In addition to its high-level APIs, ``pkg_resources`` also includes several generally-useful utility routines. These routines are used to implement the high-level APIs, but can also be quite useful by themselves. Parsing Utilities ----------------- ``parse_version(version)`` Parsed a project's version string as defined by PEP 440. The returned value will be an object that represents the version. These objects may be compared to each other and sorted. The sorting algorithm is as defined by PEP 440 with the addition that any version which is not a valid PEP 440 version will be considered less than any valid PEP 440 version and the invalid versions will continue sorting using the original algorithm. .. _yield_lines(): ``yield_lines(strs)`` Yield non-empty/non-comment lines from a string/unicode or a possibly- nested sequence thereof. If ``strs`` is an instance of ``basestring``, it is split into lines, and each non-blank, non-comment line is yielded after stripping leading and trailing whitespace. (Lines whose first non-blank character is ``#`` are considered comment lines.) If ``strs`` is not an instance of ``basestring``, it is iterated over, and each item is passed recursively to ``yield_lines()``, so that an arbitrarily nested sequence of strings, or sequences of sequences of strings can be flattened out to the lines contained therein. So for example, passing a file object or a list of strings to ``yield_lines`` will both work. (Note that between each string in a sequence of strings there is assumed to be an implicit line break, so lines cannot bridge two strings in a sequence.) This routine is used extensively by ``pkg_resources`` to parse metadata and file formats of various kinds, and most other ``pkg_resources`` parsing functions that yield multiple values will use it to break up their input. However, this routine is idempotent, so calling ``yield_lines()`` on the output of another call to ``yield_lines()`` is completely harmless. ``split_sections(strs)`` Split a string (or possibly-nested iterable thereof), yielding ``(section, content)`` pairs found using an ``.ini``-like syntax. Each ``section`` is a whitespace-stripped version of the section name ("``[section]``") and each ``content`` is a list of stripped lines excluding blank lines and comment-only lines. If there are any non-blank, non-comment lines before the first section header, they're yielded in a first ``section`` of ``None``. This routine uses ``yield_lines()`` as its front end, so you can pass in anything that ``yield_lines()`` accepts, such as an open text file, string, or sequence of strings. ``ValueError`` is raised if a malformed section header is found (i.e. a line starting with ``[`` but not ending with ``]``). Note that this simplistic parser assumes that any line whose first nonblank character is ``[`` is a section heading, so it can't support .ini format variations that allow ``[`` as the first nonblank character on other lines. ``safe_name(name)`` Return a "safe" form of a project's name, suitable for use in a ``Requirement`` string, as a distribution name, or a PyPI project name. All non-alphanumeric runs are condensed to single "-" characters, such that a name like "The $$$ Tree" becomes "The-Tree". Note that if you are generating a filename from this value you should combine it with a call to ``to_filename()`` so all dashes ("-") are replaced by underscores ("_"). See ``to_filename()``. ``safe_version(version)`` This will return the normalized form of any PEP 440 version. If the version string is not PEP 440 compatible, this function behaves similar to ``safe_name()`` except that spaces in the input become dots, and dots are allowed to exist in the output. As with ``safe_name()``, if you are generating a filename from this you should replace any "-" characters in the output with underscores. ``safe_extra(extra)`` Return a "safe" form of an extra's name, suitable for use in a requirement string or a setup script's ``extras_require`` keyword. This routine is similar to ``safe_name()`` except that non-alphanumeric runs are replaced by a single underbar (``_``), and the result is lowercased. ``to_filename(name_or_version)`` Escape a name or version string so it can be used in a dash-separated filename (or ``#egg=name-version`` tag) without ambiguity. You should only pass in values that were returned by ``safe_name()`` or ``safe_version()``. Platform Utilities ------------------ ``get_build_platform()`` Return this platform's identifier string. For Windows, the return value is ``"win32"``, and for macOS it is a string of the form ``"macosx-10.4-ppc"``. All other platforms return the same uname-based string that the ``distutils.util.get_platform()`` function returns. This string is the minimum platform version required by distributions built on the local machine. (Backward compatibility note: setuptools versions prior to 0.6b1 called this function ``get_platform()``, and the function is still available under that name for backward compatibility reasons.) ``get_supported_platform()`` (New in 0.6b1) This is the similar to ``get_build_platform()``, but is the maximum platform version that the local machine supports. You will usually want to use this value as the ``provided`` argument to the ``compatible_platforms()`` function. ``compatible_platforms(provided, required)`` Return true if a distribution built on the ``provided`` platform may be used on the ``required`` platform. If either platform value is ``None``, it is considered a wildcard, and the platforms are therefore compatible. Likewise, if the platform strings are equal, they're also considered compatible, and ``True`` is returned. Currently, the only non-equal platform strings that are considered compatible are macOS platform strings with the same hardware type (e.g. ``ppc``) and major version (e.g. ``10``) with the ``provided`` platform's minor version being less than or equal to the ``required`` platform's minor version. ``get_default_cache()`` Determine the default cache location for extracting resources from zipped eggs. This routine returns the ``PYTHON_EGG_CACHE`` environment variable, if set. Otherwise, on Windows, it returns a "Python-Eggs" subdirectory of the user's "Application Data" directory. On all other systems, it returns ``os.path.expanduser("~/.python-eggs")`` if ``PYTHON_EGG_CACHE`` is not set. PEP 302 Utilities ----------------- ``get_importer(path_item)`` A deprecated alias for ``pkgutil.get_importer()`` File/Path Utilities ------------------- ``ensure_directory(path)`` Ensure that the parent directory (``os.path.dirname``) of ``path`` actually exists, using ``os.makedirs()`` if necessary. ``normalize_path(path)`` Return a "normalized" version of ``path``, such that two paths represent the same filesystem location if they have equal ``normalized_path()`` values. Specifically, this is a shortcut for calling ``os.path.realpath`` and ``os.path.normcase`` on ``path``. Unfortunately, on certain platforms (notably Cygwin and macOS) the ``normcase`` function does not accurately reflect the platform's case-sensitivity, so there is always the possibility of two apparently-different paths being equal on such platforms. History ------- 0.6c9 * Fix ``resource_listdir('')`` always returning an empty list for zipped eggs. 0.6c7 * Fix package precedence problem where single-version eggs installed in ``site-packages`` would take precedence over ``.egg`` files (or directories) installed in ``site-packages``. 0.6c6 * Fix extracted C extensions not having executable permissions under Cygwin. * Allow ``.egg-link`` files to contain relative paths. * Fix cache dir defaults on Windows when multiple environment vars are needed to construct a path. 0.6c4 * Fix "dev" versions being considered newer than release candidates. 0.6c3 * Python 2.5 compatibility fixes. 0.6c2 * Fix a problem with eggs specified directly on ``PYTHONPATH`` on case-insensitive filesystems possibly not showing up in the default working set, due to differing normalizations of ``sys.path`` entries. 0.6b3 * Fixed a duplicate path insertion problem on case-insensitive filesystems. 0.6b1 * Split ``get_platform()`` into ``get_supported_platform()`` and ``get_build_platform()`` to work around a Mac versioning problem that caused the behavior of ``compatible_platforms()`` to be platform specific. * Fix entry point parsing when a standalone module name has whitespace between it and the extras. 0.6a11 * Added ``ExtractionError`` and ``ResourceManager.extraction_error()`` so that cache permission problems get a more user-friendly explanation of the problem, and so that programs can catch and handle extraction errors if they need to. 0.6a10 * Added the ``extras`` attribute to ``Distribution``, the ``find_plugins()`` method to ``WorkingSet``, and the ``__add__()`` and ``__iadd__()`` methods to ``Environment``. * ``safe_name()`` now allows dots in project names. * There is a new ``to_filename()`` function that escapes project names and versions for safe use in constructing egg filenames from a Distribution object's metadata. * Added ``Distribution.clone()`` method, and keyword argument support to other ``Distribution`` constructors. * Added the ``DEVELOP_DIST`` precedence, and automatically assign it to eggs using ``.egg-info`` format. 0.6a9 * Don't raise an error when an invalid (unfinished) distribution is found unless absolutely necessary. Warn about skipping invalid/unfinished eggs when building an Environment. * Added support for ``.egg-info`` files or directories with version/platform information embedded in the filename, so that system packagers have the option of including ``PKG-INFO`` files to indicate the presence of a system-installed egg, without needing to use ``.egg`` directories, zipfiles, or ``.pth`` manipulation. * Changed ``parse_version()`` to remove dashes before pre-release tags, so that ``0.2-rc1`` is considered an *older* version than ``0.2``, and is equal to ``0.2rc1``. The idea that a dash *always* meant a post-release version was highly non-intuitive to setuptools users and Python developers, who seem to want to use ``-rc`` version numbers a lot. 0.6a8 * Fixed a problem with ``WorkingSet.resolve()`` that prevented version conflicts from being detected at runtime. * Improved runtime conflict warning message to identify a line in the user's program, rather than flagging the ``warn()`` call in ``pkg_resources``. * Avoid giving runtime conflict warnings for namespace packages, even if they were declared by a different package than the one currently being activated. * Fix path insertion algorithm for case-insensitive filesystems. * Fixed a problem with nested namespace packages (e.g. ``peak.util``) not being set as an attribute of their parent package. 0.6a6 * Activated distributions are now inserted in ``sys.path`` (and the working set) just before the directory that contains them, instead of at the end. This allows e.g. eggs in ``site-packages`` to override unmanaged modules in the same location, and allows eggs found earlier on ``sys.path`` to override ones found later. * When a distribution is activated, it now checks whether any contained non-namespace modules have already been imported and issues a warning if a conflicting module has already been imported. * Changed dependency processing so that it's breadth-first, allowing a depender's preferences to override those of a dependee, to prevent conflicts when a lower version is acceptable to the dependee, but not the depender. * Fixed a problem extracting zipped files on Windows, when the egg in question has had changed contents but still has the same version number. 0.6a4 * Fix a bug in ``WorkingSet.resolve()`` that was introduced in 0.6a3. 0.6a3 * Added ``safe_extra()`` parsing utility routine, and use it for Requirement, EntryPoint, and Distribution objects' extras handling. 0.6a1 * Enhanced performance of ``require()`` and related operations when all requirements are already in the working set, and enhanced performance of directory scanning for distributions. * Fixed some problems using ``pkg_resources`` w/PEP 302 loaders other than ``zipimport``, and the previously-broken "eager resource" support. * Fixed ``pkg_resources.resource_exists()`` not working correctly, along with some other resource API bugs. * Many API changes and enhancements: * Added ``EntryPoint``, ``get_entry_map``, ``load_entry_point``, and ``get_entry_info`` APIs for dynamic plugin discovery. * ``list_resources`` is now ``resource_listdir`` (and it actually works) * Resource API functions like ``resource_string()`` that accepted a package name and resource name, will now also accept a ``Requirement`` object in place of the package name (to allow access to non-package data files in an egg). * ``get_provider()`` will now accept a ``Requirement`` instance or a module name. If it is given a ``Requirement``, it will return a corresponding ``Distribution`` (by calling ``require()`` if a suitable distribution isn't already in the working set), rather than returning a metadata and resource provider for a specific module. (The difference is in how resource paths are interpreted; supplying a module name means resources path will be module-relative, rather than relative to the distribution's root.) * ``Distribution`` objects now implement the ``IResourceProvider`` and ``IMetadataProvider`` interfaces, so you don't need to reference the (no longer available) ``metadata`` attribute to get at these interfaces. * ``Distribution`` and ``Requirement`` both have a ``project_name`` attribute for the project name they refer to. (Previously these were ``name`` and ``distname`` attributes.) * The ``path`` attribute of ``Distribution`` objects is now ``location``, because it isn't necessarily a filesystem path (and hasn't been for some time now). The ``location`` of ``Distribution`` objects in the filesystem should always be normalized using ``pkg_resources.normalize_path()``; all of the setuptools' code that generates distributions from the filesystem (including ``Distribution.from_filename()``) ensure this invariant, but if you use a more generic API like ``Distribution()`` or ``Distribution.from_location()`` you should take care that you don't create a distribution with an un-normalized filesystem path. * ``Distribution`` objects now have an ``as_requirement()`` method that returns a ``Requirement`` for the distribution's project name and version. * Distribution objects no longer have an ``installed_on()`` method, and the ``install_on()`` method is now ``activate()`` (but may go away altogether soon). The ``depends()`` method has also been renamed to ``requires()``, and ``InvalidOption`` is now ``UnknownExtra``. * ``find_distributions()`` now takes an additional argument called ``only``, that tells it to only yield distributions whose location is the passed-in path. (It defaults to False, so that the default behavior is unchanged.) * ``AvailableDistributions`` is now called ``Environment``, and the ``get()``, ``__len__()``, and ``__contains__()`` methods were removed, because they weren't particularly useful. ``__getitem__()`` no longer raises ``KeyError``; it just returns an empty list if there are no distributions for the named project. * The ``resolve()`` method of ``Environment`` is now a method of ``WorkingSet`` instead, and the ``best_match()`` method now uses a working set instead of a path list as its second argument. * There is a new ``pkg_resources.add_activation_listener()`` API that lets you register a callback for notifications about distributions added to ``sys.path`` (including the distributions already on it). This is basically a hook for extensible applications and frameworks to be able to search for plugin metadata in distributions added at runtime. 0.5a13 * Fixed a bug in resource extraction from nested packages in a zipped egg. 0.5a12 * Updated extraction/cache mechanism for zipped resources to avoid inter- process and inter-thread races during extraction. The default cache location can now be set via the ``PYTHON_EGGS_CACHE`` environment variable, and the default Windows cache is now a ``Python-Eggs`` subdirectory of the current user's "Application Data" directory, if the ``PYTHON_EGGS_CACHE`` variable isn't set. 0.5a10 * Fix a problem with ``pkg_resources`` being confused by non-existent eggs on ``sys.path`` (e.g. if a user deletes an egg without removing it from the ``easy-install.pth`` file). * Fix a problem with "basket" support in ``pkg_resources``, where egg-finding never actually went inside ``.egg`` files. * Made ``pkg_resources`` import the module you request resources from, if it's not already imported. 0.5a4 * ``pkg_resources.AvailableDistributions.resolve()`` and related methods now accept an ``installer`` argument: a callable taking one argument, a ``Requirement`` instance. The callable must return a ``Distribution`` object, or ``None`` if no distribution is found. This feature is used by EasyInstall to resolve dependencies by recursively invoking itself. 0.4a4 * Fix problems with ``resource_listdir()``, ``resource_isdir()`` and resource directory extraction for zipped eggs. 0.4a3 * Fixed scripts not being able to see a ``__file__`` variable in ``__main__`` * Fixed a problem with ``resource_isdir()`` implementation that was introduced in 0.4a2. 0.4a1 * Fixed a bug in requirements processing for exact versions (i.e. ``==`` and ``!=``) when only one condition was included. * Added ``safe_name()`` and ``safe_version()`` APIs to clean up handling of arbitrary distribution names and versions found on PyPI. 0.3a4 * ``pkg_resources`` now supports resource directories, not just the resources in them. In particular, there are ``resource_listdir()`` and ``resource_isdir()`` APIs. * ``pkg_resources`` now supports "egg baskets" -- .egg zipfiles which contain multiple distributions in subdirectories whose names end with ``.egg``. Having such a "basket" in a directory on ``sys.path`` is equivalent to having the individual eggs in that directory, but the contained eggs can be individually added (or not) to ``sys.path``. Currently, however, there is no automated way to create baskets. * Namespace package manipulation is now protected by the Python import lock. 0.3a1 * Initial release. PK`_). Setuptools as a project continues to support Python 2 with bugfixes and important features on Setuptools 44.x. By design, most users will be unaffected by this change. That's because Setuptools 45 declares its supported Python versions to exclude Python 2.7, and installers such as pip 9 or later will honor this declaration and prevent installation of Setuptools 45 or later in Python 2 environments. Users that do import any portion of Setuptools 45 or later on Python 2 are directed to this documentation to provide guidance on how to work around the issues. Workarounds ----------- The best recommendation is to avoid Python 2 and move to Python 3 where possible. This project acknowledges that not all environments can drop Python 2 support, so provides other options. In less common scenarios, later versions of Setuptools can be installed on unsupported Python versions. In these environments, the installer is advised to first install ``setuptools<45`` to "pin Setuptools" to a compatible version. - When using older versions of pip (before 9.0), the ``Requires-Python`` directive is not honored and invalid versions can be installed. Users are advised first to upgrade pip and retry or to pin Setuptools. Use ``pip --version`` to determine the version of pip. - When using ``easy_install``, ``Requires-Python`` is not honored and later versions can be installed. In this case, users are advised to pin Setuptools. This applies to ``setup.py install`` invocations as well, as they use Setuptools under the hood. It's still not working ---------------------- If after trying the above steps, the Python environment still has incompatible versions of Setuptools installed, here are some things to try. 1. Uninstall and reinstall Setuptools. Run ``pip uninstall -y setuptools`` for the relevant environment. Repeat until there is no Setuptools installed. Then ``pip install setuptools``. 2. If possible, attempt to replicate the problem in a second environment (virtual machine, friend's computer, etc). If the issue is isolated to just one unique environment, first determine what is different about those environments (or reinstall/reset the failing one to defaults). 3. End users who are not themselves the maintainers for the package they are trying to install should contact the support channels for the relevant application. Please be considerate of those projects by searching for existing issues and following the latest guidance before reaching out for support. When filing an issue, be sure to give as much detail as possible to help the maintainers understand what factors led to the issue after following their recommended guidance. 4. Reach out to your local support groups. There's a good chance someone nearby has the expertise and willingness to help. 5. If all else fails, `file this template `_ with Setuptools. Please complete the whole template, providing as much detail about what factors led to the issue. Setuptools maintainers will summarily close tickets filed without any meaningful detail or engagement with the issue. PK`. ``extras_require`` A dictionary mapping names of "extras" (optional features of your project) to strings or lists of strings specifying what other distributions must be installed to support those features. See the section on :ref:`Declaring Dependencies` for details and examples of the format of this argument. ``python_requires`` A string corresponding to a version specifier (as defined in PEP 440) for the Python version, used to specify the Requires-Python defined in PEP 345. ``setup_requires`` .. warning:: Using ``setup_requires`` is discouraged in favor of `PEP-518`_ A string or list of strings specifying what other distributions need to be present in order for the *setup script* to run. ``setuptools`` will attempt to obtain these (even going so far as to download them using ``EasyInstall``) before processing the rest of the setup script or commands. This argument is needed if you are using distutils extensions as part of your build process; for example, extensions that process setup() arguments and turn them into EGG-INFO metadata files. (Note: projects listed in ``setup_requires`` will NOT be automatically installed on the system where the setup script is being run. They are simply downloaded to the ./.eggs directory if they're not locally available already. If you want them to be installed, as well as being available when the setup script is run, you should add them to ``install_requires`` **and** ``setup_requires``.) .. _PEP-518: http://www.python.org/dev/peps/pep-0518/ ``dependency_links`` .. warning:: ``dependency_links`` is deprecated. It is not supported anymore by pip. A list of strings naming URLs to be searched when satisfying dependencies. These links will be used if needed to install packages specified by ``setup_requires`` or ``tests_require``. They will also be written into the egg's metadata for use by tools like EasyInstall to use when installing an ``.egg`` file. ``namespace_packages`` A list of strings naming the project's "namespace packages". A namespace package is a package that may be split across multiple project distributions. For example, Zope 3's ``zope`` package is a namespace package, because subpackages like ``zope.interface`` and ``zope.publisher`` may be distributed separately. The egg runtime system can automatically merge such subpackages into a single parent package at runtime, as long as you declare them in each project that contains any subpackages of the namespace package, and as long as the namespace package's ``__init__.py`` does not contain any code other than a namespace declaration. See the section on :ref:`Namespace Packages` for more information. ``test_suite`` A string naming a ``unittest.TestCase`` subclass (or a package or module containing one or more of them, or a method of such a subclass), or naming a function that can be called with no arguments and returns a ``unittest.TestSuite``. If the named suite is a module, and the module has an ``additional_tests()`` function, it is called and the results are added to the tests to be run. If the named suite is a package, any submodules and subpackages are recursively added to the overall test suite. Specifying this argument enables use of the :ref:`test` command to run the specified test suite, e.g. via ``setup.py test``. See the section on the :ref:`test` command below for more details. New in 41.5.0: Deprecated the test command. ``tests_require`` If your project's tests need one or more additional packages besides those needed to install it, you can use this option to specify them. It should be a string or list of strings specifying what other distributions need to be present for the package's tests to run. When you run the ``test`` command, ``setuptools`` will attempt to obtain these (even going so far as to download them using ``EasyInstall``). Note that these required projects will *not* be installed on the system where the tests are run, but only downloaded to the project's setup directory if they're not already installed locally. New in 41.5.0: Deprecated the test command. .. _test_loader: ``test_loader`` If you would like to use a different way of finding tests to run than what setuptools normally uses, you can specify a module name and class name in this argument. The named class must be instantiable with no arguments, and its instances must support the ``loadTestsFromNames()`` method as defined in the Python ``unittest`` module's ``TestLoader`` class. Setuptools will pass only one test "name" in the ``names`` argument: the value supplied for the ``test_suite`` argument. The loader you specify may interpret this string in any way it likes, as there are no restrictions on what may be contained in a ``test_suite`` string. The module name and class name must be separated by a ``:``. The default value of this argument is ``"setuptools.command.test:ScanningLoader"``. If you want to use the default ``unittest`` behavior, you can specify ``"unittest:TestLoader"`` as your ``test_loader`` argument instead. This will prevent automatic scanning of submodules and subpackages. The module and class you specify here may be contained in another package, as long as you use the ``tests_require`` option to ensure that the package containing the loader class is available when the ``test`` command is run. New in 41.5.0: Deprecated the test command. ``eager_resources`` A list of strings naming resources that should be extracted together, if any of them is needed, or if any C extensions included in the project are imported. This argument is only useful if the project will be installed as a zipfile, and there is a need to have all of the listed resources be extracted to the filesystem *as a unit*. Resources listed here should be '/'-separated paths, relative to the source root, so to list a resource ``foo.png`` in package ``bar.baz``, you would include the string ``bar/baz/foo.png`` in this argument. If you only need to obtain resources one at a time, or you don't have any C extensions that access other files in the project (such as data files or shared libraries), you probably do NOT need this argument and shouldn't mess with it. For more details on how this argument works, see the section below on :ref:`Automatic Resource Extraction`. ``project_urls`` An arbitrary map of URL names to hyperlinks, allowing more extensible documentation of where various resources can be found than the simple ``url`` and ``download_url`` options provide. PK`_ - a single-file importable distribution format * Enhanced support for accessing data files hosted in zipped packages. * Automatically include all packages in your source tree, without listing them individually in setup.py * Automatically include all relevant files in your source distributions, without needing to create a ``MANIFEST.in`` file, and without having to force regeneration of the ``MANIFEST`` file when your source tree changes. * Automatically generate wrapper scripts or Windows (console and GUI) .exe files for any number of "main" functions in your project. (Note: this is not a py2exe replacement; the .exe files rely on the local Python installation.) * Transparent Cython support, so that your setup.py can list ``.pyx`` files and still work even when the end-user doesn't have Cython installed (as long as you include the Cython-generated C in your source distribution) * Command aliases - create project-specific, per-user, or site-wide shortcut names for commonly used commands and options * Deploy your project in "development mode", such that it's available on ``sys.path``, yet can still be edited directly from its source checkout. * Easily extend the distutils with new commands or ``setup()`` arguments, and distribute/reuse your extensions for multiple projects, without copying code. * Create extensible applications and frameworks that automatically discover extensions, using simple "entry points" declared in a project's setup script. * Full support for PEP 420 via ``find_namespace_packages()``, which is also backwards compatible to the existing ``find_packages()`` for Python >= 3.3. ----------------- Developer's Guide ----------------- The developer's guide has been updated. See the :doc:`most recent version `. TRANSITIONAL NOTE ~~~~~~~~~~~~~~~~~ Setuptools automatically calls ``declare_namespace()`` for you at runtime, but future versions may *not*. This is because the automatic declaration feature has some negative side effects, such as needing to import all namespace packages during the initialization of the ``pkg_resources`` runtime, and also the need for ``pkg_resources`` to be explicitly imported before any namespace packages work at all. In some future releases, you'll be responsible for including your own declaration lines, and the automatic declaration feature will be dropped to get rid of the negative side effects. During the remainder of the current development cycle, therefore, setuptools will warn you about missing ``declare_namespace()`` calls in your ``__init__.py`` files, and you should correct these as soon as possible before the compatibility support is removed. Namespace packages without declaration lines will not work correctly once a user has upgraded to a later version, so it's important that you make this change now in order to avoid having your code break in the field. Our apologies for the inconvenience, and thank you for your patience. setup.cfg-only projects ======================= .. versionadded:: 40.9.0 If ``setup.py`` is missing from the project directory when a :pep:`517` build is invoked, ``setuptools`` emulates a dummy ``setup.py`` file containing only a ``setuptools.setup()`` call. .. note:: :pep:`517` doesn't support editable installs so this is currently incompatible with ``pip install -e .``. This means that you can have a Python project with all build configuration specified in ``setup.cfg``, without a ``setup.py`` file, if you **can rely on** your project always being built by a :pep:`517`/:pep:`518` compatible frontend. To use this feature: * Specify build requirements and :pep:`517` build backend in ``pyproject.toml``. For example: .. code-block:: toml [build-system] requires = [ "setuptools >= 40.9.0", "wheel", ] build-backend = "setuptools.build_meta" * Use a :pep:`517` compatible build frontend, such as ``pip >= 19`` or ``build``. .. warning:: As :pep:`517` is new, support is not universal, and frontends that do support it may still have bugs. For compatibility, you may want to put a ``setup.py`` file containing only a ``setuptools.setup()`` invocation. Configuration API ================= Some automation tools may wish to access data from a configuration file. ``Setuptools`` exposes a ``read_configuration()`` function for parsing ``metadata`` and ``options`` sections into a dictionary. .. code-block:: python from setuptools.config import read_configuration conf_dict = read_configuration("/home/user/dev/package/setup.cfg") By default, ``read_configuration()`` will read only the file provided in the first argument. To include values from other configuration files which could be in various places, set the ``find_others`` keyword argument to ``True``. If you have only a configuration file but not the whole package, you can still try to get data out of it with the help of the ``ignore_option_errors`` keyword argument. When it is set to ``True``, all options with errors possibly produced by directives, such as ``attr:`` and others, will be silently ignored. As a consequence, the resulting dictionary will include no such options. Mailing List and Bug Tracker ============================ Please use the `distutils-sig mailing list`_ for questions and discussion about setuptools, and the `setuptools bug tracker`_ ONLY for issues you have confirmed via the list are actual bugs, and which you have reduced to a minimal set of steps to reproduce. .. _distutils-sig mailing list: http://mail.python.org/pipermail/distutils-sig/ .. _setuptools bug tracker: https://github.com/pypa/setuptools/ PK build_meta pkg_resources references/keywords roadmap setuptools Development guide Backward compatibility & deprecated practice Changelog .. tidelift-referral-banner:: PK`_ to make Setuptools the reference API for distutils. Since the 49.1.2 release, Setuptools includes a local, vendored copy of distutils (from late copies of CPython) that is disabled by default. To enable the use of this copy of distutils when invoking setuptools, set the enviroment variable: SETUPTOOLS_USE_DISTUTILS=local This behavior is planned to become the default. Prefer Setuptools ----------------- As Distutils is deprecated, any usage of functions or objects from distutils is similarly discouraged, and Setuptools aims to replace or deprecate all such uses. This section describes the recommended replacements. ``distutils.core.setup`` → ``setuptools.setup`` ``distutils.cmd.Command`` → ``setuptools.Command`` ``distutils.log`` → (no replacement yet) ``distutils.version.*`` → ``packaging.version.*`` If a project relies on uses of ``distutils`` that do not have a suitable replacement above, please search the `Setuptools issue tracker `_ and file a request, describing the use-case so that Setuptools' maintainers can investigate. Please provide enough detail to help the maintainers understand how distutils is used, what value it provides, and why that behavior should be supported. PK`_. (Note: please DO NOT send private email directly to the author of setuptools; it will be discarded. The mailing list is a searchable archive of previously-asked and answered questions; you should begin your research there before reporting something as a bug -- and then do so via list discussion first.) (Also, if you'd like to learn about how you can use ``setuptools`` to make your own packages work better with EasyInstall, or provide EasyInstall-like features without requiring your users to use EasyInstall directly, you'll probably want to check out the full documentation as well.) Using "Easy Install" ==================== .. _installation instructions: Installing "Easy Install" ------------------------- Please see the `setuptools PyPI page `_ for download links and basic installation instructions for each of the supported platforms. You will need at least Python 3.5 or 2.7. An ``easy_install`` script will be installed in the normal location for Python scripts on your platform. Note that the instructions on the setuptools PyPI page assume that you are are installing to Python's primary ``site-packages`` directory. If this is not the case, you should consult the section below on `Custom Installation Locations`_ before installing. (And, on Windows, you should not use the ``.exe`` installer when installing to an alternate location.) Note that ``easy_install`` normally works by downloading files from the internet. If you are behind an NTLM-based firewall that prevents Python programs from accessing the net directly, you may wish to first install and use the `APS proxy server `_, which lets you get past such firewalls in the same way that your web browser(s) do. (Alternately, if you do not wish easy_install to actually download anything, you can restrict it from doing so with the ``--allow-hosts`` option; see the sections on `restricting downloads with --allow-hosts`_ and `command-line options`_ for more details.) Troubleshooting ~~~~~~~~~~~~~~~ If EasyInstall/setuptools appears to install correctly, and you can run the ``easy_install`` command but it fails with an ``ImportError``, the most likely cause is that you installed to a location other than ``site-packages``, without taking any of the steps described in the `Custom Installation Locations`_ section below. Please see that section and follow the steps to make sure that your custom location will work correctly. Then re-install. Similarly, if you can run ``easy_install``, and it appears to be installing packages, but then you can't import them, the most likely issue is that you installed EasyInstall correctly but are using it to install packages to a non-standard location that hasn't been properly prepared. Again, see the section on `Custom Installation Locations`_ for more details. Windows Notes ~~~~~~~~~~~~~ Installing setuptools will provide an ``easy_install`` command according to the techniques described in `Executables and Launchers`_. If the ``easy_install`` command is not available after installation, that section provides details on how to configure Windows to make the commands available. Downloading and Installing a Package ------------------------------------ For basic use of ``easy_install``, you need only supply the filename or URL of a source distribution or .egg file (`Python Egg`__). __ http://peak.telecommunity.com/DevCenter/PythonEggs **Example 1**. Install a package by name, searching PyPI for the latest version, and automatically downloading, building, and installing it:: easy_install SQLObject **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: easy_install -f http://pythonpaste.org/package_index.html SQLObject **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: easy_install http://example.com/path/to/MyPackage-1.2.3.tgz **Example 4**. Install an already-downloaded .egg file:: easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg **Example 5**. Upgrade an already-installed package to the latest version listed on PyPI:: easy_install --upgrade PyProtocols **Example 6**. Install a source distribution that's already downloaded and extracted in the current directory (New in 0.5a9):: easy_install . **Example 7**. (New in 0.6a1) Find a source distribution or Subversion checkout URL for a package, and extract it or check it out to ``~/projects/sqlobject`` (the name will always be in all-lowercase), where it can be examined or edited. (The package will not be installed, but it can easily be installed with ``easy_install ~/projects/sqlobject``. See `Editing and Viewing Source Packages`_ below for more info.):: easy_install --editable --build-directory ~/projects SQLObject **Example 7**. (New in 0.6.11) Install a distribution within your home dir:: easy_install --user SQLAlchemy Easy Install accepts URLs, filenames, PyPI package names (i.e., ``distutils`` "distribution" names), and package+version specifiers. In each case, it will attempt to locate the latest available version that meets your criteria. When downloading or processing downloaded files, Easy Install recognizes distutils source distribution files with extensions of .tgz, .tar, .tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg distributions as well as ``.win32.exe`` installers built using distutils. By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` option to specify an alternative directory, or specify an alternate location using distutils configuration files. (See `Configuration Files`_, below.) By default, any scripts included with the package are installed to the running Python installation's standard script installation location. However, if you specify an installation directory via the command line or a config file, then the default directory for installing scripts will be the same as the package installation directory, to ensure that the script will have access to the installed package. You can override this using the ``-s`` or ``--script-dir`` option. Installed packages are added to an ``easy-install.pth`` file in the install directory, so that Python will always use the most-recently-installed version of the package. If you would like to be able to select which version to use at runtime, you should use the ``-m`` or ``--multi-version`` option. Upgrading a Package ------------------- You don't need to do anything special to upgrade a package: just install the new version, either by requesting a specific version, e.g.:: easy_install "SomePackage==2.0" a version greater than the one you have now:: easy_install "SomePackage>2.0" using the upgrade flag, to find the latest available version on PyPI:: easy_install --upgrade SomePackage or by using a download page, direct download URL, or package filename:: easy_install -f http://example.com/downloads ExamplePackage easy_install http://example.com/downloads/ExamplePackage-2.0-py2.4.egg easy_install my_downloads/ExamplePackage-2.0.tgz If you're using ``-m`` or ``--multi-version`` , using the ``require()`` function at runtime automatically selects the newest installed version of a package that meets your version criteria. So, installing a newer version is the only step needed to upgrade such packages. If you're installing to a directory on PYTHONPATH, or a configured "site" directory (and not using ``-m``), installing a package automatically replaces any previous version in the ``easy-install.pth`` file, so that Python will import the most-recently installed version by default. So, again, installing the newer version is the only upgrade step needed. If you haven't suppressed script installation (using ``--exclude-scripts`` or ``-x``), then the upgraded version's scripts will be installed, and they will be automatically patched to ``require()`` the corresponding version of the package, so that you can use them even if they are installed in multi-version mode. ``easy_install`` never actually deletes packages (unless you're installing a package with the same name and version number as an existing package), so if you want to get rid of older versions of a package, please see `Uninstalling Packages`_, below. Changing the Active Version --------------------------- If you've upgraded a package, but need to revert to a previously-installed version, you can do so like this:: easy_install PackageName==1.2.3 Where ``1.2.3`` is replaced by the exact version number you wish to switch to. If a package matching the requested name and version is not already installed in a directory on ``sys.path``, it will be located via PyPI and installed. If you'd like to switch to the latest installed version of ``PackageName``, you can do so like this:: easy_install PackageName This will activate the latest installed version. (Note: if you have set any ``find_links`` via distutils configuration files, those download pages will be checked for the latest available version of the package, and it will be downloaded and installed if it is newer than your current version.) Note that changing the active version of a package will install the newly active version's scripts, unless the ``--exclude-scripts`` or ``-x`` option is specified. Uninstalling Packages --------------------- If you have replaced a package with another version, then you can just delete the package(s) you don't need by deleting the PackageName-versioninfo.egg file or directory (found in the installation directory). If you want to delete the currently installed version of a package (or all versions of a package), you should first run:: easy_install -m PackageName This will ensure that Python doesn't continue to search for a package you're planning to remove. After you've done this, you can safely delete the .egg files or directories, along with any scripts you wish to remove. Managing Scripts ---------------- Whenever you install, upgrade, or change versions of a package, EasyInstall automatically installs the scripts for the selected package version, unless you tell it not to with ``-x`` or ``--exclude-scripts``. If any scripts in the script directory have the same name, they are overwritten. Thus, you do not normally need to manually delete scripts for older versions of a package, unless the newer version of the package does not include a script of the same name. However, if you are completely uninstalling a package, you may wish to manually delete its scripts. EasyInstall's default behavior means that you can normally only run scripts from one version of a package at a time. If you want to keep multiple versions of a script available, however, you can simply use the ``--multi-version`` or ``-m`` option, and rename the scripts that EasyInstall creates. This works because EasyInstall installs scripts as short code stubs that ``require()`` the matching version of the package the script came from, so renaming the script has no effect on what it executes. For example, suppose you want to use two versions of the ``rst2html`` tool provided by the `docutils `_ package. You might first install one version:: easy_install -m docutils==0.3.9 then rename the ``rst2html.py`` to ``r2h_039``, and install another version:: easy_install -m docutils==0.3.10 This will create another ``rst2html.py`` script, this one using docutils version 0.3.10 instead of 0.3.9. You now have two scripts, each using a different version of the package. (Notice that we used ``-m`` for both installations, so that Python won't lock us out of using anything but the most recently-installed version of the package.) Executables and Launchers ------------------------- On Unix systems, scripts are installed with as natural files with a "#!" header and no extension and they launch under the Python version indicated in the header. On Windows, there is no mechanism to "execute" files without extensions, so EasyInstall provides two techniques to mirror the Unix behavior. The behavior is indicated by the SETUPTOOLS_LAUNCHER environment variable, which may be "executable" (default) or "natural". Regardless of the technique used, the script(s) will be installed to a Scripts directory (by default in the Python installation directory). It is recommended for EasyInstall that you ensure this directory is in the PATH environment variable. The easiest way to ensure the Scripts directory is in the PATH is to run ``Tools\Scripts\win_add2path.py`` from the Python directory. Note that instead of changing your ``PATH`` to include the Python scripts directory, you can also retarget the installation location for scripts so they go on a directory that's already on the ``PATH``. For more information see `Command-Line Options`_ and `Configuration Files`_. During installation, pass command line options (such as ``--script-dir``) to control where scripts will be installed. Windows Executable Launcher ~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the "executable" launcher is used, EasyInstall will create a '.exe' launcher of the same name beside each installed script (including ``easy_install`` itself). These small .exe files launch the script of the same name using the Python version indicated in the '#!' header. This behavior is currently default. To force the use of executable launchers, set ``SETUPTOOLS_LAUNCHER`` to "executable". Natural Script Launcher ~~~~~~~~~~~~~~~~~~~~~~~ EasyInstall also supports deferring to an external launcher such as `pylauncher `_ for launching scripts. Enable this experimental functionality by setting the ``SETUPTOOLS_LAUNCHER`` environment variable to "natural". EasyInstall will then install scripts as simple scripts with a .pya (or .pyw) extension appended. If these extensions are associated with the pylauncher and listed in the PATHEXT environment variable, these scripts can then be invoked simply and directly just like any other executable. This behavior may become default in a future version. EasyInstall uses the .pya extension instead of simply the typical '.py' extension. This distinct extension is necessary to prevent Python from treating the scripts as importable modules (where name conflicts exist). Current releases of pylauncher do not yet associate with .pya files by default, but future versions should do so. Tips & Techniques ----------------- Multiple Python Versions ~~~~~~~~~~~~~~~~~~~~~~~~ EasyInstall installs itself under two names: ``easy_install`` and ``easy_install-N.N``, where ``N.N`` is the Python version used to install it. Thus, if you install EasyInstall for both Python 3.2 and 2.7, you can use the ``easy_install-3.2`` or ``easy_install-2.7`` scripts to install packages for the respective Python version. Setuptools also supplies easy_install as a runnable module which may be invoked using ``python -m easy_install`` for any Python with Setuptools installed. Restricting Downloads with ``--allow-hosts`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can use the ``--allow-hosts`` (``-H``) option to restrict what domains EasyInstall will look for links and downloads on. ``--allow-hosts=None`` prevents downloading altogether. You can also use wildcards, for example to restrict downloading to hosts in your own intranet. See the section below on `Command-Line Options`_ for more details on the ``--allow-hosts`` option. By default, there are no host restrictions in effect, but you can change this default by editing the appropriate `configuration files`_ and adding: .. code-block:: ini [easy_install] allow_hosts = *.myintranet.example.com,*.python.org The above example would then allow downloads only from hosts in the ``python.org`` and ``myintranet.example.com`` domains, unless overridden on the command line. Installing on Un-networked Machines ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Just copy the eggs or source packages you need to a directory on the target machine, then use the ``-f`` or ``--find-links`` option to specify that directory's location. For example:: easy_install -H None -f somedir SomePackage will attempt to install SomePackage using only eggs and source packages found in ``somedir`` and disallowing all remote access. You should of course make sure you have all of SomePackage's dependencies available in somedir. If you have another machine of the same operating system and library versions (or if the packages aren't platform-specific), you can create the directory of eggs using a command like this:: easy_install -zmaxd somedir SomePackage This will tell EasyInstall to put zipped eggs or source packages for SomePackage and all its dependencies into ``somedir``, without creating any scripts or .pth files. You can then copy the contents of ``somedir`` to the target machine. (``-z`` means zipped eggs, ``-m`` means multi-version, which prevents .pth files from being used, ``-a`` means to copy all the eggs needed, even if they're installed elsewhere on the machine, and ``-d`` indicates the directory to place the eggs in.) You can also build the eggs from local development packages that were installed with the ``setup.py develop`` command, by including the ``-l`` option, e.g.:: easy_install -zmaxld somedir SomePackage This will use locally-available source distributions to build the eggs. Packaging Others' Projects As Eggs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Need to distribute a package that isn't published in egg form? You can use EasyInstall to build eggs for a project. You'll want to use the ``--zip-ok``, ``--exclude-scripts``, and possibly ``--no-deps`` options (``-z``, ``-x`` and ``-N``, respectively). Use ``-d`` or ``--install-dir`` to specify the location where you'd like the eggs placed. By placing them in a directory that is published to the web, you can then make the eggs available for download, either in an intranet or to the internet at large. If someone distributes a package in the form of a single ``.py`` file, you can wrap it in an egg by tacking an ``#egg=name-version`` suffix on the file's URL. So, something like this:: easy_install -f "http://some.example.com/downloads/foo.py#egg=foo-1.0" foo will install the package as an egg, and this:: easy_install -zmaxd. \ -f "http://some.example.com/downloads/foo.py#egg=foo-1.0" foo will create a ``.egg`` file in the current directory. Creating your own Package Index ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In addition to local directories and the Python Package Index, EasyInstall can find download links on most any web page whose URL is given to the ``-f`` (``--find-links``) option. In the simplest case, you can simply have a web page with links to eggs or Python source packages, even an automatically generated directory listing (such as the Apache web server provides). If you are setting up an intranet site for package downloads, you may want to configure the target machines to use your download site by default, adding something like this to their `configuration files`_: .. code-block:: ini [easy_install] find_links = http://mypackages.example.com/somedir/ http://turbogears.org/download/ http://peak.telecommunity.com/dist/ As you can see, you can list multiple URLs separated by whitespace, continuing on multiple lines if necessary (as long as the subsequent lines are indented. If you are more ambitious, you can also create an entirely custom package index or PyPI mirror. See the ``--index-url`` option under `Command-Line Options`_, below, and also the section on `Package Index "API"`_. Password-Protected Sites ------------------------ If a site you want to download from is password-protected using HTTP "Basic" authentication, you can specify your credentials in the URL, like so:: http://some_userid:some_password@some.example.com/some_path/ You can do this with both index page URLs and direct download URLs. As long as any HTML pages read by easy_install use *relative* links to point to the downloads, the same user ID and password will be used to do the downloading. Using .pypirc Credentials ------------------------- In additional to supplying credentials in the URL, ``easy_install`` will also honor credentials if present in the .pypirc file. Teams maintaining a private repository of packages may already have defined access credentials for uploading packages according to the distutils documentation. ``easy_install`` will attempt to honor those if present. Refer to the distutils documentation for Python 2.5 or later for details on the syntax. Controlling Build Options ~~~~~~~~~~~~~~~~~~~~~~~~~ EasyInstall respects standard distutils `Configuration Files`_, so you can use them to configure build options for packages that it installs from source. For example, if you are on Windows using the MinGW compiler, you can configure the default compiler by putting something like this: .. code-block:: ini [build] compiler = mingw32 into the appropriate distutils configuration file. In fact, since this is just normal distutils configuration, it will affect any builds using that config file, not just ones done by EasyInstall. For example, if you add those lines to ``distutils.cfg`` in the ``distutils`` package directory, it will be the default compiler for *all* packages you build. See `Configuration Files`_ below for a list of the standard configuration file locations, and links to more documentation on using distutils configuration files. Editing and Viewing Source Packages ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sometimes a package's source distribution contains additional documentation, examples, configuration files, etc., that are not part of its actual code. If you want to be able to examine these files, you can use the ``--editable`` option to EasyInstall, and EasyInstall will look for a source distribution or Subversion URL for the package, then download and extract it or check it out as a subdirectory of the ``--build-directory`` you specify. If you then wish to install the package after editing or configuring it, you can do so by rerunning EasyInstall with that directory as the target. Note that using ``--editable`` stops EasyInstall from actually building or installing the package; it just finds, obtains, and possibly unpacks it for you. This allows you to make changes to the package if necessary, and to either install it in development mode using ``setup.py develop`` (if the package uses setuptools, that is), or by running ``easy_install projectdir`` (where ``projectdir`` is the subdirectory EasyInstall created for the downloaded package. In order to use ``--editable`` (``-e`` for short), you *must* also supply a ``--build-directory`` (``-b`` for short). The project will be placed in a subdirectory of the build directory. The subdirectory will have the same name as the project itself, but in all-lowercase. If a file or directory of that name already exists, EasyInstall will print an error message and exit. Also, when using ``--editable``, you cannot use URLs or filenames as arguments. You *must* specify project names (and optional version requirements) so that EasyInstall knows what directory name(s) to create. If you need to force EasyInstall to use a particular URL or filename, you should specify it as a ``--find-links`` item (``-f`` for short), and then also specify the project name, e.g.:: easy_install -eb ~/projects \ -fhttp://prdownloads.sourceforge.net/ctypes/ctypes-0.9.6.tar.gz?download \ ctypes==0.9.6 Dealing with Installation Conflicts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (NOTE: As of 0.6a11, this section is obsolete; it is retained here only so that people using older versions of EasyInstall can consult it. As of version 0.6a11, installation conflicts are handled automatically without deleting the old or system-installed packages, and without ignoring the issue. Instead, eggs are automatically shifted to the front of ``sys.path`` using special code added to the ``easy-install.pth`` file. So, if you are using version 0.6a11 or better of setuptools, you do not need to worry about conflicts, and the following issues do not apply to you.) EasyInstall installs distributions in a "managed" way, such that each distribution can be independently activated or deactivated on ``sys.path``. However, packages that were not installed by EasyInstall are "unmanaged", in that they usually live all in one directory and cannot be independently activated or deactivated. As a result, if you are using EasyInstall to upgrade an existing package, or to install a package with the same name as an existing package, EasyInstall will warn you of the conflict. (This is an improvement over ``setup.py install``, because the ``distutils`` just install new packages on top of old ones, possibly combining two unrelated packages or leaving behind modules that have been deleted in the newer version of the package.) EasyInstall will stop the installation if it detects a conflict between an existing, "unmanaged" package, and a module or package in any of the distributions you're installing. It will display a list of all of the existing files and directories that would need to be deleted for the new package to be able to function correctly. To proceed, you must manually delete these conflicting files and directories and re-run EasyInstall. Of course, once you've replaced all of your existing "unmanaged" packages with versions managed by EasyInstall, you won't have any more conflicts to worry about! Compressed Installation ~~~~~~~~~~~~~~~~~~~~~~~ EasyInstall tries to install packages in zipped form, if it can. Zipping packages can improve Python's overall import performance if you're not using the ``--multi-version`` option, because Python processes zipfile entries on ``sys.path`` much faster than it does directories. As of version 0.5a9, EasyInstall analyzes packages to determine whether they can be safely installed as a zipfile, and then acts on its analysis. (Previous versions would not install a package as a zipfile unless you used the ``--zip-ok`` option.) The current analysis approach is fairly conservative; it currently looks for: * Any use of the ``__file__`` or ``__path__`` variables (which should be replaced with ``pkg_resources`` API calls) * Possible use of ``inspect`` functions that expect to manipulate source files (e.g. ``inspect.getsource()``) * Top-level modules that might be scripts used with ``python -m`` (Python 2.4) If any of the above are found in the package being installed, EasyInstall will assume that the package cannot be safely run from a zipfile, and unzip it to a directory instead. You can override this analysis with the ``-zip-ok`` flag, which will tell EasyInstall to install the package as a zipfile anyway. Or, you can use the ``--always-unzip`` flag, in which case EasyInstall will always unzip, even if its analysis says the package is safe to run as a zipfile. Normally, however, it is simplest to let EasyInstall handle the determination of whether to zip or unzip, and only specify overrides when needed to work around a problem. If you find you need to override EasyInstall's guesses, you may want to contact the package author and the EasyInstall maintainers, so that they can make appropriate changes in future versions. (Note: If a package uses ``setuptools`` in its setup script, the package author has the option to declare the package safe or unsafe for zipped usage via the ``zip_safe`` argument to ``setup()``. If the package author makes such a declaration, EasyInstall believes the package's author and does not perform its own analysis. However, your command-line option, if any, will still override the package author's choice.) Reference Manual ================ Configuration Files ------------------- (New in 0.4a2) You may specify default options for EasyInstall using the standard distutils configuration files, under the command heading ``easy_install``. EasyInstall will look first for a ``setup.cfg`` file in the current directory, then a ``~/.pydistutils.cfg`` or ``$HOME\\pydistutils.cfg`` (on Unix-like OSes and Windows, respectively), and finally a ``distutils.cfg`` file in the ``distutils`` package directory. Here's a simple example: .. code-block:: ini [easy_install] # set the default location to install packages install_dir = /home/me/lib/python # Notice that indentation can be used to continue an option # value; this is especially useful for the "--find-links" # option, which tells easy_install to use download links on # these pages before consulting PyPI: # find_links = http://sqlobject.org/ http://peak.telecommunity.com/dist/ In addition to accepting configuration for its own options under ``[easy_install]``, EasyInstall also respects defaults specified for other distutils commands. For example, if you don't set an ``install_dir`` for ``[easy_install]``, but *have* set an ``install_lib`` for the ``[install]`` command, this will become EasyInstall's default installation directory. Thus, if you are already using distutils configuration files to set default install locations, build options, etc., EasyInstall will respect your existing settings until and unless you override them explicitly in an ``[easy_install]`` section. For more information, see also the current Python documentation on the `use and location of distutils configuration files `_. Notice that ``easy_install`` will use the ``setup.cfg`` from the current working directory only if it was triggered from ``setup.py`` through the ``install_requires`` option. The standalone command will not use that file. Command-Line Options -------------------- ``--zip-ok, -z`` Install all packages as zip files, even if they are marked as unsafe for running as a zipfile. This can be useful when EasyInstall's analysis of a non-setuptools package is too conservative, but keep in mind that the package may not work correctly. (Changed in 0.5a9; previously this option was required in order for zipped installation to happen at all.) ``--always-unzip, -Z`` Don't install any packages as zip files, even if the packages are marked as safe for running as a zipfile. This can be useful if a package does something unsafe, but not in a way that EasyInstall can easily detect. EasyInstall's default analysis is currently very conservative, however, so you should only use this option if you've had problems with a particular package, and *after* reporting the problem to the package's maintainer and to the EasyInstall maintainers. (Note: the ``-z/-Z`` options only affect the installation of newly-built or downloaded packages that are not already installed in the target directory; if you want to convert an existing installed version from zipped to unzipped or vice versa, you'll need to delete the existing version first, and re-run EasyInstall.) ``--multi-version, -m`` "Multi-version" mode. Specifying this option prevents ``easy_install`` from adding an ``easy-install.pth`` entry for the package being installed, and if an entry for any version the package already exists, it will be removed upon successful installation. In multi-version mode, no specific version of the package is available for importing, unless you use ``pkg_resources.require()`` to put it on ``sys.path``. This can be as simple as:: from pkg_resources import require require("SomePackage", "OtherPackage", "MyPackage") which will put the latest installed version of the specified packages on ``sys.path`` for you. (For more advanced uses, like selecting specific versions and enabling optional dependencies, see the ``pkg_resources`` API doc.) Changed in 0.6a10: this option is no longer silently enabled when installing to a non-PYTHONPATH, non-"site" directory. You must always explicitly use this option if you want it to be active. ``--upgrade, -U`` (New in 0.5a4) By default, EasyInstall only searches online if a project/version requirement can't be met by distributions already installed on sys.path or the installation directory. However, if you supply the ``--upgrade`` or ``-U`` flag, EasyInstall will always check the package index and ``--find-links`` URLs before selecting a version to install. In this way, you can force EasyInstall to use the latest available version of any package it installs (subject to any version requirements that might exclude such later versions). ``--install-dir=DIR, -d DIR`` Set the installation directory. It is up to you to ensure that this directory is on ``sys.path`` at runtime, and to use ``pkg_resources.require()`` to enable the installed package(s) that you need. (New in 0.4a2) If this option is not directly specified on the command line or in a distutils configuration file, the distutils default installation location is used. Normally, this would be the ``site-packages`` directory, but if you are using distutils configuration files, setting things like ``prefix`` or ``install_lib``, then those settings are taken into account when computing the default installation directory, as is the ``--prefix`` option. ``--script-dir=DIR, -s DIR`` Set the script installation directory. If you don't supply this option (via the command line or a configuration file), but you *have* supplied an ``--install-dir`` (via command line or config file), then this option defaults to the same directory, so that the scripts will be able to find their associated package installation. Otherwise, this setting defaults to the location where the distutils would normally install scripts, taking any distutils configuration file settings into account. ``--exclude-scripts, -x`` Don't install scripts. This is useful if you need to install multiple versions of a package, but do not want to reset the version that will be run by scripts that are already installed. ``--user`` (New in 0.6.11) Use the user-site-packages as specified in :pep:`370` instead of the global site-packages. ``--always-copy, -a`` (New in 0.5a4) Copy all needed distributions to the installation directory, even if they are already present in a directory on sys.path. In older versions of EasyInstall, this was the default behavior, but now you must explicitly request it. By default, EasyInstall will no longer copy such distributions from other sys.path directories to the installation directory, unless you explicitly gave the distribution's filename on the command line. Note that as of 0.6a10, using this option excludes "system" and "development" eggs from consideration because they can't be reliably copied. This may cause EasyInstall to choose an older version of a package than what you expected, or it may cause downloading and installation of a fresh copy of something that's already installed. You will see warning messages for any eggs that EasyInstall skips, before it falls back to an older version or attempts to download a fresh copy. ``--find-links=URLS_OR_FILENAMES, -f URLS_OR_FILENAMES`` Scan the specified "download pages" or directories for direct links to eggs or other distributions. Any existing file or directory names or direct download URLs are immediately added to EasyInstall's search cache, and any indirect URLs (ones that don't point to eggs or other recognized archive formats) are added to a list of additional places to search for download links. As soon as EasyInstall has to go online to find a package (either because it doesn't exist locally, or because ``--upgrade`` or ``-U`` was used), the specified URLs will be downloaded and scanned for additional direct links. Eggs and archives found by way of ``--find-links`` are only downloaded if they are needed to meet a requirement specified on the command line; links to unneeded packages are ignored. If all requested packages can be found using links on the specified download pages, the Python Package Index will not be consulted unless you also specified the ``--upgrade`` or ``-U`` option. (Note: if you want to refer to a local HTML file containing links, you must use a ``file:`` URL, as filenames that do not refer to a directory, egg, or archive are ignored.) You may specify multiple URLs or file/directory names with this option, separated by whitespace. Note that on the command line, you will probably have to surround the URL list with quotes, so that it is recognized as a single option value. You can also specify URLs in a configuration file; see `Configuration Files`_, above. Changed in 0.6a10: previously all URLs and directories passed to this option were scanned as early as possible, but from 0.6a10 on, only directories and direct archive links are scanned immediately; URLs are not retrieved unless a package search was already going to go online due to a package not being available locally, or due to the use of the ``--update`` or ``-U`` option. ``--no-find-links`` Blocks the addition of any link. This parameter is useful if you want to avoid adding links defined in a project easy_install is installing (whether it's a requested project or a dependency). When used, ``--find-links`` is ignored. Added in Distribute 0.6.11 and Setuptools 0.7. ``--index-url=URL, -i URL`` (New in 0.4a1; default changed in 0.6c7) Specifies the base URL of the Python Package Index. The default is https://pypi.org/simple/ if not specified. When a package is requested that is not locally available or linked from a ``--find-links`` download page, the package index will be searched for download pages for the needed package, and those download pages will be searched for links to download an egg or source distribution. ``--editable, -e`` (New in 0.6a1) Only find and download source distributions for the specified projects, unpacking them to subdirectories of the specified ``--build-directory``. EasyInstall will not actually build or install the requested projects or their dependencies; it will just find and extract them for you. See `Editing and Viewing Source Packages`_ above for more details. ``--build-directory=DIR, -b DIR`` (UPDATED in 0.6a1) Set the directory used to build source packages. If a package is built from a source distribution or checkout, it will be extracted to a subdirectory of the specified directory. The subdirectory will have the same name as the extracted distribution's project, but in all-lowercase. If a file or directory of that name already exists in the given directory, a warning will be printed to the console, and the build will take place in a temporary directory instead. This option is most useful in combination with the ``--editable`` option, which forces EasyInstall to *only* find and extract (but not build and install) source distributions. See `Editing and Viewing Source Packages`_, above, for more information. ``--verbose, -v, --quiet, -q`` (New in 0.4a4) Control the level of detail of EasyInstall's progress messages. The default detail level is "info", which prints information only about relatively time-consuming operations like running a setup script, unpacking an archive, or retrieving a URL. Using ``-q`` or ``--quiet`` drops the detail level to "warn", which will only display installation reports, warnings, and errors. Using ``-v`` or ``--verbose`` increases the detail level to include individual file-level operations, link analysis messages, and distutils messages from any setup scripts that get run. If you include the ``-v`` option more than once, the second and subsequent uses are passed down to any setup scripts, increasing the verbosity of their reporting as well. ``--dry-run, -n`` (New in 0.4a4) Don't actually install the package or scripts. This option is passed down to any setup scripts run, so packages should not actually build either. This does *not* skip downloading, nor does it skip extracting source distributions to a temporary/build directory. ``--optimize=LEVEL``, ``-O LEVEL`` (New in 0.4a4) If you are installing from a source distribution, and are *not* using the ``--zip-ok`` option, this option controls the optimization level for compiling installed ``.py`` files to ``.pyo`` files. It does not affect the compilation of modules contained in ``.egg`` files, only those in ``.egg`` directories. The optimization level can be set to 0, 1, or 2; the default is 0 (unless it's set under ``install`` or ``install_lib`` in one of your distutils configuration files). ``--record=FILENAME`` (New in 0.5a4) Write a record of all installed files to FILENAME. This is basically the same as the same option for the standard distutils "install" command, and is included for compatibility with tools that expect to pass this option to "setup.py install". ``--site-dirs=DIRLIST, -S DIRLIST`` (New in 0.6a1) Specify one or more custom "site" directories (separated by commas). "Site" directories are directories where ``.pth`` files are processed, such as the main Python ``site-packages`` directory. As of 0.6a10, EasyInstall automatically detects whether a given directory processes ``.pth`` files (or can be made to do so), so you should not normally need to use this option. It is is now only necessary if you want to override EasyInstall's judgment and force an installation directory to be treated as if it supported ``.pth`` files. ``--no-deps, -N`` (New in 0.6a6) Don't install any dependencies. This is intended as a convenience for tools that wrap eggs in a platform-specific packaging system. (We don't recommend that you use it for anything else.) ``--allow-hosts=PATTERNS, -H PATTERNS`` (New in 0.6a6) Restrict downloading and spidering to hosts matching the specified glob patterns. E.g. ``-H *.python.org`` restricts web access so that only packages listed and downloadable from machines in the ``python.org`` domain. The glob patterns must match the *entire* user/host/port section of the target URL(s). For example, ``*.python.org`` will NOT accept a URL like ``http://python.org/foo`` or ``http://www.python.org:8080/``. Multiple patterns can be specified by separating them with commas. The default pattern is ``*``, which matches anything. In general, this option is mainly useful for blocking EasyInstall's web access altogether (e.g. ``-Hlocalhost``), or to restrict it to an intranet or other trusted site. EasyInstall will do the best it can to satisfy dependencies given your host restrictions, but of course can fail if it can't find suitable packages. EasyInstall displays all blocked URLs, so that you can adjust your ``--allow-hosts`` setting if it is more strict than you intended. Some sites may wish to define a restrictive default setting for this option in their `configuration files`_, and then manually override the setting on the command line as needed. ``--prefix=DIR`` (New in 0.6a10) Use the specified directory as a base for computing the default installation and script directories. On Windows, the resulting default directories will be ``prefix\\Lib\\site-packages`` and ``prefix\\Scripts``, while on other platforms the defaults will be ``prefix/lib/python2.X/site-packages`` (with the appropriate version substituted) for libraries and ``prefix/bin`` for scripts. Note that the ``--prefix`` option only sets the *default* installation and script directories, and does not override the ones set on the command line or in a configuration file. ``--local-snapshots-ok, -l`` (New in 0.6c6) Normally, EasyInstall prefers to only install *released* versions of projects, not in-development ones, because such projects may not have a currently-valid version number. So, it usually only installs them when their ``setup.py`` directory is explicitly passed on the command line. However, if this option is used, then any in-development projects that were installed using the ``setup.py develop`` command, will be used to build eggs, effectively upgrading the "in-development" project to a snapshot release. Normally, this option is used only in conjunction with the ``--always-copy`` option to create a distributable snapshot of every egg needed to run an application. Note that if you use this option, you must make sure that there is a valid version number (such as an SVN revision number tag) for any in-development projects that may be used, as otherwise EasyInstall may not be able to tell what version of the project is "newer" when future installations or upgrades are attempted. .. _non-root installation: Custom Installation Locations ----------------------------- By default, EasyInstall installs python packages into Python's main ``site-packages`` directory, and manages them using a custom ``.pth`` file in that same directory. Very often though, a user or developer wants ``easy_install`` to install and manage python packages in an alternative location, usually for one of 3 reasons: 1. They don't have access to write to the main Python site-packages directory. 2. They want a user-specific stash of packages, that is not visible to other users. 3. They want to isolate a set of packages to a specific python application, usually to minimize the possibility of version conflicts. Historically, there have been many approaches to achieve custom installation. The following section lists only the easiest and most relevant approaches [1]_. `Use the "--user" option`_ `Use the "--user" option and customize "PYTHONUSERBASE"`_ `Use "virtualenv"`_ .. [1] There are older ways to achieve custom installation using various ``easy_install`` and ``setup.py install`` options, combined with ``PYTHONPATH`` and/or ``PYTHONUSERBASE`` alterations, but all of these are effectively deprecated by the User scheme brought in by `PEP-370`_. .. _PEP-370: http://www.python.org/dev/peps/pep-0370/ Use the "--user" option ~~~~~~~~~~~~~~~~~~~~~~~ Python provides a User scheme for installation, which means that all python distributions support an alternative install location that is specific to a user [3]_. The Default location for each OS is explained in the python documentation for the ``site.USER_BASE`` variable. This mode of installation can be turned on by specifying the ``--user`` option to ``setup.py install`` or ``easy_install``. This approach serves the need to have a user-specific stash of packages. .. [3] Prior to the User scheme, there was the Home scheme, which is still available, but requires more effort than the User scheme to get packages recognized. Use the "--user" option and customize "PYTHONUSERBASE" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The User scheme install location can be customized by setting the ``PYTHONUSERBASE`` environment variable, which updates the value of ``site.USER_BASE``. To isolate packages to a specific application, simply set the OS environment of that application to a specific value of ``PYTHONUSERBASE``, that contains just those packages. Use "virtualenv" ~~~~~~~~~~~~~~~~ "virtualenv" is a 3rd-party python package that effectively "clones" a python installation, thereby creating an isolated location to install packages. The evolution of "virtualenv" started before the existence of the User installation scheme. "virtualenv" provides a version of ``easy_install`` that is scoped to the cloned python install and is used in the normal way. "virtualenv" does offer various features that the User installation scheme alone does not provide, e.g. the ability to hide the main python site-packages. Please refer to the `virtualenv`_ documentation for more details. .. _virtualenv: https://pypi.org/project/virtualenv/ Package Index "API" ------------------- Custom package indexes (and PyPI) must follow the following rules for EasyInstall to be able to look up and download packages: 1. Except where stated otherwise, "pages" are HTML or XHTML, and "links" refer to ``href`` attributes. 2. Individual project version pages' URLs must be of the form ``base/projectname/version``, where ``base`` is the package index's base URL. 3. Omitting the ``/version`` part of a project page's URL (but keeping the trailing ``/``) should result in a page that is either: a) The single active version of that project, as though the version had been explicitly included, OR b) A page with links to all of the active version pages for that project. 4. Individual project version pages should contain direct links to downloadable distributions where possible. It is explicitly permitted for a project's "long_description" to include URLs, and these should be formatted as HTML links by the package index, as EasyInstall does no special processing to identify what parts of a page are index-specific and which are part of the project's supplied description. 5. Where available, MD5 information should be added to download URLs by appending a fragment identifier of the form ``#md5=...``, where ``...`` is the 32-character hex MD5 digest. EasyInstall will verify that the downloaded file's MD5 digest matches the given value. 6. Individual project version pages should identify any "homepage" or "download" URLs using ``rel="homepage"`` and ``rel="download"`` attributes on the HTML elements linking to those URLs. Use of these attributes will cause EasyInstall to always follow the provided links, unless it can be determined by inspection that they are downloadable distributions. If the links are not to downloadable distributions, they are retrieved, and if they are HTML, they are scanned for download links. They are *not* scanned for additional "homepage" or "download" links, as these are only processed for pages that are part of a package index site. 7. The root URL of the index, if retrieved with a trailing ``/``, must result in a page containing links to *all* projects' active version pages. (Note: This requirement is a workaround for the absence of case-insensitive ``safe_name()`` matching of project names in URL paths. If project names are matched in this fashion (e.g. via the PyPI server, mod_rewrite, or a similar mechanism), then it is not necessary to include this all-packages listing page.) 8. If a package index is accessed via a ``file://`` URL, then EasyInstall will automatically use ``index.html`` files, if present, when trying to read a directory with a trailing ``/`` on the URL. PK`_ Collection of recipes showing how to achieve more control over distutils. .. _pure-mod: Pure Python distribution (by module) ==================================== If you're just distributing a couple of modules, especially if they don't live in a particular package, you can specify them individually using the ``py_modules`` option in the setup script. In the simplest case, you'll have two files to worry about: a setup script and the single module you're distributing, :file:`foo.py` in this example:: / setup.py foo.py (In all diagrams in this section, ** will refer to the distribution root directory.) A minimal setup script to describe this situation would be:: from distutils.core import setup setup(name='foo', version='1.0', py_modules=['foo'], ) Note that the name of the distribution is specified independently with the ``name`` option, and there's no rule that says it has to be the same as the name of the sole module in the distribution (although that's probably a good convention to follow). However, the distribution name is used to generate filenames, so you should stick to letters, digits, underscores, and hyphens. Since ``py_modules`` is a list, you can of course specify multiple modules, eg. if you're distributing modules ``foo`` and ``bar``, your setup might look like this:: / setup.py foo.py bar.py and the setup script might be :: from distutils.core import setup setup(name='foobar', version='1.0', py_modules=['foo', 'bar'], ) You can put module source files into another directory, but if you have enough modules to do that, it's probably easier to specify modules by package rather than listing them individually. .. _pure-pkg: Pure Python distribution (by package) ===================================== If you have more than a couple of modules to distribute, especially if they are in multiple packages, it's probably easier to specify whole packages rather than individual modules. This works even if your modules are not in a package; you can just tell the Distutils to process modules from the root package, and that works the same as any other package (except that you don't have to have an :file:`__init__.py` file). The setup script from the last example could also be written as :: from distutils.core import setup setup(name='foobar', version='1.0', packages=[''], ) (The empty string stands for the root package.) If those two files are moved into a subdirectory, but remain in the root package, e.g.:: / setup.py src/ foo.py bar.py then you would still specify the root package, but you have to tell the Distutils where source files in the root package live:: from distutils.core import setup setup(name='foobar', version='1.0', package_dir={'': 'src'}, packages=[''], ) More typically, though, you will want to distribute multiple modules in the same package (or in sub-packages). For example, if the ``foo`` and ``bar`` modules belong in package ``foobar``, one way to layout your source tree is :: / setup.py foobar/ __init__.py foo.py bar.py This is in fact the default layout expected by the Distutils, and the one that requires the least work to describe in your setup script:: from distutils.core import setup setup(name='foobar', version='1.0', packages=['foobar'], ) If you want to put modules in directories not named for their package, then you need to use the ``package_dir`` option again. For example, if the :file:`src` directory holds modules in the ``foobar`` package:: / setup.py src/ __init__.py foo.py bar.py an appropriate setup script would be :: from distutils.core import setup setup(name='foobar', version='1.0', package_dir={'foobar': 'src'}, packages=['foobar'], ) Or, you might put modules from your main package right in the distribution root:: / setup.py __init__.py foo.py bar.py in which case your setup script would be :: from distutils.core import setup setup(name='foobar', version='1.0', package_dir={'foobar': ''}, packages=['foobar'], ) (The empty string also stands for the current directory.) If you have sub-packages, they must be explicitly listed in ``packages``, but any entries in ``package_dir`` automatically extend to sub-packages. (In other words, the Distutils does *not* scan your source tree, trying to figure out which directories correspond to Python packages by looking for :file:`__init__.py` files.) Thus, if the default layout grows a sub-package:: / setup.py foobar/ __init__.py foo.py bar.py subfoo/ __init__.py blah.py then the corresponding setup script would be :: from distutils.core import setup setup(name='foobar', version='1.0', packages=['foobar', 'foobar.subfoo'], ) .. _single-ext: Single extension module ======================= Extension modules are specified using the ``ext_modules`` option. ``package_dir`` has no effect on where extension source files are found; it only affects the source for pure Python modules. The simplest case, a single extension module in a single C source file, is:: / setup.py foo.c If the ``foo`` extension belongs in the root package, the setup script for this could be :: from distutils.core import setup from distutils.extension import Extension setup(name='foobar', version='1.0', ext_modules=[Extension('foo', ['foo.c'])], ) If the extension actually belongs in a package, say ``foopkg``, then With exactly the same source tree layout, this extension can be put in the ``foopkg`` package simply by changing the name of the extension:: from distutils.core import setup from distutils.extension import Extension setup(name='foobar', version='1.0', ext_modules=[Extension('foopkg.foo', ['foo.c'])], ) Checking a package ================== The ``check`` command allows you to verify if your package meta-data meet the minimum requirements to build a distribution. To run it, just call it using your :file:`setup.py` script. If something is missing, ``check`` will display a warning. Let's take an example with a simple script:: from distutils.core import setup setup(name='foobar') Running the ``check`` command will display some warnings: .. code-block:: shell-session $ python setup.py check running check warning: check: missing required meta-data: version, url warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied If you use the reStructuredText syntax in the ``long_description`` field and `docutils`_ is installed you can check if the syntax is fine with the ``check`` command, using the ``restructuredtext`` option. For example, if the :file:`setup.py` script is changed like this:: from distutils.core import setup desc = """\ My description ============== This is the description of the ``foobar`` package. """ setup(name='foobar', version='1', author='tarek', author_email='tarek@ziade.org', url='http://example.com', long_description=desc) Where the long description is broken, ``check`` will be able to detect it by using the :mod:`docutils` parser: .. code-block:: shell-session $ python setup.py check --restructuredtext running check warning: check: Title underline too short. (line 2) warning: check: Could not finish the parsing. Reading the metadata ===================== The :func:`distutils.core.setup` function provides a command-line interface that allows you to query the metadata fields of a project through the ``setup.py`` script of a given project: .. code-block:: shell-session $ python setup.py --name distribute This call reads the ``name`` metadata by running the :func:`distutils.core.setup` function. Although, when a source or binary distribution is created with Distutils, the metadata fields are written in a static file called :file:`PKG-INFO`. When a Distutils-based project is installed in Python, the :file:`PKG-INFO` file is copied alongside the modules and packages of the distribution under :file:`NAME-VERSION-pyX.X.egg-info`, where ``NAME`` is the name of the project, ``VERSION`` its version as defined in the Metadata, and ``pyX.X`` the major and minor version of Python like ``2.7`` or ``3.2``. You can read back this static file, by using the :class:`distutils.dist.DistributionMetadata` class and its :func:`~distutils.dist.DistributionMetadata.read_pkg_file` method:: >>> from distutils.dist import DistributionMetadata >>> metadata = DistributionMetadata() >>> metadata.read_pkg_file(open('distribute-0.6.8-py2.7.egg-info')) >>> metadata.name 'distribute' >>> metadata.version '0.6.8' >>> metadata.description 'Easily download, build, install, upgrade, and uninstall Python packages' Notice that the class can also be instantiated with a metadata file path to loads its values:: >>> pkg_info_path = 'distribute-0.6.8-py2.7.egg-info' >>> DistributionMetadata(pkg_info_path).name 'distribute' .. % \section{Multiple extension modules} .. % \label{multiple-ext} .. % \section{Putting it all together} .. _docutils: http://docutils.sourceforge.net PKyq!!*docs/deprecated/distutils/introduction.rstnu[.. _distutils-intro: **************************** An Introduction to Distutils **************************** .. include:: ./_setuptools_disclaimer.rst This document covers using the Distutils to distribute your Python modules, concentrating on the role of developer/distributor: if you're looking for information on installing Python modules, you should refer to the :ref:`install-index` chapter. .. _distutils-concepts: Concepts & Terminology ====================== Using the Distutils is quite simple, both for module developers and for users/administrators installing third-party modules. As a developer, your responsibilities (apart from writing solid, well-documented and well-tested code, of course!) are: * write a setup script (:file:`setup.py` by convention) * (optional) write a setup configuration file * create a source distribution * (optional) create one or more built (binary) distributions Each of these tasks is covered in this document. Not all module developers have access to a multitude of platforms, so it's not always feasible to expect them to create a multitude of built distributions. It is hoped that a class of intermediaries, called *packagers*, will arise to address this need. Packagers will take source distributions released by module developers, build them on one or more platforms, and release the resulting built distributions. Thus, users on the most popular platforms will be able to install most popular Python module distributions in the most natural way for their platform, without having to run a single setup script or compile a line of code. .. _distutils-simple-example: A Simple Example ================ The setup script is usually quite simple, although since it's written in Python, there are no arbitrary limits to what you can do with it, though you should be careful about putting arbitrarily expensive operations in your setup script. Unlike, say, Autoconf-style configure scripts, the setup script may be run multiple times in the course of building and installing your module distribution. If all you want to do is distribute a module called ``foo``, contained in a file :file:`foo.py`, then your setup script can be as simple as this:: from distutils.core import setup setup(name='foo', version='1.0', py_modules=['foo'], ) Some observations: * most information that you supply to the Distutils is supplied as keyword arguments to the :func:`~distutils.core.setup` function * those keyword arguments fall into two categories: package metadata (name, version number) and information about what's in the package (a list of pure Python modules, in this case) * modules are specified by module name, not filename (the same will hold true for packages and extensions) * it's recommended that you supply a little more metadata, in particular your name, email address and a URL for the project (see section :ref:`setup-script` for an example) To create a source distribution for this module, you would create a setup script, :file:`setup.py`, containing the above code, and run this command from a terminal:: python setup.py sdist For Windows, open a command prompt window (:menuselection:`Start --> Accessories`) and change the command to:: setup.py sdist :command:`sdist` will create an archive file (e.g., tarball on Unix, ZIP file on Windows) containing your setup script :file:`setup.py`, and your module :file:`foo.py`. The archive file will be named :file:`foo-1.0.tar.gz` (or :file:`.zip`), and will unpack into a directory :file:`foo-1.0`. If an end-user wishes to install your ``foo`` module, all they have to do is download :file:`foo-1.0.tar.gz` (or :file:`.zip`), unpack it, and---from the :file:`foo-1.0` directory---run :: python setup.py install which will ultimately copy :file:`foo.py` to the appropriate directory for third-party modules in their Python installation. This simple example demonstrates some fundamental concepts of the Distutils. First, both developers and installers have the same basic user interface, i.e. the setup script. The difference is which Distutils *commands* they use: the :command:`sdist` command is almost exclusively for module developers, while :command:`install` is more often for installers (although most developers will want to install their own code occasionally). If you want to make things really easy for your users, you can create one or more built distributions for them. For instance, if you are running on a Windows machine, and want to make things easy for other Windows users, you can create an executable installer (the most appropriate type of built distribution for this platform) with the :command:`bdist_wininst` command. For example:: python setup.py bdist_wininst will create an executable installer, :file:`foo-1.0.win32.exe`, in the current directory. Other useful built distribution formats are RPM, implemented by the :command:`bdist_rpm` command, Solaris :program:`pkgtool` (:command:`bdist_pkgtool`), and HP-UX :program:`swinstall` (:command:`bdist_sdux`). For example, the following command will create an RPM file called :file:`foo-1.0.noarch.rpm`:: python setup.py bdist_rpm (The :command:`bdist_rpm` command uses the :command:`rpm` executable, therefore this has to be run on an RPM-based system such as Red Hat Linux, SuSE Linux, or Mandrake Linux.) You can find out what distribution formats are available at any time by running :: python setup.py bdist --help-formats .. _python-terms: General Python terminology ========================== If you're reading this document, you probably have a good idea of what modules, extensions, and so forth are. Nevertheless, just to be sure that everyone is operating from a common starting point, we offer the following glossary of common Python terms: module the basic unit of code reusability in Python: a block of code imported by some other code. Three types of modules concern us here: pure Python modules, extension modules, and packages. pure Python module a module written in Python and contained in a single :file:`.py` file (and possibly associated :file:`.pyc` files). Sometimes referred to as a "pure module." extension module a module written in the low-level language of the Python implementation: C/C++ for Python, Java for Jython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object (:file:`.so`) file for Python extensions on Unix, a DLL (given the :file:`.pyd` extension) for Python extensions on Windows, or a Java class file for Jython extensions. (Note that currently, the Distutils only handles C/C++ extensions for Python.) package a module that contains other modules; typically contained in a directory in the filesystem and distinguished from other directories by the presence of a file :file:`__init__.py`. root package the root of the hierarchy of packages. (This isn't really a package, since it doesn't have an :file:`__init__.py` file. But we have to call it something.) The vast majority of the standard library is in the root package, as are many small, standalone third-party modules that don't belong to a larger module collection. Unlike regular packages, modules in the root package can be found in many directories: in fact, every directory listed in ``sys.path`` contributes modules to the root package. .. _distutils-term: Distutils-specific terminology ============================== The following terms apply more specifically to the domain of distributing Python modules using the Distutils: module distribution a collection of Python modules distributed together as a single downloadable resource and meant to be installed *en masse*. Examples of some well-known module distributions are NumPy, SciPy, Pillow, or mxBase. (This would be called a *package*, except that term is already taken in the Python context: a single module distribution may contain zero, one, or many Python packages.) pure module distribution a module distribution that contains only pure Python modules and packages. Sometimes referred to as a "pure distribution." non-pure module distribution a module distribution that contains at least one extension module. Sometimes referred to as a "non-pure distribution." distribution root the top-level directory of your source tree (or source distribution); the directory where :file:`setup.py` exists. Generally :file:`setup.py` will be run from this directory. PK`_. | +--------------------+--------------------------------+-------------------------------------------------------------+ | *distclass* | the :class:`Distribution` | a subclass of | | | class to use | :class:`distutils.core.Distribution` | +--------------------+--------------------------------+-------------------------------------------------------------+ | *script_name* | The name of the setup.py | a string | | | script - defaults to | | | | ``sys.argv[0]`` | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *script_args* | Arguments to supply to the | a list of strings | | | setup script | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *options* | default options for the setup | a dictionary | | | script | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *license* | The license for the package | a string | +--------------------+--------------------------------+-------------------------------------------------------------+ | *keywords* | Descriptive meta-data, see | a list of strings or a comma-separated string | | | :pep:`314` | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *platforms* | | a list of strings or a comma-separated string | +--------------------+--------------------------------+-------------------------------------------------------------+ | *cmdclass* | A mapping of command names to | a dictionary | | | :class:`Command` subclasses | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *data_files* | A list of data files to | a list | | | install | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *package_dir* | A mapping of package to | a dictionary | | | directory names | | +--------------------+--------------------------------+-------------------------------------------------------------+ .. function:: run_setup(script_name[, script_args=None, stop_after='run']) Run a setup script in a somewhat controlled environment, and return the :class:`distutils.dist.Distribution` instance that drives things. This is useful if you need to find out the distribution meta-data (passed as keyword args from *script* to :func:`setup`), or the contents of the config files or command-line. *script_name* is a file that will be read and run with :func:`exec`. ``sys.argv[0]`` will be replaced with *script* for the duration of the call. *script_args* is a list of strings; if supplied, ``sys.argv[1:]`` will be replaced by *script_args* for the duration of the call. *stop_after* tells :func:`setup` when to stop processing; possible values: .. tabularcolumns:: |l|L| +---------------+---------------------------------------------+ | value | description | +===============+=============================================+ | *init* | Stop after the :class:`Distribution` | | | instance has been created and populated | | | with the keyword arguments to :func:`setup` | +---------------+---------------------------------------------+ | *config* | Stop after config files have been parsed | | | (and their data stored in the | | | :class:`Distribution` instance) | +---------------+---------------------------------------------+ | *commandline* | Stop after the command-line | | | (``sys.argv[1:]`` or *script_args*) have | | | been parsed (and the data stored in the | | | :class:`Distribution` instance.) | +---------------+---------------------------------------------+ | *run* | Stop after all commands have been run (the | | | same as if :func:`setup` had been called | | | in the usual way). This is the default | | | value. | +---------------+---------------------------------------------+ In addition, the :mod:`distutils.core` module exposed a number of classes that live elsewhere. * :class:`~distutils.extension.Extension` from :mod:`distutils.extension` * :class:`~distutils.cmd.Command` from :mod:`distutils.cmd` * :class:`~distutils.dist.Distribution` from :mod:`distutils.dist` A short description of each of these follows, but see the relevant module for the full reference. .. class:: Extension The Extension class describes a single C or C++ extension module in a setup script. It accepts the following keyword arguments in its constructor: .. tabularcolumns:: |l|L|l| +------------------------+--------------------------------+---------------------------+ | argument name | value | type | +========================+================================+===========================+ | *name* | the full name of the | a string | | | extension, including any | | | | packages --- ie. *not* a | | | | filename or pathname, but | | | | Python dotted name | | +------------------------+--------------------------------+---------------------------+ | *sources* | list of source filenames, | a list of strings | | | relative to the distribution | | | | root (where the setup script | | | | lives), in Unix form | | | | (slash-separated) for | | | | portability. | | | | Source files may be C, C++, | | | | SWIG (.i), platform-specific | | | | resource files, or whatever | | | | else is recognized by the | | | | :command:`build_ext` command | | | | as source for a Python | | | | extension. | | +------------------------+--------------------------------+---------------------------+ | *include_dirs* | list of directories to search | a list of strings | | | for C/C++ header files (in | | | | Unix form for portability) | | +------------------------+--------------------------------+---------------------------+ | *define_macros* | list of macros to define; each | a list of tuples | | | macro is defined using a | | | | 2-tuple ``(name, value)``, | | | | where *value* is | | | | either the string to define it | | | | to or ``None`` to define it | | | | without a particular value | | | | (equivalent of ``#define FOO`` | | | | in source or :option:`!-DFOO` | | | | on Unix C compiler command | | | | line) | | +------------------------+--------------------------------+---------------------------+ | *undef_macros* | list of macros to undefine | a list of strings | | | explicitly | | +------------------------+--------------------------------+---------------------------+ | *library_dirs* | list of directories to search | a list of strings | | | for C/C++ libraries at link | | | | time | | +------------------------+--------------------------------+---------------------------+ | *libraries* | list of library names (not | a list of strings | | | filenames or paths) to link | | | | against | | +------------------------+--------------------------------+---------------------------+ | *runtime_library_dirs* | list of directories to search | a list of strings | | | for C/C++ libraries at run | | | | time (for shared extensions, | | | | this is when the extension is | | | | loaded) | | +------------------------+--------------------------------+---------------------------+ | *extra_objects* | list of extra files to link | a list of strings | | | with (eg. object files not | | | | implied by 'sources', static | | | | library that must be | | | | explicitly specified, binary | | | | resource files, etc.) | | +------------------------+--------------------------------+---------------------------+ | *extra_compile_args* | any extra platform- and | a list of strings | | | compiler-specific information | | | | to use when compiling the | | | | source files in 'sources'. For | | | | platforms and compilers where | | | | a command line makes sense, | | | | this is typically a list of | | | | command-line arguments, but | | | | for other platforms it could | | | | be anything. | | +------------------------+--------------------------------+---------------------------+ | *extra_link_args* | any extra platform- and | a list of strings | | | compiler-specific information | | | | to use when linking object | | | | files together to create the | | | | extension (or to create a new | | | | static Python interpreter). | | | | Similar interpretation as for | | | | 'extra_compile_args'. | | +------------------------+--------------------------------+---------------------------+ | *export_symbols* | list of symbols to be exported | a list of strings | | | from a shared extension. Not | | | | used on all platforms, and not | | | | generally necessary for Python | | | | extensions, which typically | | | | export exactly one symbol: | | | | ``init`` + extension_name. | | +------------------------+--------------------------------+---------------------------+ | *depends* | list of files that the | a list of strings | | | extension depends on | | +------------------------+--------------------------------+---------------------------+ | *language* | extension language (i.e. | a string | | | ``'c'``, ``'c++'``, | | | | ``'objc'``). Will be detected | | | | from the source extensions if | | | | not provided. | | +------------------------+--------------------------------+---------------------------+ | *optional* | specifies that a build failure | a boolean | | | in the extension should not | | | | abort the build process, but | | | | simply skip the extension. | | +------------------------+--------------------------------+---------------------------+ .. versionchanged:: 3.8 On Unix, C extensions are no longer linked to libpython except on Android and Cygwin. .. class:: Distribution A :class:`Distribution` describes how to build, install and package up a Python software package. See the :func:`setup` function for a list of keyword arguments accepted by the Distribution constructor. :func:`setup` creates a Distribution instance. .. versionchanged:: 3.7 :class:`~distutils.core.Distribution` now warns if ``classifiers``, ``keywords`` and ``platforms`` fields are not specified as a list or a string. .. class:: Command A :class:`Command` class (or rather, an instance of one of its subclasses) implement a single distutils command. :mod:`distutils.ccompiler` --- CCompiler base class =================================================== .. module:: distutils.ccompiler :synopsis: Abstract CCompiler class This module provides the abstract base class for the :class:`CCompiler` classes. A :class:`CCompiler` instance can be used for all the compile and link steps needed to build a single project. Methods are provided to set options for the compiler --- macro definitions, include directories, link path, libraries and the like. This module provides the following functions. .. function:: gen_lib_options(compiler, library_dirs, runtime_library_dirs, libraries) Generate linker options for searching library directories and linking with specific libraries. *libraries* and *library_dirs* are, respectively, lists of library names (not filenames!) and search directories. Returns a list of command-line options suitable for use with some compiler (depending on the two format strings passed in). .. function:: gen_preprocess_options(macros, include_dirs) Generate C pre-processor options (:option:`!-D`, :option:`!-U`, :option:`!-I`) as used by at least two types of compilers: the typical Unix compiler and Visual C++. *macros* is the usual thing, a list of 1- or 2-tuples, where ``(name,)`` means undefine (:option:`!-U`) macro *name*, and ``(name, value)`` means define (:option:`!-D`) macro *name* to *value*. *include_dirs* is just a list of directory names to be added to the header file search path (:option:`!-I`). Returns a list of command-line options suitable for either Unix compilers or Visual C++. .. function:: get_default_compiler(osname, platform) Determine the default compiler to use for the given platform. *osname* should be one of the standard Python OS names (i.e. the ones returned by ``os.name``) and *platform* the common value returned by ``sys.platform`` for the platform in question. The default values are ``os.name`` and ``sys.platform`` in case the parameters are not given. .. function:: new_compiler(plat=None, compiler=None, verbose=0, dry_run=0, force=0) Factory function to generate an instance of some CCompiler subclass for the supplied platform/compiler combination. *plat* defaults to ``os.name`` (eg. ``'posix'``, ``'nt'``), and *compiler* defaults to the default compiler for that platform. Currently only ``'posix'`` and ``'nt'`` are supported, and the default compilers are "traditional Unix interface" (:class:`UnixCCompiler` class) and Visual C++ (:class:`MSVCCompiler` class). Note that it's perfectly possible to ask for a Unix compiler object under Windows, and a Microsoft compiler object under Unix---if you supply a value for *compiler*, *plat* is ignored. .. % Is the posix/nt only thing still true? Mac OS X seems to work, and .. % returns a UnixCCompiler instance. How to document this... hmm. .. function:: show_compilers() Print list of available compilers (used by the :option:`!--help-compiler` options to :command:`build`, :command:`build_ext`, :command:`build_clib`). .. class:: CCompiler([verbose=0, dry_run=0, force=0]) The abstract base class :class:`CCompiler` defines the interface that must be implemented by real compiler classes. The class also has some utility methods used by several compiler classes. The basic idea behind a compiler abstraction class is that each instance can be used for all the compile/link steps in building a single project. Thus, attributes common to all of those compile and link steps --- include directories, macros to define, libraries to link against, etc. --- are attributes of the compiler instance. To allow for variability in how individual files are treated, most of those attributes may be varied on a per-compilation or per-link basis. The constructor for each subclass creates an instance of the Compiler object. Flags are *verbose* (show verbose output), *dry_run* (don't actually execute the steps) and *force* (rebuild everything, regardless of dependencies). All of these flags default to ``0`` (off). Note that you probably don't want to instantiate :class:`CCompiler` or one of its subclasses directly - use the :func:`distutils.CCompiler.new_compiler` factory function instead. The following methods allow you to manually alter compiler options for the instance of the Compiler class. .. method:: CCompiler.add_include_dir(dir) Add *dir* to the list of directories that will be searched for header files. The compiler is instructed to search directories in the order in which they are supplied by successive calls to :meth:`add_include_dir`. .. method:: CCompiler.set_include_dirs(dirs) Set the list of directories that will be searched to *dirs* (a list of strings). Overrides any preceding calls to :meth:`add_include_dir`; subsequent calls to :meth:`add_include_dir` add to the list passed to :meth:`set_include_dirs`. This does not affect any list of standard include directories that the compiler may search by default. .. method:: CCompiler.add_library(libname) Add *libname* to the list of libraries that will be included in all links driven by this compiler object. Note that *libname* should \*not\* be the name of a file containing a library, but the name of the library itself: the actual filename will be inferred by the linker, the compiler, or the compiler class (depending on the platform). The linker will be instructed to link against libraries in the order they were supplied to :meth:`add_library` and/or :meth:`set_libraries`. It is perfectly valid to duplicate library names; the linker will be instructed to link against libraries as many times as they are mentioned. .. method:: CCompiler.set_libraries(libnames) Set the list of libraries to be included in all links driven by this compiler object to *libnames* (a list of strings). This does not affect any standard system libraries that the linker may include by default. .. method:: CCompiler.add_library_dir(dir) Add *dir* to the list of directories that will be searched for libraries specified to :meth:`add_library` and :meth:`set_libraries`. The linker will be instructed to search for libraries in the order they are supplied to :meth:`add_library_dir` and/or :meth:`set_library_dirs`. .. method:: CCompiler.set_library_dirs(dirs) Set the list of library search directories to *dirs* (a list of strings). This does not affect any standard library search path that the linker may search by default. .. method:: CCompiler.add_runtime_library_dir(dir) Add *dir* to the list of directories that will be searched for shared libraries at runtime. .. method:: CCompiler.set_runtime_library_dirs(dirs) Set the list of directories to search for shared libraries at runtime to *dirs* (a list of strings). This does not affect any standard search path that the runtime linker may search by default. .. method:: CCompiler.define_macro(name[, value=None]) Define a preprocessor macro for all compilations driven by this compiler object. The optional parameter *value* should be a string; if it is not supplied, then the macro will be defined without an explicit value and the exact outcome depends on the compiler used. .. XXX true? does ANSI say anything about this? .. method:: CCompiler.undefine_macro(name) Undefine a preprocessor macro for all compilations driven by this compiler object. If the same macro is defined by :meth:`define_macro` and undefined by :meth:`undefine_macro` the last call takes precedence (including multiple redefinitions or undefinitions). If the macro is redefined/undefined on a per-compilation basis (ie. in the call to :meth:`compile`), then that takes precedence. .. method:: CCompiler.add_link_object(object) Add *object* to the list of object files (or analogues, such as explicitly named library files or the output of "resource compilers") to be included in every link driven by this compiler object. .. method:: CCompiler.set_link_objects(objects) Set the list of object files (or analogues) to be included in every link to *objects*. This does not affect any standard object files that the linker may include by default (such as system libraries). The following methods implement methods for autodetection of compiler options, providing some functionality similar to GNU :program:`autoconf`. .. method:: CCompiler.detect_language(sources) Detect the language of a given file, or list of files. Uses the instance attributes :attr:`~CCompiler.language_map` (a dictionary), and :attr:`~CCompiler.language_order` (a list) to do the job. .. method:: CCompiler.find_library_file(dirs, lib[, debug=0]) Search the specified list of directories for a static or shared library file *lib* and return the full path to that file. If *debug* is true, look for a debugging version (if that makes sense on the current platform). Return ``None`` if *lib* wasn't found in any of the specified directories. .. method:: CCompiler.has_function(funcname [, includes=None, include_dirs=None, libraries=None, library_dirs=None]) Return a boolean indicating whether *funcname* is supported on the current platform. The optional arguments can be used to augment the compilation environment by providing additional include files and paths and libraries and paths. .. method:: CCompiler.library_dir_option(dir) Return the compiler option to add *dir* to the list of directories searched for libraries. .. method:: CCompiler.library_option(lib) Return the compiler option to add *lib* to the list of libraries linked into the shared library or executable. .. method:: CCompiler.runtime_library_dir_option(dir) Return the compiler option to add *dir* to the list of directories searched for runtime libraries. .. method:: CCompiler.set_executables(**args) Define the executables (and options for them) that will be run to perform the various stages of compilation. The exact set of executables that may be specified here depends on the compiler class (via the 'executables' class attribute), but most will have: +--------------+------------------------------------------+ | attribute | description | +==============+==========================================+ | *compiler* | the C/C++ compiler | +--------------+------------------------------------------+ | *linker_so* | linker used to create shared objects and | | | libraries | +--------------+------------------------------------------+ | *linker_exe* | linker used to create binary executables | +--------------+------------------------------------------+ | *archiver* | static library creator | +--------------+------------------------------------------+ On platforms with a command-line (Unix, DOS/Windows), each of these is a string that will be split into executable name and (optional) list of arguments. (Splitting the string is done similarly to how Unix shells operate: words are delimited by spaces, but quotes and backslashes can override this. See :func:`distutils.util.split_quoted`.) The following methods invoke stages in the build process. .. method:: CCompiler.compile(sources[, output_dir=None, macros=None, include_dirs=None, debug=0, extra_preargs=None, extra_postargs=None, depends=None]) Compile one or more source files. Generates object files (e.g. transforms a :file:`.c` file to a :file:`.o` file.) *sources* must be a list of filenames, most likely C/C++ files, but in reality anything that can be handled by a particular compiler and compiler class (eg. :class:`MSVCCompiler` can handle resource files in *sources*). Return a list of object filenames, one per source filename in *sources*. Depending on the implementation, not all source files will necessarily be compiled, but all corresponding object filenames will be returned. If *output_dir* is given, object files will be put under it, while retaining their original path component. That is, :file:`foo/bar.c` normally compiles to :file:`foo/bar.o` (for a Unix implementation); if *output_dir* is *build*, then it would compile to :file:`build/foo/bar.o`. *macros*, if given, must be a list of macro definitions. A macro definition is either a ``(name, value)`` 2-tuple or a ``(name,)`` 1-tuple. The former defines a macro; if the value is ``None``, the macro is defined without an explicit value. The 1-tuple case undefines a macro. Later definitions/redefinitions/undefinitions take precedence. *include_dirs*, if given, must be a list of strings, the directories to add to the default include file search path for this compilation only. *debug* is a boolean; if true, the compiler will be instructed to output debug symbols in (or alongside) the object file(s). *extra_preargs* and *extra_postargs* are implementation-dependent. On platforms that have the notion of a command-line (e.g. Unix, DOS/Windows), they are most likely lists of strings: extra command-line arguments to prepend/append to the compiler command line. On other platforms, consult the implementation class documentation. In any event, they are intended as an escape hatch for those occasions when the abstract compiler framework doesn't cut the mustard. *depends*, if given, is a list of filenames that all targets depend on. If a source file is older than any file in depends, then the source file will be recompiled. This supports dependency tracking, but only at a coarse granularity. Raises :exc:`CompileError` on failure. .. method:: CCompiler.create_static_lib(objects, output_libname[, output_dir=None, debug=0, target_lang=None]) Link a bunch of stuff together to create a static library file. The "bunch of stuff" consists of the list of object files supplied as *objects*, the extra object files supplied to :meth:`add_link_object` and/or :meth:`set_link_objects`, the libraries supplied to :meth:`add_library` and/or :meth:`set_libraries`, and the libraries supplied as *libraries* (if any). *output_libname* should be a library name, not a filename; the filename will be inferred from the library name. *output_dir* is the directory where the library file will be put. .. XXX defaults to what? *debug* is a boolean; if true, debugging information will be included in the library (note that on most platforms, it is the compile step where this matters: the *debug* flag is included here just for consistency). *target_lang* is the target language for which the given objects are being compiled. This allows specific linkage time treatment of certain languages. Raises :exc:`LibError` on failure. .. method:: CCompiler.link(target_desc, objects, output_filename[, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None]) Link a bunch of stuff together to create an executable or shared library file. The "bunch of stuff" consists of the list of object files supplied as *objects*. *output_filename* should be a filename. If *output_dir* is supplied, *output_filename* is relative to it (i.e. *output_filename* can provide directory components if needed). *libraries* is a list of libraries to link against. These are library names, not filenames, since they're translated into filenames in a platform-specific way (eg. *foo* becomes :file:`libfoo.a` on Unix and :file:`foo.lib` on DOS/Windows). However, they can include a directory component, which means the linker will look in that specific directory rather than searching all the normal locations. *library_dirs*, if supplied, should be a list of directories to search for libraries that were specified as bare library names (ie. no directory component). These are on top of the system default and those supplied to :meth:`add_library_dir` and/or :meth:`set_library_dirs`. *runtime_library_dirs* is a list of directories that will be embedded into the shared library and used to search for other shared libraries that \*it\* depends on at run-time. (This may only be relevant on Unix.) *export_symbols* is a list of symbols that the shared library will export. (This appears to be relevant only on Windows.) *debug* is as for :meth:`compile` and :meth:`create_static_lib`, with the slight distinction that it actually matters on most platforms (as opposed to :meth:`create_static_lib`, which includes a *debug* flag mostly for form's sake). *extra_preargs* and *extra_postargs* are as for :meth:`compile` (except of course that they supply command-line arguments for the particular linker being used). *target_lang* is the target language for which the given objects are being compiled. This allows specific linkage time treatment of certain languages. Raises :exc:`LinkError` on failure. .. method:: CCompiler.link_executable(objects, output_progname[, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, debug=0, extra_preargs=None, extra_postargs=None, target_lang=None]) Link an executable. *output_progname* is the name of the file executable, while *objects* are a list of object filenames to link in. Other arguments are as for the :meth:`link` method. .. method:: CCompiler.link_shared_lib(objects, output_libname[, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None]) Link a shared library. *output_libname* is the name of the output library, while *objects* is a list of object filenames to link in. Other arguments are as for the :meth:`link` method. .. method:: CCompiler.link_shared_object(objects, output_filename[, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None]) Link a shared object. *output_filename* is the name of the shared object that will be created, while *objects* is a list of object filenames to link in. Other arguments are as for the :meth:`link` method. .. method:: CCompiler.preprocess(source[, output_file=None, macros=None, include_dirs=None, extra_preargs=None, extra_postargs=None]) Preprocess a single C/C++ source file, named in *source*. Output will be written to file named *output_file*, or *stdout* if *output_file* not supplied. *macros* is a list of macro definitions as for :meth:`compile`, which will augment the macros set with :meth:`define_macro` and :meth:`undefine_macro`. *include_dirs* is a list of directory names that will be added to the default list, in the same way as :meth:`add_include_dir`. Raises :exc:`PreprocessError` on failure. The following utility methods are defined by the :class:`CCompiler` class, for use by the various concrete subclasses. .. method:: CCompiler.executable_filename(basename[, strip_dir=0, output_dir='']) Returns the filename of the executable for the given *basename*. Typically for non-Windows platforms this is the same as the basename, while Windows will get a :file:`.exe` added. .. method:: CCompiler.library_filename(libname[, lib_type='static', strip_dir=0, output_dir='']) Returns the filename for the given library name on the current platform. On Unix a library with *lib_type* of ``'static'`` will typically be of the form :file:`liblibname.a`, while a *lib_type* of ``'dynamic'`` will be of the form :file:`liblibname.so`. .. method:: CCompiler.object_filenames(source_filenames[, strip_dir=0, output_dir='']) Returns the name of the object files for the given source files. *source_filenames* should be a list of filenames. .. method:: CCompiler.shared_object_filename(basename[, strip_dir=0, output_dir='']) Returns the name of a shared object file for the given file name *basename*. .. method:: CCompiler.execute(func, args[, msg=None, level=1]) Invokes :func:`distutils.util.execute`. This method invokes a Python function *func* with the given arguments *args*, after logging and taking into account the *dry_run* flag. .. method:: CCompiler.spawn(cmd) Invokes :func:`distutils.spawn.spawn`. This invokes an external process to run the given command. .. method:: CCompiler.mkpath(name[, mode=511]) Invokes :func:`distutils.dir_util.mkpath`. This creates a directory and any missing ancestor directories. .. method:: CCompiler.move_file(src, dst) Invokes :meth:`distutils.file_util.move_file`. Renames *src* to *dst*. .. method:: CCompiler.announce(msg[, level=1]) Write a message using :func:`distutils.log.debug`. .. method:: CCompiler.warn(msg) Write a warning message *msg* to standard error. .. method:: CCompiler.debug_print(msg) If the *debug* flag is set on this :class:`CCompiler` instance, print *msg* to standard output, otherwise do nothing. .. % \subsection{Compiler-specific modules} .. % .. % The following modules implement concrete subclasses of the abstract .. % \class{CCompiler} class. They should not be instantiated directly, but should .. % be created using \function{distutils.ccompiler.new_compiler()} factory .. % function. :mod:`distutils.unixccompiler` --- Unix C Compiler ================================================== .. module:: distutils.unixccompiler :synopsis: UNIX C Compiler This module provides the :class:`UnixCCompiler` class, a subclass of :class:`CCompiler` that handles the typical Unix-style command-line C compiler: * macros defined with :option:`!-Dname[=value]` * macros undefined with :option:`!-Uname` * include search directories specified with :option:`!-Idir` * libraries specified with :option:`!-llib` * library search directories specified with :option:`!-Ldir` * compile handled by :program:`cc` (or similar) executable with :option:`!-c` option: compiles :file:`.c` to :file:`.o` * link static library handled by :program:`ar` command (possibly with :program:`ranlib`) * link shared library handled by :program:`cc` :option:`!-shared` :mod:`distutils.msvccompiler` --- Microsoft Compiler ==================================================== .. module:: distutils.msvccompiler :synopsis: Microsoft Compiler .. XXX: This is *waaaaay* out of date! This module provides :class:`MSVCCompiler`, an implementation of the abstract :class:`CCompiler` class for Microsoft Visual Studio. Typically, extension modules need to be compiled with the same compiler that was used to compile Python. For Python 2.3 and earlier, the compiler was Visual Studio 6. For Python 2.4 and 2.5, the compiler is Visual Studio .NET 2003. :class:`MSVCCompiler` will normally choose the right compiler, linker etc. on its own. To override this choice, the environment variables *DISTUTILS_USE_SDK* and *MSSdk* must be both set. *MSSdk* indicates that the current environment has been setup by the SDK's ``SetEnv.Cmd`` script, or that the environment variables had been registered when the SDK was installed; *DISTUTILS_USE_SDK* indicates that the distutils user has made an explicit choice to override the compiler selection by :class:`MSVCCompiler`. :mod:`distutils.bcppcompiler` --- Borland Compiler ================================================== .. module:: distutils.bcppcompiler This module provides :class:`BorlandCCompiler`, a subclass of the abstract :class:`CCompiler` class for the Borland C++ compiler. :mod:`distutils.cygwinccompiler` --- Cygwin Compiler ==================================================== .. module:: distutils.cygwinccompiler This module provides the :class:`CygwinCCompiler` class, a subclass of :class:`UnixCCompiler` that handles the Cygwin port of the GNU C compiler to Windows. It also contains the Mingw32CCompiler class which handles the mingw32 port of GCC (same as cygwin in no-cygwin mode). :mod:`distutils.archive_util` --- Archiving utilities ====================================================== .. module:: distutils.archive_util :synopsis: Utility functions for creating archive files (tarballs, zip files, ...) This module provides a few functions for creating archive files, such as tarballs or zipfiles. .. function:: make_archive(base_name, format[, root_dir=None, base_dir=None, verbose=0, dry_run=0]) Create an archive file (eg. ``zip`` or ``tar``). *base_name* is the name of the file to create, minus any format-specific extension; *format* is the archive format: one of ``zip``, ``tar``, ``gztar``, ``bztar``, ``xztar``, or ``ztar``. *root_dir* is a directory that will be the root directory of the archive; ie. we typically ``chdir`` into *root_dir* before creating the archive. *base_dir* is the directory where we start archiving from; ie. *base_dir* will be the common prefix of all files and directories in the archive. *root_dir* and *base_dir* both default to the current directory. Returns the name of the archive file. .. versionchanged:: 3.5 Added support for the ``xztar`` format. .. function:: make_tarball(base_name, base_dir[, compress='gzip', verbose=0, dry_run=0]) 'Create an (optional compressed) archive as a tar file from all files in and under *base_dir*. *compress* must be ``'gzip'`` (the default), ``'bzip2'``, ``'xz'``, ``'compress'``, or ``None``. For the ``'compress'`` method the compression utility named by :program:`compress` must be on the default program search path, so this is probably Unix-specific. The output tar file will be named :file:`base_dir.tar`, possibly plus the appropriate compression extension (``.gz``, ``.bz2``, ``.xz`` or ``.Z``). Return the output filename. .. versionchanged:: 3.5 Added support for the ``xz`` compression. .. function:: make_zipfile(base_name, base_dir[, verbose=0, dry_run=0]) Create a zip file from all files in and under *base_dir*. The output zip file will be named *base_name* + :file:`.zip`. Uses either the :mod:`zipfile` Python module (if available) or the InfoZIP :file:`zip` utility (if installed and found on the default search path). If neither tool is available, raises :exc:`DistutilsExecError`. Returns the name of the output zip file. :mod:`distutils.dep_util` --- Dependency checking ================================================= .. module:: distutils.dep_util :synopsis: Utility functions for simple dependency checking This module provides functions for performing simple, timestamp-based dependency of files and groups of files; also, functions based entirely on such timestamp dependency analysis. .. function:: newer(source, target) Return true if *source* exists and is more recently modified than *target*, or if *source* exists and *target* doesn't. Return false if both exist and *target* is the same age or newer than *source*. Raise :exc:`DistutilsFileError` if *source* does not exist. .. function:: newer_pairwise(sources, targets) Walk two filename lists in parallel, testing if each source is newer than its corresponding target. Return a pair of lists (*sources*, *targets*) where source is newer than target, according to the semantics of :func:`newer`. .. % % equivalent to a listcomp... .. function:: newer_group(sources, target[, missing='error']) Return true if *target* is out-of-date with respect to any file listed in *sources*. In other words, if *target* exists and is newer than every file in *sources*, return false; otherwise return true. *missing* controls what we do when a source file is missing; the default (``'error'``) is to blow up with an :exc:`OSError` from inside :func:`os.stat`; if it is ``'ignore'``, we silently drop any missing source files; if it is ``'newer'``, any missing source files make us assume that *target* is out-of-date (this is handy in "dry-run" mode: it'll make you pretend to carry out commands that wouldn't work because inputs are missing, but that doesn't matter because you're not actually going to run the commands). :mod:`distutils.dir_util` --- Directory tree operations ======================================================= .. module:: distutils.dir_util :synopsis: Utility functions for operating on directories and directory trees This module provides functions for operating on directories and trees of directories. .. function:: mkpath(name[, mode=0o777, verbose=0, dry_run=0]) Create a directory and any missing ancestor directories. If the directory already exists (or if *name* is the empty string, which means the current directory, which of course exists), then do nothing. Raise :exc:`DistutilsFileError` if unable to create some directory along the way (eg. some sub-path exists, but is a file rather than a directory). If *verbose* is true, print a one-line summary of each mkdir to stdout. Return the list of directories actually created. .. function:: create_tree(base_dir, files[, mode=0o777, verbose=0, dry_run=0]) Create all the empty directories under *base_dir* needed to put *files* there. *base_dir* is just the name of a directory which doesn't necessarily exist yet; *files* is a list of filenames to be interpreted relative to *base_dir*. *base_dir* + the directory portion of every file in *files* will be created if it doesn't already exist. *mode*, *verbose* and *dry_run* flags are as for :func:`mkpath`. .. function:: copy_tree(src, dst[, preserve_mode=1, preserve_times=1, preserve_symlinks=0, update=0, verbose=0, dry_run=0]) Copy an entire directory tree *src* to a new location *dst*. Both *src* and *dst* must be directory names. If *src* is not a directory, raise :exc:`DistutilsFileError`. If *dst* does not exist, it is created with :func:`mkpath`. The end result of the copy is that every file in *src* is copied to *dst*, and directories under *src* are recursively copied to *dst*. Return the list of files that were copied or might have been copied, using their output name. The return value is unaffected by *update* or *dry_run*: it is simply the list of all files under *src*, with the names changed to be under *dst*. *preserve_mode* and *preserve_times* are the same as for :func:`distutils.file_util.copy_file`; note that they only apply to regular files, not to directories. If *preserve_symlinks* is true, symlinks will be copied as symlinks (on platforms that support them!); otherwise (the default), the destination of the symlink will be copied. *update* and *verbose* are the same as for :func:`~distutils.file_util.copy_file`. Files in *src* that begin with :file:`.nfs` are skipped (more information on these files is available in answer D2 of the `NFS FAQ page `_). .. versionchanged:: 3.3.1 NFS files are ignored. .. function:: remove_tree(directory[, verbose=0, dry_run=0]) Recursively remove *directory* and all files and directories underneath it. Any errors are ignored (apart from being reported to ``sys.stdout`` if *verbose* is true). :mod:`distutils.file_util` --- Single file operations ===================================================== .. module:: distutils.file_util :synopsis: Utility functions for operating on single files This module contains some utility functions for operating on individual files. .. function:: copy_file(src, dst[, preserve_mode=1, preserve_times=1, update=0, link=None, verbose=0, dry_run=0]) Copy file *src* to *dst*. If *dst* is a directory, then *src* is copied there with the same name; otherwise, it must be a filename. (If the file exists, it will be ruthlessly clobbered.) If *preserve_mode* is true (the default), the file's mode (type and permission bits, or whatever is analogous on the current platform) is copied. If *preserve_times* is true (the default), the last-modified and last-access times are copied as well. If *update* is true, *src* will only be copied if *dst* does not exist, or if *dst* does exist but is older than *src*. *link* allows you to make hard links (using :func:`os.link`) or symbolic links (using :func:`os.symlink`) instead of copying: set it to ``'hard'`` or ``'sym'``; if it is ``None`` (the default), files are copied. Don't set *link* on systems that don't support it: :func:`copy_file` doesn't check if hard or symbolic linking is available. It uses :func:`~distutils.file_util._copy_file_contents` to copy file contents. Return a tuple ``(dest_name, copied)``: *dest_name* is the actual name of the output file, and *copied* is true if the file was copied (or would have been copied, if *dry_run* true). .. % XXX if the destination file already exists, we clobber it if .. % copying, but blow up if linking. Hmmm. And I don't know what .. % macostools.copyfile() does. Should definitely be consistent, and .. % should probably blow up if destination exists and we would be .. % changing it (ie. it's not already a hard/soft link to src OR .. % (not update) and (src newer than dst)). .. function:: move_file(src, dst[, verbose, dry_run]) Move file *src* to *dst*. If *dst* is a directory, the file will be moved into it with the same name; otherwise, *src* is just renamed to *dst*. Returns the new full name of the file. .. warning:: Handles cross-device moves on Unix using :func:`copy_file`. What about other systems? .. function:: write_file(filename, contents) Create a file called *filename* and write *contents* (a sequence of strings without line terminators) to it. :mod:`distutils.util` --- Miscellaneous other utility functions =============================================================== .. module:: distutils.util :synopsis: Miscellaneous other utility functions This module contains other assorted bits and pieces that don't fit into any other utility module. .. function:: get_platform() Return a string that identifies the current platform. This is used mainly to distinguish platform-specific build directories and platform-specific built distributions. Typically includes the OS name and version and the architecture (as supplied by 'os.uname()'), although the exact information included depends on the OS; e.g., on Linux, the kernel version isn't particularly important. Examples of returned values: * ``linux-i586`` * ``linux-alpha`` * ``solaris-2.6-sun4u`` For non-POSIX platforms, currently just returns ``sys.platform``. For Mac OS X systems the OS version reflects the minimal version on which binaries will run (that is, the value of ``MACOSX_DEPLOYMENT_TARGET`` during the build of Python), not the OS version of the current system. For universal binary builds on Mac OS X the architecture value reflects the universal binary status instead of the architecture of the current processor. For 32-bit universal binaries the architecture is ``fat``, for 64-bit universal binaries the architecture is ``fat64``, and for 4-way universal binaries the architecture is ``universal``. Starting from Python 2.7 and Python 3.2 the architecture ``fat3`` is used for a 3-way universal build (ppc, i386, x86_64) and ``intel`` is used for a universal build with the i386 and x86_64 architectures Examples of returned values on Mac OS X: * ``macosx-10.3-ppc`` * ``macosx-10.3-fat`` * ``macosx-10.5-universal`` * ``macosx-10.6-intel`` For AIX, Python 3.9 and later return a string starting with "aix", followed by additional fields (separated by ``'-'``) that represent the combined values of AIX Version, Release and Technology Level (first field), Build Date (second field), and bit-size (third field). Python 3.8 and earlier returned only a single additional field with the AIX Version and Release. Examples of returned values on AIX: * ``aix-5307-0747-32`` # 32-bit build on AIX ``oslevel -s``: 5300-07-00-0000 * ``aix-7105-1731-64`` # 64-bit build on AIX ``oslevel -s``: 7100-05-01-1731 * ``aix-7.2`` # Legacy form reported in Python 3.8 and earlier .. versionchanged:: 3.9 The AIX platform string format now also includes the technology level, build date, and ABI bit-size. .. function:: convert_path(pathname) Return 'pathname' as a name that will work on the native filesystem, i.e. split it on '/' and put it back together again using the current directory separator. Needed because filenames in the setup script are always supplied in Unix style, and have to be converted to the local convention before we can actually use them in the filesystem. Raises :exc:`ValueError` on non-Unix-ish systems if *pathname* either starts or ends with a slash. .. function:: change_root(new_root, pathname) Return *pathname* with *new_root* prepended. If *pathname* is relative, this is equivalent to ``os.path.join(new_root,pathname)`` Otherwise, it requires making *pathname* relative and then joining the two, which is tricky on DOS/Windows. .. function:: check_environ() Ensure that 'os.environ' has all the environment variables we guarantee that users can use in config files, command-line options, etc. Currently this includes: * :envvar:`HOME` - user's home directory (Unix only) * :envvar:`PLAT` - description of the current platform, including hardware and OS (see :func:`get_platform`) .. function:: subst_vars(s, local_vars) Perform shell/Perl-style variable substitution on *s*. Every occurrence of ``$`` followed by a name is considered a variable, and variable is substituted by the value found in the *local_vars* dictionary, or in ``os.environ`` if it's not in *local_vars*. *os.environ* is first checked/augmented to guarantee that it contains certain values: see :func:`check_environ`. Raise :exc:`ValueError` for any variables not found in either *local_vars* or ``os.environ``. Note that this is not a fully-fledged string interpolation function. A valid ``$variable`` can consist only of upper and lower case letters, numbers and an underscore. No { } or ( ) style quoting is available. .. function:: split_quoted(s) Split a string up according to Unix shell-like rules for quotes and backslashes. In short: words are delimited by spaces, as long as those spaces are not escaped by a backslash, or inside a quoted string. Single and double quotes are equivalent, and the quote characters can be backslash-escaped. The backslash is stripped from any two-character escape sequence, leaving only the escaped character. The quote characters are stripped from any quoted string. Returns a list of words. .. % Should probably be moved into the standard library. .. function:: execute(func, args[, msg=None, verbose=0, dry_run=0]) Perform some action that affects the outside world (for instance, writing to the filesystem). Such actions are special because they are disabled by the *dry_run* flag. This method takes care of all that bureaucracy for you; all you have to do is supply the function to call and an argument tuple for it (to embody the "external action" being performed), and an optional message to print. .. function:: strtobool(val) Convert a string representation of truth to true (1) or false (0). True values are ``y``, ``yes``, ``t``, ``true``, ``on`` and ``1``; false values are ``n``, ``no``, ``f``, ``false``, ``off`` and ``0``. Raises :exc:`ValueError` if *val* is anything else. .. function:: byte_compile(py_files[, optimize=0, force=0, prefix=None, base_dir=None, verbose=1, dry_run=0, direct=None]) Byte-compile a collection of Python source files to :file:`.pyc` files in a :file:`__pycache__` subdirectory (see :pep:`3147` and :pep:`488`). *py_files* is a list of files to compile; any files that don't end in :file:`.py` are silently skipped. *optimize* must be one of the following: * ``0`` - don't optimize * ``1`` - normal optimization (like ``python -O``) * ``2`` - extra optimization (like ``python -OO``) If *force* is true, all files are recompiled regardless of timestamps. The source filename encoded in each :term:`bytecode` file defaults to the filenames listed in *py_files*; you can modify these with *prefix* and *basedir*. *prefix* is a string that will be stripped off of each source filename, and *base_dir* is a directory name that will be prepended (after *prefix* is stripped). You can supply either or both (or neither) of *prefix* and *base_dir*, as you wish. If *dry_run* is true, doesn't actually do anything that would affect the filesystem. Byte-compilation is either done directly in this interpreter process with the standard :mod:`py_compile` module, or indirectly by writing a temporary script and executing it. Normally, you should let :func:`byte_compile` figure out to use direct compilation or not (see the source for details). The *direct* flag is used by the script generated in indirect mode; unless you know what you're doing, leave it set to ``None``. .. versionchanged:: 3.2.3 Create ``.pyc`` files with an :func:`import magic tag ` in their name, in a :file:`__pycache__` subdirectory instead of files without tag in the current directory. .. versionchanged:: 3.5 Create ``.pyc`` files according to :pep:`488`. .. function:: rfc822_escape(header) Return a version of *header* escaped for inclusion in an :rfc:`822` header, by ensuring there are 8 spaces space after each newline. Note that it does no other modification of the string. .. % this _can_ be replaced .. % \subsection{Distutils objects} :mod:`distutils.dist` --- The Distribution class ================================================ .. module:: distutils.dist :synopsis: Provides the Distribution class, which represents the module distribution being built/installed/distributed This module provides the :class:`~distutils.core.Distribution` class, which represents the module distribution being built/installed/distributed. :mod:`distutils.extension` --- The Extension class ================================================== .. module:: distutils.extension :synopsis: Provides the Extension class, used to describe C/C++ extension modules in setup scripts This module provides the :class:`~distutils.extension.Extension` class, used to describe C/C++ extension modules in setup scripts. .. % \subsection{Ungrouped modules} .. % The following haven't been moved into a more appropriate section yet. :mod:`distutils.debug` --- Distutils debug mode =============================================== .. module:: distutils.debug :synopsis: Provides the debug flag for distutils This module provides the DEBUG flag. :mod:`distutils.errors` --- Distutils exceptions ================================================ .. module:: distutils.errors :synopsis: Provides standard distutils exceptions Provides exceptions used by the Distutils modules. Note that Distutils modules may raise standard exceptions; in particular, SystemExit is usually raised for errors that are obviously the end-user's fault (eg. bad command-line arguments). This module is safe to use in ``from ... import *`` mode; it only exports symbols whose names start with ``Distutils`` and end with ``Error``. :mod:`distutils.fancy_getopt` --- Wrapper around the standard getopt module =========================================================================== .. module:: distutils.fancy_getopt :synopsis: Additional getopt functionality This module provides a wrapper around the standard :mod:`getopt` module that provides the following additional features: * short and long options are tied together * options have help strings, so :func:`fancy_getopt` could potentially create a complete usage summary * options set attributes of a passed-in object * boolean options can have "negative aliases" --- eg. if :option:`!--quiet` is the "negative alias" of :option:`!--verbose`, then :option:`!--quiet` on the command line sets *verbose* to false. .. function:: fancy_getopt(options, negative_opt, object, args) Wrapper function. *options* is a list of ``(long_option, short_option, help_string)`` 3-tuples as described in the constructor for :class:`FancyGetopt`. *negative_opt* should be a dictionary mapping option names to option names, both the key and value should be in the *options* list. *object* is an object which will be used to store values (see the :meth:`~FancyGetopt.getopt` method of the :class:`FancyGetopt` class). *args* is the argument list. Will use ``sys.argv[1:]`` if you pass ``None`` as *args*. .. function:: wrap_text(text, width) Wraps *text* to less than *width* wide. .. class:: FancyGetopt([option_table=None]) The option_table is a list of 3-tuples: ``(long_option, short_option, help_string)`` If an option takes an argument, its *long_option* should have ``'='`` appended; *short_option* should just be a single character, no ``':'`` in any case. *short_option* should be ``None`` if a *long_option* doesn't have a corresponding *short_option*. All option tuples must have long options. The :class:`FancyGetopt` class provides the following methods: .. method:: FancyGetopt.getopt([args=None, object=None]) Parse command-line options in args. Store as attributes on *object*. If *args* is ``None`` or not supplied, uses ``sys.argv[1:]``. If *object* is ``None`` or not supplied, creates a new :class:`OptionDummy` instance, stores option values there, and returns a tuple ``(args, object)``. If *object* is supplied, it is modified in place and :func:`getopt` just returns *args*; in both cases, the returned *args* is a modified copy of the passed-in *args* list, which is left untouched. .. % and args returned are? .. method:: FancyGetopt.get_option_order() Returns the list of ``(option, value)`` tuples processed by the previous run of :meth:`getopt` Raises :exc:`RuntimeError` if :meth:`getopt` hasn't been called yet. .. method:: FancyGetopt.generate_help([header=None]) Generate help text (a list of strings, one per suggested line of output) from the option table for this :class:`FancyGetopt` object. If supplied, prints the supplied *header* at the top of the help. :mod:`distutils.filelist` --- The FileList class ================================================ .. module:: distutils.filelist :synopsis: The FileList class, used for poking about the file system and building lists of files. This module provides the :class:`FileList` class, used for poking about the filesystem and building lists of files. :mod:`distutils.log` --- Simple :pep:`282`-style logging ======================================================== .. module:: distutils.log :synopsis: A simple logging mechanism, :pep:`282`-style :mod:`distutils.spawn` --- Spawn a sub-process ============================================== .. module:: distutils.spawn :synopsis: Provides the spawn() function This module provides the :func:`~distutils.spawn.spawn` function, a front-end to various platform-specific functions for launching another program in a sub-process. Also provides :func:`~distutils.spawn.find_executable` to search the path for a given executable name. :mod:`distutils.sysconfig` --- System configuration information =============================================================== .. module:: distutils.sysconfig :synopsis: Low-level access to configuration information of the Python interpreter. .. moduleauthor:: Fred L. Drake, Jr. .. moduleauthor:: Greg Ward .. sectionauthor:: Fred L. Drake, Jr. The :mod:`distutils.sysconfig` module provides access to Python's low-level configuration information. The specific configuration variables available depend heavily on the platform and configuration. The specific variables depend on the build process for the specific version of Python being run; the variables are those found in the :file:`Makefile` and configuration header that are installed with Python on Unix systems. The configuration header is called :file:`pyconfig.h` for Python versions starting with 2.2, and :file:`config.h` for earlier versions of Python. Some additional functions are provided which perform some useful manipulations for other parts of the :mod:`distutils` package. .. data:: PREFIX The result of ``os.path.normpath(sys.prefix)``. .. data:: EXEC_PREFIX The result of ``os.path.normpath(sys.exec_prefix)``. .. function:: get_config_var(name) Return the value of a single variable. This is equivalent to ``get_config_vars().get(name)``. .. function:: get_config_vars(...) Return a set of variable definitions. If there are no arguments, this returns a dictionary mapping names of configuration variables to values. If arguments are provided, they should be strings, and the return value will be a sequence giving the associated values. If a given name does not have a corresponding value, ``None`` will be included for that variable. .. function:: get_config_h_filename() Return the full path name of the configuration header. For Unix, this will be the header generated by the :program:`configure` script; for other platforms the header will have been supplied directly by the Python source distribution. The file is a platform-specific text file. .. function:: get_makefile_filename() Return the full path name of the :file:`Makefile` used to build Python. For Unix, this will be a file generated by the :program:`configure` script; the meaning for other platforms will vary. The file is a platform-specific text file, if it exists. This function is only useful on POSIX platforms. .. function:: get_python_inc([plat_specific[, prefix]]) Return the directory for either the general or platform-dependent C include files. If *plat_specific* is true, the platform-dependent include directory is returned; if false or omitted, the platform-independent directory is returned. If *prefix* is given, it is used as either the prefix instead of :const:`PREFIX`, or as the exec-prefix instead of :const:`EXEC_PREFIX` if *plat_specific* is true. .. function:: get_python_lib([plat_specific[, standard_lib[, prefix]]]) Return the directory for either the general or platform-dependent library installation. If *plat_specific* is true, the platform-dependent include directory is returned; if false or omitted, the platform-independent directory is returned. If *prefix* is given, it is used as either the prefix instead of :const:`PREFIX`, or as the exec-prefix instead of :const:`EXEC_PREFIX` if *plat_specific* is true. If *standard_lib* is true, the directory for the standard library is returned rather than the directory for the installation of third-party extensions. The following function is only intended for use within the :mod:`distutils` package. .. function:: customize_compiler(compiler) Do any platform-specific customization of a :class:`distutils.ccompiler.CCompiler` instance. This function is only needed on Unix at this time, but should be called consistently to support forward-compatibility. It inserts the information that varies across Unix flavors and is stored in Python's :file:`Makefile`. This information includes the selected compiler, compiler and linker options, and the extension used by the linker for shared objects. This function is even more special-purpose, and should only be used from Python's own build procedures. .. function:: set_python_build() Inform the :mod:`distutils.sysconfig` module that it is being used as part of the build process for Python. This changes a lot of relative locations for files, allowing them to be located in the build area rather than in an installed Python. :mod:`distutils.text_file` --- The TextFile class ================================================= .. module:: distutils.text_file :synopsis: Provides the TextFile class, a simple interface to text files This module provides the :class:`TextFile` class, which gives an interface to text files that (optionally) takes care of stripping comments, ignoring blank lines, and joining lines with backslashes. .. class:: TextFile([filename=None, file=None, **options]) This class provides a file-like object that takes care of all the things you commonly want to do when processing a text file that has some line-by-line syntax: strip comments (as long as ``#`` is your comment character), skip blank lines, join adjacent lines by escaping the newline (ie. backslash at end of line), strip leading and/or trailing whitespace. All of these are optional and independently controllable. The class provides a :meth:`warn` method so you can generate warning messages that report physical line number, even if the logical line in question spans multiple physical lines. Also provides :meth:`unreadline` for implementing line-at-a-time lookahead. :class:`TextFile` instances are create with either *filename*, *file*, or both. :exc:`RuntimeError` is raised if both are ``None``. *filename* should be a string, and *file* a file object (or something that provides :meth:`readline` and :meth:`close` methods). It is recommended that you supply at least *filename*, so that :class:`TextFile` can include it in warning messages. If *file* is not supplied, :class:`TextFile` creates its own using the :func:`open` built-in function. The options are all boolean, and affect the values returned by :meth:`readline` .. tabularcolumns:: |l|L|l| +------------------+--------------------------------+---------+ | option name | description | default | +==================+================================+=========+ | *strip_comments* | strip from ``'#'`` to | true | | | end-of-line, as well as any | | | | whitespace leading up to the | | | | ``'#'``\ ---unless it is | | | | escaped by a backslash | | +------------------+--------------------------------+---------+ | *lstrip_ws* | strip leading whitespace from | false | | | each line before returning it | | +------------------+--------------------------------+---------+ | *rstrip_ws* | strip trailing whitespace | true | | | (including line terminator!) | | | | from each line before | | | | returning it. | | +------------------+--------------------------------+---------+ | *skip_blanks* | skip lines that are empty | true | | | \*after\* stripping comments | | | | and whitespace. (If both | | | | lstrip_ws and rstrip_ws are | | | | false, then some lines may | | | | consist of solely whitespace: | | | | these will \*not\* be skipped, | | | | even if *skip_blanks* is | | | | true.) | | +------------------+--------------------------------+---------+ | *join_lines* | if a backslash is the last | false | | | non-newline character on a | | | | line after stripping comments | | | | and whitespace, join the | | | | following line to it to form | | | | one logical line; if N | | | | consecutive lines end with a | | | | backslash, then N+1 physical | | | | lines will be joined to form | | | | one logical line. | | +------------------+--------------------------------+---------+ | *collapse_join* | strip leading whitespace from | false | | | lines that are joined to their | | | | predecessor; only matters if | | | | ``(join_lines and not | | | | lstrip_ws)`` | | +------------------+--------------------------------+---------+ Note that since *rstrip_ws* can strip the trailing newline, the semantics of :meth:`readline` must differ from those of the built-in file object's :meth:`readline` method! In particular, :meth:`readline` returns ``None`` for end-of-file: an empty string might just be a blank line (or an all-whitespace line), if *rstrip_ws* is true but *skip_blanks* is not. .. method:: TextFile.open(filename) Open a new file *filename*. This overrides any *file* or *filename* constructor arguments. .. method:: TextFile.close() Close the current file and forget everything we know about it (including the filename and the current line number). .. method:: TextFile.warn(msg[,line=None]) Print (to stderr) a warning message tied to the current logical line in the current file. If the current logical line in the file spans multiple physical lines, the warning refers to the whole range, such as ``"lines 3-5"``. If *line* is supplied, it overrides the current line number; it may be a list or tuple to indicate a range of physical lines, or an integer for a single physical line. .. method:: TextFile.readline() Read and return a single logical line from the current file (or from an internal buffer if lines have previously been "unread" with :meth:`unreadline`). If the *join_lines* option is true, this may involve reading multiple physical lines concatenated into a single string. Updates the current line number, so calling :meth:`warn` after :meth:`readline` emits a warning about the physical line(s) just read. Returns ``None`` on end-of-file, since the empty string can occur if *rstrip_ws* is true but *strip_blanks* is not. .. method:: TextFile.readlines() Read and return the list of all logical lines remaining in the current file. This updates the current line number to the last line of the file. .. method:: TextFile.unreadline(line) Push *line* (a string) onto an internal buffer that will be checked by future :meth:`readline` calls. Handy for implementing a parser with line-at-a-time lookahead. Note that lines that are "unread" with :meth:`unreadline` are not subsequently re-cleansed (whitespace stripped, or whatever) when read with :meth:`readline`. If multiple calls are made to :meth:`unreadline` before a call to :meth:`readline`, the lines will be returned most in most recent first order. :mod:`distutils.version` --- Version number classes =================================================== .. module:: distutils.version :synopsis: Implements classes that represent module version numbers. .. % todo .. % \section{Distutils Commands} .. % .. % This part of Distutils implements the various Distutils commands, such .. % as \code{build}, \code{install} \&c. Each command is implemented as a .. % separate module, with the command name as the name of the module. :mod:`distutils.cmd` --- Abstract base class for Distutils commands =================================================================== .. module:: distutils.cmd :synopsis: Provides the abstract base class :class:`~distutils.cmd.Command`. This class is subclassed by the modules in the distutils.command subpackage. This module supplies the abstract base class :class:`Command`. .. class:: Command(dist) Abstract base class for defining command classes, the "worker bees" of the Distutils. A useful analogy for command classes is to think of them as subroutines with local variables called *options*. The options are declared in :meth:`initialize_options` and defined (given their final values) in :meth:`finalize_options`, both of which must be defined by every command class. The distinction between the two is necessary because option values might come from the outside world (command line, config file, ...), and any options dependent on other options must be computed after these outside influences have been processed --- hence :meth:`finalize_options`. The body of the subroutine, where it does all its work based on the values of its options, is the :meth:`run` method, which must also be implemented by every command class. The class constructor takes a single argument *dist*, a :class:`~distutils.core.Distribution` instance. Creating a new Distutils command ================================ This section outlines the steps to create a new Distutils command. A new command lives in a module in the :mod:`distutils.command` package. There is a sample template in that directory called :file:`command_template`. Copy this file to a new module with the same name as the new command you're implementing. This module should implement a class with the same name as the module (and the command). So, for instance, to create the command ``peel_banana`` (so that users can run ``setup.py peel_banana``), you'd copy :file:`command_template` to :file:`distutils/command/peel_banana.py`, then edit it so that it's implementing the class ``peel_banana``, a subclass of :class:`distutils.cmd.Command`. Subclasses of :class:`Command` must define the following methods. .. method:: Command.initialize_options() Set default values for all the options that this command supports. Note that these defaults may be overridden by other commands, by the setup script, by config files, or by the command-line. Thus, this is not the place to code dependencies between options; generally, :meth:`initialize_options` implementations are just a bunch of ``self.foo = None`` assignments. .. method:: Command.finalize_options() Set final values for all the options that this command supports. This is always called as late as possible, ie. after any option assignments from the command-line or from other commands have been done. Thus, this is the place to code option dependencies: if *foo* depends on *bar*, then it is safe to set *foo* from *bar* as long as *foo* still has the same value it was assigned in :meth:`initialize_options`. .. method:: Command.run() A command's raison d'etre: carry out the action it exists to perform, controlled by the options initialized in :meth:`initialize_options`, customized by other commands, the setup script, the command-line, and config files, and finalized in :meth:`finalize_options`. All terminal output and filesystem interaction should be done by :meth:`run`. .. attribute:: Command.sub_commands *sub_commands* formalizes the notion of a "family" of commands, e.g. ``install`` as the parent with sub-commands ``install_lib``, ``install_headers``, etc. The parent of a family of commands defines *sub_commands* as a class attribute; it's a list of 2-tuples ``(command_name, predicate)``, with *command_name* a string and *predicate* a function, a string or ``None``. *predicate* is a method of the parent command that determines whether the corresponding command is applicable in the current situation. (E.g. ``install_headers`` is only applicable if we have any C header files to install.) If *predicate* is ``None``, that command is always applicable. *sub_commands* is usually defined at the *end* of a class, because predicates can be methods of the class, so they must already have been defined. The canonical example is the :command:`install` command. :mod:`distutils.command` --- Individual Distutils commands ========================================================== .. module:: distutils.command :synopsis: Contains one module for each standard Distutils command. .. % \subsubsection{Individual Distutils commands} .. % todo :mod:`distutils.command.bdist` --- Build a binary installer =========================================================== .. module:: distutils.command.bdist :synopsis: Build a binary installer for a package .. % todo :mod:`distutils.command.bdist_packager` --- Abstract base class for packagers ============================================================================= .. module:: distutils.command.bdist_packager :synopsis: Abstract base class for packagers .. % todo :mod:`distutils.command.bdist_dumb` --- Build a "dumb" installer ================================================================ .. module:: distutils.command.bdist_dumb :synopsis: Build a "dumb" installer - a simple archive of files .. % todo :mod:`distutils.command.bdist_msi` --- Build a Microsoft Installer binary package ================================================================================= .. module:: distutils.command.bdist_msi :synopsis: Build a binary distribution as a Windows MSI file .. class:: bdist_msi .. deprecated:: 3.9 Use bdist_wheel (wheel packages) instead. Builds a `Windows Installer`_ (.msi) binary package. .. _Windows Installer: https://msdn.microsoft.com/en-us/library/cc185688(VS.85).aspx In most cases, the ``bdist_msi`` installer is a better choice than the ``bdist_wininst`` installer, because it provides better support for Win64 platforms, allows administrators to perform non-interactive installations, and allows installation through group policies. :mod:`distutils.command.bdist_rpm` --- Build a binary distribution as a Redhat RPM and SRPM =========================================================================================== .. module:: distutils.command.bdist_rpm :synopsis: Build a binary distribution as a Redhat RPM and SRPM .. % todo :mod:`distutils.command.bdist_wininst` --- Build a Windows installer ==================================================================== .. module:: distutils.command.bdist_wininst :synopsis: Build a Windows installer .. deprecated:: 3.8 Use bdist_wheel (wheel packages) instead. .. % todo :mod:`distutils.command.sdist` --- Build a source distribution ============================================================== .. module:: distutils.command.sdist :synopsis: Build a source distribution .. % todo :mod:`distutils.command.build` --- Build all files of a package =============================================================== .. module:: distutils.command.build :synopsis: Build all files of a package .. % todo :mod:`distutils.command.build_clib` --- Build any C libraries in a package ========================================================================== .. module:: distutils.command.build_clib :synopsis: Build any C libraries in a package .. % todo :mod:`distutils.command.build_ext` --- Build any extensions in a package ======================================================================== .. module:: distutils.command.build_ext :synopsis: Build any extensions in a package .. % todo :mod:`distutils.command.build_py` --- Build the .py/.pyc files of a package =========================================================================== .. module:: distutils.command.build_py :synopsis: Build the .py/.pyc files of a package .. class:: build_py :mod:`distutils.command.build_scripts` --- Build the scripts of a package ========================================================================= .. module:: distutils.command.build_scripts :synopsis: Build the scripts of a package .. % todo :mod:`distutils.command.clean` --- Clean a package build area ============================================================= .. module:: distutils.command.clean :synopsis: Clean a package build area This command removes the temporary files created by :command:`build` and its subcommands, like intermediary compiled object files. With the ``--all`` option, the complete build directory will be removed. Extension modules built :ref:`in place ` will not be cleaned, as they are not in the build directory. :mod:`distutils.command.config` --- Perform package configuration ================================================================= .. module:: distutils.command.config :synopsis: Perform package configuration .. % todo :mod:`distutils.command.install` --- Install a package ====================================================== .. module:: distutils.command.install :synopsis: Install a package .. % todo :mod:`distutils.command.install_data` --- Install data files from a package =========================================================================== .. module:: distutils.command.install_data :synopsis: Install data files from a package .. % todo :mod:`distutils.command.install_headers` --- Install C/C++ header files from a package ====================================================================================== .. module:: distutils.command.install_headers :synopsis: Install C/C++ header files from a package .. % todo :mod:`distutils.command.install_lib` --- Install library files from a package ============================================================================= .. module:: distutils.command.install_lib :synopsis: Install library files from a package .. % todo :mod:`distutils.command.install_scripts` --- Install script files from a package ================================================================================ .. module:: distutils.command.install_scripts :synopsis: Install script files from a package .. % todo :mod:`distutils.command.register` --- Register a module with the Python Package Index ===================================================================================== .. module:: distutils.command.register :synopsis: Register a module with the Python Package Index The ``register`` command registers the package with the Python Package Index. This is described in more detail in :pep:`301`. .. % todo :mod:`distutils.command.check` --- Check the meta-data of a package =================================================================== .. module:: distutils.command.check :synopsis: Check the meta-data of a package The ``check`` command performs some tests on the meta-data of a package. For example, it verifies that all required meta-data are provided as the arguments passed to the :func:`~distutils.core.setup` function. .. % todo PK python setup.py build_ext --swig-opts="-modern -I../include" On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just means Windows message text (:file:`.mc`) files and resource definition (:file:`.rc`) files for Visual C++. These will be compiled to binary resource (:file:`.res`) files and linked into the executable. Preprocessor options -------------------- Three optional arguments to :class:`~distutils.core.Extension` will help if you need to specify include directories to search or preprocessor macros to define/undefine: ``include_dirs``, ``define_macros``, and ``undef_macros``. For example, if your extension requires header files in the :file:`include` directory under your distribution root, use the ``include_dirs`` option:: Extension('foo', ['foo.c'], include_dirs=['include']) You can specify absolute directories there; if you know that your extension will only be built on Unix systems with X11R6 installed to :file:`/usr`, you can get away with :: Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11']) You should avoid this sort of non-portable usage if you plan to distribute your code: it's probably better to write C code like :: #include If you need to include header files from some other Python extension, you can take advantage of the fact that header files are installed in a consistent way by the Distutils :command:`install_headers` command. For example, the Numerical Python header files are installed (on a standard Unix installation) to :file:`/usr/local/include/python1.5/Numerical`. (The exact location will differ according to your platform and Python installation.) Since the Python include directory---\ :file:`/usr/local/include/python1.5` in this case---is always included in the search path when building Python extensions, the best approach is to write C code like :: #include If you must put the :file:`Numerical` include directory right into your header search path, though, you can find that directory using the Distutils :mod:`distutils.sysconfig` module:: from distutils.sysconfig import get_python_inc incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical') setup(..., Extension(..., include_dirs=[incdir]), ) Even though this is quite portable---it will work on any Python installation, regardless of platform---it's probably easier to just write your C code in the sensible way. You can define and undefine pre-processor macros with the ``define_macros`` and ``undef_macros`` options. ``define_macros`` takes a list of ``(name, value)`` tuples, where ``name`` is the name of the macro to define (a string) and ``value`` is its value: either a string or ``None``. (Defining a macro ``FOO`` to ``None`` is the equivalent of a bare ``#define FOO`` in your C source: with most compilers, this sets ``FOO`` to the string ``1``.) ``undef_macros`` is just a list of macros to undefine. For example:: Extension(..., define_macros=[('NDEBUG', '1'), ('HAVE_STRFTIME', None)], undef_macros=['HAVE_FOO', 'HAVE_BAR']) is the equivalent of having this at the top of every C source file:: #define NDEBUG 1 #define HAVE_STRFTIME #undef HAVE_FOO #undef HAVE_BAR Library options --------------- You can also specify the libraries to link against when building your extension, and the directories to search for those libraries. The ``libraries`` option is a list of libraries to link against, ``library_dirs`` is a list of directories to search for libraries at link-time, and ``runtime_library_dirs`` is a list of directories to search for shared (dynamically loaded) libraries at run-time. For example, if you need to link against libraries known to be in the standard library search path on target systems :: Extension(..., libraries=['gdbm', 'readline']) If you need to link with libraries in a non-standard location, you'll have to include the location in ``library_dirs``:: Extension(..., library_dirs=['/usr/X11R6/lib'], libraries=['X11', 'Xt']) (Again, this sort of non-portable construct should be avoided if you intend to distribute your code.) .. XXX Should mention clib libraries here or somewhere else! Other options ------------- There are still some other options which can be used to handle special cases. The ``optional`` option is a boolean; if it is true, a build failure in the extension will not abort the build process, but instead simply not install the failing extension. The ``extra_objects`` option is a list of object files to be passed to the linker. These files must not have extensions, as the default extension for the compiler is used. ``extra_compile_args`` and ``extra_link_args`` can be used to specify additional command line options for the respective compiler and linker command lines. ``export_symbols`` is only useful on Windows. It can contain a list of symbols (functions or variables) to be exported. This option is not needed when building compiled extensions: Distutils will automatically add ``initmodule`` to the list of exported symbols. The ``depends`` option is a list of files that the extension depends on (for example header files). The build command will call the compiler on the sources to rebuild extension if any on this files has been modified since the previous build. Relationships between Distributions and Packages ================================================ A distribution may relate to packages in three specific ways: #. It can require packages or modules. #. It can provide packages or modules. #. It can obsolete packages or modules. These relationships can be specified using keyword arguments to the :func:`distutils.core.setup` function. Dependencies on other Python modules and packages can be specified by supplying the *requires* keyword argument to :func:`~distutils.core.setup`. The value must be a list of strings. Each string specifies a package that is required, and optionally what versions are sufficient. To specify that any version of a module or package is required, the string should consist entirely of the module or package name. Examples include ``'mymodule'`` and ``'xml.parsers.expat'``. If specific versions are required, a sequence of qualifiers can be supplied in parentheses. Each qualifier may consist of a comparison operator and a version number. The accepted comparison operators are:: < > == <= >= != These can be combined by using multiple qualifiers separated by commas (and optional whitespace). In this case, all of the qualifiers must be matched; a logical AND is used to combine the evaluations. Let's look at a bunch of examples: +-------------------------+----------------------------------------------+ | Requires Expression | Explanation | +=========================+==============================================+ | ``==1.0`` | Only version ``1.0`` is compatible | +-------------------------+----------------------------------------------+ | ``>1.0, !=1.5.1, <2.0`` | Any version after ``1.0`` and before ``2.0`` | | | is compatible, except ``1.5.1`` | +-------------------------+----------------------------------------------+ Now that we can specify dependencies, we also need to be able to specify what we provide that other distributions can require. This is done using the *provides* keyword argument to :func:`~distutils.core.setup`. The value for this keyword is a list of strings, each of which names a Python module or package, and optionally identifies the version. If the version is not specified, it is assumed to match that of the distribution. Some examples: +---------------------+----------------------------------------------+ | Provides Expression | Explanation | +=====================+==============================================+ | ``mypkg`` | Provide ``mypkg``, using the distribution | | | version | +---------------------+----------------------------------------------+ | ``mypkg (1.1)`` | Provide ``mypkg`` version 1.1, regardless of | | | the distribution version | +---------------------+----------------------------------------------+ A package can declare that it obsoletes other packages using the *obsoletes* keyword argument. The value for this is similar to that of the *requires* keyword: a list of strings giving module or package specifiers. Each specifier consists of a module or package name optionally followed by one or more version qualifiers. Version qualifiers are given in parentheses after the module or package name. The versions identified by the qualifiers are those that are obsoleted by the distribution being described. If no qualifiers are given, all versions of the named module or package are understood to be obsoleted. .. _distutils-installing-scripts: Installing Scripts ================== So far we have been dealing with pure and non-pure Python modules, which are usually not run by themselves but imported by scripts. Scripts are files containing Python source code, intended to be started from the command line. Scripts don't require Distutils to do anything very complicated. The only clever feature is that if the first line of the script starts with ``#!`` and contains the word "python", the Distutils will adjust the first line to refer to the current interpreter location. By default, it is replaced with the current interpreter location. The :option:`!--executable` (or :option:`!-e`) option will allow the interpreter path to be explicitly overridden. The ``scripts`` option simply is a list of files to be handled in this way. From the PyXML setup script:: setup(..., scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val'] ) .. versionchanged:: 3.1 All the scripts will also be added to the ``MANIFEST`` file if no template is provided. See :ref:`manifest`. .. _distutils-installing-package-data: Installing Package Data ======================= Often, additional files need to be installed into a package. These files are often data that's closely related to the package's implementation, or text files containing documentation that might be of interest to programmers using the package. These files are called :dfn:`package data`. Package data can be added to packages using the ``package_data`` keyword argument to the :func:`~distutils.core.setup` function. The value must be a mapping from package name to a list of relative path names that should be copied into the package. The paths are interpreted as relative to the directory containing the package (information from the ``package_dir`` mapping is used if appropriate); that is, the files are expected to be part of the package in the source directories. They may contain glob patterns as well. The path names may contain directory portions; any necessary directories will be created in the installation. For example, if a package should contain a subdirectory with several data files, the files can be arranged like this in the source tree:: setup.py src/ mypkg/ __init__.py module.py data/ tables.dat spoons.dat forks.dat The corresponding call to :func:`~distutils.core.setup` might be:: setup(..., packages=['mypkg'], package_dir={'mypkg': 'src/mypkg'}, package_data={'mypkg': ['data/*.dat']}, ) .. versionchanged:: 3.1 All the files that match ``package_data`` will be added to the ``MANIFEST`` file if no template is provided. See :ref:`manifest`. .. _distutils-additional-files: Installing Additional Files =========================== The ``data_files`` option can be used to specify additional files needed by the module distribution: configuration files, message catalogs, data files, anything which doesn't fit in the previous categories. ``data_files`` specifies a sequence of (*directory*, *files*) pairs in the following way:: setup(..., data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']), ('config', ['cfg/data.cfg'])], ) Each (*directory*, *files*) pair in the sequence specifies the installation directory and the files to install there. Each file name in *files* is interpreted relative to the :file:`setup.py` script at the top of the package source distribution. Note that you can specify the directory where the data files will be installed, but you cannot rename the data files themselves. The *directory* should be a relative path. It is interpreted relative to the installation prefix (Python's ``sys.prefix`` for system installations; ``site.USER_BASE`` for user installations). Distutils allows *directory* to be an absolute installation path, but this is discouraged since it is incompatible with the wheel packaging format. No directory information from *files* is used to determine the final location of the installed file; only the name of the file is used. You can specify the ``data_files`` options as a simple sequence of files without specifying a target directory, but this is not recommended, and the :command:`install` command will print a warning in this case. To install data files directly in the target directory, an empty string should be given as the directory. .. versionchanged:: 3.1 All the files that match ``data_files`` will be added to the ``MANIFEST`` file if no template is provided. See :ref:`manifest`. .. _meta-data: Additional meta-data ==================== The setup script may include additional meta-data beyond the name and version. This information includes: +----------------------+---------------------------+-----------------+--------+ | Meta-Data | Description | Value | Notes | +======================+===========================+=================+========+ | ``name`` | name of the package | short string | \(1) | +----------------------+---------------------------+-----------------+--------+ | ``version`` | version of this release | short string | (1)(2) | +----------------------+---------------------------+-----------------+--------+ | ``author`` | package author's name | short string | \(3) | +----------------------+---------------------------+-----------------+--------+ | ``author_email`` | email address of the | email address | \(3) | | | package author | | | +----------------------+---------------------------+-----------------+--------+ | ``maintainer`` | package maintainer's name | short string | \(3) | +----------------------+---------------------------+-----------------+--------+ | ``maintainer_email`` | email address of the | email address | \(3) | | | package maintainer | | | +----------------------+---------------------------+-----------------+--------+ | ``url`` | home page for the package | URL | \(1) | +----------------------+---------------------------+-----------------+--------+ | ``description`` | short, summary | short string | | | | description of the | | | | | package | | | +----------------------+---------------------------+-----------------+--------+ | ``long_description`` | longer description of the | long string | \(4) | | | package | | | +----------------------+---------------------------+-----------------+--------+ | ``download_url`` | location where the | URL | | | | package may be downloaded | | | +----------------------+---------------------------+-----------------+--------+ | ``classifiers`` | a list of classifiers | list of strings | (6)(7) | +----------------------+---------------------------+-----------------+--------+ | ``platforms`` | a list of platforms | list of strings | (6)(8) | +----------------------+---------------------------+-----------------+--------+ | ``keywords`` | a list of keywords | list of strings | (6)(8) | +----------------------+---------------------------+-----------------+--------+ | ``license`` | license for the package | short string | \(5) | +----------------------+---------------------------+-----------------+--------+ Notes: (1) These fields are required. (2) It is recommended that versions take the form *major.minor[.patch[.sub]]*. (3) Either the author or the maintainer must be identified. If maintainer is provided, distutils lists it as the author in :file:`PKG-INFO`. (4) The ``long_description`` field is used by PyPI when you publish a package, to build its project page. (5) The ``license`` field is a text indicating the license covering the package where the license is not a selection from the "License" Trove classifiers. See the ``Classifier`` field. Notice that there's a ``licence`` distribution option which is deprecated but still acts as an alias for ``license``. (6) This field must be a list. (7) The valid classifiers are listed on `PyPI `_. (8) To preserve backward compatibility, this field also accepts a string. If you pass a comma-separated string ``'foo, bar'``, it will be converted to ``['foo', 'bar']``, Otherwise, it will be converted to a list of one string. 'short string' A single line of text, not more than 200 characters. 'long string' Multiple lines of plain text in reStructuredText format (see http://docutils.sourceforge.net/). 'list of strings' See below. Encoding the version information is an art in itself. Python packages generally adhere to the version format *major.minor[.patch][sub]*. The major number is 0 for initial, experimental releases of software. It is incremented for releases that represent major milestones in a package. The minor number is incremented when important new features are added to the package. The patch number increments when bug-fix releases are made. Additional trailing version information is sometimes used to indicate sub-releases. These are "a1,a2,...,aN" (for alpha releases, where functionality and API may change), "b1,b2,...,bN" (for beta releases, which only fix bugs) and "pr1,pr2,...,prN" (for final pre-release release testing). Some examples: 0.1.0 the first, experimental release of a package 1.0.1a2 the second alpha release of the first patch version of 1.0 ``classifiers`` must be specified in a list:: setup(..., classifiers=[ 'Development Status :: 4 - Beta', 'Environment :: Console', 'Environment :: Web Environment', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: Python Software Foundation License', 'Operating System :: MacOS :: MacOS X', 'Operating System :: Microsoft :: Windows', 'Operating System :: POSIX', 'Programming Language :: Python', 'Topic :: Communications :: Email', 'Topic :: Office/Business', 'Topic :: Software Development :: Bug Tracking', ], ) .. versionchanged:: 3.7 :class:`~distutils.core.setup` now warns when ``classifiers``, ``keywords`` or ``platforms`` fields are not specified as a list or a string. .. _debug-setup-script: Debugging the setup script ========================== Sometimes things go wrong, and the setup script doesn't do what the developer wants. Distutils catches any exceptions when running the setup script, and print a simple error message before the script is terminated. The motivation for this behaviour is to not confuse administrators who don't know much about Python and are trying to install a package. If they get a big long traceback from deep inside the guts of Distutils, they may think the package or the Python installation is broken because they don't read all the way down to the bottom and see that it's a permission problem. On the other hand, this doesn't help the developer to find the cause of the failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set to anything except an empty string, and distutils will now print detailed information about what it is doing, dump the full traceback when an exception occurs, and print the whole command line when an external program (like a C compiler) fails. PK python setup.py bdist_rpm --spec-only .. % # ...edit dist/FooBar-1.0.spec .. % > python setup.py bdist_rpm --spec-file=dist/FooBar-1.0.spec .. % \ end{verbatim} .. % .. % (Although a better way to do this is probably to override the standard .. % \command{bdist\_rpm} command with one that writes whatever else you want .. % to the \file{.spec} file.) .. _creating-wininst: Creating Windows Installers =========================== .. warning:: bdist_wininst is deprecated since Python 3.8. .. warning:: bdist_msi is deprecated since Python 3.9. Executable installers are the natural format for binary distributions on Windows. They display a nice graphical user interface, display some information about the module distribution to be installed taken from the metadata in the setup script, let the user select a few options, and start or cancel the installation. Since the metadata is taken from the setup script, creating Windows installers is usually as easy as running:: python setup.py bdist_wininst or the :command:`bdist` command with the :option:`!--formats` option:: python setup.py bdist --formats=wininst If you have a pure module distribution (only containing pure Python modules and packages), the resulting installer will be version independent and have a name like :file:`foo-1.0.win32.exe`. Note that creating ``wininst`` binary distributions in only supported on Windows systems. If you have a non-pure distribution, the extensions can only be created on a Windows platform, and will be Python version dependent. The installer filename will reflect this and now has the form :file:`foo-1.0.win32-py2.0.exe`. You have to create a separate installer for every Python version you want to support. The installer will try to compile pure modules into :term:`bytecode` after installation on the target system in normal and optimizing mode. If you don't want this to happen for some reason, you can run the :command:`bdist_wininst` command with the :option:`!--no-target-compile` and/or the :option:`!--no-target-optimize` option. By default the installer will display the cool "Python Powered" logo when it is run, but you can also supply your own 152x261 bitmap which must be a Windows :file:`.bmp` file with the :option:`!--bitmap` option. The installer will also display a large title on the desktop background window when it is run, which is constructed from the name of your distribution and the version number. This can be changed to another text by using the :option:`!--title` option. The installer file will be written to the "distribution directory" --- normally :file:`dist/`, but customizable with the :option:`!--dist-dir` option. .. _cross-compile-windows: Cross-compiling on Windows ========================== Starting with Python 2.6, distutils is capable of cross-compiling between Windows platforms. In practice, this means that with the correct tools installed, you can use a 32bit version of Windows to create 64bit extensions and vice-versa. To build for an alternate platform, specify the :option:`!--plat-name` option to the build command. Valid values are currently 'win32', and 'win-amd64'. For example, on a 32bit version of Windows, you could execute:: python setup.py build --plat-name=win-amd64 to build a 64bit version of your extension. The Windows Installers also support this option, so the command:: python setup.py build --plat-name=win-amd64 bdist_wininst would create a 64bit installation executable on your 32bit version of Windows. To cross-compile, you must download the Python source code and cross-compile Python itself for the platform you are targeting - it is not possible from a binary installation of Python (as the .lib etc file for other platforms are not included.) In practice, this means the user of a 32 bit operating system will need to use Visual Studio 2008 to open the :file:`PCbuild/PCbuild.sln` solution in the Python source tree and build the "x64" configuration of the 'pythoncore' project before cross-compiling extensions is possible. Note that by default, Visual Studio 2008 does not install 64bit compilers or tools. You may need to reexecute the Visual Studio setup process and select these tools (using Control Panel->[Add/Remove] Programs is a convenient way to check or modify your existing install.) .. _postinstallation-script: The Postinstallation script --------------------------- Starting with Python 2.3, a postinstallation script can be specified with the :option:`!--install-script` option. The basename of the script must be specified, and the script filename must also be listed in the scripts argument to the setup function. This script will be run at installation time on the target system after all the files have been copied, with ``argv[1]`` set to :option:`!-install`, and again at uninstallation time before the files are removed with ``argv[1]`` set to :option:`!-remove`. The installation script runs embedded in the windows installer, every output (``sys.stdout``, ``sys.stderr``) is redirected into a buffer and will be displayed in the GUI after the script has finished. Some functions especially useful in this context are available as additional built-in functions in the installation script. .. function:: directory_created(path) file_created(path) These functions should be called when a directory or file is created by the postinstall script at installation time. It will register *path* with the uninstaller, so that it will be removed when the distribution is uninstalled. To be safe, directories are only removed if they are empty. .. function:: get_special_folder_path(csidl_string) This function can be used to retrieve special folder locations on Windows like the Start Menu or the Desktop. It returns the full path to the folder. *csidl_string* must be one of the following strings:: "CSIDL_APPDATA" "CSIDL_COMMON_STARTMENU" "CSIDL_STARTMENU" "CSIDL_COMMON_DESKTOPDIRECTORY" "CSIDL_DESKTOPDIRECTORY" "CSIDL_COMMON_STARTUP" "CSIDL_STARTUP" "CSIDL_COMMON_PROGRAMS" "CSIDL_PROGRAMS" "CSIDL_FONTS" If the folder cannot be retrieved, :exc:`OSError` is raised. Which folders are available depends on the exact Windows version, and probably also the configuration. For details refer to Microsoft's documentation of the :c:func:`SHGetSpecialFolderPath` function. .. function:: create_shortcut(target, description, filename[, arguments[, workdir[, iconpath[, iconindex]]]]) This function creates a shortcut. *target* is the path to the program to be started by the shortcut. *description* is the description of the shortcut. *filename* is the title of the shortcut that the user will see. *arguments* specifies the command line arguments, if any. *workdir* is the working directory for the program. *iconpath* is the file containing the icon for the shortcut, and *iconindex* is the index of the icon in the file *iconpath*. Again, for details consult the Microsoft documentation for the :class:`IShellLink` interface. Vista User Access Control (UAC) =============================== Starting with Python 2.6, bdist_wininst supports a :option:`!--user-access-control` option. The default is 'none' (meaning no UAC handling is done), and other valid values are 'auto' (meaning prompt for UAC elevation if Python was installed for all users) and 'force' (meaning always prompt for elevation). .. note:: bdist_wininst is deprecated since Python 3.8. .. note:: bdist_msi is deprecated since Python 3.9. PK doc_files = CHANGES.txt README.txt USAGE.txt doc/ examples/ Note that the ``doc_files`` option is simply a whitespace-separated string split across multiple lines for readability. .. seealso:: :ref:`inst-config-syntax` in "Installing Python Modules" More information on the configuration files is available in the manual for system administrators. .. rubric:: Footnotes .. [#] This ideal probably won't be achieved until auto-configuration is fully supported by the Distutils. PK`__ in the Python Packaging User Guide for more information. This document describes the Python Distribution Utilities ("Distutils") from the module developer's point of view, describing the underlying capabilities that ``setuptools`` builds on to allow Python developers to make Python modules and extensions readily available to a wider audience. .. toctree:: :maxdepth: 2 :numbered: introduction.rst setupscript.rst configfile.rst sourcedist.rst builtdist.rst examples.rst extending.rst commandref.rst apiref.rst PKyq!!*docs/deprecated/distutils/introduction.rstnu[PK