EasyInstall/setuptools 0.5a4: significant new features, including automatic

installation of dependencies, the ability to specify dependencies in a
setup script, and several new options to control EasyInstall's behavior.

--HG--
branch : setuptools
extra : convert_revision : svn%3A6015fed2-1504-0410-9fe1-9d1591cc4771/sandbox/trunk/setuptools%4041073

9 files changed, 486 insertions, 208 deletions

diff --git a/EasyInstall.txt b/EasyInstall.txt
index 39301540..95b14b4a 100755
--- a/EasyInstall.txt
+++ b/EasyInstall.txt
@@ -23,7 +23,7 @@ Installing "Easy Install"
 -------------------------
 
 Windows users can just download and run the `setuptools binary installer for
-Windows <http://peak.telecommunity.com/dist/setuptools-0.5a3.win32.exe>`_.
+Windows <http://peak.telecommunity.com/dist/setuptools-0.5a4.win32.exe>`_.
 All others should just download `ez_setup.py
 <http://peak.telecommunity.com/dist/ez_setup.py>`_, and run it; this will
 download and install the correct version of ``setuptools`` for your Python
@@ -62,7 +62,7 @@ version, and automatically downloading, building, and installing it::
 **Example 2**. Install or upgrade a package by name and version by finding
 links on a given "download page"::
 
-    easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a3"
+    easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a4"
 
 **Example 3**. Download a source distribution from a specified URL,
 automatically building and installing it::
@@ -73,6 +73,11 @@ automatically building and installing it::
 
     easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg
 
+**Example 5**.  Upgrade an already-installed package to the latest version
+listed on PyPI:
+
+    easy_install --upgrade PyProtocols
+
 Easy Install accepts URLs, filenames, PyPI package names (i.e., ``distutils``
 "distribution" names), and package+version specifiers.  In each case, it will
 attempt to locate the latest available version that meets your criteria.
@@ -118,23 +123,29 @@ a version greater than the one you have now::
 
     easy_install "SomePackage>2.0"
 
+using the upgrade flag, to find the latest available version on PyPI::
+
+    easy_install --upgrade SomePackage
+
 or by using a download page, direct download URL, or package filename::
 
-    easy_install -s http://example.com/downloads ExamplePackage
+    easy_install -f http://example.com/downloads ExamplePackage
 
     easy_install http://example.com/downloads/ExamplePackage-2.0-py2.4.egg
 
     easy_install my_downloads/ExamplePackage-2.0.tgz
 
 If you're using ``-m`` or ``--multi`` (or installing outside of
-``site-packages``), the ``require()`` function automatically selects the newest
-available version of a package that meets your version criteria at runtime, so
-installation is the only step needed.
+``site-packages``), using the ``require()`` function at runtime automatically
+selects the newest installed version of a package that meets your version
+criteria.  So, installing a newer version is the only step needed to upgrade
+such packages.
 
-If you're installing to ``site-packages`` and not using ``-m``, installing a
-package automatically replaces any previous version in the ``easy-install.pth``
-file, so that Python will import the most-recently installed version by
-default.
+If you're installing to Python's ``site-packages`` directory (and not
+using ``-m``), installing a package automatically replaces any previous version
+in the ``easy-install.pth`` file, so that Python will import the most-recently
+installed version by default.  So, again, installing the newer version is the
+only upgrade step needed.
 
 If you haven't suppressed script installation (using ``--exclude-scripts`` or
 ``-x``), then the upgraded version's scripts will be installed, and they will
@@ -339,6 +350,16 @@ Command-Line Options
     the ``--install-dir`` or ``-d`` option (or they are set via configuration
     file(s)) you must also use ``require()`` to enable packages at runtime.
 
+``--upgrade, -U``   (New in 0.5a4)
+    By default, EasyInstall only searches the Python Package Index if a
+    project/version requirement can't be met by distributions already installed
+    on sys.path or the installation directory.  However, if you supply the
+    ``--upgrade`` or ``-U`` flag, EasyInstall will always check the package
+    index before selecting a version to install.  In this way, you can force
+    EasyInstall to use the latest available version of any package it installs
+    (subject to any version requirements that might exclude such later
+    versions).
+
 ``--install-dir=DIR, -d DIR``
     Set the installation directory. It is up to you to ensure that this
     directory is on ``sys.path`` at runtime, and to use
@@ -366,6 +387,14 @@ Command-Line Options
     versions of a package, but do not want to reset the version that will be
     run by scripts that are already installed.
 
+``--always-copy, -a``   (New in 0.5a4)
+    Copy all needed distributions to the installation directory, even if they
+    are already present in a directory on sys.path.  In older versions of
+    EasyInstall, this was the default behavior, but now you must explicitly
+    request it.  By default, EasyInstall will no longer copy such distributions
+    from other sys.path directories to the installation directory, unless you
+    explicitly gave the distribution's filename on the command line.
+
 ``--find-links=URL, -f URL`` (Option renamed in 0.4a2)
     Scan the specified "download pages" for direct links to downloadable eggs
     or source distributions.  Any usable packages will be downloaded if they
@@ -434,6 +463,12 @@ Command-Line Options
     the default is 0 (unless it's set under ``install`` or ``install_lib`` in
     one of your distutils configuration files).
 
+``--record=FILENAME``  (New in 0.5a4)
+    Write a record of all installed files to FILENAME.  This is basically the
+    same as the same option for the standard distutils "install" command, and
+    is included for compatibility with tools that expect to pass this option
+    to "setup.py install".
+
 
 Release Notes/Change History
 ============================
@@ -442,6 +477,46 @@ Known Issues
  * There's no automatic retry for borked Sourceforge mirrors, which can easily
    time out or be missing a file.
 
+0.5a4
+ * Added ``--always-copy/-a`` option to always copy needed packages to the
+   installation directory, even if they're already present elsewhere on
+   sys.path. (In previous versions, this was the default behavior, but now
+   you must request it.)
+ 
+ * Added ``--upgrade/-U`` option to force checking PyPI for latest available
+   version(s) of all packages requested by name and version, even if a matching
+   version is available locally.
+
+ * Setup scripts using setuptools can now list their dependencies directly in
+   the setup.py file, without having to manually create a ``depends.txt`` file.
+   The ``install_requires`` and ``extras_require`` arguments to ``setup()``
+   are used to create a dependencies file automatically.  If you are manually
+   creating ``depends.txt`` right now, please switch to using these setup
+   arguments as soon as practical, because ``depends.txt`` support will be
+   removed in the 0.6 release cycle.  For documentation on the new arguments,
+   see the ``setuptools.dist.Distribution`` class.
+
+ * Added automatic installation of dependencies declared by a distribution
+   being installed.  These dependencies must be listed in the distribution's
+   ``EGG-INFO`` directory, so the distribution has to have declared its
+   dependencies by using setuptools.  If a package has requirements it didn't
+   declare, you'll still have to deal with them yourself.  (E.g., by asking
+   EasyInstall to find and install them.)
+
+ * Setup scripts using setuptools now always install using ``easy_install``
+   internally, for ease of uninstallation and upgrading.  Note: you *must*
+   remove any ``extra_path`` argument from your setup script, as it conflicts
+   with the proper functioning of the ``easy_install`` command.  (Also, added   
+   the ``--record`` option to ``easy_install`` for the benefit of tools that
+   run ``setup.py install --record=filename`` on behalf of another packaging
+   system.)
+
+ * ``pkg_resources.AvailableDistributions.resolve()`` and related methods now
+   accept an ``installer`` argument: a callable taking one argument, a
+   ``Requirement`` instance.  The callable must return a ``Distribution``
+   object, or ``None`` if no distribution is found.  This feature is used by
+   EasyInstall to resolve dependencies by recursively invoking itself.
+
 0.5a3
  * Fixed not setting script permissions to allow execution.
  
@@ -584,10 +659,8 @@ Known Issues
 Future Plans
 ============
 
-* Process the installed package's dependencies as well as the base package
 * Additional utilities to list/remove/verify packages
 * Signature checking?  SSL?  Ability to suppress PyPI search?
 * Display byte progress meter when downloading distributions and long pages?
 * Redirect stdout/stderr to log during run_setup?
 
-
diff --git a/easy_install.py b/easy_install.py
index 689ee6ba..40950541 100755
--- a/easy_install.py
+++ b/easy_install.py
@@ -9,7 +9,6 @@ packages.  For detailed documentation, see the accompanying EasyInstall.txt
 file, or visit the `EasyInstall home page`__.
 
 __ http://peak.telecommunity.com/DevCenter/EasyInstall
-
 """
 
 import sys, os.path, zipimport, shutil, tempfile, zipfile
@@ -21,6 +20,7 @@ from distutils.sysconfig import get_python_lib
 from distutils.errors import DistutilsArgError, DistutilsOptionError
 from setuptools.archive_util import unpack_archive
 from setuptools.package_index import PackageIndex, parse_bdist_wininst
+from setuptools.package_index import URL_SCHEME
 from setuptools.command import bdist_egg
 from pkg_resources import *
 
@@ -47,9 +47,11 @@ class easy_install(Command):
     user_options = [
         ("zip-ok", "z", "install package as a zipfile"),
         ("multi-version", "m", "make apps have to require() a version"),
+        ("upgrade", "U", "force upgrade (searches PyPI for latest versions)"),
         ("install-dir=", "d", "install package to DIR"),
         ("script-dir=", "s", "install scripts to DIR"),
         ("exclude-scripts", "x", "Don't install scripts"),
+        ("always-copy", "a", "Copy all needed packages to install dir"),
         ("index-url=", "i", "base URL of Python Package Index"),
         ("find-links=", "f", "additional URL(s) to search for packages"),
         ("build-directory=", "b",
@@ -57,29 +59,27 @@ class easy_install(Command):
         ('optimize=', 'O',
          "also compile with optimization: -O1 for \"python -O\", "
          "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"),
+        ('record=', None,
+         "filename in which to record list of installed files"),
+    ]
+    boolean_options = [
+        'zip-ok', 'multi-version', 'exclude-scripts', 'upgrade', 'always-copy'
     ]
-
-    boolean_options = [ 'zip-ok', 'multi-version', 'exclude-scripts' ]
     create_index = PackageIndex
 
     def initialize_options(self):
         self.zip_ok = None
-        self.multi_version = None
         self.install_dir = self.script_dir = self.exclude_scripts = None
         self.index_url = None
         self.find_links = None
         self.build_directory = None
         self.args = None
-        self.optimize = None
-
+        self.optimize = self.record = None
+        self.upgrade = self.always_copy = self.multi_version = None
         # Options not specifiable via command line
         self.package_index = None
         self.pth_file = None
 
-
-
-
-
     def finalize_options(self):
         # If a non-default installation directory was specified, default the
         # script directory to match it.
@@ -96,6 +96,8 @@ class easy_install(Command):
         self.set_undefined_options('install_scripts',
             ('install_dir', 'script_dir')
         )
+        # default --record from the install command
+        self.set_undefined_options('install', ('record', 'record'))
 
         site_packages = get_python_lib()
         instdir = self.install_dir
@@ -118,8 +120,19 @@ class easy_install(Command):
             )
 
         self.index_url = self.index_url or "http://www.python.org/pypi"
+
+        self.shadow_path = sys.path[:]
+
+        for path_item in self.install_dir, self.script_dir:
+            if path_item not in self.shadow_path:
+                self.shadow_path.insert(0, self.install_dir)
+        
         if self.package_index is None:
-            self.package_index = self.create_index(self.index_url)
+            self.package_index = self.create_index(
+                self.index_url, search_path = self.shadow_path
+            )
+
+        self.local_index = AvailableDistributions(self.shadow_path)
 
         if self.find_links is not None:
             if isinstance(self.find_links, basestring):
@@ -145,20 +158,7 @@ class easy_install(Command):
                 "Build directory can only be set when using one URL"
             )
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+        self.outputs = []
 
 
 
@@ -179,30 +179,112 @@ class easy_install(Command):
                 self.package_index.scan_url(link)
             for spec in self.args:
                 self.easy_install(spec)
+            if self.record:
+                from distutils import file_util
+                self.execute(
+                    file_util.write_file, (self.record, self.outputs),
+                    "writing list of installed files to '%s'" %
+                    self.record
+                )
         finally:
             log.set_verbosity(self.distribution.verbose)
 
 
+    def add_output(self, path):
+        if os.path.isdir(path):
+            for base, dirs, files in walk(path):
+                for filename in files:
+                    self.outputs.append(os.path.join(base,filename))
+        else:
+            self.outputs.append(path)
+
+
+
+
+
+
     def easy_install(self, spec):
         tmpdir = self.alloc_tmp()
-        try:
-            download = self.package_index.download(spec, tmpdir)
+        download = None
+
+        try:           
+            if not isinstance(spec,Requirement):
+                if URL_SCHEME(spec):
+                    # It's a url, download it to tmpdir and process
+                    download = self.package_index.download(spec, tmpdir)
+                    return self.install_item(None, download, tmpdir, True)                   
+
+                elif os.path.exists(spec):
+                    # Existing file or directory, just process it directly
+                    return self.install_item(None, spec, tmpdir, True)
+                else:
+                    try:
+                        spec = Requirement.parse(spec)
+                    except ValueError:
+                        raise RuntimeError(
+                            "Not a URL, existing file, or requirement spec: %r"
+                            % (spec,)
+                        )
+
+            if isinstance(spec, Requirement):
+                download = self.package_index.fetch(spec, tmpdir, self.upgrade)
+            else:
+                spec = None
+
             if download is None:
                 raise RuntimeError(
                     "Could not find distribution for %r" % spec
                 )
 
-            log.info("Processing %s", os.path.basename(download))
-            for dist in self.install_eggs(download, self.zip_ok, tmpdir):
-                self.package_index.add(dist)
-                self.install_egg_scripts(dist)
-                log.warn(self.installation_report(dist))
+            return self.install_item(spec, download, tmpdir)
 
         finally:
             if self.build_directory is None:
                 shutil.rmtree(tmpdir)
 
 
+
+    def install_item(self, spec, download, tmpdir, install_needed=False):
+        # Installation is also needed if file in tmpdir or is not an egg
+        install_needed = install_needed or os.path.dirname(download) == tmpdir
+        install_needed = install_needed or not download.endswith('.egg')
+        log.info("Processing %s", os.path.basename(download))
+        if install_needed or self.always_copy:
+            dists = self.install_eggs(download, self.zip_ok, tmpdir)
+            for dist in dists:
+                self.process_distribution(spec, dist)
+        else:
+            dists = [self.egg_distribution(download)]
+            self.process_distribution(spec, dists[0], "Using")
+        if spec is not None:
+            for dist in dists:
+                if dist in spec:
+                    return dist
+
+    def process_distribution(self, requirement, dist, *info):
+        self.update_pth(dist)
+        self.package_index.add(dist)
+        self.local_index.add(dist)
+        self.install_egg_scripts(dist)
+        log.warn(self.installation_report(dist, *info))
+        if requirement is None:
+            requirement = Requirement.parse('%s==%s'%(dist.name,dist.version))
+        if dist in requirement:
+            log.info("Processing dependencies for %s", requirement)
+            try:
+                self.local_index.resolve(
+                    [requirement], self.shadow_path, self.easy_install
+                )
+            except DistributionNotFound, e:
+                raise RuntimeError(
+                    "Could not find required distribution %s" % e.args
+                )
+            except VersionConflict, e:
+                raise RuntimeError(
+                    "Installed distribution %s conflicts with requirement %s"
+                    % e.args
+                )
+
     def install_egg_scripts(self, dist):
         metadata = dist.metadata
         if self.exclude_scripts or not metadata.metadata_isdir('scripts'):
@@ -285,26 +367,36 @@ class easy_install(Command):
 
 
 
+    def egg_distribution(self, egg_path):
+        if os.path.isdir(egg_path):
+            metadata = PathMetadata(egg_path,os.path.join(egg_path,'EGG-INFO'))
+        else:
+            metadata = EggMetadata(zipimport.zipimporter(egg_path))
+        return Distribution.from_filename(egg_path,metadata=metadata)
+        
     def install_egg(self, egg_path, zip_ok, tmpdir):
         destination = os.path.join(self.install_dir,os.path.basename(egg_path))
         destination = os.path.abspath(destination)
+
         if not self.dry_run:
             ensure_directory(destination)
 
         if not samefile(egg_path, destination):
             if os.path.isdir(destination):
                 dir_util.remove_tree(destination, dry_run=self.dry_run)
-
             elif os.path.isfile(destination):
                 self.execute(os.unlink,(destination,),"Removing "+destination)
 
-            if zip_ok:
+            if os.path.isdir(egg_path):
+                if egg_path.startswith(tmpdir):
+                    f,m = shutil.move, "Moving"
+                else:
+                    f,m = shutil.copytree, "Copying"
+            elif zip_ok:
                 if egg_path.startswith(tmpdir):
                     f,m = shutil.move, "Moving"
                 else:
                     f,m = shutil.copy2, "Copying"
-            elif os.path.isdir(egg_path):
-                f,m = shutil.move, "Moving"
             else:
                 self.mkpath(destination)
                 f,m = self.unpack_and_compile, "Extracting"
@@ -313,18 +405,8 @@ class easy_install(Command):
                 (m+" %s to %s") %
                 (os.path.basename(egg_path),os.path.dirname(destination)))
 
-        if os.path.isdir(destination):
-            dist = Distribution.from_filename(
-                destination, metadata=PathMetadata(
-                    destination, os.path.join(destination,'EGG-INFO')
-                )
-            )
-        else:
-            metadata = EggMetadata(zipimport.zipimporter(destination))
-            dist = Distribution.from_filename(destination,metadata=metadata)
-
-        self.update_pth(dist)
-        return dist
+        self.add_output(destination)
+        return self.egg_distribution(destination)
 
     def install_exe(self, dist_filename, tmpdir):
         # See if it's valid, get data
@@ -408,10 +490,10 @@ class easy_install(Command):
             ensure_directory(nl_txt)
             open(nl_txt,'w').write('\n'.join(native_libs)+'\n')
 
-    def installation_report(self, dist):
+    def installation_report(self, dist, what="Installed"):
         """Helpful installation message for display to package users"""
 
-        msg = "\nInstalled %(eggloc)s to %(instdir)s"
+        msg = "\n%(what)s %(eggloc)s"
         if self.multi_version:
             msg += """
 
@@ -431,8 +513,7 @@ Note also that the installation directory must be on sys.path at runtime for
 this to work.  (e.g. by being the application's script directory, by being on
 PYTHONPATH, or by being added to sys.path by your code.)
 """
-        eggloc = os.path.basename(dist.path)
-        instdir = os.path.realpath(self.install_dir)
+        eggloc = dist.path
         name = dist.name
         version = dist.version
         return msg % locals()
@@ -449,6 +530,7 @@ PYTHONPATH, or by being added to sys.path by your code.)
 
 
 
+
     def build_egg(self, tmpdir, setup_script):
         sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg)
 
@@ -472,22 +554,54 @@ PYTHONPATH, or by being added to sys.path by your code.)
         finally:
             log.set_verbosity(self.verbose) # restore our log verbosity
 
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
     def update_pth(self,dist):
-        if self.pth_file is not None:
-            remove = self.pth_file.remove
-            for d in self.pth_file.get(dist.key,()):    # drop old entries
-                log.info("Removing %s from .pth file", d)
-                remove(d)
-            if not self.multi_version:
-                log.info("Adding %s to .pth file", dist)
+        if self.pth_file is None:
+            return
+
+        for d in self.pth_file.get(dist.key,()):    # drop old entries
+            if self.multi_version or d.path != dist.path:
+                log.info("Removing %s from easy-install.pth file", d)
+                self.pth_file.remove(d)
+                if d.path in self.shadow_path:
+                    self.shadow_path.remove(d.path)
+
+        if not self.multi_version:
+            if dist.path in self.pth_file.paths:
+                log.info(
+                    "%s is already the active version in easy-install.pth",
+                    dist
+                )
+            else:
+                log.info("Adding %s to easy-install.pth file", dist)
                 self.pth_file.add(dist) # add new entry
-            self.pth_file.save()
+                if dist.path not in self.shadow_path:
+                    self.shadow_path.append(dist.path)
 
-            if dist.name=='setuptools':
-                # Ensure that setuptools itself never becomes unavailable!
-                f = open(os.path.join(self.install_dir,'setuptools.pth'), 'w')
-                f.write(dist.path+'\n')
-                f.close()
+        self.pth_file.save()
+
+        if dist.name=='setuptools':
+            # Ensure that setuptools itself never becomes unavailable!
+            f = open(os.path.join(self.install_dir,'setuptools.pth'), 'w')
+            f.write(dist.path+'\n')
+            f.close()
 
 
     def unpack_progress(self, src, dst):
@@ -496,6 +610,9 @@ PYTHONPATH, or by being added to sys.path by your code.)
         return dst     # only unpack-and-compile skips files for dry run
 
 
+
+
+
     def unpack_and_compile(self, egg_path, destination):
         to_compile = []
 
@@ -531,6 +648,12 @@ PYTHONPATH, or by being added to sys.path by your code.)
 
 
 
+
+
+
+
+
+
 def extract_wininst_cfg(dist_filename):
     """Extract configuration data from a bdist_wininst .exe
 
diff --git a/ez_setup.py b/ez_setup.py
index 82cabb93..d0c98b14 100755
--- a/ez_setup.py
+++ b/ez_setup.py
@@ -14,7 +14,7 @@ the appropriate options to ``use_setuptools()``.
 This file can also be run as a script to install or upgrade setuptools.
 """
 
-DEFAULT_VERSION = "0.5a3"
+DEFAULT_VERSION = "0.5a4"
 DEFAULT_URL     = "http://peak.telecommunity.com/dist/"
 
 import sys, os
diff --git a/pkg_resources.py b/pkg_resources.py
index 603975d2..7aba57ce 100644
--- a/pkg_resources.py
+++ b/pkg_resources.py
@@ -296,7 +296,7 @@ class AvailableDistributions(object):
         """Remove `dist` from the distribution map"""
         self._distmap[dist.key].remove(dist)
 
-    def best_match(self,requirement,path=None):
+    def best_match(self, requirement, path=None, installer=None):
         """Find distribution best matching `requirement` and usable on `path`
 
         If a distribution that's already installed on `path` is unsuitable,
@@ -324,9 +324,9 @@ class AvailableDistributions(object):
         for dist in distros:
             if dist in requirement:
                 return dist
-        return self.obtain(requirement) # try and download
+        return self.obtain(requirement, installer) # try and download/install
 
-    def resolve(self, requirements, path=None):
+    def resolve(self, requirements, path=None, installer=None):
         """List all distributions needed to (recursively) meet requirements"""
 
         if path is None:
@@ -344,26 +344,26 @@ class AvailableDistributions(object):
                 continue
 
             dist = best.get(req.key)
-
             if dist is None:
                 # Find the best distribution and add it to the map
-                dist = best[req.key] = self.best_match(req,path)
+                dist = best[req.key] = self.best_match(req, path, installer)
                 if dist is None:
                     raise DistributionNotFound(req)  # XXX put more info here
                 to_install.append(dist)
 
             elif dist not in req:
                 # Oops, the "best" so far conflicts with a dependency
-                raise VersionConflict(req,dist) # XXX put more info here
+                raise VersionConflict(dist,req) # XXX put more info here
 
             requirements.extend(dist.depends(req.options)[::-1])
             processed[req] = True
 
         return to_install    # return list of distros to install
 
-    def obtain(self, requirement):
+    def obtain(self, requirement, installer=None):
         """Obtain a distro that matches requirement (e.g. via download)"""
-        return None     # override this in subclasses
+        if installer is not None:
+            return installer(requirement)
 
     def __len__(self): return len(self._distmap)
 
@@ -1316,10 +1316,9 @@ class Distribution(object):
             return self.__dep_map
         except AttributeError:
             dm = self.__dep_map = {None: []}
-            for section,contents in split_sections(
-                self._get_metadata('depends.txt')
-            ):
-                dm[section] = list(parse_requirements(contents))
+            for name in 'requires.txt', 'depends.txt':
+                for extra,reqs in split_sections(self._get_metadata(name)):
+                    dm.setdefault(extra,[]).extend(parse_requirements(reqs))
             return dm
 
     _dep_map = property(_dep_map)
@@ -1351,6 +1350,7 @@ class Distribution(object):
             fixup_namespace_packages(self.path)
             map(declare_namespace, self._get_metadata('namespace_packages.txt'))
 
+
     def egg_name(self):
         """Return what this distribution's standard .egg filename should be"""
         filename = "%s-%s-py%s" % (
diff --git a/setup.py b/setup.py
index 5b86a8c6..df9c9aec 100755
--- a/setup.py
+++ b/setup.py
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 """Distutils setup file, used to install or test 'setuptools'"""
 
-VERSION = "0.5a3"
+VERSION = "0.5a4"
 from setuptools import setup, find_packages, Require
 
 setup(
@@ -42,7 +42,6 @@ setup(
     packages = find_packages(),
     py_modules = ['pkg_resources', 'easy_install'],
     scripts = ['easy_install.py'],
-    extra_path = ('setuptools', 'setuptools-%s.egg' % VERSION),
 
     classifiers = [f.strip() for f in """
     Development Status :: 3 - Alpha
@@ -80,3 +79,4 @@ setup(
 
 
 
+
diff --git a/setuptools/__init__.py b/setuptools/__init__.py
index d469f3f7..ebebd509 100644
--- a/setuptools/__init__.py
+++ b/setuptools/__init__.py
@@ -8,7 +8,7 @@ from distutils.core import Command as _Command
 from distutils.util import convert_path
 import os.path
 
-__version__ = '0.5a3'
+__version__ = '0.5a4'
 
 __all__ = [
     'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require',
diff --git a/setuptools/command/bdist_egg.py b/setuptools/command/bdist_egg.py
index cd3d3ecb..8abcb855 100644
--- a/setuptools/command/bdist_egg.py
+++ b/setuptools/command/bdist_egg.py
@@ -11,7 +11,7 @@ from distutils.sysconfig import get_python_version, get_python_lib
 from distutils.errors import *
 from distutils import log
 from pkg_resources import parse_requirements, get_platform, safe_name, \
-    safe_version, Distribution
+    safe_version, Distribution, yield_lines
 
 
 def write_stub(resource, pyfile):
@@ -78,7 +78,7 @@ class bdist_egg(Command):
         self.tag_build = None
         self.tag_svn_revision = 0
         self.tag_date = 0
-
+        self.egg_output = None
 
     def finalize_options (self):
         self.egg_name = safe_name(self.distribution.get_name())
@@ -105,19 +105,19 @@ class bdist_egg(Command):
             self.bdist_dir = os.path.join(bdist_base, 'egg')
         if self.plat_name is None:
             self.plat_name = get_platform()
-        self.set_undefined_options('bdist',('dist_dir', 'dist_dir'))
-
-
-
-
-
-
-
-
 
+        self.set_undefined_options('bdist',('dist_dir', 'dist_dir'))
 
+        if self.egg_output is None:
 
+            # Compute filename of the output egg
+            basename = Distribution(
+                None, None, self.egg_name, self.egg_version,
+                get_python_version(),
+                self.distribution.has_ext_modules() and self.plat_name
+            ).egg_name()
 
+            self.egg_output = os.path.join(self.dist_dir, basename+'.egg')
 
 
 
@@ -146,22 +146,22 @@ class bdist_egg(Command):
         finally:
             self.distribution.data_files = old
 
+    def get_outputs(self):
+        return [self.egg_output]
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+    def write_requirements(self):
+        dist = self.distribution
+        if not getattr(dist,'install_requires',None) and \
+           not getattr(dist,'extras_require',None): return
+        requires = os.path.join(self.egg_info,"requires.txt")
+        log.info("writing %s", requires)
+        if not self.dry_run:
+            f = open(requires, 'wt')
+            f.write('\n'.join(yield_lines(dist.install_requires)))
+            for extra,reqs in dist.extras_require.items():
+                f.write('\n\n[%s]\n%s' % (extra, '\n'.join(yield_lines(reqs))))
+            f.close()
+            
     def run(self):
         # We run install_lib before install_data, because some data hacks
         # pull their data path from the install_lib command.
@@ -189,24 +189,19 @@ class bdist_egg(Command):
         if self.distribution.data_files:
             self.do_install_data()
 
+        # Make the EGG-INFO directory
+        archive_root = self.bdist_dir
+        egg_info = os.path.join(archive_root,'EGG-INFO')
+        self.mkpath(egg_info)
+        self.mkpath(self.egg_info)
+
         if self.distribution.scripts:
-            script_dir = os.path.join(self.bdist_dir,'EGG-INFO','scripts')
+            script_dir = os.path.join(egg_info, 'scripts')
             log.info("installing scripts to %s" % script_dir)
             self.call_command('install_scripts', install_dir=script_dir)
 
-        # And make an archive relative to the root of the
-        # pseudo-installation tree.
-        archive_basename = Distribution(
-            None, None, self.egg_name, self.egg_version, get_python_version(),
-            ext_outputs and self.plat_name
-        ).egg_name()
-        pseudoinstall_root = os.path.join(self.dist_dir, archive_basename)
-        archive_root = self.bdist_dir
+        self.write_requirements()
 
-        # Make the EGG-INFO directory
-        egg_info = os.path.join(archive_root,'EGG-INFO')
-        self.mkpath(egg_info)
-        self.mkpath(self.egg_info)
 
         log.info("writing %s" % os.path.join(self.egg_info,'PKG-INFO'))
         if not self.dry_run:
@@ -231,15 +226,20 @@ class bdist_egg(Command):
             if not self.dry_run:
                 os.unlink(native_libs)
 
-        if self.egg_info and os.path.exists(self.egg_info):
-            for filename in os.listdir(self.egg_info):
-                path = os.path.join(self.egg_info,filename)
-                if os.path.isfile(path):
-                    self.copy_file(path,os.path.join(egg_info,filename))
+        for filename in os.listdir(self.egg_info):
+            path = os.path.join(self.egg_info,filename)
+            if os.path.isfile(path):
+                self.copy_file(path,os.path.join(egg_info,filename))
 
+        if os.path.exists(os.path.join(self.egg_info,'depends.txt')):
+            log.warn(
+                "WARNING: 'depends.txt' will not be used by setuptools 0.6!"
+            )
+            log.warn(
+                "Use the install_requires/extras_require setup() args instead."
+            )
         # Make the archive
-        make_zipfile(pseudoinstall_root+'.egg',
-                          archive_root, verbose=self.verbose,
+        make_zipfile(self.egg_output, archive_root, verbose=self.verbose,
                           dry_run=self.dry_run)
         if not self.keep_temp:
             remove_tree(self.bdist_dir, dry_run=self.dry_run)
diff --git a/setuptools/dist.py b/setuptools/dist.py
index f46a02f8..742ce7a1 100644
--- a/setuptools/dist.py
+++ b/setuptools/dist.py
@@ -8,50 +8,44 @@ from setuptools.command.install import install
 from setuptools.command.install_lib import install_lib
 from distutils.errors import DistutilsOptionError, DistutilsPlatformError
 from distutils.errors import DistutilsSetupError
-import setuptools
+import setuptools, pkg_resources
 
 sequence = tuple, list
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
 class Distribution(_Distribution):
     """Distribution with support for features, tests, and package data
 
     This is an enhanced version of 'distutils.dist.Distribution' that
     effectively adds the following new optional keyword arguments to 'setup()':
 
+     'install_requires' -- a string or sequence of strings specifying project
+        versions that the distribution requires when installed, in the format
+        used by 'pkg_resources.require()'.  They will be installed
+        automatically when the package is installed.  If you wish to use
+        packages that are not available in PyPI, or want to give your users an
+        alternate download location, you can add a 'find_links' option to the
+        '[easy_install]' section of your project's 'setup.cfg' file, and then
+        setuptools will scan the listed web pages for links that satisfy the
+        requirements.
+
+     'extras_require' -- a dictionary mapping names of optional "extras" to the
+        additional requirement(s) that using those extras incurs. For example,
+        this::
+
+            extras_require = dict(reST = ["docutils>=0.3", "reSTedit"])
+
+        indicates that the distribution can optionally provide an extra
+        capability called "reST", but it can only be used if docutils and
+        reSTedit are installed.  If the user installs your package using
+        EasyInstall and requests one of your extras, the corresponding
+        additional requirements will be installed if needed.
+
      'features' -- a dictionary mapping option names to 'setuptools.Feature'
         objects.  Features are a portion of the distribution that can be
         included or excluded based on user options, inter-feature dependencies,
         and availability on the current system.  Excluded features are omitted
         from all setup commands, including source and binary distributions, so
         you can create multiple distributions from the same source tree.
-
         Feature names should be valid Python identifiers, except that they may
         contain the '-' (minus) sign.  Features can be included or excluded
         via the command line options '--with-X' and '--without-X', where 'X' is
@@ -84,6 +78,8 @@ class Distribution(_Distribution):
     the distribution.  They are used by the feature subsystem to configure the
     distribution for the included and excluded features.
     """
+
+
     def __init__ (self, attrs=None):
         have_package_data = hasattr(self, "package_data")
         if not have_package_data:
@@ -91,16 +87,68 @@ class Distribution(_Distribution):
         self.features = {}
         self.test_suite = None
         self.requires = []
+        self.install_requires = []
+        self.extras_require = {}
         _Distribution.__init__(self,attrs)
         if not have_package_data:
             from setuptools.command.build_py import build_py
             self.cmdclass.setdefault('build_py',build_py)
+
         self.cmdclass.setdefault('build_ext',build_ext)
         self.cmdclass.setdefault('install',install)
         self.cmdclass.setdefault('install_lib',install_lib)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+    def finalize_options(self):
+        _Distribution.finalize_options(self)
+
         if self.features:
             self._set_global_opts_from_features()
 
+        if self.extra_path:
+            raise DistutilsSetupError(
+                "The 'extra_path' parameter is not needed when using "
+                "setuptools.  Please remove it from your setup script."
+            )
+        try:
+            list(pkg_resources.parse_requirements(self.install_requires))
+        except (TypeError,ValueError):
+            raise DistutilsSetupError(
+                "'install_requires' must be a string or list of strings "
+                "containing valid project/version requirement specifiers"
+            )
+        try:
+            for k,v in self.extras_require.items():
+                list(pkg_resources.parse_requirements(v))
+        except (TypeError,ValueError,AttributeError):
+            raise DistutilsSetupError(
+                "'extras_require' must be a dictionary whose values are "
+                "strings or lists of strings containing valid project/version "
+                "requirement specifiers."
+            )
+
     def parse_command_line(self):
         """Process features after parsing command line options"""
         result = _Distribution.parse_command_line(self)
@@ -108,19 +156,12 @@ class Distribution(_Distribution):
             self._finalize_features()
         return result
 
+
     def _feature_attrname(self,name):
         """Convert feature name to corresponding option attribute name"""
         return 'with_'+name.replace('-','_')
 
 
-
-
-
-
-
-
-
-
     def _set_global_opts_from_features(self):
         """Add --with-X/--without-X options based on optional features"""
 
@@ -343,29 +384,29 @@ class Distribution(_Distribution):
 
         return nargs
 
-
     def has_dependencies(self):
         return not not self.requires
 
-
     def run_commands(self):
-        if setuptools.bootstrap_install_from and 'install' in self.commands:
+        for cmd in self.commands:
+            if cmd=='install' and not cmd in self.have_run:
+                self.install_eggs()
+            else:
+                self.run_command(cmd)
+
+    def install_eggs(self):
+        from easy_install import easy_install
+        cmd = easy_install(self, args="x")
+        cmd.ensure_finalized()  # finalize before bdist_egg munges install cmd
+        self.run_command('bdist_egg')
+        args = [self.get_command_obj('bdist_egg').egg_output]
+        if setuptools.bootstrap_install_from:
             # Bootstrap self-installation of setuptools
-            from easy_install import easy_install
-            cmd = easy_install(
-                self, args=[setuptools.bootstrap_install_from], zip_ok=1
-            )
-            cmd.ensure_finalized()
-            cmd.run()
-            setuptools.bootstrap_install_from = None
-
-        _Distribution.run_commands(self)
-
-
-
-
-
-
+            args.insert(0, setuptools.bootstrap_install_from)
+        cmd.args = args
+        cmd.run()
+        self.have_run['install'] = 1
+        setuptools.bootstrap_install_from = None
 
     def get_cmdline_options(self):
         """Return a '{cmd: {opt:val}}' map of all command-line options
diff --git a/setuptools/package_index.py b/setuptools/package_index.py
index a4f81882..78962eee 100755
--- a/setuptools/package_index.py
+++ b/setuptools/package_index.py
@@ -203,7 +203,7 @@ class PackageIndex(AvailableDistributions):
 
 
 
-    def find_packages(self,requirement):
+    def find_packages(self, requirement):
         self.scan_url(self.index_url + requirement.distname+'/')
         if not self.package_pages.get(requirement.key):
             # We couldn't find the target package, so search the index page too
@@ -221,13 +221,13 @@ class PackageIndex(AvailableDistributions):
             # scan each page that might be related to the desired package
             self.scan_url(url)
 
-    def obtain(self,requirement):
+    def obtain(self, requirement, installer=None):
         self.find_packages(requirement)
         for dist in self.get(requirement.key, ()):
             if dist in requirement:
                 return dist
             self.debug("%s does not match %s", requirement, dist)                
-
+        return super(PackageIndex, self).obtain(requirement,installer)
 
 
 
@@ -245,19 +245,20 @@ class PackageIndex(AvailableDistributions):
 
 
     def download(self, spec, tmpdir):
-        """Locate and/or download `spec`, returning a local filename
+        """Locate and/or download `spec` to `tmpdir`, returning a local path
 
         `spec` may be a ``Requirement`` object, or a string containing a URL,
-        an existing local filename, or a package/version requirement spec
+        an existing local filename, or a project/version requirement spec
         (i.e. the string form of a ``Requirement`` object).
 
-        If necessary, the requirement is searched for in the package index.
-        If the download is successful, the return value is a local file path,
-        and it is a subpath of `tmpdir` if the distribution had to be
-        downloaded.  If no matching distribution is found, return ``None``.
-        Various errors may be raised if a problem occurs during downloading.
+        If `spec` is a ``Requirement`` object or a string containing a
+        project/version requirement spec, this method is equivalent to
+        the ``fetch()`` method.  If `spec` is a local, existing file or
+        directory name, it is simply returned unchanged.  If `spec` is a URL,
+        it is downloaded to a subpath of `tmpdir`, and the local filename is
+        returned.  Various errors may be raised if a problem occurs during
+        downloading.
         """
-
         if not isinstance(spec,Requirement):
             scheme = URL_SCHEME(spec)
             if scheme:
@@ -275,16 +276,56 @@ class PackageIndex(AvailableDistributions):
                         "Not a URL, existing file, or requirement spec: %r" %
                         (spec,)
                     )
+
+        return self.fetch(spec, tmpdir, force_scan)
+
+
+
+
+
+
+
+    def fetch(self, requirement, tmpdir, force_scan=False):
+        """Obtain a file suitable for fulfilling `requirement`
+
+        `requirement` must be a ``pkg_resources.Requirement`` instance.
+        If necessary, or if the `force_scan` flag is set, the requirement is
+        searched for in the (online) package index as well as the locally
+        installed packages.  If a distribution matching `requirement` is found,
+        the return value is the same as if you had called the ``download()``
+        method with the matching distribution's URL.  If no matching
+        distribution is found, returns ``None``.
+        """
+
         # process a Requirement
-        self.info("Searching for %s", spec)
-        dist = self.best_match(spec,[])
+        self.info("Searching for %s", requirement)
+
+        if force_scan:
+            self.find_packages(requirement)
+
+        dist = self.best_match(requirement, [])     # XXX
+
         if dist is not None:
             self.info("Best match: %s", dist)
             return self.download(dist.path, tmpdir)
 
-        self.warn("No local packages or download links found for %s", spec)
+        self.warn(
+            "No local packages or download links found for %s", requirement
+        )
         return None
 
+
+
+
+
+
+
+
+
+
+
+
+
     dl_blocksize = 8192
 
     def _download_to(self, url, filename):