Prevent pip from installing some dependencies

pip install command
pip install from local directory
pip install setuppy
python pip install --upgrade pip
pip install git
pip list dependencies
pip install without dependencies
pip install -e

We're developing an AWS Lambda function for Alexa skill in Python and using pip to install the ask-sdk package to our dist/ directory:

pip install -t dist/ ask-sdk

The trouble is with the -t dist/ because pip wants to have all the dependencies there, even if they are installed system-wide.

Now, ask-sdk has a dependency on boto3 which pulls in a whole lot of other packages. However the AWS Lambda runtime environment provides boto3 and there is no need to package that and its dependencies with our code. I do have boto3 installed in the system and import boto3 works, so I thought pip should be happy, but because of -t dist/ it always installs it.

Can I somehow install just ask-sdk and its dependencies that don't exist in the system, e.g. ask-sdk-core, but not those that are already installed?

One way is to list all the modules and use pip --no-deps but that means a constantly keeping track of the dependencies manually, we would like to avoid that.

Somehow I would like to tell pip: if the package is already installed, even if not in -t dist/ don't put a copy in dist/.

Is that possible?

Thanks!


Although you can't tell pip to "install all dependencies except those required by boto3", you can generate the needed requirements.txt by computing the difference between boto3 and ask-sdk from pip freeze output (tested with Python 3.6.6):

# get boto3 requirements
pip install boto3 -t py_lib.boto3
PYTHONPATH=py_lib.boto3 pip freeze > requirements-boto3.txt

# get ask-sdk requirements
pip install ask-sdk -t py_lib.ask-sdk
PYTHONPATH=py_lib.ask-sdk pip freeze > requirements-ask-sdk.txt

# compute their difference
grep -v -x -f requirements-boto3.txt requirements-ask-sdk.txt > requirements-final.txt

# patch to add one missing dep from boto3
# aws don't have this for some reason
grep urllib3 requirements-boto3.txt >> requirements-final.txt

The requirements-final.txt contains the following:

ask-sdk==1.5.0
ask-sdk-core==1.5.0
ask-sdk-dynamodb-persistence-adapter==1.5.0
ask-sdk-model==1.6.2
ask-sdk-runtime==1.5.0
certifi==2018.11.29
chardet==3.0.4
idna==2.8
requests==2.21.0
urllib3==1.24.1

To install the final set of dependencies to a folder:

pip install --no-deps -r requirements-final.txt -t py_lib

By skipping the boto3 dependencies, you can save about 45M of data from your python dependencies. The ask-sdk dependencies are only about 7.5M (2.1M compressed), allow you to use the build-in lambda code editor if you need to.

A way to blacklist 'provided' dependencies. · Issue #3090 · pypa/pip , pip install --log pip.log -r req.txt -target /mypath/pypkgs/ --exclude for installing plugins for an application that already provides some of the  Running conda after pip has the potential to overwrite and potentially break packages installed via pip. Similarly, pip may upgrade or remove a package which a conda-installed package requires. In some cases these breakages are cosmetic, where a few files are present that should have been removed, but in other cases the environment may evolve into an unusable state.


You can try the option

  --no-dependencies

To ignore all dependencies.

To exclude specific, you can put it in requirements file and pass it:

pip install --no-deps -r requirements.txt

pip install --user ignores system-installed dependencies, installs , pip install --user ignores system-installed dependencies, installs older When inside virtual environments or when running as root, keep the default distributions support an alternative install location that is specific to a user. Installing packages using pip and virtual environments¶ This guide discusses how to install packages using pip and a virtual environment manager: either venv for Python 3 or virtualenv for Python 2. These are the lowest-level tools for managing Python packages and are recommended if higher-level tools do not suit your needs.


This will work

$ pip install -t dist --no-deps ask-sdk

After the above command I checked out the dist directory content with tree and it had installed only ask-sdk without its dependencies

dist/
├── ask_sdk
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── __version__.py
│   ├── __version__.pyc
│   ├── standard.py
│   └── standard.pyc
└── ask_sdk-0.1.3.dist-info
    ├── INSTALLER
    ├── METADATA
    ├── RECORD
    ├── WHEEL
    └── top_level.txt

pip install, As of v6.1.0, pip installs dependencies before their dependents, i.e. in to declare build dependencies, it may help certain projects install from sdist (that might While this cache attempts to minimize network activity, it does not prevent​  setup.py référence the dependencies (I use PBR to do it, i have not tested with find_packages, but in both cases I don’t see how the markers can be described in it) symptom is: pip install mypackage does install the conditioned dependency on python 2.7, pipenv install mypackage does not


The Nine Circles of Python Dependency Hell - Knerd, Dependency hell in Python often happens because pip does not have a dependency… If you just run pip install -r requirements.txt , pip will happily ignore any Some dependency management systems, like npm, avoid this problem by  When installing from a version control repository, it is strongly recommended to pin the package by specifying a tag or commit, rather than branch. Sometimes your dependencies may themselves have unpinned dependencies. In this case, it can be worth explicitly pinning those too - you can easily pin all dependencies in a project automatically.


pipdeptree · PyPI, pipdeptree is a command line utility for displaying the installed python packages in form of a dependency tree. It works for packages installed globally on a  Requirements files are used to force pip to properly resolve dependencies. As it is now, pip doesn’t have true dependency resolution, but instead simply uses the first specification it finds for a project. E.g. if pkg1 requires pkg3>=1.0 and pkg2 requires pkg3>=1.0,<=2.0, and if pkg1 is resolved first,


Managing Python packages the right way, When installing packages, pip will first resolve the dependencies, check if and some of these tools install into a non-user-writeable location. Packaging and distributing projects¶ This section covers the basics of how to configure, package and distribute your own Python projects. It assumes that you are already familiar with the contents of the Installing Packages page. The section does not aim to cover best practices for Python project development as a whole. For example, it does