How can I develop on a Python library package while testing it a project using Pycharm + Docker?

pycharm docker compose
pycharm docker-compose config failed
pycharm cannot connect to docker daemon
pycharm remote interpreter
pycharm community remote interpreter
docker python
docker utility container
docker environment in pycharm

This is a pretty specific ask, so I'm open to helpful suggestions that get part of the way there.

I have a python project that runs inside a docker container configured to work with the Pycharm debugger. I have a package, installed in a virtual env with pip, used in this project, that I'd like to develop on.

I haven't found a way to link the package into my project's docker container in such a way that I can change the package and have the code update in my project. Currently, the debugger works on codepaths that enter into the package, as long as I don't change any code in the package.

These two problems combined make it hard to test changes to the package without installing it over and over.

Is there a better way to accomplish this goal?

Given the two source trees that should work together:

  1. Create a virtual environment for them, python -m venv vpy.
  2. Activate it, setting relevant shell environment variables, . vpy/bin/activate.
  3. Install the library, cd library && pip install -e . (-e causes pip to remember a pointer to the live source tree.)
  4. Install the application, cd app && pip install -e .. (Pip should know you already have the library installed.)
  5. Do whatever you need to do, $EDITOR file.py; pytest; the_app; $SCM commit.
  6. Once it all works correctly, docker build && docker run.

I'd leave any interaction with Docker until the very end, once you've convinced yourself you've fixed the library bug or built the feature. That avoids troubles with your editor and the container disagreeing on paths, and it means you don't need root privileges for any of your ordinary development work.

Configure an interpreter using Docker - Help, PyCharm integration with Docker allows you to run your applications in the variously configured development environments deployed in Docker containers. Create a Python project QuadraticEquation , add the Solver.py file and enter the following Whether it's running, debugging, running with coverage, testing - each  One of the open tasks is to create a working setup.py for it and install it and its sub-packages in site-packages or dist-packages. It contains executable scripts that get installed into bin (of the active virtualenv if there is one). I can already install the whole project in site-packages using distutils respectively python setup.py install.

Install the package in editable mode.

pip install -e .

This will allow you to make changes to the code and update the package simultaneously.

Using Docker in PyCharm, This is ideal, not just for development, but for deployment as well. Unlike other interpreters in PyCharm, you don't visit the Project Interpreter In the Configure Remote Python Interpreter dialog, click the Docker button. In fact, we can go on to test running, debugging, code coverage, profiling, and all the  The following is only valid when Docker Integration and Python Docker plugins are installed and enabled! Docker enables developers to deploy applications inside containers for testing code in an environment identical to production. PyCharm provides Docker support using the Docker plugin. The plugin is bundled and enabled by default.

You can try using combination of what @pbskumar proposed and docker volumes.

First run your container with option --volume /path/to/your/package/on/host/:/path/in/your/container

And then you execute this inside the container: pip install -e /path/in/your/container

It should work.

Configure an interpreter using Docker Compose, You can install Docker on the various platforms, but here we'll use the Windows installation. Get the project from GitHub, and open it in PyCharm (File | Open). FROM python:3.6.7 WORKDIR /app # By copying over requirements first, we requirements rather than reinstall them on every build COPY requirements.txt  Open the Add Python Interpreter dialog by either way: When you're in the Editor, the most convenient way is to use the Python Interpreter widget in the . Click the widget and select Add Interpreter If you are in the Settings/Preferences dialog Ctrl+Alt+S, select Project <project name> | Project Interpreter.

I think I've covered all the bases I outlined above with this system:

My project structure is:

projects
   projectA (my docker project)
   projectB (the library used in my docker project that I want to develop on)
  1. In projectA: project settings -> project interpreter -> add path mappings -> map local library path to remote install path on container

    ex. local path: /user/{username}/projects/projectB remote path: /usr/local/lib/python3.6/site-packages/projectB

  2. In projectA: project settings -> project structure -> add content root (projectB) -> choose mark as sources

  3. In projectA: mark path on to projectB in container as a volume in dockerfile

    ex. VOLUME /usr/local/lib/python3.6/site-packages/projectB

  4. In projectA: load local library as volume to library installation on container in docker-compose.yml

    ex.

volumes:

- ../projectB:/usr/local/lib/python3.6/site-packages/projectB

Using python 3.6 and Pycharm 2018.2

Docker - Help, The following is only valid when Docker Integration and Python Docker plugins are inside containers for testing code in an environment identical to production. Docker images are executable packages for running containers. For example, you can build an image that runs a container with some specific version of  Do not use SSH (if you need to step into container, you can use the docker exec command). Avoid manual configurations (or actions) inside container. Conclusion. To summarize this tutorial, alongside with IDE and Git, Docker has become a must-have developer tool that is not only used for delivering Python development services. It’s a production-ready tool with a rich and mature infrastructure.

Pip install with Docker remote interpreter is ephimeral – IDEs , Hello, we are evaluating to use PyCharm Professional as IDE to develop in Python. (e.g. python:3.5.4); let PyCharm to provision the required packages running pip install -r requirements.txt in the docker container; develop, debug and test The project's dockerfile doesn't install any of the dev and test  Creating a Python file. Select the project root in the Project tool window, then select File | New from the main menu or press Alt+Insert. Choose the option Python file from the popup, and then type the new filename. PyCharm creates a new Python file and opens it for editing.

Run/Debug Configuration: Python Unit Test, Use this dialog to create a run/debug configuration for Python unit tests. Name of the module in your project, for example, my_tests . This field only appears when a Docker-based remote interpreter is selected for a project. If your application uses Expo, you need to run the development server via the start npm task. This script below (package.py) is a setuptools command extension for creating a distribution from a project. normal python setup sdist will only pack files and folders, while this script go over

Configure a Python interpreter - Help, When you configure a project Python interpreter, you need to specify the path to the one virtual environment based on Python 3.6 to develop Django applications and environment based on the same Python 3.6 to work with scientific libraries. Note that SSH, WSL, Vagrant, Docker, and Docker Compose are available  The standard site-packages is included by default, and this is where packages you install using via pip go. How to Package a Python Library. Now that we have our code and tests, let's package it all into a proper library. Python provides an easy way via the setup module. You create a file called setup.py in your package's root directory.

Comments
  • I misspoke, I'm not using a virtualenv anymore now that dependencies are installed in my docker container. Using pip install -e worked before moving the project to Docker, but I have can't find a way to make it work now.
  • I don't have a complete answer to this yet, but the closest I've gotten is loading my package as a volume, with the destination path set to where pip packages end up installed on the docker container. Using pip install -e as part of my image build didn't work for me, because I believe under the hood -e uses symlinks, which don't work as expected in docker containers. the above method allows hot reloading package changes, although using the debugger in the package is still pretty hit or miss.