Cannot activate Conda environment from subprocess

python subprocess
activate conda environment in python script
commandnotfounderror: your shell has not been properly configured to use 'conda activate
activate conda environment in script
conda activate and run script
python subprocess get output
python subprocess conda activate
use conda environment in python script

I am building an Electron app (running an Angular application) which acts as the User Interface for a python program underneath.

The python program uses anaconda for package management (I am using miniconda for development).

When the app boots up, it checks whether the required conda environment exists, and if not, creates it.

The following code is part of a Service which is responsible for managing the python program.

public doEnvironmentSetup() {
    let stdOutSub = new Subject<string>();
    let stdErrSub = new Subject<string>();
    let completeSubject = new Subject<string>();
    this.runningSetup = true;

    const TEMP_ENV_FILE = join(tmpdir(), "env.yml");

    return Promise.resolve()
      .then(() => {

        // Copy packaged environment.yml to TEMP_ENV_FILE

      })
      .then(() => this.electron.getApplicationStoragePath())
      .then((stor) => {

        setTimeout(() => {

          let runProcess = this.electron.childProcess.spawn("conda", ["env", "create", "--file", TEMP_ENV_FILE, "--name", CONDA_ENV_NAME], {
            cwd: stor
          });

          const stdOutReaderInterface = createInterface(runProcess.stdout);
          const stdErrReaderInterface = createInterface(runProcess.stderr);

          stdOutReaderInterface.on('line', (line) => {
            stdOutSub.next(line);
          });

          stdErrReaderInterface.on('line', (line) => {
            stdErrSub.next(line);
          });

          runProcess.on('close', (code: number) => {
            this.electron.fs.unlinkSync(TEMP_ENV_FILE);
            this.runningSetup = false;
            completeSubject.next("");
          });

        }, 2000);

        return {
          stdOut: stdOutSub,
          stdErr: stdErrSub,
          onComplete: completeSubject
        };

      });

  }

Now, when I need to run the actual python code, the piece of code which runs is (not giving the whole thing, since it is too long for our purpose here) :

        execCmd.push(
          `conda init ${this.electron.os.platform() === "win32" ? "powershell" : "bash"}`,
          `conda activate ${CONDA_ENV_NAME}`,
          // long python spawn command
          `conda deactivate`,
        )

        setTimeout(() => {

          logLineSubject.next({ out: "--- Setting up Execution Environment ---", err: "" });

          logLineSubject.next({ out: `Running in ${dir}`, err: "" });

          const cmd = execCmd.join(" && ");

          let runProcess = this.electron.childProcess.spawn(cmd, {
            detached: false,
            windowsHide: true,
            cwd: cwd,
            shell: this.getShell()
          });

          const stdOutInterface = createInterface(runProcess.stdout);
          const stdErrInterface = createInterface(runProcess.stderr);

          stdOutInterface.on('line', (line) => {
            // get this line back to the component
          });

          stdErrInterface.on('line', (line) => {
            // get this line back to the component
          });

          runProcess.on("error", (err) => {
            // get this back to the component
          });

          runProcess.on('close', (code: number) => {
            // get this line back to the component
          });

        }, 1000);

where getShell is defined as:

private getShell() {
  return process.env[this.electron.os.platform() === "win32" ? "COMSPEC" : "SHELL"];
}

However, whenever I try to run this, it comes back with:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
    $ conda init <SHELL_NAME>
blah blah blah ...

When I try with source activate ${CONDA_ENV_NAME}, it comes back with:

/bin/bash: activate: No such file or directory

Not really sure what I am doing wrong here. Can somebody please point me in the right direction?

PS: It works with source $(conda info --root)/etc/profile.d/conda.sh, but I can't really use it since I need to support Windows as well!

Disclaimer: I am new to python/anaconda.

The main problem is that your shell (CMD in this case) is not configured to handle conda. You have to add it to your system path by providing the Miniconda/Anaconda to the PATH enviroment variable.

Check this StackOverflow Question to know how to do it.

Using subprocess in anaconda environment, Run source activate root in linux, or activate root in Windows to activate the environment before running your code. If this does not help you,� Cannot activate Conda environment from subprocess. 72. When the app boots up, it checks whether the required conda environment exists, and if not, creates it.

I'm not sure what needs to get run in Windows Powershell, but for bash, you need to run the script that conda init configures bash to run at startup, rather than conda init. That is,

miniconda3/etc/profile.d/conda.sh

so it should be something like

execCmd.push(
    `. ${CONDA_ROOT}/etc/profile.d/conda.sh`,
    `conda activate ${CONDA_ENV_NAME}`,
    // long python spawn command
     `conda deactivate`,
)

I suspect the Windows case is running one of the .bat or .ps1 files in the condabin directory.

Alternatively, if conda is defined and you have a Python script (e.g., script.py), then you might be able to get away with using conda run, e.g.,

execCmd.push(
    `conda run -n ${CONDA_ENV_NAME} python script.py`
)

and that could potentially work cross-platform. However, conda run is rather rudimentary and lacks support for interactive I/O (it simply buffers everything hitting stdout/stderr and doesn't return it until the process exits).

`conda activate` does not work with `Popen` class � Issue #9296 , Cannot run conda activate successfully using Popen class. PIPE proc = Popen( "conda activate my_test && conda env list", shell=True,� Current Behavior. When creating an environment from an environment.yml with pip dependencies, the pip subprocess fails. pip sometimes does fail for me outside of conda, I believe due to some issues with my office network & SSL interception.

Update:

I was referring the & for Windows platform and I also run conda by absolute path to conda.bat instead of call global installed conda. So it is easy for user to install it:

const execPath = path.dirname(process.execPath)
// silent install Miniconda3 to this path (electron-forge resources)
const condaPath = [execPath, "resources", "Miniconda3"].join(path.sep)
const conda = [condaPath, "condabin", "conda.bat"].join(path.sep)
cmd = `${condaPath} activate ${venvPath} & python ${filename} ${arg1} ${arg2}`

I was looking for this too and I got this working just by & as mentioned here

I have the same setup using electron and conda for UI talked to python back. This is in my node:

spawn(
  `conda activate ${name} & python ${filename} ${arg1} ${arg2}`,
  { shell: true }
);

I got it working to stream stdin/out too. Make sure the shell is true.

Subprocess using conda python - Support, Hi, I'm trying to use subprocess to get the output of another script running in conda environment since I want other python packages there. subprocess. for a way to use this https://pypi.org/project/pyimagej/ in Slicer as well but could not do it If you use virtual environment then you must activate it before you can use it (if� Toggle navigation. busco Project overview Project overview Details; Activity

How to Activate a Python Virtual Environment from a Script File, Depending on your configuration and requirements, you may be able to simply use the virtual environment's bin/activate program for activation, and then run� By default, conda activate will deactivate the current environment before activating the new environment and reactivate it when deactivating the new environment. Sometimes you may want to leave the current environment PATH entries in place so that you can continue to easily access command-line programs from the first environment.

subprocess — Subprocess management — Python 3.8.5 , subprocess. run (args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, If env is not None , it must be a mapping that defines the environment variables for the new process; these Another window will be activated. This value cannot be used with CREATE_NEW_CONSOLE. Cannot run conda activate successfully using Popen class. Steps to Reproduce from subprocess import Popen, PIPE proc = Popen(&quot;conda activate my_test &amp;&amp; conda env list&quot;, shell=True

How to Activate/Deactivate virtualenv on linux with subprocess , Consequently, you can't edit environment variables of a process by spawning a sub-process. That's why the activate command has to be sourced, not executed,� Current Behavior. When I create a environment from a environment.yml file, which has pip dependencies, if the pip dependencies fail to install (for example if I misspelled a package name) then the entire env creation still succeeds and returns a 0 exit code.

Comments
  • That may not be the case. I say this because, if you notice, in the line before conda activate, I do issue a conda init which runs successfully. It's failing in activate. If PATH were the issue, then init wouldn't have run in the first place.
  • Have you tried running the conda init command on powershell then conda --version to checkout if conda is successfully initialized on it? The answer below is pointing out such behaviour as conda init couldn't set the proper configuration to run conda commands on powershell.
  • Disclaimer: I am developing on Linux. I have tried running the same command set on the shell (Terminal) and it's working!
  • As I mentioned in my question, it seems to be working when I fire up source $(conda info --root)/etc/profile.d/conda.sh before running the conda commands. However, since the audience needs Windows support, that particular command won't cut it. conda run cannot be used, since as you mentioned that it buffers up everything, and as you can see from the code, this program runs for a really long time (around 15 minutes) and spits out a lot of logs which are relevant. Need to show the logs as they come.
  • The meaning of & vs && are different in the bash context. & would essentially just run the conda activate ${name} in the background and proceed onto python, without any heed to whether the activate actually worked or not. This answer has a good explanation: stackoverflow.com/a/26770612/951243 In bash, Try $ false && echo "something". This will return (or show) nothing. If you try $ false & echo "something", you will get the pid of false and the output of echo.
  • While I notice the deferences (plus between other platforms), the concern is to solve this on Windows as your OP. So this is mainly for Windows. & means run after another similar to ; in some unix shells. I tried and it works as I mentioned above.
  • BTW, regarding init error. There is a difference between conda init and ${condaPath}\condabin\conda.bat activate ${condaPath} before we can activate our environment. The earlier will install conda (add to %PATH% too) for the first time while the later is just to activate "base" so you will have "conda" cmd for current session. The later is useful to activate conda by path without messing or asking user to install it first.