Making python wait until subprocess.call has finished its command

Related searches

So this question is following on from this (Read comments aswell as that is the path I took). I just want the first call robocopy to finish executing before moving on to the rest of the code. As I want the second robocopy to just skip all of the files as they have already been copied over. However what is happening is that the rest of the script will run (ie starts the second robocopy) while the first robocopy is copying over the files. Below is the code:

call(["start", "cmd", "/K", "RoboCopy.exe", f"{self.srcEntry.get()}", f"{self.dstEntry.get()}", "*.*", "/E", "/Z", "/MT:8"], stdout=PIPE, shell=True) 
temp2 = Popen(["RoboCopy.exe", f"{self.srcEntry.get()}", f"{self.dstEntry.get()}", "*.*", "/E", "/Z"], stdout=PIPE, stdin=PIPE, shell=True)

EDIT 1:

Issue is noticeable when copying over large files. I'm thinking about putting in a sleep function which is dependent upon the total size of the files to be copied over. However this doesn't take into account upload/download speeds as the files will be transferred over a network.


I use the following function to launch my commands which waits until the action has finished, but has a timeout:

import os
import logging

logger = logging.getLogger()

def command_runner(command, valid_exit_codes=None, timeout=30, shell=False, decoder='utf-8'):
    """
    command_runner 2019103101
    Whenever we can, we need to avoid shell=True in order to preseve better security
    Runs system command, returns exit code and stdout/stderr output, and logs output on error
    valid_exit_codes is a list of codes that don't trigger an error
    """

    try:
        # universal_newlines=True makes netstat command fail under windows
        # timeout does not work under Python 2.7 with subprocess32 < 3.5
        # decoder may be unicode_escape for dos commands or utf-8 for powershell
        if sys.version_info >= (3, 0):
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                             timeout=timeout, universal_newlines=False)
        else:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                             universal_newlines=False)
        output = output.decode(decoder, errors='backslashreplace')
    except subprocess.CalledProcessError as exc:
        exit_code = exc.returncode
        # noinspection PyBroadException
        try:
            output = exc.output
            try:
                output = output.decode(decoder, errors='backslashreplace')
            except Exception as subexc:
                logger.debug(subexc, exc_info=True)
                logger.debug('Cannot properly decode error. Text is %s' % output)
        except Exception:
            output = "command_runner: Could not obtain output from command."
        if exit_code in valid_exit_codes if valid_exit_codes is not None else [0]:
            logger.debug('Command [%s] returned with exit code [%s]. Command output was:' % (command, exit_code))
            if output:
                logger.debug(output)
            return exc.returncode, output
        else:
            logger.error('Command [%s] failed with exit code [%s]. Command output was:' %
                         (command, exc.returncode))
            logger.error(output)
            return exc.returncode, output
    # OSError if not a valid executable
    except OSError as exc:
        logger.error('Command [%s] faild because of OS [%s].' % (command, exc))
        return None, exc
    except subprocess.TimeoutExpired:
        logger.error('Timeout [%s seconds] expired for command [%s] execution.' % (timeout, command))
        return None, 'Timeout of %s seconds expired.' % timeout
    except Exception as exc:
        logger.error('Command [%s] failed for unknown reasons [%s].' % (command, exc))
        logger.debug('Error:', exc_info=True)
        return None, exc
    else:
        logger.debug('Command [%s] returned with exit code [0]. Command output was:' % command)
        if output:
            logger.debug(output)
        return 0, output


# YOUR CODE HERE

executable = os.path.join(os.environ['SYSTEMROOT'], 'system32', 'robocopy.exe')
mycommand = '"%s" "%s" "%s" "%s"' % (executable, source, dest, options)
result, output = command_runner(mycommand, shell=True)

17.5. subprocess — Subprocess management — Python 3.4.10 , The timeout argument is passed to Popen.wait(). Wait for command to complete. Run command with arguments and return its output. corresponding argument to the open() function when creating the stdin/stdout/ stderr pipe file objects:. The .wait() makes the code wait until the system call finished. Get Output. To make a system call and get its output, pass the argument stdout=subprocess.PIPE, and get output from the method communicate()[0]. # python 3 import subprocess # call a unix command, wait for it to finish, get its output myOutput =subprocess.Popen([r "tail", "-n 1", "x.txt"], stdout=subprocess.PIPE).communicate()[0]


What I've found out: Thanks to QuantumChris. I've found out that robocopy returns from the terminal and back into my script although I've used subprocess.run which should have paused my script until it had finished running. I'm stalling the second robocopy from running by checking if the files have been copied over to the destination folder before progressing with the second robocopy. The issue is that if the last file is big then os.path.isfile() detects the file WHILE it is still being copied over. So it engages the second robocopy however the second robocopy doesn't detect that last file and so attempts to copy the file over but recognises that it can't access the file as it is already in use (by the first robocopy) so it waits 30 secs before trying again. After the 30 secs it is able to access the file and copies it over. What I would like to do now is to make my last file an empty dummy file which I don't care about it being copied twice as it is empty. Robocopy seems to copy over the files according to ASCII order. So I've named the file ~~~~~.txt :D

17.1. subprocess — Subprocess management — Python v3.2.6 , Wait for command to complete, then return the returncode attribute. Run command with arguments and return its output as a byte string. The underlying process creation and management in this module is handled by the Popen class. It� If you need to read from a network socket, the default configuration of the socket will make your read block until data arrives, so this works well as an efficient wait. If you need to wait on data arriving on multiple sockets or other file descriptors, then the select package from the Python standard library contains wrappers for operating


Try:

while temp2.poll() is not None:
    # ... do something else, sleep, etc

out, err = temp2.communicate()

subprocess — Subprocess management — Python 3.8.5 , Wait for command to complete, then return a CompletedProcess instance. Stderr output of the child process if it was captured by run() . The underlying process creation and management in this module is handled by the Popen class. It� Python System Command. While making a program in python, you may need to exeucte some shell commands for your program. For example, if you use Pycharm IDE, you may notice that there is option to share your project on github. And you probably know that file transferring is done by git, which is operated using command line. So, Pycharm executes


Python popen command. Wait until the command is finished, The problem is that the script doesn't wait until that popen command is finished and go continues right How can I tell to my Python script to wait until the shell command has finished? If you want to do things while it is executing or feed things into stdin , you can use Make Popen wait for first command to finish then start� A major part of the python code's task is to generate and run certain matlab scripts. The code starts matlab subprocesses by an os.system(command). Under Mac/Linux, it seems that python code waits until the matlab script terminates; however, under windows, the code simply proceeds before the matlab script terminates which crashes the application.


How to Execute Shell Commands with Python, Python is a wonderful language for scripting and automating Next, the os. popen() command opens a pipe from or to the command line. Popen() class is responsible for the creation and management of the executed process. When you run .communicate() , it will wait until the process is complete. A Popen object has a .wait() method exactly defined for this: to wait for the completion of a given subprocess (and, besides, for retuning its exit status). If you use this method, you'll prevent that the process zombies are lying around for too long. (Alternatively, you can use subprocess.call() or subprocess.check_call() for calling and


However, Event() has the added benefit of being more responsive. The reason for this is that when the event is set, the program will break out of the loop immediately. With time.sleep(), your code will need to wait for the Python sleep() call to finish before the thread can exit.