Python subprocess multiple non blocking communicates

python subprocess popen
python subprocess get output
python subprocess async read stdout
python subprocess multiple commands
python subprocess example
python subprocess stdin
python subprocess stdout
python subprocess example windows

I start a script with:

t = subprocess.Popen('rosrun ros_pkg ros_node', shell=True, 
                     stdout = subprocess.PIPE, 
                     stdin = subprocess.PIPE, universal_newlines=True)

I then want to communictate with that process like this:

stdout = t.communicate('new command')[0]
if stdout == []:
stdout = t.communicate('new command')[0]

The problem is that after t.commincate the subprocess closes

There are solutions for similar problems but nothing worked for me yet please help



Note that if you want to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE. Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

PEP 3145 -- Asynchronous I/O For subprocess.Popen, A search for "python asynchronous subprocess" will turn up accounts of people wanting to execute a child process and communicate with it from time to make code that uses these functions work across multiple platforms. (args, *, stdin=None, stdout=None, stderr=None, shell=False) ¶ Run the command described by args . Wait for command to complete, then return the returncode attribute.

Using t.communicate() will close the input pipe after sending the data, meaning it can only be called once in order to send something to the subprocess.

However you can use t.stdin.write() to do sequential writes without closing the pipe and then use t.stdin.readline() to get the output. These work the same way the handler returned by open() does.

import subprocess

t = subprocess.Popen("cat", stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True, shell=True)

#These calls will have their results written to the output pipe
t.stdin.write('hi there ')
t.stdin.write('hi again\n')
#However the input data is buffered first, so call flush() before reading

#Read from the pipe
a = t.stdout.readline()

t.stdin.write('hi back at you\n')

a = t.stdout.readline()

Subprocesses — Python 3.8.5 documentation, PIPE) stdout, stderr = await proc.communicate() print(f'[{cmd!r} exited with Because all asyncio subprocess functions are asynchronous and asyncio provides functions, it is easy to execute and monitor multiple subprocesses in parallel. To read a stream without blocking regardless of the operating system in Python you would do it by using Queue.get_nowait() function: import sys. from subprocess import PIPE, Popen from threading. import Thread. try: from queue import Queue, Empty. except ImportError: from Queue import Queue, Empty # python 2.x. ON_POSIX = 'posix' in sys.builtin

I now switched from using subprocess to using pexpect. My syntax is now as follows:

child = pexpect.spawn('rosrun ros_pkg ros_node')
command = child.sendline('new command')
output = child.read_nonblocking(10000, timeout=1)
command = child.sendline('new command')
output = child.read_nonblocking(10000, timeout=1)

Many thanks to novel_yet_trivial on reddit:

subprocess — Subprocess management — Python 3.8.5 , This module intends to replace several older modules and functions: The input argument is passed to Popen.communicate() and thus to the subprocess's The function is implemented using a busy loop (non-blocking call and short sleeps). the Process.wait() method is asynchronous, whereas subprocess.Popen.wait() method is implemented as a blocking busy loop; the universal_newlines parameter is not supported. This class is not thread safe. See also the Subprocess and Threads section. coroutine wait ¶ Wait for the child process to terminate. Set and return the returncode attribute.

Issue 1191964: add non-blocking read and write , Several times in the past few years I've done stuff with os.system, popenN, If there were a blocking "read x bytes" type call, could you not do some Personally, I would factor out the code for Popen.communicate() in to a� The subprocess module provides a consistent interface to creating and working with additional processes. It offers a higher-level interface than some of the other available modules, and is intended to replace functions such as os.system(), os.spawn*(), os.popen*(), popen2.*() and commands.*().

subprocess, Invoke a blocking process; Invoke a non-blocking process; Using PIPE; When to use shell=True; References Not using Popen.communicate() or call() will result in a zombie process. To make the child process flush, there are several ways like pexpect. The run()function was added in Python 3.5; if you need to retain compatibility with older versions, see the Older high-level APIsection., *, stdin=None, input=None, stdout=None, stderr=None, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None) Run the command described by args.

Interacting with a long-running child process in Python, The Python subprocess module is a powerful swiss-army knife for launching It comes with several high-level APIs like call, check_output and (starting there's no more calls to communicate; instead, proc.wait just waits for the As long as the child process is running, the thread will dutifully block on that� The Python subprocess module is a powerful swiss-army knife for launching and interacting with child processes. It comes with several high-level APIs like call, check_output and (starting with Python 3.5) run that are focused at child processes our program runs and waits to complete.