Run and wait for asynchronous function from a synchronous one using Python asyncio
In my code I have a class with properties, that occasionally need to run asynchronous code. Sometimes I need to access the property from asynchronous function, sometimes from synchronous - that's why I don't want my properties to be asynchronous. Besides, I have an impression that asynchronous properties in general is a code smell. Correct me if I'm wrong.
I have a problem with executing the asynchronous method from the synchronous property and blocking the further execution until the asynchronous method will finish.
Here is a sample code:
import asyncio async def main(): print('entering main') synchronous_property() print('exiting main') def synchronous_property(): print('entering synchronous_property') loop = asyncio.get_event_loop() try: # this will raise an exception, so I catch it and ignore loop.run_until_complete(asynchronous()) except RuntimeError: pass print('exiting synchronous_property') async def asynchronous(): print('entering asynchronous') print('exiting asynchronous') asyncio.run(main())
entering main entering synchronous_property exiting synchronous_property exiting main entering asynchronous exiting asynchronous
RuntimeError capturing seems wrong, but if I won't do that, I'll get
RuntimeError: This event loop is already running exception.
asynchronous() function is executed last, after the synchronous one finish. I want to do some processing on the data set by asynchronous method so I need to wait for it to finish.
If I'll add
await asyncio.sleep(0) after calling
synchronous_property(), it will call
main() finish, but it doesn't help me. I need to run
What am I missing? I'm running python 3.7.
Asyncio is really insistent on not allowing nested loops, by design. However, you can always run another event loop in a different thread. Here is a variant that uses a thread pool to avoid having to create a new thread each time around:
import asyncio, concurrent.futures async def main(): print('entering main') synchronous_property() print('exiting main') pool = concurrent.futures.ThreadPoolExecutor() def synchronous_property(): print('entering synchronous_property') result = pool.submit(asyncio.run, asynchronous()).result() print('exiting synchronous_property', result) async def asynchronous(): print('entering asynchronous') await asyncio.sleep(1) print('exiting asynchronous') return 42 asyncio.run(main())
This code creates a new event loop on each sync->async boundary, so don't expect high performance if you're doing that a lot. It could be improved by creating only one event loop per thread using
asyncio.new_event_loop, and caching it in a thread-local variable.
Python & Async Simplified, Coroutines run synchronously until they hit an await and then they pause Try to await a sync function and you'll see Python complain, forget to current coroutine until the one you asked to wait on exits with a result, like this:. The next version of the program has been modified quite a bit. It makes use of Python async features using asyncio/await provided in Python 3. The time and queue modules have been replaced with the asyncio package. This gives your program access to asynchronous friendly (non-blocking) sleep and queue functionality.
There appears to a problem with the question as stated. Restating the question: How to communicate between a thread (containing no async processes and hence considered sync) and an async proces (running in some event loop). One approach is to use two sync Queues. The sync process puts its request/parameters into the QtoAsync, and waits on the QtoSync. The async process reads the QtoAsync WITHOUT wait, and if it finds a request/parameters, executes the request, and places the result in QtoSync.
import queue QtoAsync = queue.Queue() QtoSync = queue.Queue() ... async def asyncProc(): while True: try: data=QtoAsync.get_nowait() result = await <the async that you wish to execute> QtoAsync.put(result) #This can block if queue is full. you can use put_nowait and handle the exception. except queue.Empty: await asyncio.sleep(0.001) #put a nominal delay forcing this to wait in event loop .... #start the sync process in a different thread here.. asyncio.run(main()) #main invokes the async tasks including the asyncProc The sync thread puts it request to async using: req = <the async that you wish to execute> QtoAsync.put(req) result = QtoSync.get()
This should work.
Problem with the question as stated: 1. When the async processes are started with asyncio.run (or similar) execution blocks until the async processes are completed. A separate sync thread has to be started explicity before calling asyncio.run 2. In general asyncio processes depend on other asyncio processes in that loop. So calling a async process from another thread is not permitted directly. The interaction should be with the event loop, and using two queues is one approach.
Stop Waiting! Start using Async and Await!, A native coroutine is a python function defined with async def. the synchronous call took 52.7 seconds, while the async one took 6.5 seconds� This allows a synchronous function to be run asynchronously without blocking an event loop. In the sleep example I posted above, it might look like this: import asyncio from time import sleep async def sleep_async(loop, delay): # None uses the default executor (ThreadPoolExecutor) await loop.run_in_executor(None, sleep, delay) return 'I slept
The easiest way is using an existing "wheel", like asgiref.async_to_sync
from asgiref.sync import async_to_sync
async_to_sync(<your_async_func>)(<.. arguments for async function ..>)
This is a caller class which turns an awaitable that only works on the thread with the event loop into a synchronous callable that works in a subthread. If the call stack contains an async loop, the code runs there. Otherwise, the code runs in a new loop in a new thread. Either way, this thread then pauses and waits to run any thread_sensitive code called from further down the call stack using SyncToAsync, before finally exiting once the async task returns.
Python async/await Tutorial, An asynchronous function in Python is typically called a 'coroutine', which is just a There are a few ways to actually call a coroutine, one of which is the yield� Waiting Synchronously for an Asynchronous Coroutine. If an asyncio event loop is already running by calling loop.run_forever, it will block the executing thread until loop.stop is called [see the docs]. Therefore, the only way for a synchronous wait is to run the event loop on a dedicated thread, schedule the asynchronous function on the loop
I want to make the async call to execute from sync and block it's execution
Just make the sync func async and await the asynchronous function. Async functions are just like normal functions and you can put whatever code you want in them. If you still have a problem modify your question using actual code you are trying to run.
import asyncio async def main(): print('entering main') await synchronous_property() print('exiting main') async def synchronous_property(): print('entering synchronous_property') await asynchronous() # Do whatever sync stuff you want who cares print('exiting synchronous_property') async def asynchronous(): print('entering asynchronous') print('exiting asynchronous') asyncio.run(main())
Developing with asyncio — Python 3.8.5 documentation, While a Task is running in the event loop, no other Tasks can run in the same thread. async def coro_func(): return await asyncio.sleep(1, 42) # Later in another OS thread: future = asyncio.run_coroutine_threadsafe(coro_func(), loop) # Wait for When a coroutine function is called, but not awaited (e.g. coro() instead of� If you want your sync code to call async one, just use reference to running loop and do asyncio.run_coroutine_threadsafe this will return concurrent.Future (NOT asyncio.Future) that you can wait in sync way. As has been mentioned here by /u/isinfinity , asyncio.run_coroutine_threadsafe will help you out.
asyncio — Asynchronous I/O — Python 3.8.5 documentation, asyncio is a library to write concurrent code using the async/await syntax. run Python coroutines concurrently and have full control over their execution;. Introduction This article is the second part of a series on using Python for developing asynchronous web applications. The first part provides a more in-depth coverage of concurrency in Python and asyncio, as well as aiohttp. If you'd like to read more about Asynchronous Python for Web Development [/asynchronous-python-for-web-development/], we've got it covered. Due to the non-blocking nature
Getting Started With Async Features in Python – Real Python, repertoire. You'll learn how to use Python async features to take advantage of IO processes and free up your CPU. A synchronous program is executed one step at a time. The kids are a long-running task with high priority. It's an asynchronous function because you don't have to wait for it to finish. Running an asyncio Program ¶ asyncio.run (coro, *, debug=False) ¶ Execute the coroutine coro and return the result. This function runs the passed coroutine, taking care of managing the asyncio event loop and finalizing asynchronous generators. This function cannot be called when another asyncio event loop is running in the same thread.
Async IO in Python: A Complete Walkthrough – Real Python, It suggests that multiple tasks have the ability to run in an overlapping manner. Synchronous version: Judit plays one game at a time, never two at the same time, Async IO takes long waiting periods in which functions would otherwise be� Asynchronous programming is a type of parallel programming in which a unit of work is allowed to run separately from the primary application thread. When the work is complete, it notifies the main…