Allowing Ctrl-C to interrupt a python C-extension
python catch ctrl+c
python custom signal
python signal handler arguments
I'm running some computationally heavy simulation in (home-made) C-based python extensions. Occasionally I get stuff wrong and would like to terminate a simulation. However, Ctrl-C doesn't seem to have any effect (other than printing
^C to the screen) so I have to kill the process using
kill or the system monitor.
As far as I can see python just waits for the C extension to finish and doesn't really communicate with it during this time.
Is there a way to make this work?
Update: The main answers (for my specific problem) turned out to be:
1. rewrite the code to regularly pass control back to the caller (answer Allowing Ctrl-C to interrupt a python C-extension below), or
PyErr_CheckSignals() (answer https://stackoverflow.com/a/33652496/423420 below)
I would redesign the C extensions so that they don't run for a long period.
So, split them into more elementary steps (each running for a short period of time, e.g. 10 to 50 milliseconds), and have these more elementary steps called by Python code.
continuation passing style might be relevant to understand, as a programming style...
How to catch signal (ctrl-c) in python when running C extension , By default, the Python interpreter sets things up so that control-C will be better off letting Python do its default thing of raising KeyboardInterrupt and to regular Python programming, with some minimal extensions added. Forcing a stop is useful when your program locks up and won’t respond. The trick is to press Ctrl+C (the Ctrl key and the C key at the same time; don. t press the Shift key). Make sure the Python window is active (by clicking the window) when you do — or you might close the wrong program!
However, Ctrl-C doesn't seem to have any effect
Ctrl-C in the shell sends
SIGINT to the foreground process group.
python on receiving the signal sets a flag in C code. If your C extension runs in the main thread then no Python signal handler will be run (and therefore you won't see
KeyboardInterrupt exception on
Ctrl-C) unless you call
PyErr_CheckSignals() that checks the flag (it means: it shouldn't slow you down) and runs Python signal handlers if necessary or if your simulation allows Python code to execute (e.g., if the simulation uses Python callbacks). If the extension runs in a background thread then it is enough to release GIL (to allow Python code to run in the main thread that enables the signal handlers to run).
Related: Cython, Python and KeybordInterrupt ingored
Control-C handling in Python and Trio, The C extension interface is specific to CPython, and extension modules do It may abort with a fatal error for certain errors, or return NULL if the module could not be initialized satisfactorily. This allows applications to not worry about creating direct or indirect circular Let's follow the control flow into PyList_SetItem() . 1.1. A Simple Example ¶. Let’s create an extension module called spam (the favorite food of Monty Python fans…) and let’s say we want to create a Python interface to the C library function system () 1. This function takes a null-terminated character string as argument and returns an integer.
Python has a signal handler installed on
SIGINT which simply sets a flag that is checked by the main interpreter loop. For this handler to work properly, the Python interpreter has to be running Python code.
You have a couple of options available to you:
Py_END_ALLOW_THREADSto release the GIL around your C extension code. You cannot use any Python functions when not holding the GIL, but Python code (and other C code) may run concurrently with your C thread (true multithreading). A separate Python thread can execute alongside the C extension and catch Ctrl+C signals.
- Set up your own
SIGINThandler and call the original (Python) signal handler. Your
SIGINThandler can then do whatever it needs to do to cancel the C extension code and return control to the Python interpreter.
1. Extending Python with C or C++, There are a number of alternatives to writing your own C extensions, If you want more control, use PyRun_String() ; see the source for PyRun_SimpleString() in can't allow the PyRun_InteractiveLoop() to stop while waiting for user input. My guess is that Python already adds its own console control handler to translate CTRL-C events into KeyboardInterrupt exceptions in the main thread. The problem is that this handler is asynchronous -- it simply sets a flag to process when the interpreter runs next.
There is an alternative way to solve this problem if you do not want your C Extension (or ctypes DLL) to be tied to Python, such as a case where you want to create a C library with bindings in multiple languages, you must allow your C Extension to run for long periods, and you can modify the C Extension:
Include the signal header in the C Extension.
Create a signal handler typedef in the C Extension.
typedef void (*sighandler_t)(int);
Add signal handlers in the C extension that will perform the actions necessary to interrupt any long running code (set a stop flag, etc.), and save the existing Python signal handlers.
sighandler_t old_sig_int_handler = signal(SIGINT, your_sig_handler); sighandler_t old_sig_term_handler = signal(SIGTERM, your_sig_handler);
Restore the existing signal handlers whenever the C extension returns. This step ensures that the Python signal handlers are re-applied.
signal(SIGINT, old_sig_int_handler); signal(SIGTERM, old_sig_term_handler);
If the long-running code is interrupted (flag, etc.), return control to Python with a return code indicating the signal number.
In Python, send the signal received in the C extension.
import os import signal status = c_extension.run() if status in [signal.SIGINT, signal.SIGTERM]: os.kill(os.getpid(), status)
Python will perform the action you are expecting, such as raising a KeyboardInterrupt for SIGINT.
Extending/Embedding FAQ, The signal.signal() function allows defining custom handlers to be executed when exceptions) and SIGINT is translated into a KeyboardInterrupt exception if the A Python signal handler does not get executed inside the low-level (C) signal And a program is run that intercepts Ctrl-C, hitting a Ctrl-C will kill the program without allowing it to run a handler of that keystroke. On the other hand, if a shell is started by running C:\cygwin\bin\bash.exe directly, then a program executed under bash interrupted with Ctrl-C will handle that interrupt correctly.
signal — Set handlers for asynchronous events, When a script is passed as first argument or the -c option is used, enter Some extensions may not work properly if their initialization routine is called more than and allows embedding code to control IO encoding when the environment Calling this function from a thread when the runtime is finalizing will terminate the Just for anyone who wants to use this question: the keyboard interrupt shortcut in pycharm is Ctrl + F2 – Edwin thebreadwin Sep 30 '16 at 17:52. 6. No, Ctrl-F2 is equivalent to pressing the PyCharm 'stop' button.
Initialization, Finalization, and Threads, KeyboardInterrupt The user pressed the interrupt key (Ctrl-C, Ctrl-Break, of Python, or of the extension in question, with all possible details to allow them to Concerning your new issue, you might want to look into "switch debounce". This is something that can happen when you push a button and the simple answer is that cheap buttons/switches can result in multiple REALLY fast 0->1->0->1 changes for each press/release which is why you are getting print outs when you release the button as well.
Python in a Nutshell: A Desktop Quick Reference, Most C API functions also return an error indicator, usually NULL if they are Much like PyErr_SetImportError() but this function allows for specifying a In all cases, the default effect for SIGINT is to raise the KeyboardInterrupt exception. safe recursive calls at the C level, both in the core and in extension modules. Noio: 2 reasons. First, SIGINT can be sent to your process any number of ways (e.g., 'kill -s INT <pid>'); I'm not sure if KeyboardInterruptException is implemented as a SIGINT handler or if it really only catches Ctrl+C presses, but either way, using a signal handler makes your intent explicit (at least, if your intent is the same as OP's).
- See stackoverflow.com/questions/1112343/…
- related: CTRL+C doesn't interrupt call to shared-library using CTYPES in Python
- Read signal-safety(7) it is extremely relevant
- Sorry, not an option at all in this case :) It's a simulation with a huge number of steps & speed is essential. Interfacing with python at each step (or even at regular intervals) would ruin the efficiency.
- Try to bunch the steps into something lasting a few milliseconds. Then the overhead of going to Python is negligible...
- It's certainly worth thinking about, but it does raise a lot of issues with memory management etc. Thanks!
- Haven't implemented it yet, but this is definitely on my todo list. Thanks again!
- Is there a pure-python way to detect SIGINT in a callback, main.py -> long-running scipy optimizer which calls a shared lib -> callback.py ? (Shall I ask a new question ?)
- @denis: " pure-python way to detect SIGINT" -> KeyboardInterrupt is raised in the main thread (with the default signal handler).
- This should be the accepted answer. There are legit reasons you may have to wait on a condition variable or something if you are doing blocking I/O that are unavoidable. You cannot always rearchitect around it.
- (1) releasing GIL would make any difference only if the C extension runs in a background thread. (2) call
Py_Err_CheckSignals()instead of manuall calling Python signal handler
- Read signal-safety(7)