How do I step into arbitrary generator function call from pdb console?

pdb step into
pip install pdb
python breakpoint()
python debugger online
pdb cheat sheet
python debugger vscode
pdb list breakpoints
python debug flag

I have a generator function that misbehaves, which I would like to call from the pdb console and then step through each iteration of it to see which part of it is wrong. I was hoping I could

(Pdb) !pdb.runcall(broken_function, with_arg)

but it since the function is a generator all I got back was

<generator object broken_function at 0x2badc30>

Does anyone have any idea of what I can do at this point?

Edit: I should have made this clear earlier: I'd of course like to just set a breakpoint in the relevant loop, but I'm running the code off of a (for me) read-only file system, which makes that impractical.

You can easily step into any generator and step through it, my answer is somewhat similar to toes.

Let say we have a function

no = 4
def func(no):
    for i in range(0,no):
        yield i

To step into the function func and iterating through it, you can use pdb.runcall and an iterator next, so I have used both and created a function debug_gen and this function will help in debugging the generator.

def debug_gen(func,*args,**kwargs):
x = func(*args, **kwargs)
results = []
try:
    while True:
            result = pdb.runcall(next, x)
            results.append(result)
except StopIteration:
    return results

so you just have to pass the function to debug_gen with arguments and keyword arguments (if any) and after that, you have to call the debug_gen function anywhere to debug any generator function.

How do I step into arbitrary generator function call from pdb console?, You can easily step into any generator and step through it, my answer is somewhat similar to toes. Let say we have a function no = 4 def func(no): for i in range(0  Add import pdb; pdb.set_trace() to your code at points where you want to debug. In the debugger you can step into code and you can execute arbitrary statements by prefixing them with !. For details, continue reading. Naive debugging. Let’s assume for a moment that you have a new virtualenv.

Edit: Short version:

Options:

  1. Target your function using pdb.run('my_module.my_method()') and step through the generator.

  2. Call pdb as a script python -m pdb my_script.py.

Using test file yippy_dog.py:

def yippy_dog():
    for i in range(10):
        print("YIP YIP!")

yippy_dog()

To target a generator from the pdb console, import your required modules and then call .run() on an invocation of the relevant method:

>>> import pdb
>>> import yippy_dog
YIP YIP!
~~~
YIP YIP!
YIP YIP!
>>> pdb.run('yippy_dog.yippy_dog()')
> <string>(1)<module>()
(Pdb) s
--Call--
> /private/tmp/yippy_dog.py(1)yippy_dog()
-> def yippy_dog():
(Pdb) s
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):
(Pdb) s
> /private/tmp/yippy_dog.py(3)yippy_dog()
-> print("YIP YIP!")
(Pdb) s
YIP YIP!
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):
(Pdb) s
> /private/tmp/yippy_dog.py(3)yippy_dog()
-> print("YIP YIP!")
(Pdb) s
YIP YIP!
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):
(Pdb) s
> /private/tmp/yippy_dog.py(3)yippy_dog()
-> print("YIP YIP!")
(Pdb) s
YIP YIP!
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):
(Pdb)

To run through a generator calling pdb as a script from the command line, invoke the pdb module and step to the relevant frame:

$ python -m pdb yippy_dog.py
> /private/tmp/yippy_dog.py(1)<module>()
-> def yippy_dog():
(Pdb) step
> /private/tmp/yippy_dog.py(5)<module>()
-> yippy_dog()
(Pdb) step
--Call--
> /private/tmp/yippy_dog.py(1)yippy_dog()
-> def yippy_dog():
(Pdb) step
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):
(Pdb) step
> /private/tmp/yippy_dog.py(3)yippy_dog()
-> print("YIP YIP!")
(Pdb) step
YIP YIP!
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):
(Pdb) step
> /private/tmp/yippy_dog.py(3)yippy_dog()
-> print("YIP YIP!")
(Pdb) step
YIP YIP!
> /private/tmp/yippy_dog.py(2)yippy_dog()
-> for i in range(10):

You can, of course, manually force breakpoints by placing set_trace() inside your generator, but that requires modification/placing debugging in your target code, which may not be ideal.

26.2. pdb — The Python Debugger, pdb.py can also be invoked as a script to debug other scripts. For example: python -m pdb When runcall() returns, it returns whatever the function call returned. When you run a debugger like pdb, realgud in contrast to gud, makes an attempt to sort the specific debugger options, e.g to pdb, from the debugged script options. In other words it parses what you type; undoubtedly that parsing is failing here.

This is what I would try:

  1. Set a break point in the body of the generator.
  2. At the pdb prompt call a function which iterates through all of the values of the generator.

Example:

def mygen():
  for e in ...:
    ...               -- set a break point here, for instance
    yield ...

def testgen(g):       -- call this function from the pdb prompt
  for e in g: pass

You can also have testgen() directly call mygen() instead of requiring you to pass the generator in.

pdb — The Python Debugger, pdb.py can also be invoked as a script to debug other scripts. When runcall() returns, it returns whatever the function call returned. If given, header is printed to the console just before debugging begins. the code argument (which is an arbitrary expression or statement to be executed in the current environment). The debugger will not step into frames that originate in a module that matches one of these patterns. 1 By default, Pdb sets a handler for the SIGINT signal (which is sent when the user presses Ctrl-C on the console) when you give a continue command.

I tried this. Seems to work, but maybe it's not what you are looking for:

# gen.py

def gen():
    while True:
        yield "foo"

if __name__ == "__main__":
    g = gen()

Then I fire up pdb:

$ python2.7 -m pdb gen.py 
> /home/elasand/prog/gen.py(1)<module>()
-> def gen():
(Pdb) s
> /home/elasand/prog/gen.py(5)<module>()
-> if __name__ == "__main__":
(Pdb) s
> /home/elasand/prog/gen.py(6)<module>()
-> g = gen()
(Pdb) s
--Return--
> /home/elasand/prog/gen.py(6)<module>()->None
-> g = gen()
(Pdb) !next(g)
'foo'

Python Debugging, When runcall() returns, it returns whatever the function call returned. The run* functions and set_trace() are aliases for instantiating the Pdb class and The debugger will not step into frames that originate in a module that matches one of listing, and evaluation of arbitrary Python code in the context of any stack frame​. The <Targetname>.PDB files are generated by the linker and it is related with the target executable or the DLL. This file contains the complete debug info. When we are debugging, we need this ‘.pdb’ file for aligning to the symbols. The timestamp of the target file and the PDB should match.

Working with pdb in Python - Makimo Tech Blog, As with every console tool, pdb may seem a bit intimidating. (place in which program should halt the execution and yield the control to the programmer). By calling python -m pdb <script> , then, we call pdb as a script to debug other scripts. To select a function breakpoint from the call stack. While debugging, open the Call Stack window by selecting Debug > Windows > Call Stack. In the Call Stack window, right-click a function and select Run To Cursor, or press Ctrl + F10. To visually trace the call stack, see Map methods on the call stack while debugging.

PyCharm Educational 183.4588.180 Release Notes, call from "Override Method" helper for methods whose parent class implementation raises Feature, PY-10956, View numpy array in debug or console Bug, PY-27910, Using Python 3.7's breakpoint() starts pdb session during Performance, PY-29869, Generating Skeletons slow for docker-​compose. The Python debugger will automatically start over when it reaches the end of your program. Whenever you want to leave the pdb console, type the command quit or exit. If you would like to explicitly restart a program at any place within the program, you can do so with the command run. Using the Debugger to Move through a Program.

javascripthon · PyPI, Javascript for refined palates: a Python 3 to ES6 Javascript translator. Function's args and call parameters is designed to be the first step in a pipeline that translates your Pyhton code into standard functions, generator functions, async functions;; parameters defaults; A pj console script is also automatically installed: A set of debugging decorators which respects Django's settings in case the package is withing a Django project. It allows a user to PDB into a function, do a Line profiler, inspect an object and Disasemble the function. Fork of winpdb after this was unmaintained. Improved version of pdb that is part of IPython but also can be used separately.

Comments
  • I saw your edit and added an edit to my answer that makes the answer clear (instead of having it be in the middle of the example). You don't need to set a breakpoint in the relevant loop.
  • This seems by far like the cleanest solution. I don't have the problem anymore, but I'll accept yours as the answer unless someone has a strong objection. :)
  • Thanks, appreciate that.
  • This does not seem to work when yippy_dog is a generator function, since it returns immediately with the generator.
  • The problem is that I'm running the code from a read-only directory, so modifying the code is unfortunately not an option! I guess I should have specified that.
  • perhaps you can call pdb.run("for e in mygen(): pass") instead of defining testgen().
  • You might be able to call next(the_gen) from pdb?
  • Not sure if this answer is what the OP wanted but this solved the problem for me. You can do this in ipython by importing the generator function, grabbing a generator (e.g. g= gen()), and then calling ipdb.runcall on the next method (e.g. ipdb.runcall(g.next)).