How to escape an asterisk in a Python related library?

python 3 escape characters
python regex
python escape backslash
python escape backslash in string
python escape percent
python escape string for sql
python escape newline
t in python

I'm working with PYKD, a library, used for writing Python scripts for Windbg.

One of the functions, typedVar, seems not to work when handling type names who contain an asterisk:

(In a Windbg session):

.load pykd.pyd // Load PYKD library
!py            // enter an interactive Python session

>>> print typedVar('CMap<unsigned long,unsigned long,int,int>', 0x02d729dc).m_nCount
Int4B at 0x2d729e8 Value: 0x4 (4)
=> ok!
>>> print typedVar('CMap<int,int,ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > > >', 0x02ca2568).m_nCount
Int4B at 0x2ca2574 Value: 0x7 (7)
=> ok!

>>> print typedVar('CMap<int,int,void *,void *>', 0x0054ac10).m_nCount
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeException: CMap<int,int,void *,void *> : invalid type name
=> NOK (most probably because of the asterisk)

I believe this is caused by the asterisk, being interpreted as a wildcard, so I'd like to use an escape character, in order to use the asterisk as a "normal" character, but this seems not to be that simple:

Using a backslash:

>>> print typedVar('CMap<int,int,void \*,void \*>', 0x0054ac10).m_nCount
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeException: CMap<int,int,void \*,void \*> : invalid type name

Doubling the asterisk:

>>> print typedVar('CMap<int,int,void **,void **>', 0x0054ac10).m_nCount
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeException: CMap<int,int,void **,void **> : invalid type name

Does anybody know the escape character for regular expressions in Python (in case it's not a backslash or doubling the character)?

Thanks in advance

Oh, before I forget: the mentioned type is present in the application's symbols, as you can see here:

for tp in app.enumTypes("*CMap<*"):
  print tp
...
CMap<int,int,void *,void *>
...

Edit after first comment and reply

These are the commands I used (with the r and the u):

>>> print typedVar(r'CMap<int,int,void *,void *>', 0x0054ac10).m_nCount
>>> print typedVar(u'CMap<int,int,void *,void *>', 0x0054ac10).m_nCount
>>> print typedVar(rr'CMap<int,int,void *,void *>', 0x0054ac10).m_nCount
>>> print typedVar(r'''CMap<int,int,void *,void *>''', 0x0054ac10).m_nCount
>>> print typedVar(r''CMap<int,int,void *,void *>'', 0x0054ac10).m_nCount
>>> print typedVar(ru'CMap<int,int,void *,void *>', 0x0054ac10).m_nCount
>>> print typedVar(ru'CMap<int,int,void *,void *>', 0x0054ac10).m_nCount

Edit after further use of the newest PYKD library

Unfortunately there still is an issue, as you can see from following excerpt:

0:000> .load pykd.pyd
0:000> .chain
Extension DLL search Path:
    ...
Extension DLL chain:
    pykd.pyd: image 0.3.4.2, API 1.0.0, built Sat Nov 17 13:06:54 2018
        [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x86\winext\pykd.pyd]
    ...
0:000> !py

  >>> dprintln("%d" % typedVar('Application!CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>', 0x064ad440).m_nCount)
  2
  => An asterisk in the type name can sometimes be handled
  >>> dprintln("%d" % typedVar('Application!CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>', 0x064ad328).m_nCount)
  Traceback (most recent call last):
    File "<console>", line 1, in <module>
  AttributeError: typed var has no field 'm_nCount'
  => But sometimes there still are issues.
     The error message clearly shows that the typename is known.

What might be the issue here? Can I do anything to add more detailed debugging information?

Edit after new proposal from ussrhero

typeInfo seems to be empty:

>>> print(typeInfo('Application!CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>'))
class/struct : CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *> Size: 0x0 (0)

Here another CMap related typeInfo for reference reasons:

>>> print(typeInfo('Application!CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>'))
class/struct : CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int> Size: 0x1c (28)
   +0000 __VFN_table             : VTable*
   =0000000000 classCObject      : CRuntimeClass
   +0004 m_pHashTable            : CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>::CAssoc**
   +0008 m_nHashTableSize        : UInt4B
   +000c m_nCount                : Int4B
   +0010 m_pFreeList             : CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>::CAssoc*
   +0014 m_pBlocks               : CPlex*
   +0018 m_nBlockSize            : Int4B

Here a typeInfo of a non-existing class (also for reference reasons):

>>> print(typeInfo('Application!NonExisting_Class'))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
SymbolException: 'NonExisting_Class' - symbol not found

=> So the class, giving the problem, is known, but can apparently not be handled.

Edit after last update from ussrhero:

dt and dx seem to mention everything is ok for the other CMap:

0:000> dt 0x064ad440 CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>
Application!CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>
   +0x000 __VFN_table : 0x01503444 
   +0x004 m_pHashTable     : 0x06ab9ad0  -> (null) 
   +0x008 m_nHashTableSize : 0x186ab
   +0x00c m_nCount         : 0n2
   +0x010 m_pFreeList      : 0x063c953c CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>::CAssoc
   +0x014 m_pBlocks        : 0x063c9518 CPlex
   +0x018 m_nBlockSize     : 0n10
0:000> dt 0x064ad440 CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int> m_nCount
Application!CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>
   +0x00c m_nCount : 0n2
0:000> dx (CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>*) 0x064ad440
(CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>*) 0x064ad440                 : 0x64ad440 [Type: CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int> *]
    [+0x004] m_pHashTable     : 0x6ab9ad0 [Type: CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>::CAssoc * *]
    [+0x008] m_nHashTableSize : 0x186ab [Type: unsigned int]
    [+0x00c] m_nCount         : 2 [Type: int]
    [+0x010] m_pFreeList      : 0x63c953c [Type: CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>::CAssoc *]
    [+0x014] m_pBlocks        : 0x63c9518 [Type: CPlex *]
    [+0x018] m_nBlockSize     : 10 [Type: int]

dt and dx seem to mention there is a problem for that particular CMap:

0:000> dt 0x064ad328 CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>
Application!CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>
0:000> dt 0x064ad328 CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *> m_nCount
Application!CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>
0:000> dx (CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>*) 0x064ad328
Error: Unable to find type 'CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *> *' for cast.

However, the symbols seem not to have a problem:

0:000> x /2 Application!CMap<*vftable*
...
0152e944          Application!CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION *,_RTL_CRITICAL_SECTION *>::`vftable'
...
01503444          Application!CMap<ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > >,wchar_t const *,unsigned int,unsigned int>::`vftable'

Apparently we are dealing here with a Windbg issue. How can we find out if this is a known Windbg issue or a new one? Where are those issues collected? (For your info, I'm working with Windbg 10.0.16299.15 X86 for Windows 10, Version 1803 (OS Build 17134.345), but also Windbg Preview seems to have this bug.

It is a pykd bug: https://githomelab.ru/pykd/pykd/issues/33

It will be fixed next release

Python 2.7 Tutorial, : "\t" is a tab, "\n" is a newline, and "\r" is a carriage return. The answer is simple: escape the asterisk character in your regular expression using the backslash. In particular, use ‘\*’ instead of ‘*’.

try to print out type information:

>>>print( typeInfo('Application!CMap<unsigned int,unsigned int,_RTL_CRITICAL_SECTION,_RTL_CRITICAL_SECTION *>') )

3: Comments and Quotes, Well organized and easy to understand Web building tutorials with lots of examples of how to use HTML, CSS, JavaScript, SQL, PHP, Python, Bootstrap, Java  The PEP that added this to Python 3.0 is PEP 3132 and it’s not a very long one. Asterisks in list literals. Python 3.5 introduced a ton of new *-related features through PEP 448. One of the biggest new features is the ability to use * to dump an iterable into a new list.

Unable to comment but you could try r'string' which is explained here What exactly do "u" and "r" string flags do, and what are raw string literals?

Or if you're able to, swap the pointer for any unsigned 64 bit data type.

What does '\\r' do in Python?, On this page: commenting with #, multi-line strings with """ """, printing multiple objects, the backslash "\" as the escape character, '\t', '\n', '\r', and '\\'. Get Started  I'm quite busy with other non-asterisk related projects in my personal time, so if you're a good Python programmer, and working with this library and Asterisk on a daily basis -- please consider maintaining the project! Contact me for more information: r@rdegges.com. In the meantime, I'm happy to accept pull requests and cut releases as needed.

typedVar does not interpet type name. You must use the same type name as it is contained in the symbol information.

Try to find it with dt command:

dt CMap*

Or with pykd:

print( "\n".join( module('ModuleName').enumTypes('CMap*') )

I believe it will look like 'CMap<int,int,void*,void*>' ( without any space )

2.4.1 String literals, The glob module finds all the pathnames matching a specified pattern according to the Special characters in drive/UNC sharepoints are not escaped, e.g. on  I’m quite busy with other non-asterisk related projects in my personal time, so if you’re a good Python programmer, and working with this library and Asterisk on a daily basis – please consider maintaining the project! Contact me for more information: r @ rdegges. com. In the meantime, I’m happy to accept pull requests and cut releases

Python Escape Characters, They're used for specifying a character class, which is a set of It's also used to escape all the metacharacters so you can still match them in To use a similar example, ca+t will match cat (1 a), caaat (3 a's), but won't match ct. The re module provides an interface to the regular expression engine,  You may want to add a logdir if you call agitb.enable() before you have an agi.AGI() handle. Alternatively, if you have caught an exception and want agitb to display it for you, call agitb.handler(). The optional argument to handler() is a 3-item tuple (etype, evalue, etb) just like the value of sys.exc_info().

glob — Unix style pathname pattern expansion, This module provides regular expression matching operations similar to those found in Perl To match a literal '|', use \|, or enclose it inside a character class, as in [|]. Escape all the characters in pattern except ASCII letters, numbers and '_'. Powered by a free Atlassian Confluence Open Source Project License granted to Asterisk Project. Evaluate Confluence today . Powered by Atlassian Confluence 5.6.6 , Team Collaboration Software

Regular Expression HOWTO, Wildcards, also known as globbing, look very similar in their syntax to regular expressions. If you want to use regular expressions in Python, you have to import the re module, in regular expressions, but is also used as an escape character in strings. Square brackets, "[" and "]", are used to include a character class. I’m not sure if this is a brainfart of mine, or a bug, but I can’t get a literal asterisk in my markdown. The following: md`I want a *literal \\*asterisk*` shows as " I want a literal asterisk* " instead of “I want a literal *asterisk” (here in the forum is does work as expected). The quick fix employed, btw: md`I want a *literal ${html`*`}asterisk*`

Comments
  • Have you tried typedVar(r'CMap***...') (note the r)?
  • @Jan: thanks for the quick reply, but adding the r, or u, or ru, ..., does not solve the problem.
  • Potentially related: stackoverflow.com/questions/46235976/…
  • @ThomasWeller: my question is a consequence of my original question "stackoverflow.com/questions/47651846/…", and after having visited the URL you mentioned, I believe you're on the right track for solving my original question. Unfortunately my tryouts, using the "@!" prefix seem not to work. Can you have a look at my original question and see how it can be handled? Thanks in advance.
  • Wow. Unfortunately I broke PyKd on my machine. I'll try again in a virtual machine. Is it possible for you to provide the C++ code as a minimal reproducible example so that I can focus on the PyKd part and don't need to bother with implementation?
  • Thanks for your reply. If you reply this on my original question "stackoverflow.com/questions/47651846/…", I'll award you the bounty for it. Do you know if the issue only applies on void* or also on other pointer members?
  • Any templates with * or [] are affected
  • try to update pykd to 0.3.4.2 pip install pykd githomelab.ru/pykd/pykd/wikis/0.3.4.2
  • Thanks. I've downloaded the latest version and I can confirm that it works: the typedVar() function is working fine now, also for templates, containing an asterisk.
  • Thank you for the confirmation
  • Thanks for your quick reply. I've edited the question accordingly, see "Edit after new proposal from ussrhero".
  • try to compare with output by dt / dx / ?? Does windbg print out this correctly?
  • I've edited the question, based on your last comment: the issue seems to be caused by Windbg, not by PYKD, as Windbg commands dt and dx also don't give a correct result.
  • In the meanwhile, I've found out that the issue is caused by missing symbols (the mentioned object was embedded inside a library whose symbols are not included in the global symbols of the application), so this question can be closed.
  • Thanks for the quick reply, but as mentioned to Jan, I followed the advise, mentioned in the URL, but I didn't succeed.