How do I add custom field to Python log format string?

python logging multiple modules
python logging multiple files
python logging json
python logging filehandler
python logging dictconfig
python logging format
python custom logger
python root logger

My current format string is:

formatter = logging.Formatter('%(asctime)s : %(message)s')

and I want to add a new field called app_name and which will have a different value in each script that contains this formatter.

import logging
formatter = logging.Formatter('%(asctime)s %(app_name)s : %(message)s')
syslog.setFormatter(formatter)
logger.addHandler(syslog)

But I'm not sure how to pass that app_name value to the logger to interpolate into the format string. I can obviously get it to appear in the log message but passing it each time but this is messy.

I've tried:

logging.info('Log message', app_name='myapp')
logging.info('Log message', {'app_name', 'myapp'})
logging.info('Log message', 'myapp')

but none work.

You could use a LoggerAdapter so you don't have to pass the extra info with every logging call:

import logging
extra = {'app_name':'Super App'}

logger = logging.getLogger(__name__)
syslog = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s %(app_name)s : %(message)s')
syslog.setFormatter(formatter)
logger.setLevel(logging.INFO)
logger.addHandler(syslog)

logger = logging.LoggerAdapter(logger, extra)
logger.info('The sky is so blue')

logs (something like)

2013-07-09 17:39:33,596 Super App : The sky is so blue

Filters can also be used to add contextual information.

import logging

class AppFilter(logging.Filter):
    def filter(self, record):
        record.app_name = 'Super App'
        return True

logger = logging.getLogger(__name__)
logger.addFilter(AppFilter())
syslog = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s %(app_name)s : %(message)s')
syslog.setFormatter(formatter)
logger.setLevel(logging.INFO)
logger.addHandler(syslog)

logger.info('The sky is so blue')

produces a similar log record.

How do I add custom field to Python log format string , My current format string is: formatter = logging.Formatter('%(asctime)s : %(​message)s'). and I want to add a new field called app_name and  Teams. Q&A for Work. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.

You need to pass the dict as a parameter to extra to do it that way.

logging.info('Log message', extra={'app_name': 'myapp'})

Proof:

>>> import logging
>>> logging.basicConfig(format="%(foo)s - %(message)s")
>>> logging.warning('test', extra={'foo': 'bar'})
bar - test 

Also, as a note, if you try to log a message without passing the dict, then it will fail.

>>> logging.warning('test')
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 846, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
    s = self._fmt % record.__dict__
KeyError: 'foo'
Logged from file <stdin>, line 1

logging — Logging facility for Python, Handlers send the log records (created by loggers) to the appropriate destination​. (Note that this means that you can use keywords in the format string, together with a single These custom attributes can then be used as you like. args , which are combined using msg % args to create the message field of the record. Example of how to filter or apply custom formatting using Python's logging library - custom-log-filtering-and-formatting.py Skip to content All gists Back to GitHub

Another way is to create a custom LoggerAdapter. This is particularly useful when you can't change the format OR if your format is shared with code that does not send the unique key (in your case app_name):

class LoggerAdapter(logging.LoggerAdapter):
    def __init__(self, logger, prefix):
        super(LoggerAdapter, self).__init__(logger, {})
        self.prefix = prefix

    def process(self, msg, kwargs):
        return '[%s] %s' % (self.prefix, msg), kwargs

And in your code, you would create and initialize your logger as usual:

    logger = logging.getLogger(__name__)
    # Add any custom handlers, formatters for this logger
    myHandler = logging.StreamHandler()
    myFormatter = logging.Formatter('%(asctime)s %(message)s')
    myHandler.setFormatter(myFormatter)
    logger.addHandler(myHandler)
    logger.setLevel(logging.INFO)

Finally, you would create the wrapper adapter to add a prefix as needed:

    logger = LoggerAdapter(logger, 'myapp')
    logger.info('The world bores you when you are cool.')

The output will look something like this:

2013-07-09 17:39:33,596 [myapp] The world bores you when you are cool.

Logging Cookbook, ERROR) # create formatter and add it to the handlers formatter = logging. can then be output using a suitable format string, or if needed a custom Formatter . Formatters work by putting in one or more replacement fields or placeholders — defined by a pair of curly braces {} — into a string and calling the str.format() method. You’ll pass into the method the value you want to concatenate with the string.

15.7. logging — Logging facility for Python, Handlers send the log records (created by loggers) to the appropriate destination​. (Note that this means that you can use keywords in the format string, together with a single These custom attributes can then be used as you like. args , which are combined using msg % args to create the message field of the record. You can format strings in a number of ways using Python. The main emphasis of formatting is to present the string in a form that is both pleasing to the user and easy to understand. Formatting doesn’t mean adding effects in this case, but refers merely to the presentation of the data.

Using mr2ert's answer, I came up with this comfortable solution (Though I guess it's not recommended) - Override the built-in logging methods to accept the custom argument and create the extra dictionary inside the methods:

import logging

class CustomLogger(logging.Logger):

   def debug(self, msg, foo, *args, **kwargs):
       extra = {'foo': foo}

       if self.isEnabledFor(logging.DEBUG):
            self._log(logging.DEBUG, msg, args, extra=extra, **kwargs)

   *repeat for info, warning, etc*

logger = CustomLogger('CustomLogger', logging.DEBUG)
formatter = logging.Formatter('%(asctime)s [%(foo)s] %(message)s') 
handler = logging.StreamHandler()
handler.setFormatter(formatter) 
logger.addHandler(handler)

logger.debug('test', 'bar')

Output:

2019-03-02 20:06:51,998 [bar] test

This is the built in function for reference:

def debug(self, msg, *args, **kwargs):
    """
    Log 'msg % args' with severity 'DEBUG'.

    To pass exception information, use the keyword argument exc_info with
    a true value, e.g.

    logger.debug("Houston, we have a %s", "thorny problem", exc_info=1)
    """
    if self.isEnabledFor(DEBUG):
        self._log(DEBUG, msg, args, **kwargs)

Python Logging, error("testing", elapsed_seconds=4, model_name='example') . I know I could use something like Python structlog module but I though it might be  In the W3C Logging Fields dialog box, click Add Field. In the Add Custom Field dialog box, enter a name in Field name, and select one of the following for the Source Type: Request Header, Response Header, or Server Variable. In Source, select a source from the list, or enter the name of a custom source. Click OK, then click OK again.

Logging: Ability to add extra fields to logging calls · Issue #5888 , My current format string is: formatter = logging.Formatter('%(asctime)s : %(​message)s') and I want to add a new field called app_name and  The format () reads the type of arguments passed to it and formats it according to the format codes defined in the string. For positional arguments. Here, Argument 0 is a string "Adam" and Argument 1 is a floating number 230.2346. Note: Argument list starts from 0 in Python. The string "Hello {0}, your balance is {1:9.3f}" is the template

How do I add custom field to Python log format string?, In this post, I am going to write a Python custom logging handler that will send This version lets you send JSON data in addition to string data. String Formatting in Python. In Python, we can take advantage of two separate methods of string interpolation. String interpolation is a term used to describe the process of evaluating a string value that is contained as one or more placeholders. To put it simply, it helps developers with string formatting and concatenation.

Python Custom Logging Handler Example, import coloredlogs, logging >>> coloredlogs.install(level='DEBUG') >>> logger = logging. Screen shot of colored logging with custom log format. Formatter class in Python 3.2+) to strings containing regular expression patterns A string containing a regular expression that matches a “replacement field” as defined in​  To add to the other answers, here are the variable list from Python Documentation. Directive Meaning Notes %a Locale’s abbreviated weekday name. %A Locale’s full weekday name.

Comments
  • Do you really want to pass this in to every log call? If so, look at the docs where it says "This functionality can be used to inject your own values into a LogRecord…" But this seems like a prime case for using logger = logging.getLogger('myapp') and having it baked into the logger.info call.
  • How can we specify that in a config.ini file? I wish to add the current host name socket.gethostname().
  • I have this sample not working for me. import uuid uniqueId = str(uuid.uuid4()) extra = {"u_id" : uniqueId} RotatingHandler = RotatingFileHandler(LOG_FILENAME,encoding='utf-8',maxBytes=maxSize, backupCount=batchSize) logger.basicConfig(handlers=[RotatingHandler],level=logLevel.upper(),format='%(levelname)s %(u_id)s %(funcName)s %(asctime)s %(message)s ',datefmt='%m/%d/%Y %I:%M:%S %p') logger = logger.LoggerAdapter(logger=logger, extra=extra)
  • Is it possible to add a field "level" which is equal to "levelname"? See: How can I rename "levelname" to "level" in Python log messages?
  • Can I just pass a string of Extra Info. Something like this: "Error occurred for employee ID 1029382" Without creating any dictionary.
  • Will this also work for logging.info()? It failed when I tried last. :/
  • I like @mr2ert answer. You can give a default value to the extra field by extending the logging.Formatter class: class CustomFormatter(logging.Formatter): def format(self, record): if not hasattr(record, 'foo'): record.foo = 'default_foo' return super(CustomFormatter, self.format(record) h = loggin.StreamHandler() h.setFormatter(CustomFormatter('%(foo)s %(message)s') logger = logging.getLogger('bar') logger.addHandler(h) logger.error('hey!', extra={'foo': 'FOO'}) logger.error('hey!')
  • This method is faster but you need to add the extra lines on each log message which is easy to forget and prone to errors. Replacing the super() calls is messier than the answer from unutbu.
  • @Prakhar Mohan Srivastava Yes It'll work fine for logging.info() as well. What error message are you getting ?
  • Can I just pass a string of Extra Info. Something like this: "Error occurred for employee ID 1029382" Without creating any dictionary and passing the keys
  • Is any alternative for python 2.7?