How to use different formatters with the same logging handler in python

python logging multiple modules
python logging filehandler
python logging from threads
python remote logging
python multiple loggers
python logging source code
python logging timestamp
python logging async

Is it possible to log to a single destination (i.e. using one FileHandler) with multiple loggers (i.e. logging.getLogger("base.foo") and logging.getLogger("base.bar")), and use different formatters for each of the loggers.

To my understanding it's only possible to assign one formatter to each handle. Maybe it's possible to associate the formatter with a logger rather than the handler?

It's easy to dispatch to different formatters based on record.name. Below is prove-of-concept sample code:

import logging


class DispatchingFormatter:

    def __init__(self, formatters, default_formatter):
        self._formatters = formatters
        self._default_formatter = default_formatter

    def format(self, record):
        formatter = self._formatters.get(record.name, self._default_formatter)
        return formatter.format(record)


handler = logging.StreamHandler()
handler.setFormatter(DispatchingFormatter({
        'base.foo': logging.Formatter('FOO: %(message)s'),
        'base.bar': logging.Formatter('BAR: %(message)s'),
    },
    logging.Formatter('%(message)s'),
))
logging.getLogger().addHandler(handler)

logging.getLogger('base.foo').error('Log from foo')
logging.getLogger('base.bar').error('Log from bar')
logging.getLogger('base.baz').error('Log from baz')

Another way is to open file manually and create two stream handlers from it with different formatters.

Logging Cookbook, Notice that the 'application' code does not care about multiple handlers. In that case, the same format string can be used to get similar output to that shown  The example creates a console logger and adds a formatter to its handler. formatter = logging.Formatter('%(asctime)s %(name)s %(levelname)s: %(message)s') The formatter is created. It includes the date time, logger name, logging level name, and log message. consoleHandler.setFormatter(formatter)

Little fix to excellent Denis's solution.

Logging name system based on hierarchical structure:

The name is potentially a period-separated hierarchical value, like foo.bar.baz (though it could also be just plain foo, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo.

For example, when you setLevel() for some logger, this level will be also applied to child loggers. That's why you might want your formatter will be used for logger and it's child loggers too. For example, 'one.two' formatter should also be applied to 'one.two.three' logger (if no formatter for 'one.two.three' set). Here's version of DispatchingFormatter that do job (Python 3 code):

class DispatchingFormatter:
    """Dispatch formatter for logger and it's sub logger."""
    def __init__(self, formatters, default_formatter):
        self._formatters = formatters
        self._default_formatter = default_formatter

    def format(self, record):
        # Search from record's logger up to it's parents:
        logger = logging.getLogger(record.name)
        while logger:
            # Check if suitable formatter for current logger exists:
            if logger.name in self._formatters:
                formatter = self._formatters[logger.name]
                break
            else:
                logger = logger.parent
        else:
            # If no formatter found, just use default:
            formatter = self._default_formatter
        return formatter.format(record)

Example:

handler = logging.StreamHandler()
handler.setFormatter(DispatchingFormatter({
        'one': logging.Formatter('%(message)s -> one'),
        'one.two': logging.Formatter('%(message)s -> one.two'),
    },
    logging.Formatter('%(message)s -> <default>'),
))
logging.getLogger().addHandler(handler)

print('Logger used -> formatter used:')
logging.getLogger('one').error('one')
logging.getLogger('one.two').error('one.two')
logging.getLogger('one.two.three').error('one.two.three')  # parent formatter 'one.two' will be used here
logging.getLogger('other').error('other')

# OUTPUT:
# Logger used -> formatter used:
# one -> one
# one.two -> one.two
# one.two.three -> one.two
# other -> <default>

Logging Cookbook, It is true for references to the same object; additionally, application code can setFormatter(formatter) # add the handlers to the logger logger. Let's say you want to log to console and file with different message formats and  If you set the formatter for a handler that was already attached to the logger you want to modify the output of, you are fine, otherwise you have to retrieve a logger object with logging.getLogger()and call its addHandler()method with the instance of your handler class that you set the formatter on as the argument.

This work for you ? Different log level output formats and different log destinations, file vs stdout (and different levels for each destination):

    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)

    format_for_stdout = logging.Formatter('%(message)s')
    format_for_logfile = logging.Formatter('%(asctime)s: %(name)s: %(levelname)s: %(message)s')

    handler_logfile = logging.FileHandler('my_awesome_logs.log')
    handler_logfile.setLevel(logging.DEBUG)
    handler_logfile.setFormatter(format_for_logfile)

    handler_stdout = logging.StreamHandler()
    handler_stdout.setLevel(logging.INFO)
    handler_stdout.setFormatter(format_for_stdout)

    logger.addHandler(handler_logfile)
    logger.addHandler(handler_stdout)

    logging.addLevelName(logging.INFO, logging.getLevelName(logging.INFO))
    logging.addLevelName(logging.ERROR, logging.getLevelName(logging.ERROR))

How to Collect, Customize, and Centralize Python Logs, Logging the traceback in your exception logs can be the exception type ( FileNotFoundError ) and the This automatically captures the same You can use Python's standard traceback library to format the traceback and include it in the log message. def setup_logger(logger_name, log_file, level=logging.INFO): l = logging.getLogger(logger_name) formatter = logging.Formatter('%(message)s') fileHandler = logging.FileHandler(log_file, mode='w') fileHandler.setFormatter(formatter) streamHandler = logging.StreamHandler() streamHandler.setFormatter(formatter) l.setLevel(level) l.addHandler(fileHandler) l.addHandler(streamHandler) setup_logger('log1', txtName+"txt") setup_logger('log2', txtName+"small.txt") logger_1 = logging.getLogger('log1

Logging Cookbook, It is true for references to the same object; additionally, application code can define ERROR) # create formatter and add it to the handlers formatter = logging. Let's say you want to log to console and file with different message formats and  I plan to use the default logging module to print messages to stdout and a log file. To use the logging module, some initialization is required - import logging.config logging.config.fileConfig('logging.conf') logger = logging.getLogger('pyApp') logger.info('testing')

Python Logging: In-Depth Tutorial, The Python logging module comes with the standard library and provides basic For example, when a log “hello world” is sent through a log formatter: for common use cases: https://docs.python.org/3/library/logging.handlers.html#​module- The effective level is the same as logger level if the level is not NOTSET, i.e.,  Creating loggers, handlers, and formatters explicitly using Python code that calls the configuration methods listed above. Creating a logging config file and reading it using the fileConfig() function. Creating a dictionary of configuration information and passing it to the dictConfig() function.

15.6. logging — Logging facility for Python, The logging module supports a hierarchy of loggers with different names. setFormatter() selects a Formatter object for this handler to use. module creates a logger, handler, and formatter nearly identical to those in the example listed above  When logging was added to the Python standard library, the only way of formatting messages with variable content was to use the %-formatting method. Since then, Python has gained two new formatting approaches: string.Template (added in Python 2.4) and str.format () (added in Python 2.6).

Comments
  • Excellent, elegant solution. For the record this also works based on record.levelno, so that the formatters dictionary key could be logging.DEBUG instead of 'base.foo'
  • do you know how to do this to filehandler? Same way?
  • @Henry, yes, just change in second snippet 'StreamHandler()' to 'FileHandler(filename)'. Formatter itself can be applied to any type of handler.