How to retain slf4j MDC logging context in CompletableFuture?

completablefuture logging
sl4j mdc context
logging system offers mdc functionality
logback-access mdc
mdc getcopyofcontextmap
mdc setcontextmap nullpointerexception
context aware completablefuture
logback output mdc

When executing async CompletableFuture, the parent threadcontext and moreover the org.slf4j.MDC context is lost.

This is bad as I'm using some kind of "fish tagging" to track logs from one request among multiple logfiles.

MDC.put("fishid", randomId())

Question: how can I retain that id during the tasks of CompletableFutures in general?

List<CompletableFuture<UpdateHotelAllotmentsRsp>> futures =
    tasks.stream()
        .map(task -> CompletableFuture.supplyAsync(
            () -> businesslogic(task))
        .collect(Collectors.toList());

List results = futures.stream()
    .map(CompletableFuture::join)
    .collect(Collectors.toList());

public void businesslogic(Task task) {
       LOGGER.info("mdc fishtag context is lost here");
}

The most readable way I solved this problem was as below -

---------------Thread utils class--------------------

public static Runnable withMdc(Runnable runnable) {
    Map<String, String> mdc = MDC.getCopyOfContextMap();
    return () -> {
        MDC.setContextMap(mdc);
        runnable.run();
    };
}

public static <U> Supplier<U> withMdc(Supplier<U> supplier) {
    Map<String, String> mdc = MDC.getCopyOfContextMap();
    return (Supplier) () -> {
        MDC.setContextMap(mdc);
        return supplier.get();
    };
}

---------------Usage--------------

CompletableFuture.supplyAsync(withMdc(() -> someSupplier()))
                 .thenRunAsync(withMdc(() -> someRunnable())
                 ....

WithMdc in ThreadUtils would have to be overloaded to include other functional interfaces which are accepted by CompletableFuture

Please note that the withMdc() method is statically imported to improve readability.

Logging in a multithreaded environment and with , Retaining MDC context with CompletableFuture construct. We would use learnings from the previous step to achieve it. Retaining MDC context� Contributing MDC-aware Executors to slf4j. Hi, When working with asynchronous programming models ( Netty, CompletableFuture, OSGi Promises ) it becomes problematic to propagate the MDC between the

At the end I created a Supplier wrapper retaining the MDC. If anyone has a better idea feel free to comment.

public static <U> CompletableFuture<U> supplyAsync(Supplier<U> supplier, Executor executor) {
    return CompletableFuture.supplyAsync(new SupplierMDC(supplier), executor);
}

private static class SupplierMDC<T> implements Supplier<T> {
    private final Supplier<T> delegate;
    private final Map<String, String> mdc;

    public SupplierMDC(Supplier<T> delegate) {
        this.delegate = delegate;
        this.mdc = MDC.getCopyOfContextMap();
    }

    @Override
    public T get() {
        MDC.setContextMap(mdc);
        return delegate.get();
    }
}

MDC and Threadpools – Ashton Kemerling, And while many systems will keep logs in their reverse proxy like Nginx, a well I have found, the standard way to resolve this is to use the Mapped Diagnostic Context, or MDC. LoggerFactory; import org.slf4j.MDC; import java.util. concurrent.CompletableFuture; public class Main { public static final String� Each of them calls a function func() where the attributes are logged using MDC. But when I later retrieve the MDC to log these attributes in a buffer appender, only service1's attributes are available in the MDC, service2's MDC context returns null. I am not clearing the MDC anywhere in my code. What could be the reason.

YES, Twitter Future did this correctly. They have a class Local.scala that Future.scala knows about.

The fix is for java authors to fix this issue so your Local state travels through ALL libaries that use CompletableFutures. Basically, Local.scala is used by Future and internally uses a ThreadLocal up until .thenApply or .thenAccept and it will capture state and transfer it when needed to the next one on and on. This works through all third party libraries with ZERO 3rd party library changes.

Here is more but poke Java Authors to fix their stuff... http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-May/047867.html

until then, MDC will NEVER work through 3rd party libraries.

My SO post on this Does CompletableFuture have a corresponding Local context?

MDC Logger with CompletableFuture, I am using MDC Logger, which is perfectly working for me except in one case. Wherever Below is my logback.xml configuration, where sessionID is MDC data task) { //save the current MDC context Map<String, String> contextMap = MDC. 7 How to retain slf4j MDC logging context in CompletableFuture? Jun 1 '19. 5 Jython Remote debugging Intellij Idea, using Pycharm plugin Apr 5 '13.

Code Breeze !: Using log MDC in multi-thread, concurrent instead of the old style Thread way to show how to keep MDC context inheritance. 0. What you need. JDK 1.7; Maven 3.2.1; SLF4J 1.7. /** * A SLF4J MDC-compatible {@link ThreadPoolExecutor}. * <p/> * In general, MDC is used to store diagnostic information (e.g. a user's session id) in per-thread variables, to facilitate * logging. However, although MDC data is passed to thread children, this doesn't work when threads are reused in a * thread pool.

How to use MDC with ForkJoinPool?, Completablefuture mdc. How to retain slf4j MDC logging context in CompletableFuture , The most readable way I solved this problem was as below - . Stack Overflow Public questions and answers; Teams Private questions and answers for your team; Enterprise Private self-hosted questions and answers for your enterprise; Jobs Programming and related technical career opportunities

How to use MDC with ForkJoinPool?, ForkJoinTask; import java.util.concurrent.atomic.AtomicReference; import org. slf4j.MDC; /** * A {@link ForkJoinPool} that inherits MDC contexts from the thread � Bean Integration. Camel supports the integration of beans and POJOs in a number of ways. Annotations. If a bean is defined in Spring XML or scanned using the Spring component scanning mechanism and a <camelContext> is used or a CamelBeanPostProcessor then we process a number of Camel annotations to do various things such as injecting resources or producing, consuming or routing messages.

Logging • Akka Documentation, SLF4J is used for logging and Akka provides access to an org.slf4j. getLogger( getClass()); void run() { CompletableFuture<String> task = CompletableFuture. MDC allows for adding additional context dependent attributes to log entries. if the logging backend can't keep up with the throughput of produced log events. This is a boon for non-blocking applications and continuation-style APIs (as popularized by CompletableFuture and ReactiveX) that allow declarative composition of asynchronous logic. At the programming-model level, Java 8 enabled Spring WebFlux to offer functional web endpoints alongside annotated controllers.

Comments
  • Wouldn't the solution from How to use MDC with thread pools? work? Only restriction is that you need to pass the executor in all *Async() calls. If you use Spring @Async you just have to configure that executor for @Async.
  • Also closely related, but didn't get an answer: Does CompletableFuture have a corresponding Local context?
  • any solution may work internally with JUST your code. It won't work through libraries that then use futures as well. Java will have more and more libraries using future and MDC won't be transferred into it correctly until Java fixes this.
  • Thanks for your improvement. This is even better as my original answer, so I will accept yours instead. It's also more consistent even if only using one single .supplyAsync() statement.
  • trying it, looks good.I was thinking similar, got the code handy :)
  • Can you please share the code , how you have used it
  • Please note : Usually CompletableFuture is used as a fluid API, so if you chain another supplier with the static supplyAsync method above, it will be a normal supplier and not a SupplierMDC. Thus the MDC context will be lost for subsequent asynch processing
  • @Bhushan that's true. Feel free to add your suggestions how that could be solved.
  • @Membersound Thanks to your original answer, I refactored it a little bit and added some code-sugar so that it is compatible with the fluidity of CompletableFuture stackoverflow.com/a/56403808/1594534
  • The other option being to have EVERY 3rd party library that uses CompletableFuture to switch to a new kind of Future that we all do. heh, good luck with that one though!