Execute a for loop in parallel using CompletableFuture in Java and log the execution

completablefuture callback
completablefuture android
completablefuture java
completablefuture exceptionally
spring boot completablefuture
completablefuture exception handling
java stream completablefuture
thenapply vs thencompose

I have a for loop which I am trying to parallelize using CompletableFuture.

for (int i = 0; i < 10000; i++) {
    doSomething();
    doSomethingElse();
}

What I have till now is:

for (int i = 0; i < 10000; i++) {
    CompletableFuture.runAsync(() -> doSomething());
    CompletableFuture.runAsync(() -> doSomethingElse());
}

I guess this serves the purpose but there is a requirement to print log just before the start and end of all the processing. If I do this:

log("Started doing things");
for (int i = 0; i < 10000; i++) {
    CompletableFuture.runAsync(() -> doSomething());
    CompletableFuture.runAsync(() -> doSomethingElse());
}
log("Ended doing things");

Does this guarantee that the second log statement will be printed once all the for loop is over since that is executing in a separate thread? If not, is there a way to do this without blocking the main thread?

You have to collect all CompletableFutures and wait for their complete:

log("Started doing things");
List<CompletableFuture> futures = new ArrayList();
for (int i = 0; i < 10000; i++) {
    futures.add(CompletableFuture.runAsync(() -> doSomething()));
    futures.add(CompletableFuture.runAsync(() -> doSomethingElse()));
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]))
                 .thenRunAsync(() -> log("Ended doing things"));

Or when you use the ExecutorService:

CompletableFuture.runAsync(() -> {
    try {
        executorService.invokeAll(tasks);
    } catch (InterruptedException) {
        e.printStackTrace();
    }
    log("Ended doing things");
});

Getting a result from a parallel task in Java using CompletableFuture , We'll simulate a number of long running calculation processes that all return an integer but are independent of each other. We have the following  Some time ago I wrote how elegant and rapid is to make parallel calls in NodeJS with async-await and Promise.all capabilities.Well, it turns out in Java is just as elegant and succinct with the help of CompletableFuture which was introduced in Java 8.

I suppose CompletableFuture is the wrong concept for your needs. If you want to execute an arbitrary number of similar tasks in parallel, the easiest thing is to use the method invokeAll(...) on an ExecutionService:

// First, create a list with all tasks you want to execute in parallel
List<Callable<?>> tasks = new ArrayList<>(10000);
for (int i = 0; i < 10000; ++i) {
    // we need to create Callables, so if your doSomething method returns void, we have to convert it to a Callable using helper method from class Executors
    tasks.add(Executors.callable(this::doSomething));
}

// Then, create an executor service that can execute these tasks
// There are different executors you can choose from, I take one that has a fixed pool of threads
ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

// Last but not least, call invokeAll to execute all tasks and wait for them to complete
executorService.invokeAll(tasks);

// This method will be called when all tasks have been completed successfully:
System.out.println("done");

20 Examples of Using Java's CompletableFuture, Running a Simple Asynchronous Stage. The next example is how to create a stage that executes a Runnable asynchronously:. A very useful feature of asynchronous methods is the ability to provide an Executor to use it to execute the desired CompletableFuture. This example shows how to use a fixed thread pool to apply

The trick here is to use the CompletableFuture.allOf. Try this out.

public static void main(String[] args) {
    System.out.println("Started doing things");
    List<CompletableFuture<Void>> cfList = new ArrayList<>();
    for (int i = 0; i < 10; i++) {
        CompletableFuture<Void> cfOne = CompletableFuture.runAsync(() -> doSomething());
        CompletableFuture<Void> cfTwo = CompletableFuture.runAsync(() -> doSomethingElse());
        cfList.addAll(Arrays.asList(cfOne, cfTwo));
    }
    CompletableFuture<Void> allCfs = CompletableFuture.allOf(cfList.toArray(new CompletableFuture<?>[0]));
    allCfs.thenRunAsync(() -> System.out.println("Ended doing things"));
}

This implementation will NOT keep your main thread waiting till you complete all the submitted tasks.

Fantastic CompletableFuture.allOf() and how to handle errors., allOf() method to achieve the parallel computation of the calculation. For this particular use-case, I need the async task to return the result, runAsync() practice is providing a separate thread pool for our asynchronous tasks to run. log.info("Task execution stopped. The java document of allOf() says. In this video tutorial we will learn how to run Multiple Futures in Parallel in java 8 using a demo project. Below is the GitHub link to download source http

Java CompletableFuture Tutorial with Examples, CompletableFuture is used for asynchronous programming in Java. thread does not block/wait for the completion of the task and it can execute other tasks in parallel. Sometimes you need to execute a long-running computation and when the You can log the exception here and return a default value. When the matrix is small, the sequential version will run faster because of the overhead in setting up the parallel loop. Synchronous calls to shared resources, like the Console or the File System, will significantly degrade the performance of a parallel loop. When measuring performance, try to avoid calls such as Console.WriteLine within the loop.

How to make parallel calls in Java with CompletableFuture example , How to make parallel calls in Java with CompletableFuture example service, given their Ids. Of course you could iterate through the list of Ids and which contains the code you want to execute asynchronously - in this case  Loading…. Typically, for loops aren't independent blocks of execution when you say parallel. The next iteration depends on the incremented value of, most probably, a collection data structure. Parallel execution is possible when you have multiple threads created that can execute on its own independently.

Chapter 11. CompletableFuture: composable asynchronous , The fork/join framework and parallel streams discussed in chapter 7 are valuable tools for The Future interface was introduced in Java 5 to model a result made available at Executing a long-lasting operation asynchronously in a Future. 1 Using CompletableFutures to build an asynchronous application sign in now  The most optimal approach in this case is to chunk the SQL used in the FOR loop. Let's say the SQL is "select * from emp".Create a procedure that processes a chunk of emp.

Comments
  • Why do you want to use CompletableFuture? (It doesn't seem you use these objects outside the loop any more, so there is no need of doing it with CompletableFutures.)
  • In order to ensure the loop has finished executing without blocking the main thread
  • Therefore you don't need a CompletableFuture. There are other mechanisms in Java that are better suited for this purpose.
  • Is futures.foreach(CompletableFuture::join) blocking? I don't want to block the main thread.
  • It is, but you can execute the join operations in an other thread if you want the main Thread to keep running.
  • This is neat. Thanks!
  • Remove the second example using the ExecutorService as it doesn't work this way (invokeAll is a blocking call).
  • If you want to use an explicit ExecutorService, you can specify it as second argument to CompletableFuture.runAsync(...)
  • Is invokeAll() a blocking call? I don't want to block the main thread.
  • Yes it is. But you wrote that you want the second log statement to be called when all tasks have been finished. This is exactly what blocking is.
  • Maybe I could do that in a different thread itself? I was using CompletableFuture because it has thenRun() which does not block the main thread. Which would be the better way?
  • So you don't care about the thread that runs your second log statement?
  • No, I just want it to execute immediately after the for loop without blocking the main thread.