Executing tasks in parallel

run list of tasks in parallel c#
c# run multiple tasks in parallel
task parallel library vs multithreading
parallel.invoke c#
c# run same method in parallel
c# start multiple tasks simultaneously
c# task
task vs parallel c#

Ok, so basically I have a bunch of tasks (10) and I want to start them all at the same time and wait for them to complete. When completed I want to execute other tasks. I read a bunch of resources about this but I can't get it right for my particular case...

Here is what I currently have (code has been simplified):

public async Task RunTasks()
{
    var tasks = new List<Task>
    {
        new Task(async () => await DoWork()),
        //and so on with the other 9 similar tasks
    }


    Parallel.ForEach(tasks, task =>
    {
        task.Start();
    });

    Task.WhenAll(tasks).ContinueWith(done =>
    {
        //Run the other tasks
    });
}

//This function perform some I/O operations
public async Task DoWork()
{
    var results = await GetDataFromDatabaseAsync();
    foreach (var result in results)
    {
        await ReadFromNetwork(result.Url);
    }
}

So my problem is that when I'm waiting for tasks to complete with the WhenAll call, it tells me that all tasks are over even though none of them are completed. I tried adding Console.WriteLine in my foreach and when I have entered the continuation task, data keeps coming in from my previous Tasks that aren't really finished.

What am I doing wrong here?

You should almost never use the Task constructor directly. In your case that task only fires the actual task that you can't wait for.

You can simply call DoWork and get back a task, store it in a list and wait for all the tasks to complete. Meaning:

tasks.Add(DoWork());
// ...
await Task.WhenAll(tasks);

However, async methods run synchronously until the first await on an uncompleted task is reached. If you worry about that part taking too long then use Task.Run to offload it to another ThreadPool thread and then store that task in the list:

tasks.Add(Task.Run(() => DoWork()));
// ...
await Task.WhenAll(tasks);

Tasks and Parallelism: The New Wave of Multithreading, This example shows how to parallelize operations by using Invoke in the Task Parallel Library. Three operations are performed on a shared  3.3 Running tasks in parallel Running Tasks In Parallel The yield statement is great for representing asynchronous control flow in a linear style, but we also need to do things in parallel.

How to: Use Parallel.Invoke to Execute Parallel Operations , Threading is complicated, because it's non-deterministic when exactly thread runs, and how long they run for. The "problem" is that each thread  Ok, so basically I have a bunch of tasks (10) and I want to start them all at the same time and wait for them to complete. When completed I want to execute other tasks. I read a bunch of resources about this but I can't get it right for my particular case Here is what I currently have (code has b

If you want to run those task's parallel in different threads using TPL you may need something like this:

public async Task RunTasks()
{
    var tasks = new List<Func<Task>>
    {
       DoWork,
       //...
    };

    await Task.WhenAll(tasks.AsParallel().Select(async task => await task()));

    //Run the other tasks
}

These approach parallelizing only small amount of code: the queueing of the method to the thread pool and the return of an uncompleted Task. Also for such small amount of task parallelizing can take more time than just running asynchronously. This could make sense only if your tasks do some longer (synchronous) work before their first await.

For most cases better way will be:

public async Task RunTasks()
{
    await Task.WhenAll(new [] 
    {
        DoWork(),
        //...
    });
    //Run the other tasks
}

To my opinion in your code:

  1. You should not wrap your code in Task before passing to Parallel.ForEach.

  2. You can just await Task.WhenAll instead of using ContinueWith.

Task-based asynchronous programming, This time let's talk a bit about the difference between serial and parallel task execution. The idea is simple: if we have two or more operations  Parallel.For[Each]() will only return when all the parallel tasks are complete, and you can then move onto the other tasks. The code has some other issues too: you mark the method async but don't await in it (the await you do have is in the delegate, not the method);

The DoWork method is an asynchronous I/O method. It means that you don't need multiple threads to execute several of them, as most of the time the method will asynchronously wait for the I/O to complete. One thread is enough to do that.

public async Task RunTasks()
{
    var tasks = new List<Task>
    {
        DoWork(),
        //and so on with the other 9 similar tasks
    };

    await Task.WhenAll(tasks);

    //Run the other tasks            
}

You should almost never use the Task constructor to create a new task. To create an asynchronous I/O task, simply call the async method. To create a task that will be executed on a thread pool thread, use Task.Run. You can read this article for a detailed explanation of Task.Run and other options of creating tasks.

C# - how to run 5 tasks in parallel, Future offers a very elegant way to implement parallel execution of tasks in Java. As I found the concept quite interesting I thought I give it a try  The better approach is to build a process in a semi-parallel way, meaning that a group of Data Flow Tasks (not all of them!) are executed concurrently. Such an approach shortens the time of execution and gives a more balanced usage of CPU and RAM. One possible design of such an approach is shown below.

Just also add a try-catch block around the Task.WhenAll

NB: An instance of System.AggregateException is thrown that acts as a wrapper around one or more exceptions that have occurred. This is important for methods that coordinate multiple tasks like Task.WaitAll() and Task.WaitAny() so the AggregateException is able to wrap all the exceptions within the running tasks that have occurred.

try
{ 
    Task.WaitAll(tasks.ToArray());  
}
catch(AggregateException ex)
{ 
    foreach (Exception inner in ex.InnerExceptions)
    {
    Console.WriteLine(String.Format("Exception type {0} from {1}", inner.GetType(), inner.Source));
    }
}

Serial vs Parallel task execution, Task parallelism allows you to run multiple tasks simultaneously. It is similar to thread pooling, but has some extra benefits like you can return a  In simple words, If you place more than one Task on Control Flow pane and do not connect them by using Precedence constraint, the Tasks will run in Parallel. This can be helpful to speedup the process when we load data from Source Database to Staging Database and there is no dependency which table should be loaded first.

Executing tasks in parallel using Java Future, 1) The library you are trying to use clearly does not fit your requirements. So why force it? Aren't you essentially just calling: var tasks  The Task Parallel Library (TPL) is based on the concept of a task, which represents an asynchronous operation. In some ways, a task resembles a thread or ThreadPool work item, but at a higher level of abstraction. The term task parallelism refers to one or more independent tasks running concurrently.

Task Parallelism in C#, You can create a task or parallel thread of execution using Task class. It provides a callback, that we can use to define a subsequent task to run asynchronously. Concurrent execution is the generalized form of parallel execution. For example parallel program can also be called concurrent but reverse is not true. Concurrent execution is possible on single processor (multiple threads, managed by scheduler) Parallel execution is not possible on single processor but on multiple processors.

Running parallel tasks until they're done, To execute chunks in parallel, the user must have CREATE JOB system privilege. The CHUNK_BY_SQL, RUN_TASK, and RESUME_TASK subprograms require a query, and are executed using DBMS_SQL. Invokers of the DBMS_SQL interface must ensure that none of the queries passed-in contains SQL injection.

Comments
  • I suspect in all this you have a Task<Task> somewhere (calling the Task constructor directly is somewhat suspect) and you really want to await the inner task. Can you use Task.Run instead of the Task constructor/Start? Task.Run will do the unwrapping for you.
  • So you think I should do: var tasks = new List<Task> { Task.Run(()=> DoWork()) }
  • By run synchronously until the first await did you want to mean they would actually run async only when we called Task.WhenAll? Thanks!
  • @Alisson I was more referring to the synchronous part of DoWork and not the code calling it.
  • @i3arnon, as you said, However, async methods run synchronously until the first await on an uncompleted task is reached. ---- SO here can I use "await task.delay(5).ConfigureAwait(false); statement as a first line in my DoWork() method? to return the main thread and start the Threadpool Thread for to execute remianing code in the method, ? Please advise.
  • @SaddamBinSyed you don't need Task.Delay. You have Task.Yield exactly for that purpose. But as I see it using Task.Run is better in that case because it's more explicit about "firing up" a new task.
  • An inspiring code snippet, perhaps the best on the net which combines task parallel with async using PLINQ.