Is there a way to combine LINQ and async

async linq to sql
using async in linq
async efcore
asynchronous query
linq join
efcore async get
ef core async
c# async inside linq select

Basically I have a procedure like

var results = await Task.WhenAll(
    from input in inputs
    select Task.Run(async () => await InnerMethodAsync(input))
);
.
.
.
private static async Task<Output> InnerMethodAsync(Input input)
{
    var x = await Foo(input);
    var y = await Bar(x);
    var z = await Baz(y);
    return z;
}

and I'm wondering whether there's a fancy way to combine this into a single LINQ query that's like an "async stream" (best way I can describe it).

When you use LINQ, there are generally two parts to it: creation and iteration.

Creation:

var query = list.Select( a => a.Name);

These calls are always synchronous. But this code doesn't do much more than create an object that exposes an IEnumerable. The actual work isn't done till later, due to a pattern called deferred execution.

Iteration:

var results = query.ToList();

This code takes the enumerable and gets the value of each item, which typically will involve the invocation of your callback delegates (in this case, a => a.Name ). This is the part that is potentially expensive, and could benefit from asychronousness, e.g. if your callback is something like async a => await httpClient.GetByteArrayAsync(a).

So it's the iteration part that we're interested in, if we want to make it async.

The issue here is that ToList() (and most of the other methods that force iteration, like Any() or Last()) are not asynchronous methods, so your callback delegate will be invoked synchronously, and you’ll end up with a list of tasks instead of the data you want.

We can get around that with a piece of code like this:

public static class ExtensionMethods
{
    static public async Task<List<T>> ToListAsync<T>(this IEnumerable<Task<T>> This)
    {
        var tasks = This.ToList();     //Force LINQ to iterate and create all the tasks. Tasks always start when created.
        var results = new List<T>();   //Create a list to hold the results (not the tasks)
        foreach (var item in tasks)
        {
            results.Add(await item);   //Await the result for each task and add to results list
        }
        return results;
    }
}

With this extension method, we can rewrite your code:

var results = await inputs.Select( async i => await InnerMethodAsync(i) ).ToListAsync();

^That should give you the async behavior you're looking for, and avoids creating thread pool tasks, as your example does.

Note: If you are using LINQ-to-entities, the expensive part (the data retrieval) isn't exposed to you. For LINQ-to-entities, you'd want to use the ToListAsync() that comes with the EF framework instead.

Try it out and see the timings in my demo on DotNetFiddle.

Query: Async query with Join() and an access to a dependent entity , Is there a way I could re-write the code below to run in async mode by using "​await" or "Task.Run()" etc? I have tried various things back and  The result is a EnumerableMonad<T> that is itself a IEnumerable<T> (i.e. you can further use LINQ on it). Because in this case there is no way to hijack the C# yield operator directly (like F# would) this is instead represented here with the expression await Yield for intermediary values and return for the last one.

Running LINQ to SQL query async?, ToListAsync();. Be aware that this will query the data. Also, you may reconsider your method name. When a method is async the naming  When calling with Task.Run(), both the LINQ query itself and the data bidning seems to work since the GridView has items efter data bind when I debug. It seems to be the actual HTML that does not RENDER with new data items as soon as the method is called in async mode.

Try using Microsoft's Reactive Framework. Then you can do this:

IObservable<Output[]> query =
    from input in inputs.ToObservable()
    from x in Observable.FromAsync(() => Foo(input))
    from y in Observable.FromAsync(() => Bar(x))
    from z in Observable.FromAsync(() => Baz(y))
    select z;

Output[] results = await query.ToArray();

Simple.

Just NuGet "System.Reactive" and add using System.Reactive.Linq; to your code.

[SOLVED], Large systems are complex and hierarchical and one of the best ways to fight accidental complexity is to compose a system from smaller  What About LINQ? Language Integrated Query, or LINQ, provides both a set of helper methods for operating on synchronous enumerables and a set of keywords in the language for writing queries that then compile down to these helper methods. .NET Core 3.0 and C# 8 don’t include either of those for asynchronous enumerables.

Combining iterator blocks and async methods in C#, Entity Framework Core provides a set of async extension methods similar to the LINQ methods, which execute a query and return results. Asynchronous queries avoid blocking a thread while the query is executed in the database. Async queries are important for keeping a responsive UI in thick-client applications. They can also increase throughput in web applications where they free up the thread to service other requests in web applications.

Asynchronous Queries, Similarly, in Entity Framework, the LINQ Join is used to load data from two or The following query combines Authors and Books tables using the Join() method. Unfortunately, LINQ doesn't appear to support asynchronous predicates, so something like this doesn't work: var filteredAddresses = addresses.Where(MeetsCriteria); Is there a similarly convenient way to do this?

Joining, The asynchronous version will always be slower than the synchronous version when there is no concurrency. It's doing all of the same work as the non-async  Blocking asynchronous operations is another anti-pattern that should never be used. But similarly to another anti-pattern mentioned above, you have to use it in practice from time to time to avoid viral asynchrony of the code. There are 3 ways to synchronously wait for the result of a task-based operation: .Result, .Wait and GetAwaiter

Comments
  • @StuartLC I had a type-o
  • Note that ToListAsync is for IQueryable<>, not IEnumerable<>.
  • @xanatos Indeed. Also is not generally applicable because is from EntityFramework.dll
  • Thanks. Based on this feedback I've rewritten the answer.
  • var tasks = This.ToList(); in your solution would be a problem, its same as calling ToList on actual IEnumerable generated via Linq, when ideally you shall enumerate the IEnumerable, foreach (var item in This), no List conversion required
  • @MrinalKamboj I find your feedback frustrating, since you state "would be a problem" without specifying the problem, and offer an alternative which is "ideal" but don't say why. I suggest for technical discussions you avoid judgmental words like those and force yourself to be specific. As it is your comment is unactionable.
  • select z shall result in IObservable type, does that need an explicit await or would it work otherwise too. Does await ensures data streaming.
  • Yes, this gives an IObservable<Output>. I realized a small mistake and have corrected it - I should have written await query.ToArray() rather than await query as the latter only returns the last item from an observable. Using .ToArray() it returns all values when awaited. Awaiting an observable performs a .Subscribe under the hood.