nurdyguy nurdyguy - 2 months ago 72
C# Question

.net Core Parallel.ForEach issues

I've switched over to .net Core for some projects and am now having a problem with Parallel.ForEach. In the past I often had a List of id values which I then would use to make web requests in order to get the full data. It would look something like this:

Parallel.ForEach(myList, l =>
{
// make web request using l.id
// process the data somehow
});


Well, in .net Core the web requests must all be tagged
await
which means the Parallel.ForEach action must be tagged with
async
. But, tagging a Parallel.ForEach action as
async
means we have a
void async
method which causes issues. In my particular case that means the response returns to the application before all of the web requests in the Parallel loop are completed which is both awkward and causes errors.

Question: What are the alternatives to using Parallel.ForEach here?

One possible solution I found was to wrap the Parallel loop inside of a Task and await the task:

await Task.Run(() => Parallel.ForEach(myList, l =>
{
// stuff here
}));


(found here:Parallel.ForEach vs Task.Run and Task.WhenAll)

But, that isn't working for me. When I use that I still end up returning to the application before the loop is completed.

Another option:

var tasks = new List<Task>();
foreach (var l in myList)
{
tasks.Add(Task.Run(async () =>
{
// stuff here
}));
}
await Task.WhenAll(tasks);


This appears to work, but is that the only option? It seems that the new .net Core has rendered Parallel.ForEach virtually useless (at least when it comes to nested web calls).

Any assistance/advice is appreciated.

Answer

Neither of those 3 apporaches are good.

You should not use the Parallel class, or Task.Run on this scenario.

Instead, have an async handler method:

private async Task HandleResponse(Task<HttpResponseMessage> gettingResponse)
{
     HttpResponseMessage response = await gettingResponse;
     // Process the data
}

And then use Task.WhenAll:

Task[] requests = myList.Select(l => SendWebRequest(l.Id))
                        .Select(r => HandleResponse(r))
                        .ToArray();

await Task.WhenAll(requests);