nurdyguy nurdyguy - 1 year ago 408
C# Question

.net Core Parallel.ForEach issues

I've switched over to .net Core for some projects and am now having a problem with Parallel.ForEach. In the past I often had a List of id values which I then would use to make web requests in order to get the full data. It would look something like this:

Parallel.ForEach(myList, l =>
// make web request using
// process the data somehow

Well, in .net Core the web requests must all be tagged
which means the Parallel.ForEach action must be tagged with
. But, tagging a Parallel.ForEach action as
means we have a
void async
method which causes issues. In my particular case that means the response returns to the application before all of the web requests in the Parallel loop are completed which is both awkward and causes errors.

Question: What are the alternatives to using Parallel.ForEach here?

One possible solution I found was to wrap the Parallel loop inside of a Task and await the task:

await Task.Run(() => Parallel.ForEach(myList, l =>
// stuff here

(found here:Parallel.ForEach vs Task.Run and Task.WhenAll)

But, that isn't working for me. When I use that I still end up returning to the application before the loop is completed.

Another option:

var tasks = new List<Task>();
foreach (var l in myList)
tasks.Add(Task.Run(async () =>
// stuff here
await Task.WhenAll(tasks);

This appears to work, but is that the only option? It seems that the new .net Core has rendered Parallel.ForEach virtually useless (at least when it comes to nested web calls).

Any assistance/advice is appreciated.

Answer Source

Neither of those 3 apporaches are good.

You should not use the Parallel class, or Task.Run on this scenario.

Instead, have an async handler method:

private async Task HandleResponse(Task<HttpResponseMessage> gettingResponse)
     HttpResponseMessage response = await gettingResponse;
     // Process the data

And then use Task.WhenAll:

Task[] requests = myList.Select(l => SendWebRequest(l.Id))
                        .Select(r => HandleResponse(r))

await Task.WhenAll(requests);
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download