Mahdi Mahdi - 10 days ago 6
HTTP Question

How to get response from an endpoint which sometimes throws timeout exception

There is a third party API which is supposed to return a large number of text messages. I do not know how many messages it has for a specific query so I have to call it in a loop using take and skip parameters like follows:

while (result.messages.Count == postCount)
{
using (var handler = new HttpClientHandler { CookieContainer = cookieContainer })
{
using (var client = new HttpClient(handler) { BaseAddress = url, Timeout = new TimeSpan(0, 2, 0) })
{
cookieContainer.Add(domain, new Cookie(cookie.Name, cookie.Value));
take = 500;
postCount += 500;
query["Skip"] = "0";
query["Take"] = "0";
builder.Query = query.ToString();
url = builder.Uri;
client.BaseAddress = url;
var tempStringResult = client.GetStringAsync(url).Result;
skip += 500;
}
}
}


The issue is that the endpoint typically throws timeout exception. So I decided to call it offline (in a worker) and cache the results to be used be the web server.
Now the question is how can I retry calling it when it throws error? Normally, if I call it after some time it may work.

Answer

When you receive a timeout response code, you can reduce the number of messages to take and retry with the same skip index.

Something as:

var take        = 500;
var skip        = 0;
var postCount   = 0;

while(result.messages.Count == postCount)
{
    using(var handler = new HttpClientHandler { CookieContainer = cookieContainer })
    {
        using(var client = new HttpClient(handler) { BaseAddress = url, Timeout = new TimeSpan(0, 2, 0) })
        {
            cookieContainer.Add(domain, new Cookie(cookie.Name, cookie.Value));

            query["Skip"]        = skip;
            query["Take"]        = take;
            builder.Query        = query.ToString();
            url                  = builder.Uri;
            client.BaseAddress   = url;

            var response    = await client.GetAsync(url);

            if(response.IsSuccessStatusCode)
            {
                // succeeded
                skip        += take;
                postCount   += take;
            }
            else if(response.StatusCode == System.Net.HttpStatusCode.RequestTimeout)
            {
                // timeout, we reduce the number of messages to take. 
                take        -= 100;
            }
        }
    }
}