Dovile Dovile - 20 days ago 6
C# Question

Checking if URL exists - HTTP Request always returns an exception

There are numerous posts on how to check if a URL is valid. All of them feature basically the same code, which seems to work for everyone - not for me though and I don't get why.

public static bool ifURLexists(string url)
{

try
{
var request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "HEAD";
//response ALWAYS throws an exception
using (var response = (HttpWebResponse)request.GetResponse())
{
return response.StatusCode == HttpStatusCode.OK;
}
}
catch
{
return false;
}
}


I have tested the method with parameters such as "http://www.nonexistingwebsiteblabla.com" and "http://www.google.com". No matter if I insert an existing or a non existing URL, I get a WebException at this line:

using (var response = (HttpWebResponse)request.GetResponse())


Why could it be not working?

Answer

Check the status WebException.Status This will let you know what specific web exception has occured.

Update: Try change the request.Method = "HEAD"; to GET and try.

Try with a unavailable (404) url, compare the status. Check whether anything is blocking your request.

This is how i manage in my code, i am handling using only ftp specific status.'CommStatus' is an ENUM with error codes which is available in whole application.

catch (WebException ex)
        {
            FtpWebResponse response = (FtpWebResponse)ex.Response;               
          switch(response.StatusCode)
            {
                case FtpStatusCode.ActionNotTakenFileUnavailable:
                    return CommStatus.PathNotFound; 
                case FtpStatusCode.NotLoggedIn:
                    return CommStatus.AuthenticationError;
                default: return CommStatus.UnhandledException;

            }


        }

Below are the available Status of WebException.

CacheEntryNotFound
This API supports the product infrastructure and is not intended to be used directly from your code. The specified cache entry was not found.

ConnectFailure
This API supports the product infrastructure and is not intended to be used directly from your code. The remote service point could not be contacted at the transport level.

ConnectionClosed
This API supports the product infrastructure and is not intended to be used directly from your code. The connection was prematurely closed.

KeepAliveFailure
This API supports the product infrastructure and is not intended to be used directly from your code. The connection for a request that specifies the Keep-alive header was closed unexpectedly.

MessageLengthLimitExceeded
This API supports the product infrastructure and is not intended to be used directly from your code. A message was received that exceeded the specified limit when sending a request or receiving a response from the server.

NameResolutionFailure
This API supports the product infrastructure and is not intended to be used directly from your code. The name resolver service could not resolve the host name.

Pending This API supports the product infrastructure and is not intended to be used directly from your code. An internal asynchronous request is pending.

PipelineFailure This API supports the product infrastructure and is not intended to be used directly from your code. The request was a piplined request and the connection was closed before the response was received.

ProtocolError
This API supports the product infrastructure and is not intended to be used directly from your code. The response received from the server was complete but indicated a protocol-level error. For example, an HTTP protocol error such as 401 Access Denied would use this status.

ProxyNameResolutionFailure
This API supports the product infrastructure and is not intended to be used directly from your code. The name resolver service could not resolve the proxy host name.

ReceiveFailure
This API supports the product infrastructure and is not intended to be used directly from your code. A complete response was not received from the remote server.

RequestCanceled This API supports the product infrastructure and is not intended to be used directly from your code. The request was canceled, the WebRequest.Abort method was called, or an unclassifiable error occurred. This is the default value for Status.

RequestProhibitedByCachePolicy
This API supports the product infrastructure and is not intended to be used directly from your code. The request was not permitted by the cache policy. In general, this occurs when a request is not cacheable and the effective policy prohibits sending the request to the server. You might receive this status if a request method implies the presence of a request body, a request method requires direct interaction with the server, or a request contains a conditional header.

RequestProhibitedByProxy
This API supports the product infrastructure and is not intended to be used directly from your code. This request was not permitted by the proxy.

SecureChannelFailure
This API supports the product infrastructure and is not intended to be used directly from your code. An error occurred while establishing a connection using SSL.

SendFailure This API supports the product infrastructure and is not intended to be used directly from your code. A complete request could not be sent to the remote server.

ServerProtocolViolation This API supports the product infrastructure and is not intended to be used directly from your code. The server response was not a valid HTTP response.

Success This API supports the product infrastructure and is not intended to be used directly from your code. No error was encountered.

Timeout This API supports the product infrastructure and is not intended to be used directly from your code. No response was received during the time-out period for a request.

TrustFailure
This API supports the product infrastructure and is not intended to be used directly from your code. A server certificate could not be validated.

UnknownError
This API supports the product infrastructure and is not intended to be used directly from your code. An exception of unknown type has occurred.

More details here: https://msdn.microsoft.com/en-us/library/system.net.webexceptionstatus(v=vs.110).aspx