WAFL WAFL - 5 months ago 20
HTML Question

Get HTML code from website in C#

How to get the HTML code from a website, save it, and find some text by a LINQ expression?

I'm using the following code to get the source of a web page:

public static String code(string Url)
{
HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(Url);
myRequest.Method = "GET";
WebResponse myResponse = myRequest.GetResponse();
StreamReader sr = new StreamReader(myResponse.GetResponseStream(), System.Text.Encoding.UTF8);
string result = sr.ReadToEnd();
sr.Close();
myResponse.Close();

return result;
}


How do I find the text in a div in the source of the web page?

Answer

Getting HTML code from a website. You can use code like this.

string urlAddress = "http://google.com";

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

if (response.StatusCode == HttpStatusCode.OK)
{
  Stream receiveStream = response.GetResponseStream();
  StreamReader readStream = null;

  if (response.CharacterSet == null)
  {
     readStream = new StreamReader(receiveStream);
  }
  else
  {
     readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
  }

  string data = readStream.ReadToEnd();

  response.Close();
  readStream.Close();
}

This will give you the returned HTML code from the website. But find text via LINQ is not that easy. Perhaps it is better to use regular expression but that does not play well with HTML code