anothershrubery anothershrubery - 4 years ago 69
C# Question

Screen scraping web page containing button with AJAX

I am trying to automate some of our processes, one includes logging in to an external web page, clicking a link to expand details, then grab all details displayed.

I have got the process logging in, and can grab all of the details once they are expanded.

The problem is with clicking the link. The link is defined like below (I have removed what the

Submit
method actually does as the code is long and probably irrelevant. Obviously the
img
is placeholder just as an example):

<a id="form:SummarySubView:closedToggleControl" onclick="A4J.AJAX.Submit(...); return false;" href="#">
<img ... />
</a>


I use this data as below:

void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
WebBrowser browser = (WebBrowser)sender;

HtmlElement expandDetails = browser.Document.GetElementById("form:SummarySubView:closedToggleControl");
//When open ID for element is "form:SummarySubView:openToggleControl"

if(expandDetails == null) //If already expanded
{
//Stuff
}
else
{
expandDetails.InvokeMember("click"); //Click on element to run AJAX
}
}


Upon running
expandDetails.InvokeMember("click");
browser_DocumentCompleted
gets called again as expected but the document is same as before and
expandDetails
is found again with the "closed" id. This means that the details I am looking for are never shown.

How do I get access to the document AFTER the AJAX script runs correctly?

Adding a
Timer
to delay checking the document doesn't seem to have worked.

Answer Source

So a really simple solution seems to have worked. My code now looks like:

void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
    WebBrowser browser = (WebBrowser)sender;

    HtmlElement expandDetails = browser.Document.GetElementById("form:SummarySubView:closedToggleControl");
    //When open ID for element is "form:SummarySubView:openToggleControl"

    if(expandDetails == null) //If already expanded
    {
        //Stuff
    }
    else
    {
        expandDetails.InvokeMember("click"); //Click on element to run AJAX

        while (expandDetails != null)
        {
            expandDetails = browser.Document.GetElementById("form:SummarySubView0:closedToggleControl");

            Application.DoEvents();
            System.Threading.Thread.Sleep(200);
        }

        //Stuff
    }
}

So running the while loop works fine for my case.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download