Andrey Andrey - 14 days ago 6
C# Question

Parsing new "barchart.com" site

I'm looking for a way to parse data (one time per day) from barchart.com (for example, this link https://www.barchart.com/futures/quotes/HG*0/all-futures#/viewName=main).

The problem is, the page code does not contain the values I want to get (and I cannot find the js script which loads it on the page). I mean the table with close/open/high prices.

I guess the easiest way is to download the excel file from the page using the "download" button on the right top corner of the table (and then work with it). But I cannot find the download link too, cause the click event is also processed in some script.

I'm using C#, but not asking for a code. Any ideas and hints will be really usefull. Thanks.

Answer

If you check the network tab in the web inspector then you will find the URL used for loading the data. Simply call that directly and do as you wish. Here's a jQuery example (because I can demonstrate the concept using the snippet tool), but calling it from C# is trivial using the WebClient.DownloadString() method.

var url = 'https://core-api.barchart.com/v1/quotes/get?fields=symbol%2CcontractName%2ClastPrice%2CpriceChange%2CopenPrice%2ChighPrice%2ClowPrice%2CpreviousPrice%2Cvolume%2CopenInterest%2CtradeTime%2CsymbolCode%2CsymbolType%2ChasOptions&list=futures.contractInRoot&root=HG&meta=field.shortName%2Cfield.description&hasOptions=true&raw=1';

$('#data').load(url);
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

Data: <br />
<textarea id="data" cols="50" rows="8"></textarea>

C# example:

public static void DownloadString (string address)
{
    WebClient client = new WebClient ();
    string reply = client.DownloadString (address);

    Console.WriteLine (reply);
    return reply;
}

var url = 'https://core-api.barchart.com/v1/quotes/get?fields=symbol%2CcontractName%2ClastPrice%2CpriceChange%2CopenPrice%2ChighPrice%2ClowPrice%2CpreviousPrice%2Cvolume%2CopenInterest%2CtradeTime%2CsymbolCode%2CsymbolType%2ChasOptions&list=futures.contractInRoot&root=HG&meta=field.shortName%2Cfield.description&hasOptions=true&raw=1';

var data = DownloadString(url);
Comments