H.Fadlallah H.Fadlallah - 1 month ago 11
Vb.net Question

Ignoring tags in XPATH using html agility pack

I am using the following code to parse html tables from an html file into a dataset:

Public Function GetDataSet(html As String) As DataSet
Dim ds As DataSet = New DataSet
Dim htmldoc As New HtmlAgilityPack.HtmlDocument
htmldoc.LoadHtml(html)
Dim tables = htmldoc.DocumentNode.SelectNodes("//table/tr") _
.GroupBy(Function(x) x.ParentNode)
For i As Integer = 0 To tables.Count - 1
Dim rows = tables(i).ToList()
ds.Tables.Add(String.Format("Table {0}", i))
Dim headers = rows(0).Elements("th").Select(Function(x) x.InnerText.Trim).ToList()
For Each Hr In headers
ds.Tables(i).Columns.Add(Hr)
Next
For j As Integer = 1 To rows.Count - 1
Dim row = rows(j)
Dim dr = row.Elements("td").Select(Function(x) x.InnerText.Trim).ToArray()
ds.Tables(i).Rows.Add(dr)
Next
Next
Return ds
End Function


and it works fine. But When There are a Tag placed inside the
<Table>
Tag before
<tr>
tag the table is not parsed

Simple Example:

<html>
<head><title>Test</title></head>
<body>
<div>Contents:</div>
<table>
<tr>
<th>Column1</th> <th>Column2</th>
</tr>
<tr>
<td>1</td> <td>11</td>
</tr>
<tr>
<td>2</td> <td>22</td>
</tr>
</table>
<table>
<tbody>
<tr>
<th>Column1</th> <th>Column2</th> <th>Column3</th>
</tr>
<tr>
<td>a</td> <td>aa</td> <td>aaa</td>
</tr>
<tr>
<td>b</td> <td>bb</td> <td>bbb</td>
</tr>
</tbody>
</table>
<table>
<div>
<tr>
<th>Column1</th> <th>Column2</th> <th>Column3</th>
</tr>
<tr>
<td>a</td> <td>aa</td> <td>aaa</td>
</tr>
<tr>
<td>b</td> <td>bb</td> <td>bbb</td>
</tr>
</div>
</table>
</body>
</html>


In This Example only the first table is parsed.

My question is how to ignore any tag between
<Table>
tag and
<tr>
tag in the following line of code:

Dim tables = htmldoc.DocumentNode.SelectNodes("//table/tr") _
.GroupBy(Function(x) x.ParentNode)


and all the tables will be parsed.

Answer

You can use // to select from all descendants:

Dim rows = htmldoc.DocumentNode.SelectNodes("//table//tr");

Also based on your requirement, it seems it's better to group the result based on the first ancestor table, because the parent of tr may be a tbody or thead and you need to group rows in tables:

Dim tables = htmldoc.DocumentNode.SelectNodes("//table//tr") _
                    .GroupBy(Function(x) x.Ancestors("table").First())
Comments