Nithin B Nithin B - 3 months ago 15
HTML Question

Correct syntex of XPath function 'substring-after' for html that selects only substring of all nodes?

I need a x path that selects only sub-string of all nodes.
I have be using this x-path but selects all text instead of sub string.

//span[@class="feed-date"]/text()[substring-after(., "on ")]


Html I have: I am willing to extract only date after 'Published on'

<span class="feed-date">Published on 2016-07-07</span>
<span class="feed-date">Published on 2015-02-23</span>
<span class="feed-date">Published on 2014-11-13</span>
<span class="feed-date">Published on 2014-04-28</span>


I found this link that say you can do it in xml

But I cant do it with html. Is there any way to achieve this?

Answer

In XPath 2.0 and later respectively XQuery 1.0 and later or XSLT 2.0 and later you can use //span[@class = 'feed-date']/substring-after(., 'on ') to get a sequence of string values. With XPath 1.0 that functionality does not exist, you would need to iterate all your span elements in a host language and extract the string for each span.

As for using XPath 2.0 with HTMLAgilityPack, it looks as if that is possible making use of https://github.com/StefH/XPath2.Net which is also available on NuGet, that way the Microsoft XPathNavigator gets various extension methods like XPath2Evaluate which then allow you to use XPath 2.0 functions both on an XPathNavigator created from Microsoft's XPathDocument as well as the HTMLAgilityPack's HtmlDocument.

Here is an example:

using System;
using System.Xml.XPath;
using Wmhelp.XPath2;
using HtmlAgilityPack;

namespace XPath20Net1
{
    class Program
    {
        static void Main(string[] args)
        {
            XPathNavigator nav = new XPathDocument("XMLFile1.xml").CreateNavigator();
            Console.WriteLine(nav.XPath2Evaluate("string-join(//span[@class = 'feed-date']/substring-after(., 'on '), ' ')"));

            HtmlDocument doc = new HtmlDocument();
            doc.Load("HTMLPage1.html");

            Console.WriteLine(doc.CreateNavigator().XPath2Evaluate("string-join(//span[@class = 'feed-date']/substring-after(., 'on '), ' ')"));
        }
    }
}

With the the XML document being

<?xml version="1.0" encoding="utf-8" ?>
<html>
  <body>
    <span class="feed-date">Published on 2016-07-07</span>
    <span class="feed-date">Published on 2015-02-23</span>
    <span class="feed-date">Published on 2014-11-13</span>
    <span class="feed-date">Published on 2014-04-28</span>
  </body>
</html>

and the HTML document being

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>Test</title>
</head>
<body>
 <p id=test>

         <span class="feed-date">Published on 2016-07-07</span>
         <span class="feed-date">Published on 2015-02-23</span>
         <span class="feed-date">Published on 2014-11-13</span>
         <span class="feed-date">Published on 2014-04-28</span>

</body>
</html>

then output is

2016-07-07 2015-02-23 2014-11-13 2014-04-28
2016-07-07 2015-02-23 2014-11-13 2014-04-28