David L-R David L-R - 3 months ago 7
Ruby Question

Nokogiri get all HTML nodes

I want to get all nodes from a HTML document using Nokogiri.
Example HTML input string:

<html>
<body>
<h1>Test</h1>
<p>test <strong> Jojo </strong></p>
</body>
</html>


Expected output:

['<html>','<body>','<h1>','</h1>','<p>','<strong>','</strong>','</p>','</body>','</html>']


Closing tags and the correct order is important!

I tried this code already:

require 'nokogiri'
string_page = "<html><body><h1>Header1</h1></body></html>"
doc = Nokogiri::HTML(string_page)
doc.search('*').map(&:name)
# => ["html", "body", "h1"]


But it doesn't return closing tags.

Answer

You could split the OuterXml over InnerXml of all opening elements that are not self closing, store the corresponding closing elements if any to retrieve it and parse the document using the Nokogiri reader to build the list according to the order within the document.

It requires that your document is a valid XML fragment as it is using the XML parser and not the HTML one.

require 'nokogiri'
[ "<html><body><h1>Header1</h1></body></html>",
"<html><body><div><h1>Title</h1><hr /></div><div><p>Lorem Ipsum<br />sit <span class=\"style\">d</span>olor</p></div></body></html>", <<END
<html>
  <body>
      <h1>Test</h1>
      <p>test <strong> Jojo </strong></p>
  </body>
</html>
END
].each { |string_page|
  elem_all = Array.new
  elem_ends = Hash.new
  reader = Nokogiri::XML::Reader(string_page)
  reader.each { |node|
    if node.node_type.eql?(1)
      if node.self_closing?
        elem_all << node.outer_xml
      else
        elem_tags = node.outer_xml.split(node.inner_xml)
        elem_all << elem_tags.first
        elem_ends[node.local_name] = elem_tags[1] unless elem_tags.one?
      end
    end
    elem_all << elem_ends[node.local_name] if node.node_type.eql?(15) and elem_ends.has_key?(node.local_name)
  }

  puts string_page
  puts elem_all.to_s
  puts
}

Outputs:

<html><body><h1>Header1</h1></body></html>
["<html>", "<body>", "<h1>", "</h1>", "</body>", "</html>"]

<html><body><div><h1>Title</h1><hr /></div><div><p>Lorem Ipsum<br />sit <span class="style">d</span>olor</p></div></body></html>
["<html>", "<body>", "<div>", "<h1>", "</h1>", "<hr/>", "</div>", "<div>", "<p>", "<br/>", "<span class=\"style\">", "</span>", "</p>", "</div>", "</body>", "</html>"]

<html>
  <body>
      <h1>Test</h1>
      <p>test <strong> Jojo </strong></p>
  </body>
</html>
["<html>", "<body>", "<h1>", "</h1>", "<p>", "<strong>", "</strong>", "</p>", "</body>", "</html>"]
Comments