Godzilla74 Godzilla74 - 6 months ago 11
Ruby Question

How to do a Nokogiri search based on an XML attribute

I'm trying to search through an XML file and display results based on whether an attribute exists in an nmap file. I have the basic parsing working (or so I think), however my Ruby script is obviously incorrect since it's yielding the same results for every host in my file, even if the given XML field doesn't exist for a given host.

The XML file is:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/local/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 7.12 scan initiated Mon May 23 15:53:44 2016 as: nmap -&#45;script /Users/me/Coding/nse_scripts/ssl-poodle.nse -oX poodle_results.txt 10.10.10.0/24 -->
<nmaprun scanner="nmap" args="nmap -&#45;script /Users/me/Coding/nse_scripts/ssl-poodle.nse -oX poodle_results.txt 10.10.10.0/24" start="1464033224" startstr="Mon May 23 15:53:44 2016" version="7.12" xmloutputversion="1.04">
<scaninfo type="connect" protocol="tcp" numservices="1000" services="1,3-4,6-7,9,13,17,19-26,30,32-33,37,42-43,49,53,70,79-85,88-90,99-100,106,109-111,113,119,125,135,139,143-144,146,161,163,179,199,211-212,222,254-256,259,264,280,301,306,311,340,366,389,406-407,416-417,425,427,443-445,458,464-465,481,497,500,512-515,524,541,543-545,548,554-555,563,587,593,616-617,625,631,636,646,648,666-668,683,687,691,700,705,711,714,720,722,726,749,765,777,783,787,800-801,808,843,873,880,888,898,900-903,911-912,981,987,990,992-993,995,999-1002,1007,1009-1011,1021-1100,1102,1104-1108,1110-1114,1117,1119,1121-1124,1126,1130-1132,1137-1138,1141,1145,1147-1149,1151-1152,1154,1163-1166,1169,1174-1175,1183,1185-1187,1192,1198-1199,1201,1213,1216-1218,1233-1234,1236,1244,1247-1248,1259,1271-1272,1277,1287,1296,1300-1301,1309-1311,1322,1328,1334,1352,1417,1433-1434,1443,1455,1461,1494,1500-1501,1503,1521,1524,1533,1556,1580,1583,1594,1600,1641,1658,1666,1687-1688,1700,1717-1721,1723,1755,1761,1782-1783,1801,1805,1812,1839-1840,1862-1864,1875,1900,1914,1935,1947,1971-1972,1974,1984,1998-2010,2013,2020-2022,2030,2033-2035,2038,2040-2043,2045-2049,2065,2068,2099-2100,2103,2105-2107,2111,2119,2121,2126,2135,2144,2160-2161,2170,2179,2190-2191,2196,2200,2222,2251,2260,2288,2301,2323,2366,2381-2383,2393-2394,2399,2401,2492,2500,2522,2525,2557,2601-2602,2604-2605,2607-2608,2638,2701-2702,2710,2717-2718,2725,2800,2809,2811,2869,2875,2909-2910,2920,2967-2968,2998,3000-3001,3003,3005-3007,3011,3013,3017,3030-3031,3052,3071,3077,3128,3168,3211,3221,3260-3261,3268-3269,3283,3300-3301,3306,3322-3325,3333,3351,3367,3369-3372,3389-3390,3404,3476,3493,3517,3527,3546,3551,3580,3659,3689-3690,3703,3737,3766,3784,3800-3801,3809,3814,3826-3828,3851,3869,3871,3878,3880,3889,3905,3914,3918,3920,3945,3971,3986,3995,3998,4000-4006,4045,4111,4125-4126,4129,4224,4242,4279,4321,4343,4443-4446,4449,4550,4567,4662,4848,4899-4900,4998,5000-5004,5009,5030,5033,5050-5051,5054,5060-5061,5080,5087,5100-5102,5120,5190,5200,5214,5221-5222,5225-5226,5269,5280,5298,5357,5405,5414,5431-5432,5440,5500,5510,5544,5550,5555,5560,5566,5631,5633,5666,5678-5679,5718,5730,5800-5802,5810-5811,5815,5822,5825,5850,5859,5862,5877,5900-5904,5906-5907,5910-5911,5915,5922,5925,5950,5952,5959-5963,5987-5989,5998-6007,6009,6025,6059,6100-6101,6106,6112,6123,6129,6156,6346,6389,6502,6510,6543,6547,6565-6567,6580,6646,6666-6669,6689,6692,6699,6779,6788-6789,6792,6839,6881,6901,6969,7000-7002,7004,7007,7019,7025,7070,7100,7103,7106,7200-7201,7402,7435,7443,7496,7512,7625,7627,7676,7741,7777-7778,7800,7911,7920-7921,7937-7938,7999-8002,8007-8011,8021-8022,8031,8042,8045,8080-8090,8093,8099-8100,8180-8181,8192-8194,8200,8222,8254,8290-8292,8300,8333,8383,8400,8402,8443,8500,8600,8649,8651-8652,8654,8701,8800,8873,8888,8899,8994,9000-9003,9009-9011,9040,9050,9071,9080-9081,9090-9091,9099-9103,9110-9111,9200,9207,9220,9290,9415,9418,9485,9500,9502-9503,9535,9575,9593-9595,9618,9666,9876-9878,9898,9900,9917,9929,9943-9944,9968,9998-10004,10009-10010,10012,10024-10025,10082,10180,10215,10243,10566,10616-10617,10621,10626,10628-10629,10778,11110-11111,11967,12000,12174,12265,12345,13456,13722,13782-13783,14000,14238,14441-14442,15000,15002-15004,15660,15742,16000-16001,16012,16016,16018,16080,16113,16992-16993,17877,17988,18040,18101,18988,19101,19283,19315,19350,19780,19801,19842,20000,20005,20031,20221-20222,20828,21571,22939,23502,24444,24800,25734-25735,26214,27000,27352-27353,27355-27356,27715,28201,30000,30718,30951,31038,31337,32768-32785,33354,33899,34571-34573,35500,38292,40193,40911,41511,42510,44176,44442-44443,44501,45100,48080,49152-49161,49163,49165,49167,49175-49176,49400,49999-50003,50006,50300,50389,50500,50636,50800,51103,51493,52673,52822,52848,52869,54045,54328,55055-55056,55555,55600,56737-56738,57294,57797,58080,60020,60443,61532,61900,62078,63331,64623,64680,65000,65129,65389"/>
<verbose level="0"/>
<debugging level="0"/>
<host starttime="1464033224" endtime="1464034228"><status state="up" reason="syn-ack" reason_ttl="0"/>
<address addr="10.10.10.1" addrtype="ipv4"/>
<hostnames>
<hostname name="DD-WRT" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="996">
<extrareasons reason="conn-refused" count="996"/>
</extraports>
<port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" method="table" conf="3"/></port>
<port protocol="tcp" portid="53"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="domain" method="table" conf="3"/></port>
<port protocol="tcp" portid="80"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" method="table" conf="3"/></port>
<port protocol="tcp" portid="443"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="https" method="table" conf="3"/><script id="ssl-poodle" output="&#xa; VULNERABLE:&#xa; SSL POODLE information leak&#xa; State: VULNERABLE&#xa; IDs: OSVDB:113251 CVE:CVE-2014-3566&#xa; The SSL protocol 3.0, as used in OpenSSL through 1.0.1i and&#xa; other products, uses nondeterministic CBC padding, which makes it easier&#xa; for man-in-the-middle attackers to obtain cleartext data via a&#xa; padding-oracle attack, aka the &quot;POODLE&quot; issue.&#xa; Disclosure date: 2014-10-14&#xa; Check results:&#xa; TLS_RSA_WITH_3DES_EDE_CBC_SHA&#xa; References:&#xa; https://www.imperialviolet.org/2014/10/14/poodle.html&#xa; http://osvdb.org/113251&#xa; https://www.openssl.org/~bodo/ssl-poodle.pdf&#xa; https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3566&#xa;"><table key="CVE-2014-3566">
<elem key="title">SSL POODLE information leak</elem>
<elem key="state">VULNERABLE</elem>
<table key="ids">
<elem>OSVDB:113251</elem>
<elem>CVE:CVE-2014-3566</elem>
</table>
<table key="description">
<elem> The SSL protocol 3.0, as used in OpenSSL through 1.0.1i and&#xa; other products, uses nondeterministic CBC padding, which makes it easier&#xa; for man-in-the-middle attackers to obtain cleartext data via a&#xa; padding-oracle attack, aka the &quot;POODLE&quot; issue.</elem>
</table>
<table key="dates">
<table key="disclosure">
<elem key="day">14</elem>
<elem key="month">10</elem>
<elem key="year">2014</elem>
</table>
</table>
<elem key="disclosure">2014-10-14</elem>
<table key="check_results">
<elem>TLS_RSA_WITH_3DES_EDE_CBC_SHA</elem>
</table>
<table key="refs">
<elem>https://www.imperialviolet.org/2014/10/14/poodle.html</elem>
<elem>http://osvdb.org/113251</elem>
<elem>https://www.openssl.org/~bodo/ssl-poodle.pdf</elem>
<elem>https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3566</elem>
</table>
</table>
</script></port>
</ports>
<times srtt="17618" rttvar="3666" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.100" addrtype="ipv4"/>
<hostnames>
</hostnames>
<ports><extraports state="closed" count="1000">
<extrareasons reason="conn-refused" count="1000"/>
</extraports>
</ports>
<times srtt="15580" rttvar="11919" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.101" addrtype="ipv4"/>
<hostnames>
<hostname name="office" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="999">
<extrareasons reason="conn-refused" count="999"/>
</extraports>
<port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" method="table" conf="3"/></port>
</ports>
<times srtt="12241" rttvar="4711" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.104" addrtype="ipv4"/>
<hostnames>
<hostname name="main" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="999">
<extrareasons reason="conn-refused" count="999"/>
</extraports>
<port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" method="table" conf="3"/></port>
</ports>
<times srtt="14409" rttvar="1988" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.115" addrtype="ipv4"/>
<hostnames>
<hostname name="c8d10f5c386d4e4bb01d7aa0edac7a24" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="998">
<extrareasons reason="conn-refused" count="998"/>
</extraports>
<port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" method="table" conf="3"/></port>
<port protocol="tcp" portid="10000"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="snet-sensor-mgmt" method="table" conf="3"/></port>
</ports>
<times srtt="4009" rttvar="2037" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="syn-ack" reason_ttl="0"/>
<address addr="10.10.10.121" addrtype="ipv4"/>
<hostnames>
<hostname name="OpenELEC" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="995">
<extrareasons reason="conn-refused" count="995"/>
</extraports>
<port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" method="table" conf="3"/></port>
<port protocol="tcp" portid="80"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" method="table" conf="3"/></port>
<port protocol="tcp" portid="111"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="rpcbind" method="table" conf="3"/></port>
<port protocol="tcp" portid="445"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="microsoft-ds" method="table" conf="3"/></port>
<port protocol="tcp" portid="1049"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="td-postman" method="table" conf="3"/></port>
</ports>
<times srtt="11099" rttvar="4538" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464033238"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.130" addrtype="ipv4"/>
<hostnames>
<hostname name="Justins-iMac" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="1000">
<extrareasons reason="conn-refused" count="1000"/>
</extraports>
</ports>
<times srtt="455" rttvar="155" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034248"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.135" addrtype="ipv4"/>
<hostnames>
<hostname name="HarmonyHub" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="997">
<extrareasons reason="conn-refused" count="997"/>
</extraports>
<port protocol="tcp" portid="5222"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="xmpp-client" method="table" conf="3"/></port>
<port protocol="tcp" portid="8088"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="radan-http" method="table" conf="3"/></port>
<port protocol="tcp" portid="8222"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="unknown" method="table" conf="3"/></port>
</ports>
<times srtt="28872" rttvar="2401" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="syn-ack" reason_ttl="0"/>
<address addr="10.10.10.137" addrtype="ipv4"/>
<hostnames>
<hostname name="HP1A4DC4" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="985">
<extrareasons reason="conn-refused" count="985"/>
</extraports>
<port protocol="tcp" portid="80"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" method="table" conf="3"/></port>
<port protocol="tcp" portid="139"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="netbios-ssn" method="table" conf="3"/></port>
<port protocol="tcp" portid="443"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="https" method="table" conf="3"/></port>
<port protocol="tcp" portid="445"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="microsoft-ds" method="table" conf="3"/></port>
<port protocol="tcp" portid="631"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ipp" method="table" conf="3"/></port>
<port protocol="tcp" portid="6839"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="unknown" method="table" conf="3"/></port>
<port protocol="tcp" portid="7435"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="unknown" method="table" conf="3"/></port>
<port protocol="tcp" portid="8080"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http-proxy" method="table" conf="3"/></port>
<port protocol="tcp" portid="9100"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="jetdirect" method="table" conf="3"/></port>
<port protocol="tcp" portid="9101"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="jetdirect" method="table" conf="3"/></port>
<port protocol="tcp" portid="9102"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="jetdirect" method="table" conf="3"/></port>
<port protocol="tcp" portid="9110"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="unknown" method="table" conf="3"/></port>
<port protocol="tcp" portid="9111"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="DragonIDSConsole" method="table" conf="3"/></port>
<port protocol="tcp" portid="9220"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="unknown" method="table" conf="3"/></port>
<port protocol="tcp" portid="9290"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="unknown" method="table" conf="3"/></port>
</ports>
<times srtt="6912" rttvar="5486" to="100000"/>
</host>

<host starttime="1464033224" endtime="1464034228"><status state="up" reason="conn-refused" reason_ttl="0"/>
<address addr="10.10.10.144" addrtype="ipv4"/>
<hostnames>
<hostname name="time-capsule" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="995">
<extrareasons reason="conn-refused" count="995"/>
</extraports>
<port protocol="tcp" portid="139"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="netbios-ssn" method="table" conf="3"/></port>
<port protocol="tcp" portid="445"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="microsoft-ds" method="table" conf="3"/></port>
<port protocol="tcp" portid="548"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="afp" method="table" conf="3"/></port>
<port protocol="tcp" portid="5009"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="airport-admin" method="table" conf="3"/></port>
<port protocol="tcp" portid="10000"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="snet-sensor-mgmt" method="table" conf="3"/></port>
</ports>
<times srtt="24675" rttvar="20640" to="107235"/>
</host>

<runstats><finished time="1464034248" timestr="Mon May 23 16:10:48 2016" elapsed="1024.26" summary="Nmap done at Mon May 23 16:10:48 2016; 256 IP addresses (10 hosts up) scanned in 1024.26 seconds" exit="success"/><hosts up="10" down="246" total="256"/>
</runstats>
</nmaprun>


My Ruby script is:

require 'nokogiri'

document = Nokogiri::XML(File.open("poodle_results.txt"))

document.xpath("//host").each do |host|
ip_address = host.at_xpath("address[@addrtype='ipv4']").at_xpath("@addr").value
result = host.at_xpath("//ports//elem[@key='state']").content
puts "#{ip_address} #{result}"
end


In this file there are 10 hosts, but only one that has the
<elem key="state">
tag. However my output looks like:

10.10.10.1 VULNERABLE
10.10.10.100 VULNERABLE
10.10.10.101 VULNERABLE
10.10.10.104 VULNERABLE
10.10.10.115 VULNERABLE
10.10.10.121 VULNERABLE
10.10.10.130 VULNERABLE
10.10.10.135 VULNERABLE
10.10.10.137 VULNERABLE
10.10.10.144 VULNERABLE


I'm confused at this point about what I've done wrong, since it's clearly iterating over each host. The last 9 hosts shown don't have the
<elem key="state">
tag within their
<host></host>
section, so they should not be included in the output.

If the
<elem key="state">
tag doesn't exist, I'd expect the outcome to just list the hosts that do have the key and it equals "VULNERABLE".

10.10.10.1 VULNERABLE

Answer

This gives you the correct XPath initially, so that you don't have to expend the effort looping through mismatched elements to get your results:

document.xpath("//host[.//elem/@key='state']").each do |host|
  ip_address = host.at_xpath("address[@addrtype='ipv4']").at_xpath("@addr").value
  result =  host.at_xpath("//ports//elem[@key='state']").content
  puts "#{ip_address} #{result}"
end

This chooses all of the host elements that have a sub-element with a key attribute with a 'state' value. No need to check for correct matches within the loop, because this will actually only return the elements that you want. It's a complete XPath solution to your question.