holaSenor holaSenor - 6 months ago 23
Ruby Question

How to parse XML to CSV where data is in attributes only

The XML file I am trying to parse has all the data contained in attributes. I found how to build the string to insert into the text file.

I have this XML file:

<ig:prescribed_item class_ref="0161-1#01-765557#1">
<ig:prescribed_property property_ref="0161-1#02-016058#1" is_required="false" combination_allowed="false" one_of_allowed="false">
<dt:measure_number_type representation_ref="0161-1#04-000005#1">
<dt:real_type>
<dt:real_format pattern="\d(1,)\.\d(1,)"/>
</dt:real_type>
<dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
</dt:measure_number_type>
</ig:prescribed_property>
<ig:prescribed_property property_ref="0161-1#02-016059#1" is_required="false" combination_allowed="false" one_of_allowed="false">
<dt:measure_number_type representation_ref="0161-1#04-000005#1">
<dt:real_type>
<dt:real_format pattern="\d(1,)\.\d(1,)"/>
</dt:real_type>
<dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
</dt:measure_number_type>
</ig:prescribed_property>
</ig:prescribed_item>
</ig:identification_guide>


And I want to parse it into a text file like this with the class ref duplicated for each property:

class_ref|property_ref|is_required|UOM_ref
0161-1#01-765557#1|0161-1#02-016058#1|false|0161-1#05-003260#1
0161-1#01-765557#1|0161-1#02-016059#1|false|0161-1#05-003260#1


This is the code I have so far:

require 'nokogiri'

doc = Nokogiri::XML(File.open("file.xml"), 'UTF-8') do |config|
config.strict
end

content = doc.xpath("//ig:prescribed_item/@class_ref").map {|i|
i.search("//ig:prescribed_item/ig:prescribed_property/@property_ref").map { |d| d.text }
}

puts content.inspect

content.each do |c|
puts c.join('|')
end

Answer

I'd simplify it a bit using CSS accessors:

xml = <<EOT
<ig:prescribed_item class_ref="0161-1#01-765557#1">
    <ig:prescribed_property property_ref="0161-1#02-016058#1" is_required="false" combination_allowed="false" one_of_allowed="false">
        <dt:measure_number_type representation_ref="0161-1#04-000005#1">
            <dt:real_type>
                <dt:real_format pattern="\d(1,)\.\d(1,)"/>
            </dt:real_type>
            <dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
        </dt:measure_number_type>
    </ig:prescribed_property>
    <ig:prescribed_property property_ref="0161-1#02-016059#1" is_required="false" combination_allowed="false" one_of_allowed="false">
        <dt:measure_number_type representation_ref="0161-1#04-000005#1">
            <dt:real_type>
                <dt:real_format pattern="\d(1,)\.\d(1,)"/>
            </dt:real_type>
            <dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
        </dt:measure_number_type>
    </ig:prescribed_property>
</ig:prescribed_item>
</ig:identification_guide>
EOT

require 'nokogiri'

doc = Nokogiri::XML(xml)

data = [ %w[ class_ref property_ref is_required UOM_ref] ]

doc.css('|prescribed_item').each do |pi|
  pi.css('|prescribed_property').each do |pp|
    data << [
      pi['class_ref'],
      pp['property_ref'],
      pp['is_required'],
      pp.at_css('|prescribed_unit_of_measure')['UOM_ref']
    ]
  end
end

puts data.map{ |row| row.join('|') }

Which outputs:

class_ref|property_ref|is_required|UOM_ref
0161-1#01-765557#1|0161-1#02-016058#1|false|0161-1#05-003260#1
0161-1#01-765557#1|0161-1#02-016059#1|false|0161-1#05-003260#1

Could you explain this line in greater detail "pp.at_css('|prescribed_unit_of_measure')['UOM_ref']"

In Nokogiri, there are two types of "find a node" methods: The "search" methods return all nodes that match a particular accessor as a NodeSet, and the "at" methods return the first Node of the NodeSet which will be the first encountered Node that matched the accessor.

The "search" methods are things like search, css, xpath and /. The "at" methods are things like at, at_css, at_xpath and %. Both search and at accept either XPath or CSS accessors.

Back to pp.at_css('|prescribed_unit_of_measure')['UOM_ref']: At that point in the code pp is a local variable containing a "prescribed_property" Node. So, I'm telling the code to find the first node under pp that matches the CSS |prescribed_unit_of_measure accessor, in other words the first <dt:prescribed_unit_of_measure> tag contained by the pp node. When Nokogiri finds that node, it returns the value of the UOM_ref attribute of the node.

As a FYI, the / and % operators are aliased to search and at respectively in Nokogiri. They're part of its "Hpricot" compatability; We used to use them a lot when Hpricot was the XML/HTML parser of choice, but they're not idiomatic for most Nokogiri developers. I suspect it's to avoid confusion with the regular use of the operators, at least it is in my case.

Also, Nokogiri's CSS accessors have some extra-special juiciness; They support namespaces, like the XPath accessors do, only they use |. Nokogiri will let us ignore the namespaces, which is what I did. You'll want to nose around in the Nokogiri docs for CSS and namespaces for more information.

Comments