sab sab - 2 months ago 17
Java Question

how converte large XML file to stream-like-java8 of object

Hy,
I have a (very)large XML file (100GB) with a list of foo, I want to convert-it into a stream like they where introduce into java 8 of object:

Any idea of lib or code sample?

at the beginning:

<foos>
<foo>...</foo>
<foo>...</foo>
</foos>


at the end:

Stream<Foo> foosStream = ????("foo.xml")
streamFoos.forEach(foo->foo.doFooStuffs());


Edit:
@Pierre Thank-you, here is the implementation of your solution:

try {
XMLEventReader reader = XMLInputFactory.newInstance().
createXMLEventReader(stream);
final Unmarshaller unmarshaller = JAXBContext.newInstance(XXXXX.class).createUnmarshaller();

Iterator<XXXXX> it = new XmlIterator<>(reader, unmarshaller, "xxxxxx");
return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, Spliterator.ORDERED), false);
} catch (XMLStreamException e1) {
logger.error("XMLStreamException", e1);
} catch (JAXBException e) {
logger.error("JAXBException", e);
}


and

public class XmlIterator<T> implements Iterator<T> {

private final Logger logger = LoggerFactory.getLogger(this.getClass());

XMLEventReader reader;

XMLEvent event;

Unmarshaller unmarshaller;
String name;

public XmlIterator(XMLEventReader reader, Unmarshaller unmarshaller, String name) {
this.reader = reader;
this.unmarshaller = unmarshaller;
this.name = name;
try {
reader.next();
this.event = reader.peek();
} catch (XMLStreamException e) {
logger.error("", e);
event = null;
}
}

@Override
public boolean hasNext() {
try {
while (event != null && !(event.isStartElement() && name.equals(event.asStartElement().getName().getLocalPart()))) {
Object a = reader.next();
event = reader.peek();
}
return event != null;

} catch (XMLStreamException e) {
logger.error("", e);
event = null;
}
return event != null;
}

@Override
public T next() {
try {
T next = ((JAXBElement<T>) unmarshaller.unmarshal(reader)).getValue();
event = reader.peek();
return next;
} catch (JAXBException e) {
logger.error("error during unmarshalling ", e);
return null;
} catch (XMLStreamException e) {
logger.error("error during stream ", e);
return null;
}
}
}

Answer
Comments