Arkay Arkay - 2 months ago 17
Java Question

Using a Java OutputFormat to emit Scala Tuples

I am trying to write a custom

OutputFormat
for ArangoDB. I've used
CassandraOutputFormat.java
as inspiration, swapping out the relevant drivers and calls etc. I wrote this in Java as I could not find a Scala example of an
OutputFormat
. The class therefore is defined by:

public class ArangoOutputFormat<OUT extends Tuple> extends RichOutputFormat<OUT>


Now, my Flink program where I want to write to ArangoDB is written in Scala. The dataset
test
is
DataSet[Tuple2[Long, Long]]
. The call to write to ArangoDB looks like:

test.output(
new ArangoOutputFormat[(Long, Long)]()
);


However when compiling I get the following error:


Error:(47, 11) type arguments [(Long, Long)] do not conform to class ArangoOutputFormat's type parameter bounds [OUT <:
org.apache.flink.api.java.tuple.Tuple]
new ArangoOutputFormat(Long, Long)


I'm generally a little confused on the type system when using both Scala and Java classes, can anyone explain what is happening here?

Thanks in advance,
Rob

Answer

The ArangoOutputFormat is typed to Flink's Java Tuples (org.apache.flink.api.java.tuple.Tuple) whereas your Scala program uses Scala's Tuples. Flink's Java tuples are not compatible with Scala's tuples. You need to explicitly create Java Tuples for instance in a Mapper before the sink.

Comments