Zefu Hu Zefu Hu - 2 years ago 141
Scala Question

Can Apache Kafka send non-string messages through a topic?

A mllib model is trained somewhere and I want it to be sent to somewhere else. When I try to send it through a kafka topic like this

val model = LogisticRegressionModel.load(sc, "/PATH/To/Model")
val producer=new Producer[String, LogisticRegressionModel](config)
val data=new KeyedMessage[String, LogisticRegressionModel(topic2,key,model)

I would encounter an error like this:

org.apache.spark.mllib.classification.LogisticRegressionModel cannot be cast to java.lang.String

So, is it possible for kafka to send non-string messages through a topic?

Answer Source

You can send non-string messages to Kafka topic using Kafka Producer. From 0.9.0 version its better to use Java Client instead of Scala Client.

All you need to do is specifying the correct Key, Value serializer in Properties like below.

props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
 props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download