Implementing divolteStream usign DirectStream API

[fnozarian]

Hello.
I noticed that the divolteStream uses createStream method of KafkaUtils to establish a stream, however since Spark 1.3 it is recomended to use DirectStream API (createDirectStream) to read data from Kafka. Do you have any plan to re-implement divolteStream using this API?

[asnare]

These examples really need a good polish: although the general strategies are roughly right, the implementation details have changed significantly. In particular, the Spark project has evolved considerably since the examples were written:

• Avro files can now be easily read using the DataFrame API. See https://github.com/databricks/spark-avro.
• Kafka streaming support has improved considerably.

I'm currently planning some work on improving the way Kafka is supported and hope to revisit the examples after that.