Skip to content

Instantly share code, notes, and snippets.

@adilsonbna

adilsonbna/AnalyzeTweetsFromEventHub

Created December 15, 2018 14:55
Show Gist options
  • Save adilsonbna/e7fb947cac78a72e42672ac80fb2da08 to your computer and use it in GitHub Desktop.
Save adilsonbna/e7fb947cac78a72e42672ac80fb2da08 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
AnalyzeTweetsFromEventHub
import org.apache.spark.eventhubs._
// Build connection string with the above information
val connectionString = ConnectionStringBuilder("<EVENT HUBS CONNECTION STRING>")
.setEventHubName("<EVENT HUB NAME>")
.build
val customEventhubParameters =
EventHubsConf(connectionString)
.setMaxEventsPerTrigger(5)
val incomingStream = spark.readStream.format("eventhubs").options(customEventhubParameters.toMap).load()
incomingStream.printSchema
// Sending the incoming stream into the console.
// Data comes in batches!
incomingStream.writeStream.outputMode("append").format("console").option("truncate", false).start().awaitTermination()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment