Skip to content

Instantly share code, notes, and snippets.

@chy168

chy168/README

Last active December 3, 2015 13:32
Show Gist options
  • Save chy168/d83bf0e1e6a8477d2a0f to your computer and use it in GitHub Desktop.
Save chy168/d83bf0e1e6a8477d2a0f to your computer and use it in GitHub Desktop.
Download ZIP
Spark-redis example
Raw
README
./bin/spark-shell -packages redis.clients:jedis:2.7.2,com.redislabs:spark-redis:0.5.1 --repositories https://raw.github.com/RedisLabs/spark-redis/mvn-repo/
bad option: '-packages'
Raw
gistfile1.txt
import com.redislabs.provider.redis._
val projTable = sc.parallelize(Array( ("A", List("A1","A2","A3","A4")), ("B", List("B1","B2")), ("C", List("C1", "C2","C3")), ("D", List("D1")) ))
val gg = projTable.flatMap { case (k,v) =>
v.map { x=>
(k -> x)
}
}
gg.map{ case (x,y) =>
println(x, y)
sc.toRedisSET(y, x, ("127.0.0.1", 6379))
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment