Skip to content

Commit

Permalink
fix bcast example (#93)
Browse files Browse the repository at this point in the history
  • Loading branch information
holdenk authored Sep 11, 2017
1 parent d9893b8 commit 6680eed
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -108,11 +108,11 @@ object RDDJoinExamples {
V2 : ClassTag](bigRDD : RDD[(K, V1)],
smallRDD : RDD[(K, V2)])= {
val smallRDDLocal: Map[K, V2] = smallRDD.collectAsMap()
bigRDD.sparkContext.broadcast(smallRDDLocal)
val smallRDDLocalBcast = bigRDD.sparkContext.broadcast(smallRDDLocal)
bigRDD.mapPartitions(iter => {
iter.flatMap{
case (k,v1 ) =>
smallRDDLocal.get(k) match {
smallRDDLocalBcast.value.get(k) match {
case None => Seq.empty[(K, (V1, V2))]
case Some(v2) => Seq((k, (v1, v2)))
}
Expand Down

0 comments on commit 6680eed

Please sign in to comment.