MongoSpark

This program illustrates how to insert JSON data to MongoDB and query from MongoDB using Java and Spark.

Details

MongoDB

• Database name: test-db
• Collection name: test-collection
• User: admin
• Password: superadmin
• _id: country-city (example: canada-vancouver, united_states-las_vegas)

Program Arguments

Write

args[0]

{
  "user": "MongoDB user",
  "password": "MongoDB password",
  "ip": "MongoDB IP address",
  "port": "MongoDB port",
  "database": "Database name",
  "collection": "Collection name"
}

args[1]: path to JSON file

Read

args[0]

{
  "user": "MongoDB user",
  "password": "MongoDB password",
  "ip": "MongoDB IP address",
  "port": "MongoDB port",
  "database": "Database name",
  "collection": "Collection name"
}

args[1]: query

How to compile

mvn install

Running the program

Write

spark-submit --class com.malik.main.WriteMain --master local[2] malik/engine/MongoSpark-1.0-SNAPSHOT-jar-with-dependencies.jar '{"user":"admin","password":"superadmin","ip":"192.168.20.99","port":"27017","database":"test-db","collection":"test-collection"}' '/home/malik/data/city_attributes.json'

Read

spark-submit --class com.malik.main.ReadMain --master local[2] malik/engine/MongoSpark-1.0-SNAPSHOT-jar-with-dependencies.jar '{"user":"admin","password":"superadmin","ip":"192.168.20.99","port":"27017","database":"test-db","collection":"test-collection"}' '{ $match: { country: "Canada" } }'