-
Notifications
You must be signed in to change notification settings - Fork 708
Quickstart
Rivers were deprecated in Elasticsearch 1.5 and removed in Elasticsearch 2.0.
See https://www.elastic.co/blog/deprecating_rivers for more details. So please take care if you are using 2.0 above.
===========================
A short guide for the impatient!
Prerequisites:
A running MySQL database test
, with user test
and password test
, a table orders
A terminal / console with commands curl
and unzip
Internet access (of course)
If you are installing elasticsearch with homebrew you will need to locate your $ES_HOME directory. The easiest way to do this is by navigating to
PATH_TO_SERVER:9200/_cluster/nodes?settings=true&pretty=true
See this Stack Overflow question for more details http://stackoverflow.com/questions/11954677/how-do-i-find-where-elasticsearch-is-installing-my-plugins
-
Download elasticsearch (current version is 1.0.0.RC1, check for the most recent version on http://elasticsearch.org) from
curl -OL https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.0.0RC1.zip
-
Unpack zip file into you favorite elasticsearch directory, we call it $ES_HOME
cd $ES_HOME
unzip path/to/elasticsearch-1.0.0.RC1.zip
-
Install JDBC river plugin (current version is 1.0.0.RC1.2 check for the most recent version, and if you have the file permissions to write into the
plugins
folder). If you have installed a JDBC river plugin before, it is recommended to remove the folderplugins/river-jdbc
before installing a new version../bin/plugin -install river-jdbc -url http://bit.ly/1dKqNJy
-
Download MySQL JDBC driver (current version is 5.1.28, check for the most recent version)
curl -o mysql-connector-java-5.1.28.zip -L 'http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.28.zip/from/http://cdn.mysql.com/'
-
Add MySQL JDBC driver jar to JDBC river plugin directory and set access permission for .jar file (at least chmod 644)
cp mysql-connector-java-5.1.28-bin.jar $ES_HOME/plugins/river-jdbc/
-
Start elasticsearch in terminal window with logging on the console
./bin/elasticsearch
-
Start another terminal window, and create a new JDBC river with name
my_jdbc_river
with thiscurl
commandcurl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{ "type" : "jdbc", "jdbc" : { "url" : "jdbc:mysql://localhost:3306/test", "user" : "test", "password" : "test", "sql" : "select * from orders" } }'
-
The river runs! It will run exactly once. Watch the log on the elasticsearch console for the river activity. When the river fetched the data, you can query your elasticsearch node for the data you just indexed with the following
curl
commandcurl -XGET 'localhost:9200/jdbc/_search?pretty&q=*'
-
Enjoy the result!
-
If you want to stop the
my_jdbc_river
river fetching data from theorders
table after the quick demonstration, use thiscurl
command:curl -XDELETE 'localhost:9200/_river/my_jdbc_river'
Now, if you want more fine-tuning, add a schedule for fetching data regularly, you can change the index namejdbc
, add more SQL statements, tune bulk indexing, change the mapping, change the river creation settings - whatever you like.