Skip to content
bill4mobile edited this page Apr 6, 2016 · 18 revisions

Rivers were deprecated in Elasticsearch 1.5 and removed in Elasticsearch 2.0.

See https://www.elastic.co/blog/deprecating_rivers for more details. So please take care if you are using 2.0 above.

===========================

A short guide for the impatient!

Prerequisites:

A running MySQL database test, with user test and password test, a table orders

A terminal / console with commands curl and unzip

Internet access (of course)

Homebrew users

If you are installing elasticsearch with homebrew you will need to locate your $ES_HOME directory. The easiest way to do this is by navigating to

PATH_TO_SERVER:9200/_cluster/nodes?settings=true&pretty=true

See this Stack Overflow question for more details http://stackoverflow.com/questions/11954677/how-do-i-find-where-elasticsearch-is-installing-my-plugins

Let's go!

  1. Download elasticsearch (current version is 1.0.0.RC1, check for the most recent version on http://elasticsearch.org) from

    curl -OL https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.0.0RC1.zip

  2. Unpack zip file into you favorite elasticsearch directory, we call it $ES_HOME

    cd $ES_HOME

    unzip path/to/elasticsearch-1.0.0.RC1.zip

  3. Install JDBC river plugin (current version is 1.0.0.RC1.2 check for the most recent version, and if you have the file permissions to write into the plugins folder). If you have installed a JDBC river plugin before, it is recommended to remove the folder plugins/river-jdbc before installing a new version.

    ./bin/plugin -install river-jdbc -url http://bit.ly/1dKqNJy

  4. Download MySQL JDBC driver (current version is 5.1.28, check for the most recent version)

    curl -o mysql-connector-java-5.1.28.zip -L 'http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.28.zip/from/http://cdn.mysql.com/'

  5. Add MySQL JDBC driver jar to JDBC river plugin directory and set access permission for .jar file (at least chmod 644)

    cp mysql-connector-java-5.1.28-bin.jar $ES_HOME/plugins/river-jdbc/

  6. Start elasticsearch in terminal window with logging on the console

    ./bin/elasticsearch

  7. Start another terminal window, and create a new JDBC river with name my_jdbc_river with this curl command

    curl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{
        "type" : "jdbc",
        "jdbc" : {
            "url" : "jdbc:mysql://localhost:3306/test",
            "user" : "test",
            "password" : "test",
            "sql" : "select * from orders"
        }
    }'
    
  8. The river runs! It will run exactly once. Watch the log on the elasticsearch console for the river activity. When the river fetched the data, you can query your elasticsearch node for the data you just indexed with the following curl command

    curl -XGET 'localhost:9200/jdbc/_search?pretty&q=*'

  9. Enjoy the result!

  10. If you want to stop the my_jdbc_river river fetching data from the orders table after the quick demonstration, use this curl command:

    curl -XDELETE 'localhost:9200/_river/my_jdbc_river'

What next?

Now, if you want more fine-tuning, add a schedule for fetching data regularly, you can change the index namejdbc, add more SQL statements, tune bulk indexing, change the mapping, change the river creation settings - whatever you like.