eventstreams110.md

copyright	lastupdated
years 2015, 2018	2018-02-06

{:new_window: target="_blank"} {:shortdesc: .shortdesc} {:screen: .screen} {:codeblock: .codeblock} {:pre: .pre}

Using KSQL with {{site.data.keyword.messagehub}}

{: #ksql_using}

You can use KSQL {:new_window} with {{site.data.keyword.messagehub}} for stream processing. Ensure that you use KSQL 0.4, or later.

Complete the following steps:

1. Create an {{site.data.keyword.messagehub}} KSQL configuration file.

   For example:

   bootstrap.servers=BOOTSTRAP_SERVERS
   application.id=ksql_server_quickstart
   ksql.command.topic.suffix=commands
   listeners=http://localhost:8080
   security.protocol=SASL_SSL
   sasl.mechanism=PLAIN
   ssl.protocol=TLSv1.2
   ssl.enabled.protocols=TLSv1.2
   sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="USERNAME" password="PASSWORD";
   ksql.sink.replications.default=3
   

   where BOOTSTRAP_SERVERS, USERNAME, and PASSWORD are the values from your {{site.data.keyword.messagehub}} Service Credentials tab in {{site.data.keyword.Bluemix_notm}}.

2. Use the {{site.data.keyword.messagehub}} dashboard in the {{site.data.keyword.Bluemix_notm}} console to create a topic called ksql__commands with a single partition and the default retention period.

3. From a Docker terminal, start KSQL using the following command:

/bin/ksql-cli local 
--properties-file ./config/ksqlserver.properties

4. Use the {{site.data.keyword.messagehub}} dashboard in the {{site.data.keyword.Bluemix_notm}} console to create two topics with one partition each: users and pageviews.

   Then start DataGen twice as follows:

   i. With bootstrap-server=HOSTNAME:PORTNUMBER quickstart=users format=json topic=users maxInterval=10000 to start creating users events.

   ii. With bootstrap-server=HOSTNAME:PORTNUMBER quickstart=pageviews format=delimited topic=pageviews maxInterval=10000 to start creating pageviews events.

   Ensure you insert all the Kafka hosts listed in the Service Credentials page as values for bootstrap-server. To find this information, go to your {{site.data.keyword.messagehub}} instance in {{site.data.keyword.Bluemix_notm}}, go to the Service Credentials tab, and select the Credentials that you want to use.

When you have completed these steps, you can run all queries listed in the KSQL Quick Start Guide {:new_window}