Skip to content
This repository has been archived by the owner on Aug 23, 2021. It is now read-only.

Comdb2 addition #184

Open
wants to merge 28 commits into
base: master
Choose a base branch
from
Open

Conversation

saatviks
Copy link

This pull request provides functionality to benchmark ComDB2 with Oltpbench. The benchmarks worked on are:

  1. TPCC: Fully integrated.
  2. Wikipedia: All but one procedure has been integrated. The one procedure not integrated is due to many required features still not being supported by ComDB2.
  3. TPCH: Partially Integrated.

Copy link
Member

@apavlo apavlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address my questions/remarks.

@dvanaken Can you coordinate with @saatviks whether we want to add the submodule dependency for TPC-H dbgen?

<!-- Connection details -->
<dbtype>comdb2</dbtype>
<driver>com.bloomberg.comdb2.jdbc.Driver</driver>
<DBUrl><![CDATA[jdbc:comdb2://localhost/tpcc?statement_query_effects=1&verify_retry=1]]></DBUrl>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need the CDATA part here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The '&' operator causes some XML parsing issue - The 'CDATA' makes XML escape the '&'

log4j.properties Outdated
@@ -12,6 +12,7 @@ log4j.logger.com.oltpbenchmark=INFO
log4j.logger.com.oltpbenchmark.api=INFO

# Benchmarks
log4j.logger.com.oltpbenchmark.benchmarks.tpcc=DEBUG
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switch to 'INFO'

pom.xml Outdated
@@ -150,5 +150,7 @@
<artifactId>ojdbc14</artifactId>
<version>10.2.0.1.0</version>
</dependency>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What did you change in this file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No changes - We'll revert this.

@@ -173,7 +173,7 @@ public void unload(Catalog catalog) throws SQLException {
Statement st = conn.createStatement();
for (Table catalog_tbl : catalog.getTables()) {
LOG.debug(String.format("Deleting data from %s.%s", workConf.getDBName(), catalog_tbl.getName()));
String sql = "DELETE FROM " + catalog_tbl.getEscapedName();
String sql = "truncate " + catalog_tbl.getEscapedName();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comdb2 specific?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were a number of issues in trying to get TPCH play well with ComDB2 - Most of the changes in the Oltpbench source were our attempts(and suggestions from ComDB2 devs) to get it to work. @QinyuHu and I will revert these changes soon since TPC-H didnt end up working.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will revert the changes - they only change "DELETE FROM" to "truncate" for speed up reason. Also System.out was used for debug purpose so we will also revert them.

@@ -0,0 +1,19 @@
<?xml version="1.0"?>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good! Nice work!

@@ -229,7 +229,8 @@ public void load() throws SQLException {
static void truncateTable(String strTable) throws SQLException {
LOG.debug("Truncating '" + strTable + "' ...");
try {
conn.createStatement().execute("DELETE FROM " + strTable);
System.out.println("truncate " + strTable);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove all System.out calls.

@@ -229,7 +229,8 @@ public void load() throws SQLException {
static void truncateTable(String strTable) throws SQLException {
LOG.debug("Truncating '" + strTable + "' ...");
try {
conn.createStatement().execute("DELETE FROM " + strTable);
System.out.println("truncate " + strTable);
conn.createStatement().execute("truncate " + strTable);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, is this comdb2 specific?

if (loaders[i] != null)
loaders[i].start();

// for (int i = 0; i < 8; ++i)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What did you change here?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part changes the loading from multiple threads at the same time to one at a time. The reason is some unknown issue (till now) of comdb2 (or something between oltp and comdb2) part that stopped multi-thread running. Should we keep some comment here or just revert everything?

@QinyuHu
Copy link

QinyuHu commented Dec 22, 2017

@apavlo Prof., unnecessary changes (only TPCH config files for comdb2 remains) have been reverted and other changes also made according to previous comments.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants