Hadoop Benchmarks

A list of Hadoop Ecosystem Benchmark Links for reference

HDFS & MapReduce

http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraValidate.java

https://community.hortonworks.com/content/kbentry/44315/teragen-terasort-and-teravalidate-performance-test.html

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java

https://github.com/sunileman/MapReduce-Performance_Testing

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java

HBase

https://github.com/brianfrankcooper/YCSB/wiki

https://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation

https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java

http://hbase.apache.org/0.94/book/hbase.tests.html

https://github.com/cartershanklin/phoenix-performance

Hive

https://github.com/hortonworks/hive-testbench

http://www.tpc.org/tpcx-hs/

http://www.tpc.org/tpch/

http://www.tpc.org/tpcds/

Spark

https://github.com/intel-hadoop/HiBench/tree/master/sparkbench

https://github.com/databricks/spark-perf

https://github.com/databricks/spark-sql-perf

http://01org.github.io/sparkscore/plaf1.html

https://github.com/wardbekker/benchmark

Kafka

https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines

https://gist.github.com/jkreps/c7ddb4041ef62a900e6c

Various

http://hadoop.apache.org/docs/r2.7.3/hadoop-gridmix/GridMix.html

https://github.com/intel-hadoop/HiBench

https://github.com/Netflix/chaosmonkey

https://github.com/saurabhmishra/benchmarkhadoop

http://hadoop.apache.org/docs/r2.5.2/hadoop-project-dist/hadoop-hdfs/SLGUserGuide.html

https://hadoop.apache.org/docs/current/hadoop-gridmix/GridMix.html