Merge pull request #176 from aerospike/2.1.1

2.1.1

BUILD.md

@@ -66,6 +66,25 @@ download the appropriate C client. However, if one is present this will not
 happen, and a build may fail against an old client. At that point you should
 remove the directory `aerospike-client-c` and run the build command again.
 
+If you see the error `error: need permission to copy the Lua system files to /usr/local/aerospike/lua or change the --lua-system-path`
+
+You may need to either run the command with sudo, or alternatively specify a different location for the Lua system modules to be installed:
+
+	python setup.py build --force --lua-system-path=/path/to/lua
+
+### Troubleshooting OS X builds
+Building on OS X versions >= 10.11 , may cause a few additional errors to be generated. If the build command fails with an
+error similar to: `error: could not create '/usr/local/aerospike/lua': Permission denied` there are a couple of options:
+
+- Rerun the build command with the additional command line flags `--user --prefix=` *Note that there are no charcters after the '='.* This will cause the library to only be installed for the current user, and store the library's data files in a user specific location.
+- rerun the command with sudo.
+
+If an error similar to `ld: targeted OS version does not support use of thread local variables` appears, it can be fixed by temporarily setting the `MACOSX_DEPLOYMENT_TARGET` environment variable to `'10.12'` e.g.
+
+```
+MACOSX_DEPLOYMENT_TARGET=10.12 python setup.py build --force
+MACOSX_DEPLOYMENT_TARGET=10.12 python setup.py install --force
+```
 
 ### Building on an Unsupported Linux Distro
 
@@ -98,6 +117,20 @@ To install the library:
 
     python setup.py install --force
 
+If you see the error `error: need permission to copy the Lua system files to /usr/local/aerospike/lua or change the --lua-system-path`
+
+You may need to either run the command with sudo, or alternatively specify a different location for the Lua system modules to be installed:
+
+	python setup.py install --force --lua-system-path=/path/to/lua
+
+
+### Troubleshooting OS X Installation
+Installing on OS X versions >= 10.11 , may cause a few additional errors to be generated. If the install command fails with an
+error similar to: `error: could not create '/usr/local/aerospike/lua': Permission denied` there are a couple of options:
+
+- Rerun the install command with the additional command line flags `--user --prefix=` *Note that there are no charcters after the '='.* This will cause the library to only be installed for the current user, and store the library's data files in a user specific location.
+- rerun the command with sudo.
+
 ### Lua System Modules
 
 Stream UDF functionality requires a local copy of the system Lua modules.
@@ -107,6 +140,11 @@ A different directory can be created, then set:
     python setup.py install --lua-system-path=/path/to/lua
 
 
+**Note** If you do not use the default location, and you wish to perform Stream UDF operations it will be necessary to specify the locations of the system modules as a configuration parameter to the Aerospike client constructor:
+
+	config = {'hosts': [('127.0.0.1', 3000)], 'lua': {'system_path': '/path/to/lua'} ...}
+	my_client = aerospike.client(config)
+
 ## Examples
 
 **Note** If you did not install the library, then you will need to setup your `PYTHONPATH` environment variable. The `PYTHONPATH` should contain an entry for the directory where the Python module is stored. This is usually in `build/lib.*`.

README.rst

@@ -77,7 +77,7 @@ Aerospike Python Client can be installed using ``pip``:
     # to troubleshoot pip versions >= 6.0 you can
     pip install --no-cache-dir aerospike
 
-    # to trouleshoot installation on OS X El-Capitan (10.11)
+    # to trouleshoot installation on OS X El-Capitan (10.11) or OS X Sierra (10.12)
     pip install --no-cache-dir --user aerospike
 
     # to have pip copy the Lua system files to a dir other than /usr/local/aerospike/lua
@@ -86,9 +86,19 @@ Aerospike Python Client can be installed using ``pip``:
 If you run into trouble installing the client on a supported OS, you may be
 using an outdated ``pip``.
 Versions of ``pip`` older than 7.0.0 should be upgraded, as well as versions of
-``setuptools`` older than 18.0.0. Upgrading ``pip`` on OS X El-Capitan (10.11)
+``setuptools`` older than 18.0.0.
+
+OS X Installation
+~~~~~~~~~~~~~~~~~~
+Upgrading ``pip`` on OS X El-Capitan (10.11) or OS X Sierra(10.12)
 runs into `SIP issues <https://apple.stackexchange.com/questions/209572/how-to-use-pip-after-the-el-capitan-max-os-x-upgrade>`__
-with ``pip install --user <module>`` as the recommended workaround.
+with ``pip install --user aerospike`` as the recommended workaround to install aerospike on those versions of OS X.
+
+Attempting to install the client with pip for the system default Python may cause permssions issues when copying necesarry files. In order to avoid
+those issues the client can be installed for the current user only with the command: ``pip install --user aerospike``
+
+If the version of Python is not in the officially supported list, or the ``--install-option`` argument is provided, pip will attempt to compile the client from source. Please see the `build directions in the GitHub repository <https://github.com/aerospike/aerospike-client-python/blob/master/BUILD.md>`__
+to troubleshoot any issues caused by compiling the client.
 
 
 Build
@@ -118,9 +128,15 @@ For examples, to run the ``kvs.py``:
 Benchmarks
 ----------
 
-To run the benchmarks the python modules 'guppy' and 'tabulate' need to be installed.
+To run the benchmarks the python module 'tabulate' need to be installed. In order to display heap information the module `guppy` must be installed.
+Note that `guppy` is only available for Python2. If `guppy` is not installed the benchmarks will still be runnable.
 Benchmark applications are provided in the `benchmarks directory of the GitHub repository <https://github.com/aerospike/aerospike-client-python/tree/master/benchmarks>`__
 
+By default the benchmarks will try to connect to a server located at 127.0.0.1:3000 , instructions on changing that setting and other command line flags may be displayed by appending the `--help` argument to the benchmark script. For example:
+::
+
+    python benchmarks/keygen.py --help
+
 License
 -------
 

VERSION

@@ -1 +1 @@
-2.1.0
+2.1.1

benchmarks/README.rst

@@ -0,0 +1,53 @@
+Python client Benchmarks
+=========================
+
+
+Prerequisites
+~~~~~~~~~~~~~~
+To run the benchmarks the python module 'tabulate' needs to be installed. In order to display heap information the module `guppy` must be installed.
+Note that `guppy` is only available for Python2. If `guppy` is not installed the benchmarks will still be runnable, but no heap information
+will be displayed.
+
+Available Benchmarks
+~~~~~~~~~~~~~~~~~~~~~
+There are currently two benchmarks provided for the Aerospike Python client:
+
+keygen.py
+-------------------
+This benchmark will write small distinct records to the database.
+Command line usage help is available by running.
+::
+	python keygen.py --help
+
+It will report:
+- Numer of keys generated
+- Number of operations
+- Runtime
+- Operations per second
+
+
+kvs.py
+-------
+This benchmark will perform writes and reads to the database. It is possible to specify a read and write ratio.
+Command line usage help is available by running.
+::
+	python kvs.py --help
+
+It will report
+- Numer of keys generated
+- Number of operations
+- Runtime
+- Operations per second
+- Latency statistics for read and write operations
+
+
+Example Usage
+~~~~~~~~~~~~~~
+To run keygen.py against a server located at 127.0.0.1 listening on port 3000 to the set named "benchmark"
+::
+	python keygen.py -h "127.0.0.1" -p 3000 -s "benchmark"
+
+To run kvs.py against a server located at 127.0.0.1 listening on port 3000 to the set named "benchmark" with 75% reads
+and 25% writes of string data with minimum length of 15 and maximum length of 20 characters
+::
+	python kvs.py -h "127.0.0.1" -p 3000 -s "benchmark" --reads 75 --writes 25 --gen "str" --str-min 15 --str-max 20

benchmarks/keygen.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 ##########################################################################
-# Copyright 2013-2016 Aerospike, Inc.
+# Copyright 2013-2017 Aerospike, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -55,11 +55,11 @@
 
 optparser.add_option(
     "-n", "--namespace", dest="namespace", type="string", default="test", metavar="<NS>",
-    help="Port of the Aerospike server.")
+    help="Namespace that records will be stored and retrieved from.")
 
 optparser.add_option(
     "-s", "--set", dest="set", type="string", default="demo", metavar="<SET>",
-    help="Port of the Aerospike server.")
+    help="Set that records will be stored and retrieved from.")
 
 optparser.add_option(
     "-v", "--verbose", dest="verbose", action="store_true", metavar="<PORT>",

benchmarks/kvs.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 ##########################################################################
-# Copyright 2013-2016 Aerospike, Inc.
+# Copyright 2013-2017 Aerospike, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -24,11 +24,23 @@
 import string
 import time
 
-# from guppy import hpy
+
 from optparse import OptionParser
 import copy
 from tabulate import tabulate
 
+# Guppy is only available in python 2.7, try to load it
+try:
+    from guppy import hpy
+    HAVE_HEAPY = True
+except ImportError:
+    HAVE_HEAPY = False
+# set maximum integer value compatible in python2 and 3
+try:
+    MAX_INT = sys.maxint
+except AttributeError:
+    MAX_INT = sys.maxsize
+
 ##########################################################################
 # Options Parsing
 ##########################################################################
@@ -63,11 +75,11 @@
 
 optparser.add_option(
     "-n", "--namespace", dest="namespace", type="string", default="test", metavar="<NS>",
-    help="Port of the Aerospike server.")
+    help="Namespace that records will be stored and retrieved from.")
 
 optparser.add_option(
     "-s", "--set", dest="set", type="string", default="demo", metavar="<SET>",
-    help="Port of the Aerospike server.")
+    help="Set that records will be stored and retrieved from.")
 
 optparser.add_option(
     "-v", "--verbose", dest="verbose", action="store_true", metavar="<PORT>",
@@ -94,7 +106,7 @@
     help="Minimum value for generated integers.")
 
 optparser.add_option(
-    "--int-max", dest="int_max", type="int", default=sys.maxint,
+    "--int-max", dest="int_max", type="int", default=MAX_INT,
     help="Maximum value for generated integere.")
 
 optparser.add_option(
@@ -158,6 +170,7 @@
 def operation(r, w):
     rn = r
     wn = w
+    total = r + w
     while rn + wn > 0:
         if rn > 0 and wn > 0:
             op = random.randint(0, 100)
@@ -210,7 +223,8 @@ def genint():
 write_count = 0
 start = 0
 intervals = []
-heapy = hpy()
+if HAVE_HEAPY:
+    heapy = hpy()
 
 no_of_buckets, range_increment = [
     int(elem.strip()) for elem in options.latency.split(',')]
@@ -275,8 +289,9 @@ def total_summary():
     table = [info for info in interprete_summary()]
     print(tabulate(table, headers=get_latency_table_headers()))
     print()
-    print("Heap: ")
-    print(heapy.heap())
+    if HAVE_HEAPY:
+        print("Heap: ")
+        print(heapy.heap())
 
     sys.exit(0)
 
@@ -317,7 +332,8 @@ def total_summary():
 
         start = time.time()
 
-        print("HEAP@{0}: {1}".format(0, heapy.heap()))
+        if HAVE_HEAPY:
+            print("HEAP@{0}: {1}".format(0, heapy.heap()))
 
         # run the operatons
         for op in operation(r, w):
@@ -351,7 +367,8 @@ def total_summary():
                 write_count += 1
 
             if options.heap_interval > 0 and (total_count % options.heap_interval) == 0:
-                print("HEAP@{0}: {1}".format(total_count, heapy.heap()))
+                if HAVE_HEAPY:
+                    print("HEAP@{0}: {1}".format(total_count, heapy.heap()))
 
     except KeyboardInterrupt:
         total_summary()