Skip to content

Commit

Permalink
Merge pull request #175 from aerospike/2.1.0
Browse files Browse the repository at this point in the history
2.1.0
  • Loading branch information
Jeff Boone authored May 25, 2017
2 parents a70e980 + 8295002 commit 6d20dc9
Show file tree
Hide file tree
Showing 34 changed files with 571 additions and 286 deletions.
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.0.13
2.1.0
19 changes: 17 additions & 2 deletions doc/aerospike.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,10 @@ in an in-memory primary index.
.. hlist::
:columns: 1

* **hosts** a required :class:`list` of (address, port) tuples identifying a node (or multiple nodes) in the cluster. \
* **hosts** a required :class:`list` of (address, port, [tls-name]) tuples identifying a node (or multiple nodes) in the cluster. \
The client will connect to the first available node in the list, the *seed node*, \
and will learn about the cluster and partition map from it.
and will learn about the cluster and partition map from it. If tls-name is specified, it must match the tls-name specified in the node's
server configuration file and match the server's CA certificate. **Note: use of TLS requires Aerospike Enterprise Edition**
* **lua** an optional :class:`dict` containing the paths to two types of Lua modules
* **system_path** the location of the system modules such as ``aerospike.lua`` (default: ``/usr/local/aerospike/lua``)
* **user_path** the location of the user's record and stream UDFs
Expand All @@ -66,6 +67,20 @@ in an in-memory primary index.
* **takeover_threshold_sec** take over tending if the cluster hasn't been checked for this many seconds (default: 30)
* **shm_key** explicitly set the shm key for this client. It is otherwise implicitly evaluated per unique hostname, and can be inspected with :meth:`~aerospike.Client.shm_key` (default: 0xA5000000)
* **use_shared_connection** :class:`bool` indicating whether this instance should share its connection to the Aerospike cluster with other client instances in the same process. (default: ``False``)
* **tls** a :class:`dict` of optional TLS configuration parameters. **TLS usage requires Aerospike Enterprise Edition**
* **enable** a :class:`bool` indicating whether tls should be enabled or not. Default: ``False``
* **cafile** :class:`str` Path to a trusted CA certificate file. By default TLS will use system standard trusted CA certificates
* **capath** :class:`str` Path to a directory of trusted certificates. See the OpenSSL SSL_CTX_load_verify_locations manual page for more information about the format of the directory.
* **protocols** Specifies enabled protocols. This format is the same as Apache's SSLProtocol documented at https://httpd.apache.org/docs/current/mod/mod_ssl.html#sslprotocol . If not specified the client will use "-all +TLSv1.2".
* **cipher_suite** :class:`str` Specifies enabled cipher suites. The format is the same as OpenSSL's Cipher List Format documented at https://www.openssl.org/docs/manmaster/apps/ciphers.html .If not specified the OpenSSL default cipher suite described in the ciphers documentation will be used. If you are not sure what cipher suite to select this option is best left unspecified
* **keyfile** :class:`str` Path to the client's key for mutual authentication. By default mutual authentication is disabled.
* **cert_blacklist** :class:`str` Path to a certificate blacklist file. The file should contain one line for each blacklisted certificate. Each line starts with the certificate serial number expressed in hex. Each entry may optionally specify the issuer name of the certificate (serial numbers are only required to be unique per issuer). Example records: 867EC87482B2 /C=US/ST=CA/O=Acme/OU=Engineering/CN=Test Chain CA E2D4B0E570F9EF8E885C065899886461
* **certfile** :class:`str` Path to the client's certificate chain file for mutual authentication. By default mutual authentication is disabled.
* **encrypt_only** :class:`bool` If ``True`` Only encrypt connections; do not verify certificates. By default TLS will verify certificates.
* **crl_check** :class:`bool` Enable CRL checking for the certificate chain leaf certificate. An error occurs if a suitable CRL cannot be found. By default CRL checking is disabled.
* **crl_check_all** :class:`bool` Enable CRL checking for the entire certificate chain. An error occurs if a suitable CRL cannot be found. By default CRL checking is disabled.
* **log_session_info** :class:`bool` Log session information for each connection.
* **max_socket_idle** :class:`int` Maximum socket idle in seconds for TLS connections. TLS Socket connection pools will discard sockets that have been idle longer than the maximum. The value is limited to 24 hours (86400). It's important to set this value to a few seconds less than the server's proto-fd-idle-ms (default 60000 milliseconds or 1 minute), so the client does not attempt to use a socket that has already been reaped by the server. Default: 55 seconds
* **serialization** an optional instance-level :py:func:`tuple` of (serializer, deserializer). Takes precedence over a class serializer registered with :func:`~aerospike.set_serializer`.
* **thread_pool_size** number of threads in the pool that is used in batch/scan/query commands (default: 16)
* **max_threads** size of the synchronous connection pool for each server node (default: 300) *DEPRECATED*
Expand Down
3 changes: 3 additions & 0 deletions doc/client.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2126,6 +2126,7 @@ a cluster-tending thread.
.. versionchanged:: 1.0.41
.. warning:: ``get_nodes`` will not work when using TLS
.. method:: info(command[, hosts[, policy]]) -> {}
Expand Down Expand Up @@ -2174,6 +2175,8 @@ a cluster-tending thread.
.. versionchanged:: 1.0.41
.. warning:: ``info_node`` will not work when using TLS
.. method:: has_geo() -> bool
Check whether the connected cluster supports geospatial data and indexes.
Expand Down
3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def run(self):
os.environ['ARCHFLAGS'] = '-arch x86_64'
AEROSPIKE_C_VERSION = os.getenv('AEROSPIKE_C_VERSION')
if not AEROSPIKE_C_VERSION:
AEROSPIKE_C_VERSION = '4.1.5'
AEROSPIKE_C_VERSION = '4.1.6'
DOWNLOAD_C_CLIENT = os.getenv('DOWNLOAD_C_CLIENT')
AEROSPIKE_C_HOME = os.getenv('AEROSPIKE_C_HOME')
PREFIX = None
Expand Down Expand Up @@ -370,6 +370,7 @@ def resolve_c_client(lua_src_path, lua_system_path):
'src/main/policy.c',
'src/main/calc_digest.c',
'src/main/predicates.c',
'src/main/tls_config.c',
'src/main/global_hosts/type.c',
'src/main/nullobject/type.c',
],
Expand Down
23 changes: 23 additions & 0 deletions src/include/tls_config.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*******************************************************************************
* Copyright 2017 Aerospike, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
******************************************************************************/

#include <Python.h>

#include <aerospike/as_config.h>

#include "macros.h"

void setup_tls_config(as_config* config, PyObject* tls_config);
2 changes: 1 addition & 1 deletion src/main/aerospike.c
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ AerospikeConstants operator_constants[] = {
MOD_INIT(aerospike)
{

const char version[8] = "2.0.13";
const char version[8] = "2.1.0";
// Makes things "thread-safe"
PyEval_InitThreads();
int i = 0;
Expand Down
9 changes: 8 additions & 1 deletion src/main/client/info.c
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ typedef struct foreach_callback_info_udata_t {
#define INET6_ADDRSTRLEN 46
#define INET_PORT 5
#define IP_PORT_SEPARATOR_LEN 1
#define IP_PORT_MAX_LEN INET6_ADDRSTRLEN + INET_PORT + IP_PORT_SEPARATOR_LEN
// Add 2 to the length to facilitate [] around the edges
#define IP_PORT_MAX_LEN INET6_ADDRSTRLEN + INET_PORT + IP_PORT_SEPARATOR_LEN + 2

/**
*******************************************************************************************************
Expand Down Expand Up @@ -153,6 +154,12 @@ static bool AerospikeClient_Info_each(as_error * err, const as_node * node, cons
if ( !strcmp(ip_port, addr->name) ) {
PyObject * py_nodes = (PyObject *) udata_ptr->udata_p;
PyDict_SetItemString(py_nodes, node->name, py_res);
} else {
sprintf(ip_port, "[%s]:%d", host_addr, port);
if ( !strcmp(ip_port, addr->name) ) {
PyObject * py_nodes = (PyObject *) udata_ptr->udata_p;
PyDict_SetItemString(py_nodes, node->name, py_res);
}
}
}

Expand Down
31 changes: 27 additions & 4 deletions src/main/client/type.c
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,13 @@
#include "policy.h"
#include "conversions.h"
#include "exceptions.h"

#include "tls_config.h"

enum {INIT_NO_CONFIG_ERR = 1, INIT_CONFIG_TYPE_ERR, INIT_LUA_USER_ERR,
INIT_LUA_SYS_ERR, INIT_HOST_TYPE_ERR, INIT_EMPTY_HOSTS_ERR,
INIT_INVALID_ADRR_ERR, INIT_SERIALIZE_ERR, INIT_DESERIALIZE_ERR,
INIT_COMPRESSION_ERR} ;

/*******************************************************************************
* PYTHON TYPE METHODS
******************************************************************************/
Expand Down Expand Up @@ -455,6 +456,11 @@ static int AerospikeClient_Type_Init(AerospikeClient * self, PyObject * args, Py
}
}

PyObject * py_tls = PyDict_GetItemString(py_config, "tls");
if (py_tls && PyDict_Check(py_tls)) {
setup_tls_config(&config, py_tls);
}

PyObject * py_hosts = PyDict_GetItemString(py_config, "hosts");
if (py_hosts && PyList_Check(py_hosts)) {
int size = (int) PyList_Size(py_hosts);
Expand All @@ -463,11 +469,12 @@ static int AerospikeClient_Type_Init(AerospikeClient * self, PyObject * args, Py
}
for (int i = 0; i < size; i++) {
char *addr = NULL;
char *tls_name = NULL;
uint16_t port = 3000;
PyObject * py_host = PyList_GetItem(py_hosts, i);
PyObject * py_addr, * py_port;
PyObject * py_addr, * py_port, *py_tls_name;

if (PyTuple_Check(py_host) && PyTuple_Size(py_host) == 2) {
if (PyTuple_Check(py_host) && PyTuple_Size(py_host) >= 2 && PyTuple_Size(py_host) <= 3 ) {

py_addr = PyTuple_GetItem(py_host, 0);
if (PyString_Check(py_addr)) {
Expand All @@ -484,6 +491,17 @@ static int AerospikeClient_Type_Init(AerospikeClient * self, PyObject * args, Py
else {
port = 0;
}
// Set TLS Name if provided
if (PyTuple_Size(py_host) == 3) {
py_tls_name = PyTuple_GetItem(py_host, 2);
if (PyString_Check(py_tls_name)) {
tls_name = strdup(PyString_AsString(py_tls_name));
} else if (PyUnicode_Check(py_tls_name)) {
PyObject* py_ustr = PyUnicode_AsUTF8String(py_tls_name);
tls_name = strdup(PyBytes_AsString(py_ustr));
Py_DECREF(py_ustr);
}
}
}
else if (PyString_Check(py_host)) {
addr = strdup( strtok( PyString_AsString(py_host), ":" ) );
Expand All @@ -494,7 +512,12 @@ static int AerospikeClient_Type_Init(AerospikeClient * self, PyObject * args, Py
}
}
if (addr) {
as_config_add_host(&config, addr, port);
if (tls_name) {
as_config_tls_add_host(&config, addr, tls_name, port);
free(tls_name);
} else {
as_config_add_host(&config, addr, port);
}
free(addr);
} else {
return INIT_INVALID_ADRR_ERR;
Expand Down
2 changes: 1 addition & 1 deletion src/main/conversions.c
Original file line number Diff line number Diff line change
Expand Up @@ -555,7 +555,7 @@ as_status pyobject_to_record(AerospikeClient * self, as_error * err, PyObject *
}

if (aerospike_has_geo(self->as)) {
ret_val = as_record_set_geojson_str(rec, name, geo_value);
ret_val = as_record_set_geojson_strp(rec, name, strdup(geo_value), true);
} else {
as_bytes *bytes;
GET_BYTES_POOL(bytes, static_pool, err);
Expand Down
187 changes: 187 additions & 0 deletions src/main/tls_config.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
/*******************************************************************************
* Copyright 2017 Aerospike, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
******************************************************************************/

#include "tls_config.h"

static void
_set_config_str_if_present(as_config* config, PyObject* tls_config,
const char* key);

static char*
get_string_from_string_like(PyObject* string_like);


/***
* Param: tls_conf PyDict.
* Fill in the appropriate TLS values of config based on the contents of
* tls_config
***/
void
setup_tls_config(as_config* config, PyObject* tls_config)
{
uint32_t max_val = 0xFFFFFFFF; // 2 ^ 32 -1
long long_timeout;
PyObject* config_value = NULL;
int truth_value = -1;

// Setup string values in the tls config struct
_set_config_str_if_present(config, tls_config, "cafile");
_set_config_str_if_present(config, tls_config, "capath");
_set_config_str_if_present(config, tls_config, "protocols");
_set_config_str_if_present(config, tls_config, "cipher_suite");
_set_config_str_if_present(config, tls_config, "cert_blacklist");
_set_config_str_if_present(config, tls_config, "keyfile");
_set_config_str_if_present(config, tls_config, "certfile");

// Setup The boolean values of the struct if they are present
config_value = PyDict_GetItemString(tls_config, "enable");
if (config_value) {
truth_value = PyObject_IsTrue(config_value);
if(truth_value != -1) {
config->tls.enable = (bool)truth_value;
truth_value = -1;
}
}

config_value = PyDict_GetItemString(tls_config, "encrypt_only");
if (config_value) {
truth_value = PyObject_IsTrue(config_value);
if(truth_value != -1) {
config->tls.encrypt_only = (bool)truth_value;
truth_value = -1;
}
}

config_value = PyDict_GetItemString(tls_config, "crl_check");
if (config_value) {
truth_value = PyObject_IsTrue(config_value);
if(truth_value != -1) {
config->tls.crl_check = (bool)truth_value;
truth_value = -1;
}
}

config_value = PyDict_GetItemString(tls_config, "crl_check_all");
if (config_value) {
truth_value = PyObject_IsTrue(config_value);
if(truth_value != -1) {
config->tls.crl_check_all = (bool)truth_value;
truth_value = -1;
}
}

config_value = PyDict_GetItemString(tls_config, "log_session_info");
if (config_value) {
truth_value = PyObject_IsTrue(config_value);
if(truth_value != -1) {
config->tls.log_session_info = (bool)truth_value;
truth_value = -1;
}
}

// Setup the uint32_t item if present and valid
config_value = PyDict_GetItemString(tls_config, "max_socket_idle");
if (config_value && PyInt_Check(config_value)) {
long_timeout = PyInt_AsLong(config_value);
if (long_timeout >= 0 && long_timeout <= max_val) {
config->tls.max_socket_idle = (uint32_t)long_timeout;
}
}

}

/***
* Param key: name of the tls field to be set
* Param tls_config: PyObject of a string type Unicode or String
* Param config: the as_config in which to store information
* If tls_config is a string type, and key is valid,
* the appropriate field is set
***/
static void
_set_config_str_if_present(as_config* config, PyObject* tls_config,
const char* key)
{
PyObject* config_value = NULL;
char* config_value_str = NULL;

config_value = PyDict_GetItemString(tls_config, key);
if (config_value) {
// Get the char* value of the python config dictionary's entry for key
config_value_str = get_string_from_string_like(config_value);
/* Depending on the key, call the correct setter function */
if (config_value_str) {
if (strcmp("cafile", key) == 0) {
as_config_tls_set_cafile(config,
(const char*)config_value_str);

}
else if (strcmp("capath", key) == 0) {
as_config_tls_set_capath(config,
(const char*)config_value_str);

}
else if (strcmp("protocols", key) == 0) {
as_config_tls_set_protocols(config,
(const char*)config_value_str);

}
else if (strcmp("cipher_suite", key) == 0) {
as_config_tls_set_cipher_suite(config,
(const char*)config_value_str);

}
else if (strcmp("cert_blacklist", key) == 0) {
as_config_tls_set_cert_blacklist(config,
(const char*)config_value_str);

}
else if (strcmp("keyfile", key) == 0) {
as_config_tls_set_keyfile(config,
(const char*)config_value_str);

}
else if (strcmp("certfile", key) == 0) {
as_config_tls_set_certfile(config,
(const char*)config_value_str);
}
}
}
}

/***
* Param string_like: PyObject
*
* If string_like is a String or Unicode,
* the char* representation of it's string value is returned,
* else null is returned
***/
static char*
get_string_from_string_like(PyObject* string_like)
{
char* ret_str = NULL;
PyObject* ustr = NULL;
if (PyString_Check(string_like)) {
ret_str = PyString_AsString(string_like);
}
else if (PyUnicode_Check(string_like)) {
ustr = PyUnicode_AsUTF8String(string_like);
if (ustr) {
ret_str = PyBytes_AsString(ustr);
}
}
return ret_str;
}
Loading

0 comments on commit 6d20dc9

Please sign in to comment.