From 20bdabca32e83297e4bee738e22e6007e2fe13fe Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Thu, 6 Sep 2018 13:52:27 -0700 Subject: [PATCH 01/52] propose a gRFC that creates a new TLS credential API. --- A19-tls-credential-API | 281 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 A19-tls-credential-API diff --git a/A19-tls-credential-API b/A19-tls-credential-API new file mode 100644 index 000000000..57cba4d45 --- /dev/null +++ b/A19-tls-credential-API @@ -0,0 +1,281 @@ +Title +---- +* Author(s): yihuazhang +* Approver: vjpai, markdroth +* Status: Draft +* Implemented in: +* Last updated: September 3, 2018 +* Discussion at: https://groups.google.com/forum/#!topic/grpc-io/gMrROsizrtc + +## Abstract +This proposal aims to incorporate SPIFFE-based mutual TLS into gRPC C core auth stack. Towards the goal, it proposes a new credential API model that 1) allows flexible addition of new TLS-related features (e.g., credential reload) without breaking the API and 2) supports both synchronous and asynchronous implementations of those features. It then proposes SPIFFE and HTTPS-based TLS credential API's that comply with the new model. + +## Background + +The current TLS implementation in gRPC C core has the following problems when it comes to support SPIFFE-based mutual TLS. + +1: It is originally designed to comply with HTTPS semantics which makes it difficult to be configured to support SPIFFE-based mutual TLS. For instance, the server authorization check in the current implementation is enforced locally by leveraging whitelisted endpoint information embedded in peer certificates while in SPIFFE use case, it may be enforced via an external server authorization check service. + +2: It realizes TLS-related features such as credential reload and server authorization check in a synchronous manner. It is undesirable in SPIFFE use case as those operations could involve expensive RPC calls to other services which may block gRPC core threads. + +3: The current TLS credential API is inflexible in incorporating new TLS-related features in future as not only does it break the API, it also requires all wrapped languages to update their implementations in a lockstep manner. + +This proposal introduces a new credential API model that allows flexible addition of TLS-related features that can be implemented in either a synchronous or asynchronous manner. It then proposes new credential API's for both HTTPS and SPIFFE-based TLS that complies with the API model. + +### Related Proposals: +N/A + +## Proposal + +The first part of proposal introduces credential options and credential create APIs that will be added to include/grpc/grpc_security.h. + +```c++ +/** Config for direct provision of TLS key materials. */ +typedef struct grpc_tls_key_materials_config grpc_tls_key_materials_config; + +/** Config for TLS credential reload. */ +typedef struct grpc_tls_credential_reload_config grpc_tls_credential_reload_config; + +/** Config for TLS server authorization check. */ +typedef struct grpc_tls_server_authorization_check +grpc_tls_server_authorization_check; + +/** Config for TLS context customization. */ +typedef struct grpc_tls_ctx_customize_config grpc_tls_ctx_customize_config; + +/** TLS credentials options. */ +typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; + +/** Create an empty TLS credentials options. */ +GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(); + +/** Set grpc_ssl_client_certificate_request_type field in credentials options + with the provided type. */ +GRPCAPI void grpc_tls_credentials_options_set_cert_request_type( + grpc_tls_credentials_options* options, + grpc_ssl_client_certificate_request_type type); + +/** Set grpc_tls_credential_reload_config field in credentials options + with the provided config struct whose ownership is transferred. */ +GRPCAPI void grpc_tls_credentials_options_set_credential_reload_config( + grpc_tls_credentials_options* options, + grpc_tls_credential_reload_config* config); + + +/** Set grpc_tls_ctx_customize_config field in credentials options + with the provided config struct whose ownership is transferred. */ +GRPCAPI void grpc_tls_credentials_options_set_tls_ctx_customize_config( + grpc_tls_credentials_options* options, + grpc_tls_ctx_customize_config* config); + +/** Set grpc_tls_server_authorization_check_config field in credentials options + with the provided config struct whose ownership is transferred. */ +GRPCAPI void grpc_tls_credentials_options_set_server_authorization_check_config( + grpc_tls_credentials_options* options, + grpc_tls_server_authorization_check_config* config); + +/** Destroy a TLS credentials options. */ +GRPCAPI void grpc_tls_credentials_options_destroy( + grpc_tls_credentials_options* options); + +/** Create an HTTPS-based TLS server credential object using the provided + options struct whose ownership is transferred. */ +GRPCAPI grpc_ssl_server_credentials* grpc_tls_https_server_credentials_create( + const grpc_tls_credentials_options* options); + +/** Create an HTTPS-based TLS channel credential object using the provided + options struct whose ownership is transferred. */ +GRPCAPI grpc_ssl_credentials* grpc_tls_https_credentials_create( + const grpc_tls_credentials_options* options); + +/** Create a SPIFFE-based TLS server credential object using the provided + options struct whose ownership is transferred. */ +GRPCAPI grpc_ssl_server_credentials* grpc_tls_spiffe_server_credentials_create( + const grpc_tls_credentials_options* options); + +/** Create a SPIFFE-based TLS channel credential object using the provided + options struct whose ownership is transferred. */ +GRPCAPI grpc_ssl_credentials* grpc_tls_spiffe_credentials_create( + const grpc_tls_credentials_options* options); +``` + +The second part of proposal introduces API's related to configs of TLS-related features that will be added to include/grpc/grpc_security.h. + +```c++ +/** --- TLS key materials config. --- */ + +/** Create an empty grpc_tls_key_materials_config instance. */ +GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create(); + +/** Add a set of root certficiates, private key and certificate chain to a config + instance. + - config is an instance of grpc_tls_key_materials_config struct. + - pem_root_certs is the NULL-terminated string containing the PEM encoding of + root certificates. If it is NULL, a default system root certificate or the + one shipped with gRPC package will be used. + - private_key is the NULL-terminated string containing the PEM encoding + of private key. + - cert_chain is the NULL-terminated string containing the PEM encoding of + certificate chain. +*/ +GRPCAPI void grpc_tls_key_materials_config_add( + grpc_tls_key_materials_config* config, + const char* pem_root_certs, + const char* private_key, + const char* cert_chain); + +/** Destroy a grpc_tls_key_materials_config instance. */ +GRPCAPI void grpc_tls_key_materials_config_destroy( + grpc_tls_key_materials_config* config); + +/** --- TLS credential reload config. --- */ + +/** A callback function provided by gRPC to handle the result of credential reload. + It is used when schedule API is implemented asynchronously, and serves to + bring the control back to grpc C core. */ +typedef void (*grpc_tls_on_credential_reload_done_cb)(void* user_data); + +/** A struct containing all information necessary to schedule/cancel + a credential reload request. cb and cb_user_data represent a gRPC-provided + callback and an argument passed to it. key_materials is an in/output parameter + containing currently used/newly reloaded credentials. status and error_details + are used to hold information about errors occurred when a credential reload request + is scheduled/cancelled. */ +typedef struct { + grpc_tls_on_credential_reload_done_cb cb; + void *cb_user_data; + grpc_tls_key_materials *key_materials; + grpc_status_code *status; + const char **error_details; +} grpc_tls_credential_reload_arg; + +/** Create a grpc_tls_credential_reload_config instance. + - config_user_data is config-specific, read-only user data + that works for all channels created with a credential using the config. + - schedule is a pointer to an application-provided callback used to invoke + credential reload API. The implementation of this method has to be + non-blocking, but can be performed synchronously or asynchronously. + 1) If processing occurs synchronously, it populates arg->key_materials, + arg->status, and arg->error_details and returns zero. + 2) If processing occurs asynchronously, it returns a non-zero value. + The application then invokes arg->cb when processing is completed. Note that + arg->cb cannot be invoked before schedule API returns. + - cancel is a pointer to an application-provided callback used to cancel + a credential reload request scheduled via an asynchronous schedule API. + arg is used to pinpoint an exact reloading request to be cancelled. + The operation may not have any effect if the request has already been + processed. + - destruct is a pointer to an application-provided callback used to clean up + any data associated with the config. +*/ +GRPCAPI grpc_tls_credential_reload_config* + grpc_tls_credential_reload_config_create( + const void* config_user_data, + int (*schedule)(const void* config_user_data, + grpc_tls_credential_reload_arg* arg), + void (*cancel)(const void* config_user_data, + grpc_tls_credential_reload_arg* arg), + void (*destruct)(const void* config_user_data)); + + +/** Destroy a grpc_tls_credential_reload_config instance. */ +GRPCAPI void grpc_tls_credential_reload_config_destroy( + grpc_tls_credential_reload_config* config); + +/** --- TLS context customization config. --- */ + +/** A struct containing all information necessary to customize a TLS context. + context represents the underlying TLS context whose ownership is not transferred. + status and error_details are used to hold information about errors occurred + when scheduling a TLS context customize request. +*/ +typedef struct { + void* context; + grpc_status_code* status; + const char** error_details; +} grpc_tls_ctx_customize_arg; + +/** Create a grpc_tls_ctx_customize_config instance. + - config_user_data is config-specific, read-only user data + that works for all channels created with a credential using the config. + - schedule is a pointer to an application-provided callback used to invoke + TLS context customization API. This method should be implemented synchronously. + - destruct is a pointer to an application-provided callback used to clean up + any data associated with the config. +*/ +GRPCAPI grpc_tls_ctx_customize_config* grpc_tls_ctx_customize_config_create( + const void* config_user_data, + void (*schedule)(const void* config_user_data, + grpc_tls_ctx_customize_arg* arg), + void (*destruct)(const void* config_user_data)); + +/** Destroy a grpc_tls_ctx_customize_config instance. */ +GRPCAPI void grpc_tls_ctx_customize_config_destroy(grpc_tls_ctx_customize_config* config); + +/** --- TLS server authorization check config. --- */ + +/** callback function provided by gRPC used to handle the result of server authorization + check. It is used when schedule API is implemented asynchronously, and + serves to bring the control back to gRPC C core. */ +typedef void (*grpc_tls_on_server_authorization_check_done_cb) (void* user_data); + +/** A struct containing all information necessary to schedule/cancel a server + authorization check request. cb and cb_user_data represent a gRPC-provided callback and + an argument passed to it. result will store the result of server authorization + check. target_name is the name of an endpoint the channel is connecting to and + certificate represents a complete certificate chain including both signing and + leaf certificates. status and error_details contain information about errors + occurred when a server authorization check request is scheduled/cancelled. */ +typedef struct { + grpc_tls_on_server_authorization_check_done_cb cb; + void *cb_user_data; + bool *result; + const char *target_name; + const char *certificate; + grpc_status_code *status; + const char **error_details; +} grpc_tls_server_authorization_check_arg; + +/** Create a grpc_tls_server_authorization_check_config instance. + - config_user_data is config-specific, read-only user data + that works for all channels created with a credential using the config. + - schedule is a pointer to an application-provided callback used to invoke server + authorization check API. The implementation of this method has to be non-blocking, + but can be performed synchronously or asynchronously. + 1) If processing occurs synchronously, it populates arg->result, + arg->status, and arg->error_details and returns zero. + 2) If processing occurs asynchronously, it returns a non-zero value. + The application then invokes arg->cb when processing is completed. Note that + arg->cb cannot be invoked before schedule API returns. + - cancel is a pointer to an application-provided callback used to cancel a server + authorization check request scheduled via an asynchronous schedule API. + arg is used to pinpoint an exact check request to be cancelled. + The operation may not have any effect if the request has already been processed. + - destruct is a pointer to an application-provided callback used to clean up any data + associated with the config. +*/ +GRPCAPI grpc_tls_server_authorization_check_config* + grpc_tls_server_authorization_check_config_create( + const void* config_user_data, + int (*schedule)(const void* config_user_data, + grpc_tls_server_authorization_check_arg* arg), + void (*cancel)(const void* config_user_data, + grpc_tls_server_authorization_check_arg* arg), + void (*destruct)(const void* config_user_data)); + +/** Destroy a grpc_tls_server_authorization_check_config instance. */ +GRPCAPI void grpc_tls_server_authorization_check_config_destroy(grpc_tls_server_authorization_check_config* config); +``` + +## Rationale + +Regarding grpc_tls_ctx_customize_config, the config is used when a caller wants to +configure an underlying TLS context with customized primitives. For example, there could +be a use case in which raw private keys are not directly accessible (e.g., hardware-backed) to a caller, +but private key methods (e.g., “Sign” functions) are, in which case the config provides a means +to customize an TLS context with an SSL_PRIVATE_KEY_METHOD object and associated signing algorithms. + +## Implementation + +The implementation of this proposal is straightforward as all API changes will be +made to include/grpc/grpc_security.h. From 28253466e402ae03f5b0900bf250c55100d55250 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Tue, 11 Sep 2018 13:50:01 -0700 Subject: [PATCH 02/52] add more explanation on server authorization check. --- A19-tls-credential-API | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/A19-tls-credential-API b/A19-tls-credential-API index 57cba4d45..c99fadc33 100644 --- a/A19-tls-credential-API +++ b/A19-tls-credential-API @@ -14,7 +14,7 @@ This proposal aims to incorporate SPIFFE-based mutual TLS into gRPC C core auth The current TLS implementation in gRPC C core has the following problems when it comes to support SPIFFE-based mutual TLS. -1: It is originally designed to comply with HTTPS semantics which makes it difficult to be configured to support SPIFFE-based mutual TLS. For instance, the server authorization check in the current implementation is enforced locally by leveraging whitelisted endpoint information embedded in peer certificates while in SPIFFE use case, it may be enforced via an external server authorization check service. +1: It is originally designed to comply with HTTPS semantics which makes it difficult to be configured to support SPIFFE-based mutual TLS. For instance, the server authorization check in the current implementation is enforced locally by leveraging whitelisted endpoint information embedded in peer certificates while in SPIFFE use case, it may be enforced via an external server authorization check service. Here, the server authorization check is used to validate if a peer is authorized to run a job with a specific target name, and is invoked only at the client-side. 2: It realizes TLS-related features such as credential reload and server authorization check in a synchronous manner. It is undesirable in SPIFFE use case as those operations could involve expensive RPC calls to other services which may block gRPC core threads. @@ -27,7 +27,7 @@ N/A ## Proposal -The first part of proposal introduces credential options and credential create APIs that will be added to include/grpc/grpc_security.h. +The first part of proposal introduces credential options and credential create APIs that will be added to include/grpc/grpc_security.h. Note that although the credentials options provided to HTTPS and SPIFFE-based TLS credential API have the same format (i.e., grpc_tls_credentials_options), some of its members will not get populated and hence used in some API's, for instance, grpc_tls_server_authorization_check_config will be ignored in server-side API's. ```c++ /** Config for direct provision of TLS key materials. */ @@ -37,8 +37,7 @@ typedef struct grpc_tls_key_materials_config grpc_tls_key_materials_config; typedef struct grpc_tls_credential_reload_config grpc_tls_credential_reload_config; /** Config for TLS server authorization check. */ -typedef struct grpc_tls_server_authorization_check -grpc_tls_server_authorization_check; +typedef struct grpc_tls_server_authorization_check_config grpc_tls_server_authorization_check_config; /** Config for TLS context customization. */ typedef struct grpc_tls_ctx_customize_config grpc_tls_ctx_customize_config; @@ -277,5 +276,4 @@ to customize an TLS context with an SSL_PRIVATE_KEY_METHOD object and associated ## Implementation -The implementation of this proposal is straightforward as all API changes will be -made to include/grpc/grpc_security.h. +The implementation of this proposal is straightforward as all API changes will be made to include/grpc/grpc_security.h. Note that once the new HTTPS-based TLS credential API gets implemented, the old HTTPS-based TLS credential API will get deprecated and be discouraged to use. From 9c4e64ff8e536b0354ddea0f80615cf095c94466 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Wed, 12 Sep 2018 10:24:02 -0700 Subject: [PATCH 03/52] fixed some errors --- A19-tls-credential-API | 62 +++++++++++++++++++++++------------------- 1 file changed, 34 insertions(+), 28 deletions(-) diff --git a/A19-tls-credential-API b/A19-tls-credential-API index c99fadc33..d0f29371d 100644 --- a/A19-tls-credential-API +++ b/A19-tls-credential-API @@ -79,22 +79,22 @@ GRPCAPI void grpc_tls_credentials_options_destroy( /** Create an HTTPS-based TLS server credential object using the provided options struct whose ownership is transferred. */ -GRPCAPI grpc_ssl_server_credentials* grpc_tls_https_server_credentials_create( +GRPCAPI grpc_server_credentials* grpc_tls_https_server_credentials_create( const grpc_tls_credentials_options* options); /** Create an HTTPS-based TLS channel credential object using the provided options struct whose ownership is transferred. */ -GRPCAPI grpc_ssl_credentials* grpc_tls_https_credentials_create( +GRPCAPI grpc_channel_credentials* grpc_tls_https_credentials_create( const grpc_tls_credentials_options* options); /** Create a SPIFFE-based TLS server credential object using the provided options struct whose ownership is transferred. */ -GRPCAPI grpc_ssl_server_credentials* grpc_tls_spiffe_server_credentials_create( +GRPCAPI grpc_server_credentials* grpc_tls_spiffe_server_credentials_create( const grpc_tls_credentials_options* options); /** Create a SPIFFE-based TLS channel credential object using the provided options struct whose ownership is transferred. */ -GRPCAPI grpc_ssl_credentials* grpc_tls_spiffe_credentials_create( +GRPCAPI grpc_channel_credentials* grpc_tls_spiffe_credentials_create( const grpc_tls_credentials_options* options); ``` @@ -103,25 +103,24 @@ The second part of proposal introduces API's related to configs of TLS-related f ```c++ /** --- TLS key materials config. --- */ +/** A struct containing TLS credential key materials in a PEM format. + * pem_root_certs is the NULL-terminated string containing the PEM encoding of + * root certificates. If it is NULL, a default system root certificate or the + * one shipped with gRPC package will be used. + */ +typedef struct grpc_tls_key_materials { + grpc_ssl_pem_key_cert_pair* pem_key_cert_pair; + const char* pem_root_certs; +} grpc_tls_key_materials; + /** Create an empty grpc_tls_key_materials_config instance. */ GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create(); -/** Add a set of root certficiates, private key and certificate chain to a config - instance. - - config is an instance of grpc_tls_key_materials_config struct. - - pem_root_certs is the NULL-terminated string containing the PEM encoding of - root certificates. If it is NULL, a default system root certificate or the - one shipped with gRPC package will be used. - - private_key is the NULL-terminated string containing the PEM encoding - of private key. - - cert_chain is the NULL-terminated string containing the PEM encoding of - certificate chain. -*/ +/** Add a grpc_tls_key_materials instance to a grpc_tls_key_materials_config + * instance. */ GRPCAPI void grpc_tls_key_materials_config_add( - grpc_tls_key_materials_config* config, - const char* pem_root_certs, - const char* private_key, - const char* cert_chain); + grpc_tls_key_materials_config* config, + grpc_tls_key_materials* key_materials); /** Destroy a grpc_tls_key_materials_config instance. */ GRPCAPI void grpc_tls_key_materials_config_destroy( @@ -129,10 +128,13 @@ GRPCAPI void grpc_tls_key_materials_config_destroy( /** --- TLS credential reload config. --- */ +typedef struct grpc_tls_credential_reload_arg grpc_tls_credential_reload_arg; + /** A callback function provided by gRPC to handle the result of credential reload. It is used when schedule API is implemented asynchronously, and serves to bring the control back to grpc C core. */ -typedef void (*grpc_tls_on_credential_reload_done_cb)(void* user_data); +typedef void (*grpc_tls_on_credential_reload_done_cb)( + grpc_tls_credential_reload_arg* arg); /** A struct containing all information necessary to schedule/cancel a credential reload request. cb and cb_user_data represent a gRPC-provided @@ -140,13 +142,13 @@ typedef void (*grpc_tls_on_credential_reload_done_cb)(void* user_data); containing currently used/newly reloaded credentials. status and error_details are used to hold information about errors occurred when a credential reload request is scheduled/cancelled. */ -typedef struct { +struct grpc_tls_credential_reload_arg { grpc_tls_on_credential_reload_done_cb cb; void *cb_user_data; grpc_tls_key_materials *key_materials; grpc_status_code *status; const char **error_details; -} grpc_tls_credential_reload_arg; +}; /** Create a grpc_tls_credential_reload_config instance. - config_user_data is config-specific, read-only user data @@ -213,10 +215,14 @@ GRPCAPI void grpc_tls_ctx_customize_config_destroy(grpc_tls_ctx_customize_config /** --- TLS server authorization check config. --- */ -/** callback function provided by gRPC used to handle the result of server authorization - check. It is used when schedule API is implemented asynchronously, and - serves to bring the control back to gRPC C core. */ -typedef void (*grpc_tls_on_server_authorization_check_done_cb) (void* user_data); +typedef struct grpc_tls_server_authorization_check_arg + grpc_tls_server_authorization_check_arg; + +/** callback function provided by gRPC used to handle the result of server + authorization check. It is used when schedule API is implemented + asynchronously, and serves to bring the control back to gRPC C core. */ +typedef void (*grpc_tls_on_server_authorization_check_done_cb)( + grpc_tls_server_authorization_check_arg* arg); /** A struct containing all information necessary to schedule/cancel a server authorization check request. cb and cb_user_data represent a gRPC-provided callback and @@ -225,7 +231,7 @@ typedef void (*grpc_tls_on_server_authorization_check_done_cb) (void* user_data) certificate represents a complete certificate chain including both signing and leaf certificates. status and error_details contain information about errors occurred when a server authorization check request is scheduled/cancelled. */ -typedef struct { +struct grpc_tls_server_authorization_check_arg { grpc_tls_on_server_authorization_check_done_cb cb; void *cb_user_data; bool *result; @@ -233,7 +239,7 @@ typedef struct { const char *certificate; grpc_status_code *status; const char **error_details; -} grpc_tls_server_authorization_check_arg; +}; /** Create a grpc_tls_server_authorization_check_config instance. - config_user_data is config-specific, read-only user data From b15f234900c2f6247a5c305d37f5ee1f8bfcdd79 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Wed, 12 Sep 2018 16:23:34 -0700 Subject: [PATCH 04/52] add correct suffix --- A19-tls-credential-API => A19-tls-credential-API.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename A19-tls-credential-API => A19-tls-credential-API.md (100%) diff --git a/A19-tls-credential-API b/A19-tls-credential-API.md similarity index 100% rename from A19-tls-credential-API rename to A19-tls-credential-API.md From 2cc5fac749686db20ba4d0584ec5b05c1a42af82 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Fri, 14 Sep 2018 11:50:25 -0700 Subject: [PATCH 05/52] minor changes to API's --- A19-tls-credential-API.md | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/A19-tls-credential-API.md b/A19-tls-credential-API.md index d0f29371d..dc64c7aad 100644 --- a/A19-tls-credential-API.md +++ b/A19-tls-credential-API.md @@ -54,6 +54,12 @@ GRPCAPI void grpc_tls_credentials_options_set_cert_request_type( grpc_tls_credentials_options* options, grpc_ssl_client_certificate_request_type type); +/** Set grpc_tls_key_materials_config field in credentials options + with the provided config struct whose ownership is transferred. */ +GRPCAPI void grpc_tls_credentials_options_set_key_materials_config( + grpc_tls_credentials_options* options, + grpc_tls_key_materials_config* config); + /** Set grpc_tls_credential_reload_config field in credentials options with the provided config struct whose ownership is transferred. */ GRPCAPI void grpc_tls_credentials_options_set_credential_reload_config( @@ -113,14 +119,10 @@ typedef struct grpc_tls_key_materials { const char* pem_root_certs; } grpc_tls_key_materials; -/** Create an empty grpc_tls_key_materials_config instance. */ -GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create(); - -/** Add a grpc_tls_key_materials instance to a grpc_tls_key_materials_config - * instance. */ -GRPCAPI void grpc_tls_key_materials_config_add( - grpc_tls_key_materials_config* config, - grpc_tls_key_materials* key_materials); +/** Create a grpc_tls_key_materials_config instance. */ +GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create( + grpc_ssl_pem_key_cert_pair* key_cert_pair_set, const char** root_certs_set, + size_t num); /** Destroy a grpc_tls_key_materials_config instance. */ GRPCAPI void grpc_tls_key_materials_config_destroy( @@ -172,11 +174,11 @@ struct grpc_tls_credential_reload_arg { GRPCAPI grpc_tls_credential_reload_config* grpc_tls_credential_reload_config_create( const void* config_user_data, - int (*schedule)(const void* config_user_data, + int (*schedule)(void* config_user_data, grpc_tls_credential_reload_arg* arg), - void (*cancel)(const void* config_user_data, + void (*cancel)(void* config_user_data, grpc_tls_credential_reload_arg* arg), - void (*destruct)(const void* config_user_data)); + void (*destruct)(void* config_user_data)); /** Destroy a grpc_tls_credential_reload_config instance. */ @@ -206,9 +208,9 @@ typedef struct { */ GRPCAPI grpc_tls_ctx_customize_config* grpc_tls_ctx_customize_config_create( const void* config_user_data, - void (*schedule)(const void* config_user_data, + void (*schedule)(void* config_user_data, grpc_tls_ctx_customize_arg* arg), - void (*destruct)(const void* config_user_data)); + void (*destruct)(void* config_user_data)); /** Destroy a grpc_tls_ctx_customize_config instance. */ GRPCAPI void grpc_tls_ctx_customize_config_destroy(grpc_tls_ctx_customize_config* config); @@ -262,11 +264,11 @@ struct grpc_tls_server_authorization_check_arg { GRPCAPI grpc_tls_server_authorization_check_config* grpc_tls_server_authorization_check_config_create( const void* config_user_data, - int (*schedule)(const void* config_user_data, + int (*schedule)(void* config_user_data, grpc_tls_server_authorization_check_arg* arg), - void (*cancel)(const void* config_user_data, + void (*cancel)(void* config_user_data, grpc_tls_server_authorization_check_arg* arg), - void (*destruct)(const void* config_user_data)); + void (*destruct)(void* config_user_data)); /** Destroy a grpc_tls_server_authorization_check_config instance. */ GRPCAPI void grpc_tls_server_authorization_check_config_destroy(grpc_tls_server_authorization_check_config* config); From 0091a1a7bccc82d38655c55f21552994897761b9 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Fri, 14 Sep 2018 15:17:12 -0700 Subject: [PATCH 06/52] updated grpc_tls_key_materials_config struct --- A19-tls-credential-API.md | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/A19-tls-credential-API.md b/A19-tls-credential-API.md index dc64c7aad..8f34f3f02 100644 --- a/A19-tls-credential-API.md +++ b/A19-tls-credential-API.md @@ -109,19 +109,14 @@ The second part of proposal introduces API's related to configs of TLS-related f ```c++ /** --- TLS key materials config. --- */ -/** A struct containing TLS credential key materials in a PEM format. - * pem_root_certs is the NULL-terminated string containing the PEM encoding of - * root certificates. If it is NULL, a default system root certificate or the - * one shipped with gRPC package will be used. +/** Create an empty grpc_tls_key_materials_config instance. */ +GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create(); + +/** Set grpc_tls_key_materials_config instance with a provided TLS certificate. */ -typedef struct grpc_tls_key_materials { - grpc_ssl_pem_key_cert_pair* pem_key_cert_pair; - const char* pem_root_certs; -} grpc_tls_key_materials; - -/** Create a grpc_tls_key_materials_config instance. */ -GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create( - grpc_ssl_pem_key_cert_pair* key_cert_pair_set, const char** root_certs_set, +GRPCAPI void grpc_tls_key_materials_config_set_certificate( + grpc_tls_key_materials_config* config, + grpc_ssl_pem_key_cert_pair* pem_key_cert_pairs, const char* pem_root_certs, size_t num); /** Destroy a grpc_tls_key_materials_config instance. */ @@ -147,7 +142,7 @@ typedef void (*grpc_tls_on_credential_reload_done_cb)( struct grpc_tls_credential_reload_arg { grpc_tls_on_credential_reload_done_cb cb; void *cb_user_data; - grpc_tls_key_materials *key_materials; + grpc_tls_key_materials_config *key_materials_config; grpc_status_code *status; const char **error_details; }; From ab946247edf8643fb8a87d99344f3f338386d693 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Wed, 3 Oct 2018 15:11:23 -0700 Subject: [PATCH 07/52] remove TLS context customization from proposal --- A19-tls-credential-API.md | 47 --------------------------------------- 1 file changed, 47 deletions(-) diff --git a/A19-tls-credential-API.md b/A19-tls-credential-API.md index 8f34f3f02..dd1a23610 100644 --- a/A19-tls-credential-API.md +++ b/A19-tls-credential-API.md @@ -39,9 +39,6 @@ typedef struct grpc_tls_credential_reload_config grpc_tls_credential_reload_conf /** Config for TLS server authorization check. */ typedef struct grpc_tls_server_authorization_check_config grpc_tls_server_authorization_check_config; -/** Config for TLS context customization. */ -typedef struct grpc_tls_ctx_customize_config grpc_tls_ctx_customize_config; - /** TLS credentials options. */ typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; @@ -67,12 +64,6 @@ GRPCAPI void grpc_tls_credentials_options_set_credential_reload_config( grpc_tls_credential_reload_config* config); -/** Set grpc_tls_ctx_customize_config field in credentials options - with the provided config struct whose ownership is transferred. */ -GRPCAPI void grpc_tls_credentials_options_set_tls_ctx_customize_config( - grpc_tls_credentials_options* options, - grpc_tls_ctx_customize_config* config); - /** Set grpc_tls_server_authorization_check_config field in credentials options with the provided config struct whose ownership is transferred. */ GRPCAPI void grpc_tls_credentials_options_set_server_authorization_check_config( @@ -180,36 +171,6 @@ GRPCAPI grpc_tls_credential_reload_config* GRPCAPI void grpc_tls_credential_reload_config_destroy( grpc_tls_credential_reload_config* config); -/** --- TLS context customization config. --- */ - -/** A struct containing all information necessary to customize a TLS context. - context represents the underlying TLS context whose ownership is not transferred. - status and error_details are used to hold information about errors occurred - when scheduling a TLS context customize request. -*/ -typedef struct { - void* context; - grpc_status_code* status; - const char** error_details; -} grpc_tls_ctx_customize_arg; - -/** Create a grpc_tls_ctx_customize_config instance. - - config_user_data is config-specific, read-only user data - that works for all channels created with a credential using the config. - - schedule is a pointer to an application-provided callback used to invoke - TLS context customization API. This method should be implemented synchronously. - - destruct is a pointer to an application-provided callback used to clean up - any data associated with the config. -*/ -GRPCAPI grpc_tls_ctx_customize_config* grpc_tls_ctx_customize_config_create( - const void* config_user_data, - void (*schedule)(void* config_user_data, - grpc_tls_ctx_customize_arg* arg), - void (*destruct)(void* config_user_data)); - -/** Destroy a grpc_tls_ctx_customize_config instance. */ -GRPCAPI void grpc_tls_ctx_customize_config_destroy(grpc_tls_ctx_customize_config* config); - /** --- TLS server authorization check config. --- */ typedef struct grpc_tls_server_authorization_check_arg @@ -269,14 +230,6 @@ GRPCAPI grpc_tls_server_authorization_check_config* GRPCAPI void grpc_tls_server_authorization_check_config_destroy(grpc_tls_server_authorization_check_config* config); ``` -## Rationale - -Regarding grpc_tls_ctx_customize_config, the config is used when a caller wants to -configure an underlying TLS context with customized primitives. For example, there could -be a use case in which raw private keys are not directly accessible (e.g., hardware-backed) to a caller, -but private key methods (e.g., “Sign” functions) are, in which case the config provides a means -to customize an TLS context with an SSL_PRIVATE_KEY_METHOD object and associated signing algorithms. - ## Implementation The implementation of this proposal is straightforward as all API changes will be made to include/grpc/grpc_security.h. Note that once the new HTTPS-based TLS credential API gets implemented, the old HTTPS-based TLS credential API will get deprecated and be discouraged to use. From 4b53d8edfc828715d0ae3f4900c2981c0977cbc4 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Mon, 29 Oct 2018 09:12:05 -0700 Subject: [PATCH 08/52] Change to --- A19-tls-credential-API.md => L42-tls-credential-API.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename A19-tls-credential-API.md => L42-tls-credential-API.md (100%) diff --git a/A19-tls-credential-API.md b/L42-tls-credential-API.md similarity index 100% rename from A19-tls-credential-API.md rename to L42-tls-credential-API.md From b4e56022358ecf5cb4639a7e71e15bb58a3e54aa Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Mon, 29 Oct 2018 09:54:06 -0700 Subject: [PATCH 09/52] rename the proposal with L --- L42-tls-credential-API.md => L42-core-tls-credential-API.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename L42-tls-credential-API.md => L42-core-tls-credential-API.md (100%) diff --git a/L42-tls-credential-API.md b/L42-core-tls-credential-API.md similarity index 100% rename from L42-tls-credential-API.md rename to L42-core-tls-credential-API.md From 5de6a09eb83d3d318162163cdb36bff782016e13 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Mon, 28 Jan 2019 09:52:42 -0800 Subject: [PATCH 10/52] Updated L# and synced with #17549 --- ...l-API.md => L46-core-tls-credential-API.md | 181 +++++++++--------- 1 file changed, 91 insertions(+), 90 deletions(-) rename L42-core-tls-credential-API.md => L46-core-tls-credential-API.md (62%) diff --git a/L42-core-tls-credential-API.md b/L46-core-tls-credential-API.md similarity index 62% rename from L42-core-tls-credential-API.md rename to L46-core-tls-credential-API.md index dd1a23610..5cac18827 100644 --- a/L42-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -30,14 +30,17 @@ N/A The first part of proposal introduces credential options and credential create APIs that will be added to include/grpc/grpc_security.h. Note that although the credentials options provided to HTTPS and SPIFFE-based TLS credential API have the same format (i.e., grpc_tls_credentials_options), some of its members will not get populated and hence used in some API's, for instance, grpc_tls_server_authorization_check_config will be ignored in server-side API's. ```c++ -/** Config for direct provision of TLS key materials. */ + +/** Config for TLS key materials. */ typedef struct grpc_tls_key_materials_config grpc_tls_key_materials_config; /** Config for TLS credential reload. */ -typedef struct grpc_tls_credential_reload_config grpc_tls_credential_reload_config; +typedef struct grpc_tls_credential_reload_config + grpc_tls_credential_reload_config; /** Config for TLS server authorization check. */ -typedef struct grpc_tls_server_authorization_check_config grpc_tls_server_authorization_check_config; +typedef struct grpc_tls_server_authorization_check_config + grpc_tls_server_authorization_check_config; /** TLS credentials options. */ typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; @@ -46,29 +49,35 @@ typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(); /** Set grpc_ssl_client_certificate_request_type field in credentials options - with the provided type. */ -GRPCAPI void grpc_tls_credentials_options_set_cert_request_type( - grpc_tls_credentials_options* options, - grpc_ssl_client_certificate_request_type type); + with the provided type. options should not be NULL. + It returns 1 on success and 0 on failure. */ +GRPCAPI int grpc_tls_credentials_options_set_cert_request_type( + grpc_tls_credentials_options* options, + grpc_ssl_client_certificate_request_type type); /** Set grpc_tls_key_materials_config field in credentials options - with the provided config struct whose ownership is transferred. */ -GRPCAPI void grpc_tls_credentials_options_set_key_materials_config( + with the provided config struct whose ownership is transferred. + Both parameters should not be NULL. + It returns 1 on success and 0 on failure. */ +GRPCAPI int grpc_tls_credentials_options_set_key_materials_config( grpc_tls_credentials_options* options, grpc_tls_key_materials_config* config); /** Set grpc_tls_credential_reload_config field in credentials options - with the provided config struct whose ownership is transferred. */ -GRPCAPI void grpc_tls_credentials_options_set_credential_reload_config( - grpc_tls_credentials_options* options, - grpc_tls_credential_reload_config* config); - + with the provided config struct whose ownership is transferred. + Both parameters should not be NULL. + It returns 1 on success and 0 on failure. */ +GRPCAPI int grpc_tls_credentials_options_set_credential_reload_config( + grpc_tls_credentials_options* options, + grpc_tls_credential_reload_config* config); /** Set grpc_tls_server_authorization_check_config field in credentials options - with the provided config struct whose ownership is transferred. */ -GRPCAPI void grpc_tls_credentials_options_set_server_authorization_check_config( - grpc_tls_credentials_options* options, - grpc_tls_server_authorization_check_config* config); + with the provided config struct whose ownership is transferred. + Both parameters should not be NULL. + It returns 1 on success and 0 on failure. */ +GRPCAPI int grpc_tls_credentials_options_set_server_authorization_check_config( + grpc_tls_credentials_options* options, + grpc_tls_server_authorization_check_config* config); /** Destroy a TLS credentials options. */ GRPCAPI void grpc_tls_credentials_options_destroy( @@ -103,39 +112,39 @@ The second part of proposal introduces API's related to configs of TLS-related f /** Create an empty grpc_tls_key_materials_config instance. */ GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create(); -/** Set grpc_tls_key_materials_config instance with a provided TLS certificate. +/** Set grpc_tls_key_materials_config instance with provided a TLS certificate. + config will take the ownership of pem_root_certs and pem_key_cert_pairs. + It's valid for the caller to provide nullptr pem_root_certs, in which case + the gRPC-provided root cert will be used. pem_key_cert_pairs should not be + NULL. It returns 1 on success and 0 on failure. */ -GRPCAPI void grpc_tls_key_materials_config_set_certificate( - grpc_tls_key_materials_config* config, - grpc_ssl_pem_key_cert_pair* pem_key_cert_pairs, const char* pem_root_certs, - size_t num); - -/** Destroy a grpc_tls_key_materials_config instance. */ -GRPCAPI void grpc_tls_key_materials_config_destroy( - grpc_tls_key_materials_config* config); +GRPCAPI int grpc_tls_key_materials_config_set_key_materials( + grpc_tls_key_materials_config* config, const char* pem_root_certs, + const grpc_ssl_pem_key_cert_pair** pem_key_cert_pairs, + size_t num_key_cert_pairs); /** --- TLS credential reload config. --- */ typedef struct grpc_tls_credential_reload_arg grpc_tls_credential_reload_arg; -/** A callback function provided by gRPC to handle the result of credential reload. - It is used when schedule API is implemented asynchronously, and serves to - bring the control back to grpc C core. */ +/** A callback function provided by gRPC to handle the result of credential + reload. It is used when schedule API is implemented asynchronously and + serves to bring the control back to grpc C core. */ typedef void (*grpc_tls_on_credential_reload_done_cb)( grpc_tls_credential_reload_arg* arg); /** A struct containing all information necessary to schedule/cancel a credential reload request. cb and cb_user_data represent a gRPC-provided - callback and an argument passed to it. key_materials is an in/output parameter - containing currently used/newly reloaded credentials. status and error_details - are used to hold information about errors occurred when a credential reload request - is scheduled/cancelled. */ + callback and an argument passed to it. key_materials is an in/output + parameter containing currently used/newly reloaded credentials. status and + error_details are used to hold information about errors occurred when a + credential reload request is scheduled/cancelled. */ struct grpc_tls_credential_reload_arg { grpc_tls_on_credential_reload_done_cb cb; - void *cb_user_data; - grpc_tls_key_materials_config *key_materials_config; - grpc_status_code *status; - const char **error_details; + void* cb_user_data; + grpc_tls_key_materials_config* key_materials_config; + grpc_status_code status; + const char* error_details; }; /** Create a grpc_tls_credential_reload_config instance. @@ -147,8 +156,8 @@ struct grpc_tls_credential_reload_arg { 1) If processing occurs synchronously, it populates arg->key_materials, arg->status, and arg->error_details and returns zero. 2) If processing occurs asynchronously, it returns a non-zero value. - The application then invokes arg->cb when processing is completed. Note that - arg->cb cannot be invoked before schedule API returns. + The application then invokes arg->cb when processing is completed. Note + that arg->cb cannot be invoked before schedule API returns. - cancel is a pointer to an application-provided callback used to cancel a credential reload request scheduled via an asynchronous schedule API. arg is used to pinpoint an exact reloading request to be cancelled. @@ -158,76 +167,68 @@ struct grpc_tls_credential_reload_arg { any data associated with the config. */ GRPCAPI grpc_tls_credential_reload_config* - grpc_tls_credential_reload_config_create( - const void* config_user_data, - int (*schedule)(void* config_user_data, - grpc_tls_credential_reload_arg* arg), - void (*cancel)(void* config_user_data, - grpc_tls_credential_reload_arg* arg), - void (*destruct)(void* config_user_data)); - - -/** Destroy a grpc_tls_credential_reload_config instance. */ -GRPCAPI void grpc_tls_credential_reload_config_destroy( - grpc_tls_credential_reload_config* config); +grpc_tls_credential_reload_config_create( + const void* config_user_data, + int (*schedule)(void* config_user_data, + grpc_tls_credential_reload_arg* arg), + void (*cancel)(void* config_user_data, grpc_tls_credential_reload_arg* arg), + void (*destruct)(void* config_user_data)); /** --- TLS server authorization check config. --- */ - typedef struct grpc_tls_server_authorization_check_arg grpc_tls_server_authorization_check_arg; /** callback function provided by gRPC used to handle the result of server - authorization check. It is used when schedule API is implemented - asynchronously, and serves to bring the control back to gRPC C core. */ + authorization check. It is used when schedule API is implemented + asynchronously, and serves to bring the control back to gRPC C core. */ typedef void (*grpc_tls_on_server_authorization_check_done_cb)( grpc_tls_server_authorization_check_arg* arg); /** A struct containing all information necessary to schedule/cancel a server - authorization check request. cb and cb_user_data represent a gRPC-provided callback and - an argument passed to it. result will store the result of server authorization - check. target_name is the name of an endpoint the channel is connecting to and - certificate represents a complete certificate chain including both signing and - leaf certificates. status and error_details contain information about errors - occurred when a server authorization check request is scheduled/cancelled. */ + authorization check request. cb and cb_user_data represent a gRPC-provided + callback and an argument passed to it. result will store the result of + server authorization check. target_name is the name of an endpoint the + channel is connecting to and certificate represents a complete certificate + chain including both signing and leaf certificates. status and error_details + contain information about errors occurred when a server authorization check + request is scheduled/cancelled. */ struct grpc_tls_server_authorization_check_arg { - grpc_tls_on_server_authorization_check_done_cb cb; - void *cb_user_data; - bool *result; - const char *target_name; - const char *certificate; - grpc_status_code *status; - const char **error_details; + grpc_tls_on_server_authorization_check_done_cb cb; + void* cb_user_data; + int result; + const char* target_name; + const char* peer_cert; + grpc_status_code status; + const char* error_details; }; /** Create a grpc_tls_server_authorization_check_config instance. - config_user_data is config-specific, read-only user data that works for all channels created with a credential using the config. - - schedule is a pointer to an application-provided callback used to invoke server - authorization check API. The implementation of this method has to be non-blocking, - but can be performed synchronously or asynchronously. - 1) If processing occurs synchronously, it populates arg->result, + - schedule is a pointer to an application-provided callback used to invoke + server authorization check API. The implementation of this method has to + be non-blocking, but can be performed synchronously or asynchronously. + 1)If processing occurs synchronously, it populates arg->result, arg->status, and arg->error_details and returns zero. - 2) If processing occurs asynchronously, it returns a non-zero value. - The application then invokes arg->cb when processing is completed. Note that + 2) If processing occurs asynchronously, it returns a non-zero value. The + application then invokes arg->cb when processing is completed. Note that arg->cb cannot be invoked before schedule API returns. - - cancel is a pointer to an application-provided callback used to cancel a server - authorization check request scheduled via an asynchronous schedule API. - arg is used to pinpoint an exact check request to be cancelled. - The operation may not have any effect if the request has already been processed. - - destruct is a pointer to an application-provided callback used to clean up any data - associated with the config. + - cancel is a pointer to an application-provided callback used to cancel a + server authorization check request scheduled via an asynchronous schedule + API. arg is used to pinpoint an exact check request to be cancelled. The + operation may not have any effect if the request has already been + processed. + - destruct is a pointer to an application-provided callback used to clean up + any data associated with the config. */ GRPCAPI grpc_tls_server_authorization_check_config* - grpc_tls_server_authorization_check_config_create( - const void* config_user_data, - int (*schedule)(void* config_user_data, - grpc_tls_server_authorization_check_arg* arg), - void (*cancel)(void* config_user_data, - grpc_tls_server_authorization_check_arg* arg), - void (*destruct)(void* config_user_data)); - -/** Destroy a grpc_tls_server_authorization_check_config instance. */ -GRPCAPI void grpc_tls_server_authorization_check_config_destroy(grpc_tls_server_authorization_check_config* config); +grpc_tls_server_authorization_check_config_create( + const void* config_user_data, + int (*schedule)(void* config_user_data, + grpc_tls_server_authorization_check_arg* arg), + void (*cancel)(void* config_user_data, + grpc_tls_server_authorization_check_arg* arg), + void (*destruct)(void* config_user_data)); ``` ## Implementation From 0f0c87b90ae77ebcf2e5c6c4463c21a676cc9133 Mon Sep 17 00:00:00 2001 From: Yihua Zhang Date: Mon, 28 Jan 2019 15:50:43 -0800 Subject: [PATCH 11/52] explicitly mention that the proposal does not work for Java and Go. --- L46-core-tls-credential-API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 5cac18827..f8c295fbb 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -8,7 +8,7 @@ Title * Discussion at: https://groups.google.com/forum/#!topic/grpc-io/gMrROsizrtc ## Abstract -This proposal aims to incorporate SPIFFE-based mutual TLS into gRPC C core auth stack. Towards the goal, it proposes a new credential API model that 1) allows flexible addition of new TLS-related features (e.g., credential reload) without breaking the API and 2) supports both synchronous and asynchronous implementations of those features. It then proposes SPIFFE and HTTPS-based TLS credential API's that comply with the new model. +This proposal aims to incorporate SPIFFE-based mutual TLS into gRPC C core auth stack. Towards the goal, it proposes a new credential API model that 1) allows flexible addition of new TLS-related features (e.g., credential reload) without breaking the API and 2) supports both synchronous and asynchronous implementations of those features. It then proposes SPIFFE and HTTPS-based TLS credential API's that comply with the new model. Note that the API changes proposed in this gRFC only apply to gRPC core and its wrapped languages (not Java and Go). ## Background From ca3e8725405b6c74a8b49f47ef64a2375670bbd2 Mon Sep 17 00:00:00 2001 From: ZhenLian Date: Mon, 7 Sep 2020 23:32:36 -0700 Subject: [PATCH 12/52] first edition: redundant version --- L46-core-tls-credential-API.md | 476 ++++++++++++++++++++------------- 1 file changed, 284 insertions(+), 192 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index f8c295fbb..43058a117 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -1,236 +1,328 @@ Title ---- -* Author(s): yihuazhang -* Approver: vjpai, markdroth +* Author(s): ZhenLian +* Approver: markdroth, yashykt, jiangtaoli2016 * Status: Draft * Implemented in: -* Last updated: September 3, 2018 -* Discussion at: https://groups.google.com/forum/#!topic/grpc-io/gMrROsizrtc +* Last updated: September 7, 2020 ## Abstract -This proposal aims to incorporate SPIFFE-based mutual TLS into gRPC C core auth stack. Towards the goal, it proposes a new credential API model that 1) allows flexible addition of new TLS-related features (e.g., credential reload) without breaking the API and 2) supports both synchronous and asynchronous implementations of those features. It then proposes SPIFFE and HTTPS-based TLS credential API's that comply with the new model. Note that the API changes proposed in this gRFC only apply to gRPC core and its wrapped languages (not Java and Go). + +This proposal aims to enhance the security features provided by [this already implemented proposal](https://github.com/grpc/proposal/pull/98). +The improvements include: +1. replacing the per-connection callback-based credential reloading mechanism with the new distributor/provider mechanism +2. changing the server authorization check to an authorization check that could be applied to both the client side and the server side +3. adding options for users to configure their TLS version ## Background -The current TLS implementation in gRPC C core has the following problems when it comes to support SPIFFE-based mutual TLS. +The current [TLS credential API](https://github.com/grpc/grpc/blob/master/src/core/lib/security/credentials/tls/tls_credentials.h) in gRPC C core implemented the following two features: + +1. per-connection based credential reloading callback that allows callers to reload transport credentials without shutting down +2. per-connection based server authorization check on the client side that allows callers to do custom checks at the end of the handshake -1: It is originally designed to comply with HTTPS semantics which makes it difficult to be configured to support SPIFFE-based mutual TLS. For instance, the server authorization check in the current implementation is enforced locally by leveraging whitelisted endpoint information embedded in peer certificates while in SPIFFE use case, it may be enforced via an external server authorization check service. Here, the server authorization check is used to validate if a peer is authorized to run a job with a specific target name, and is invoked only at the client-side. +We recently have the following two use cases that we want to support: -2: It realizes TLS-related features such as credential reload and server authorization check in a synchronous manner. It is undesirable in SPIFFE use case as those operations could involve expensive RPC calls to other services which may block gRPC core threads. +1. In gRPC Proxyless Service Mesh, gRPC client has no initial credential to bootstrap with, which means the credentials fetching may not be instantaneous. Therefore, the current TLS credentials reloading mechanism cannot be used directly. +2. In some internal use cases, the credentials are stored in certain file locations and updated asynchronously. Application does not need to be notified of when/how the credentials are updated. gRPC core would be responsible for reading credentials from files and reloading the credentials. -3: The current TLS credential API is inflexible in incorporating new TLS-related features in future as not only does it break the API, it also requires all wrapped languages to update their implementations in a lockstep manner. +These two cases indicate the following design requirements: +1. Support both use cases above. We cannot assume there are bootstrapping key materials. +2. The credentials reloading C core API needs to be simple, so that it can be easily wrapped by different languages. The credentials reloading C++ API needs to be user-friendly and simple to use. If possible, the API shall avoid complex asynchronous programming (schedule, cancel, mutex, etc.) by the gRPC users. +3. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per handshake. -This proposal introduces a new credential API model that allows flexible addition of TLS-related features that can be implemented in either a synchronous or asynchronous manner. It then proposes new credential API's for both HTTPS and SPIFFE-based TLS that complies with the API model. +Based on the requirements, we are going to propose a new distributor/provider API that will meet these requirements. +The distributor is responsible for caching the credentials and distributing them in each handshake. +The users will need to specify a particular provider implementation when creating their credentials, and we will provide one implementation for reloading from files. +If that doesn't meet users' requirements, they could implement their own providers. + +Besides major credential reloading change, we are also planning to: +1. make the server authorization original designed for client side only applicable to both the client side and the server side. +This is based on some recent user requirement, and it may also be used as the CRL check since CRL is not officially supported in gRPC. +2. add the options to configure the min/max TLS version that will be used in handshake ### Related Proposals: -N/A + +[L46: Create a new TLS credential API that supports SPIFFE mutual TLS](https://github.com/grpc/proposal/pull/98) ## Proposal -The first part of proposal introduces credential options and credential create APIs that will be added to include/grpc/grpc_security.h. Note that although the credentials options provided to HTTPS and SPIFFE-based TLS credential API have the same format (i.e., grpc_tls_credentials_options), some of its members will not get populated and hence used in some API's, for instance, grpc_tls_server_authorization_check_config will be ignored in server-side API's. +### Changes around TLS credentials and Options +This part of the proposal introduces changes made to existing TLS credential APIs. Changes include: +1. Replacing all the occurrence of "*_server_authorization_*" to "*_authorization_" +2. Removing `grpc_tls_key_materials_config`, `grpc_tls_credential_reload_config` and `grpc_tls_credential_reload_arg` +3. Adding API `grpc_tls_credentials_options_set_certificate_provider`, `grpc_ssl_server_credentials_set_min_tls_version` and `grpc_ssl_server_credentials_set_max_tls_version` -```c++ +Here is how the API would look like after these changes. For simplicity, all the comments are omitted. -/** Config for TLS key materials. */ -typedef struct grpc_tls_key_materials_config grpc_tls_key_materials_config; +```c +/** --- TLS channel/server credentials --- + * It is used for experimental purpose for now and subject to change. */ -/** Config for TLS credential reload. */ -typedef struct grpc_tls_credential_reload_config - grpc_tls_credential_reload_config; +typedef struct grpc_tls_error_details grpc_tls_error_details; -/** Config for TLS server authorization check. */ -typedef struct grpc_tls_server_authorization_check_config - grpc_tls_server_authorization_check_config; +typedef struct grpc_tls_authorization_check_config grpc_tls_authorization_check_config; -/** TLS credentials options. */ typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; -/** Create an empty TLS credentials options. */ -GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(); +GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(void); -/** Set grpc_ssl_client_certificate_request_type field in credentials options - with the provided type. options should not be NULL. - It returns 1 on success and 0 on failure. */ GRPCAPI int grpc_tls_credentials_options_set_cert_request_type( grpc_tls_credentials_options* options, grpc_ssl_client_certificate_request_type type); -/** Set grpc_tls_key_materials_config field in credentials options - with the provided config struct whose ownership is transferred. - Both parameters should not be NULL. - It returns 1 on success and 0 on failure. */ -GRPCAPI int grpc_tls_credentials_options_set_key_materials_config( +GRPCAPI int grpc_tls_credentials_options_set_verification_option( grpc_tls_credentials_options* options, - grpc_tls_key_materials_config* config); + grpc_tls_verification_option verification_option); -/** Set grpc_tls_credential_reload_config field in credentials options - with the provided config struct whose ownership is transferred. - Both parameters should not be NULL. - It returns 1 on success and 0 on failure. */ -GRPCAPI int grpc_tls_credentials_options_set_credential_reload_config( +GRPCAPI int grpc_tls_credentials_options_set_authorization_check_config( grpc_tls_credentials_options* options, - grpc_tls_credential_reload_config* config); + grpc_tls_authorization_check_config* config); -/** Set grpc_tls_server_authorization_check_config field in credentials options - with the provided config struct whose ownership is transferred. - Both parameters should not be NULL. - It returns 1 on success and 0 on failure. */ -GRPCAPI int grpc_tls_credentials_options_set_server_authorization_check_config( +/** Sets the credential provider. */ +GRPCAPI int grpc_tls_credentials_options_set_certificate_provider( grpc_tls_credentials_options* options, - grpc_tls_server_authorization_check_config* config); - -/** Destroy a TLS credentials options. */ -GRPCAPI void grpc_tls_credentials_options_destroy( - grpc_tls_credentials_options* options); - -/** Create an HTTPS-based TLS server credential object using the provided - options struct whose ownership is transferred. */ -GRPCAPI grpc_server_credentials* grpc_tls_https_server_credentials_create( - const grpc_tls_credentials_options* options); - -/** Create an HTTPS-based TLS channel credential object using the provided - options struct whose ownership is transferred. */ -GRPCAPI grpc_channel_credentials* grpc_tls_https_credentials_create( - const grpc_tls_credentials_options* options); - -/** Create a SPIFFE-based TLS server credential object using the provided - options struct whose ownership is transferred. */ -GRPCAPI grpc_server_credentials* grpc_tls_spiffe_server_credentials_create( - const grpc_tls_credentials_options* options); - -/** Create a SPIFFE-based TLS channel credential object using the provided - options struct whose ownership is transferred. */ -GRPCAPI grpc_channel_credentials* grpc_tls_spiffe_credentials_create( - const grpc_tls_credentials_options* options); -``` + grpc_tls_certificate_provider* provider); + +/** Sets the minimum TLS version that will be negotiated during the TLS + handshake. */ +GRPCAPI void grpc_ssl_server_credentials_set_min_tls_version(grpc_tls_credentials_options* options, + grpc_tls_version min_tls_version); + +/** Sets the maximum TLS version that will be negotiated during the TLS + handshake. */ +GRPCAPI void grpc_ssl_server_credentials_set_max_tls_version(grpc_tls_credentials_options* options, + grpc_tls_version max_tls_version); + +typedef struct grpc_tls_authorization_check_arg grpc_tls_authorization_check_arg; -The second part of proposal introduces API's related to configs of TLS-related features that will be added to include/grpc/grpc_security.h. - -```c++ -/** --- TLS key materials config. --- */ - -/** Create an empty grpc_tls_key_materials_config instance. */ -GRPCAPI grpc_tls_key_materials_config* grpc_tls_key_materials_config_create(); - -/** Set grpc_tls_key_materials_config instance with provided a TLS certificate. - config will take the ownership of pem_root_certs and pem_key_cert_pairs. - It's valid for the caller to provide nullptr pem_root_certs, in which case - the gRPC-provided root cert will be used. pem_key_cert_pairs should not be - NULL. It returns 1 on success and 0 on failure. - */ -GRPCAPI int grpc_tls_key_materials_config_set_key_materials( - grpc_tls_key_materials_config* config, const char* pem_root_certs, - const grpc_ssl_pem_key_cert_pair** pem_key_cert_pairs, - size_t num_key_cert_pairs); - -/** --- TLS credential reload config. --- */ - -typedef struct grpc_tls_credential_reload_arg grpc_tls_credential_reload_arg; - -/** A callback function provided by gRPC to handle the result of credential - reload. It is used when schedule API is implemented asynchronously and - serves to bring the control back to grpc C core. */ -typedef void (*grpc_tls_on_credential_reload_done_cb)( - grpc_tls_credential_reload_arg* arg); - -/** A struct containing all information necessary to schedule/cancel - a credential reload request. cb and cb_user_data represent a gRPC-provided - callback and an argument passed to it. key_materials is an in/output - parameter containing currently used/newly reloaded credentials. status and - error_details are used to hold information about errors occurred when a - credential reload request is scheduled/cancelled. */ -struct grpc_tls_credential_reload_arg { - grpc_tls_on_credential_reload_done_cb cb; - void* cb_user_data; - grpc_tls_key_materials_config* key_materials_config; - grpc_status_code status; - const char* error_details; +struct grpc_tls_authorization_check_arg { + /** nothing changed; omitted */ }; -/** Create a grpc_tls_credential_reload_config instance. - - config_user_data is config-specific, read-only user data - that works for all channels created with a credential using the config. - - schedule is a pointer to an application-provided callback used to invoke - credential reload API. The implementation of this method has to be - non-blocking, but can be performed synchronously or asynchronously. - 1) If processing occurs synchronously, it populates arg->key_materials, - arg->status, and arg->error_details and returns zero. - 2) If processing occurs asynchronously, it returns a non-zero value. - The application then invokes arg->cb when processing is completed. Note - that arg->cb cannot be invoked before schedule API returns. - - cancel is a pointer to an application-provided callback used to cancel - a credential reload request scheduled via an asynchronous schedule API. - arg is used to pinpoint an exact reloading request to be cancelled. - The operation may not have any effect if the request has already been - processed. - - destruct is a pointer to an application-provided callback used to clean up - any data associated with the config. -*/ -GRPCAPI grpc_tls_credential_reload_config* -grpc_tls_credential_reload_config_create( - const void* config_user_data, - int (*schedule)(void* config_user_data, - grpc_tls_credential_reload_arg* arg), - void (*cancel)(void* config_user_data, grpc_tls_credential_reload_arg* arg), - void (*destruct)(void* config_user_data)); - -/** --- TLS server authorization check config. --- */ -typedef struct grpc_tls_server_authorization_check_arg - grpc_tls_server_authorization_check_arg; - -/** callback function provided by gRPC used to handle the result of server - authorization check. It is used when schedule API is implemented - asynchronously, and serves to bring the control back to gRPC C core. */ -typedef void (*grpc_tls_on_server_authorization_check_done_cb)( - grpc_tls_server_authorization_check_arg* arg); - -/** A struct containing all information necessary to schedule/cancel a server - authorization check request. cb and cb_user_data represent a gRPC-provided - callback and an argument passed to it. result will store the result of - server authorization check. target_name is the name of an endpoint the - channel is connecting to and certificate represents a complete certificate - chain including both signing and leaf certificates. status and error_details - contain information about errors occurred when a server authorization check - request is scheduled/cancelled. */ -struct grpc_tls_server_authorization_check_arg { - grpc_tls_on_server_authorization_check_done_cb cb; - void* cb_user_data; - int result; - const char* target_name; - const char* peer_cert; - grpc_status_code status; - const char* error_details; +GRPCAPI grpc_tls_authorization_check_config* +grpc_tls_authorization_check_config_create(/** omitted */); + +grpc_channel_credentials* grpc_tls_credentials_create(grpc_tls_credentials_options* options); + +grpc_server_credentials* grpc_tls_server_credentials_create(grpc_tls_credentials_options* options); +``` + +### Provider +The provider API offers a general interface for different implementations to interact with the `distributor`. + +`grpc_tls_certificate_provider` is the provider interface we will use in `grpc_tls_credentials_options`. +Callers could directly pass in the already implemented providers, e.g. file-watcher provider, or a custom provider `grpc_tls_certificate_provider_external`, whose function pointer implementations are completely determined by the callers. + +```c +/* Opaque type. */ +typedef struct grpc_tls_certificate_provider grpc_tls_certificate_provider; +typedef struct grpc_tls_certificate_provider_external grpc_tls_certificate_provider_external; + +/* Factory function for file-watcher provider (implemented inside core). */ +grpc_tls_certificate_provider* grpc_tls_certificate_provider_file_watcher_create(const char* private_key_file_name, const char* identity_certificate_file_name, const char* root_certificate_file_name); + +/* API for creating external provider. */ +struct grpc_tls_certificate_provider_external { + grpc_tls_certificate_distributor* (*get_distributor)(grpc_tls_certificate_provider_external*); + void (*destroy)(grpc_tls_certificate_provider_external*); }; -/** Create a grpc_tls_server_authorization_check_config instance. - - config_user_data is config-specific, read-only user data - that works for all channels created with a credential using the config. - - schedule is a pointer to an application-provided callback used to invoke - server authorization check API. The implementation of this method has to - be non-blocking, but can be performed synchronously or asynchronously. - 1)If processing occurs synchronously, it populates arg->result, - arg->status, and arg->error_details and returns zero. - 2) If processing occurs asynchronously, it returns a non-zero value. The - application then invokes arg->cb when processing is completed. Note that - arg->cb cannot be invoked before schedule API returns. - - cancel is a pointer to an application-provided callback used to cancel a - server authorization check request scheduled via an asynchronous schedule - API. arg is used to pinpoint an exact check request to be cancelled. The - operation may not have any effect if the request has already been - processed. - - destruct is a pointer to an application-provided callback used to clean up - any data associated with the config. +/* The ownership of grpc_tls_certificate_provider_external will be taken by grpc_tls_certificate_provider. */ +grpc_tls_certificate_provider* grpc_tls_certificate_provider_external_create( + grpc_tls_certificate_provider_external* external_provider); + +``` + +### Distributor +The distributor is the actual component for caching and distributing the credentials to the underlying transport connections(security connectors). +```c +typedef struct grpc_tls_certificate_distributor grpc_tls_certificate_distributor; + +/* Creates a TLS certificate distributor object. This object is ref-counted. */ +GRPCAPI grpc_tls_certificate_distributor* grpc_tls_certificate_distributor_create(); + +/* Unrefs a TLS certificate distributor object. */ +GRPCAPI void grpc_tls_certificate_distributor_unref(grpc_tls_certificate_distributor* distributor); + +/* Sets root certificates and key and certificate chain pairs. The Ownerships of + cert_name, pem_root_certs and pem_key_cert_pairs are not transferred. */ +GRPCAPI int grpc_tls_certificate_distributor_set_key_materials( + grpc_tls_certificate_distributor* distributor, + const char* cert_name, + const char* pem_root_certs, + const grpc_ssl_pem_key_cert_pair** pem_key_cert_pairs, size_t num_key_cert_pairs); + +/* Sets root certificates. The Ownership of cert_name and pem_root_certs are not transferred. */ +GRPCAPI int grpc_tls_certificate_distributor_set_root_certs( + grpc_tls_certificate_distributor* distributor, const char* cert_name, + const char* pem_root_certs); + +/* Sets key and certificate chain pairs. The Ownership of cert_name, and pem_key_cert_pairs are not transferred. */ +GRPCAPI int grpc_tls_certificate_distributor_set_key_cert_pairs( + grpc_tls_certificate_distributor* distributor, + const char* cert_name, + const grpc_ssl_pem_key_cert_pair** pem_key_cert_pairs, size_t num_key_cert_pairs); + +/* Sets errors for both the root and the identity certificates of name |cert_name|. + At least one of root_cert_error and identity_cert_error must be set. */ +GRPCAPI int grpc_tls_certificate_distributor_set_error_for_key_materials( + grpc_tls_certificate_distributor* distributor, + const char* cert_name, + grpc_tls_error_details* root_cert_error, + grpc_tls_error_details* identity_cert_error); + +/* Callback function to be called by the TLS certificate distributor to inform + the TLS certificate provider when the watch status changed. + - user_data is the opaque pointer that was passed to TLS certificate distributor + - watching_root_certs is a boolean value indicating that root certificates are + being watched. + - watching_key_cert_pairs is a boolean value indicating that the key certificate + pairs are being watched. + - cert_name is the name of the certificates being watched. */ -GRPCAPI grpc_tls_server_authorization_check_config* -grpc_tls_server_authorization_check_config_create( - const void* config_user_data, - int (*schedule)(void* config_user_data, - grpc_tls_server_authorization_check_arg* arg), - void (*cancel)(void* config_user_data, - grpc_tls_server_authorization_check_arg* arg), - void (*destruct)(void* config_user_data)); +typedef void (*grpc_tls_certificate_watch_status_cb)( + void* user_data, const char* cert_name, bool watching_root_certs, bool watching_key_cert_pairs); + +/* Sets the watch status callback on the distributor. + Callbacks are invoked synchronously. The user_data parameter will be passed + to the callback when it is invoked. + The callback can be set to null to disable callbacks. Callers should generally + set it to null before they unref the distributor, to ensure that no subsequent + callbacks are invoked. */ +GRPCAPI void grpc_tls_certificate_distributor_set_watch_status_callback( + grpc_tls_certificate_distributor* distributor, + grpc_tls_certificate_watch_status_cb cb, void* user_data); +``` + +## Example Use + +We will provide an example showing how to put all these APIs together. Note that users wouldn't be able to interact directly with the core APIs. +Each wrap language needs to wrap these APIs in a way that meets the inherent core logic. + +```c +/* Create option and set basic security primitives. */ +grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); +grpc_tls_credentials_options_set_verification_option(options, GRPC_TLS_SKIP_HOSTNAME_VERIFICATION); +grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(.....); +grpc_tls_credentials_options_set_authorization_check_config(options, config); +grpc_ssl_server_credentials_set_min_tls_version(options, TLS1_2); +grpc_ssl_server_credentials_set_max_tls_version(options, TLS1_3); +/* Use the file-based provider for reloading certificates. */ +grpc_tls_certificate_provider* file_provider = grpc_tls_certificate_provider_file_watcher_create(...); +grpc_tls_credentials_options_set_certificate_provider(options, file_provider); +/* Create TLS channel credentials. */ +grpc_channel_credentials* creds = grpc_tls_credentials_create(options); ``` +Wrap language should also allow users to define their own provider implementations: +```c +static grpc_tls_certificate_distributor* GetDistributor(grpc_tls_certificate_provider_external* external_provider) { + /* .... */ +} +static void Destroy(grpc_tls_certificate_provider_external* external_provider) { + /* .... */ + delete external_provider; +} +grpc_tls_certificate_provider_external* external_provider = new grpc_tls_certificate_provider_external(); +external_provider->get_distributor = GetDistributor; +external_provider->destroy = Destroy; +/* The ownership will be taken by provider. */ +grpc_tls_certificate_provider* provider grpc_tls_certificate_provider_external_create(external_provider); -## Implementation +``` + +## Wrap Languages +Each wrap language might have their own way to wrap these core APIs, but in general, there are a few things to consider: +1. create an interface-like class `ProviderInterface` which contains two functions GetDistributor() and Destroy() +2. create a concrete class `FileCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_watcher_create` +3. create an interface-like class `CertificateProvider` that implements `ProviderInterface`, and exposes several APIs in the distributor to users, such as `grpc_tls_certificate_distributor_set_key_materials`, etc +4. users could extend `CertificateProvider` to define their own provider implementations + +For point 3 and 4, here is an example of `CertificateProvider` in C++: +```cpp +namespace grpc { + +template +class CertificateProvider { + public: + CertificateProvider() + : distributor_(grpc_tls_certificate_distributor_create()) { + base.get_distributor = GetDistributor; + base.destroy = Destroy; + grpc_tls_certificate_distributor_set_watch_status_callback( + OnWatchStatusChangedCallback, this); + } + + virtual ~CertificateProvider() { + grpc_tls_certificate_distributor_destroy(distributor_); + } + + grpc_tls_certificate_provider* c_provider() const { + return grpc_tls_certificate_provider_external_create(&base_); + } + + virtual void OnWatchStatusChanged( + const std::string& cert_name, bool watching_root, + bool watching_identity) = 0; + + void SetRootCert(const std::string& cert_name, + const std::string& contents) { + grpc_tls_certificate_distributor_set_root_certs( + distributor_, cert_name.c_str(), contents.c_str()); + } + + // ...similar functions for SetCertKeyPairs() and SetKeyMaterials()... + + private: + static grpc_tls_certificate_distributor* GetDistributor( + grpc_tls_certificate_provider_external* external_provider) { + auto* provider = static_cast(external_provider); + return provider->distributor_; + } + + static void Destroy( + grpc_tls_certificate_provider_external* external_provider) { + auto* provider = static_cast(external_provider); + delete provider; + } + + static void OnWatchStatusChangedCallback( + void* arg, const std::string& cert_name, bool watching_root, + bool watching_identity) { + auto* provider = static_cast(arg); + provider->OnWatchStatusChanged(cert_name, watching_root, + watching_identity); + } + + // First data element to simulate inheritance. + grpc_tls_certificate_provider_external base_; + + grpc_tls_certificate_distributor* distributor_; +}; + +} // namespace grpc +``` +and an example of custom provider implementation: +```cpp +class MyCertificateProvider: public grpc::CertificateProvider { + public: + void OnWatchStatusChanged( + const std::string& cert_name, bool watching_root, + bool watching_identity) override { + if (!watching_root && !watching_identity) { + certs_watching_.erase(cert_name); + } else { + certs_watching_[cert_name] = {watching_root, watching_identity}; + // ...start or stop watching as needed... + } + } + + private: + struct CertsWatching { + bool watching_root = false; + bool watching_identity = false; + }; + std::map certs_watching_; +}; +``` -The implementation of this proposal is straightforward as all API changes will be made to include/grpc/grpc_security.h. Note that once the new HTTPS-based TLS credential API gets implemented, the old HTTPS-based TLS credential API will get deprecated and be discouraged to use. From d7cb92b48bf5305452178cadf4bb745a296af6f0 Mon Sep 17 00:00:00 2001 From: ZhenLian Date: Mon, 7 Sep 2020 23:59:17 -0700 Subject: [PATCH 13/52] add APIs to select names --- L46-core-tls-credential-API.md | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 43058a117..d59a66642 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -24,7 +24,7 @@ The current [TLS credential API](https://github.com/grpc/grpc/blob/master/src/co We recently have the following two use cases that we want to support: 1. In gRPC Proxyless Service Mesh, gRPC client has no initial credential to bootstrap with, which means the credentials fetching may not be instantaneous. Therefore, the current TLS credentials reloading mechanism cannot be used directly. -2. In some internal use cases, the credentials are stored in certain file locations and updated asynchronously. Application does not need to be notified of when/how the credentials are updated. gRPC core would be responsible for reading credentials from files and reloading the credentials. +2. In some internal use cases, the credentials are stored in certain file locations and updated asynchronously. Application does not need to be notified of when/how the credentials are updated. gRPC core would be responsible for routinely reading credentials from files and reloading the credentials. These two cases indicate the following design requirements: 1. Support both use cases above. We cannot assume there are bootstrapping key materials. @@ -38,7 +38,7 @@ If that doesn't meet users' requirements, they could implement their own provide Besides major credential reloading change, we are also planning to: 1. make the server authorization original designed for client side only applicable to both the client side and the server side. -This is based on some recent user requirement, and it may also be used as the CRL check since CRL is not officially supported in gRPC. +This is based on some recent user requirement, and it may also be used as the CRL check since CRL is not supported in gRPC core. 2. add the options to configure the min/max TLS version that will be used in handshake ### Related Proposals: @@ -51,7 +51,8 @@ This is based on some recent user requirement, and it may also be used as the CR This part of the proposal introduces changes made to existing TLS credential APIs. Changes include: 1. Replacing all the occurrence of "*_server_authorization_*" to "*_authorization_" 2. Removing `grpc_tls_key_materials_config`, `grpc_tls_credential_reload_config` and `grpc_tls_credential_reload_arg` -3. Adding API `grpc_tls_credentials_options_set_certificate_provider`, `grpc_ssl_server_credentials_set_min_tls_version` and `grpc_ssl_server_credentials_set_max_tls_version` +3. Adding API `grpc_tls_credentials_options_set_certificate_provider`, `grpc_tls_server_credentials_set_min_tls_version` and `grpc_tls_server_credentials_set_max_tls_version` +4. Adding API `grpc_tls_credentials_options_set_root_cert_name` and `grpc_tls_credentials_options_set_identity_cert_name` Here is how the API would look like after these changes. For simplicity, all the comments are omitted. @@ -71,14 +72,22 @@ GRPCAPI int grpc_tls_credentials_options_set_cert_request_type( grpc_tls_credentials_options* options, grpc_ssl_client_certificate_request_type type); -GRPCAPI int grpc_tls_credentials_options_set_verification_option( +GRPCAPI int grpc_tls_credentials_options_set_server_verification_option( grpc_tls_credentials_options* options, - grpc_tls_verification_option verification_option); + grpc_tls_server_verification_option server_verification_option); GRPCAPI int grpc_tls_credentials_options_set_authorization_check_config( grpc_tls_credentials_options* options, grpc_tls_authorization_check_config* config); +GRPCAPI int grpc_tls_credentials_options_set_root_cert_name( + grpc_tls_credentials_options* options, + const char* root_cert_name); + +GRPCAPI int grpc_tls_credentials_options_set_identity_cert_name( + grpc_tls_credentials_options* options, + const char* identity_cert_name); + /** Sets the credential provider. */ GRPCAPI int grpc_tls_credentials_options_set_certificate_provider( grpc_tls_credentials_options* options, @@ -86,12 +95,12 @@ GRPCAPI int grpc_tls_credentials_options_set_certificate_provider( /** Sets the minimum TLS version that will be negotiated during the TLS handshake. */ -GRPCAPI void grpc_ssl_server_credentials_set_min_tls_version(grpc_tls_credentials_options* options, +GRPCAPI void grpc_tls_server_credentials_set_min_tls_version(grpc_tls_credentials_options* options, grpc_tls_version min_tls_version); /** Sets the maximum TLS version that will be negotiated during the TLS handshake. */ -GRPCAPI void grpc_ssl_server_credentials_set_max_tls_version(grpc_tls_credentials_options* options, +GRPCAPI void grpc_tls_server_credentials_set_max_tls_version(grpc_tls_credentials_options* options, grpc_tls_version max_tls_version); typedef struct grpc_tls_authorization_check_arg grpc_tls_authorization_check_arg; @@ -203,11 +212,13 @@ Each wrap language needs to wrap these APIs in a way that meets the inherent cor ```c /* Create option and set basic security primitives. */ grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); -grpc_tls_credentials_options_set_verification_option(options, GRPC_TLS_SKIP_HOSTNAME_VERIFICATION); +grpc_tls_credentials_options_set_server_verification_option(options, GRPC_TLS_SKIP_HOSTNAME_VERIFICATION); grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(.....); grpc_tls_credentials_options_set_authorization_check_config(options, config); -grpc_ssl_server_credentials_set_min_tls_version(options, TLS1_2); -grpc_ssl_server_credentials_set_max_tls_version(options, TLS1_3); +grpc_tls_server_credentials_set_min_tls_version(options, TLS1_2); +grpc_tls_server_credentials_set_max_tls_version(options, TLS1_3); +grpc_tls_credentials_options_set_root_cert_name(options, "root_PEM_certs"); +grpc_tls_credentials_options_set_identity_cert_name(options, "identity_PEM_certs"); /* Use the file-based provider for reloading certificates. */ grpc_tls_certificate_provider* file_provider = grpc_tls_certificate_provider_file_watcher_create(...); grpc_tls_credentials_options_set_certificate_provider(options, file_provider); From 5e057b446b35a0779b7e6a4f31c5c10db09cdde8 Mon Sep 17 00:00:00 2001 From: ZhenLian Date: Wed, 9 Sep 2020 23:46:02 -0700 Subject: [PATCH 14/52] second edition --- L46-core-tls-credential-API.md | 341 ++++++++++++++++++++++----------- 1 file changed, 226 insertions(+), 115 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index d59a66642..0e2e74ddc 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -1,45 +1,34 @@ -Title +L46: C-core: New TLS Credentials API ---- * Author(s): ZhenLian * Approver: markdroth, yashykt, jiangtaoli2016 * Status: Draft * Implemented in: -* Last updated: September 7, 2020 +* Last updated: September 9, 2020 ## Abstract -This proposal aims to enhance the security features provided by [this already implemented proposal](https://github.com/grpc/proposal/pull/98). -The improvements include: -1. replacing the per-connection callback-based credential reloading mechanism with the new distributor/provider mechanism -2. changing the server authorization check to an authorization check that could be applied to both the client side and the server side -3. adding options for users to configure their TLS version +This proposal aims to provide the following security features in a new TLS credential API: +1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down +2. custom authorization check: optionally bypass the certificate check, bypass the hostname check on the client side, and perform an additional per-connection authorization callback at the end of authentication +3. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake ## Background -The current [TLS credential API](https://github.com/grpc/grpc/blob/master/src/core/lib/security/credentials/tls/tls_credentials.h) in gRPC C core implemented the following two features: +The current TLS implementation in gRPC C core has some restrictions: +1. lack of the credential reloading support. Once an application starts, it can never reload its identity key/cert-chain and root certificates without shutting down +2. inflexibility in the peer verification. In a mutual TLS scenario, servers would always perform certificate verification, and clients would always perform certificate verification as well as default hostname check. No customized checks can be introduced. +3. inability to choose the TLS version that will be used -1. per-connection based credential reloading callback that allows callers to reload transport credentials without shutting down -2. per-connection based server authorization check on the client side that allows callers to do custom checks at the end of the handshake +There have always been some ad hoc requests on GitHub Issue Page for supporting these features. The demands for a more flexible and configurable API have increased with the emerging needs in cloud platform. +One common example is to support SPIFFE ID as the identity format, which requires us to disable the default hostname check on the client side. +Another example is to reload the identity certificates without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. -We recently have the following two use cases that we want to support: +Hence we proposed an API that will meet the following requirements: -1. In gRPC Proxyless Service Mesh, gRPC client has no initial credential to bootstrap with, which means the credentials fetching may not be instantaneous. Therefore, the current TLS credentials reloading mechanism cannot be used directly. -2. In some internal use cases, the credentials are stored in certain file locations and updated asynchronously. Application does not need to be notified of when/how the credentials are updated. gRPC core would be responsible for routinely reading credentials from files and reloading the credentials. - -These two cases indicate the following design requirements: -1. Support both use cases above. We cannot assume there are bootstrapping key materials. -2. The credentials reloading C core API needs to be simple, so that it can be easily wrapped by different languages. The credentials reloading C++ API needs to be user-friendly and simple to use. If possible, the API shall avoid complex asynchronous programming (schedule, cancel, mutex, etc.) by the gRPC users. -3. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per handshake. - -Based on the requirements, we are going to propose a new distributor/provider API that will meet these requirements. -The distributor is responsible for caching the credentials and distributing them in each handshake. -The users will need to specify a particular provider implementation when creating their credentials, and we will provide one implementation for reloading from files. -If that doesn't meet users' requirements, they could implement their own providers. - -Besides major credential reloading change, we are also planning to: -1. make the server authorization original designed for client side only applicable to both the client side and the server side. -This is based on some recent user requirement, and it may also be used as the CRL check since CRL is not supported in gRPC core. -2. add the options to configure the min/max TLS version that will be used in handshake +1. Flexible enough to support multiple use cases. Users should be able to use their own implementations if the provided ones don't fit their needs +2. C core API needs to be simple, so that it can be easily wrapped by different languages +3. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per handshake ### Related Proposals: @@ -47,43 +36,96 @@ This is based on some recent user requirement, and it may also be used as the CR ## Proposal -### Changes around TLS credentials and Options -This part of the proposal introduces changes made to existing TLS credential APIs. Changes include: -1. Replacing all the occurrence of "*_server_authorization_*" to "*_authorization_" -2. Removing `grpc_tls_key_materials_config`, `grpc_tls_credential_reload_config` and `grpc_tls_credential_reload_arg` -3. Adding API `grpc_tls_credentials_options_set_certificate_provider`, `grpc_tls_server_credentials_set_min_tls_version` and `grpc_tls_server_credentials_set_max_tls_version` -4. Adding API `grpc_tls_credentials_options_set_root_cert_name` and `grpc_tls_credentials_options_set_identity_cert_name` - -Here is how the API would look like after these changes. For simplicity, all the comments are omitted. +### TLS credentials and TLS Options +This part of the proposal introduces `grpc_tls_credentials_options` and TLS credentials. Users use `grpc_tls_credentials_options` to configure the specific security properties they want, and build the client/server credential by passing `grpc_tls_credentials_options` in. +This section only covers the configuration not related to credential reloading and custom verification. Configurations related to those two topics are explained in the latter sections. ```c /** --- TLS channel/server credentials --- * It is used for experimental purpose for now and subject to change. */ -typedef struct grpc_tls_error_details grpc_tls_error_details; - -typedef struct grpc_tls_authorization_check_config grpc_tls_authorization_check_config; - typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(void); +/** + * Set grpc_ssl_client_certificate_request_type in credentials options. + * If not set, we will use the default value GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. + */ GRPCAPI int grpc_tls_credentials_options_set_cert_request_type( grpc_tls_credentials_options* options, grpc_ssl_client_certificate_request_type type); +/** + * Set grpc_tls_server_verification_option. + * If not set, we will use the default value GRPC_TLS_SERVER_VERIFICATION. + * This should be called only on the client side. + */ GRPCAPI int grpc_tls_credentials_options_set_server_verification_option( grpc_tls_credentials_options* options, grpc_tls_server_verification_option server_verification_option); -GRPCAPI int grpc_tls_credentials_options_set_authorization_check_config( - grpc_tls_credentials_options* options, - grpc_tls_authorization_check_config* config); +/** + * Sets the minimum TLS version that will be negotiated during the TLS handshake. + * If not set, the underlying SSL library will pick the version automatically. + */ +GRPCAPI void grpc_tls_credentials_options_set_min_tls_version(grpc_tls_credentials_options* options, + grpc_tls_version min_tls_version); + +/** + * Sets the maximum TLS version that will be negotiated during the TLS handshake. + * If not set, the underlying SSL library will pick the version automatically. + */ +GRPCAPI void grpc_tls_credentials_options_set_max_tls_version(grpc_tls_credentials_options* options, + grpc_tls_version max_tls_version); + +/** + * This method creates a TLS channel credential object. + * It takes ownership of the options parameter. + */ +grpc_channel_credentials* grpc_tls_credentials_create(grpc_tls_credentials_options* options); + +grpc_server_credentials* grpc_tls_server_credentials_create(grpc_tls_credentials_options* options); +``` +Here is an example of how to use this API: +```c +/* Create option and set basic security primitives. */ +grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); +grpc_tls_credentials_options_set_server_verification_option(options, GRPC_TLS_SKIP_HOSTNAME_VERIFICATION); +grpc_tls_server_credentials_set_min_tls_version(options, TLS1_2); +grpc_tls_server_credentials_set_max_tls_version(options, TLS1_3); +/* Create TLS channel credentials. */ +grpc_channel_credentials* creds = grpc_tls_credentials_create(options); +``` + +### Credential Reloading +The credential reloading API basically consists of two parts: the top-level `Provider` and the low-level `Distributor`. +The `Distributor` is the actual component for caching and distributing the credentials to the underlying transport connections(security connectors). +The `Provider` offers a general interface for different implementations to interact with the `Distributor`. +```c +/* Opaque type. */ +typedef struct grpc_tls_certificate_provider grpc_tls_certificate_provider; +typedef struct grpc_tls_certificate_distributor grpc_tls_certificate_distributor; +typedef struct grpc_tls_certificate_provider_external grpc_tls_certificate_provider_external; +/* The ownership of grpc_tls_certificate_provider_external will be taken by grpc_tls_certificate_provider. */ +grpc_tls_certificate_provider* grpc_tls_certificate_provider_external_create( + grpc_tls_certificate_provider_external* external_provider); + +/** + * Sets the name of the root certificates being used in the distributor. + * Most users don't need to set this value. + * If not set, we will use the default name, which is an empty string. + */ GRPCAPI int grpc_tls_credentials_options_set_root_cert_name( grpc_tls_credentials_options* options, const char* root_cert_name); +/** + * Sets the name of the identity certificates being used in the distributor. + * Most users don't need to set this value. + * If not set, we will use the default name, which is an empty string. + */ GRPCAPI int grpc_tls_credentials_options_set_identity_cert_name( grpc_tls_credentials_options* options, const char* identity_cert_name); @@ -93,43 +135,12 @@ GRPCAPI int grpc_tls_credentials_options_set_certificate_provider( grpc_tls_credentials_options* options, grpc_tls_certificate_provider* provider); -/** Sets the minimum TLS version that will be negotiated during the TLS - handshake. */ -GRPCAPI void grpc_tls_server_credentials_set_min_tls_version(grpc_tls_credentials_options* options, - grpc_tls_version min_tls_version); - -/** Sets the maximum TLS version that will be negotiated during the TLS - handshake. */ -GRPCAPI void grpc_tls_server_credentials_set_max_tls_version(grpc_tls_credentials_options* options, - grpc_tls_version max_tls_version); - -typedef struct grpc_tls_authorization_check_arg grpc_tls_authorization_check_arg; - -struct grpc_tls_authorization_check_arg { - /** nothing changed; omitted */ -}; - -GRPCAPI grpc_tls_authorization_check_config* -grpc_tls_authorization_check_config_create(/** omitted */); - -grpc_channel_credentials* grpc_tls_credentials_create(grpc_tls_credentials_options* options); - -grpc_server_credentials* grpc_tls_server_credentials_create(grpc_tls_credentials_options* options); -``` - -### Provider -The provider API offers a general interface for different implementations to interact with the `distributor`. - -`grpc_tls_certificate_provider` is the provider interface we will use in `grpc_tls_credentials_options`. -Callers could directly pass in the already implemented providers, e.g. file-watcher provider, or a custom provider `grpc_tls_certificate_provider_external`, whose function pointer implementations are completely determined by the callers. - -```c -/* Opaque type. */ -typedef struct grpc_tls_certificate_provider grpc_tls_certificate_provider; -typedef struct grpc_tls_certificate_provider_external grpc_tls_certificate_provider_external; +/** ------------------------------------ Provider ------------------------------------ */ /* Factory function for file-watcher provider (implemented inside core). */ grpc_tls_certificate_provider* grpc_tls_certificate_provider_file_watcher_create(const char* private_key_file_name, const char* identity_certificate_file_name, const char* root_certificate_file_name); +/* Factory function for static file provider (implemented inside core). */ +grpc_tls_certificate_provider* grpc_tls_certificate_provider_file_static_create(const char* private_key_file_name, const char* identity_certificate_file_name, const char* root_certificate_file_name); /* API for creating external provider. */ struct grpc_tls_certificate_provider_external { @@ -137,17 +148,7 @@ struct grpc_tls_certificate_provider_external { void (*destroy)(grpc_tls_certificate_provider_external*); }; -/* The ownership of grpc_tls_certificate_provider_external will be taken by grpc_tls_certificate_provider. */ -grpc_tls_certificate_provider* grpc_tls_certificate_provider_external_create( - grpc_tls_certificate_provider_external* external_provider); - -``` - -### Distributor -The distributor is the actual component for caching and distributing the credentials to the underlying transport connections(security connectors). -```c -typedef struct grpc_tls_certificate_distributor grpc_tls_certificate_distributor; - +/** ------------------------------------ Distributor ------------------------------------ */ /* Creates a TLS certificate distributor object. This object is ref-counted. */ GRPCAPI grpc_tls_certificate_distributor* grpc_tls_certificate_distributor_create(); @@ -203,29 +204,19 @@ GRPCAPI void grpc_tls_certificate_distributor_set_watch_status_callback( grpc_tls_certificate_distributor* distributor, grpc_tls_certificate_watch_status_cb cb, void* user_data); ``` - -## Example Use - -We will provide an example showing how to put all these APIs together. Note that users wouldn't be able to interact directly with the core APIs. -Each wrap language needs to wrap these APIs in a way that meets the inherent core logic. - +If only the already implemented providers are needed, e.g. file-watcher provider or static-cert provider, those provider implementations can be directly passed in without caring about `Distributor`. +In that case, How to wrap it should be straightforward. +The Core API will be called like this: ```c -/* Create option and set basic security primitives. */ grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); -grpc_tls_credentials_options_set_server_verification_option(options, GRPC_TLS_SKIP_HOSTNAME_VERIFICATION); -grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(.....); -grpc_tls_credentials_options_set_authorization_check_config(options, config); -grpc_tls_server_credentials_set_min_tls_version(options, TLS1_2); -grpc_tls_server_credentials_set_max_tls_version(options, TLS1_3); -grpc_tls_credentials_options_set_root_cert_name(options, "root_PEM_certs"); -grpc_tls_credentials_options_set_identity_cert_name(options, "identity_PEM_certs"); -/* Use the file-based provider for reloading certificates. */ +// Use the file-based provider for reloading certificates. grpc_tls_certificate_provider* file_provider = grpc_tls_certificate_provider_file_watcher_create(...); grpc_tls_credentials_options_set_certificate_provider(options, file_provider); -/* Create TLS channel credentials. */ -grpc_channel_credentials* creds = grpc_tls_credentials_create(options); ``` -Wrap language should also allow users to define their own provider implementations: +And wrap languages can just wrap `grpc_tls_certificate_provider_file_watcher_create`. + +If the wrap language also wants users to be able to implement their own `Provider` implementations, a bit of the design is needed, because the wrapper class needs to expose part of the `Distributor` API. +The Core API will be called like this: ```c static grpc_tls_certificate_distributor* GetDistributor(grpc_tls_certificate_provider_external* external_provider) { /* .... */ @@ -238,18 +229,17 @@ grpc_tls_certificate_provider_external* external_provider = new grpc_tls_certifi external_provider->get_distributor = GetDistributor; external_provider->destroy = Destroy; /* The ownership will be taken by provider. */ -grpc_tls_certificate_provider* provider grpc_tls_certificate_provider_external_create(external_provider); - +grpc_tls_certificate_provider* my_provider = grpc_tls_certificate_provider_external_create(external_provider); +grpc_tls_credentials_options_set_certificate_provider(options, my_provider); ``` - -## Wrap Languages -Each wrap language might have their own way to wrap these core APIs, but in general, there are a few things to consider: +Each wrap language is free to choose the design suitable for its language characteristics, but consider some general advices first: 1. create an interface-like class `ProviderInterface` which contains two functions GetDistributor() and Destroy() 2. create a concrete class `FileCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_watcher_create` -3. create an interface-like class `CertificateProvider` that implements `ProviderInterface`, and exposes several APIs in the distributor to users, such as `grpc_tls_certificate_distributor_set_key_materials`, etc -4. users could extend `CertificateProvider` to define their own provider implementations +3. create a concrete class `StaticCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_static_create` +4. create an interface-like class `CertificateProvider` that implements `ProviderInterface`, and exposes several APIs in the distributor to users, such as `grpc_tls_certificate_distributor_set_key_materials`, etc +5. users could extend `CertificateProvider` to define their own provider implementations -For point 3 and 4, here is an example of `CertificateProvider` in C++: +As a reference, here is how `CertificateProvider` might be defined in C++: ```cpp namespace grpc { @@ -313,7 +303,7 @@ class CertificateProvider { } // namespace grpc ``` -and an example of custom provider implementation: +and an example of custom provider implementation in C++: ```cpp class MyCertificateProvider: public grpc::CertificateProvider { public: @@ -336,4 +326,125 @@ class MyCertificateProvider: public grpc::CertificateProvider certs_watching_; }; ``` +### Custom Authorization +```c +typedef struct grpc_tls_authorization_check_arg grpc_tls_authorization_check_arg; +typedef struct grpc_tls_authorization_check_config grpc_tls_authorization_check_config; +GRPCAPI int grpc_tls_credentials_options_set_authorization_check_config( + grpc_tls_credentials_options* options, + grpc_tls_authorization_check_config* config); + +/** ------------------------------------ Authorization Check ------------------------------------ */ + +/** callback function provided by gRPC used to handle the result of server + authorization check. It is used when schedule API is implemented + asynchronously, and serves to bring the control back to gRPC C core. */ +typedef void (*grpc_tls_on_authorization_check_done_cb)( + grpc_tls_authorization_check_arg* arg); + +/** A struct containing all information necessary to schedule/cancel a server + authorization check request. + - cb and cb_user_data represent a gRPC-provided callback and an argument + passed to it. + - success will store the result of server authorization check. That is, + if success returns a non-zero value, it means the authorization check + passes and if returning zero, it means the check fails. + - target_name is the name of an endpoint the channel is connecting to. + - peer_cert represents a complete certificate chain including both + signing and leaf certificates. + - status and error_details contain information + about errors occurred when a server authorization check request is + scheduled/cancelled. + - config is a pointer to the unique + grpc_tls_authorization_check_config instance that this argument + corresponds to. + - context is a pointer to a wrapped language implementation of this + grpc_tls_authorization_check_arg instance. + - destroy_context is a pointer to a caller-provided method that cleans + up any data associated with the context pointer. +*/ +struct grpc_tls_authorization_check_arg { + grpc_tls_on_authorization_check_done_cb cb; + void* cb_user_data; + int success; + const char* target_name; + const char* peer_cert; + const char* peer_cert_full_chain; + grpc_status_code status; + grpc_tls_error_details* error_details; + grpc_tls_authorization_check_config* config; + void* context; + void (*destroy_context)(void* ctx); +}; + +/** Create a grpc_tls_authorization_check_config instance. + - config_user_data is config-specific, read-only user data + that works for all channels created with a credential using the config. + - schedule is a pointer to an application-provided callback used to invoke + server authorization check API. The implementation of this method has to + be non-blocking, but can be performed synchronously or asynchronously. + 1)If processing occurs synchronously, it populates arg->result, + arg->status, and arg->error_details and returns zero. + 2) If processing occurs asynchronously, it returns a non-zero value. The + application then invokes arg->cb when processing is completed. Note that + arg->cb cannot be invoked before schedule API returns. + - cancel is a pointer to an application-provided callback used to cancel a + server authorization check request scheduled via an asynchronous schedule + API. arg is used to pinpoint an exact check request to be cancelled. The + operation may not have any effect if the request has already been + processed. + - destruct is a pointer to an application-provided callback used to clean up + any data associated with the config. +*/ +GRPCAPI grpc_tls_authorization_check_config* +grpc_tls_authorization_check_config_create( + const void* config_user_data, + int (*schedule)(void* config_user_data, + grpc_tls_authorization_check_arg* arg), + void (*cancel)(void* config_user_data, + grpc_tls_authorization_check_arg* arg), + void (*destruct)(void* config_user_data)); +``` +Custom authorization can be performed either synchronously or asynchronously. +Here is an example of how it could be done synchronously: +```c +int TlsAuthorizationCheckSchedule( + void* /*config_user_data*/, grpc_tls_authorization_check_arg* arg) { + if (arg->target_name == "google.com") { + arg->status = GRPC_STATUS_OK; + arg->success = 1; + return 0; + } + arg->status = GRPC_STATUS_CANCELLED; + arg->success = 0; + return 0; +} +grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(nullptr, + &TlsAuthorizationCheckSchedule, nullptr, nullptr); +grpc_tls_credentials_options_set_authorization_check_config(options, config); +``` +And example for asynchronous check: +```c +void TimeConsumingOperation(grpc_tls_authorization_check_arg* arg) { + // ... Some time-consuming operations ... + if (arg->target_name == "google.com") { + arg->status = GRPC_STATUS_OK; + arg->success = 1; + (*arg->cb)(arg); + } else { + arg->status = GRPC_STATUS_CANCELLED; + arg->success = 0; + (*arg->cb)(arg); + } +} +int TlsAuthorizationCheckSchedule( + void* /*config_user_data*/, grpc_tls_authorization_check_arg* arg) { + pthread_t thread; + pthread_create(&thread, NULL, TimeConsumingOperation, arg); + return 1; +} +grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(nullptr, + &TlsAuthorizationCheckSchedule, nullptr, nullptr); +grpc_tls_credentials_options_set_authorization_check_config(options, config); +``` \ No newline at end of file From 2e257bd542820a2b831e2d21bd3d54647f83834c Mon Sep 17 00:00:00 2001 From: ZhenLian Date: Thu, 10 Sep 2020 11:24:17 -0700 Subject: [PATCH 15/52] third edition --- L46-core-tls-credential-API.md | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 0e2e74ddc..8e9d10639 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -8,9 +8,16 @@ L46: C-core: New TLS Credentials API ## Abstract -This proposal aims to provide the following security features in a new TLS credential API: +This proposal aims to provide the following features in a new TLS credential API: 1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down -2. custom authorization check: optionally bypass the certificate check, bypass the hostname check on the client side, and perform an additional per-connection authorization callback at the end of authentication +2. custom authorization check: support the following cases + - on the client side: + - perform both the certificate and default hostname check(default option) + - perform certificate check but skip default hostname check(a custom authorization check is mandatory) + - skip both certificate check and default hostname check(a custom authorization check is mandatory) + - perform a custom authorization check, if needed + - on the server side: + - perform a custom authorization check, if needed 3. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake ## Background @@ -18,17 +25,20 @@ This proposal aims to provide the following security features in a new TLS crede The current TLS implementation in gRPC C core has some restrictions: 1. lack of the credential reloading support. Once an application starts, it can never reload its identity key/cert-chain and root certificates without shutting down 2. inflexibility in the peer verification. In a mutual TLS scenario, servers would always perform certificate verification, and clients would always perform certificate verification as well as default hostname check. No customized checks can be introduced. -3. inability to choose the TLS version that will be used +3. inability to choose the maximum/minimum TLS version supported in OpenSSL/BoringSSL for use There have always been some ad hoc requests on GitHub Issue Page for supporting these features. The demands for a more flexible and configurable API have increased with the emerging needs in cloud platform. -One common example is to support SPIFFE ID as the identity format, which requires us to disable the default hostname check on the client side. -Another example is to reload the identity certificates without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. -Hence we proposed an API that will meet the following requirements: +One common example is to support SPIFFE ID as the identity format, which requires us to disable the default hostname check on the client side. +The default hostname check is an exact check on the hostname sent from client and the identity shown on server's certificate, but in SPIFFE world, the identity on server's certificate is a SPIFEE ID, which doesn't necessarily match up with the host name. -1. Flexible enough to support multiple use cases. Users should be able to use their own implementations if the provided ones don't fit their needs -2. C core API needs to be simple, so that it can be easily wrapped by different languages -3. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per handshake +Another example is to reload the root and identity credentials without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. + +Hence we propose an API that will meet the following requirements: + +1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or authorization logic if the provided ones doesn't fit their needs +2. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection +3. C core API needs to be simple enough, so that most of existing SSL credentials in wrapped languages can migrate to call this new API ### Related Proposals: From 53440db9701f820a02fdc6e42d4724ee3da6f19e Mon Sep 17 00:00:00 2001 From: ZhenLian Date: Fri, 11 Sep 2020 10:15:26 -0700 Subject: [PATCH 16/52] small fix --- L46-core-tls-credential-API.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 8e9d10639..7f8651cfe 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -27,12 +27,14 @@ The current TLS implementation in gRPC C core has some restrictions: 2. inflexibility in the peer verification. In a mutual TLS scenario, servers would always perform certificate verification, and clients would always perform certificate verification as well as default hostname check. No customized checks can be introduced. 3. inability to choose the maximum/minimum TLS version supported in OpenSSL/BoringSSL for use -There have always been some ad hoc requests on GitHub Issue Page for supporting these features. The demands for a more flexible and configurable API have increased with the emerging needs in cloud platform. +There have always been some ad hoc requests on GitHub Issue Page for supporting these features. The demands for a more flexible and configurable API have increased with the emerging needs in the cloud platform. -One common example is to support SPIFFE ID as the identity format, which requires us to disable the default hostname check on the client side. -The default hostname check is an exact check on the hostname sent from client and the identity shown on server's certificate, but in SPIFFE world, the identity on server's certificate is a SPIFEE ID, which doesn't necessarily match up with the host name. +One common requirement is to skip the hostname verification check if a client has a different way of validating the server's identity in different platforms. +For example, it's becoming more and more popular to use SPIFFE ID as the identity format, but a valid certificate using SPIFFE ID doesn't necessarily contain the dnsName in the SAN field, which might cause hostname verification to fail. -Another example is to reload the root and identity credentials without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. +Another requirement is to reload the root and identity credentials without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. + +Beisdes requests from Open Source Community, with the growth of GCP, we've seen more and more internal use cases of these features. Hence we propose an API that will meet the following requirements: From ad7d3b05e21cded4089d46097c6494833f8609bb Mon Sep 17 00:00:00 2001 From: ZhenLian Date: Wed, 16 Sep 2020 09:52:41 -0700 Subject: [PATCH 17/52] small changes as of 09/16 --- L46-core-tls-credential-API.md | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 7f8651cfe..4370ec842 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -10,14 +10,7 @@ L46: C-core: New TLS Credentials API This proposal aims to provide the following features in a new TLS credential API: 1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down -2. custom authorization check: support the following cases - - on the client side: - - perform both the certificate and default hostname check(default option) - - perform certificate check but skip default hostname check(a custom authorization check is mandatory) - - skip both certificate check and default hostname check(a custom authorization check is mandatory) - - perform a custom authorization check, if needed - - on the server side: - - perform a custom authorization check, if needed +2. custom authorization check: perform a user-specified check at the end of authentication, and allow the client side to disable certificate verification and hostname verification check 3. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake ## Background @@ -78,14 +71,14 @@ GRPCAPI int grpc_tls_credentials_options_set_server_verification_option( grpc_tls_server_verification_option server_verification_option); /** - * Sets the minimum TLS version that will be negotiated during the TLS handshake. + * Sets the minimum supported TLS protocol version for SSL CTX. * If not set, the underlying SSL library will pick the version automatically. */ GRPCAPI void grpc_tls_credentials_options_set_min_tls_version(grpc_tls_credentials_options* options, grpc_tls_version min_tls_version); /** - * Sets the maximum TLS version that will be negotiated during the TLS handshake. + * Sets the maximum supported TLS protocol version for SSL CTX. * If not set, the underlying SSL library will pick the version automatically. */ GRPCAPI void grpc_tls_credentials_options_set_max_tls_version(grpc_tls_credentials_options* options, From 3a2486044b6115481841a299f00f90719b687e86 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Wed, 13 Mar 2024 14:17:17 +0000 Subject: [PATCH 18/52] changes to core APIs plus some provider --- L46-core-tls-credential-API.md | 146 ++++++++++++++++++++++++--------- 1 file changed, 108 insertions(+), 38 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 4370ec842..f42bb276a 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -1,24 +1,24 @@ L46: C-core: New TLS Credentials API ---- -* Author(s): ZhenLian +* Author(s): ZhenLian, gtcooke94 * Approver: markdroth, yashykt, jiangtaoli2016 * Status: Draft * Implemented in: -* Last updated: September 9, 2020 +* Last updated: March 13, 2024 ## Abstract This proposal aims to provide the following features in a new TLS credential API: 1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down -2. custom authorization check: perform a user-specified check at the end of authentication, and allow the client side to disable certificate verification and hostname verification check -3. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake +1. custom authorization check: perform a user-specified check at the end of authentication, and allow the client side to disable certificate verification and hostname verification check +1. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake ## Background The current TLS implementation in gRPC C core has some restrictions: 1. lack of the credential reloading support. Once an application starts, it can never reload its identity key/cert-chain and root certificates without shutting down -2. inflexibility in the peer verification. In a mutual TLS scenario, servers would always perform certificate verification, and clients would always perform certificate verification as well as default hostname check. No customized checks can be introduced. -3. inability to choose the maximum/minimum TLS version supported in OpenSSL/BoringSSL for use +1. inflexibility in the peer verification. In a mutual TLS scenario, servers would always perform certificate verification, and clients would always perform certificate verification as well as default hostname check. No customized checks can be introduced. +1. inability to choose the maximum/minimum TLS version supported in OpenSSL/BoringSSL for use There have always been some ad hoc requests on GitHub Issue Page for supporting these features. The demands for a more flexible and configurable API have increased with the emerging needs in the cloud platform. @@ -32,17 +32,18 @@ Beisdes requests from Open Source Community, with the growth of GCP, we've seen Hence we propose an API that will meet the following requirements: 1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or authorization logic if the provided ones doesn't fit their needs -2. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection -3. C core API needs to be simple enough, so that most of existing SSL credentials in wrapped languages can migrate to call this new API +1. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection +1. C core API needs to be simple enough, so that most of existing SSL credentials in wrapped languages can migrate to call this new API ### Related Proposals: -[L46: Create a new TLS credential API that supports SPIFFE mutual TLS](https://github.com/grpc/proposal/pull/98) +1. [L46: Create a new TLS credential API that supports SPIFFE mutual TLS](https://github.com/grpc/proposal/pull/98) - this was subsumed into this proposal. +1. [A69: Crl Providers](https://github.com/grpc/proposal/pull/382) ## Proposal ### TLS credentials and TLS Options -This part of the proposal introduces `grpc_tls_credentials_options` and TLS credentials. Users use `grpc_tls_credentials_options` to configure the specific security properties they want, and build the client/server credential by passing `grpc_tls_credentials_options` in. +This part of the proposal introduces `grpc_tls_credentials_options` and TLS credentials. Users use `grpc_tls_credentials_options` to configure the specific security properties they want, and build the client/server credential by passing in `grpc_tls_credentials_options`. This section only covers the configuration not related to credential reloading and custom verification. Configurations related to those two topics are explained in the latter sections. ```c @@ -57,46 +58,81 @@ GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(void); * Set grpc_ssl_client_certificate_request_type in credentials options. * If not set, we will use the default value GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. */ -GRPCAPI int grpc_tls_credentials_options_set_cert_request_type( +GRPCAPI void grpc_tls_credentials_options_set_cert_request_type( grpc_tls_credentials_options* options, grpc_ssl_client_certificate_request_type type); /** - * Set grpc_tls_server_verification_option. - * If not set, we will use the default value GRPC_TLS_SERVER_VERIFICATION. - * This should be called only on the client side. + * EXPERIMENTAL API - Subject to change + * + * Sets the options of whether to verify server certs on the client side. + * Passing in a non-zero value indicates verifying the certs. */ -GRPCAPI int grpc_tls_credentials_options_set_server_verification_option( - grpc_tls_credentials_options* options, - grpc_tls_server_verification_option server_verification_option); +GRPCAPI void grpc_tls_credentials_options_set_verify_server_cert( + grpc_tls_credentials_options* options, int verify_server_cert); /** - * Sets the minimum supported TLS protocol version for SSL CTX. - * If not set, the underlying SSL library will pick the version automatically. + * EXPERIMENTAL API - Subject to change + * + * Sets whether or not a TLS server should send a list of CA names in the + * ServerHello. This list of CA names is read from the server's trust bundle, so + * that the client can use this list as a hint to know which certificate it + * should send to the server. + * + * WARNING: This API is extremely dangerous and should not be used. If the + * server's trust bundle is too large, then the TLS server will be unable to + * form a ServerHello, and hence will be unusable. The definition of "too large" + * depends on the underlying SSL library being used and on the size of the CN + * fields of the certificates in the trust bundle. */ -GRPCAPI void grpc_tls_credentials_options_set_min_tls_version(grpc_tls_credentials_options* options, - grpc_tls_version min_tls_version); +GRPCAPI void grpc_tls_credentials_options_set_send_client_ca_list( + grpc_tls_credentials_options* options, bool send_client_ca_list); /** - * Sets the maximum supported TLS protocol version for SSL CTX. - * If not set, the underlying SSL library will pick the version automatically. + * EXPERIMENTAL API - Subject to change + * + * Sets the minimum TLS version that will be negotiated during the TLS + * handshake. If not set, the underlying SSL library will set it to TLS v1.2. */ -GRPCAPI void grpc_tls_credentials_options_set_max_tls_version(grpc_tls_credentials_options* options, - grpc_tls_version max_tls_version); +GRPCAPI void grpc_tls_credentials_options_set_min_tls_version( + grpc_tls_credentials_options* options, grpc_tls_version min_tls_version); /** - * This method creates a TLS channel credential object. - * It takes ownership of the options parameter. + * EXPERIMENTAL API - Subject to change + * + * Sets the maximum TLS version that will be negotiated during the TLS + * handshake. If not set, the underlying SSL library will set it to TLS v1.3. */ -grpc_channel_credentials* grpc_tls_credentials_create(grpc_tls_credentials_options* options); +GRPCAPI void grpc_tls_credentials_options_set_max_tls_version( + grpc_tls_credentials_options* options, grpc_tls_version max_tls_version); -grpc_server_credentials* grpc_tls_server_credentials_create(grpc_tls_credentials_options* options); +/** + * EXPERIMENTAL API - Subject to change + * + * Creates a TLS channel credential object based on the + * grpc_tls_credentials_options specified by callers. The + * grpc_channel_credentials will take the ownership of the |options|. The + * security level of the resulting connection is GRPC_PRIVACY_AND_INTEGRITY. + */ +grpc_channel_credentials* grpc_tls_credentials_create( + grpc_tls_credentials_options* options); + +/** + * EXPERIMENTAL API - Subject to change + * + * Creates a TLS server credential object based on the + * grpc_tls_credentials_options specified by callers. The + * grpc_server_credentials will take the ownership of the |options|. + */ +grpc_server_credentials* grpc_tls_server_credentials_create( + grpc_tls_credentials_options* options); ``` + Here is an example of how to use this API: ```c /* Create option and set basic security primitives. */ grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); -grpc_tls_credentials_options_set_server_verification_option(options, GRPC_TLS_SKIP_HOSTNAME_VERIFICATION); +grpc_tls_credentials_options_set_verify_server_cert(options, 1); grpc_tls_server_credentials_set_min_tls_version(options, TLS1_2); grpc_tls_server_credentials_set_max_tls_version(options, TLS1_3); /* Create TLS channel credentials. */ @@ -111,18 +147,24 @@ The `Provider` offers a general interface for different implementations to inter /* Opaque type. */ typedef struct grpc_tls_certificate_provider grpc_tls_certificate_provider; typedef struct grpc_tls_certificate_distributor grpc_tls_certificate_distributor; -typedef struct grpc_tls_certificate_provider_external grpc_tls_certificate_provider_external; +/** + * A struct that stores the credential data presented to the peer in handshake + * to show local identity. + */ +typedef struct grpc_tls_identity_pairs grpc_tls_identity_pairs; -/* The ownership of grpc_tls_certificate_provider_external will be taken by grpc_tls_certificate_provider. */ -grpc_tls_certificate_provider* grpc_tls_certificate_provider_external_create( - grpc_tls_certificate_provider_external* external_provider); +/** + * Creates a grpc_tls_identity_pairs that stores a list of identity credential + * data, including identity private key and identity certificate chain. + */ +GRPCAPI grpc_tls_identity_pairs* grpc_tls_identity_pairs_create(); /** * Sets the name of the root certificates being used in the distributor. * Most users don't need to set this value. * If not set, we will use the default name, which is an empty string. */ -GRPCAPI int grpc_tls_credentials_options_set_root_cert_name( +GRPCAPI void grpc_tls_credentials_options_set_root_cert_name( grpc_tls_credentials_options* options, const char* root_cert_name); @@ -131,15 +173,43 @@ GRPCAPI int grpc_tls_credentials_options_set_root_cert_name( * Most users don't need to set this value. * If not set, we will use the default name, which is an empty string. */ -GRPCAPI int grpc_tls_credentials_options_set_identity_cert_name( +GRPCAPI void grpc_tls_credentials_options_set_identity_cert_name( grpc_tls_credentials_options* options, const char* identity_cert_name); -/** Sets the credential provider. */ -GRPCAPI int grpc_tls_credentials_options_set_certificate_provider( +/** Sets the credential provider. + * Sets the credential provider in the options. + * The |options| will implicitly take a new ref to the |provider|. + */ +GRPCAPI void grpc_tls_credentials_options_set_certificate_provider( grpc_tls_credentials_options* options, grpc_tls_certificate_provider* provider); +/** + * EXPERIMENTAL API - Subject to change + * + * If set, gRPC stack will keep watching the root certificates with + * name |root_cert_name|. + * If this is not set on the client side, we will use the root certificates + * stored in the default system location, since client side must provide root + * certificates in TLS. + * If this is not set on the server side, we will not watch any root certificate + * updates, and assume no root certificates needed for the server(single-side + * TLS). Default root certs on the server side is not supported. + */ +GRPCAPI void grpc_tls_credentials_options_watch_root_certs( + grpc_tls_credentials_options* options); + +/** + * EXPERIMENTAL API - Subject to change + * + * If set, gRPC stack will keep watching the identity key-cert pairs + * with name |identity_cert_name|. + * This is required on the server side, and optional on the client side. + */ +GRPCAPI void grpc_tls_credentials_options_watch_identity_key_cert_pairs( + grpc_tls_credentials_options* options); + /** ------------------------------------ Provider ------------------------------------ */ /* Factory function for file-watcher provider (implemented inside core). */ From 40fa3c70809714ed857a545a818afdce5a9455c1 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Wed, 20 Mar 2024 19:07:37 +0000 Subject: [PATCH 19/52] external provider changes --- L46-core-tls-credential-API.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index f42bb276a..9b83a5337 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -213,16 +213,17 @@ GRPCAPI void grpc_tls_credentials_options_watch_identity_key_cert_pairs( /** ------------------------------------ Provider ------------------------------------ */ /* Factory function for file-watcher provider (implemented inside core). */ -grpc_tls_certificate_provider* grpc_tls_certificate_provider_file_watcher_create(const char* private_key_file_name, const char* identity_certificate_file_name, const char* root_certificate_file_name); -/* Factory function for static file provider (implemented inside core). */ -grpc_tls_certificate_provider* grpc_tls_certificate_provider_file_static_create(const char* private_key_file_name, const char* identity_certificate_file_name, const char* root_certificate_file_name); +GRPCAPI grpc_tls_certificate_provider* +grpc_tls_certificate_provider_file_watcher_create( + const char* private_key_path, const char* identity_certificate_path, + const char* root_cert_path, unsigned int refresh_interval_sec); -/* API for creating external provider. */ -struct grpc_tls_certificate_provider_external { - grpc_tls_certificate_distributor* (*get_distributor)(grpc_tls_certificate_provider_external*); - void (*destroy)(grpc_tls_certificate_provider_external*); -}; +/* Factory function for static file provider (implemented inside core). */ +GRPCAPI grpc_tls_certificate_provider* +grpc_tls_certificate_provider_static_data_create( + const char* root_certificate, grpc_tls_identity_pairs* pem_key_cert_pairs); +// TODO(gtcooke94) Approach to distributor, it's not currently exposed, none of these APIs exist publicly. None of this is how it got implemented. There is also no `ExternalCertificateProvider`. /** ------------------------------------ Distributor ------------------------------------ */ /* Creates a TLS certificate distributor object. This object is ref-counted. */ GRPCAPI grpc_tls_certificate_distributor* grpc_tls_certificate_distributor_create(); From f4bd8052d3b5bd7e61450e9370589639d08bea55 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 22 Mar 2024 17:34:33 +0000 Subject: [PATCH 20/52] remove layers about external certificate provider --- L46-core-tls-credential-API.md | 25 +++---------------------- 1 file changed, 3 insertions(+), 22 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 9b83a5337..d89b8f3f1 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -289,25 +289,7 @@ grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); grpc_tls_certificate_provider* file_provider = grpc_tls_certificate_provider_file_watcher_create(...); grpc_tls_credentials_options_set_certificate_provider(options, file_provider); ``` -And wrap languages can just wrap `grpc_tls_certificate_provider_file_watcher_create`. - -If the wrap language also wants users to be able to implement their own `Provider` implementations, a bit of the design is needed, because the wrapper class needs to expose part of the `Distributor` API. -The Core API will be called like this: -```c -static grpc_tls_certificate_distributor* GetDistributor(grpc_tls_certificate_provider_external* external_provider) { - /* .... */ -} -static void Destroy(grpc_tls_certificate_provider_external* external_provider) { - /* .... */ - delete external_provider; -} -grpc_tls_certificate_provider_external* external_provider = new grpc_tls_certificate_provider_external(); -external_provider->get_distributor = GetDistributor; -external_provider->destroy = Destroy; -/* The ownership will be taken by provider. */ -grpc_tls_certificate_provider* my_provider = grpc_tls_certificate_provider_external_create(external_provider); -grpc_tls_credentials_options_set_certificate_provider(options, my_provider); -``` + Each wrap language is free to choose the design suitable for its language characteristics, but consider some general advices first: 1. create an interface-like class `ProviderInterface` which contains two functions GetDistributor() and Destroy() 2. create a concrete class `FileCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_watcher_create` @@ -315,6 +297,7 @@ Each wrap language is free to choose the design suitable for its language charac 4. create an interface-like class `CertificateProvider` that implements `ProviderInterface`, and exposes several APIs in the distributor to users, such as `grpc_tls_certificate_distributor_set_key_materials`, etc 5. users could extend `CertificateProvider` to define their own provider implementations +// TODO(gtcooke94) we can directly use C++ APIs, remove external stuff from the below definitions As a reference, here is how `CertificateProvider` might be defined in C++: ```cpp namespace grpc { @@ -334,9 +317,6 @@ class CertificateProvider { grpc_tls_certificate_distributor_destroy(distributor_); } - grpc_tls_certificate_provider* c_provider() const { - return grpc_tls_certificate_provider_external_create(&base_); - } virtual void OnWatchStatusChanged( const std::string& cert_name, bool watching_root, @@ -411,6 +391,7 @@ GRPCAPI int grpc_tls_credentials_options_set_authorization_check_config( grpc_tls_credentials_options* options, grpc_tls_authorization_check_config* config); +// TODO(gtcooke94) This is really custom verification, not authorization? Change it all? /** ------------------------------------ Authorization Check ------------------------------------ */ /** callback function provided by gRPC used to handle the result of server From 880b4e0c4a04a340db87b58d03972ba694dc0c93 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 22 Mar 2024 17:35:56 +0000 Subject: [PATCH 21/52] clean up header --- L46-core-tls-credential-API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index d89b8f3f1..41a76fda0 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -1,7 +1,7 @@ L46: C-core: New TLS Credentials API ---- * Author(s): ZhenLian, gtcooke94 -* Approver: markdroth, yashykt, jiangtaoli2016 +* Approver: * Status: Draft * Implemented in: * Last updated: March 13, 2024 From 09397aa9436d3490972aafb5038f833bdc1d67e1 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 22 Mar 2024 17:40:06 +0000 Subject: [PATCH 22/52] remove EXPERIMENTAL notations in the grfc --- L46-core-tls-credential-API.md | 18 +----------------- 1 file changed, 1 insertion(+), 17 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 41a76fda0..be15a2ceb 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -4,7 +4,7 @@ L46: C-core: New TLS Credentials API * Approver: * Status: Draft * Implemented in: -* Last updated: March 13, 2024 +* Last updated: March 22, 2024 ## Abstract @@ -63,8 +63,6 @@ GRPCAPI void grpc_tls_credentials_options_set_cert_request_type( grpc_ssl_client_certificate_request_type type); /** - * EXPERIMENTAL API - Subject to change - * * Sets the options of whether to verify server certs on the client side. * Passing in a non-zero value indicates verifying the certs. */ @@ -72,8 +70,6 @@ GRPCAPI void grpc_tls_credentials_options_set_verify_server_cert( grpc_tls_credentials_options* options, int verify_server_cert); /** - * EXPERIMENTAL API - Subject to change - * * Sets whether or not a TLS server should send a list of CA names in the * ServerHello. This list of CA names is read from the server's trust bundle, so * that the client can use this list as a hint to know which certificate it @@ -89,8 +85,6 @@ GRPCAPI void grpc_tls_credentials_options_set_send_client_ca_list( grpc_tls_credentials_options* options, bool send_client_ca_list); /** - * EXPERIMENTAL API - Subject to change - * * Sets the minimum TLS version that will be negotiated during the TLS * handshake. If not set, the underlying SSL library will set it to TLS v1.2. */ @@ -98,8 +92,6 @@ GRPCAPI void grpc_tls_credentials_options_set_min_tls_version( grpc_tls_credentials_options* options, grpc_tls_version min_tls_version); /** - * EXPERIMENTAL API - Subject to change - * * Sets the maximum TLS version that will be negotiated during the TLS * handshake. If not set, the underlying SSL library will set it to TLS v1.3. */ @@ -107,8 +99,6 @@ GRPCAPI void grpc_tls_credentials_options_set_max_tls_version( grpc_tls_credentials_options* options, grpc_tls_version max_tls_version); /** - * EXPERIMENTAL API - Subject to change - * * Creates a TLS channel credential object based on the * grpc_tls_credentials_options specified by callers. The * grpc_channel_credentials will take the ownership of the |options|. The @@ -118,8 +108,6 @@ grpc_channel_credentials* grpc_tls_credentials_create( grpc_tls_credentials_options* options); /** - * EXPERIMENTAL API - Subject to change - * * Creates a TLS server credential object based on the * grpc_tls_credentials_options specified by callers. The * grpc_server_credentials will take the ownership of the |options|. @@ -186,8 +174,6 @@ GRPCAPI void grpc_tls_credentials_options_set_certificate_provider( grpc_tls_certificate_provider* provider); /** - * EXPERIMENTAL API - Subject to change - * * If set, gRPC stack will keep watching the root certificates with * name |root_cert_name|. * If this is not set on the client side, we will use the root certificates @@ -201,8 +187,6 @@ GRPCAPI void grpc_tls_credentials_options_watch_root_certs( grpc_tls_credentials_options* options); /** - * EXPERIMENTAL API - Subject to change - * * If set, gRPC stack will keep watching the identity key-cert pairs * with name |identity_cert_name|. * This is required on the server side, and optional on the client side. From 20b34f993f5d68696258ceda85efb7e931f2e1b9 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 3 May 2024 19:24:55 +0000 Subject: [PATCH 23/52] C++ify certificate provider --- L46-core-tls-credential-API.md | 94 +++++++++++++++++++++++++--------- 1 file changed, 70 insertions(+), 24 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index be15a2ceb..9f7069809 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -132,20 +132,14 @@ The credential reloading API basically consists of two parts: the top-level `Pro The `Distributor` is the actual component for caching and distributing the credentials to the underlying transport connections(security connectors). The `Provider` offers a general interface for different implementations to interact with the `Distributor`. ```c -/* Opaque type. */ -typedef struct grpc_tls_certificate_provider grpc_tls_certificate_provider; -typedef struct grpc_tls_certificate_distributor grpc_tls_certificate_distributor; -/** - * A struct that stores the credential data presented to the peer in handshake - * to show local identity. - */ -typedef struct grpc_tls_identity_pairs grpc_tls_identity_pairs; - -/** - * Creates a grpc_tls_identity_pairs that stores a list of identity credential - * data, including identity private key and identity certificate chain. - */ -GRPCAPI grpc_tls_identity_pairs* grpc_tls_identity_pairs_create(); +/* Opaque types. */ +// A struct that stores the credential data presented to the peer in handshake +// to show local identity. The private_key and certificate_chain should always +// match. +struct GRPCXX_DLL IdentityKeyCertPair { + std::string private_key; + std::string certificate_chain; +}; /** * Sets the name of the root certificates being used in the distributor. @@ -195,19 +189,71 @@ GRPCAPI void grpc_tls_credentials_options_watch_identity_key_cert_pairs( grpc_tls_credentials_options* options); /** ------------------------------------ Provider ------------------------------------ */ +/** Interface for a class that handles the process to fetch credential data. +* Implementations should be a wrapper class of an internal provider +* implementation. +*/ +class CertificateProviderInterface { + public: + virtual ~CertificateProviderInterface() = default; +}; + +// A basic CertificateProviderInterface implementation that will load credential +// data from static string during initialization. This provider will always +// return the same cert data for all cert names, and reloading is not supported. +class StaticDataCertificateProvider + : public CertificateProviderInterface { + public: + StaticDataCertificateProvider( + const std::string& root_certificate, + const std::vector& identity_key_cert_pairs); -/* Factory function for file-watcher provider (implemented inside core). */ -GRPCAPI grpc_tls_certificate_provider* -grpc_tls_certificate_provider_file_watcher_create( - const char* private_key_path, const char* identity_certificate_path, - const char* root_cert_path, unsigned int refresh_interval_sec); + explicit StaticDataCertificateProvider(const std::string& root_certificate); -/* Factory function for static file provider (implemented inside core). */ -GRPCAPI grpc_tls_certificate_provider* -grpc_tls_certificate_provider_static_data_create( - const char* root_certificate, grpc_tls_identity_pairs* pem_key_cert_pairs); + explicit StaticDataCertificateProvider( + const std::vector& identity_key_cert_pairs); + } + +// A CertificateProviderInterface implementation that will watch the credential +// changes on the file system. This provider will always return the up-to-date +// cert data for all the cert names callers set through |TlsCredentialsOptions|. +// Several things to note: +// 1. This API only supports one key-cert file and hence one set of identity +// key-cert pair, so SNI(Server Name Indication) is not supported. +// 2. The private key and identity certificate should always match. This API +// guarantees atomic read, and it is the callers' responsibility to do atomic +// updates. There are many ways to atomically update the key and certs in the +// file system. To name a few: +// 1) creating a new directory, renaming the old directory to a new name, and +// then renaming the new directory to the original name of the old directory. +// 2) using a symlink for the directory. When need to change, put new +// credential data in a new directory, and change symlink. +class GRPCXX_DLL FileWatcherCertificateProvider final + : public CertificateProviderInterface { + public: + // Constructor to get credential updates from root and identity file paths. + // + // @param private_key_path is the file path of the private key. + // @param identity_certificate_path is the file path of the identity + // certificate chain. + // @param root_cert_path is the file path to the root certificate bundle. + // @param refresh_interval_sec is the refreshing interval that we will check + // the files for updates. + FileWatcherCertificateProvider(const std::string& private_key_path, + const std::string& identity_certificate_path, + const std::string& root_cert_path, + unsigned int refresh_interval_sec); + // Constructor to get credential updates from identity file paths only. + FileWatcherCertificateProvider(const std::string& private_key_path, + const std::string& identity_certificate_path, + unsigned int refresh_interval_sec); + + // Constructor to get credential updates from root file path only. + FileWatcherCertificateProvider(const std::string& root_cert_path, + unsigned int refresh_interval_sec); + } -// TODO(gtcooke94) Approach to distributor, it's not currently exposed, none of these APIs exist publicly. None of this is how it got implemented. There is also no `ExternalCertificateProvider`. +// TODO(gtcooke94) C++ify /** ------------------------------------ Distributor ------------------------------------ */ /* Creates a TLS certificate distributor object. This object is ref-counted. */ GRPCAPI grpc_tls_certificate_distributor* grpc_tls_certificate_distributor_create(); From 3ef301b567ba023fb3a9381f8ed56dc1b2a713a3 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 3 May 2024 19:29:07 +0000 Subject: [PATCH 24/52] clean up some C++ provider impl --- L46-core-tls-credential-API.md | 80 ++++------------------------------ 1 file changed, 8 insertions(+), 72 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 9f7069809..a3a009b8f 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -310,88 +310,23 @@ GRPCAPI void grpc_tls_certificate_distributor_set_watch_status_callback( grpc_tls_certificate_distributor* distributor, grpc_tls_certificate_watch_status_cb cb, void* user_data); ``` + If only the already implemented providers are needed, e.g. file-watcher provider or static-cert provider, those provider implementations can be directly passed in without caring about `Distributor`. -In that case, How to wrap it should be straightforward. -The Core API will be called like this: -```c -grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); -// Use the file-based provider for reloading certificates. -grpc_tls_certificate_provider* file_provider = grpc_tls_certificate_provider_file_watcher_create(...); -grpc_tls_credentials_options_set_certificate_provider(options, file_provider); -``` +In that case, how to wrap it should be straightforward. -Each wrap language is free to choose the design suitable for its language characteristics, but consider some general advices first: +Each wrap language is free to choose the design suitable for its language characteristics, but consider some general advice first: 1. create an interface-like class `ProviderInterface` which contains two functions GetDistributor() and Destroy() 2. create a concrete class `FileCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_watcher_create` 3. create a concrete class `StaticCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_static_create` 4. create an interface-like class `CertificateProvider` that implements `ProviderInterface`, and exposes several APIs in the distributor to users, such as `grpc_tls_certificate_distributor_set_key_materials`, etc 5. users could extend `CertificateProvider` to define their own provider implementations -// TODO(gtcooke94) we can directly use C++ APIs, remove external stuff from the below definitions -As a reference, here is how `CertificateProvider` might be defined in C++: -```cpp -namespace grpc { - -template -class CertificateProvider { - public: - CertificateProvider() - : distributor_(grpc_tls_certificate_distributor_create()) { - base.get_distributor = GetDistributor; - base.destroy = Destroy; - grpc_tls_certificate_distributor_set_watch_status_callback( - OnWatchStatusChangedCallback, this); - } - - virtual ~CertificateProvider() { - grpc_tls_certificate_distributor_destroy(distributor_); - } - - - virtual void OnWatchStatusChanged( - const std::string& cert_name, bool watching_root, - bool watching_identity) = 0; - - void SetRootCert(const std::string& cert_name, - const std::string& contents) { - grpc_tls_certificate_distributor_set_root_certs( - distributor_, cert_name.c_str(), contents.c_str()); - } - - // ...similar functions for SetCertKeyPairs() and SetKeyMaterials()... - - private: - static grpc_tls_certificate_distributor* GetDistributor( - grpc_tls_certificate_provider_external* external_provider) { - auto* provider = static_cast(external_provider); - return provider->distributor_; - } - - static void Destroy( - grpc_tls_certificate_provider_external* external_provider) { - auto* provider = static_cast(external_provider); - delete provider; - } +// TODO(gtcooke94) we can directly use C++ APIs, remove external stuff from the below definitions, and C++ will just alias the C-core - static void OnWatchStatusChangedCallback( - void* arg, const std::string& cert_name, bool watching_root, - bool watching_identity) { - auto* provider = static_cast(arg); - provider->OnWatchStatusChanged(cert_name, watching_root, - watching_identity); - } - - // First data element to simulate inheritance. - grpc_tls_certificate_provider_external base_; - - grpc_tls_certificate_distributor* distributor_; -}; - -} // namespace grpc -``` -and an example of custom provider implementation in C++: +An example of custom provider implementation in C++: +// TODO(gtcooke94) precise semantics ```cpp -class MyCertificateProvider: public grpc::CertificateProvider { +class MyCertificateProvider: public grpc::CertificateProviderInterface { public: void OnWatchStatusChanged( const std::string& cert_name, bool watching_root, @@ -411,6 +346,7 @@ class MyCertificateProvider: public grpc::CertificateProvider certs_watching_; }; + ``` ### Custom Authorization ```c From 618adb04031a0c3ebb1e3af3644bea736e9e0aa4 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 3 May 2024 19:35:37 +0000 Subject: [PATCH 25/52] custom verifier stuff --- L46-core-tls-credential-API.md | 208 ++++++++++++++------------------- 1 file changed, 87 insertions(+), 121 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index a3a009b8f..987c803ca 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -10,7 +10,7 @@ L46: C-core: New TLS Credentials API This proposal aims to provide the following features in a new TLS credential API: 1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down -1. custom authorization check: perform a user-specified check at the end of authentication, and allow the client side to disable certificate verification and hostname verification check +1. custom verification check: perform a user-specified check at the end of authentication, and allow the client side to disable certificate verification and hostname verification check 1. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake ## Background @@ -31,7 +31,7 @@ Beisdes requests from Open Source Community, with the growth of GCP, we've seen Hence we propose an API that will meet the following requirements: -1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or authorization logic if the provided ones doesn't fit their needs +1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or verification logic if the provided ones doesn't fit their needs 1. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection 1. C core API needs to be simple enough, so that most of existing SSL credentials in wrapped languages can migrate to call this new API @@ -348,126 +348,92 @@ class MyCertificateProvider: public grpc::CertificateProviderInterface { }; ``` -### Custom Authorization -```c -typedef struct grpc_tls_authorization_check_arg grpc_tls_authorization_check_arg; -typedef struct grpc_tls_authorization_check_config grpc_tls_authorization_check_config; +### Custom Verification +```c++ +// Contains the verification-related information associated with a connection +// request. Users should not directly create or destroy this request object, but +// shall interact with it through CertificateVerifier's Verify() and Cancel(). +class TlsCustomVerificationCheckRequest { + public: + explicit TlsCustomVerificationCheckRequest( + grpc_tls_custom_verification_check_request* request); + ~TlsCustomVerificationCheckRequest() {} + + grpc::string_ref target_name() const; + grpc::string_ref peer_cert() const; + grpc::string_ref peer_cert_full_chain() const; + grpc::string_ref common_name() const; + // The subject name of the root certificate used to verify the peer chain + // If verification fails or the peer cert is self-signed, this will be an + // empty string. If verification is successful, it is a comma-separated list, + // where the entries are of the form "FIELD_ABBREVIATION=string" + // ex: "CN=testca,O=Internet Widgits Pty Ltd,ST=Some-State,C=AU" + // ex: "CN=GTS Root R1,O=Google Trust Services LLC,C=US" + grpc::string_ref verified_root_cert_subject() const; + std::vector uri_names() const; + std::vector dns_names() const; + std::vector email_names() const; + std::vector ip_names() const; -GRPCAPI int grpc_tls_credentials_options_set_authorization_check_config( - grpc_tls_credentials_options* options, - grpc_tls_authorization_check_config* config); - -// TODO(gtcooke94) This is really custom verification, not authorization? Change it all? -/** ------------------------------------ Authorization Check ------------------------------------ */ - -/** callback function provided by gRPC used to handle the result of server - authorization check. It is used when schedule API is implemented - asynchronously, and serves to bring the control back to gRPC C core. */ -typedef void (*grpc_tls_on_authorization_check_done_cb)( - grpc_tls_authorization_check_arg* arg); - -/** A struct containing all information necessary to schedule/cancel a server - authorization check request. - - cb and cb_user_data represent a gRPC-provided callback and an argument - passed to it. - - success will store the result of server authorization check. That is, - if success returns a non-zero value, it means the authorization check - passes and if returning zero, it means the check fails. - - target_name is the name of an endpoint the channel is connecting to. - - peer_cert represents a complete certificate chain including both - signing and leaf certificates. - - status and error_details contain information - about errors occurred when a server authorization check request is - scheduled/cancelled. - - config is a pointer to the unique - grpc_tls_authorization_check_config instance that this argument - corresponds to. - - context is a pointer to a wrapped language implementation of this - grpc_tls_authorization_check_arg instance. - - destroy_context is a pointer to a caller-provided method that cleans - up any data associated with the context pointer. -*/ -struct grpc_tls_authorization_check_arg { - grpc_tls_on_authorization_check_done_cb cb; - void* cb_user_data; - int success; - const char* target_name; - const char* peer_cert; - const char* peer_cert_full_chain; - grpc_status_code status; - grpc_tls_error_details* error_details; - grpc_tls_authorization_check_config* config; - void* context; - void (*destroy_context)(void* ctx); +} + +// The base class of all internal verifier implementations, and the ultimate +// class that all external verifiers will eventually be transformed into. +// To implement a custom verifier, do not extend this class; instead, +// implement a subclass of ExternalCertificateVerifier. Note that custom +// verifier implementations can compose their functionality with existing +// implementations of this interface, such as HostnameVerifier, by delegating +// to an instance of that class. +class CertificateVerifier { + public: + ~CertificateVerifier(); + + // Verifies a connection request. The check on each internal verifier could be + // either synchronous or asynchronous, and we will need to use return value to + // know. + // + // request: the verification information associated with this request + // callback: This will only take effect if the verifier is asynchronous. + // The function that gRPC will invoke when the verifier has already + // completed its asynchronous check. Callers can use this function + // to perform any additional checks. The input parameter of the + // std::function indicates the status of the verifier check. + // sync_status: This will only be useful if the verifier is synchronous. + // The status of the verifier as it has already done it's + // synchronous check. + // return: return true if executed synchronously, otherwise return false + bool Verify(TlsCustomVerificationCheckRequest* request, + std::function callback, + grpc::Status* sync_status); + + // Cancels a verification request previously started via Verify(). + // Used when the connection attempt times out or is cancelled while an async + // verification request is pending. + // + // request: the verification information associated with this request + void Cancel(TlsCustomVerificationCheckRequest* request); }; -/** Create a grpc_tls_authorization_check_config instance. - - config_user_data is config-specific, read-only user data - that works for all channels created with a credential using the config. - - schedule is a pointer to an application-provided callback used to invoke - server authorization check API. The implementation of this method has to - be non-blocking, but can be performed synchronously or asynchronously. - 1)If processing occurs synchronously, it populates arg->result, - arg->status, and arg->error_details and returns zero. - 2) If processing occurs asynchronously, it returns a non-zero value. The - application then invokes arg->cb when processing is completed. Note that - arg->cb cannot be invoked before schedule API returns. - - cancel is a pointer to an application-provided callback used to cancel a - server authorization check request scheduled via an asynchronous schedule - API. arg is used to pinpoint an exact check request to be cancelled. The - operation may not have any effect if the request has already been - processed. - - destruct is a pointer to an application-provided callback used to clean up - any data associated with the config. -*/ -GRPCAPI grpc_tls_authorization_check_config* -grpc_tls_authorization_check_config_create( - const void* config_user_data, - int (*schedule)(void* config_user_data, - grpc_tls_authorization_check_arg* arg), - void (*cancel)(void* config_user_data, - grpc_tls_authorization_check_arg* arg), - void (*destruct)(void* config_user_data)); -``` -Custom authorization can be performed either synchronously or asynchronously. -Here is an example of how it could be done synchronously: -```c -int TlsAuthorizationCheckSchedule( - void* /*config_user_data*/, grpc_tls_authorization_check_arg* arg) { - if (arg->target_name == "google.com") { - arg->status = GRPC_STATUS_OK; - arg->success = 1; - return 0; - } - arg->status = GRPC_STATUS_CANCELLED; - arg->success = 0; - return 0; -} -grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(nullptr, - &TlsAuthorizationCheckSchedule, nullptr, nullptr); -grpc_tls_credentials_options_set_authorization_check_config(options, config); +// A CertificateVerifier that doesn't perform any additional checks other than +// certificate verification, if specified. +// Note: using this solely without any other authentication mechanisms on the +// peer identity will leave your applications to the MITM(Man-In-The-Middle) +// attacks. Users should avoid doing so in production environments. +class NoOpCertificateVerifier : public CertificateVerifier { + public: + NoOpCertificateVerifier(); +}; + +// A CertificateVerifier that will perform hostname verification, to see if the +// target name set from the client side matches the identity information +// specified on the server's certificate. +class HostNameCertificateVerifier : public CertificateVerifier { + public: + HostNameCertificateVerifier(); +}; ``` -And example for asynchronous check: -```c -void TimeConsumingOperation(grpc_tls_authorization_check_arg* arg) { - // ... Some time-consuming operations ... - if (arg->target_name == "google.com") { - arg->status = GRPC_STATUS_OK; - arg->success = 1; - (*arg->cb)(arg); - } else { - arg->status = GRPC_STATUS_CANCELLED; - arg->success = 0; - (*arg->cb)(arg); - } -} -int TlsAuthorizationCheckSchedule( - void* /*config_user_data*/, grpc_tls_authorization_check_arg* arg) { - pthread_t thread; - pthread_create(&thread, NULL, TimeConsumingOperation, arg); - return 1; -} -grpc_tls_authorization_check_config* config = grpc_tls_authorization_check_config_create(nullptr, - &TlsAuthorizationCheckSchedule, nullptr, nullptr); -grpc_tls_credentials_options_set_authorization_check_config(options, config); -``` \ No newline at end of file + + + + + From fbed040a208128ed9934004bf7424ac56abe7974 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 3 May 2024 19:43:06 +0000 Subject: [PATCH 26/52] TlsCredentialOptions to C++ --- L46-core-tls-credential-API.md | 160 +++++++++++++++++---------------- 1 file changed, 84 insertions(+), 76 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 987c803ca..c17b6595a 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -1,6 +1,6 @@ L46: C-core: New TLS Credentials API ---- -* Author(s): ZhenLian, gtcooke94 +* Author(s): gtcooke94, ZhenLian * Approver: * Status: Draft * Implemented in: @@ -27,7 +27,7 @@ For example, it's becoming more and more popular to use SPIFFE ID as the identit Another requirement is to reload the root and identity credentials without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. -Beisdes requests from Open Source Community, with the growth of GCP, we've seen more and more internal use cases of these features. +Besides requests from Open Source Community, with the growth of GCP, we've seen more and more internal use cases of these features. Hence we propose an API that will meet the following requirements: @@ -43,86 +43,94 @@ Hence we propose an API that will meet the following requirements: ## Proposal ### TLS credentials and TLS Options -This part of the proposal introduces `grpc_tls_credentials_options` and TLS credentials. Users use `grpc_tls_credentials_options` to configure the specific security properties they want, and build the client/server credential by passing in `grpc_tls_credentials_options`. +This part of the proposal introduces `TlsCredentialsOptions` and TLS credentials. Users use `TlsCredentialsOptions` to configure the specific security properties they want, and build the client/server credential by passing in `TlsCredentialsOptions`. This section only covers the configuration not related to credential reloading and custom verification. Configurations related to those two topics are explained in the latter sections. -```c -/** --- TLS channel/server credentials --- - * It is used for experimental purpose for now and subject to change. */ - -typedef struct grpc_tls_credentials_options grpc_tls_credentials_options; - -GRPCAPI grpc_tls_credentials_options* grpc_tls_credentials_options_create(void); - -/** - * Set grpc_ssl_client_certificate_request_type in credentials options. - * If not set, we will use the default value GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. - */ -GRPCAPI void grpc_tls_credentials_options_set_cert_request_type( - grpc_tls_credentials_options* options, - grpc_ssl_client_certificate_request_type type); - -/** - * Sets the options of whether to verify server certs on the client side. - * Passing in a non-zero value indicates verifying the certs. - */ -GRPCAPI void grpc_tls_credentials_options_set_verify_server_cert( - grpc_tls_credentials_options* options, int verify_server_cert); - -/** - * Sets whether or not a TLS server should send a list of CA names in the - * ServerHello. This list of CA names is read from the server's trust bundle, so - * that the client can use this list as a hint to know which certificate it - * should send to the server. - * - * WARNING: This API is extremely dangerous and should not be used. If the - * server's trust bundle is too large, then the TLS server will be unable to - * form a ServerHello, and hence will be unusable. The definition of "too large" - * depends on the underlying SSL library being used and on the size of the CN - * fields of the certificates in the trust bundle. - */ -GRPCAPI void grpc_tls_credentials_options_set_send_client_ca_list( - grpc_tls_credentials_options* options, bool send_client_ca_list); - -/** - * Sets the minimum TLS version that will be negotiated during the TLS - * handshake. If not set, the underlying SSL library will set it to TLS v1.2. - */ -GRPCAPI void grpc_tls_credentials_options_set_min_tls_version( - grpc_tls_credentials_options* options, grpc_tls_version min_tls_version); - -/** - * Sets the maximum TLS version that will be negotiated during the TLS - * handshake. If not set, the underlying SSL library will set it to TLS v1.3. - */ -GRPCAPI void grpc_tls_credentials_options_set_max_tls_version( - grpc_tls_credentials_options* options, grpc_tls_version max_tls_version); - -/** - * Creates a TLS channel credential object based on the - * grpc_tls_credentials_options specified by callers. The - * grpc_channel_credentials will take the ownership of the |options|. The - * security level of the resulting connection is GRPC_PRIVACY_AND_INTEGRITY. - */ -grpc_channel_credentials* grpc_tls_credentials_create( - grpc_tls_credentials_options* options); - -/** - * Creates a TLS server credential object based on the - * grpc_tls_credentials_options specified by callers. The - * grpc_server_credentials will take the ownership of the |options|. - */ -grpc_server_credentials* grpc_tls_server_credentials_create( - grpc_tls_credentials_options* options); +```c++ +// Base class of configurable options specified by users to configure their +// certain security features supported in TLS. It is used for experimental +// purposes for now and it is subject to change. +class TlsCredentialsOptions { + public: + // Constructor for base class TlsCredentialsOptions. + // + // @param certificate_provider the provider which fetches TLS credentials that + // will be used in the TLS handshake + TlsCredentialsOptions(); + ~TlsCredentialsOptions(); + + // Copy constructor does a deep copy of the underlying pointer. No assignment + // permitted + TlsCredentialsOptions(const TlsCredentialsOptions& other); + TlsCredentialsOptions& operator=(const TlsCredentialsOptions& other) = delete; + + // ---- Setters for member fields ---- + // Sets the certificate provider used to store root certs and identity certs. + void set_certificate_provider( + std::shared_ptr certificate_provider); + // Watches the updates of root certificates with name |root_cert_name|. + // If used in TLS credentials, setting this field is optional for both the + // client side and the server side. + // If this is not set on the client side, we will use the root certificates + // stored in the default system location, since client side must provide root + // certificates in TLS(no matter single-side TLS or mutual TLS). + // If this is not set on the server side, we will not watch any root + // certificate updates, and assume no root certificates needed for the server + // (in the one-side TLS scenario, the server is not required to provide root + // certs). We don't support default root certs on server side. + void watch_root_certs(); + // Sets the name of root certificates being watched, if |watch_root_certs| is + // called. If not set, an empty string will be used as the name. + // + // @param root_cert_name the name of root certs being set. + void set_root_cert_name(const std::string& root_cert_name); + // Watches the updates of identity key-cert pairs with name + // |identity_cert_name|. If used in TLS credentials, it is required to be set + // on the server side, and optional for the client side(in the one-side + // TLS scenario, the client is not required to provide identity certs). + void watch_identity_key_cert_pairs(); + // Sets the name of identity key-cert pairs being watched, if + // |watch_identity_key_cert_pairs| is called. If not set, an empty string will + // be used as the name. + // + // @param identity_cert_name the name of identity key-cert pairs being set. + void set_identity_cert_name(const std::string& identity_cert_name); + // Sets the Tls session key logging configuration. If not set, tls + // session key logging is disabled. Note that this should be used only for + // debugging purposes. It should never be used in a production environment + // due to security concerns. + // + // @param tls_session_key_log_file_path: Path where tls session keys would + // be logged. + void set_tls_session_key_log_file_path( + const std::string& tls_session_key_log_file_path); + // Sets the certificate verifier used to perform post-handshake peer identity + // checks. + void set_certificate_verifier( + std::shared_ptr certificate_verifier); + + // Sets the crl provider, see CrlProvider for more details. + void set_crl_provider(std::shared_ptr crl_provider); + + // Sets the minimum TLS version that will be negotiated during the TLS + // handshake. If not set, the underlying SSL library will use TLS v1.2. + // @param tls_version: The minimum TLS version. + void set_min_tls_version(grpc_tls_version tls_version); + // Sets the maximum TLS version that will be negotiated during the TLS + // handshake. If not set, the underlying SSL library will use TLS v1.3. + // @param tls_version: The maximum TLS version. + void set_max_tls_version(grpc_tls_version tls_version); ``` Here is an example of how to use this API: -```c +```c++ /* Create option and set basic security primitives. */ -grpc_tls_credentials_options* options = grpc_tls_credentials_options_create(); -grpc_tls_credentials_options_set_verify_server_cert(options, 1); -grpc_tls_server_credentials_set_min_tls_version(options, TLS1_2); -grpc_tls_server_credentials_set_max_tls_version(options, TLS1_3); +TlsCredentialsOptions options = TlsCredentialsOptions(); +options.set_verify_server_cert(true); +options.set_min_tls_version(TLS1_2); +options.set_max_tls_version(TLS1_3); + +//TODO(gtcooke94 C++ify) /* Create TLS channel credentials. */ grpc_channel_credentials* creds = grpc_tls_credentials_create(options); ``` From 7947f8b5461e285251e8e0cdd57302ad63a2f1ce Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 16 May 2024 15:29:22 +0000 Subject: [PATCH 27/52] cert provider --- L46-core-tls-credential-API.md | 182 ++++++++------------------------- 1 file changed, 40 insertions(+), 142 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index c17b6595a..47f8c03ce 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -130,7 +130,7 @@ options.set_verify_server_cert(true); options.set_min_tls_version(TLS1_2); options.set_max_tls_version(TLS1_3); -//TODO(gtcooke94 C++ify) +// The core credentials APIs are still C, so this is still a grpc_channel_credential /* Create TLS channel credentials. */ grpc_channel_credentials* creds = grpc_tls_credentials_create(options); ``` @@ -149,61 +149,52 @@ struct GRPCXX_DLL IdentityKeyCertPair { std::string certificate_chain; }; -/** - * Sets the name of the root certificates being used in the distributor. - * Most users don't need to set this value. - * If not set, we will use the default name, which is an empty string. - */ -GRPCAPI void grpc_tls_credentials_options_set_root_cert_name( - grpc_tls_credentials_options* options, - const char* root_cert_name); - -/** - * Sets the name of the identity certificates being used in the distributor. - * Most users don't need to set this value. - * If not set, we will use the default name, which is an empty string. - */ -GRPCAPI void grpc_tls_credentials_options_set_identity_cert_name( - grpc_tls_credentials_options* options, - const char* identity_cert_name); - -/** Sets the credential provider. - * Sets the credential provider in the options. - * The |options| will implicitly take a new ref to the |provider|. - */ -GRPCAPI void grpc_tls_credentials_options_set_certificate_provider( - grpc_tls_credentials_options* options, - grpc_tls_certificate_provider* provider); - -/** - * If set, gRPC stack will keep watching the root certificates with - * name |root_cert_name|. - * If this is not set on the client side, we will use the root certificates - * stored in the default system location, since client side must provide root - * certificates in TLS. - * If this is not set on the server side, we will not watch any root certificate - * updates, and assume no root certificates needed for the server(single-side - * TLS). Default root certs on the server side is not supported. - */ -GRPCAPI void grpc_tls_credentials_options_watch_root_certs( - grpc_tls_credentials_options* options); - -/** - * If set, gRPC stack will keep watching the identity key-cert pairs - * with name |identity_cert_name|. - * This is required on the server side, and optional on the client side. - */ -GRPCAPI void grpc_tls_credentials_options_watch_identity_key_cert_pairs( - grpc_tls_credentials_options* options); /** ------------------------------------ Provider ------------------------------------ */ /** Interface for a class that handles the process to fetch credential data. -* Implementations should be a wrapper class of an internal provider -* implementation. */ class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; + // Provider implementations MUST provide a WatchStatusCallback that will be + // set on the distributor. Sets the TLS certificate watch status callback + // function. This will be invoked when a new certificate name is watched by a + // newly registered watcher, or when a certificate name is no longer watched + // by any watchers. Note that when the callback shows a cert is no longer + // being watched, corresponding certificate data will be deleted from the + // internal cache, and any corresponding errors will be cleared, if there are + // any. This means that if the callback subsequently says the same cert is now + // being watched again, the provider must re-provide the credentials or + // re-invoke the errors to indicate a successful or failed reloading. + // + // For the parameters in the callback function: + // cert_name The name of the certificates being watched. + // root_being_watched If the root certificates with the specific name are being + // watched. + // identity_being-Watched If the identity certificates with the specific name + // are being watched. + virtual void WatchStatusCallback(string cert_name, bool root_being_watched, bool identity_being_watched) = 0; + // Sets the key materials based on their certificate name. + // @param cert_name The name of the certificates being updated. + // @param pem_root_certs The content of root certificates. + // @param pem_key_cert_pairs The content of identity key-cert pairs. + void SetKeyMaterials( + const std::string& cert_name, absl::optional pem_root_certs, + absl::optional pem_key_cert_pairs); + // Propagates an error encountered in the provider layer to the internal TLS stack. + // + // @param cert_name The watching cert name that the caller + // wants to notify about when encountering error. + // @param root_cert_error The error that the caller encounters when reloading + // root certs. + // @param identity_cert_error The error that the caller encounters when + // reloading identity certs. + void SetErrorForCert(const std::string& cert_name, + absl::Status root_cert_error, + absl::Status identity_cert_error); + + private: + std::unique_ptr distributor_; }; // A basic CertificateProviderInterface implementation that will load credential @@ -261,99 +252,6 @@ class GRPCXX_DLL FileWatcherCertificateProvider final unsigned int refresh_interval_sec); } -// TODO(gtcooke94) C++ify -/** ------------------------------------ Distributor ------------------------------------ */ -/* Creates a TLS certificate distributor object. This object is ref-counted. */ -GRPCAPI grpc_tls_certificate_distributor* grpc_tls_certificate_distributor_create(); - -/* Unrefs a TLS certificate distributor object. */ -GRPCAPI void grpc_tls_certificate_distributor_unref(grpc_tls_certificate_distributor* distributor); - -/* Sets root certificates and key and certificate chain pairs. The Ownerships of - cert_name, pem_root_certs and pem_key_cert_pairs are not transferred. */ -GRPCAPI int grpc_tls_certificate_distributor_set_key_materials( - grpc_tls_certificate_distributor* distributor, - const char* cert_name, - const char* pem_root_certs, - const grpc_ssl_pem_key_cert_pair** pem_key_cert_pairs, size_t num_key_cert_pairs); - -/* Sets root certificates. The Ownership of cert_name and pem_root_certs are not transferred. */ -GRPCAPI int grpc_tls_certificate_distributor_set_root_certs( - grpc_tls_certificate_distributor* distributor, const char* cert_name, - const char* pem_root_certs); - -/* Sets key and certificate chain pairs. The Ownership of cert_name, and pem_key_cert_pairs are not transferred. */ -GRPCAPI int grpc_tls_certificate_distributor_set_key_cert_pairs( - grpc_tls_certificate_distributor* distributor, - const char* cert_name, - const grpc_ssl_pem_key_cert_pair** pem_key_cert_pairs, size_t num_key_cert_pairs); - -/* Sets errors for both the root and the identity certificates of name |cert_name|. - At least one of root_cert_error and identity_cert_error must be set. */ -GRPCAPI int grpc_tls_certificate_distributor_set_error_for_key_materials( - grpc_tls_certificate_distributor* distributor, - const char* cert_name, - grpc_tls_error_details* root_cert_error, - grpc_tls_error_details* identity_cert_error); - -/* Callback function to be called by the TLS certificate distributor to inform - the TLS certificate provider when the watch status changed. - - user_data is the opaque pointer that was passed to TLS certificate distributor - - watching_root_certs is a boolean value indicating that root certificates are - being watched. - - watching_key_cert_pairs is a boolean value indicating that the key certificate - pairs are being watched. - - cert_name is the name of the certificates being watched. -*/ -typedef void (*grpc_tls_certificate_watch_status_cb)( - void* user_data, const char* cert_name, bool watching_root_certs, bool watching_key_cert_pairs); - -/* Sets the watch status callback on the distributor. - Callbacks are invoked synchronously. The user_data parameter will be passed - to the callback when it is invoked. - The callback can be set to null to disable callbacks. Callers should generally - set it to null before they unref the distributor, to ensure that no subsequent - callbacks are invoked. */ -GRPCAPI void grpc_tls_certificate_distributor_set_watch_status_callback( - grpc_tls_certificate_distributor* distributor, - grpc_tls_certificate_watch_status_cb cb, void* user_data); -``` - -If only the already implemented providers are needed, e.g. file-watcher provider or static-cert provider, those provider implementations can be directly passed in without caring about `Distributor`. -In that case, how to wrap it should be straightforward. - -Each wrap language is free to choose the design suitable for its language characteristics, but consider some general advice first: -1. create an interface-like class `ProviderInterface` which contains two functions GetDistributor() and Destroy() -2. create a concrete class `FileCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_watcher_create` -3. create a concrete class `StaticCertificateProvider` that implements `ProviderInterface` and wraps the provider created by `grpc_tls_certificate_provider_file_static_create` -4. create an interface-like class `CertificateProvider` that implements `ProviderInterface`, and exposes several APIs in the distributor to users, such as `grpc_tls_certificate_distributor_set_key_materials`, etc -5. users could extend `CertificateProvider` to define their own provider implementations - -// TODO(gtcooke94) we can directly use C++ APIs, remove external stuff from the below definitions, and C++ will just alias the C-core - -An example of custom provider implementation in C++: -// TODO(gtcooke94) precise semantics -```cpp -class MyCertificateProvider: public grpc::CertificateProviderInterface { - public: - void OnWatchStatusChanged( - const std::string& cert_name, bool watching_root, - bool watching_identity) override { - if (!watching_root && !watching_identity) { - certs_watching_.erase(cert_name); - } else { - certs_watching_[cert_name] = {watching_root, watching_identity}; - // ...start or stop watching as needed... - } - } - - private: - struct CertsWatching { - bool watching_root = false; - bool watching_identity = false; - }; - std::map certs_watching_; -}; ``` ### Custom Verification From ea7920d6740b76fc08cb80802bc008888a8591fe Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 16 May 2024 15:41:39 +0000 Subject: [PATCH 28/52] add custom chain building --- L46-core-tls-credential-API.md | 36 ++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 47f8c03ce..d55b40ec6 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -120,6 +120,9 @@ class TlsCredentialsOptions { // handshake. If not set, the underlying SSL library will use TLS v1.3. // @param tls_version: The maximum TLS version. void set_max_tls_version(grpc_tls_version tls_version); + + // Sets a custom chain builder implementation that replaces the default chain building from the underlying SSL library. + void set_custom_chain_builder(std::shared_ptr chain_builder); ``` Here is an example of how to use this API: @@ -255,6 +258,8 @@ class GRPCXX_DLL FileWatcherCertificateProvider final ``` ### Custom Verification +We aim to provide distinct interfaces for chain building customization and for additional validation of an already built chain. These two features have distinctly different expected users. Doing some extra validation based on the contents of the peer certificate chain is an operation that anyone doing (m)TLS might be interested in, and it should not and does not require any deep X.509 expertise. Replacing how chain building works is a relatively rare requirement and it does require deep X.509 expertise to do correctly. We want extra validation to be easy and clear for users to add without worrying about interacting with chain building itself. On the other hand, chain building is complex to implement, and we suspect that most users will stay with the default behavior. However, to enable advanced use, such as dynamic selection of the trust bundle based on the peer certificate (e.g. SPIFFE federation), this is needed. +#### Post Handshake Verification ```c++ // Contains the verification-related information associated with a connection // request. Users should not directly create or destroy this request object, but @@ -265,21 +270,21 @@ class TlsCustomVerificationCheckRequest { grpc_tls_custom_verification_check_request* request); ~TlsCustomVerificationCheckRequest() {} - grpc::string_ref target_name() const; - grpc::string_ref peer_cert() const; - grpc::string_ref peer_cert_full_chain() const; - grpc::string_ref common_name() const; + absl::string_view target_name() const; + absl::string_view peer_cert() const; + absl::string_view peer_cert_full_chain() const; + absl::string_view common_name() const; // The subject name of the root certificate used to verify the peer chain // If verification fails or the peer cert is self-signed, this will be an // empty string. If verification is successful, it is a comma-separated list, // where the entries are of the form "FIELD_ABBREVIATION=string" // ex: "CN=testca,O=Internet Widgits Pty Ltd,ST=Some-State,C=AU" // ex: "CN=GTS Root R1,O=Google Trust Services LLC,C=US" - grpc::string_ref verified_root_cert_subject() const; - std::vector uri_names() const; - std::vector dns_names() const; - std::vector email_names() const; - std::vector ip_names() const; + absl::string_view verified_root_cert_subject() const; + std::vector uri_names() const; + std::vector dns_names() const; + std::vector email_names() const; + std::vector ip_names() const; } @@ -339,7 +344,14 @@ class HostNameCertificateVerifier : public CertificateVerifier { }; ``` +#### Custom Chain Building +```c++ +class CustomChainBuilder { +public: + virtual absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) = 0; +} +``` - - - +The default behavior for chain building is based on the underlying SSL library. For example, [X509_verify_cert](https://www.openssl.org/docs/man1.1.1/man3/X509_verify_cert.html) is the implementation in OpenSSL that builds and verifies a certificate chain. +gRPC calls that function by default [here](https://github.com/grpc/grpc/blob/2d2d5a3c411a2bade319a08085e55821cf2d5ed9/src/core/tsi/ssl_transport_security.cc#L1150). +Once the `CustomChainBuilder` is implemented, this will be where it is used. The `X509_verify_cert` call will be replaced by the `CustomChainBuilder's BuildAndVerifyChain` that fully replaces chain building. \ No newline at end of file From 1883d4c3422134387c1665f61b933598dba9248e Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 16 May 2024 15:47:14 +0000 Subject: [PATCH 29/52] small cleanup --- L46-core-tls-credential-API.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index d55b40ec6..40f9bb5f1 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -10,8 +10,9 @@ L46: C-core: New TLS Credentials API This proposal aims to provide the following features in a new TLS credential API: 1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down -1. custom verification check: perform a user-specified check at the end of authentication, and allow the client side to disable certificate verification and hostname verification check +1. custom verification behaviors: performing a user-specified check at the end of normal authentication and separately giving the ability to fully customize chain building and verification. 1. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake +1. support for certificate revocation via Certificate Revocation Lists (CRLs) ## Background @@ -160,15 +161,15 @@ class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; // Provider implementations MUST provide a WatchStatusCallback that will be - // set on the distributor. Sets the TLS certificate watch status callback - // function. This will be invoked when a new certificate name is watched by a - // newly registered watcher, or when a certificate name is no longer watched - // by any watchers. Note that when the callback shows a cert is no longer - // being watched, corresponding certificate data will be deleted from the - // internal cache, and any corresponding errors will be cleared, if there are - // any. This means that if the callback subsequently says the same cert is now - // being watched again, the provider must re-provide the credentials or - // re-invoke the errors to indicate a successful or failed reloading. + // called by the internal stack. This will be invoked when a new certificate + // name is watched by a newly registered watcher, or when a certificate name + // is no longer watched by any watchers. Note that when the callback shows a + // cert is no longer being watched, corresponding certificate data will be + // deleted from the internal cache, and any corresponding errors will be + // cleared, if there are any. This means that if the callback subsequently + // says the same cert is now being watched again, the provider must re-provide + // the credentials or re-invoke the errors to indicate a successful or failed + // reloading. // // For the parameters in the callback function: // cert_name The name of the certificates being watched. From 25764dc0fcacfe6c17ec9a4195ab452daf41cbeb Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 16 May 2024 15:47:43 +0000 Subject: [PATCH 30/52] update date --- L46-core-tls-credential-API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 40f9bb5f1..d3d44e992 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -4,7 +4,7 @@ L46: C-core: New TLS Credentials API * Approver: * Status: Draft * Implemented in: -* Last updated: March 22, 2024 +* Last updated: May 16, 2024 ## Abstract From d8e5e1d81066e5b48276f456bb1c583830f93407 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 16 May 2024 15:54:19 +0000 Subject: [PATCH 31/52] rewrite CredentialReloading summary --- L46-core-tls-credential-API.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index d3d44e992..8ffe39a91 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -140,15 +140,16 @@ grpc_channel_credentials* creds = grpc_tls_credentials_create(options); ``` ### Credential Reloading -The credential reloading API basically consists of two parts: the top-level `Provider` and the low-level `Distributor`. -The `Distributor` is the actual component for caching and distributing the credentials to the underlying transport connections(security connectors). -The `Provider` offers a general interface for different implementations to interact with the `Distributor`. +Credential reloading will be implemented via a `CertificateProvider`. This provider is responsible for sourcing key material and supplying it to the internal stack via `SetKeyMaterials`. +We provide a `StaticDataCertificateProvider` and a `FileWatcherCertificateProvider` that should cover many use cases. +If these providers do not support the intricacies for a specific use case, a user can provide their own implementations of the `CertificateProviderInterface`. + ```c /* Opaque types. */ // A struct that stores the credential data presented to the peer in handshake // to show local identity. The private_key and certificate_chain should always // match. -struct GRPCXX_DLL IdentityKeyCertPair { +struct IdentityKeyCertPair { std::string private_key; std::string certificate_chain; }; @@ -231,7 +232,7 @@ class StaticDataCertificateProvider // then renaming the new directory to the original name of the old directory. // 2) using a symlink for the directory. When need to change, put new // credential data in a new directory, and change symlink. -class GRPCXX_DLL FileWatcherCertificateProvider final +class FileWatcherCertificateProvider final : public CertificateProviderInterface { public: // Constructor to get credential updates from root and identity file paths. From 4830127b0ff14123f90f03095fe484a193a7ff4f Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 16 May 2024 19:07:33 +0000 Subject: [PATCH 32/52] add a few methods to credential options --- L46-core-tls-credential-API.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 8ffe39a91..31ebe9eb5 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -122,8 +122,18 @@ class TlsCredentialsOptions { // @param tls_version: The maximum TLS version. void set_max_tls_version(grpc_tls_version tls_version); - // Sets a custom chain builder implementation that replaces the default chain building from the underlying SSL library. - void set_custom_chain_builder(std::shared_ptr chain_builder); + // Sets a custom chain builder implementation that replaces the default chain + // building from the underlying SSL library. + void set_custom_chain_builder(std::shared_ptr + chain_builder); + + // Configures if the client should verify the server cert (mTLS). Is a no-op + // if using these options to create a server credential. + void set_verify_server_cert(bool verify_server_cert); + + // Sets the options of whether to request and/or verify client certs. This + // is a no-op if using these options to create a client credential. +void set_cert_request_type(grpc_ssl_client_certificate_request_type type); ``` Here is an example of how to use this API: From 5a73a42fca9da514c74997cd772e928f08672656 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 17 May 2024 15:05:25 +0000 Subject: [PATCH 33/52] testing github PR VSCode integration --- L46-core-tls-credential-API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 31ebe9eb5..96e2e4db5 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -8,7 +8,7 @@ L46: C-core: New TLS Credentials API ## Abstract -This proposal aims to provide the following features in a new TLS credential API: +This proposal aims to provide a new TLS credentials API with the following features: 1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down 1. custom verification behaviors: performing a user-specified check at the end of normal authentication and separately giving the ability to fully customize chain building and verification. 1. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake From 8400c11b7077eca579fe8739e5533907f28c3b67 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 17 May 2024 15:52:34 +0000 Subject: [PATCH 34/52] address PR comments --- L46-core-tls-credential-API.md | 89 +++++++++++++++++++++------------- 1 file changed, 55 insertions(+), 34 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 96e2e4db5..977c8a73a 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -9,32 +9,24 @@ L46: C-core: New TLS Credentials API ## Abstract This proposal aims to provide a new TLS credentials API with the following features: -1. credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down -1. custom verification behaviors: performing a user-specified check at the end of normal authentication and separately giving the ability to fully customize chain building and verification. -1. TLS version configuration: optionally configure the minimum and maximum of the TLS version that will be used in the handshake -1. support for certificate revocation via Certificate Revocation Lists (CRLs) +1. Credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down. +1. Customizable verification of the peer certificate: perform custom validation checks on the peer's certificate chain after it has been verified to chain up to a root of trust, or fully customize path building and cryptographic verification. +1. TLS version configuration: optionally configure the minimum and maximum of the TLS versions to be negotiated during the handshake. +1. Support for certificate revocation via Certificate Revocation Lists (CRLs). ## Background -The current TLS implementation in gRPC C core has some restrictions: -1. lack of the credential reloading support. Once an application starts, it can never reload its identity key/cert-chain and root certificates without shutting down -1. inflexibility in the peer verification. In a mutual TLS scenario, servers would always perform certificate verification, and clients would always perform certificate verification as well as default hostname check. No customized checks can be introduced. -1. inability to choose the maximum/minimum TLS version supported in OpenSSL/BoringSSL for use +The current TLS implementation (the SSL credentials API) in gRPC C core has some restrictions: +1. Lack of the credential reloading support. Once an application starts, it can never reload its identity key/cert-chain and root certificates without shutting down. +1. Inflexibility in the peer verification. There is no API to add additional custom verification checks. +1. Inability to choose the maximum/minimum TLS version to be negotiated during the TLS handshake. +1. No support for certificate revocation. -There have always been some ad hoc requests on GitHub Issue Page for supporting these features. The demands for a more flexible and configurable API have increased with the emerging needs in the cloud platform. +We propose an API that will meet the following requirements: -One common requirement is to skip the hostname verification check if a client has a different way of validating the server's identity in different platforms. -For example, it's becoming more and more popular to use SPIFFE ID as the identity format, but a valid certificate using SPIFFE ID doesn't necessarily contain the dnsName in the SAN field, which might cause hostname verification to fail. - -Another requirement is to reload the root and identity credentials without shutting down the server, because certificates will eventually expire and the ability to rotate the certificates is essential to a cloud application. - -Besides requests from Open Source Community, with the growth of GCP, we've seen more and more internal use cases of these features. - -Hence we propose an API that will meet the following requirements: - -1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or verification logic if the provided ones doesn't fit their needs -1. The credentials reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection -1. C core API needs to be simple enough, so that most of existing SSL credentials in wrapped languages can migrate to call this new API +1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or verification logic if the provided ones do not fit their needs. +1. The credential reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection. +1. C core API needs to be simple enough, so that most of existing SSL credentials in wrapped languages can migrate to call this new API. ### Related Proposals: @@ -44,8 +36,7 @@ Hence we propose an API that will meet the following requirements: ## Proposal ### TLS credentials and TLS Options -This part of the proposal introduces `TlsCredentialsOptions` and TLS credentials. Users use `TlsCredentialsOptions` to configure the specific security properties they want, and build the client/server credential by passing in `TlsCredentialsOptions`. -This section only covers the configuration not related to credential reloading and custom verification. Configurations related to those two topics are explained in the latter sections. +This part of the proposal introduces `TlsCredentialsOptions` and TLS credentials. Users use `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions` to configure the specific security properties they want, and build the client/server credential by passing in those options. ```c++ // Base class of configurable options specified by users to configure their @@ -126,27 +117,57 @@ class TlsCredentialsOptions { // building from the underlying SSL library. void set_custom_chain_builder(std::shared_ptr chain_builder); +} - // Configures if the client should verify the server cert (mTLS). Is a no-op - // if using these options to create a server credential. - void set_verify_server_cert(bool verify_server_cert); +// Contains configurable options on the server side. +class TlsServerCredentialsOptions final : public TlsCredentialsOptions { + public: + // Server side is required to use a provider, because server always needs to + // use identity certs. + explicit TlsServerCredentialsOptions( + std::shared_ptr certificate_provider) + : TlsCredentialsOptions() { + set_certificate_provider(certificate_provider); + } + + + // Sets option to request the certificates from the client. + // The default is GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. + void set_cert_request_type( + grpc_ssl_client_certificate_request_type cert_request_type); +} + +// Contains configurable options on the client side. +// Client side doesn't need to always use certificate provider. When the +// certificate provider is not set, we will use the root certificates stored +// in the system default locations, and assume client won't provide any +// identity certificates(single side TLS). +class TlsChannelCredentialsOptions final : public TlsCredentialsOptions { + public: + // Sets the decision of whether to do a crypto check on the server certs. + // The default is true. + void set_verify_server_certs(bool verify_server_certs); + +// Builds TLS Credentials given TLS options. +std::shared_ptr TlsCredentials( + const TlsChannelCredentialsOptions& options); +} - // Sets the options of whether to request and/or verify client certs. This - // is a no-op if using these options to create a client credential. -void set_cert_request_type(grpc_ssl_client_certificate_request_type type); +std::shared_ptr TlsServerCredentials( + const grpc::experimental::TlsServerCredentialsOptions& options); ``` Here is an example of how to use this API: ```c++ /* Create option and set basic security primitives. */ -TlsCredentialsOptions options = TlsCredentialsOptions(); +TlsChannelCredentialsOptions options = TlsChannelCredentialsOptions(); options.set_verify_server_cert(true); options.set_min_tls_version(TLS1_2); options.set_max_tls_version(TLS1_3); // The core credentials APIs are still C, so this is still a grpc_channel_credential /* Create TLS channel credentials. */ -grpc_channel_credentials* creds = grpc_tls_credentials_create(options); +std::shared_ptr creds = TlsCredentials(options); ``` ### Credential Reloading @@ -157,8 +178,8 @@ If these providers do not support the intricacies for a specific use case, a use ```c /* Opaque types. */ // A struct that stores the credential data presented to the peer in handshake -// to show local identity. The private_key and certificate_chain should always -// match. +// to show local identity. The private_key and certificate_chain must be PEM +// encoded and should always match. struct IdentityKeyCertPair { std::string private_key; std::string certificate_chain; @@ -188,7 +209,7 @@ class CertificateProviderInterface { // watched. // identity_being-Watched If the identity certificates with the specific name // are being watched. - virtual void WatchStatusCallback(string cert_name, bool root_being_watched, bool identity_being_watched) = 0; + virtual void WatchStatusCallback(std::string cert_name, bool root_being_watched, bool identity_being_watched) = 0; // Sets the key materials based on their certificate name. // @param cert_name The name of the certificates being updated. // @param pem_root_certs The content of root certificates. From ea3f164e2eae6bc6f3c38a76b9969f151f82ab30 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Mon, 20 May 2024 15:53:11 +0000 Subject: [PATCH 35/52] address PR comments --- L46-core-tls-credential-API.md | 203 +++++++++++++++++++-------------- 1 file changed, 119 insertions(+), 84 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 977c8a73a..88de57ad9 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -17,12 +17,12 @@ This proposal aims to provide a new TLS credentials API with the following featu ## Background The current TLS implementation (the SSL credentials API) in gRPC C core has some restrictions: -1. Lack of the credential reloading support. Once an application starts, it can never reload its identity key/cert-chain and root certificates without shutting down. -1. Inflexibility in the peer verification. There is no API to add additional custom verification checks. +1. Lack of the credential reloading support. Once an application starts, it can never reload its private key, certificate chain conveying the application's identity, and root certificates without shutting down. +1. Inflexibility of how the peer certificate chain is verified. There is no API to add additional custom verification checks. 1. Inability to choose the maximum/minimum TLS version to be negotiated during the TLS handshake. 1. No support for certificate revocation. -We propose an API that will meet the following requirements: +We propose an API that supports the above features and meets the following requirements: 1. Flexible enough to support multiple use cases. Users should be able to write their own reloading or verification logic if the provided ones do not fit their needs. 1. The credential reloading should be efficient. We shall reload only when needed rather than reloading per TLS connection. @@ -36,68 +36,74 @@ We propose an API that will meet the following requirements: ## Proposal ### TLS credentials and TLS Options -This part of the proposal introduces `TlsCredentialsOptions` and TLS credentials. Users use `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions` to configure the specific security properties they want, and build the client/server credential by passing in those options. +This part of the proposal introduces the base options class, called `TlsCredentialsOptions`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsOptions`, called `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. ```c++ // Base class of configurable options specified by users to configure their -// certain security features supported in TLS. It is used for experimental -// purposes for now and it is subject to change. +// certain security features supported in TLS. class TlsCredentialsOptions { public: - // Constructor for base class TlsCredentialsOptions. - // - // @param certificate_provider the provider which fetches TLS credentials that - // will be used in the TLS handshake - TlsCredentialsOptions(); ~TlsCredentialsOptions(); - // Copy constructor does a deep copy of the underlying pointer. No assignment - // permitted + // Copy constructor does a deep copy. No assignment permitted. TlsCredentialsOptions(const TlsCredentialsOptions& other); + TlsCredentialsOptions& operator=(const TlsCredentialsOptions& other) = delete; // ---- Setters for member fields ---- - // Sets the certificate provider used to store root certs and identity certs. + // Sets the certificate provider used to store root certificates and the + // certificate chain conveying the application's identity and corresponding + // private key. void set_certificate_provider( std::shared_ptr certificate_provider); - // Watches the updates of root certificates with name |root_cert_name|. - // If used in TLS credentials, setting this field is optional for both the - // client side and the server side. + + // Watches the updates of root certificates with name |root_cert_name| (set by + // the user via set_root_cert_name). If used in TLS credentials, setting this + // field is optional for both the client side and the server side. // If this is not set on the client side, we will use the root certificates // stored in the default system location, since client side must provide root - // certificates in TLS(no matter single-side TLS or mutual TLS). + // certificates in TLS (no matter single-side TLS or mutual TLS). // If this is not set on the server side, we will not watch any root // certificate updates, and assume no root certificates needed for the server // (in the one-side TLS scenario, the server is not required to provide root - // certs). We don't support default root certs on server side. - void watch_root_certs(); - // Sets the name of root certificates being watched, if |watch_root_certs| is - // called. If not set, an empty string will be used as the name. - // - // @param root_cert_name the name of root certs being set. + // certificates). We don't support default root certs on server side. + void watch_root_certificates(); + + // Sets the name of root certificates being watched, if |watch_root_certificates| is + // called. If not set, an empty string will be used as the name. For simple + // use cases this is not needed, but a certificate provider could be + // implemented that watches many sets of certificates. This allows the user to + // differentiate between different root certificates. + // + // @param root_cert_name the name of root certificates being set. void set_root_cert_name(const std::string& root_cert_name); - // Watches the updates of identity key-cert pairs with name + + // Watches the updates of identity key-certificate pairs with name // |identity_cert_name|. If used in TLS credentials, it is required to be set // on the server side, and optional for the client side(in the one-side - // TLS scenario, the client is not required to provide identity certs). + // TLS scenario, the client is not required to provide identity certificates). void watch_identity_key_cert_pairs(); - // Sets the name of identity key-cert pairs being watched, if + + // Sets the name of identity key-certificate pairs being watched, if // |watch_identity_key_cert_pairs| is called. If not set, an empty string will // be used as the name. // - // @param identity_cert_name the name of identity key-cert pairs being set. + // @param identity_cert_name the name of identity key-certificate pairs being set. void set_identity_cert_name(const std::string& identity_cert_name); - // Sets the Tls session key logging configuration. If not set, tls + + // Sets the TLS session key logging file path. If not set, TLS // session key logging is disabled. Note that this should be used only for // debugging purposes. It should never be used in a production environment // due to security concerns. // - // @param tls_session_key_log_file_path: Path where tls session keys would + // @param tls_session_key_log_file_path: Path where TLS session keys should // be logged. void set_tls_session_key_log_file_path( const std::string& tls_session_key_log_file_path); - // Sets the certificate verifier used to perform post-handshake peer identity - // checks. + + // Sets the certificate verifier. The certificate verifier performs checks on + // the peer certificate chain after the chain has been (cryptographically) + // verified to chain up to a trusted root. void set_certificate_verifier( std::shared_ptr certificate_verifier); @@ -105,25 +111,30 @@ class TlsCredentialsOptions { void set_crl_provider(std::shared_ptr crl_provider); // Sets the minimum TLS version that will be negotiated during the TLS - // handshake. If not set, the underlying SSL library will use TLS v1.2. + // handshake. If not set, the underlying SSL library will default to TLS v1.2. // @param tls_version: The minimum TLS version. void set_min_tls_version(grpc_tls_version tls_version); + // Sets the maximum TLS version that will be negotiated during the TLS - // handshake. If not set, the underlying SSL library will use TLS v1.3. + // handshake. If not set, the underlying SSL library will default to TLS v1.3. // @param tls_version: The maximum TLS version. void set_max_tls_version(grpc_tls_version tls_version); // Sets a custom chain builder implementation that replaces the default chain - // building from the underlying SSL library. + // building from the underlying SSL library. Fully replacing and implementing + // chain building is a complex task and has dangerous security implications if + // done wrong, thus this API is inteded for expert use only. void set_custom_chain_builder(std::shared_ptr chain_builder); } -// Contains configurable options on the server side. +// Server-specific options for configuring TLS. class TlsServerCredentialsOptions final : public TlsCredentialsOptions { public: - // Server side is required to use a provider, because server always needs to - // use identity certs. + // A certificate provider that provides identity credentials is required, + // because a server must always present identity credentials during any TLS + // handshake. The certificate provider may optionally provide root + // certificates, in case the server requests client certificates. explicit TlsServerCredentialsOptions( std::shared_ptr certificate_provider) : TlsCredentialsOptions() { @@ -131,33 +142,40 @@ class TlsServerCredentialsOptions final : public TlsCredentialsOptions { } - // Sets option to request the certificates from the client. - // The default is GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. + // Sets requirements for if client certificates are requested, if they + // are required, and if the client certificate must be trusted. The + // default is GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. void set_cert_request_type( grpc_ssl_client_certificate_request_type cert_request_type); } -// Contains configurable options on the client side. -// Client side doesn't need to always use certificate provider. When the -// certificate provider is not set, we will use the root certificates stored -// in the system default locations, and assume client won't provide any -// identity certificates(single side TLS). +// Client-specific options for configuring TLS. +// +// A client may optionally set a certificate provider. +// If there is no certificate provider, the system default root certificates will be used to +// verify server certificates. +// If a certificate provider is set and it provides root certifices that root +// will be used. +// If a certificate provider is set and it provides identity credentials those +// identity credentials will be used. class TlsChannelCredentialsOptions final : public TlsCredentialsOptions { public: - // Sets the decision of whether to do a crypto check on the server certs. + // Sets the decision of whether to do a crypto check on the server certificates. // The default is true. - void set_verify_server_certs(bool verify_server_certs); + void set_verify_server_certificates(bool verify_server_certs); +} -// Builds TLS Credentials given TLS options. +// Builds a ChannelCredentials instance that establishes TLS connections in the manner specified by options. std::shared_ptr TlsCredentials( const TlsChannelCredentialsOptions& options); } +// Builds a ServerCredentials instance that establishes TLS connections in the manner specified by options. std::shared_ptr TlsServerCredentials( - const grpc::experimental::TlsServerCredentialsOptions& options); + const TlsServerCredentialsOptions& options); ``` -Here is an example of how to use this API: +Here is an example of how to configure channel credentials: ```c++ /* Create option and set basic security primitives. */ TlsChannelCredentialsOptions options = TlsChannelCredentialsOptions(); @@ -165,44 +183,49 @@ options.set_verify_server_cert(true); options.set_min_tls_version(TLS1_2); options.set_max_tls_version(TLS1_3); -// The core credentials APIs are still C, so this is still a grpc_channel_credential /* Create TLS channel credentials. */ std::shared_ptr creds = TlsCredentials(options); ``` ### Credential Reloading -Credential reloading will be implemented via a `CertificateProvider`. This provider is responsible for sourcing key material and supplying it to the internal stack via `SetKeyMaterials`. -We provide a `StaticDataCertificateProvider` and a `FileWatcherCertificateProvider` that should cover many use cases. -If these providers do not support the intricacies for a specific use case, a user can provide their own implementations of the `CertificateProviderInterface`. +Credential reloading will be implemented via a `CertificateProviderInterface`. +This provider is responsible for sourcing key material and supplying it to the +internal stack via the `CertificateProvider`'s `SetKeyMaterials` API. Two +reference implementations of the `CertificateProviderInterface` are provided - +`StaticDataCertificateProvider` and `FileWatcherCertificateProvider`. The +`StaticDataCertificateProvider` provides certificates from raw string data. The +`FileWatcherCertificateProvider` will periodically and asynchronously refresh +certificate data from a file, allowing changes to certificates without +restarting the client/server. More detail about these reference implementations +is provided below. These implementations should cover many use cases. If these +providers do not support the intricacies for a specific use case, a user can +provide their own implementations of the `CertificateProviderInterface`. ```c /* Opaque types. */ // A struct that stores the credential data presented to the peer in handshake // to show local identity. The private_key and certificate_chain must be PEM -// encoded and should always match. +// encoded and the public key in the leaf certificate must correspond to the +// given private key. struct IdentityKeyCertPair { - std::string private_key; - std::string certificate_chain; + std::string private_key_pem; + std::string certificate_chain_pem; }; /** ------------------------------------ Provider ------------------------------------ */ -/** Interface for a class that handles the process to fetch credential data. +/** Provides identity credentials and root certificates. */ class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; // Provider implementations MUST provide a WatchStatusCallback that will be // called by the internal stack. This will be invoked when a new certificate - // name is watched by a newly registered watcher, or when a certificate name - // is no longer watched by any watchers. Note that when the callback shows a - // cert is no longer being watched, corresponding certificate data will be - // deleted from the internal cache, and any corresponding errors will be - // cleared, if there are any. This means that if the callback subsequently - // says the same cert is now being watched again, the provider must re-provide - // the credentials or re-invoke the errors to indicate a successful or failed - // reloading. - // + // name is starting to be used internally, or when a certificate name is no + // longer being used internally. When being invoked for a new certificate, + // this callback should call `SetKeyMaterials` to do the initial population + // the certificate data in the internal stack. + // // For the parameters in the callback function: // cert_name The name of the certificates being watched. // root_being_watched If the root certificates with the specific name are being @@ -210,21 +233,23 @@ class CertificateProviderInterface { // identity_being-Watched If the identity certificates with the specific name // are being watched. virtual void WatchStatusCallback(std::string cert_name, bool root_being_watched, bool identity_being_watched) = 0; + // Sets the key materials based on their certificate name. // @param cert_name The name of the certificates being updated. - // @param pem_root_certs The content of root certificates. - // @param pem_key_cert_pairs The content of identity key-cert pairs. + // @param pem_root_certificates The content of root certificates. + // @param pem_key_cert_pairs The content of identity key-certificate pairs. void SetKeyMaterials( - const std::string& cert_name, absl::optional pem_root_certs, + const std::string& cert_name, absl::optional pem_root_certificates, absl::optional pem_key_cert_pairs); + // Propagates an error encountered in the provider layer to the internal TLS stack. // - // @param cert_name The watching cert name that the caller + // @param cert_name The watching certificate name that the caller // wants to notify about when encountering error. // @param root_cert_error The error that the caller encounters when reloading - // root certs. + // root certificates. // @param identity_cert_error The error that the caller encounters when - // reloading identity certs. + // reloading identity certificates. void SetErrorForCert(const std::string& cert_name, absl::Status root_cert_error, absl::Status identity_cert_error); @@ -235,7 +260,7 @@ class CertificateProviderInterface { // A basic CertificateProviderInterface implementation that will load credential // data from static string during initialization. This provider will always -// return the same cert data for all cert names, and reloading is not supported. +// return the same certificate data for all cert names, and reloading is not supported. class StaticDataCertificateProvider : public CertificateProviderInterface { public: @@ -251,13 +276,13 @@ class StaticDataCertificateProvider // A CertificateProviderInterface implementation that will watch the credential // changes on the file system. This provider will always return the up-to-date -// cert data for all the cert names callers set through |TlsCredentialsOptions|. +// certificate data for all the cert names callers set through |TlsCredentialsOptions|. // Several things to note: -// 1. This API only supports one key-cert file and hence one set of identity -// key-cert pair, so SNI(Server Name Indication) is not supported. +// 1. This API only supports one key-certificate file and hence one set of identity +// key-certificate pair, so SNI(Server Name Indication) is not supported. // 2. The private key and identity certificate should always match. This API // guarantees atomic read, and it is the callers' responsibility to do atomic -// updates. There are many ways to atomically update the key and certs in the +// updates. There are many ways to atomically update the key and certificates in the // file system. To name a few: // 1) creating a new directory, renaming the old directory to a new name, and // then renaming the new directory to the original name of the old directory. @@ -278,6 +303,7 @@ class FileWatcherCertificateProvider final const std::string& identity_certificate_path, const std::string& root_cert_path, unsigned int refresh_interval_sec); + // Constructor to get credential updates from identity file paths only. FileWatcherCertificateProvider(const std::string& private_key_path, const std::string& identity_certificate_path, @@ -297,6 +323,7 @@ We aim to provide distinct interfaces for chain building customization and for a // Contains the verification-related information associated with a connection // request. Users should not directly create or destroy this request object, but // shall interact with it through CertificateVerifier's Verify() and Cancel(). +// TODO(gtcooke94) Add expected formats for all string values here. class TlsCustomVerificationCheckRequest { public: explicit TlsCustomVerificationCheckRequest( @@ -308,7 +335,7 @@ class TlsCustomVerificationCheckRequest { absl::string_view peer_cert_full_chain() const; absl::string_view common_name() const; // The subject name of the root certificate used to verify the peer chain - // If verification fails or the peer cert is self-signed, this will be an + // If verification fails or the peer certificate is self-signed, this will be an // empty string. If verification is successful, it is a comma-separated list, // where the entries are of the form "FIELD_ABBREVIATION=string" // ex: "CN=testca,O=Internet Widgits Pty Ltd,ST=Some-State,C=AU" @@ -347,8 +374,8 @@ class CertificateVerifier { // synchronous check. // return: return true if executed synchronously, otherwise return false bool Verify(TlsCustomVerificationCheckRequest* request, - std::function callback, - grpc::Status* sync_status); + std::function callback, + Status* sync_status); // Cancels a verification request previously started via Verify(). // Used when the connection attempt times out or is cancelled while an async @@ -379,12 +406,20 @@ class HostNameCertificateVerifier : public CertificateVerifier { #### Custom Chain Building ```c++ -class CustomChainBuilder { +class CustomChainBuilderInterface { public: - virtual absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) = 0; + // TODO(gtcooke94) - more detail here on output specifics. + // The output chain should consist of DER encoded certs ordered in the same way + // that Open/BoringSSL build chains in `X509_verify_cert` including the root of + // the trusted chain. + virtual absl::StatusOr> + BuildAndVerifyChain(const std::vector& peer_cert_chain_der) + = 0; + virtual ~CustomChainBuilderInterface() = 0; } ``` The default behavior for chain building is based on the underlying SSL library. For example, [X509_verify_cert](https://www.openssl.org/docs/man1.1.1/man3/X509_verify_cert.html) is the implementation in OpenSSL that builds and verifies a certificate chain. gRPC calls that function by default [here](https://github.com/grpc/grpc/blob/2d2d5a3c411a2bade319a08085e55821cf2d5ed9/src/core/tsi/ssl_transport_security.cc#L1150). -Once the `CustomChainBuilder` is implemented, this will be where it is used. The `X509_verify_cert` call will be replaced by the `CustomChainBuilder's BuildAndVerifyChain` that fully replaces chain building. \ No newline at end of file +Once the `CustomChainBuilder` is implemented, this will be where it is used. The `X509_verify_cert` call will be replaced by the `CustomChainBuilder's BuildAndVerifyChain` that fully replaces chain building. +Replacing chain building is very complex and can open dangerous security holes if done wrong. Most users should not expect to use this API. However, there exist use cases for such behavior - for example, building chains from SPIFFE IDs and trust maps. \ No newline at end of file From 82ed957937bedfbbcddf062203ffa3224fc6c655 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Mon, 20 May 2024 20:21:08 +0000 Subject: [PATCH 36/52] add decoupled option for certificate provider --- L46-core-tls-credential-API.md | 43 ++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 88de57ad9..97031fefd 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -258,6 +258,49 @@ class CertificateProviderInterface { std::unique_ptr distributor_; }; +/** ------------------------------------ Provider ------------------------------------ */ +/** Provides identity credentials and root certificates. +*/ +// This represents a potential decoupling of roots and identity chains, with +// further extension points for something like SPIFFE bundles +class CertificateProviderInterface { + public: + virtual ~CertificateProviderInterface() = default; + // Provider implementations MUST provide a LoadAndSetCredentialsCallback that + // will be called by the internal stack. This will be invoked when a new + // certificate name is starting to be used internally, or when a certificate + // name is no longer being used internally. When being invoked for a new + // certificate, this callback should call SetRootCertificates or + // SetIdentityChainAndPrivateKey to do the initial population the certificate + // data in the internal stack. + // + // For the parameters in the callback function: cert_name The name of the + // certificates being watched. type The type of certificates being watched. + virtual void LoadAndSetCredentialsCallback(std::string name, TODOType type) = 0; + + // Must be called after the constructor. + // Does important internal setup steps. + virtual void Init() final; + + enum TODOType { + RootCertificates, + IdentityChainAndPrivateKey + } + +protected: + // Sets the root certificates based on their name. + virtual void SetRootCertificates(const std::string& name, std::string pem_root_certificates) final; + + // Sets the identity chain and private key based on their name. + virtual void SetIdentityChainAndPrivateKey(const std::string& name, grpc_core::PemKeyCertPairList pem_key_cert_pairs) final; + + // Propagates an error encountered in the provider layer to the internal TLS stack. + virtual void SetError(const std::string& name, TODOType type, absl::Status error) final; + + private: + std::unique_ptr distributor_; +}; + // A basic CertificateProviderInterface implementation that will load credential // data from static string during initialization. This provider will always // return the same certificate data for all cert names, and reloading is not supported. From 36d9e1caea43a8ac7039fc4303ace5f2bc3bfcc9 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Mon, 20 May 2024 20:31:21 +0000 Subject: [PATCH 37/52] address PR comments not related to cert provider interface --- L46-core-tls-credential-API.md | 37 +++++++++++++++------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 97031fefd..fec8fb6c9 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -39,8 +39,7 @@ We propose an API that supports the above features and meets the following requi This part of the proposal introduces the base options class, called `TlsCredentialsOptions`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsOptions`, called `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. ```c++ -// Base class of configurable options specified by users to configure their -// certain security features supported in TLS. +// Base class of TLS options. class TlsCredentialsOptions { public: ~TlsCredentialsOptions(); @@ -91,11 +90,14 @@ class TlsCredentialsOptions { // @param identity_cert_name the name of identity key-certificate pairs being set. void set_identity_cert_name(const std::string& identity_cert_name); + // WARNING: EXPERT USE ONLY. MISUSE CAN LEAD TO SIGNIFICANT SECURITY DEGRADATION. + // // Sets the TLS session key logging file path. If not set, TLS // session key logging is disabled. Note that this should be used only for // debugging purposes. It should never be used in a production environment - // due to security concerns. - // + // due to security concerns - anyone who can obtain the (logged) session key + // can decrypt all traffic on a connection. + // // @param tls_session_key_log_file_path: Path where TLS session keys should // be logged. void set_tls_session_key_log_file_path( @@ -120,6 +122,8 @@ class TlsCredentialsOptions { // @param tls_version: The maximum TLS version. void set_max_tls_version(grpc_tls_version tls_version); + // WARNING: EXPERT USE ONLY. MISUSE CAN LEAD TO SIGNIFICANT SECURITY DEGRADATION. + // // Sets a custom chain builder implementation that replaces the default chain // building from the underlying SSL library. Fully replacing and implementing // chain building is a complex task and has dangerous security implications if @@ -143,8 +147,9 @@ class TlsServerCredentialsOptions final : public TlsCredentialsOptions { // Sets requirements for if client certificates are requested, if they - // are required, and if the client certificate must be trusted. The - // default is GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE. + // are required, and if the client certificate must be trusted. The + // default is GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE, which represents + // normal TLS. void set_cert_request_type( grpc_ssl_client_certificate_request_type cert_request_type); } @@ -175,29 +180,19 @@ std::shared_ptr TlsServerCredentials( const TlsServerCredentialsOptions& options); ``` -Here is an example of how to configure channel credentials: -```c++ -/* Create option and set basic security primitives. */ -TlsChannelCredentialsOptions options = TlsChannelCredentialsOptions(); -options.set_verify_server_cert(true); -options.set_min_tls_version(TLS1_2); -options.set_max_tls_version(TLS1_3); - -/* Create TLS channel credentials. */ -std::shared_ptr creds = TlsCredentials(options); -``` - ### Credential Reloading Credential reloading will be implemented via a `CertificateProviderInterface`. -This provider is responsible for sourcing key material and supplying it to the +Note that there is a naming mismatch here - there exist credentials beyond +certificates that this _could_ theoretically support. This provider is +responsible for sourcing key material and supplying it to the internal stack via the `CertificateProvider`'s `SetKeyMaterials` API. Two reference implementations of the `CertificateProviderInterface` are provided - `StaticDataCertificateProvider` and `FileWatcherCertificateProvider`. The `StaticDataCertificateProvider` provides certificates from raw string data. The `FileWatcherCertificateProvider` will periodically and asynchronously refresh certificate data from a file, allowing changes to certificates without -restarting the client/server. More detail about these reference implementations -is provided below. These implementations should cover many use cases. If these +restarting the process. More details about these reference implementations +are provided below. These implementations should cover many use cases. If these providers do not support the intricacies for a specific use case, a user can provide their own implementations of the `CertificateProviderInterface`. From 6b01a3227a69f69a671ddb0960c9290956ac7a87 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 21 May 2024 14:56:23 +0000 Subject: [PATCH 38/52] address PR comments --- L46-core-tls-credential-API.md | 105 ++++++++++++++++++++++----------- 1 file changed, 71 insertions(+), 34 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index fec8fb6c9..b3735873c 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -9,7 +9,7 @@ L46: C-core: New TLS Credentials API ## Abstract This proposal aims to provide a new TLS credentials API with the following features: -1. Credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without shutting down. +1. Credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without restarting the process. 1. Customizable verification of the peer certificate: perform custom validation checks on the peer's certificate chain after it has been verified to chain up to a root of trust, or fully customize path building and cryptographic verification. 1. TLS version configuration: optionally configure the minimum and maximum of the TLS versions to be negotiated during the handshake. 1. Support for certificate revocation via Certificate Revocation Lists (CRLs). @@ -50,9 +50,9 @@ class TlsCredentialsOptions { TlsCredentialsOptions& operator=(const TlsCredentialsOptions& other) = delete; // ---- Setters for member fields ---- - // Sets the certificate provider used to store root certificates and the - // certificate chain conveying the application's identity and corresponding - // private key. + // Sets the certificate provider which is used to store: + // 1. The root certificates used to (cryptographically) verify peer certificate chains. + // 2. The certificate chain conveying the application's identity and the corresponding private key. void set_certificate_provider( std::shared_ptr certificate_provider); @@ -180,6 +180,11 @@ std::shared_ptr TlsServerCredentials( const TlsServerCredentialsOptions& options); ``` +To make migration from the `SslCredentials` stack to the `TlsCredentials` stack easier, +we will also have a constructor for `TlsChannelCredentialsOptions` and +`TlsServerCredentialsOptions` that accept their corresponding `SslOptions` +structs. + ### Credential Reloading Credential reloading will be implemented via a `CertificateProviderInterface`. Note that there is a naming mismatch here - there exist credentials beyond @@ -199,7 +204,7 @@ provide their own implementations of the `CertificateProviderInterface`. ```c /* Opaque types. */ // A struct that stores the credential data presented to the peer in handshake -// to show local identity. The private_key and certificate_chain must be PEM +// to show local identity. The private key and certificate chain must be PEM // encoded and the public key in the leaf certificate must correspond to the // given private key. struct IdentityKeyCertPair { @@ -214,6 +219,7 @@ struct IdentityKeyCertPair { class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; + // Provider implementations MUST provide a WatchStatusCallback that will be // called by the internal stack. This will be invoked when a new certificate // name is starting to be used internally, or when a certificate name is no @@ -354,24 +360,41 @@ class FileWatcherCertificateProvider final ``` -### Custom Verification -We aim to provide distinct interfaces for chain building customization and for additional validation of an already built chain. These two features have distinctly different expected users. Doing some extra validation based on the contents of the peer certificate chain is an operation that anyone doing (m)TLS might be interested in, and it should not and does not require any deep X.509 expertise. Replacing how chain building works is a relatively rare requirement and it does require deep X.509 expertise to do correctly. We want extra validation to be easy and clear for users to add without worrying about interacting with chain building itself. On the other hand, chain building is complex to implement, and we suspect that most users will stay with the default behavior. However, to enable advanced use, such as dynamic selection of the trust bundle based on the peer certificate (e.g. SPIFFE federation), this is needed. -#### Post Handshake Verification +### Customizable Verification of the Peer Certificate + +Users should be able to customize verification of the peer certificate, and we provide two distinct interfaces for how this can be done: + +1. Allow users to perform custom validation checks on the peer's certificate chain after it has been verified to chain up to a root of trust, but before the TLS handshake completes. This is a common customization that might be needed whenever a private PKI is involved, and does not require the user to have any significant domain knowledge. + +2. Allow users to fully customize path building and cryptographic verification. This is a relatively rare requirement from users and requires deep X.509 expertise to do correctly and without introducing significant security vulnerabilities. However, it is unavoidable in order to unblock some advanced use cases, e.g. SPIFFE federation. + +The next sections explain these interfaces in more detail. + +#### Custom Validation Checks on the Peer Certificate Chain ```c++ -// Contains the verification-related information associated with a connection -// request. Users should not directly create or destroy this request object, but -// shall interact with it through CertificateVerifier's Verify() and Cancel(). -// TODO(gtcooke94) Add expected formats for all string values here. +// Contains the information from the (verified) peer certificate chain that can +// be used to perform custom validation checks. Users should not directly +// create or destroy this request object, but shall interact with it through +// CertificateVerifier's Verify() and Cancel(). +// TODO(gtcooke94) Add expected +// formats for all string values here. class TlsCustomVerificationCheckRequest { public: - explicit TlsCustomVerificationCheckRequest( - grpc_tls_custom_verification_check_request* request); - ~TlsCustomVerificationCheckRequest() {} - + // The target hostname that is expected to appear in the server leaf + // certicate, e.g. github.com. Empty on the server-side. absl::string_view target_name() const; + + // The PEM-encoded leaf cert. absl::string_view peer_cert() const; + + // The PEM-encoded unverified peer cert chain, ordered from leaf to root, but + // not including the root. absl::string_view peer_cert_full_chain() const; + + // The common name string from the Subject extension in the peer's leaf + // certificate. absl::string_view common_name() const; + // The subject name of the root certificate used to verify the peer chain // If verification fails or the peer certificate is self-signed, this will be an // empty string. If verification is successful, it is a comma-separated list, @@ -379,23 +402,31 @@ class TlsCustomVerificationCheckRequest { // ex: "CN=testca,O=Internet Widgits Pty Ltd,ST=Some-State,C=AU" // ex: "CN=GTS Root R1,O=Google Trust Services LLC,C=US" absl::string_view verified_root_cert_subject() const; + + // The (possibly empty) list of URI names from the Subject Alternative Name + // extension in the peer's leaf certificate. std::vector uri_names() const; + + // The (possibly empty) list of DNS names from the Subject Alternative Name + // extension in the peer's leaf certificate. std::vector dns_names() const; + + // The (possibly empty) list of email names from the Subject Alternative Name + // extension in the peer's leaf certificate. std::vector email_names() const; - std::vector ip_names() const; + // The (possibly empty) list of IP names from the Subject Alternative Name + // extension in the peer's leaf certificate. + std::vector ip_names() const; } -// The base class of all internal verifier implementations, and the ultimate -// class that all external verifiers will eventually be transformed into. -// To implement a custom verifier, do not extend this class; instead, -// implement a subclass of ExternalCertificateVerifier. Note that custom +// The base class of all verifier implementations. Note that custom // verifier implementations can compose their functionality with existing // implementations of this interface, such as HostnameVerifier, by delegating // to an instance of that class. class CertificateVerifier { public: - ~CertificateVerifier(); + virtual ~CertificateVerifier() = default; // Verifies a connection request. The check on each internal verifier could be // either synchronous or asynchronous, and we will need to use return value to @@ -446,18 +477,24 @@ class HostNameCertificateVerifier : public CertificateVerifier { ```c++ class CustomChainBuilderInterface { public: - // TODO(gtcooke94) - more detail here on output specifics. - // The output chain should consist of DER encoded certs ordered in the same way - // that Open/BoringSSL build chains in `X509_verify_cert` including the root of - // the trusted chain. - virtual absl::StatusOr> - BuildAndVerifyChain(const std::vector& peer_cert_chain_der) - = 0; - virtual ~CustomChainBuilderInterface() = 0; + virtual ~CustomChainBuilderInterface() = default; + + // Returns the verified DER-encoded certificate chain, ordered from leaf to + // root. Both the leaf and the root are included. Returns an error status if + // verification fails for any reason. + virtual absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) = 0; } ``` -The default behavior for chain building is based on the underlying SSL library. For example, [X509_verify_cert](https://www.openssl.org/docs/man1.1.1/man3/X509_verify_cert.html) is the implementation in OpenSSL that builds and verifies a certificate chain. -gRPC calls that function by default [here](https://github.com/grpc/grpc/blob/2d2d5a3c411a2bade319a08085e55821cf2d5ed9/src/core/tsi/ssl_transport_security.cc#L1150). -Once the `CustomChainBuilder` is implemented, this will be where it is used. The `X509_verify_cert` call will be replaced by the `CustomChainBuilder's BuildAndVerifyChain` that fully replaces chain building. -Replacing chain building is very complex and can open dangerous security holes if done wrong. Most users should not expect to use this API. However, there exist use cases for such behavior - for example, building chains from SPIFFE IDs and trust maps. \ No newline at end of file +The default behavior for chain building is to defer to the underlying SSL library, which has a built-in chain building API that has been hardened over many years of use with the web PKI. For example, [X509_verify_cert](https://www.openssl.org/docs/man1.1.1/man3/X509_verify_cert.html) is the implementation in OpenSSL that builds a chain and verifies that it is trusted by one of the root certificates. + +```c++ +class SPIFFEFederationChainBuilder : public CustomChainBuilderInterface { +public: + ~SPIFFEFederationChainBuilder() override; + + // Builds a trusted and validated certificate chain based on SPIFFE trust maps + // rather than the standard X509 approach. + absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) override; +} +``` \ No newline at end of file From 17305967cbfe7739affe20aa1a3a6b8680a6b811 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 21 May 2024 15:11:04 +0000 Subject: [PATCH 39/52] formatting --- L46-core-tls-credential-API.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index b3735873c..4bbf6e039 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -9,10 +9,11 @@ L46: C-core: New TLS Credentials API ## Abstract This proposal aims to provide a new TLS credentials API with the following features: + 1. Credential reloading: reload transport credentials periodically from various sources, e.g. from credential files, or users' customized Provider implementations, without restarting the process. 1. Customizable verification of the peer certificate: perform custom validation checks on the peer's certificate chain after it has been verified to chain up to a root of trust, or fully customize path building and cryptographic verification. 1. TLS version configuration: optionally configure the minimum and maximum of the TLS versions to be negotiated during the handshake. -1. Support for certificate revocation via Certificate Revocation Lists (CRLs). +1. Support for certificate revocation via Certificate Revocation Lists (CRLs). ## Background From ff55e27acba824a238e831f2bfda0c265a5c84bf Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 21 May 2024 17:17:26 +0000 Subject: [PATCH 40/52] pr review --- L46-core-tls-credential-API.md | 77 ++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 37 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 4bbf6e039..1ab730012 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -36,7 +36,7 @@ We propose an API that supports the above features and meets the following requi ## Proposal -### TLS credentials and TLS Options +### TLS Credentials and TLS Options This part of the proposal introduces the base options class, called `TlsCredentialsOptions`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsOptions`, called `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. ```c++ @@ -181,26 +181,24 @@ std::shared_ptr TlsServerCredentials( const TlsServerCredentialsOptions& options); ``` -To make migration from the `SslCredentials` stack to the `TlsCredentials` stack easier, -we will also have a constructor for `TlsChannelCredentialsOptions` and -`TlsServerCredentialsOptions` that accept their corresponding `SslOptions` -structs. +To make migration from the `SslCredentials` API to the `TlsCredentials` API easier, we will also have a constructor for `TlsChannelCredentialsOptions` that accepts the `SslCredentialsOptions` +struct. Similarly for `TlsServerCredentialsOptions`. ### Credential Reloading Credential reloading will be implemented via a `CertificateProviderInterface`. Note that there is a naming mismatch here - there exist credentials beyond certificates that this _could_ theoretically support. This provider is -responsible for sourcing key material and supplying it to the -internal stack via the `CertificateProvider`'s `SetKeyMaterials` API. Two -reference implementations of the `CertificateProviderInterface` are provided - +responsible for sourcing key material and supplying it to the internal stack via +the `CertificateProvider`'s `SetKeyMaterials` API. Two reference +implementations of the `CertificateProviderInterface` are provided - `StaticDataCertificateProvider` and `FileWatcherCertificateProvider`. The `StaticDataCertificateProvider` provides certificates from raw string data. The `FileWatcherCertificateProvider` will periodically and asynchronously refresh certificate data from a file, allowing changes to certificates without -restarting the process. More details about these reference implementations -are provided below. These implementations should cover many use cases. If these -providers do not support the intricacies for a specific use case, a user can -provide their own implementations of the `CertificateProviderInterface`. +restarting the process. More details about these reference implementations are +provided below. These implementations should cover many common use cases. If +these providers do not support the intricacies for a specific use case, a user +can provide their own implementations of the `CertificateProviderInterface`. ```c /* Opaque types. */ @@ -216,7 +214,11 @@ struct IdentityKeyCertPair { /** ------------------------------------ Provider ------------------------------------ */ /** Provides identity credentials and root certificates. -*/ +/* To enable the provider to manage multiple sets of credentials of each type, + * each set of credentials must be given a "name". The "name" is an opaque + * user-defined label, and the provider will reference this "name" e.g. in logs + * and error messages. + */ class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; @@ -278,13 +280,13 @@ class CertificateProviderInterface { // // For the parameters in the callback function: cert_name The name of the // certificates being watched. type The type of certificates being watched. - virtual void LoadAndSetCredentialsCallback(std::string name, TODOType type) = 0; + virtual void LoadAndSetCredentialsCallback(std::string name, CredentialType type) = 0; // Must be called after the constructor. // Does important internal setup steps. virtual void Init() final; - enum TODOType { + enum CredentialType { RootCertificates, IdentityChainAndPrivateKey } @@ -297,7 +299,7 @@ protected: virtual void SetIdentityChainAndPrivateKey(const std::string& name, grpc_core::PemKeyCertPairList pem_key_cert_pairs) final; // Propagates an error encountered in the provider layer to the internal TLS stack. - virtual void SetError(const std::string& name, TODOType type, absl::Status error) final; + virtual void SetError(const std::string& name, CredentialType type, absl::Status error) final; private: std::unique_ptr distributor_; @@ -375,10 +377,7 @@ The next sections explain these interfaces in more detail. ```c++ // Contains the information from the (verified) peer certificate chain that can // be used to perform custom validation checks. Users should not directly -// create or destroy this request object, but shall interact with it through -// CertificateVerifier's Verify() and Cancel(). -// TODO(gtcooke94) Add expected -// formats for all string values here. +// create or destroy this request object. class TlsCustomVerificationCheckRequest { public: // The target hostname that is expected to appear in the server leaf @@ -423,15 +422,18 @@ class TlsCustomVerificationCheckRequest { // The base class of all verifier implementations. Note that custom // verifier implementations can compose their functionality with existing -// implementations of this interface, such as HostnameVerifier, by delegating +// implementations of this interface, such as HostNameVerifier, by delegating // to an instance of that class. -class CertificateVerifier { +class CertificateVerifierInterface { public: - virtual ~CertificateVerifier() = default; - - // Verifies a connection request. The check on each internal verifier could be - // either synchronous or asynchronous, and we will need to use return value to - // know. + virtual ~CertificateVerifierInterface() = default; + + // Performs a custom verification check. + // Returns true if verification occurs synchronously, in which case the + // verification result is populated in the sync_status output parameter. + // Returns false if verification occurs asynchronously, in which case the + // callback must be called when verification is complete and the verification + // result must be populated in the sync_status output parameter. // // request: the verification information associated with this request // callback: This will only take effect if the verifier is asynchronous. @@ -443,16 +445,16 @@ class CertificateVerifier { // The status of the verifier as it has already done it's // synchronous check. // return: return true if executed synchronously, otherwise return false - bool Verify(TlsCustomVerificationCheckRequest* request, + virtual bool Verify(TlsCustomVerificationCheckRequest* request, std::function callback, - Status* sync_status); + Status* sync_status) = 0; // Cancels a verification request previously started via Verify(). // Used when the connection attempt times out or is cancelled while an async // verification request is pending. // // request: the verification information associated with this request - void Cancel(TlsCustomVerificationCheckRequest* request); + virtual void Cancel(TlsCustomVerificationCheckRequest* request) = 0; }; // A CertificateVerifier that doesn't perform any additional checks other than @@ -460,7 +462,7 @@ class CertificateVerifier { // Note: using this solely without any other authentication mechanisms on the // peer identity will leave your applications to the MITM(Man-In-The-Middle) // attacks. Users should avoid doing so in production environments. -class NoOpCertificateVerifier : public CertificateVerifier { +class NoOpCertificateVerifier : public CertificateVerifierInterface { public: NoOpCertificateVerifier(); }; @@ -468,7 +470,9 @@ class NoOpCertificateVerifier : public CertificateVerifier { // A CertificateVerifier that will perform hostname verification, to see if the // target name set from the client side matches the identity information // specified on the server's certificate. -class HostNameCertificateVerifier : public CertificateVerifier { +// This will be used as the default verifier if no verification is otherwise +// set. +class HostNameCertificateVerifier : public CertificateVerifierInterface { public: HostNameCertificateVerifier(); }; @@ -488,14 +492,13 @@ public: ``` The default behavior for chain building is to defer to the underlying SSL library, which has a built-in chain building API that has been hardened over many years of use with the web PKI. For example, [X509_verify_cert](https://www.openssl.org/docs/man1.1.1/man3/X509_verify_cert.html) is the implementation in OpenSSL that builds a chain and verifies that it is trusted by one of the root certificates. +One use case for the custom chain building feature is to enable SPIFFE federation (TODO: ref the SPIFFE spec), where the set of root certificates to use is determined based on the SPIFFE trust domain in the peer's leaf certificate. ```c++ -class SPIFFEFederationChainBuilder : public CustomChainBuilderInterface { -public: - ~SPIFFEFederationChainBuilder() override; - // Builds a trusted and validated certificate chain based on SPIFFE trust maps // rather than the standard X509 approach. - absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) override; +class SpiffeFederationChainBuilder : public CustomChainBuilderInterface { +public: + ~SpiffeFederationChainBuilder() override; } ``` \ No newline at end of file From 1ce4080904486abd8a7357b817fe381dfe63aad5 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 24 May 2024 16:54:45 +0000 Subject: [PATCH 41/52] cert provider changes from review --- L46-core-tls-credential-API.md | 133 +++++++++++++-------------------- 1 file changed, 52 insertions(+), 81 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 1ab730012..9f257ab3d 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -1,4 +1,4 @@ -L46: C-core: New TLS Credentials API +L46: C++: New TLS Credentials API ---- * Author(s): gtcooke94, ZhenLian * Approver: @@ -35,6 +35,7 @@ We propose an API that supports the above features and meets the following requi 1. [A69: Crl Providers](https://github.com/grpc/proposal/pull/382) ## Proposal +Until recently, C-Core needed to be written in C89, then we had C++ wrappers of those APIs in `grpcpp`. Requirements have changed such that we can use C++ directly in Core. The rest of this proposal is written as C++ that will be implemented in core, and simple aliases to everything will be written in the `grpcpp`. ### TLS Credentials and TLS Options This part of the proposal introduces the base options class, called `TlsCredentialsOptions`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsOptions`, called `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. @@ -76,7 +77,7 @@ class TlsCredentialsOptions { // differentiate between different root certificates. // // @param root_cert_name the name of root certificates being set. - void set_root_cert_name(const std::string& root_cert_name); + void set_root_cert_name(absl::string_view root_cert_name); // Watches the updates of identity key-certificate pairs with name // |identity_cert_name|. If used in TLS credentials, it is required to be set @@ -89,7 +90,7 @@ class TlsCredentialsOptions { // be used as the name. // // @param identity_cert_name the name of identity key-certificate pairs being set. - void set_identity_cert_name(const std::string& identity_cert_name); + void set_identity_cert_name(absl::string_view identity_cert_name); // WARNING: EXPERT USE ONLY. MISUSE CAN LEAD TO SIGNIFICANT SECURITY DEGRADATION. // @@ -102,7 +103,7 @@ class TlsCredentialsOptions { // @param tls_session_key_log_file_path: Path where TLS session keys should // be logged. void set_tls_session_key_log_file_path( - const std::string& tls_session_key_log_file_path); + absl::string_view tls_session_key_log_file_path); // Sets the certificate verifier. The certificate verifier performs checks on // the peer certificate chain after the chain has been (cryptographically) @@ -129,7 +130,7 @@ class TlsCredentialsOptions { // building from the underlying SSL library. Fully replacing and implementing // chain building is a complex task and has dangerous security implications if // done wrong, thus this API is inteded for expert use only. - void set_custom_chain_builder(std::shared_ptr + void set_custom_chain_builder(std::shared_ptr chain_builder); } @@ -181,8 +182,12 @@ std::shared_ptr TlsServerCredentials( const TlsServerCredentialsOptions& options); ``` -To make migration from the `SslCredentials` API to the `TlsCredentials` API easier, we will also have a constructor for `TlsChannelCredentialsOptions` that accepts the `SslCredentialsOptions` -struct. Similarly for `TlsServerCredentialsOptions`. +`TlsCredentials` features should represent a superset of `SslCredentials` +features, so we anticipate `TlsCredentials` being the API of choice now. To +make migration from the `SslCredentials` API to the `TlsCredentials` API easier, +we will also have a method on `SslCredentialsOptions` that creates a +`TlsChannelCredentialsOptions` with the same settings. We will do similarly for +`TlsServerCredentialsOptions`. ### Credential Reloading Credential reloading will be implemented via a `CertificateProviderInterface`. @@ -211,57 +216,6 @@ struct IdentityKeyCertPair { std::string certificate_chain_pem; }; - -/** ------------------------------------ Provider ------------------------------------ */ -/** Provides identity credentials and root certificates. -/* To enable the provider to manage multiple sets of credentials of each type, - * each set of credentials must be given a "name". The "name" is an opaque - * user-defined label, and the provider will reference this "name" e.g. in logs - * and error messages. - */ -class CertificateProviderInterface { - public: - virtual ~CertificateProviderInterface() = default; - - // Provider implementations MUST provide a WatchStatusCallback that will be - // called by the internal stack. This will be invoked when a new certificate - // name is starting to be used internally, or when a certificate name is no - // longer being used internally. When being invoked for a new certificate, - // this callback should call `SetKeyMaterials` to do the initial population - // the certificate data in the internal stack. - // - // For the parameters in the callback function: - // cert_name The name of the certificates being watched. - // root_being_watched If the root certificates with the specific name are being - // watched. - // identity_being-Watched If the identity certificates with the specific name - // are being watched. - virtual void WatchStatusCallback(std::string cert_name, bool root_being_watched, bool identity_being_watched) = 0; - - // Sets the key materials based on their certificate name. - // @param cert_name The name of the certificates being updated. - // @param pem_root_certificates The content of root certificates. - // @param pem_key_cert_pairs The content of identity key-certificate pairs. - void SetKeyMaterials( - const std::string& cert_name, absl::optional pem_root_certificates, - absl::optional pem_key_cert_pairs); - - // Propagates an error encountered in the provider layer to the internal TLS stack. - // - // @param cert_name The watching certificate name that the caller - // wants to notify about when encountering error. - // @param root_cert_error The error that the caller encounters when reloading - // root certificates. - // @param identity_cert_error The error that the caller encounters when - // reloading identity certificates. - void SetErrorForCert(const std::string& cert_name, - absl::Status root_cert_error, - absl::Status identity_cert_error); - - private: - std::unique_ptr distributor_; -}; - /** ------------------------------------ Provider ------------------------------------ */ /** Provides identity credentials and root certificates. */ @@ -270,6 +224,19 @@ class CertificateProviderInterface { class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; + + // Must be called after the constructor. + // Does important internal setup steps. + void Start(); + + // The case of a SPIFFE trust bundle still falls into RootCertificates, it's + // just another way of representing root information + enum CredentialType { + RootCertificates, + IdentityChainAndPrivateKey + } + +protected: // Provider implementations MUST provide a LoadAndSetCredentialsCallback that // will be called by the internal stack. This will be invoked when a new // certificate name is starting to be used internally, or when a certificate @@ -280,26 +247,30 @@ class CertificateProviderInterface { // // For the parameters in the callback function: cert_name The name of the // certificates being watched. type The type of certificates being watched. + // TODO(gtcooke94) adjust naming of functionality of this method to match previous WatchStatusCallback, it needs to handle both beginning to watch and ending watching. virtual void LoadAndSetCredentialsCallback(std::string name, CredentialType type) = 0; - // Must be called after the constructor. - // Does important internal setup steps. - virtual void Init() final; - - enum CredentialType { - RootCertificates, - IdentityChainAndPrivateKey - } - -protected: // Sets the root certificates based on their name. - virtual void SetRootCertificates(const std::string& name, std::string pem_root_certificates) final; + // This value is layered and represents the following. + // The top level `absl::StatusOr` represents setting an error or a value. If + // the input is a status, it will be propagated across the internal stack as + // an error. + // The next layer is an `absl::optional`. This allows the user to set a value + // or `absl::nullopt`, with `absl::nullopt` representing a deletion/un-setting + // of the root certificate data. + // The last layer is an `absl::variant`. This is an extension point for us to + // add other kinds of root information, for example SPIFFE trust bundles. + void SetRootCertificates(absl::string_view name, absl::StatusOr>> root_data); // Sets the identity chain and private key based on their name. - virtual void SetIdentityChainAndPrivateKey(const std::string& name, grpc_core::PemKeyCertPairList pem_key_cert_pairs) final; - - // Propagates an error encountered in the provider layer to the internal TLS stack. - virtual void SetError(const std::string& name, CredentialType type, absl::Status error) final; + // This value is layered and represents the following. + // The top level `absl::StatusOr` represents setting an error or a value. If + // the input is a status, it will be propagated across the internal stack as + // an error. + // The next layer is an `absl::optional`. This allows the user to set a value + // or `absl::nullopt`, with `absl::nullopt` representing a deletion/un-setting + // of the identity chain and private key data. + void SetIdentityChainAndPrivateKey(absl::string_view name, grpc_core::PemKeyCertPairList pem_key_cert_pairs); private: std::unique_ptr distributor_; @@ -312,10 +283,10 @@ class StaticDataCertificateProvider : public CertificateProviderInterface { public: StaticDataCertificateProvider( - const std::string& root_certificate, + absl::string_view root_certificate, const std::vector& identity_key_cert_pairs); - explicit StaticDataCertificateProvider(const std::string& root_certificate); + explicit StaticDataCertificateProvider(absl::string_view root_certificate); explicit StaticDataCertificateProvider( const std::vector& identity_key_cert_pairs); @@ -346,18 +317,18 @@ class FileWatcherCertificateProvider final // @param root_cert_path is the file path to the root certificate bundle. // @param refresh_interval_sec is the refreshing interval that we will check // the files for updates. - FileWatcherCertificateProvider(const std::string& private_key_path, - const std::string& identity_certificate_path, - const std::string& root_cert_path, + FileWatcherCertificateProvider(absl::string_view private_key_path, + absl::string_view identity_certificate_path, + absl::string_view root_cert_path, unsigned int refresh_interval_sec); // Constructor to get credential updates from identity file paths only. - FileWatcherCertificateProvider(const std::string& private_key_path, - const std::string& identity_certificate_path, + FileWatcherCertificateProvider(absl::string_view private_key_path, + absl::string_view identity_certificate_path, unsigned int refresh_interval_sec); // Constructor to get credential updates from root file path only. - FileWatcherCertificateProvider(const std::string& root_cert_path, + FileWatcherCertificateProvider(absl::string_view root_cert_path, unsigned int refresh_interval_sec); } From 163f4f900f234f6c41161ac4d523b3cdfbe168a5 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 24 May 2024 17:22:50 +0000 Subject: [PATCH 42/52] vectors to spans --- L46-core-tls-credential-API.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 9f257ab3d..2fe4068e4 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -284,12 +284,12 @@ class StaticDataCertificateProvider public: StaticDataCertificateProvider( absl::string_view root_certificate, - const std::vector& identity_key_cert_pairs); + const absl::Span& identity_key_cert_pairs); explicit StaticDataCertificateProvider(absl::string_view root_certificate); explicit StaticDataCertificateProvider( - const std::vector& identity_key_cert_pairs); + const absl::Span& identity_key_cert_pairs); } // A CertificateProviderInterface implementation that will watch the credential @@ -376,19 +376,19 @@ class TlsCustomVerificationCheckRequest { // The (possibly empty) list of URI names from the Subject Alternative Name // extension in the peer's leaf certificate. - std::vector uri_names() const; + absl::Span uri_names() const; // The (possibly empty) list of DNS names from the Subject Alternative Name // extension in the peer's leaf certificate. - std::vector dns_names() const; + absl::Span dns_names() const; // The (possibly empty) list of email names from the Subject Alternative Name // extension in the peer's leaf certificate. - std::vector email_names() const; + absl::Span email_names() const; // The (possibly empty) list of IP names from the Subject Alternative Name // extension in the peer's leaf certificate. - std::vector ip_names() const; + absl::Span ip_names() const; } // The base class of all verifier implementations. Note that custom @@ -458,7 +458,7 @@ public: // Returns the verified DER-encoded certificate chain, ordered from leaf to // root. Both the leaf and the root are included. Returns an error status if // verification fails for any reason. - virtual absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) = 0; + virtual absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) = 0; } ``` From 3ecc1a2af09bf9ce8affc68a479947aabe85b779 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Wed, 29 May 2024 18:47:04 +0000 Subject: [PATCH 43/52] more adjustments OnWatchStart and OnWatchStopped --- L46-core-tls-credential-API.md | 38 ++++++++++++++-------------------- 1 file changed, 16 insertions(+), 22 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 2fe4068e4..dc50e42a7 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -237,18 +237,22 @@ class CertificateProviderInterface { } protected: - // Provider implementations MUST provide a LoadAndSetCredentialsCallback that - // will be called by the internal stack. This will be invoked when a new - // certificate name is starting to be used internally, or when a certificate - // name is no longer being used internally. When being invoked for a new - // certificate, this callback should call SetRootCertificates or + // Provider implementations MUST provide a OnWatchStarted callback that will + // be called by the internal stack. This will be invoked when a new + // certificate name is starting to be used internally. When being invoked for + // a new certificate, this callback should call SetRootCertificates or // SetIdentityChainAndPrivateKey to do the initial population the certificate // data in the internal stack. - // - // For the parameters in the callback function: cert_name The name of the - // certificates being watched. type The type of certificates being watched. - // TODO(gtcooke94) adjust naming of functionality of this method to match previous WatchStatusCallback, it needs to handle both beginning to watch and ending watching. - virtual void LoadAndSetCredentialsCallback(std::string name, CredentialType type) = 0; + // cert_name The name of the certificates being watched. + // type The type of certificates being watched. + virtual void OnWatchStarted(std::string name, CredentialType type) = 0; + + // Provider implementations MUST provide a OnWatchStarted callback that will + // be called by the internal stack. This will be invoked when a certificate + // name is no longer being used internally. + // cert_name The name of the certificates being watched. + // type The type of certificates being watched. + virtual void OnWatchStopped(std::string name, CredentialType type) = 0; // Sets the root certificates based on their name. // This value is layered and represents the following. @@ -417,8 +421,8 @@ class CertificateVerifierInterface { // synchronous check. // return: return true if executed synchronously, otherwise return false virtual bool Verify(TlsCustomVerificationCheckRequest* request, - std::function callback, - Status* sync_status) = 0; + absl::AnyInvocable callback, + absl::Status* sync_status) = 0; // Cancels a verification request previously started via Verify(). // Used when the connection attempt times out or is cancelled while an async @@ -428,16 +432,6 @@ class CertificateVerifierInterface { virtual void Cancel(TlsCustomVerificationCheckRequest* request) = 0; }; -// A CertificateVerifier that doesn't perform any additional checks other than -// certificate verification, if specified. -// Note: using this solely without any other authentication mechanisms on the -// peer identity will leave your applications to the MITM(Man-In-The-Middle) -// attacks. Users should avoid doing so in production environments. -class NoOpCertificateVerifier : public CertificateVerifierInterface { - public: - NoOpCertificateVerifier(); -}; - // A CertificateVerifier that will perform hostname verification, to see if the // target name set from the client side matches the identity information // specified on the server's certificate. From dba8d7ffa96809689246b41033f955f82d44423b Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Fri, 31 May 2024 19:43:56 +0000 Subject: [PATCH 44/52] pr comments, append dangerous to certain fns --- L46-core-tls-credential-API.md | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index dc50e42a7..2fb2aa8da 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -60,14 +60,14 @@ class TlsCredentialsOptions { // Watches the updates of root certificates with name |root_cert_name| (set by // the user via set_root_cert_name). If used in TLS credentials, setting this - // field is optional for both the client side and the server side. - // If this is not set on the client side, we will use the root certificates - // stored in the default system location, since client side must provide root - // certificates in TLS (no matter single-side TLS or mutual TLS). - // If this is not set on the server side, we will not watch any root - // certificate updates, and assume no root certificates needed for the server - // (in the one-side TLS scenario, the server is not required to provide root - // certificates). We don't support default root certs on server side. + // field is optional for both the client side and the server side. If this is + // not set on the client side, we will use the root certificates stored in the + // default system location, since client side must provide root certificates + // in TLS (no matter single-side TLS or mutual TLS). If this is not set on + // the server side, we will not watch any root certificate updates, and assume + // no root certificates is needed for the server (in the one-side TLS + // scenario, the server is not required to provide root certificates). We + // don't support default root certs on server side. void watch_root_certificates(); // Sets the name of root certificates being watched, if |watch_root_certificates| is @@ -102,12 +102,14 @@ class TlsCredentialsOptions { // // @param tls_session_key_log_file_path: Path where TLS session keys should // be logged. - void set_tls_session_key_log_file_path( + void set_tls_session_key_log_file_path_dangerous( absl::string_view tls_session_key_log_file_path); // Sets the certificate verifier. The certificate verifier performs checks on // the peer certificate chain after the chain has been (cryptographically) // verified to chain up to a trusted root. + // If unset, this will default to the `HostNameCertificateVerifier` detailed + // below. void set_certificate_verifier( std::shared_ptr certificate_verifier); @@ -130,7 +132,7 @@ class TlsCredentialsOptions { // building from the underlying SSL library. Fully replacing and implementing // chain building is a complex task and has dangerous security implications if // done wrong, thus this API is inteded for expert use only. - void set_custom_chain_builder(std::shared_ptr + void set_custom_chain_builder_dangerous(std::shared_ptr chain_builder); } @@ -353,7 +355,7 @@ The next sections explain these interfaces in more detail. // Contains the information from the (verified) peer certificate chain that can // be used to perform custom validation checks. Users should not directly // create or destroy this request object. -class TlsCustomVerificationCheckRequest { +class VerifiedPeerCertificateChainInfo { public: // The target hostname that is expected to appear in the server leaf // certicate, e.g. github.com. Empty on the server-side. @@ -420,7 +422,7 @@ class CertificateVerifierInterface { // The status of the verifier as it has already done it's // synchronous check. // return: return true if executed synchronously, otherwise return false - virtual bool Verify(TlsCustomVerificationCheckRequest* request, + virtual bool Verify(VerifiedPeerCertificateChainInfo* request, absl::AnyInvocable callback, absl::Status* sync_status) = 0; @@ -429,7 +431,7 @@ class CertificateVerifierInterface { // verification request is pending. // // request: the verification information associated with this request - virtual void Cancel(TlsCustomVerificationCheckRequest* request) = 0; + virtual void Cancel(VerifiedPeerCertificateChainInfo* request) = 0; }; // A CertificateVerifier that will perform hostname verification, to see if the From d2eb387edd8145a7c9904a0e5a1acb23f0df6e0f Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Mon, 3 Jun 2024 21:00:22 +0000 Subject: [PATCH 45/52] address PR comments --- L46-core-tls-credential-API.md | 59 +++++++++++++++++----------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 2fb2aa8da..6246fad93 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -35,7 +35,7 @@ We propose an API that supports the above features and meets the following requi 1. [A69: Crl Providers](https://github.com/grpc/proposal/pull/382) ## Proposal -Until recently, C-Core needed to be written in C89, then we had C++ wrappers of those APIs in `grpcpp`. Requirements have changed such that we can use C++ directly in Core. The rest of this proposal is written as C++ that will be implemented in core, and simple aliases to everything will be written in the `grpcpp`. +Until recently, the C-Core boundary APIs needed to be written in C89, then we had C++ wrappers of those APIs in `grpcpp`. Requirements have changed such that we can use C++ directly in Core. The rest of this proposal is written as C++ that will be implemented in core, and simple aliases to everything will be written in the `grpcpp`. ### TLS Credentials and TLS Options This part of the proposal introduces the base options class, called `TlsCredentialsOptions`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsOptions`, called `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. @@ -65,9 +65,8 @@ class TlsCredentialsOptions { // default system location, since client side must provide root certificates // in TLS (no matter single-side TLS or mutual TLS). If this is not set on // the server side, we will not watch any root certificate updates, and assume - // no root certificates is needed for the server (in the one-side TLS - // scenario, the server is not required to provide root certificates). We - // don't support default root certs on server side. + // no root certificates are needed for the server (in the one-side TLS + // scenario, the server is not required to provide root certificates). void watch_root_certificates(); // Sets the name of root certificates being watched, if |watch_root_certificates| is @@ -185,27 +184,28 @@ std::shared_ptr TlsServerCredentials( ``` `TlsCredentials` features should represent a superset of `SslCredentials` -features, so we anticipate `TlsCredentials` being the API of choice now. To -make migration from the `SslCredentials` API to the `TlsCredentials` API easier, -we will also have a method on `SslCredentialsOptions` that creates a -`TlsChannelCredentialsOptions` with the same settings. We will do similarly for -`TlsServerCredentialsOptions`. +features, so we anticipate `TlsCredentials` being the API of choice now going +forward. To make migration from the `SslCredentials` API to the +`TlsCredentials` API easier, we will also have a method that takes +`SslCredentialsOptions` and creates a `TlsChannelCredentialsOptions` with the +same settings. We will do similarly for `TlsServerCredentialsOptions`. ### Credential Reloading -Credential reloading will be implemented via a `CertificateProviderInterface`. +Credential reloading will be implemented via the `CertificateProviderInterface`. Note that there is a naming mismatch here - there exist credentials beyond certificates that this _could_ theoretically support. This provider is -responsible for sourcing key material and supplying it to the internal stack via -the `CertificateProvider`'s `SetKeyMaterials` API. Two reference -implementations of the `CertificateProviderInterface` are provided - -`StaticDataCertificateProvider` and `FileWatcherCertificateProvider`. The -`StaticDataCertificateProvider` provides certificates from raw string data. The -`FileWatcherCertificateProvider` will periodically and asynchronously refresh -certificate data from a file, allowing changes to certificates without -restarting the process. More details about these reference implementations are -provided below. These implementations should cover many common use cases. If -these providers do not support the intricacies for a specific use case, a user -can provide their own implementations of the `CertificateProviderInterface`. +responsible for sourcing credential material and supplying it to the internal +stack via the `CertificateProvider`'s `SetRootCertificates` and +`SetIdentityChainAndPrivateKey` APIs. Two reference implementations of the +`CertificateProviderInterface` are provided - `StaticDataCertificateProvider` +and `FileWatcherCertificateProvider`. The `StaticDataCertificateProvider` +provides certificates from raw string data. The `FileWatcherCertificateProvider` +will periodically and asynchronously refresh certificate data from a file, +allowing changes to certificates without restarting the process. More details +about these reference implementations are provided below. These implementations +handle many common use cases. If these providers do not support the intricacies +for a specific use case, a user can provide their own implementations of the +`CertificateProviderInterface`. ```c /* Opaque types. */ @@ -276,7 +276,7 @@ protected: // The next layer is an `absl::optional`. This allows the user to set a value // or `absl::nullopt`, with `absl::nullopt` representing a deletion/un-setting // of the identity chain and private key data. - void SetIdentityChainAndPrivateKey(absl::string_view name, grpc_core::PemKeyCertPairList pem_key_cert_pairs); + void SetIdentityChainAndPrivateKey(absl::string_view name, absl::StatusOr>> pem_key_cert_pairs); private: std::unique_ptr distributor_; @@ -418,13 +418,12 @@ class CertificateVerifierInterface { // completed its asynchronous check. Callers can use this function // to perform any additional checks. The input parameter of the // std::function indicates the status of the verifier check. - // sync_status: This will only be useful if the verifier is synchronous. - // The status of the verifier as it has already done it's - // synchronous check. - // return: return true if executed synchronously, otherwise return false - virtual bool Verify(VerifiedPeerCertificateChainInfo* request, - absl::AnyInvocable callback, - absl::Status* sync_status) = 0; + // return: + // If error status, failed synchronously. + // If OK status and true, succeeded synchronously. + // If OK status and false, verification is still in progress asynchronously. + virtual absl::StatusOr Verify(VerifiedPeerCertificateChainInfo* request, + absl::AnyInvocable callback) = 0; // Cancels a verification request previously started via Verify(). // Used when the connection attempt times out or is cancelled while an async @@ -466,6 +465,6 @@ One use case for the custom chain building feature is to enable SPIFFE federatio // rather than the standard X509 approach. class SpiffeFederationChainBuilder : public CustomChainBuilderInterface { public: - ~SpiffeFederationChainBuilder() override; + SpiffeFederationChainBuilder(); } ``` \ No newline at end of file From 9e039808fd2080916c5a0262c6f09c4e038d0fd2 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 4 Jun 2024 19:10:05 +0000 Subject: [PATCH 46/52] address PR comments --- L46-core-tls-credential-API.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 6246fad93..bc553b7fb 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -109,6 +109,8 @@ class TlsCredentialsOptions { // verified to chain up to a trusted root. // If unset, this will default to the `HostNameCertificateVerifier` detailed // below. + // If set to nulltpr, this will overwrite the host name verifier and lead you + // to not doing any checks (aside from the cryptographic ones). void set_certificate_verifier( std::shared_ptr certificate_verifier); @@ -227,13 +229,15 @@ class CertificateProviderInterface { public: virtual ~CertificateProviderInterface() = default; - // Must be called after the constructor. - // Does important internal setup steps. + // Starts the provider. Must be called before the provider is used for any TLS + // handshakes. Does important internal setup steps. void Start(); - // The case of a SPIFFE trust bundle still falls into RootCertificates, it's - // just another way of representing root information + // CredentialType represents the different types of credentials that the + // provider can provide. enum CredentialType { + // The case of a SPIFFE trust bundle still falls into RootCertificates, it's + // just another way of representing root information RootCertificates, IdentityChainAndPrivateKey } @@ -422,7 +426,7 @@ class CertificateVerifierInterface { // If error status, failed synchronously. // If OK status and true, succeeded synchronously. // If OK status and false, verification is still in progress asynchronously. - virtual absl::StatusOr Verify(VerifiedPeerCertificateChainInfo* request, + virtual absl::StatusOr Verify(VerifiedPeerCertificateChainInfo* verified_chain_info, absl::AnyInvocable callback) = 0; // Cancels a verification request previously started via Verify(). @@ -430,7 +434,7 @@ class CertificateVerifierInterface { // verification request is pending. // // request: the verification information associated with this request - virtual void Cancel(VerifiedPeerCertificateChainInfo* request) = 0; + virtual void Cancel(VerifiedPeerCertificateChainInfo* verified_chain_info) = 0; }; // A CertificateVerifier that will perform hostname verification, to see if the From 061fa27fc14716decf4be02b1969d25bc392e8b8 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 11 Jun 2024 15:37:01 +0000 Subject: [PATCH 47/52] make tls options a builder --- L46-core-tls-credential-API.md | 41 +++++++++++++++++----------------- 1 file changed, 20 insertions(+), 21 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index bc553b7fb..1a64afee2 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -37,19 +37,19 @@ We propose an API that supports the above features and meets the following requi ## Proposal Until recently, the C-Core boundary APIs needed to be written in C89, then we had C++ wrappers of those APIs in `grpcpp`. Requirements have changed such that we can use C++ directly in Core. The rest of this proposal is written as C++ that will be implemented in core, and simple aliases to everything will be written in the `grpcpp`. -### TLS Credentials and TLS Options -This part of the proposal introduces the base options class, called `TlsCredentialsOptions`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsOptions`, called `TlsChannelCredentialsOptions` and `TlsServerCredentialsOptions`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. +### TLS Credentials and TLS Builder +This part of the proposal introduces the base builder class, called `TlsCredentialsBuilder`, that has a large set of options for configuring TLS connections. There are 2 derived classes of `TlsCredentialsBuilder`, called `TlsChannelCredentialsBuilder` and `TlsServerCredentialsBuilder`, that users create and can use to build channel credentials or server credentials instances, respectively. The API for building these channel/server credentials instances are called `TlsCredentials`/`TlsServerCredentials` respectively. ```c++ -// Base class of TLS options. -class TlsCredentialsOptions { +// Base class of TLS credentials builder. +class TlsCredentialsBuilder { public: - ~TlsCredentialsOptions(); + ~TlsCredentialsBuilder(); // Copy constructor does a deep copy. No assignment permitted. - TlsCredentialsOptions(const TlsCredentialsOptions& other); + TlsCredentialsBuilder(const TlsCredentialsBuilder& other); - TlsCredentialsOptions& operator=(const TlsCredentialsOptions& other) = delete; + TlsCredentialsBuilder& operator=(const TlsCredentialsBuilder& other) = delete; // ---- Setters for member fields ---- // Sets the certificate provider which is used to store: @@ -138,18 +138,22 @@ class TlsCredentialsOptions { } // Server-specific options for configuring TLS. -class TlsServerCredentialsOptions final : public TlsCredentialsOptions { +class TlsServerCredentialsBuilder final : public TlsCredentialsBuilder { public: // A certificate provider that provides identity credentials is required, // because a server must always present identity credentials during any TLS // handshake. The certificate provider may optionally provide root // certificates, in case the server requests client certificates. - explicit TlsServerCredentialsOptions( + explicit TlsServerCredentialsBuilder( std::shared_ptr certificate_provider) - : TlsCredentialsOptions() { + : TlsCredentialsBuilder() { set_certificate_provider(certificate_provider); } + // Builds a ServerCredentials instance that establishes TLS connections in the manner specified by options. + std::shared_ptr TlsServerCredentials( + const TlsServerCredentialsBuilder& options); + // Sets requirements for if client certificates are requested, if they // are required, and if the client certificate must be trusted. The @@ -168,29 +172,24 @@ class TlsServerCredentialsOptions final : public TlsCredentialsOptions { // will be used. // If a certificate provider is set and it provides identity credentials those // identity credentials will be used. -class TlsChannelCredentialsOptions final : public TlsCredentialsOptions { +class TlsChannelCredentialsBuilder final : public TlsCredentialsBuilder { public: // Sets the decision of whether to do a crypto check on the server certificates. // The default is true. void set_verify_server_certificates(bool verify_server_certs); -} -// Builds a ChannelCredentials instance that establishes TLS connections in the manner specified by options. -std::shared_ptr TlsCredentials( - const TlsChannelCredentialsOptions& options); + // Builds a ChannelCredentials instance that establishes TLS connections in the manner specified by options. + std::shared_ptr BuildTlsCredentials(); } -// Builds a ServerCredentials instance that establishes TLS connections in the manner specified by options. -std::shared_ptr TlsServerCredentials( - const TlsServerCredentialsOptions& options); ``` `TlsCredentials` features should represent a superset of `SslCredentials` features, so we anticipate `TlsCredentials` being the API of choice now going forward. To make migration from the `SslCredentials` API to the `TlsCredentials` API easier, we will also have a method that takes -`SslCredentialsOptions` and creates a `TlsChannelCredentialsOptions` with the -same settings. We will do similarly for `TlsServerCredentialsOptions`. +`SslCredentialsOptions` and creates a `TlsChannelCredentialsBuilder` with the +same settings. We will do similarly for `TlsServerCredentialsBuilder`. ### Credential Reloading Credential reloading will be implemented via the `CertificateProviderInterface`. @@ -304,7 +303,7 @@ class StaticDataCertificateProvider // A CertificateProviderInterface implementation that will watch the credential // changes on the file system. This provider will always return the up-to-date -// certificate data for all the cert names callers set through |TlsCredentialsOptions|. +// certificate data for all the cert names callers set through |TlsCredentialsBuilder|. // Several things to note: // 1. This API only supports one key-certificate file and hence one set of identity // key-certificate pair, so SNI(Server Name Indication) is not supported. From f433e37b51b9cd5e30d013b50c464c6bed2acbe9 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 11 Jun 2024 16:18:50 +0000 Subject: [PATCH 48/52] address comments from L46 design review followup --- L46-core-tls-credential-API.md | 87 +++++++++++++++++----------------- 1 file changed, 43 insertions(+), 44 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 1a64afee2..85302f049 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -58,38 +58,27 @@ class TlsCredentialsBuilder { void set_certificate_provider( std::shared_ptr certificate_provider); - // Watches the updates of root certificates with name |root_cert_name| (set by - // the user via set_root_cert_name). If used in TLS credentials, setting this - // field is optional for both the client side and the server side. If this is - // not set on the client side, we will use the root certificates stored in the - // default system location, since client side must provide root certificates - // in TLS (no matter single-side TLS or mutual TLS). If this is not set on - // the server side, we will not watch any root certificate updates, and assume - // no root certificates are needed for the server (in the one-side TLS - // scenario, the server is not required to provide root certificates). - void watch_root_certificates(); - - // Sets the name of root certificates being watched, if |watch_root_certificates| is - // called. If not set, an empty string will be used as the name. For simple - // use cases this is not needed, but a certificate provider could be - // implemented that watches many sets of certificates. This allows the user to - // differentiate between different root certificates. + // Watches the updates of root certificates with name |name|. If used in TLS + // credentials, setting this field is optional for both the client side and + // the server side. If this is not set on the client side, we will use the + // root certificates stored in the default system location, since client side + // must provide root certificates in TLS (no matter single-side TLS or mutual + // TLS). If this is not set on the server side, we will not watch any root + // certificate updates, and assume no root certificates are needed for the + // server (in the one-side TLS scenario, the server is not required to provide + // root certificates). // - // @param root_cert_name the name of root certificates being set. - void set_root_cert_name(absl::string_view root_cert_name); + // @param name the name of root certificates being set. + void watch_root_certificates(absl::string_view name); + // Watches the updates of identity key-certificate pairs with name - // |identity_cert_name|. If used in TLS credentials, it is required to be set + // |name|. If used in TLS credentials, it is required to be set // on the server side, and optional for the client side(in the one-side // TLS scenario, the client is not required to provide identity certificates). - void watch_identity_key_cert_pairs(); - // Sets the name of identity key-certificate pairs being watched, if - // |watch_identity_key_cert_pairs| is called. If not set, an empty string will - // be used as the name. - // - // @param identity_cert_name the name of identity key-certificate pairs being set. - void set_identity_cert_name(absl::string_view identity_cert_name); + // @param name the name of the identity key cert pairs to watch. + void watch_identity_key_cert_pairs(absl::string_view name); // WARNING: EXPERT USE ONLY. MISUSE CAN LEAD TO SIGNIFICANT SECURITY DEGRADATION. // @@ -409,23 +398,17 @@ class CertificateVerifierInterface { virtual ~CertificateVerifierInterface() = default; // Performs a custom verification check. - // Returns true if verification occurs synchronously, in which case the - // verification result is populated in the sync_status output parameter. - // Returns false if verification occurs asynchronously, in which case the - // callback must be called when verification is complete and the verification - // result must be populated in the sync_status output parameter. + // Verification occurs asynchronously. gRPC calls `Verify` on this interface. + // The callback must be called when verification is complete and the + // verification result must be populated in the output parameter of the + // callback function. // // request: the verification information associated with this request - // callback: This will only take effect if the verifier is asynchronous. - // The function that gRPC will invoke when the verifier has already + // callback: The function that gRPC will invoke when the verifier has already // completed its asynchronous check. Callers can use this function // to perform any additional checks. The input parameter of the - // std::function indicates the status of the verifier check. - // return: - // If error status, failed synchronously. - // If OK status and true, succeeded synchronously. - // If OK status and false, verification is still in progress asynchronously. - virtual absl::StatusOr Verify(VerifiedPeerCertificateChainInfo* verified_chain_info, + // any::Invocable indicates the status of the verifier check. + virtual void Verify(std::shared_ptr verified_chain_info, absl::AnyInvocable callback) = 0; // Cancels a verification request previously started via Verify(). @@ -433,7 +416,7 @@ class CertificateVerifierInterface { // verification request is pending. // // request: the verification information associated with this request - virtual void Cancel(VerifiedPeerCertificateChainInfo* verified_chain_info) = 0; + virtual void Cancel(std::shared_ptr verified_chain_info) = 0; }; // A CertificateVerifier that will perform hostname verification, to see if the @@ -449,14 +432,30 @@ class HostNameCertificateVerifier : public CertificateVerifierInterface { #### Custom Chain Building ```c++ + + +class UnverifiedPeerChain { + std::vector peer_cert_chain_der; +} + + class CustomChainBuilderInterface { public: virtual ~CustomChainBuilderInterface() = default; - // Returns the verified DER-encoded certificate chain, ordered from leaf to - // root. Both the leaf and the root are included. Returns an error status if - // verification fails for any reason. - virtual absl::StatusOr> BuildAndVerifyChain(const std::vector& peer_cert_chain_der) = 0; + // Performs custom chain building. gRPC invokes this function. When Chain + // building is complete, the implementer must invoke the callback. The output + // parameter of the callback must be populated with the verified DER-encoded + // certificate chain, ordered from leaf to root. Both the leaf and the root + // must be included. + virtual void BuildAndVerifyChain(std::shared_ptr peer_chain, absl::AnyInvocable>> callback)) = 0; + + // Cancels a chain building request request previously started via + // BuildAndVerifyChain(). Used when the connection attempt times out or is + // cancelled while an async request is pending. + // + // request: the verification information associated with this request + virtual void Cancel(std::shared_ptr peer_chain) = 0; } ``` From 84bea7c2fb91ff0c724ccc9c2940cc6d2f892a1a Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Tue, 11 Jun 2024 16:19:23 +0000 Subject: [PATCH 49/52] modify date of edit --- L46-core-tls-credential-API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index 85302f049..eab65ff93 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -4,7 +4,7 @@ L46: C++: New TLS Credentials API * Approver: * Status: Draft * Implemented in: -* Last updated: May 16, 2024 +* Last updated: June 11, 2024 ## Abstract From 8fa1ae7e80675517bcd76c31d84883eae4f58fd4 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 20 Jun 2024 15:53:08 +0000 Subject: [PATCH 50/52] address PR comments --- L46-core-tls-credential-API.md | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index eab65ff93..fe7f3b244 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -164,8 +164,10 @@ class TlsServerCredentialsBuilder final : public TlsCredentialsBuilder { class TlsChannelCredentialsBuilder final : public TlsCredentialsBuilder { public: // Sets the decision of whether to do a crypto check on the server certificates. + // Setting this to false is potentially dangerous as in most TLS setups the + // client verifies the server. // The default is true. - void set_verify_server_certificates(bool verify_server_certs); + void set_verify_server_certificates_dangerous(bool verify_server_certs); // Builds a ChannelCredentials instance that establishes TLS connections in the manner specified by options. std::shared_ptr BuildTlsCredentials(); @@ -318,16 +320,16 @@ class FileWatcherCertificateProvider final FileWatcherCertificateProvider(absl::string_view private_key_path, absl::string_view identity_certificate_path, absl::string_view root_cert_path, - unsigned int refresh_interval_sec); + absl::Duration refresh_interval_sec); // Constructor to get credential updates from identity file paths only. FileWatcherCertificateProvider(absl::string_view private_key_path, absl::string_view identity_certificate_path, - unsigned int refresh_interval_sec); + absl::Duration refresh_interval_sec); // Constructor to get credential updates from root file path only. FileWatcherCertificateProvider(absl::string_view root_cert_path, - unsigned int refresh_interval_sec); + absl::Duration refresh_interval_sec); } @@ -434,8 +436,8 @@ class HostNameCertificateVerifier : public CertificateVerifierInterface { ```c++ -class UnverifiedPeerChain { - std::vector peer_cert_chain_der; +class UnverifiedPeerCertificateChain { + absl::Span peer_cert_chain_der; } @@ -443,24 +445,23 @@ class CustomChainBuilderInterface { public: virtual ~CustomChainBuilderInterface() = default; - // Performs custom chain building. gRPC invokes this function. When Chain - // building is complete, the implementer must invoke the callback. The output - // parameter of the callback must be populated with the verified DER-encoded - // certificate chain, ordered from leaf to root. Both the leaf and the root - // must be included. - virtual void BuildAndVerifyChain(std::shared_ptr peer_chain, absl::AnyInvocable>> callback)) = 0; + // Performs custom chain building. When chain building is complete, the + // implementer must invoke the callback. The output parameter of the callback + // must be populated with the verified DER-encoded certificate chain, ordered + // from leaf to root. Both the leaf and the root must be included. + virtual void BuildAndVerifyChain(std::shared_ptr peer_chain, absl::AnyInvocable>> callback)) = 0; // Cancels a chain building request request previously started via // BuildAndVerifyChain(). Used when the connection attempt times out or is // cancelled while an async request is pending. // // request: the verification information associated with this request - virtual void Cancel(std::shared_ptr peer_chain) = 0; + virtual void Cancel(std::shared_ptr peer_chain) = 0; } ``` The default behavior for chain building is to defer to the underlying SSL library, which has a built-in chain building API that has been hardened over many years of use with the web PKI. For example, [X509_verify_cert](https://www.openssl.org/docs/man1.1.1/man3/X509_verify_cert.html) is the implementation in OpenSSL that builds a chain and verifies that it is trusted by one of the root certificates. -One use case for the custom chain building feature is to enable SPIFFE federation (TODO: ref the SPIFFE spec), where the set of root certificates to use is determined based on the SPIFFE trust domain in the peer's leaf certificate. +One use case for the custom chain building feature is to enable [SPIFFE federation](https://spiffe.io/), where the set of root certificates to use is determined based on the SPIFFE trust domain in the peer's leaf certificate. ```c++ // Builds a trusted and validated certificate chain based on SPIFFE trust maps From fcbfc7d5424797c03643ae7f3ac83b39805eea41 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Thu, 20 Jun 2024 20:03:20 +0000 Subject: [PATCH 51/52] added warning and notes about hostname check --- L46-core-tls-credential-API.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index fe7f3b244..a3fa0d1d4 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -96,8 +96,9 @@ class TlsCredentialsBuilder { // Sets the certificate verifier. The certificate verifier performs checks on // the peer certificate chain after the chain has been (cryptographically) // verified to chain up to a trusted root. - // If unset, this will default to the `HostNameCertificateVerifier` detailed - // below. + // If unset on the client-side, this will default to the + // `HostNameCertificateVerifier` detailed below. + // If unset on the server-side, this will default to a nullptr. // If set to nulltpr, this will overwrite the host name verifier and lead you // to not doing any checks (aside from the cryptographic ones). void set_certificate_verifier( @@ -163,6 +164,8 @@ class TlsServerCredentialsBuilder final : public TlsCredentialsBuilder { // identity credentials will be used. class TlsChannelCredentialsBuilder final : public TlsCredentialsBuilder { public: + // WARNING: EXPERT USE ONLY. MISUSE CAN LEAD TO SIGNIFICANT SECURITY DEGRADATION. + // // Sets the decision of whether to do a crypto check on the server certificates. // Setting this to false is potentially dangerous as in most TLS setups the // client verifies the server. From 711f19f529eddffbf08bd12f0bf5fa33a32e6617 Mon Sep 17 00:00:00 2001 From: Gregory Cooke Date: Wed, 20 Nov 2024 15:13:07 +0000 Subject: [PATCH 52/52] add note about custom verification --- L46-core-tls-credential-API.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/L46-core-tls-credential-API.md b/L46-core-tls-credential-API.md index a3fa0d1d4..d5bab9753 100644 --- a/L46-core-tls-credential-API.md +++ b/L46-core-tls-credential-API.md @@ -407,6 +407,8 @@ class CertificateVerifierInterface { // The callback must be called when verification is complete and the // verification result must be populated in the output parameter of the // callback function. + // This method will only be called in the case that regular verification has succeeded. + // Custom verification WILL NOT be called on resumed handshakes. // // request: the verification information associated with this request // callback: The function that gRPC will invoke when the verifier has already