Leverage AWS services to create cost-effective serverless architectures to enable monitoring of Couchbase clusters using Amazon CloudWatch.
Amazon CloudWtach provides scalable and reliable monitoring and observability for your resources on many levels. Using metrics monitoring you get visibility on how your resource is performing under operational conditions. You can use alarms to watch for single or composite metrics to trigger notifications or automatic actions based on the value of the metric or expression relative to a threshold over a number of time periods. Amazon CloudWatch provides an API and SDK integrations so that you can send metrics supplemented by dimensions to enable custom resources monitoring. You can leverage CloudWatch to add additional monitoring capabilities for operational system that does not natively integrate with CloudWatch.
This enables a single plane for monitoring your AWS resources as well as other systems through CloudWatch.
Couchbase clusters provide cluster endpoints that send information about the cluster health. Couchbase clusters also provide Bucket specific endpoints that monitor the performance of the operations in relation to the Buckets.
This is a serverless architecture that provides an example on how to monitor the state of a Couchbase cluster via its HTTP/JSON APIs. This information will be forwarded to Amazon CloudWatch as metrics to monitor Couchbase cluster during its operation.
The architecture above deploys the following resources.
- An AWS Secrets Manager secret to store the cluster credentials.
- An AWS Lambda function that
- Looks up cluster credentials from the mentioned secret.
- Calls the cluster endpoints for cluster and bucket telemetry.
- Parses and forwards metrics it to Amazon CloudWatch with proper dimensions.
- An AWS EventBridge rule that calls the Lambda continuously based on a schedule.
Follow the following guidelines to deploy rest monitoring sample
CLI tools like CBSTATS, that ships with Couchbase also provides telemetry capabilities. CBSTATS has a parameter to allow for text or JSON output format.
The architecture above describes a script to call the health endpoints and CBSTATS tool. The script parse the output and forwards metrics to CloudWatch with proper dimensions. You can use node host schedule capabilities to schedule contentiously calling the monitoring script.
Check sample script for node internal monitoring
Couchbase provides a lot of log events in multiple log files around significant operational events.
CloudWatch Logs provides a cost-effective, reliable and scalable service to aggregate and index your application logs. Through unified CloudWatch Logs agent, logs can be shipped to CloudWatch Logs to monitoring the Couchbase cluster logs. For more information about using the unified CloudWatch agent to get started with CloudWatch Logs, see Collect Metrics and Logs from Amazon EC2 Instances and On-Premises Servers with the CloudWatch Agent in the Amazon CloudWatch User Guide.
Check examples for Amazon CloudWatch Logs unified agent configuration.
- Follow the following guidelines to deploy rest monitoring sample. You can deploy the rest monitoring solution example by getting started with SAM model.
- Check sample script for node internal monitoring.
- Check examples for Amazon CloudWatch Logs unified agent configuration.
The samples repository provides guidance on 2 main topics,
- monitoring metrics via CloudWatch
- and shipping logs to CloudWatch Logs
For the first point you have 2 choices based on the degree you control your cluster nodes.
a. If you have access to install scripts and packages on the cluster nodes, then consider node monitoring through internal monitoring script.
b. If you are not able to install packages on the node, then you can externally monitor your cluster by deploying HTTP endpoints.
For the second point, if you want to access your node logs via CloudWatch Logs, then check the guidance to ship logs via the unified agent.
If you found yourself wishing this set of frequently asked questions had an answer for a particular problem, please submit a pull request. The chances are good that others will also benefit from having the answer listed here.
See the Contributing Guidelines for details.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.