Examples of adding metrics to Splunk Smart Agent
The general process is the following:
- Find the customization you are looking to add
- Main sources
- If what you are looking for can't be found here you may be able to find other CollectD plugins or build your own collection
Below are some common commands and tips you can use in modifying configurations and monitoring logs. They are specific to Linux/Ubuntu but could easily be adapted for other operating systems.
- Restarting the service
- Many changes don't require restarting the agent but some (like log level) do
sudo service signalfx-agent restart
- Logging datapoints
- To view the datapoints
signalfx-agent tap-dps
- To send datapoints to the log
- Change log level in
/etc/signalfx/agent.yamlto Debug - Add the following to the
writersection of/etc/signalfx/agent.yamllogDatapoints: true
- Restart the agent
- Tail the log
journalctl -f -u signalfx-agent
- Change log level in
- To view the datapoints
- To view monitors
- Long output
signalfx-agent status monitors
- Short output
signalfx-agent status monitors | egrep '^[0-9]+\.'
- To get details about a specific monitor (e.g. etcd)
signalfx-agent status monitors | sed -n '/\. etcd/,/^$/p'
- Long output