-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ha to monasca barclamp #14
base: monasca
Are you sure you want to change the base?
Add ha to monasca barclamp #14
Conversation
# ":#{node[:monasca][:db][:password]}" \ | ||
# "@#{db_settings[:address]}" \ | ||
# "/#{node[:monasca][:db][:database]}" | ||
if ha_enabled |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style/IfUnlessModifier: Favor modifier if usage when having a single-line body. Another good alternative is the usage of control flow &&/||. (https://github.com/bbatsov/ruby-style-guide#if-as-a-modifier)
chef/cookbooks/monasca/recipes/ha.rb
Outdated
address "0.0.0.0" | ||
port 1234 | ||
use_ssl false | ||
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "plain") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Metrics/LineLength: Line is too long. [104/100] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricslinelength)
|
||
default[:monasca][:ha][:enabled] = false | ||
# Ports to bind to when haproxy is used for the real ports | ||
default[:monasca][:ha][:ports][:plain] = 5580 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thats conflicting with horizon, choose a different port
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just a 'random' port, I only wanted to experiment with HA to make sure I get it right. So can I assume that, in overall, the approach itself is correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but please find a port that doesn't clash with anything else. As far as I know we do not have a fixed convention for that. Please correct me if I'm wrong about that, Dirk.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
chef/cookbooks/monasca/recipes/ha.rb
Outdated
|
||
haproxy_loadbalancer "monasca" do | ||
address "0.0.0.0" | ||
port 1234 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this the proper port?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
haproxy_loadbalancer "monasca-api" do
address network_settings 0.0.0.0 # not sure if/how we can specify multiple addresses for this resource but this will make it listen on both the admin and public interface
port 8070
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "monasca-api")
)
action :nothing
end.run_action(:create)
haproxy_loadbalancer "monasca-log-api" do
address network_settings 0.0.0.0 # not sure if/how we can specify multiple addresses for this resource but this will make it listen on both the admin and public interface
port 5607
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "monasca-log-api")
)
action :nothing
end.run_action(:create)
haproxy_loadbalancer "monasca-logs-search" do
address network_settings 0.0.0.0 # not sure if/how we can specify multiple addresses for this resource but this will make it listen on both the admin and public interface
port 5601
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "monasca-logs-search")
)
action :nothing
end.run_action(:create)
For this to work you'll need to define the ports where the regular services listen in the HA case in defaults.rb:
default[:monasca][:ha][:ports][:monasca-api] = 18070
default[:monasca][:ha][:ports][:monasca-log-api] = 15607
default[:monasca][:ha][:ports][:monasca-logs-search] = 15601
This is where haproxy_servers_for_service
is defined:
That should make it clear why this works out. It is a bit convoluted, I'm afraid.
Also, I just used the regular listen port for the haproxy_loadbalancer
resource's port attribute above for the sake of clarity. Please don't do this
in your cookbook. Use a helper the way the Barbican
barclamp
does it. You'll find it defined here:
That helper ensures the actual services listen on the alternate ports (i.e.
18070, 15607, 15601 in our case) in the HA case, and on the regular ports
(8070, 5607, 5601 in our case) otherwise. To that end it provides the network
settings in both the service recipe
and the HA recipe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not quite there, yet unfortunately :-(
See my inline comments, especially the second one. I hope that makes things somewhat clearer (our lookup mechanisms for HA/non-HA ports are a bit convoluted). Please feel free to ask if it's not clear (I took a while to wrap my mind around all these twists and turns myself the first time I added HA support to a barclamp...)
|
||
default[:monasca][:ha][:enabled] = false | ||
# Ports to bind to when haproxy is used for the real ports | ||
default[:monasca][:ha][:ports][:plain] = 5580 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but please find a port that doesn't clash with anything else. As far as I know we do not have a fixed convention for that. Please correct me if I'm wrong about that, Dirk.
chef/cookbooks/monasca/recipes/ha.rb
Outdated
|
||
haproxy_loadbalancer "monasca" do | ||
address "0.0.0.0" | ||
port 1234 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
haproxy_loadbalancer "monasca-api" do
address network_settings 0.0.0.0 # not sure if/how we can specify multiple addresses for this resource but this will make it listen on both the admin and public interface
port 8070
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "monasca-api")
)
action :nothing
end.run_action(:create)
haproxy_loadbalancer "monasca-log-api" do
address network_settings 0.0.0.0 # not sure if/how we can specify multiple addresses for this resource but this will make it listen on both the admin and public interface
port 5607
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "monasca-log-api")
)
action :nothing
end.run_action(:create)
haproxy_loadbalancer "monasca-logs-search" do
address network_settings 0.0.0.0 # not sure if/how we can specify multiple addresses for this resource but this will make it listen on both the admin and public interface
port 5601
servers CrowbarPacemakerHelper.haproxy_servers_for_service(node, "monasca", "monasca-server", "monasca-logs-search")
)
action :nothing
end.run_action(:create)
For this to work you'll need to define the ports where the regular services listen in the HA case in defaults.rb:
default[:monasca][:ha][:ports][:monasca-api] = 18070
default[:monasca][:ha][:ports][:monasca-log-api] = 15607
default[:monasca][:ha][:ports][:monasca-logs-search] = 15601
This is where haproxy_servers_for_service
is defined:
That should make it clear why this works out. It is a bit convoluted, I'm afraid.
Also, I just used the regular listen port for the haproxy_loadbalancer
resource's port attribute above for the sake of clarity. Please don't do this
in your cookbook. Use a helper the way the Barbican
barclamp
does it. You'll find it defined here:
That helper ensures the actual services listen on the alternate ports (i.e.
18070, 15607, 15601 in our case) in the HA case, and on the regular ports
(8070, 5607, 5601 in our case) otherwise. To that end it provides the network
settings in both the service recipe
and the HA recipe.
d762fd4
to
bdbacc8
Compare
Following is the first one that will add installing monasca-log-agent capabilities
…hnologyGmbH#11) * Use deticate users for monasca-log-agent Following commit provides registering custom project and dedicated user registered inside this project * Added some debug details * Added including monasca:common * Include monasca:common elsewhere * Removed return clause' * Removed action:nothing * Move log outside block and added some variables * Added newline * Removed block * Added new configuration variables into conf template * TNT * Use group instead of 2xowner * Removed extra " from log-agent.conf.erb * Changed way to access values * Added user&group into databad * Moved registering log-agent Since agents (metric&logs) will be configured with single credentials per each node it made sense to put creating them inside the common part. Additionally added randomizing the agents password * Fixing hound error * Fixing hound error * Fixed typo * Addressed matrixik comments * Fixing undefined method in monasca_service.rb * Use empty strings instead of null values * trying to make it work * trying to make it work 2 * trying to make it work 3 * trying to make it work 4 * trying to make it work 5 * trying to make it work 6 * polished * Pleasing Hound
* Configure monasca-ui plugin * Refactor the code * Addresing review comments * Deploy grafana alongside horizon * Remove obsolete changes to monasca schema * Removing extra blank line
* Add monitoring dashboards to Grafana * Change template to cookbook_file
* Provide Ansible integration - draft version * Adding parameters to UI * Adding group_all template * Fixing inventory file templating * Making sure Ansible runs successfully (at least once) * Adapt constraints for monasca-server * Set default deployment of monasca roles * Add config field for CMM installer URL * Fixing invalid merge * Fix order of roles * Add grafana database password * Fixing hound violations * Fixing hound violations 2 * Draft of inventory file template for cluster * Adjust monasca-master role to use appropriate inventory file template * Fill in inventory file for cluster * Provide Ansible integration - draft version * Adding parameters to UI * Adding group_all template * Fixing inventory file templating * Making sure Ansible runs successfully (at least once) * Adapt constraints for monasca-server * Set default deployment of monasca roles * Add config field for CMM installer URL * Fixing invalid merge * Fix order of roles * Add grafana database password * Fixing hound violations * Fixing hound violations 2 * Addressing review comments * Fix Hound violations * Fix Hound violations 2 * Fix Hound violations 3 * Addressing review comments * Addressing some additional comments * Addressing some additional comments 2
* Add missing end keyword * Fix indent
…chnologyGmbH#2) * Add monasca-reconfigure and sudoers file * Fix monasca-setup bin path * Fix template variables * Keystone settings for agent * Keystone settings for agent * Keystone settings for agent * Add log * More readable config templates * Remove log * Remove per-node credentials todos * Fix json indentations * Rename agent dimensions variable name * Run monasca-reconfigure every time with chef client * Add firts GUI elements * Add GUI elements * Update for new code arrangement for keystone * Fix yaml for translations * Fix monasca helper module name * Use root user for agent temporally * Update monasca agent GUI parts * WIP: Add custom checks for monasca-agent * Extract variables for templates * Fix testing filepath * Fix template for monasca agent checks * Fix template paths again * Address Hound warning * Add more monasca agent checkers * Fix some variables * Add log api port to data_bag * Build Kibana and log-api urls * Add more info to GUI * Add user_managed: false * Fix yaml indentation * Add some logs * Fix monasca-reconfigure * Fix ruby boolean name * Uncomment agent.yaml template code * Fix providing service variable to template * Fix insecure args for monasca-setup * Fix monasca-reconfigure * Configure more variables with monasca-reconfigure * Add more config to UI * Disable generating agent.yaml It's regenerated every time chef client is run, it's running monasca-reconfigure. * Change default overwrite config to false * Add monasca-agent to sudoers * Run monasca-setup as monasca-agent user * Remove logs from monasca-reconfigure * Rename monasca-agent to monasca-metric-agent And rename all variables to reflect this change. * Fix hound warnings * Address comments * Fix copyright * Use same order in data_bag schema like in json * Restore user mapping in log_agent schema * Change monasca-master unique to false * Fix monasca-metric-agents names in json * Use keystone settings from data_bag for metric agent * Replace `owner` with `user` in execute command * Replace group `monasca-agent` with `monasca` * Remove adding monasca-agent to sudoers Will be added from rpm. * Add port to monasca_api_url for monasca-agent * Use helper for getting api url * Fix bad default config name
…ogyGmbH#19) Enforce skip_enable option for monasca_setup 'skip_enable' option controls the creation of systemd service file for Monasca Agent. The service file is created in RPM package, so the option has to be enabled here.
…TechnologyGmbH#18) * Get proper log-api url for monasca-log-agent * Remove useless monasca urls from data_bag * Remove log_agent user and group from data_bag * Always use http in log-api url * Use only http for monasca-agent url * Remove ability to set `insecure` and `ca_file` options from UI.
…ologyGmbH#15) * Finish integration of Ansible installer * Set credentials in command line * Fix violations * Fix violations 2 * Add support for multiple networks in single node * Pass missing variables to ansible * Fix hound violations * Use monasca.yml as entry point for ansible installer * Fix paths and influxdb url * Remove agent nodes from inventory file
* Register monasca endpoints in Keystone * Fix kibana url * Rearrange helper functions
Remove unused grafana_group from cluster template
…nologyGmbH#34) They are outputted to STDERR and break applying cookbook.
* Register monasca endpoints in Keystone * Fix kibana url * Rearrange helper functions * Make agent log dir configurable
monasca_api_client_port: network_settings[:api][:bind_port], | ||
monasca_log_api_port: network_settings[:log_api][:bind_port], | ||
kibana_port: network_settings[:kibana][:bind_port], | ||
mysql_port: network_settings[:mariadb][:bind_port], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style/TrailingCommaInArguments: Avoid comma after the last parameter of a method call. (https://github.com/bbatsov/ruby-style-guide#no-trailing-params-comma)
Rename cluster_admin_ip to cluster_monitoring_ip
Also set mode "tcp" for all InfluxDB configs.
) | ||
end | ||
|
||
def network_settings(node) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Metrics/AbcSize: Assignment Branch Condition size for network_settings is too high. [118.1/30] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricsabcsize, http://c2.com/cgi/wiki?AbcMetric)
Metrics/CyclomaticComplexity: Cyclomatic complexity for network_settings is too high. [11/6]
Metrics/MethodLength: Method has too many lines. [64/50] (https://github.com/bbatsov/ruby-style-guide#short-methods)
Metrics/PerceivedComplexity: Perceived complexity for network_settings is too high. [12/7]
196eada
to
7919735
Compare
No description provided.