Skip to content

Commit

Permalink
feat(deploy/ha): Add clouddriver-ro-deck to HA deployment. (#1069)
Browse files Browse the repository at this point in the history
  • Loading branch information
joonlim authored Oct 16, 2018
1 parent b45a4ee commit 92b9963
Show file tree
Hide file tree
Showing 10 changed files with 143 additions and 32 deletions.
51 changes: 26 additions & 25 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,6 @@
* [**hal config canary datadog account list**](#hal-config-canary-datadog-account-list)
* [**hal config canary datadog disable**](#hal-config-canary-datadog-disable)
* [**hal config canary datadog enable**](#hal-config-canary-datadog-enable)
* [**hal config canary signalfx**](#hal-config-canary-signalfx)
* [**hal config canary signalfx account**](#hal-config-canary-signalfx-account)
* [**hal config canary signalfx account add**](#hal-config-canary-signalfx-account-add)
* [**hal config canary signalfx account delete**](#hal-config-canary-signalfx-account-delete)
* [**hal config canary signalfx account edit**](#hal-config-canary-signalfx-account-edit)
* [**hal config canary signalfx account get**](#hal-config-canary-signalfx-account-get)
* [**hal config canary signalfx account list**](#hal-config-canary-signalfx-account-list)
* [**hal config canary signalfx disable**](#hal-config-canary-signalfx-disable)
* [**hal config canary signalfx enable**](#hal-config-canary-signalfx-enable)
* [**hal config canary disable**](#hal-config-canary-disable)
* [**hal config canary edit**](#hal-config-canary-edit)
* [**hal config canary enable**](#hal-config-canary-enable)
Expand All @@ -134,6 +125,15 @@
* [**hal config canary prometheus disable**](#hal-config-canary-prometheus-disable)
* [**hal config canary prometheus edit**](#hal-config-canary-prometheus-edit)
* [**hal config canary prometheus enable**](#hal-config-canary-prometheus-enable)
* [**hal config canary signalfx**](#hal-config-canary-signalfx)
* [**hal config canary signalfx account**](#hal-config-canary-signalfx-account)
* [**hal config canary signalfx account add**](#hal-config-canary-signalfx-account-add)
* [**hal config canary signalfx account delete**](#hal-config-canary-signalfx-account-delete)
* [**hal config canary signalfx account edit**](#hal-config-canary-signalfx-account-edit)
* [**hal config canary signalfx account get**](#hal-config-canary-signalfx-account-get)
* [**hal config canary signalfx account list**](#hal-config-canary-signalfx-account-list)
* [**hal config canary signalfx disable**](#hal-config-canary-signalfx-disable)
* [**hal config canary signalfx enable**](#hal-config-canary-signalfx-enable)
* [**hal config ci**](#hal-config-ci)
* [**hal config ci jenkins**](#hal-config-ci-jenkins)
* [**hal config ci jenkins disable**](#hal-config-ci-jenkins-disable)
Expand Down Expand Up @@ -2558,14 +2558,14 @@ hal config canary signalfx [parameters] [subcommands]
* `--no-validate`: (*Default*: `false`) Skip validation.

#### Subcommands
* `account`: Manage and view Spinnaker configuration for the SignalFx service integration's canary accounts.
* `disable`: Set Spinnaker's canary analysis SignalFx service integration to disabled.
* `enable`: Set Spinnaker's canary analysis SignalFx service integration to enabled.
* `account`: Manage and view Spinnaker configuration for the Signalfx service integration's canary accounts.
* `disable`: Set Spinnaker's canary analysis Signalfx service integration to disabled.
* `enable`: Set Spinnaker's canary analysis Signalfx service integration to enabled.

---
## hal config canary signalfx account

Manage and view Spinnaker configuration for the SignalFx service integration's canary accounts.
Manage and view Spinnaker configuration for the Signalfx service integration's canary accounts.

#### Usage
```
Expand All @@ -2578,16 +2578,16 @@ hal config canary signalfx account ACCOUNT [parameters] [subcommands]
* `--no-validate`: (*Default*: `false`) Skip validation.

#### Subcommands
* `add`: Add a canary account to the SignalFx service integration.
* `delete`: Delete a specific SignalFx canary account by name.
* `edit`: Edit a canary account in the SignalFx service integration.
* `get`: Get the specified canary account details for the SignalFx service integration.
* `list`: List the canary account names for the SignalFx service integration.
* `add`: Add a canary account to the Signalfx service integration.
* `delete`: Delete a specific Signalfx canary account by name.
* `edit`: Edit a canary account in the Signalfx service integration.
* `get`: Get the specified canary account details for the Signalfx service integration.
* `list`: List the canary account names for the Signalfx service integration.

---
## hal config canary signalfx account add

Add a canary account to the SignalFx service integration.
Add a canary account to the Signalfx service integration.

#### Usage
```
Expand All @@ -2605,7 +2605,7 @@ hal config canary signalfx account add ACCOUNT [parameters]
---
## hal config canary signalfx account delete

Delete a specific SignalFx canary account by name.
Delete a specific Signalfx canary account by name.

#### Usage
```
Expand All @@ -2621,7 +2621,7 @@ hal config canary signalfx account delete ACCOUNT [parameters]
---
## hal config canary signalfx account edit

Edit a canary account in the SignalFx service integration.
Edit a canary account in the Signalfx service integration.

#### Usage
```
Expand All @@ -2639,7 +2639,7 @@ hal config canary signalfx account edit ACCOUNT [parameters]
---
## hal config canary signalfx account get

Get the specified canary account details for the SignalFx service integration.
Get the specified canary account details for the Signalfx service integration.

#### Usage
```
Expand All @@ -2655,7 +2655,7 @@ hal config canary signalfx account get ACCOUNT [parameters]
---
## hal config canary signalfx account list

List the canary account names for the SignalFx service integration.
List the canary account names for the Signalfx service integration.

#### Usage
```
Expand All @@ -2670,7 +2670,7 @@ hal config canary signalfx account list [parameters]
---
## hal config canary signalfx disable

Set Spinnaker's canary analysis SignalFx service integration to disabled.
Set Spinnaker's canary analysis Signalfx service integration to disabled.

#### Usage
```
Expand All @@ -2685,7 +2685,7 @@ hal config canary signalfx disable [parameters]
---
## hal config canary signalfx enable

Set Spinnaker's canary analysis SignalFx service integration to enabled.
Set Spinnaker's canary analysis Signalfx service integration to enabled.

#### Usage
```
Expand Down Expand Up @@ -3293,6 +3293,7 @@ hal config deploy ha clouddriver edit [parameters]
* `--deployment`: If supplied, use this Halyard deployment. This will _not_ create a new deployment.
* `--no-validate`: (*Default*: `false`) Skip validation.
* `--redis-master-endpoint`: Set external Redis endpoint for clouddriver-rw and clouddriver-caching. The Redis URI schema is described here: https://www.iana.org/assignments/uri-schemes/prov/redis. clouddriver-rw and clouddriver-caching are configured to use the shared Redis, by default.
* `--redis-slave-deck-endpoint`: Set external Redis endpoint for clouddriver-ro-deck. The Redis URI schema is described here: https://www.iana.org/assignments/uri-schemes/prov/redis. clouddriver-ro-deck is configured to use the shared Redis, by default.
* `--redis-slave-endpoint`: Set external Redis endpoint for clouddriver-ro. The Redis URI schema is described here: https://www.iana.org/assignments/uri-schemes/prov/redis. clouddriver-ro is configured to use the shared Redis, by default.


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,19 @@ public class ClouddriverHaServiceEditCommand extends AbstractHaServiceEditComman
)
private String redisSlaveEndpoint;

@Parameter(
names = "--redis-slave-deck-endpoint",
description = "Set external Redis endpoint for clouddriver-ro-deck. "
+ "The Redis URI schema is described here: https://www.iana.org/assignments/uri-schemes/prov/redis. "
+ "clouddriver-ro-deck is configured to use the shared Redis, by default."
)
private String redisSlaveDeckEndpoint;

@Override
protected HaService editHaService(ClouddriverHaService haService) {
haService.setRedisMasterEndpoint(isSet(redisMasterEndpoint) ? redisMasterEndpoint : haService.getRedisMasterEndpoint());
haService.setRedisSlaveEndpoint(isSet(redisSlaveEndpoint) ? redisSlaveEndpoint : haService.getRedisSlaveEndpoint());
haService.setRedisSlaveDeckEndpoint(isSet(redisSlaveDeckEndpoint) ? redisSlaveDeckEndpoint : haService.getRedisSlaveDeckEndpoint());

return haService;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
public class ClouddriverHaService extends HaService {
String redisMasterEndpoint;
String redisSlaveEndpoint;
String redisSlaveDeckEndpoint;

@Override
public HaServiceType haServiceType() { return HaServiceType.CLOUDDRIVER; }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,23 @@
import com.netflix.spinnaker.halyard.config.model.v1.node.Validator;
import com.netflix.spinnaker.halyard.config.problem.v1.ConfigProblemSetBuilder;
import com.netflix.spinnaker.halyard.core.problem.v1.Problem;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.apache.commons.lang.StringUtils;

public class ClouddriverHaServiceValidator extends Validator<ClouddriverHaService> {
@Override
public void validate(ConfigProblemSetBuilder p, ClouddriverHaService clouddriverHaService) {
if (!StringUtils.isBlank(clouddriverHaService.getRedisMasterEndpoint()) &&
StringUtils.isBlank(clouddriverHaService.getRedisSlaveEndpoint())) {
p.addProblem(Problem.Severity.ERROR, "A Clouddriver Redis master endpoint must be supplied when a slave endpoint is provided.");
}
if (StringUtils.isBlank(clouddriverHaService.getRedisMasterEndpoint()) &&
!StringUtils.isBlank(clouddriverHaService.getRedisSlaveEndpoint())) {
p.addProblem(Problem.Severity.ERROR, "A Clouddriver Redis slave endpoint must be supplied when a master endpoint is provided.");
List<String> redisEndpoints = Stream.of(
clouddriverHaService.getRedisMasterEndpoint(),
clouddriverHaService.getRedisSlaveEndpoint(),
clouddriverHaService.getRedisSlaveDeckEndpoint()
).filter(e -> !StringUtils.isBlank(e))
.collect(Collectors.toList());

if (redisEndpoints.size() != 3 && redisEndpoints.size() != 0) {
p.addProblem(Problem.Severity.ERROR, "Please provide values for Clouddriver Redis master endpoint, Redis slave endpoint, and Redis slave-deck endpoint or leave them all blank.");
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,9 @@ public Class<Gate> getEndpointClass() {
return Gate.class;
}

protected void appendReadonlyClouddriverForDeck(Profile profile, DeploymentConfiguration deploymentConfiguration, SpinnakerRuntimeSettings endpoints) {
}

@Override
public List<Profile> getProfiles(DeploymentConfiguration deploymentConfiguration, SpinnakerRuntimeSettings endpoints) {
List<Profile> profiles = super.getProfiles(deploymentConfiguration, endpoints);
Expand All @@ -76,6 +79,8 @@ public List<Profile> getProfiles(DeploymentConfiguration deploymentConfiguration
GateProfileFactory gateProfileFactory = getGateProfileFactory(deploymentConfiguration.getName());
Profile profile = gateProfileFactory.getProfile(filename, path, deploymentConfiguration, endpoints);

appendReadonlyClouddriverForDeck(profile, deploymentConfiguration, endpoints);

profiles.add(profile);
return profiles;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ public enum Type {
CLOUDDRIVER_BOOTSTRAP(CLOUDDRIVER, "bootstrap"),
CLOUDDRIVER_CACHING(CLOUDDRIVER, "caching"),
CLOUDDRIVER_RO(CLOUDDRIVER, "ro"),
CLOUDDRIVER_RO_DECK(CLOUDDRIVER, "ro-deck"),
CLOUDDRIVER_RW(CLOUDDRIVER, "rw"),
CONSUL_CLIENT("consul-client"),
CONSUL_SERVER("consul-server"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,12 @@ public class KubectlServiceProvider extends SpinnakerServiceProvider<AccountDepl
KubernetesV2ClouddriverCachingService clouddriverCachingService;

@Autowired
@Qualifier("kubernetesV2ClouddriverRoService")
KubernetesV2ClouddriverRoService clouddriverRoService;

@Autowired
KubernetesV2ClouddriverRoDeckService clouddriverRoDeckService;

@Autowired
KubernetesV2ClouddriverRwService clouddriverRwService;

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/*
* Copyright 2018 Google, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License")
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
*/

package com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.distributed.kubernetes.v2;


import com.netflix.spinnaker.halyard.config.model.v1.ha.HaServices;
import com.netflix.spinnaker.halyard.config.model.v1.node.DeploymentConfiguration;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.SpinnakerRuntimeSettings;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.profile.Profile;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.ServiceSettings;
import java.nio.file.Paths;
import java.util.List;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.springframework.stereotype.Component;

@Data
@Component
@EqualsAndHashCode(callSuper = true)
public class KubernetesV2ClouddriverRoDeckService extends KubernetesV2ClouddriverRoService{
@Override
public Type getType() {
return Type.CLOUDDRIVER_RO_DECK;
}

@Override
public List<Profile> getProfiles(DeploymentConfiguration deploymentConfiguration, SpinnakerRuntimeSettings endpoints) {
List<Profile> profiles = super.getProfiles(deploymentConfiguration, endpoints);

String filename = "clouddriver-ro-deck.yml";
String path = Paths.get(getConfigOutputPath(), filename).toString();
profiles.add(getClouddriverProfileFactory().getProfile(filename, path, deploymentConfiguration, endpoints));

return profiles;
}

@Override
protected boolean hasServiceOverrides(DeploymentConfiguration deploymentConfiguration) {
HaServices haServices = deploymentConfiguration.getDeploymentEnvironment().getHaServices();
return haServices.getClouddriver().getRedisSlaveDeckEndpoint() != null;
}

@Override
protected SpinnakerRuntimeSettings getServiceOverrides(DeploymentConfiguration deploymentConfiguration, SpinnakerRuntimeSettings endpoints) {
SpinnakerRuntimeSettings serviceOverrides = super.getServiceOverrides(deploymentConfiguration, endpoints);

serviceOverrides.setServiceSettings(Type.REDIS, new ServiceSettings(deploymentConfiguration.getDeploymentEnvironment().getHaServices().getClouddriver().getRedisSlaveDeckEndpoint()));

return serviceOverrides;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ public Type getType() {
public boolean isEnabled(DeploymentConfiguration deploymentConfiguration) {
return deploymentConfiguration.getDeploymentEnvironment().getHaServices().getClouddriver().isEnabled();
}

@Override
public List<Profile> getProfiles(DeploymentConfiguration deploymentConfiguration, SpinnakerRuntimeSettings endpoints) {
List<Profile> profiles = super.getProfiles(deploymentConfiguration, endpoints);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,16 @@

import com.netflix.spinnaker.halyard.config.model.v1.ha.HaServices;
import com.netflix.spinnaker.halyard.config.model.v1.node.DeploymentConfiguration;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.SpinnakerRuntimeSettings;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.profile.Profile;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.GateService;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.ServiceSettings;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.distributed.DistributedService.DeployPriority;
import com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.distributed.kubernetes.KubernetesSharedServiceSettings;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.experimental.Delegate;
Expand Down Expand Up @@ -71,4 +75,17 @@ protected List<Type> overrideServiceEndpoints() {
Type.ECHO_WORKER
);
}

@Override
protected void appendReadonlyClouddriverForDeck(Profile profile, DeploymentConfiguration deploymentConfiguration, SpinnakerRuntimeSettings endpoints) {
if (hasServiceOverrides(deploymentConfiguration)) {
Map<String, Map<String, Map<String, Map<String, Map<String, String>>>>> services = Collections.singletonMap(
"services", Collections.singletonMap(
"clouddriver", Collections.singletonMap(
"config", Collections.singletonMap(
"dynamicEndpoints", Collections.singletonMap(
"deck", endpoints.getServiceSettings(Type.CLOUDDRIVER_RO_DECK).getBaseUrl())))));
profile.appendContents(getYamlParser().dump(services));
}
}
}

0 comments on commit 92b9963

Please sign in to comment.