Skip to content

Commit

Permalink
Move skywalking cluster to a dedicated module and reorganize the doc
Browse files Browse the repository at this point in the history 
- Add workflow to generate and push doc.
  • Loading branch information
@kezhenxu94
kezhenxu94 committed Sep 2, 2023
1 parent d3f1b21 commit 22fc80c
Show file tree
Hide file tree
Showing 25 changed files with 800 additions and 380 deletions.
1 change: 1 addition & 0 deletions .github/actions/terraform-docs
Submodule terraform-docs added at d1c994
50 changes: 50 additions & 0 deletions .github/workflows/documentation.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: Check Docs
on:
- pull_request

jobs:
docs:
name: ${{ matrix.doc.name }}
runs-on: ubuntu-latest
strategy:
matrix:
doc:
- name: AWS
dir: aws
output-file: configurations.md
- name: AWS SkyWalking Module
dir: aws/modules/skywalking
output-file: README.md
fail-fast: true
steps:
- uses: actions/checkout@v3
with:
submodules: true
ref: ${{ github.event.pull_request.head.ref }}
- name: Render and check docs is updated
uses: ./.github/actions/terraform-docs
with:
working-dir: ${{ matrix.doc.dir }}
output-file: ${{ matrix.doc.output-file }}
output-method: inject
fail-on-diff: "true"
- name: Print diff
if: ${{ failure() }}
run: git diff
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ ansible/local.var.yaml
ansible/inventory
!ansible/inventory/template
.terraform.tfstate.lock.info
terraform.tfvars
3 changes: 3 additions & 0 deletions .gitmodules
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule ".github/actions/terraform-docs"]
path = .github/actions/terraform-docs
url = https://github.com/terraform-docs/gh-actions.git
208 changes: 17 additions & 191 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,200 +1,26 @@
# SkyWalking Terraform and Ansible

This repository contains the Terraform scripts to create the infrastructure for SkyWalking on cloud vendors,
and the Ansible playbooks to install SkyWalking on the created infrastructure, or on the existing infrastructure,
no matter on-premises or on cloud vendors, such as AWS.

# Terraform

**Notice, HashiCorp had changed the LICENSE of Terraform from MPL 2.0 to BSL/BUSL 1.1 since its 1.5.6 release. We don't have hard-dependencies on Terraform.**

**OpenTF Foundation announced to maintain the MPL 2.0 based fork of Terraform. Read their [announcement](https://opentf.org/announcement) and [website](https://opentf.org/) for more details.**

**All Terraform and/or OpenTF scripts are just for end-user convenience. The Apache 2.0 License is only for the scripts.**
# Terraform module for SkyWalking

For now, we have supported the following cloud vendors, and we welcome everyone to contribute supports for
more cloud vendors:

- Amazon Web Services (AWS): go to the [aws](aws) folder for more details.

## Prerequisites

1. Terraform installed
2. AWS Credentials: Ensure your environment is set up with the necessary AWS credentials. This can be done in various ways, such as:
- Configuring using the AWS CLI.
- Setting up environment variables (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`).
- Using IAM roles with necessary permissions if you're running Terraform on an AWS EC2 instance.
- For more information on configuring AWS credentials for Terraform, see the [official documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).
3. A working knowledge of Terraform and AWS resources

## Instructions

### 1. Initialization

Before applying any Terraform script, initialize your Terraform working directory:

```bash
cd aws/
terraform init
```

### 2. Configuration

The script is designed with modularity and reusability in mind. Various parameters like region, instance count, instance type, etc., are exposed as variables for easier customization.

For the full configuration list, please refer to [the doc](/aws/README.md).

To modify the default values, you can create a `terraform.tfvars` file in the same directory as your Terraform script:

```bash
oap_instance_count = 2
ui_instance_count = 2
region = "us-west-1"
instance_type = "t2.large"
extra_tags = {
"Environment" = "Production"
}
```

### 3. Test and apply the outcomes of the Script

After adjusting your configuration, test and apply the script:

```bash
terraform plan
terraform apply
```

After all the resources are created, you can head to the
[Ansible part](#ansible) to start deploying SkyWalking.

### 4. Accessing the Resources

#### SSH into bastion host (Optional)

You don't usually need to SSH into the bastion host, but if you want to, you can
SSH into the bastion host with the command:

```shell
KEY_FILE=$(terraform output -raw ssh-user-key-file)
BASTION_IP=$(terraform output -json bastion_ips | jq -r '.[0]')

ssh -i "$KEY_FILE" ec2-user@"$BASTION_IP"
```

- **Security Attention**: two security rules are created for the bastion host:
- `ssh-access`: Allows SSH access from any IP (`0.0.0.0/0`). **Please note** that this is potentially insecure and you should restrict the IP range wherever possible.
- `public-egress-access`: Allows egress access to the internet for the instances.

### 5. Tearing Down

To destroy the resources when they are no longer needed:

```bash
terraform destroy
```

This command will prompt you to confirm before destroying the resources.

## Security Note

SSH access is open to the entire internet (`0.0.0.0/0`). This is not recommended for production environments. Always restrict the CIDR block to known IP ranges for better security.

# Ansible

You can use the Ansible playbook in combination with the Terraform to create necessary infrastructure and install
SkyWalking on the created infrastructure, or you can use the Ansible to install SkyWalking on the existing infrastructure.

This guide provides steps on using Ansible to install Apache SkyWalking on AWS instances.

## Prerequisites

1. Ansible installed.
2. A working knowledge of Ansible and AWS resources.
3. An active SSH key and access to AWS EC2 instances.

## Instructions

### 1. Change diroectory

```shell
cd ../ansible/
```

### 2. Test Connectivity to the EC2 Instances

Before installing SkyWalking, ensure that you can connect to the EC2 instances:

```
ansible -m ping all -u ec2-user
```

**Expected Output**:

You should see output for each IP with a `SUCCESS` status:
```text
<ip1> | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
<ip2> | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
```

### 3. Install Apache SkyWalking

After confirming connectivity, proceed to install Apache SkyWalking using the Ansible playbook:

```
ansible-playbook skywalking.yml
```

### 4. Configurations

The Ansible playbook can be customized to install Apache SkyWalking with
different configurations. The following variables can be modified to suit your
needs:

> For full configurations, refer to the
> [ansible/roles/skywalking/vars/main.yml](ansible/roles/skywalking/vars/main.yml).
> file.
```yaml
# `skywalking_tarball` can be a remote URL or a local path, if it's a remote URL
# the remote file will be downloaded to the remote host and then extracted,
# if it's a local path, the local file will be copied to the remote host and
# then extracted.
skywalking_tarball: "https://dist.apache.org/repos/dist/release/skywalking/9.5.0/apache-skywalking-apm-9.5.0.tar.gz"

# `skywalking_ui_environment` is a dictionary of environment variables that will
# be sourced when running the skywalking-ui service. All environment variables
# that are supported by SkyWalking webapp can be set here.
skywalking_ui_environment: {}

# `skywalking_oap_environment` is a dictionary of environment variables that will
# be sourced when running the skywalking-oap service. All environment variables
# that are supported by SkyWalking OAP can be set here.
skywalking_oap_environment: {}

```
- [AWS](aws): Terraform scripts to provision necessary resources on Amazon Web Services.

### 5. Accessing SkyWalking UI!
> [!NOTE]
> HashiCorp had changed the LICENSE of Terraform from MPL 2.0 to BSL/BUSL 1.1
> since its 1.5.6 release. We don't have hard-dependencies on Terraform.
>
> OpenTF Foundation announced to maintain the MPL 2.0 based fork of Terraform.
> Read their [announcement](https://opentf.org/announcement) and
> [website](https://opentf.org/) for more details.
>
> All Terraform and/or OpenTF scripts are just for end-user convenience.
> The Apache 2.0 License is only for the scripts.
After the installation is complete, you can go back to the aws folder and get
the ALB domain name address that can be used to access the SkyWalking UI:

```shell
cd ../aws
terraform output -raw alb_dns_name
```
# Ansible playbook for SkyWalking

And you can open your browser and access the SkyWalking UI with the address.
You can use the Ansible playbook in combination with the Terraform to create
necessary infrastructure and install SkyWalking on the created infrastructure,
or you can use the Ansible to install SkyWalking on the existing infrastructure.

Please go to the [ansible](ansible) folder for more details.
108 changes: 108 additions & 0 deletions ansible/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
This guide provides steps on using Ansible to install Apache SkyWalking on VM instances.

# Prerequisites

- [Ansible installed](https://docs.ansible.com/ansible/latest/installation_guide/index.html).
- A working knowledge of Ansible.
- Access to instances.

# Instructions

## Change diroectory

```shell
cd ansible
```

## Test Connectivity to the Instances

Before installing SkyWalking, ensure that you can connect to the instances:

```shell
ansible -m ping all
```

**Expected Output**:

You should see output for each IP with a `SUCCESS` status:

```text
<ip1> | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
<ip2> | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
```

## Install Apache SkyWalking

After confirming connectivity, proceed to install Apache SkyWalking using the Ansible playbook:

```
ansible-playbook skywalking.yml
```

## Configurations

The Ansible playbook can be customized to install Apache SkyWalking with
different configurations. The following variables can be modified to suit your
needs:

> For full configurations, refer to the
> [roles/skywalking/vars/main.yml](roles/skywalking/vars/main.yml).
> file.
```yaml
# `skywalking_tarball` can be a remote URL or a local path, if it's a remote URL
# the remote file will be downloaded to the remote host and then extracted,
# if it's a local path, the local file will be copied to the remote host and
# then extracted.
skywalking_tarball: "https://dist.apache.org/repos/dist/release/skywalking/9.5.0/apache-skywalking-apm-9.5.0.tar.gz"

# `skywalking_ui_environment` is a dictionary of environment variables that will
# be sourced when running the skywalking-ui service. All environment variables
# that are supported by SkyWalking webapp can be set here.
skywalking_ui_environment: {}

# `skywalking_oap_environment` is a dictionary of environment variables that will
# be sourced when running the skywalking-oap service. All environment variables
# that are supported by SkyWalking OAP can be set here.
skywalking_oap_environment: {}

```
You can create a local variable file to override the default values:
```shell
cat <<EOF > local.var.yaml
skywalking_tarball: "~/workspace/skywalking/apm-dist/target/apache-skywalking-apm-bin.tar.gz"
EOF
```

And then run the playbook with the local variable file:

```shell
ansible-playbook skywalking.yml -e @local.var.yaml
```

## Accessing SkyWalking UI!

After the installation is complete, you can go back to the aws folder and get
the ALB domain name address that can be used to access the SkyWalking UI:

```shell
cd ../aws
terraform output -raw alb_dns_name
```

And you can open your browser and access the SkyWalking UI with the address.

Loading

0 comments on commit 22fc80c

Please sign in to comment.