Skip to content

Commit

Permalink
Merge pull request #14 from nzlosh/doc_update
Browse files Browse the repository at this point in the history 
Update documentation and improve unit testing
  • Loading branch information
@nzlosh
nzlosh authored May 15, 2019
2 parents b1b89bb + 791c041 commit 7a77859
Show file tree
Hide file tree
Showing 5 changed files with 317 additions and 59 deletions.
7 changes: 2 additions & 5 deletions .circleci/config.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ jobs:
command: |
python3 -m venv venv
. venv/bin/activate
pip install errbot
errbot --init
pip install -r requirements.txt
pip install -r test_requirements.txt
Expand All @@ -41,15 +43,10 @@ jobs:
key: v1-dependencies-{{ checksum "requirements.txt" }}

# run tests!
# this example uses Django's built-in test-runner
# other common Python testing frameworks include pytest and nose
# https://pytest.org
# https://nose.readthedocs.io
- run:
name: run tests
command: |
. venv/bin/activate
#python manage.py test
pytest test_err-stackstorm.py
- store_artifacts:
Expand Down
34 changes: 30 additions & 4 deletions Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,30 @@
test:
pytest test_err-stackstorm.py
flake8 --max-line-length=100 st2.py lib/*.py || true
pep8 --max-line-length=100 st2.py lib/*.py
SHELL=/bin/sh
VENVDIR=/tmp/err-stackstorm-venv

.PHONY: all
all: setup lint_test unit_test

.PHONY: clean
clean:
rm -rf "${VENVDIR}"

.PHONY: setup
setup:
test -d "${VENVDIR}" || virtualenv -p python3 "${VENVDIR}"
. ${VENVDIR}/bin/activate
${VENVDIR}/bin/pip install errbot flake8 pep8 pytest mock
${VENVDIR}/bin/pip install -r requirements.txt
cd "${VENVDIR}" && "${VENVDIR}/bin/errbot" --init

.PHONY: activate
activate:
. ${VENVDIR}/bin/activate

.PHONY: unit_test
unit_test: activate
${VENVDIR}/bin/pytest test_err-stackstorm.py

.PHONY: lint_test
lint_test: activate
${VENVDIR}/bin/flake8 --max-line-length=100 st2.py lib/*.py || true
${VENVDIR}/bin/pep8 --max-line-length=100 st2.py lib/*.py
166 changes: 117 additions & 49 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,36 @@ A plugin to run StackStorm actions, bringing StackStorm's ChatOps to Errbot.
1. [Troubleshooting](#Troubleshooting)

## Help <a name="GettingHelp">
You can find users of err-stackstorm on Gitter. https://gitter.im/err-stackstorm/community or Slack https://stackstorm-community.slack.com `#err-stackstorm`. If you think you've found a bug or need a new feature open an issue on the [github repository](https://github.com/nzlosh/err-stackstorm/issues). If you want to contribute to the err-stackstorm project, there's plenty of improvements to be made, contact nzlosh via chat or email to discuss how you can get involved.

You can find users of err-stackstorm on Gitter. https://gitter.im/err-stackstorm/community or Slack https://stackstorm-community.slack.com `#err-stackstorm`. There's also the comprehensive [troubleshooting](#Troubleshooting) section to help guide you through possible break/fix scenarios.

If you think you've found a bug or need a new feature open an issue on the [github repository](https://github.com/nzlosh/err-stackstorm/issues).

If you want to contribute to the err-stackstorm project, there are plenty of improvements to be made, contact nzlosh via chat or email to discuss how you can get involved.

## Installation <a name="Installation"></a>
Installation of the err-stackstorm plugin can be performed from within a running Errbot instance or from the command line using `git clone`. Ensure Errbot is up and running before attempting to install the plugin. See the Errbot installation documentation here https://github.com/Errbotio/Errbot for instructions on how to setup Errbot on your chat back-end. These instructions assume a running instance of StackStorm is already in place. See the official [StackStorm documentation](https://docs.stackstorm.com/install/index.html) for details.
Installation of the err-stackstorm plugin can be performed from within a running Errbot instance. Ensure Errbot is up and running before attempting to install the plugin. See the Errbot installation documentation here https://github.com/errbotio/errbot for instructions on how to setup Errbot on your chat back-end. These instructions assume a running instance of StackStorm is already in place. See the official [StackStorm documentation](https://docs.stackstorm.com/install/index.html) for details.

1. Install Errbot on the target system using standard package manager or Errbot installation method.
1. Configure Errbot, see the [Configuration](#Configuration) section for help.
1. Enable Errbot's internal web server, see the [Webhook](#Webhook) section for help.
1. Install ChatOps pack on StackStorm, see the [ChatOps Pack](#ChatOpsPack) section for help.
1. Connect to your chat back-end and starting interacting with your StackStorm/Errbot instance.

The below command will install the plugin.
```
!repos install https://github.com/nzlosh/err-stackstorm.git
```
*If errors are encountered, the plugin will fail to be installed. Check the configuration is correct.*
After completing the above steps, issue The below command will install the plugin.

1. Confirm Errbot is configured to install plugin dependencies. The below line should be present in `config.py`, if it is not add it.
```
AUTOINSTALL_DEPS = True
```
1. From the chat backend, issue the command to install err-stackstorm.
```
!repos install https://github.com/nzlosh/err-stackstorm.git
```
*The plugin will fail to install if any errors are encountered. This is often caused by configuration errors in `config.py`.*

## Requirements <a name="Requirements"></a>
The plugin has been developed and tested against the below software. For optimal operation it is recommended to use the following versions:
The plugin has been developed and tested against the below software combinations. For optimal operation it is recommended to use the following versions:

plugin tag (version) | Python | Errbot | StackStorm client
--- | --- | --- | ---
Expand All @@ -46,7 +57,7 @@ plugin tag (version) | Python | Errbot | StackStorm client
1.0 | 2.7 | 3.x | 2.1

## Supported Chat Back-ends <a name="SupportedChatBackends"></a>
Errbot provides official support for a few of major chat back-ends and many more chat back-ends are available through unofficial plugins.
Errbot provides official support for most major chat back-ends and many more chat back-ends are available through unofficial plugins.

Back end | Mode value | Support type
--- | --- | ---
Expand All @@ -67,6 +78,7 @@ Currently supported extra back-ends

## Configuration <a name="Configuration"></a>
Edit the `config.py` configuration file which is used to describe how the plugin will communicate with StackStorm's API and authentication end points.

If you followed the Errbot setup documentation this file will have been created by downloading a template from the Errbot GitHub site. If this file has not already been created, please create it following the instructions at https://github.com/Errbotio/Errbot

```
Expand All @@ -88,7 +100,7 @@ STACKSTORM = {
'rbac_auth': {
'standalone': {},
},
'timer_update': 900, # Unit: second. Interval for Errbot to refresh to list of available action aliases.
'timer_update': 900, # Unit: second. Interval to check the user token is still valid.
}
```

Expand All @@ -102,16 +114,20 @@ Option | Description
`api_auth.user.password` | Errbot password to authenticate with StackStorm.
`api_auth.token` | Errbot user token to authenticate with StackStorm. Used instead of a username/password pair.
`api_auth.key` | Errbot API key to authenticate with StackStorm. Used instead of a username/password pair or user token.
`timer_update` | Unit: seconds. Default is *60*. Interval for err-stackstorm to refresh to list of available action aliases. (deprecated)
`timer_update` | Unit: seconds. Default is *60*. Interval for err-stackstorm to the user token is valid.
`rbac_auth.standalone` | Standalone authentication.
`rbac_auth.clientside` | Clientside authentication, a chat user will supply StackStorm credentials to err-stackstorm via an authentication page.
`rbac_auth.clientside.url` | Url to the authentication web page.
`secrets_store.cleartext` | Use the in memory store.


### Locale
Errbot uses the systems locale for handling text, if your getting errors handling non ascii characters from chat
Errbot uses the systems locale for handling text, if you're getting errors handling non-ascii characters from chat
`UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 83: ordinal not in range(128)`
Make sure the systems locale is using a unicode encoding, which is indicted by the `.UTF8` extension.

Make sure the systems locale is configured for unicode encoding. In the below example, the machine
was set the English (`en`) New Zealand (`NZ`) with utf-8 encoding (`.UTF8`).

```
# locale
LANG=en_NZ.UTF8
Expand Down Expand Up @@ -160,38 +176,41 @@ The secrets store is used by err-stackstorm to cache StackStorm API credentials.
#### ClearText
The `cleartext` store maintains the cache in memory and does not encrypt the contents to disk. This is option doesn't protect the stored secrets in memory.

### Role Based Access Control Authentication

#### Standalone RBAC
### StackStorm Authentication Credentials

This is the original authentication method where by err-stackstorm uses its own credentials for all
#### Standalone RBAC Authentication

This is the original authentication method where err-stackstorm used its own credentials for all
calls to the StackStorm API. All action-alias' issued by chat service users execute the underlying
workflows with err-stackstorm credentials.
workflows with err-stackstorm credentials. Any Role Based Access Control policies would only be
applied to the bot user which gives coarse control.


##### Configuration
It is considered an error to have multiple RBAC authentication configurations present at the same time.
An empty dictionary in the *standalone* key is all that is required to maintain Err-stackstorm's
Do not configure multiple RBAC authentication configurations at the same time.
An empty dictionary in the *standalone* key is all that is required to maintain err-stackstorm's
original authentication method.
```
'rbac_auth': {
'standalone': {},
},
```

#### Client-side RBAC
#### Client-side RBAC Authentication

Err-stackstorm provides a way to associate the chat service user account with a StackStorm
username/password, user token or api token.
err-stackstorm provides a way to associate the chat service user account with a StackStorm
username/password or api token.

This implementation is specific to err-stackstorm. It is achieved by requesting a new authentication
session with err-stackstorm. A Universally Unique Identifier (UUID) is generated for the session
and the chat user is invited to follow a URL to the authentication page hosted by errbot. For
security reasons, the UUID is a one time use and is consumed when the page is accessed.
This implementation is specific to err-stackstorm. It is achieved by requesting a new
authentication session with err-stackstorm. A Universally Unique Identifier (UUID) is generated
for the session and the chat user is invited to follow a URL to the authentication page hosted by
errbot. For security reasons, the UUID is a one time use and is consumed when the page is
accessed. Any subsequent attempts to access the page will result in an error.

The login page must be protected by TLS encryption and ideally require an ssl client certificate.
The login page should not be exposed directly to the internet, but have a reverse proxy such as
nginx place between it and any external service consumers.
nginx placed between it and any external service consumers.

The user enters their StackStorm credentials via the login page which err-stackstorm will validate
against the StackStorm API. If the credentials are valid, the user token or api key will be cached
Expand All @@ -202,21 +221,62 @@ using the associated StackStorm credentials.


##### Configuration
It is considered an error to have both *extended* and *proxied* RBAC authentication configurations
present at the same time. A *proxied* key with *url* is required to correctly
configure Out-of-bands authentication for ChatOps.
A *url* is required to correctly configure client-side authentication for ChatOps.
```
'rbac_auth': {
'clientside': {
'url': 'https://<hostname>:<port>/',
'url': 'https://<hostname>:<port>/'
}
},
```

Option | Description
--- | ---
`url` | Errbot's authentication web server end point. Used for the out-of-bands authentication web page.
`url` | Errbot's authentication web server end point. Used for the client-side authentication web page.

## Errbot ACLs
Errbot comes with native Access Control List support. It can be configured to constrain command execution by grouping `command`, `channel` and `user`. Glob patterns can be used in each field to provide flexibility in ACL definitions.

As an example, a StackStorm instance has an automatic package upgrade workflow. Its progress can be viewed by executing the action alias: `apu stats <role>`, which is defined as shown below.

```
| action_ref | st2dm_apu.apu_status |
| formats | [ |
| | { |
| | "representation": [ |
| | "apu status {{role}}" |
| | ], |
| | "display": "apu status <role>" |
| | } |
| | ] |
```
The below Errbot ACL configuration allows `@user1` to view the status of the upgrade, but not start/stop the upgrade process (they are other action aliases that are triggered with `st2 apu ...`)
```
ACL_SQUAD_INFRA = ["@admin1", "@admin2", "@admin3", "@admin4"]
ACL_APU_USERS = ['@user1']
ACL_EVERYONE = ["*"]
ACCESS_CONTROLS = {
'whoami': {
'allowrooms': ['@bot_user'],
'allowusers': ACL_EVERYONE
},
'st2 apu status*':{
'allowrooms': ['#channel'],
'allowusers': ACL_SQUAD_INFRA + ACL_APU_USERS
},
'st2 apu*':{
'allowrooms': ['#channel'],
'allowusers': ACL_SQUAD_INFRA
},
}
```


Getting the correct usernames to fill into `allowusers`/`denyusers` isn't obvious. On a small
scale it's possibel to use the `!whoami` command to get the correct user account name. For large
installation it'd make more sense to use a pre-defined pattern.

Errbot matches `username` against the ACL definition. This information isn't found easily in the Slack interface. Use errbot's `!whoami` command to find the value from the `nick` field which can be used with ACL definitions.


## How to expose action-aliases as plugin commands <a name="ActionAliases"></a>
Expand All @@ -232,20 +292,21 @@ Errbot has a built in web server which is configured and enabled through the bot

To configure Errbot's web server plugin, the command below can be sent to Errbot:
```
!plugin config Webserver {'HOST': '0.0.0.0', 'PORT': 3141,
'SSL': {'enabled': False, 'host': '0.0.0.0', 'port': 3142, 'certificate': '', 'key': ''}}
!plugin config Webserver {'HOST': '0.0.0.0', 'PORT': 3141, 'SSL': {'enabled': False, 'host': '0.0.0.0', 'port': 3142, 'certificate': '', 'key': ''}}
```

**NOTE:** _The variables must be adjusted to match the operating environment in which Errbot is running. See Errbot documentation for further configuration information._

The configuration above is only applied for the current runtime and will not
persist after the errbot process being restarted. Making the configuration
change permanent is as simple as installing a special plugin:
---
### Optional
The configuration above is only applied for the current runtime and may not persist after the errbot
process is restarted. Making the configuration change permanent is as simple as installing a
special plugin:
```
!repos install https://github.com/tkit/errbot-plugin-webserverconfiguration
```
The configuration command from above is not required prior to installing this
plugin.
The configuration command from above is not required prior to installing this plugin.
---

In production environments it may be desirable to place a reverse-proxy like nginx in front of errbot.

Expand All @@ -259,12 +320,9 @@ enabled for full ChatOps support between StackStorm and Errbot.

## StackStorm ChatOps pack configuration. <a name="ChatOpsPack"></a>

StackStorm's [ChatOps pack](https://github.com/StackStorm/st2/tree/master/contrib/chatops) is required
to be installed and a notify rule file added to the pack.
StackStorm's [ChatOps pack](https://github.com/StackStorm/st2/tree/master/contrib/chatops) is required to be installed and a notify rule file added to the pack.

The notify rule must be placed in `/<stackstorm installation>/packs/chatops/rules`. The rule file
[notify_errbot.yaml](https://raw.githubusercontent.com/fmnisme/err-stackstorm/master/contrib/stackstorm-chatops/rules/notify_errbot.yaml) can be found
in this repository under
The notify rule must be placed in `/<stackstorm installation>/packs/chatops/rules`. The rule file [notify_errbot.yaml](https://raw.githubusercontent.com/nzlosh/err-stackstorm/master/contrib/stackstorm-chatops/rules/notify_errbot.yaml) can be found in this repository under

Edit the `chatops/actions/post_message.yaml` file to use the errbot route as it's default value.
```
Expand All @@ -274,6 +332,14 @@ Edit the `chatops/actions/post_message.yaml` file to use the errbot route as it'

## Troubleshooting <a name="Troubleshooting"></a>

### Is the virtual environment active?

If Errbot was installed in a python virtual environement and a `command is not found` is reported.
```
errbot --init, the command is not found.
```
Make sure the virtual environment is activated correctly.

### Is the Errbot process running?
Check an instance of Errbot is running on the host
```
Expand All @@ -282,15 +348,15 @@ root 158707 0.1 0.0 2922228 59640 pts/21 Sl+ Aug14 2:29 | \_ /opt/e
```

### Is the Errbot webhook listening?
Check Errbot's internal web server is listening on the correct interface.
Check Errbot's internal web server is listening on the correct interface. Use the PID `158707` from the previous command to filter the output to find which port the errbot process is listening on. If the result is empty, errbot is not listening on any ports.
```
# ss -tlpn | grep 158707
LISTEN 0 128 *:8888 *:* users:(("errbot",158707,21))
LISTEN 0 128 *:3141 *:* users:(("errbot",158707,21))
```
OR
```
# netstat -tlpn | grep 158707
tcp 0 0 0.0.0.0:8888 0.0.0.0:* LISTEN 158707/python3
tcp 0 0 0.0.0.0:3141 0.0.0.0:* LISTEN 158707/python3
```

### Is the Errbot machine able to communicate with the StackStorm end points?
Expand Down Expand Up @@ -376,7 +442,7 @@ _This is a basic implementation of a chatroom_
```

### Is the StackStorm ChatOps pack installed and configured correctly?
Err-stackstorm requires the ChatOps pack to be installed. To confirm it is installed, use the st2 cli.
err-stackstorm requires the ChatOps pack to be installed. To confirm it is installed, use the st2 cli.

```
$ st2 pack list
Expand Down Expand Up @@ -479,7 +545,9 @@ This indicates that the route wasn't set to `errbot`, see the Install ChatOps se

## Acknowledgements

The err-stackstorm plugin was founded by [fmnisme](https://github.com/fmnisme), thanks for starting the project.
A special thank you to [fmnisme](https://github.com/fmnisme) who started the err-stackstorm plugin project.
Thanks to the StackStorm and Errbot community for their support and interest.


## Legal

Expand Down
Loading

0 comments on commit 7a77859

Please sign in to comment.