Skip to content

Commit

Permalink
#23 | Removed bash_unit Tests and Added shellspec Tests (#30)
Browse files Browse the repository at this point in the history
* Removed old `bash_unit` tests, re-structured test directories (will need to do that again when I start adding functional tests), moved unit test README.md and made some minor updates to it, added tests of createHeaderFooter() using the new shellspec library, and setup a bunch of stuff that's needed by shellspec.

* Created some unit tests of createHeaderFooter using `shellspec`, updated top level README.md so it points to the correct location when it comes to the unit test README.md, removed the old GitHub actions files that used the old unit test library, broke out a test from one of the larger files, and fixed a number of bugs found while testing createHeaderFooter. Oh, I also made some updates to the createHeaderFooter doc.

* Corrected typo in createdHeaderFooter doc and created tests of help option.

* Moved location of unit tests so when I add functional tests in the future, I will just need to create a directory for them. Updated top level readme to reflect those changes. Moved unit test README.md for the same reason. Renamed unit test files to consistently follow a naming conv. Added a test of --prefix option of createHeaderFooter. Corrected name of output constants file and ensured new file name was updated where it's sourced. Cleaned up doc of createHeaderFooter by removing a bunch of stuff that's no longer true. Minor code refactor of createHeaderFooter so it looks cleaner.

* Added first few tests of output() constants file.

* Corrected tag name in createHeaderFooter unit test. Added some functions that are likely to be used in a bunch of places to spec_helper.sh. Finished creating tests of output() constants.

* Updated test README.md files with updated/correct info.

* Added logic to output() that allows it to differentiate between return code 141 and 142. Updated doc. Removed usage of a number of printf statments. Redirected error messages to stderr. Added logic that won't call date if not defined in environment. Added tr command to shellspec sandbox.

* Added a number of unit tests of output(). Still need to add more.

* Added last tests of output()'s message option and added some detail to the description of one of output()'s return codes.

* Revampted project README.md files.

* Updated test to follow standards documented in last commit.

* Made what I think is the final change to the unit test struc. standard. Also updated existing tests to follow it.

* Wrote a bunch more unit tests. Refactored how output() and createHeaderFooter() imports constants file so unit tests can mock it out.

* Added unit tests, moved all sourcing of `output()` util files to `main.sh` to prevent double sourcing, coverted `createHeaderFooter()` to function, and refactored all existing unit tests (along with other stuff).

shellspec config updates:
- Added standard return value globals, link to spec helper portion of shellspec docs, and methods for checking if a value is posative or not negative to spec_helper.sh.

`output()` unit test updates:
- Updated top level `Define` section of all `output()` unit tests with more accurate line comment and mocked out `createHeaderFooter()` for all test files that contain only tests that should never use that method.
- Removed `$` character from formatting characters returned by mockout out `createHeaderFooter()` because that character isn't supported.
- Added unit test of `output()` when no arguments or options are provided to message unit test file.
- Added unit tests that ensure `output()` handles `createHeaderFooter()` failures correctly.
- Added unit tests that ensure `DEFAULT_INDENT` is handled correctly.
- Moved some tests from the unit tests of the --prefix option that combined that with prefix to a seprate file. Think I might have also added some tests of that option combination.
- Added unit tests of --pretty option.
- Added unit tests of --header-footer option.
- Added unit tests of --line-length option.
- Added unit tests that ensure --pretty and --indent work together.
- Mocked out `cat` command in unit tests of --header-footer and --pretty tests.

Unit test updates:
- Removed mocking out of `inScriptSouce()` from all unit tests of `createHeaderFooter()` because the file that method is in no longer sources anything. Also updated all unit tests of `createHeaderFooter()` to source `createHeaderFooter()` in a `BeforeAll` rather than having tests source the path that the file that contains `createHeaderFooter()` is in. This might speed up the tests, but it has to be done because `createHeaderFooter()` is now a function rather than just code in a file.
- Refactored `createHeaderFooter_dateMocking_spec.sh` unit test so the structureis more simple.
- Added tests of formatting character option of `createHeaderFooter()` in order to increase code coverage.
- Added tests and refactored existing ones of line length option of `createHeaderFooter()` in order to better test boundry values and increase code coverage.
- Corrected unit test that passes nothing to `createHeaderFooter()`.
- Added test of prefix option of `createHeaderFooter()` in order to increase code coverage.

`output()` udpates:
- Moved sourcing of util files to `main.sh` to: avoid double sourcing, remove complexity from file that contains `output()`, and to make testing easier.
- Now source `main.sh`.
- Renamed varaible that stores log prefix so it's unique. This also prevents it from confilting with helper function(s).
- Refactored how options that take numbers are parsed so they are more strict and will now catch invalid input and return the correct value.
- Removed some error checking code because it's now handled by the option parsing code.
- Refactored error checking bellow option parsing so it's easier to understand that if anything fails, the code will exit.
- Refactored how `createHeaderFooter()` is called because it's now sourced rather than accessed directly.
- Removed usage of `+=` to increase POSIX compatability.

`createHeaderFooter()` updates:
- `createHeaderFooter()` is now a function. This was done mostly to make testing easier, but it also cleans up the process of calling it.
- I also removed all sourcing from the file that contains `createHeaderFooter()` because all sourcing of `output()` util files has been moved to a `main.sh` file so stuff isn't sourced more than once.
Added defenition of DOD to top level README.md.
- Removed sourcing because it's now handled by `main.sh` (which is sourced by `output()`).
- Refactored how options that should be numbers are parsed so it's more strict. This allows invalid option values to be cought.

`main.sh` created:
- This file handles all sourcing of `output()` util files.
- Sourced by `output()`.

`.gitignore` now ignores coverage results as well as a temp file I sometimes use.

* Removed use of `function` bashism.

* Converted all instances of `${@}` to `$@` because the `{` and `}` arn't needed.

* Apply suggestions from code review

Minor doc changes.

* Apply suggestions from code review

* Update src/shell/shell_aliases

* Apply suggestions from code review

* Replaced TODOs in root README.md with correct text.

* Replaced TODOs with text in functions README.md

* Removed a TODO from the src README.md

* Apply suggestions from code review
  • Loading branch information
ReedClanton authored Apr 12, 2023
1 parent 1afcfeb commit 6a4e6cd
Show file tree
Hide file tree
Showing 62 changed files with 3,260 additions and 2,287 deletions.

This file was deleted.

13 changes: 0 additions & 13 deletions .github/workflows/tests-unit-output.yml

This file was deleted.

2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
# Ensure backups of local user's bash file(s) aren't included.
/src/environmentSetup/*-homeBackUp-*
tmp.sh
coverage/
22 changes: 22 additions & 0 deletions .shellspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# For more information regarding shellspec options, see: https://github.com/shellspec/shellspec#command-options
###############################
## shellspec Default Option(s) ##
###############################
# Create virtual environment around unit tests.
# TODO #35: When I start creating functional tests I'll need change this so it only applies to unit tests.
--sandbox
# Ensure spec helper file is present and used (https://github.com/shellspec/shellspec#spec_helper).
--require spec_helper
--fail-no-examples
--warning-as-failure

## Default kcov (coverage) options
# --kcov-options "--include-path=. --path-strip-level=1"
# --kcov-options "--include-pattern=.sh"
# --kcov-options "--exclude-pattern=/.shellspec,/spec/,/coverage/,/report/"

## Example: Include script "myprog" with no extension
# --kcov-options "--include-pattern=.sh,myprog"

## Example: Only specified files/directories
# --kcov-options "--include-pattern=myprog,/lib/"
54 changes: 29 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,45 @@
# Shell Base

I create symbolic links to a local clone of this repo in my local environment. This repo provides almost all environment files I use on my machines. I provide no warranties.
This project contains all shell configuration files needed to setup a POSIX compilent shell environment. It also includes shell functions that may be utalized by user created scripts (e.g. log).

## Setup

Run `environment_setup.sh` file to remove your current bash file(s) and directory(ies), any file in your home directory that starts with `.bash*` or directory named `bash`, and replace them with the bash files from this repo.

## Functionality

This section covers what functionality is provided by this repo. For more info, see the `README.md` files in the directory that's closest to the code you want to know more about or use the links from this section. In other words, this section is just for letting you know what functionality exists.
There's a script that uses this project's contents to setup a user's shell environment.

### Source Code
## Terminology

Section covers where to go to find out more about the functionality provided by the code in this repo. There is another section that covers tests.
| Term | Meaning | Refrence |
|:------------------------------ |:-------------------------------------------------------------- |:--------------------------------------- |
| CUT | **C**ode **U**nder **T**est | |
| `Describe`/`ExampleGroup`/`Context` | | [shellspec-Basic structure](https://github.com/shellspec/shellspec#basic-structure) |
| DOD | Defenition Of Done | |
| environment setup script | Environment configuration script. | [environment setup script](src/environmentSetup/environmentSetup.sh) |
| functions readme | `README.md` covering **all** function(s). | [functions readme](src/shell/functions/README.md) |
| {local} user shell environment | Files used by Unix based/derived systems to configure a user's environment (aliases, PATH, etc). | |
| project readme | This `README.md`. | [project readme](README.md) |
| shell readme | `README.md` covering shell configureation file(s). | [shell readme](src/shell/README.md) |
| `stderr` | Output stream that error data is published to. | Google it. |
| `stdout` | Output stream that most return values are published to. | Google it. |
| source readme | `README.md` covering **all** source code. | [source readme](src/README.md) |
| test readme | `README.md` covering **all** testing. | [test readme](spec/README.md) |
| unit test readme | `README.md` covering **unit** testing. | [unit test readme](spec/unit/README.md) |

#### [Basic Environment Setup and Configuration](src/README.md)
## Goal(s)

There is an environment setup script here as well as the base file(s) used when configuring a POSIX complient environment. The `README.md` linked to above contains more info.
Make setting up a new POSIX complient shell environment as easy as possible and provided additional shell script functionality, like a logger.

Please note that an attempt is made to ensure everything is POSIX complient, but most of my systems run `bash`.

#### [Advanced Environment Configuration](src/shell/README.md)

These file(s) are used (sourced) by the files covered by [Basic Environment Setup and Configuration](#basic-environment-setup-and-configuration). The `README.md` linked to above contains more info.
## Setup

#### [POSIX Functions](src/shell/functions/README.md)
Run the [environment setup script](src/environmentSetup/environmentSetup.sh) to remove your current user shell configuration file(s) and directory(ies). For more information regarding shell environment configration, see the [shell readme](src/shell/README.md).

These file(s) contain POSIX complient function(s). The `README.md` linked to above contains more info.
## Functionality

#### [POSIX Function Constants](src/shell/functions/constants/README.md)
### Shell Environment Configuration

These file(s) contain constants used by the POSIX functions in the parient directory. The `README.md` linked to above contains more info.
See the [shell readme](src/shell/README.md) for information regarding what user shell configuration is handled by this project and how you can deploy it to your local user shell environment.

### Tests
### Shell Scripting Utility(ies)

TODO
See the [functions readme](src/shell/functions/README.md) for information regarding the functionality this project provides that you can use in the shell scripts you create.

#### [Unit Tests](tests/unit/README.md)
## Testing

The `README.md` linked to above contains more info.
See the [test readme](spec/README.md) for information regarding how tests of this project are organized, what the goals of the tests are, test requirements, etc.
11 changes: 11 additions & 0 deletions spec/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Testing

The [`shellspec`](https://github.com/shellspec/shellspec#testing-shell-functions) library is use for testing this repo.

## Unit Test(s)

For info regarding unit tests, including goals, standards, directory structure, when tests should be split up into diffrent files, etc. see the [unit test readme](unit/README.md).

## Functional Test(s)

TODO #34: Not implemented.
41 changes: 41 additions & 0 deletions spec/spec_helper.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# shellcheck shell=sh

# For more information regarding shellspec's spec_helper file, see: https://github.com/shellspec/shellspec#spec_helper

######################################
## Global (Across All specfiles) Data ##
######################################
## Constant(s) ##
# Return Value(s) #
CATCHALL_RT=1
MINOR_EXECUTION_FAILURE_RT=3
OPTION_NAME_INVALID_RT=140
OPTION_VALUE_INVALID_RT=141
OPTION_REQUIRED_NOT_PROVIDED_RT=142
CODE_NOT_ACCESSIBLE_RT=202
## Variable(s) ##
# NoOp
## Function(s) ##
# Value Checking Function(s) #
isPositive() { [ $(($1)) -gt 0 ]; }
isNotNegative() { [ $(($1)) -ge 0 ]; }
## Alias(es) ##
# NoOp

# This callback function will be invoked only once before loading specfiles.
spec_helper_precheck() {
# Available functions: info, warn, error, abort, setenv, unsetenv
# Available variables: VERSION, SHELL_TYPE, SHELL_VERSION
: minimum_version "0.28.1"
}

# This callback function will be invoked after a specfile has been loaded.
spec_helper_loaded() {
:
}

# This callback function will be invoked after core modules has been loaded.
spec_helper_configure() {
# Available functions: import, before_each, after_each, before_all, after_all
: import 'support/custom_matcher'
}
3 changes: 3 additions & 0 deletions spec/support/bin/grep
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh -e
. "$SHELLSPEC_SUPPORT_BIN"
invoke grep "$@"
3 changes: 3 additions & 0 deletions spec/support/bin/tr
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh -e
. "$SHELLSPEC_SUPPORT_BIN"
invoke tr "$@"
79 changes: 79 additions & 0 deletions spec/unit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Unit Testing

This directory and its child directories contain all unit tests of this repo's code. For generic info regarding testing, see the top level [test readme](../README.md).

## Unit Testing Goals

Unit tests should:

- Test the **smallest** possible unit of code in a **single** way.
- Test the **smallest** possible unit of code in **every** way.
- **All** code **must** have **100%** coverage:
- This includes things like files that define constants.
- Ensure **no code outside** of the Code Under Test (COT) **is executed**.
- **Environment changes** should **never break** tests:
- For example, changing the value of a value in a constants file should only cause tests to fail when the new value is invalid.
- **Functionality** changes **should** always **break** tests.
- **Implementation** changes should **not break** tests... ideally.

## Directory Structure

Each directory in this `README.md`'s directory has the name of a command that may be used when this repo is used to configure a POSIX environment. Tests of that functionality are stored in each directory. If the functionality has supporting code, constants, etc, then tests of that code will be in a `util` directory.


## Test Naming

Bellow is the unit test file naming convention:

```
<cotFileName>_[optional/required][Option/Argument]_<optionOrArgumentName>_spec.sh
```

For example, `output_optionalOption_help_spec.sh` would contain unit tests of the `--help`/`-h` option for the `output()` function.

## Tag Naming Requirement(s)

- The part of the tag name after the `:` **must** be the same as the title in cammle case with the first character in lower case.
- The part of the tag before the `:` should be the same as the tag of its parent with the `:` removed and the first character of the right hand potion of the tag capitalized.
- The top level tag **must** follow: `<userFacingFunctionName>:<nextLevelDown>`:
- Example:
- Top three level tags of a util function called `createHeaderFooter()` used by a user facing function called `output()` would be: `output:output`, `outputOutput:util`, & `outputOutputUtil:createHeaderFooter` respectivly.

[**See an example here**.](#test-structure-example)

## Unit Test Structure Requirement(s)

- All levels (`It`, `Describe`, etc) **must** be named.
- All levels, other than the bottom one, **must** have `:` as the last character in the name:
- This is done so the test output is eaiser to read.
- Each level of the test should reflect how the CUT is organized, thus:
- The top level `Describe` (see: [shellspec basic structure for more info](https://github.com/shellspec/shellspec#basic-structure)) must be named the same as the user facing function the test is most closly related to:
- For example:
- If the test if of a util fnction called `utilFunc()`, and `utilFunc()` is used by the `output()` function, then the top level `Describe` should have the title `output():`.
- The next level would be named `Util:`.
- The next would be named `utilFunc():`.
- The bottom level (`It`) must **always** have a unique tag.
- Each piece of functionality that can be tested separately from any other funcinality, even if only partialy, **must** be placed in its own file:
- Splitting tests up into different files allows `shellspec` to run them in parallel.
- Additional tests needed to get full coverage should be placed in a diffrent file.

**Note:** If an existing test file doesn't follow any of these rules, then the one who is modifying the file must update the entire file to follow all rules.

## Test Structure Example

A test of `output()`'s `--help` option would look like this:

```
Describe "output():" output:output
Describe "Optional option:" outputOutput:optionalOption
Describe "Help:" outputOutputOptionalOption:help
It "-h" outputOutputOptionalOptionsHelp:h
Test here.
End
It "--help" outputOutputOptionalOptionsHelp:help
Test here.
End
End
End
End
```
41 changes: 41 additions & 0 deletions spec/unit/output/output_createHeaderFooterFailure_spec.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Set constant(s) use here so configuration changes won't cause tests to fail.
% DEFAULT_LINE_LENGTH:100
readonly DEFAULT_LINE_LENGTH
% DEFAULT_INDENT:0
readonly DEFAULT_INDENT

Describe "output():" output:output
# Mock out sourcing of util (constants, globals, helper functions, etc).
inScriptSource() { return 0; }
# Makes test easier to read and maintain.
output=$PWD/src/shell/functions/output/output.sh

Describe "createHeaderFooter() failure:" outputOutput:createHeaderFooterFailure
DEFAULT_CHAR='#'

Describe "Non zero return code:" outputOutputCreateHeaderFooterFailure:nonZeroReturnCode
It "140" outputOutputCreateHeaderFooterFailureNonZeroReturnCode:140
createHeaderFooter() { return $OPTION_NAME_INVALID_RT; }
When run source $output -m=m -p
The stdout should not be present
The stderr should include "ERROR"
The status should equal $MINOR_EXECUTION_FAILURE_RT
End
It "141" outputOutputCreateHeaderFooterFailureNonZeroReturnCode:141
createHeaderFooter() { return $OPTION_VALUE_INVALID_RT; }
When run source $output -m=ms --header-footer
The stdout should not be present
The stderr should include "ERROR"
The status should equal $MINOR_EXECUTION_FAILURE_RT
End
It "202" outputOutputCreateHeaderFooterFailureNonZeroReturnCode:202
createHeaderFooter() { return $CODE_NOT_ACCESSIBLE_RT; }
When run source $output -m="message 1" -m=msg2 --pretty
The stdout should not be present
The stderr should include "ERROR"
The status should equal $MINOR_EXECUTION_FAILURE_RT
End
End
End
End

27 changes: 27 additions & 0 deletions spec/unit/output/output_dateMocking_spec.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
Describe "output():" output:output
# Mock out sourcing of util (constants, globals, helper functions, etc).
inScriptSource() { return 0; }
# Makes test easier to read and maintain.
output=$PWD/src/shell/functions/output/output.sh
# createHeaderFooter() shouldn't be called, but mock it out just in case.
createHeaderFooter() { return $CATCHALL_RT; }

Describe "Mock date():" outputOutput:mockDate
It "Not defined" outputOutputMockDate:notDefined
When run source $output -f
The stdout should not be present
The stderr line 1 should start with "ERROR"
The status should equal $OPTION_NAME_INVALID_RT
End
End
Describe "Mock date():" outputOutput:mockDate
date() { echo "2023/03/31 22:06:55 MDT"; }
It "Defined" outputOutputMockDate:defined
When run source $output -f
The stdout should not be present
The stderr line 1 should start with "2023/03/31 22:06:55 MDT ERROR"
The status should equal $OPTION_NAME_INVALID_RT
End
End
End

Loading

0 comments on commit 6a4e6cd

Please sign in to comment.