From bd2995503872861e36697ec993f68dd9386622fd Mon Sep 17 00:00:00 2001 From: Thiago Miotto Date: Mon, 9 Oct 2017 16:37:09 -0300 Subject: [PATCH 1/3] Adds TESTING.md --- CHANGELOG.md | 2 ++ TESTING.md | 61 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+) create mode 100644 TESTING.md diff --git a/CHANGELOG.md b/CHANGELOG.md index f6bddce8d..2fb14d918 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ #### Notes This release adds the [endpoints-support.md](endpoints-support.md) file to the repository, in order to track implemented endpoints and what is in the scope of this SDK. +Also adds the [TESTING.md](TESTING.md) file to the repository, in order to guide the testing execution and implementation for this SDK. + ## v5.1.1 #### Bug fixes & Enhancements diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 000000000..3b157c573 --- /dev/null +++ b/TESTING.md @@ -0,0 +1,61 @@ +# Testing 'oneview-sdk' gem source code +The 'oneview-sdk' source code provides a unit, integration/functional, system testing strategies, and linting tools to ensure its code style. +This document will cover on how to execute and implement them. + +## Executing tests +The tests can be executed by `rake` tasks, `guard` watches, and by their original tools, like `rspec`, and `rubocop`. + +### Rake +All the test strategies and checks can be executed by the rake command. +Please use `rake -T` to see a full list of commands. + +### Guard +Guard will watch the changes and execute unit tests and style checks. Just activate it to have real time test execution on every change. + +### Unit tests +All unit tests are inside the spec folder. You can execute them manually by using RSpec and specifying the test files, like `rspec spec/unit/path/to/my/tests`. + +### Integration tests +The integration tests are responsible for checking if the `oneview-sdk` is correctly integrated with the OneView appliance, at the same time it tests its functionalities. + +#### Categories +Each integration test has its own pre-requisites and leaves the appliance in a different post state, and given OneView composable nature, each resource needs other resources configured so it can work. + +The most top-level resources would need the appliance entirely configured, and setting up and tearing it down for each test would cost a lot of time (likely days). Because of the slow time to set up the tests have dependencies on each other that sped up the execution to few minutes or likely an hour depending on the test environment. + +So the integration tests are split into 3 categories `create`, `update`, and `delete` that are executed in the same order to allow some level of modularity between the steps, i.e. if something fails in one of the categories, the testing environment can be easily fixed so the next step won't fail because of the dependency. + +### System tests +System tests are integration-like tests, but they aim to test the whole solution executing more complex interactions (use cases), not just focusing in testing one endpoint. + +### Coverage +The tests issue a coverage report generated by SimpleCov. To see the full coverage report it is needed to run the entire suites for each individual test strategy. For instance, if a full integration coverage report is needed it is necessary to run the three testing categories. + +### Style checking +Style checking is done by `rubocop` for all the checkable code. + +## Implementing tests +All code must have associated tests, be it the already implemented or newly submitted, and this section covers what tests need to be implemented. + +### Unit tests +The unit tests are required for each new resource, bug fix, or enhancement. + +### Integration tests +The integration tests are required for each new resource, bug fix, or enhancement as far the new code modifies or adds an interaction with OneView. + +To avoid incompatibility between tests, the integration has a consolidated configuration file with all the resource names and a dependency matrix called `sequence_and_naming.rb`. + +It is far beyond from not advisable to instantiate a OneView resource in the integration tests that does not use the names in established there, if necessary, add the new names there. + +Add the new resource implemented to the dependency matrix, so the testing requisites will be met. + +Note: It is uncommon for an updated resource to change its dependency order, but it is fairly possible. + +### Shared tests +Some resource methods behave exactly like other ones, so the source code provides a shared tests folder that can be used along multiple resource tests. + +### Unit and Integration acceptance criteria +The necessary amount of testing is dictated by who is implementing and by who is reviewing and accepting the code, however, there is a floor coverage acceptance level defined in the `spec_helper.rb` file. + +### System tests +System tests focus on use cases and they may be implemented to cover specific scenarios not covered by the other test strategies. Different from the unit and integration tests they do not have an acceptance criteria. From 269ddab9ddc82984ec0ee74c4ef0b9a1e4cc5230 Mon Sep 17 00:00:00 2001 From: Thiago Miotto Date: Mon, 9 Oct 2017 16:39:41 -0300 Subject: [PATCH 2/3] Update word in Changelog --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2fb14d918..8fa7b819f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ #### Notes This release adds the [endpoints-support.md](endpoints-support.md) file to the repository, in order to track implemented endpoints and what is in the scope of this SDK. -Also adds the [TESTING.md](TESTING.md) file to the repository, in order to guide the testing execution and implementation for this SDK. +Also adds the [TESTING.md](TESTING.md) file to the repository, in order to guide the test execution and implementation for this SDK. ## v5.1.1 From 77b73311fbe1daf6683ebed7f2622805938656f6 Mon Sep 17 00:00:00 2001 From: Thiago Miotto Date: Mon, 9 Oct 2017 17:08:49 -0300 Subject: [PATCH 3/3] Rewrites some information in TESTING.md --- TESTING.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/TESTING.md b/TESTING.md index 3b157c573..9c79f2459 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1,6 +1,6 @@ # Testing 'oneview-sdk' gem source code -The 'oneview-sdk' source code provides a unit, integration/functional, system testing strategies, and linting tools to ensure its code style. -This document will cover on how to execute and implement them. +The 'oneview-sdk' source code provides a unit, integration/functional, and system testing strategies. It also provides linting tools to ensure its code style. +This document will cover how to execute and implement them. ## Executing tests The tests can be executed by `rake` tasks, `guard` watches, and by their original tools, like `rspec`, and `rubocop`. @@ -21,9 +21,9 @@ The integration tests are responsible for checking if the `oneview-sdk` is corre #### Categories Each integration test has its own pre-requisites and leaves the appliance in a different post state, and given OneView composable nature, each resource needs other resources configured so it can work. -The most top-level resources would need the appliance entirely configured, and setting up and tearing it down for each test would cost a lot of time (likely days). Because of the slow time to set up the tests have dependencies on each other that sped up the execution to few minutes or likely an hour depending on the test environment. +The most top-level resources would need the appliance entirely configured, and setting up and tearing it down for each test would cost a lot of time (likely days). So there is a chained dependency among the tests to greatly reduce its execution time. -So the integration tests are split into 3 categories `create`, `update`, and `delete` that are executed in the same order to allow some level of modularity between the steps, i.e. if something fails in one of the categories, the testing environment can be easily fixed so the next step won't fail because of the dependency. +The integration tests are split into 3 categories `create`, `update`, and `delete` that can be run separately to allow some level of modularity between them. ### System tests System tests are integration-like tests, but they aim to test the whole solution executing more complex interactions (use cases), not just focusing in testing one endpoint.