Unit Testing Overview for Omnis Developers

This page provides a brief summary of what unit testing is and why it is important to an Omnis developer. Omnis Studio does not offer a native testing suite, and some Omnis developers may be unfamiliar with the concept of an automated formal test. This page is not meant to explain the complete theory behind software testing, but instead provide a "boots on the ground" run-down on what unit testing means in Omnis Studio and to OmnisTAP.

Why test?

Omnis is known for rapid development, and its runtime debugger is a significant aid to building code quickly. Omnis also allows you to interact with the application as user would from right within the IDE. The allows a developer to debug their code as they write it and immediately perform manual user testing.

This approach to development is indeed rapid, but it doesn't provide any lasting, reproducible guarantee the application will continue to operate properly as time goes by and the code base evolves.

The unit and integration tests offered by OmnisTAP are code. They can be played back repeatedly to ensure the application still behaves as it should. Not only does this improve the product quality, it provides freedom to the developer to redesign and improve their code while ensuring their application still behaves as it should. Testable code is clean code, and as you add tests to your codebase you'll find it because easier to enhance and less prone to bugs.

What is a unit test?

A unit test describes the behavior of a single method. It provides a method with a specific input, executes the method, and verifies the output of the method is correct. Together, this process is called an "assertion".

Unit tests are designed to run quickly and focus on the smallest building blocks of your application. OmnisTAP enforces this design by automatically failing unit tests that take more than 5 seconds to execute.

Integration tests

Like a unit test, an integration tests executes part of your application with a specific set of inputs and asserts the correct output is returned. However, the method(s) your integration test calls will call other methods in turn. Where a unit test is concerned with how one unit, or method, behaves, an integration test cares about how a whole group of methods work together.

In OmnisTAP, an integration test is allowed 90 seconds to execute before it is failed for execution time. Tests that make database or other network calls are considered integration tests as they test how your code behaves when it integrates with the remote system.

Unit test example

Consider a method to build a compound name for a person given constituent name parts.

Do loPerson.$buildFullName(pcTitle,pcFirst,pcMiddle,pcLast) Returns lcFullName

Here are a set of inputs for this method and the expected output:

pcTitle	pcFirst	pcMiddle	pcLast	lcFullName
Captain	James	Tiberius	Kirk	Captain James Tiberius Kirk
	James	Tiberius	Kirk	James Tiberius Kirk
	James		Kirk	James Kirk
Captain			Kirk	Captain Kirk

OmnisTAP allows up to write code that ensure this is aways the case.

Do loPerson.$buildFullName("Captain","James","Tiberius","Kirk") Returns lcFullname
Do ioTAP.$is_char(lcFullname,"Captain James Tiberius Kirk")

Do loPerson.$buildFullName("","James","Tiberius","Kirk") Returns lcFullname
Do ioTAP.$is_char(lcFullname,"James Tiberius Kirk")

Do loPerson.$buildFullName("","James","","Kirk") Returns lcFullname
Do ioTAP.$is_char(lcFullname,"James Kirk")

Do loPerson.$buildFullName("Captain","","","Kirk") Returns lcFullname
Do ioTAP.$is_char(lcFullname,"Captain Kirk")

Fixtures

In the previous example, we have to pass specific values to the $buildFullName() method to get an expected output. These specific values are known as fixtures because they provide static data around which we execute our code and assert its result.

Types of methods that are unit-tested

Unit tests ideally cover three types of methods:

• Operation
• Accessor
• Decision

Operation methods "do" an action. The $buildFullName() method in the previous example is an operation method. Operation method might also run a SQL query, open a window, or search a list of lines matching some condition.

Accessor methods get or set an attribute of a object. Classic getters and setters are accessor methods, but interrogative methods are also accessors because they access the state of a system. For example, table class methods for $isPersonDeceased() or $hasSpouse() are accessor methods.

Decision methods use accessors to decide what operations to execute. These methods are often referred to as "glue code" because they glues together other operations instead of running the operations directly.

Testing accessor methods

Recall the definition above for a unit test:

[A unit test] provides a method with a specific input, executes the method, and verifies the output of the method is correct.

In the $buildFullName() example, the input is a set of character values and the output is another character value. When testing operation methods it's common to use variable like this as input and output, whether they're primitives like strings or more complicated data like table class-based lists or custom objects.

For accessor methods, however, the test's input and output can be a bit different. The input for an $isDeceased() method on a table class might be the values in a row or list. Its output would be a simple boolean. Assume $isDeceased() has this code:

If not(isclear($cinst.death_date))
  Quit method kTrue
End if

Quit method kFalse

The test might look like this:

Calculate lrPerson.death_date as #D
Do lrPerson.$isDeceased() Returns lbIsDeceased
Do ioTAP.$is_boolean(lbIsDeceased,kTrue,"The person is deceased when they have a death date")

Calculate lrPerson.death_date as #NULL
Do lrPerson.$isDeceased() Returns lbIsDeceased
Do ioTAP.$is_boolean(lbIsDeceased,kFalse,"The person is not deceased when they don't have a death date")

The subtle but important difference is this test first sets the internal data of the row, then calls the method being tested. In the application's normal lifecycle, death_date may come from a database and wouldn't be set directly on the row. In our test, we're directly setting some data inside an object to mimic how it would be returned from another system. This data is also known as a "stub", because we provide a pre-determined result from another system. The stub in this example is similar to the fixture data from the $buildFullName() method.

Note that it's important to set this date in a fashion that's as close to the production code as possible. If a later change to the database renamed death_date to date_deceased, this code would start failing and expose the need to update $isDeceased() to use the new column name.

Testing decision methods

Decision methods, or glue code, require a completely different technique to test them. Remember, a unit test should test a single method. Consider a loMailer.$addPersonToEmail() method that flags people for an email blast from your application:

If pfrPerson.$isDeceased()
  Do $cinst.$_excludePerson(pfrPerson)
Else
  Do $cinst.$_includePerson(pfrPerson)
End if

You could create two people records, one with a death date and one without, call this method, then check the excluded person list and included person list to ensure the right people are on the right list. This would be a perfectly valid test, but it would be an integration test.

Here's how that integration test might look:

Calculate lrDeceasedPerson.death_date as #D
Calculate lrAlivePerson.death_date as #NULL

Do llExcludeList.$add().$assignrow(lrDeceasedPerson)
Do llIncludeList.$add().$assignrow(lrAlivePerson)

Do loMailer.$addPersonToEmail(lrDeceasedPerson)
Do loMailer.$addPersonToEmail(lrAlivePerson)

Do ioTAP.$is_list(loMailer.ilExcludeList,llExcludeList,"We add the deceased person to the exclude list")
Do ioTAP.$is_list(loMailer.ilIncludeList,llIncludeList,"We add the alive person to the include list")

This works, but what if you change the requirements for knowing if a person is deceased? You have to update the unit test on $isDeceased() and and the test on $addPersonToEmail(). Plus, you also have to know how $isDeceased() determines if a person is deceased to setup the fixtures, and you also need to know how the $excludePerson() and $includePerson() methods work to test they build their lists correctly.

This is why integration tests are seen as "brittle". It's easy to break them with other changes. Integration test are useful and necessary in their own right, but they're more involved to setup and can require more maintenance down the road.

In this example, we don't really care how the program determines if a person is deceased--the test on $isDeceased() does that. And we don't care how the $excludePerson() and $includePerson() methods work. What we really want to know is that this $addPersonToEmail() method puts the right people on the right list.

Mocking

What we need is a way to provide a "fixture" or "stub" for the $isDeceased() method without worrying about how that method is implemented. And, we need a way to know that the $excludePerson() or $includePerson() methods were called for the right people.

OmnisTAP provides a feature to do exactly that, called "mocking". Mocking allow you to replace a method so that:

• You can set its result to any fixture data you want
• The method's real code isn't run
• You can test if the method was called the right number of times with the right parameters

At face value, mocking may seem like it requires just as much code that building an integration test. But, OmnisTAP makes mocking quick to write, and it keeps your tests on decision code true unit tests that focus on the decision logic. This allows you to manage your application's complexity by distilling methods to just the parts that matter, and locking down the correct behavior for each and every method with a unit test.

Here is how we might test the $addPersonToEmail() method:

Do $cinst.$mock($tables.tPerson,lrDeceasedPerson) ;; Get an instance of the person for mocking a deceased person
Do lrDeceasedPerson.$mock("$isDeceased").$return(kTrue) ;; Set the next call to $isDeceased to return true
Do loMailer.$mock("$_excludePerson").$expect(lrDeceasedPerson) ;; Expect we'll add the person to the exclude list

Do $cinst.$mock($tables.tPerson,lrAlivePerson) ;; Get an instance of the person for mocking an alive person
Do lrAlivePerson.$mock("$isDeceased").$return(kFalse) ;; Set the next call to $isDeceased to return false
Do loMailer.$mock("$_includePerson").$expect(lrAlivePerson) ;; Expect we'll add the person to the include list

Do loMailer.$addPersonToEmail(lrDeceasedPerson) ;; Add the deceased person
Do loMailer.$addPersonToEmail(lrAlivePerson) ;; Add the alive person
Do $cinst.$assertMocks()

When $assertMocks() is called, OmnisTAP will ensure:

1. We called $isDeceased() on both person records
2. We called $excludePerson() once and passed the deceased person to it
3. We called $includePerson() once and passed the alive person to it

At no time will the $isDeceased(), $excludePerson(), and $includePerson() methods actually run. If we later change the implementation of any of these methods, this test on $addPersonToEmail() will still pass.

What's next

The challenges for Omnis developers to add testing to their applications are three-fold:

1. Understanding how to write effective tests
2. Understanding how to add testing and clean code practices to existing code
3. Implementing testing as part of a continuous integration process so that no code ships without passing its test

This wiki will attempt to address how to write effective tests. For working with existing code and adopting clean coding practices, check out the resources linked below. Be on the lookout for a guide to setting up continuous integration with OmnisTAP and OmnisCLI. Remember, the tests are only useful if you run them and keep them passing. :)

Further Reading

For additional information on automated testing, clean code and how to work with existing, un-tested code, check out:

• Test Driven Development: By Example by Kent Beck
• Clean Code by Robert C. Martin
• Working Effectively with Legacy Code by Michael Feathers