Release v0.0.7

This release has some great features that have been contributed by members of the community.

Custom resource testing

This release introduces new features that makes it easy to create integration tests for custom resources. One enhancement to the klient package now allows to registration of arbitrary resource schemes allowing testing of arbitrary resources such as custom resources.

Another improvement is the introduction of helper functions, envfuncs.SetupCRDs and envfuncs.TeardownCRDs, to setup/teardown custom resource definition YAML files when testing of custom resources.

func TestMain(m *testing.M) {
	cfg, _ := envconf.NewFromFlags()
	testEnv = env.NewWithConfig(cfg)
	kindClusterName = envconf.RandomName("crdtest-", 16)
	namespace = envconf.RandomName("my-ns", 10)

	testEnv.Setup(
		envfuncs.CreateKindCluster(kindClusterName),
		envfuncs.CreateNamespace(namespace),
		envfuncs.SetupCRDs("./testdata/crds", "*"),
	)

	testEnv.Finish(
		envfuncs.DeleteNamespace(namespace),
		envfuncs.TeardownCRDs("./testdata/crds", "*"),
		envfuncs.DestroyKindCluster(kindClusterName),
	)

	os.Exit(testEnv.Run(m))
}

This feature, along with other facilities in the framework, should make it easy to test components such as custom resources and their controllers.

Read more about this feature here.

Fail fast mode

This feature allows test writers to short circuit the execution of test, causing the entire test to fail immediately when a fail signal is encountered.

func TestExample(t *testing.T) {
	failFeature := features.New("fail-feature").
		Assess("1==2", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
			if 1 != 2 {
				t.Log("1 != 2")
				t.FailNow() // mark test case as failed here, don't continue execution
			} else {
				t.Log("1 == 2")
			}
			return ctx
		}).
		Assess("print", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
			t.Log("THIS LINE SHOULDN'T BE PRINTED")
			return ctx
		}).
		Teardown(func(ctx context.Context, t *testing.T, c *envconf.Config) context.Context {
			t.Log("This teardown should not be invoked")
			return ctx
		}).
 		Feature()

	nextFeature := features.New("next-feature").
		Assess("print", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
			t.Log("THIS LINE ALSO SHOULDN'T BE PRINTED")
			return ctx
		}).
		Feature()

	testenv.Test(t, failFeature, nextFeature)
}

When the previous is executed with the --fail-fast flag, the execution of the test function will exit when t.FailNow() is encountered.

Read more about this feature here.

Dry run mode

This release introduces a new feature with a new CLI flag, --dry-fun, that lists the expected feature tests without actually running them.

go test . -test.v -args --dry-run

=== RUN   TestPodBringUp
=== RUN   TestPodBringUp/Feature_One
=== RUN   TestPodBringUp/Feature_One/Create_Nginx_Deployment_1
=== RUN   TestPodBringUp/Feature_One/Wait_for_Nginx_Deployment_1_to_be_scaled_up
=== RUN   TestPodBringUp/Feature_Two
=== RUN   TestPodBringUp/Feature_Two/Create_Nginx_Deployment_2
=== RUN   TestPodBringUp/Feature_Two/Wait_for_Nginx_Deployment_2_to_be_scaled_up
--- PASS: TestPodBringUp (0.00s)
    --- PASS: TestPodBringUp/Feature_One (0.00s)
        --- PASS: TestPodBringUp/Feature_One/Create_Nginx_Deployment_1 (0.00s)
        --- PASS: TestPodBringUp/Feature_One/Wait_for_Nginx_Deployment_1_to_be_scaled_up (0.00s)
    --- PASS: TestPodBringUp/Feature_Two (0.00s)
        --- PASS: TestPodBringUp/Feature_Two/Create_Nginx_Deployment_2 (0.00s)
        --- PASS: TestPodBringUp/Feature_Two/Wait_for_Nginx_Deployment_2_to_be_scaled_up (0.00s)
PASS
ok  	sigs.k8s.io/e2e-framework/examples/parallel_features	0.353s

Read more about this feature here.

A new API to watch resources

This release comes with a new API to facilitate resource watching during tests allowing test authors to write reactive test code by getting notifications when resource state changes.

For instance, the following shows how to setup your test to watch a Deployment object and callback functions to handle the changes.

Setup(func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
    cl, err := cfg.NewClient()
    if err != nil {
        t.Fatal(err)
    }

	dep := appsv1.Deployment{ObjectMeta: metav1.ObjectMeta{Name: "watch-dep", Namespace: cfg.Namespace()}}

// Start watching for the deployment and triger action based on the event received.
    cl.Resources().Watch(&appsv1.DeploymentList{}, resources.WithFieldSelector(labels.FormatLabels(map[string]string{"metadata.name": dep.Name}))).
	WithAddFunc(onAdd).WithDeleteFunc(onDelete).Start(ctx)

    return ctx
})

// Call backs
func onAdd(obj interface{}) {
	dep := obj.(*appsv1.Deployment)
	depName := dep.GetName()
	if depName == "watch-dep" || depName == "watchnstop-dep" {
		klog.InfoS("Deployment name matches with actual name!")
	}
}

func onDelete(obj interface{}) {
	dep := obj.(*appsv1.Deployment)
	depName := dep.GetName()
	if depName == "watch-dep" || depName == "watchnstop-dep" {
		klog.InfoS("Deployment deleted successfully!")
	}
}

For more detail on this feature, see the example.

Extended the test Feature API

The feature test type now supports arbitrary name for all steps allowing for better parsing of test results

func TestHello_WithSetup(t *testing.T) {
	e := env.NewWithConfig(envconf.New())
	var name string
	feat := features.New("Hello Feature").
		WithLabel("type", "simple").
		WithSetup("SetupName", func(ctx context.Context, t *testing.T, _ *envconf.Config) context.Context {
			name = "foobar"
			return ctx
		}).
		Assess("test message", func(ctx context.Context, t *testing.T, _ *envconf.Config) context.Context {
			result := Hello(name)
			if result != "Hello foobar" {
				t.Error("unexpected message")
			}
			return ctx
		}).WithTeardown("Teardown", func(ctx context.Context, t *testing.T, _ *envconf.Config) context.Context {
                ...
		}).Feature()

	e.Test(t, feat)
}

Fix: Parallel test scoping

This release introduces a fix for a race condition that was causing data overwrites when running feature tests in parallel.

Other improvements

• Improved Github Actions
• Automation using goreleaser

Changelog

8339ef1 k8s resource watch and triger action based on the events
7b3aabf GIT-109: enable CRD setup helper to ease the testing of operators
b69e158 GIT-112: enable framework specific fail-fast mode
d19222a GIT-62: Enable --dry-run mode
ed7d3ad Add support for registring other types with klient
a22cbf1 Update Owners: add ShwethaKumbla as approver
2b14f5e GIT-119: uptick kind and fix the install mode
e47ad0c Update config.go
3467252 GIT-119: handle scoping for parallel tests
353abeb GIT-94: enable extended Feature API to provide named setup and teardown
1b2232e add github actions/goreleaser to release the lib