Feature Flags

What is a feature flag?

In the Firefox iOS codebase, we define a feature flag as a variable, inside a feature, that controls the status of the feature in the application.

Feature flags should logically be part of their own features, even if that's the only variable in that feature. - ie. Should not be part of a generalAppFeature, or a featureFlagFeature.

How feature flags work in Firefox iOS

Feature flags are all controlled by the FeatureFlagManager singleton. To access the singleton, you must make a class conform to the FeatureFlaggable protocol, which will give access to the featureFlags variable.

class BibimbapViewModel: FeatureFlaggable {
    var isNewMenuAvailable: Bool {
        return featureFlags.isFeatureEnabled(.newBibimbapMenu, checking: .buildOnly)
    }
}

Types of features flags

Name	Description	User Togglable
Core	Core features are features that are used for developer purposes and are not directly user impacting.	No
Nimbus	A nimbus feature is a feature whose configuration comes from Nimbus.	Possibly

The vast majority of feature flags should be Nimbus flags, rather than Core flags.

The FeatureFlagManager interface

Interface	Purpose
isCoreFeatureEnabled(...)	Checking where a Core feature is enabled.
isFeatureEnabled(...)	Checking whether a boolean based Nimbus feature is enabled.
getCustomState<T>(...)	Checking the status of a non-boolean based Nimbus feature.
set(...)	Saving a boolean based Nimbus feature user preference to UserDefaults.
set<T: FlaggableFeatureOptions>(...)	Saving a non-boolean based Nimbus feature user preference to UserDefaults.

The checking parameter & how feature status is checked.

One of the complexities of feature flags is that while Nimbus may have a default, a user may turn something off. Regardless of whether or not the user is in an experiment, their preferences should be respected. To accomplish this, the previously listed interfaces that check a feature status include a specific checking parameter. This has three options which should cover 100% of use cases for needing to check the status of a feature.

• buildOnly - this will only check Nimbus configuration for status
• userOnly - this will check UserDefaults to see if the user has a preference. If they do, that is what will be returned. If they do not, then the Nimbus configuration is queried for status
• buildAndUser - this will a mix of both.

How to set up a Feature Flag

Adding a feature flag to Nimbus

To add a feature to Nimbus, please read Nimbus Feature. Once this is done, add a variable to that feature named something indicative of a status. Here is an example of what that might look like

...
  isEnabled:
    description: >
      Whether or not the feature is enabled.
    type: Boolean
    default: false

After the changes have been made, be sure to build the application

Adding a simple boolean feature flag in the app

Say you wanted to add a flag that controlled whether a user saw an old menu or a new menu. To add the flag in the app (for example, for the newBibimbapMenu flag), follow these three simple steps:

1. Add case newBibimbapMenu to the NimbusFeatureFlagID enum.
2. Add this new case to the NimbusFlaggableFeature struct.

• If the user will have a setting to interact with for the feature, you should add this such that it returns a PrefsKeys.FeatureFlags key, which you will have to also add.
• If the user doesn't have a setting for the feature, you should add it to the return nil part of the switch statement.

3. In the NimbusFeatureFlagLayer class, you should add a case for your new feature, as well as the function it will call

...
    switch featureID {
    case .newBibimbapMenu:
        return checkBibimbapFeature(for: featureID, from: nimbus)
...
private func checkBibimbapFeature(
    for featureID: NimbusFeatureFlagID,
    from nimbus: FxNimbus
) -> Bool {
    let config = nimbus.features.bibimbapFeature.value()

    switch featureID {
    case .newBibimbapMenu: return config.newBibimbapMenu
    default: return false
    }
}

At this point, your work is done and you now have a feature flag that can be checked.

Adding a complex feature flag in the app

Say you wanted a flag that had more than two options. In our example, there is a morning, afternoon, and evening version of the menu. The complexity in this case is that Nimbus features must be mapped. Improvements to this will be coming in the future, but as of now, here's how to accomplish this.

1. Add case bibimbapMenuVersion to the NimbusFeatureFlagID enum.
2. Add case bibimbapMenuVersion to the NimbusFeatureFlagWithCustomOptionsID enum.
3. Add this new case to the NimbusFlaggableFeature struct.

• If the user will have a setting to interact with for the feature, you should add this such that it returns a PrefsKeys.FeatureFlags key, which you will have to also add.
• If the user doesn't have a setting for the feature, you should add it to the return nil part of the switch statement.

4. In the FlaggableFeatureOptions file, create an enum for your feature flag, inheriting from String and FlaggableFeatureOptions.

enum BibimbapMenuVersion: String, FlaggableFeatureOptions {
    case morning
    case afternoon
    case evening
}

5. In the NimbusFeatureFlagLayer class, you should add a case for your new feature, as well as the function it will call

...
    switch featureID {
    case .newBibimbapMenu:
        return checkBibimbapFeature(for: featureID, from: nimbus)
...
private func checkBibimbapFeature(from nimbus: FxNimbus) -> BibimbapMenuVersion {
    let config = nimbus.features.bibimbapFeature.value()
    let nimbusSetting = config.bibimbapMenuVersion

    switch nimbusSetting {
    case .morning: return .morning
    case .afternoon: return .afternoon
    case .evening: return .evening
    }
}

6. In the NimbusFlaggableFeature class, under getUserPreference, you should add your case in the switch:

case .bibimbapMenuVersion:
    return nimbusLayer.checkBibimbapFeature().rawValue

7. In NimbusFeatureFlagManager's getCustomState<T>, add your case to the switch statement.

case .bibimbapMenuVersion: return BibimbapMenuVersion(rawValue: userSetting) as? T

At this point, your work is done and you now have a feature flag that can be checked:

lazy var bibimbapMenuVersion: BibimbapMenuVersion? = featureFlags.getCustomState(for: .bibimbapMenuVersion)

Feature Flags Debug Menu

In order to test your feature flag using the app, we now have a Feature Flags section in our secret settings (tap on the version number in settings 5 times). Scroll down until you see the Feature Flags cell and tap on it. This setting is only available for developer / beta builds and is hidden from production. The view contains a top portion where you can toggle certain features flags on and off and the bottom contains the current values that the app has.

Debug Menu	Feature Flags Section

How to add a new toggle

1. Add the feature flag case to debugKey for NimbusFeatureFlagID and ensure that it returns a string using rawValue + PrefsKeys.FeatureFlags.DebugSuffixKey. Cases return a debugKey that is nil by default.
2. Create a new FeatureFlagsBoolSetting specific to the feature flag case you want to toggle in FeatureFlagsDebugViewController .
3. Add the new setting to SettingSection in FeatureFlagsDebugViewController.
4. Run the app and navigate to the feature flag debug setting. Confirm that you can now see your new feature flag toggle setting and it works appropriately.

Proposal PR: https://github.com/mozilla-mobile/firefox-ios/pulls?q=is%3Apr+debug+menu+is%3Aclosed+

Unit Testing with Feature Flags

We should test that our business logic around feature flags is correct (as we should expect different outcomes when a flag is enabled than when it is disabled).

First of all, you should call the following in your setUp() method whenever your code will be calling feature flags:

override func setUp() {
   super.setUp()
   ...
   LegacyFeatureFlagsManager.shared.initializeDeveloperFeatures(with: MockProfile())
}

Setup Nimbus Defaults for Unit Testing

We can initialize our feature with a default configuration prior to executing our test. This overrides the default Nimbus settings and is similar to editing a local YAML file to "enable" a certain feature for your local dev environment.

As an example, let's look at the Sent from Firefox experiment. The default configuration can be found in sentFromFirefoxFeature.yaml. This experiment is disabled by default, but we want to test our code for when it is enabled. There are only two values we are concerned with: enabled, which tells us whether the user is enrolled in the experiment, and isTreatmentA, which is used for A/B testing two different text strings shown to users.

We can create a helper method in our unit tests to initialize Nimbus with our desired configuration for a given feature, for example:

private func setupNimbusSentFromFirefoxTesting(isEnabled: Bool, isTreatmentA: Bool) {
        FxNimbus.shared.features.sentFromFirefoxFeature.with { _, _ in
            return SentFromFirefoxFeature(
                enabled: isEnabled,
                isTreatmentA: isTreatmentA
            )
        }
    }

We can then call this in our unit tests like so, which allows us to test how our code handles each type of configuration:

setupNimbusSentFromFirefoxTesting(isEnabled: <value>, isTreatmentA: <value>)

Another example of this can be found at the end of NimbusOnboardingFeatureLayerTests.swift. The setupNimbusForStringTesting() method has a slightly more complicated setup using a dictionary.

You will also need to setup LegacyFeatureFlagsManager for testing by calling LegacyFeatureFlagsManager.shared.initializeDeveloperFeatures(with: MockProfile()). This allows the stubbed value of the feature flag to be returned instead of just returning false.

Testing .buildOnly, .buildAndUser, and .userOnly

Recall that buildOnly refers to the value in Nimbus and userOnly refers to the value of the user's preference (for example, a toggle in Settings). And finally, buildAndUser takes the logical AND of those two values.

Users can toggle some features on / off in our settings. If a user toggles off a feature, we have to ensure that even if they're enrolled in an experiment, that feature is disabled.

Once again we can look at the Sent from Firefox experiment as an example. The ShareManager should override sharing behaviour only for users enrolled in the Sent from Firefox experiment who have not explicitly opted out using the toggle in the general settings.

We want to test that users who have a false .userOnly setting do not get the Sent from Firefox experiment treatment even if .buildOnly is true (i.e. the value of enabled from Nimbus).

For unit tests, we can simulate a user preference with the following:

UserDefaults.standard.set(<override bool>, forKey: PrefsKeys.NimbusUserEnabledFeatureTestsOverride)

By setting PrefsKeys.NimbusUserEnabledFeatureTestsOverride on UserDefault, we can ensure the override value (either true or false) is returned for every user preference check. The current implementation is very basic, so if different values need to be returned the implementation will need to be improved.

The user preference override will persist across your unit tests until you reset your state. Thus, in your tearDown() method you should also clear this testing state:

 override func tearDown() {
     UserDefaults.standard.removeObject(forKey: PrefsKeys.NimbusUserEnabledFeatureTestsOverride)
     super.tearDown()
}

By using NimbusUserEnabledFeatureTestsOverride in conjunction with setting a default nimbus testing state, you can thoroughly test features that should only be enabled when .buildAndUser is true (i.e. both the nimbus setting and the user setting return true).