Click 日本語 for the Japanese page if you need.
Anjin is an autopilot framework for games made with Unity. It consists of the following two elements.
- A dispatcher that launches the corresponding Agent according to the Scene loaded in the game
- An Agent that realizes auto-execution closed to the Scene
Agents are small, isolated C# scripts that perform specific operations, such as playback of UI operations or monkey tests. In addition to the built-in ones, you can implement and use ones specific to your game title.
You can choose from two typical installation methods.
- Open the Package Manager tab in Project Settings window (Editor > Project Settings)
- Click + button under the Scoped Registries and enter the following settings (figure 1.):
- Name:
package.openupm.com
- URL:
https://package.openupm.com
- Scope(s):
com.dena
,com.cysharp
, andcom.nowsprinting
- Name:
- Open the Package Manager window (Window > Package Manager) and select My Registries in registries drop-down list (figure 2.)
- Click Install button on the
com.dena.anjin
package
Figure 1. Package Manager tab in Project Settings window.
Figure 2. Select registries drop-down list in Package Manager window.
Note
Do not forget to add com.cysharp
and com.nowsprinting
into scopes. These are used within Anjin.
Note
Required install Unity Test Framework package v1.3 or later for running tests (when adding to the testables
in package.json).
If you installed openupm-cli, run the command below:
openupm add com.dena.anjin
UNITY_INCLUDE_TESTS || DENA_AUTOPILOT_ENABLE
is set in the Define Constraints of the Assembly Definition File, So excluded from the release build in principle.
The following files are automatically generated when you start Anjin. There is no need to track it, so we recommend adding it to your project's .gitignore file.
/Assets/AutopilotState.asset*
After installing the UPM package in the game title Unity project, configure and implement the following.
- Generate and configure the AutopilotSettings.asset file
- Generate and configure the .asset file for the Agent to be used
- Implement a custom Agent if necessary
- Implement initialization process as needed
Note
Set UNITY_INCLUDE_TESTS || DENA_AUTOPILOT_ENABLE
in the Define Constraints of the Assembly Definition File to which they belong to exclude them from release builds.
Right-click in the Unity Editor's Project window to open the context menu, then select Create > Anjin > Autopilot Settings to create the file. The file name is arbitrary, and multiple files can be created and used within a project.
There are three main settings.
Set up a mapping of Agent configuration files (.asset) to be automatically executed for each Scene.
Set the combination of Scene
and Agent
.
The order does not affect the operation.
If a Scene does not exist in the list, the Agent set in Fallback Agent
will be used.
For example, if you want UGUIMonkeyAgent
to work for all Scenes, set Scene Agent Maps
to empty, and set
Set the Fallback Agent
to an UGUIMonkeyAgent
's .asset file.
Set the Agents to run by scene crossing, independent of Scene Agent Maps
and Fallback Agent
.
The specified agents will have the same lifespan as Autopilot (i.e., use DontDestroyOnLoad
)
for specifying, e.g., ErrorHandlerAgent
and UGUIEmergencyExitAgent
.
Warning
Recommend set an ErrorHandlerAgent to interrupt Autopilot if an exception occurs during execution.
This item can also be overridden from the commandline (see below).
- Lifespan
- Specifies the execution time limit in seconds. Defaults to 300 seconds, 0 specifies unlimited operation
- Random Seed
- Specify when you want to fix the seed given to the pseudo-random number generator (optional). This is a setting related to the pseudo-random number generator used by the autopilot. To fix the seed of the pseudo-random number generator in the game itself, it is necessary to implement this setting on the game title side.
- Time Scale
- Time.timeScale. Default is 1.0
- Loggers
- Logger used for this autopilot settings. If omitted,
Debug.unityLogger
will be used as default. - Reporters
- Reporter to be called on Autopilot terminate.
Whether you use the built-in Agent or implement custom Agent, you must create an instance (.asset file) of it in the Unity editor.
Instances are created by right-clicking in the Project window of the Unity editor to open the context menu, then selecting Create > Anjin > Agent type name.
Select the generated file, Agent-specific settings are displayed in the inspector and can be customized. You can prepare multiple Agents with different settings for the same Agent type and use them in different Scenes.
Logger instances are created by right-clicking in the Project window of the Unity editor to open the context menu, then selecting Create > Anjin > Logger type name.
Select the generated file, Logger-specific settings are displayed in the inspector and can be customized. You can prepare multiple Loggers with different settings for the same Logger type.
Reporter instances are created by right-clicking in the Project window of the Unity editor to open the context menu, then selecting Create > Anjin > Reporter type name.
Select the generated file, Reporter-specific settings are displayed in the inspector and can be customized. You can prepare multiple Reporters with different settings for the same Reporter type.
Autopilot can be run in the Unity editor in three ways.
Open the AutopilotSettings file you wish to run in the inspector and click the Run button to enter play mode with Autopilot enabled. After the set run time has elapsed, or as in normal play mode, clicking the Play button will stop the program.
To execute from the commandline, specify the following arguments.
$(UNITY) \
-projectPath $(PROJECT_HOME) \
-batchmode \
-executeMethod DeNA.Anjin.Editor.Commandline.Bootstrap \
-AUTOPILOT_SETTINGS Assets/Path/To/AutopilotSettings.asset
UNITY
is the path to the Unity editor, andPROJECT_HOME
is the root of the project to be autorun.-AUTOPILOT_SETTINGS
is the path to the settings file (AutopilotSettings) you want to run.- Do not specify
-quit
(it will exit without entering play mode). - Do not specify
-nographics
(do not show GameView window).
In addition, some settings can be overridden by adding the following arguments. For details on each argument, see the entry of the same name in the "Generate and configure the AutopilotSettings.asset file" mentioned above.
- LIFESPAN_SEC
- Specifies the execution time limit in seconds
- RANDOM_SEED
- Specifies when you want to fix the seed given to the pseudo-random number generator
- TIME_SCALE
- Specifies the Time.timeScale. Default is 1.0
In both cases, the key should be prefixed with -
and specified as -LIFESPAN_SEC 60
.
Autopilot works within your test code using the async method Launcher.LaunchAutopilotAsync(string)
.
Specify the AutopilotSettings
file path via the argument.
[Test]
public async Task LaunchAutopilotInTest()
{
// Load the first scene
await SceneManager.LoadSceneAsync(0);
// Launch autopilot
await Launcher.LaunchAutopilotAsync("Assets/Path/To/AutopilotSettings.asset");
}
Warning
The default timeout for tests is 3 minutes. If the autopilot execution time exceeds 3 minutes, please specify the timeout time with the Timeout
attribute.
Warning
When running tests on a player, any necessary configuration files must be placed in the Resources
folder to be included in the player build. It can use IPrebuildSetup
and IPostBuildCleanup
to insert processing into the test player build.
Note
The test will fail if the test-runner detects a LogException
or LogError
output. You can suppress this by using LogAssert.ignoreFailingMessages
assuming that you will use Anjin for error handling.
The following Agent types are provided. These can be used as they are, or game-title-specific custom Agents can be implemented and used.
This is an Agent that randomly manipulates uGUI components. This Agent implementation uses open source test-helper.monkey package.
An instance of this Agent (.asset file) can contain the following.
- Lifespan Sec
- Duration of random operation execution time in secounds. If 0 is specified, the operation is almost unlimited (TimeSpan.MaxValue). With this setting, neither Autopilot nor the app itself will exit when the Agent exits. It will not do anything until the next Scene switch
- Delay Millis
- Wait interval [milliseconds] between random operations
- Secs Searching Components
- Seconds to determine that an error has occurred when an object that can be interacted with does not exist
- Touch and Hold Millis
- Delay time for touch-and-hold [ms]
- Enable Gizmos
- Show Gizmos on GameView during running monkey test if true
- Enabled
- Whether screenshot is enabled or not
- Directory
- Use Default: Whether using a default directory path to save screenshots or specifying it manually. Default value is specified by command line argument "-testHelperScreenshotDirectory". If the command line argument is also omitted, `Application.persistentDataPath` + "/TestHelper/Screenshots/" is used.
Path: Directory path to save screenshots - Filename
- Use Default: Whether using a default prefix of screenshots filename or specifying it manually. Default value is Agent name
Prefix: Prefix of screenshots filename - Super Size
- The factor to increase resolution with. Neither this nor Stereo Capture Mode can be specified
- Stereo Capture Mode
- The eye texture to capture when stereo rendering is enabled. Neither this nor Resolution Factor can be specified
If you have a GameObject
that you want to avoid manipulation by the UGUIMonkeyAgent
,
attach the IgnoreAnnotation
component in the TestHelper.Monkey.Annotations
assembly.
See Anjin Annotations below for more information.
This is an Agent that playback uGUI operations with the Recorded Playback feature of the Automated QA package.
The following can be set in an instance (.asset file) of this Agent.
- Recorded Json
- Specify the recording file (.json) to play
Use the Recorded Playback window for recording operations with Automated QA. The window is opened via the Unity editor menu Automated QA > Automated QA Hub > Recorded Playback. The recording file (.json) is saved under the Assets/Recordings/ folder and can be freely moved or renamed.
Note that the Recorded Playback function in Automated QA can record operations across Scene transitions, but in Anjin, when the Scene is switched, the Agent is also forcibly switched, so playback is also interrupted. Therefore, please be careful to record in units of Scenes.
A screenshot of the operation by Automated QA is stored under Application.persistentDataPath/Anjin
.
The Application.persistentDataPath
for each platform can be found in the Unity manual at
Scripting API: Application.persistentDataPath.
Warning
The Automated QA package outputs LogType.Error
to the console when playback fails (e.g., the target Button cannot be found). The following setting is required to detect this and terminate the autopilot.
- Add ErrorHandlerAgent and set
Handle Error
toTerminate Autopilot
. - Add ConsoleLogger and set
Filter LogType
toError
or higher.
Note
The Automated QA package is in the preview stage. Please note that destructive changes may occur, and the package itself may be discontinued or withdrawn.
An Agent that does nothing.
The following settings can be configured for this Agent instance (.asset file).
- Lifespan Sec
- Specifies the do nothing time in seconds. 0 means unlimited time to do nothing. If 0 is specified, an unlimited amount of time for no action is taken. It will not do anything until the next Scene is switched.
An Agent that executes multiple Agents in parallel.
The instance of this Agent (.asset file) can have the following settings.
- Agents
- A list of Agents to be executed in parallel, which can be nested by specifying a CompositeAgent
An Agent that executes multiple Agents in series.
The instance of this Agent (.asset file) can have the following settings.
- Agents
- A list of Agents to be executed in series, which can be nested by specifying a CompositeAgent
An Agent that can register one child Agent and execute it only once throughout the Autopilot execution period. The second time executions will be skipped.
For example, in a game where the title scene leads to a different path only for the first time or where the user receives a login bonus only for the first time on the home scene, it is possible to construct a test scenario in which the title screen is executed as is even if a communication error or the like causes a return to the title scene.
The Agent instance (.asset file) can contain the following settings.
- Agent
- Agent that can be executed only once, which can be nested by specifying a CompositeAgent
An Agent that executes one child Agent indefinitely and repeatedly.
Combined with SerialCompositeAgent, it can be used to repeat a sequence many times or to repeat a specific Agent. Note that finite iterations are not supported (to use SerialCompositeAgent).
This Agent instance (.asset file) can contain the following.
- Agent
- Agent to be executed repeatedly, which can be nested by specifying a CompositeAgent
An Agent that will fail if a defuse message is not received before the inner Agent exits.
For example, pass the out-game tutorial (Things that can be completed with tap operations on uGUI, such as smartphone games).
- Set
UGUIMonkeyAgent
as theAgent
. It should not be able to operate except for the advancing button. Set theLifespan Sec
with a little margin. - Set the log message to be output when the tutorial is completed as the
Defuse Message
.
This Agent instance (.asset file) can contain the following.
- Agent
- Working Agent. If this Agent exits first, the TimeBombAgent will fail.
- Defuse Message
- Defuse the time bomb when this message comes to the log first. Regex is also available.
An Agent that detects abnormal log messages and terminates the Autopilot running.
It should always be started at the same time as other Agents (that actually perform game operations).
Generally, you add to Scene Crossing Agents
in AutopilotSettings
.
If you want to change the settings for each Scene, use with ParallelCompositeAgent
.
This Agent instance (.asset file) can contain the following.
- Handle Exception
- Specify an Autopilot terminates or only reports when an Exception is detected in the log.
It can be overwritten with the command line argument
-HANDLE_EXCEPTION
, but be careful if multiple ErrorHandlerAgents are defined, they will all be overwritten with the same value. - Handle Error
- Specify an Autopilot terminates or only reports when an Error is detected in the log.
It can be overwritten with the command line argument
-HANDLE_ERROR
, but be careful if multiple ErrorHandlerAgents are defined, they will all be overwritten with the same value. - Handle Assert
- Specify an Autopilot terminates or only reports when an Assert is detected in the log.
It can be overwritten with the command line argument
-HANDLE_ASSERT
, but be careful if multiple ErrorHandlerAgents are defined, they will all be overwritten with the same value. - Handle Warning
- Specify an Autopilot terminates or only reports when an Warning is detected in the log.
It can be overwritten with the command line argument
-HANDLE_WARNING
, but be careful if multiple ErrorHandlerAgents are defined, they will all be overwritten with the same value. - Ignore Messages
- Log messages containing the specified strings will be ignored from the stop condition. Regex is also available; escape is a single backslash (`\`).
Note
Recommend enabling a Handle Exception
to interrupt Autopilot if an exception occurs during execution.
An Agent that monitors the appearance of the EmergencyExitAnnotations
component in the DeNA.Anjin.Annotations
assembly and clicks on it as soon as it appears.
This can be used in games that contain behavior that is irregular in the execution of the test scenario, for example, communication errors or "return to title screen" buttons that are triggered by a daybreak.
It should always be started at the same time as other Agents (that actually perform game operations).
This can be accomplished with ParallelCompositeAgent
, but it is easier to set it up as an Scene Crossing Agents
in AutopilotSettings.
The following Logger types are provided. These can be used as they are, or game-title-specific custom Loggers can be implemented and used.
A Logger that outputs to a console.
The instance of this Logger (.asset file) can have the following settings.
- Filter LogType
- To selective enable debug log message
A Logger that outputs to a specified file.
The instance of this Logger (.asset file) can have the following settings.
- Output File Path
- Log output file path. Specify relative path from project root or absolute path. When run on player, it will be the
Application.persistentDataPath
. It can be overwritten with the command line argument-FILE_LOGGER_OUTPUT_PATH
, but be careful if multiple FileLoggers are defined, they will all be overwritten with the same value. - Filter LogType
- To selective enable debug log message
- Timestamp
- Output timestamp to log entities
The following Reporter types are provided. These can be used as they are, or game-title-specific custom Reporters can be implemented and used.
A reporter that outputs report files in JUnit XML format. If there are zero errors and zero failures, the autopilot run is considered to have completed successfully.
The instance of this Reporter (.asset file) can have the following settings.
- Output File Path
- JUnit XML report file output path. Specify relative path from project root or absolute path. When run on player, it will be the
Application.persistentDataPath
. It can be overwritten with the command line argument-JUNIT_REPORT_PATH
, but be careful if multiple JUnitXmlReporters are defined, they will all be overwritten with the same value.
A Reporter that post report to Slack.
The instance of this Reporter (.asset file) can have the following settings.
- Slack Token
- OAuth token of Slack Bot used for notifications. If omitted, no notifications will be sent.
It can be overwritten with the command line argument
-SLACK_TOKEN
, but be careful if multiple SlackReporters are defined, they will all be overwritten with the same value. - Slack Channels
- Channels to send notifications. If omitted, not notified. Multiple channels can be specified by separating them with commas.
Note that the bot must be invited to the channel.
It can be overwritten with the command line argument
-SLACK_CHANNELS
, but be careful if multiple SlackReporters are defined, they will all be overwritten with the same value. - Mention Sub Team IDs
- Comma-separated sub team IDs to mention when posting error reports.
- Add @here
- Add @here to the post when posting error reports.
- Lead Text
- Lead text for error reports. It is used in OS notifications. You can specify placeholders like a "{message}".
- Message
- Message body template for error reports. You can specify placeholders like a "{message}".
- Color
- Attachments color for error reports. Default color is same as Slack's "danger" red.
- Screenshot
- Take a screenshot for error reports. Default is on.
- Post if completes
- Also post a report if completed autopilot normally. Default is off.
You can create a Slack Bot on the following page:
Slack API: Applications
The Slack Bot needs the following permissions:
- chat:write
- files:write
The placeholders you can include in the lead and message body template are:
- "{message}": Message with terminate (e.g., error log message)
- "{settings}": Name of running AutopilotSettings
- "{env.KEY}": Environment variables
Game-title-specific custom Agents and initialization code must be avoided in the release build.
Create a unique assembly and turn off "Auto Referenced", and set the "Define Constraints" to UNITY_INCLUDE_TESTS || DENA_AUTOPILOT_ENABLE
.
This asmdef and its storage folder can be created by opening the context menu anywhere in the Project window and selecting Create > Anjin > Game Title Specific Assembly Folder.
A custom Agent is created by inheriting from Anjin.Agents.AbstractAgent
.
Simply implement the process to be executed by the Agent in the method UniTask Run()
.
The following fields defined in AbstractAgent
are available. Each instance is set before calling the Run
method.
Logger
: Implementation ofUnityEngine.ILogger
specified in AutopilotSettings. Use this if you want to log output in Agent.Random
: Implementation ofAnjin.Utilities.IRandom
. Which is created from a seed specified in AutopilotSettings or a startup argument.
Note that it is convenient to set the [CreateAssetMenu]
attribute to create an instance from the context menu.
A custom Logger is created by inheriting from Anjin.Logers.AbstractLoggerAsset
.
You must implement the Logger { get; }
property to return custom UnityEngine.ILogger
implementation.
Note that it is convenient to set the [CreateAssetMenu]
attribute to create an instance from the context menu.
A custom Reporter is created by inheriting from Anjin.Reporters.AbstractReporter
.
Simply implement the method UniTask PostReportAsync()
.
Note that it is convenient to set the [CreateAssetMenu]
attribute to create an instance from the context menu.
If your game title requires its specific initialization process, add the InitializeOnLaunchAutopilot
attribute to the static
method that does the initialization.
An added method is called from the autopilot launch process.
[InitializeOnLaunchAutopilot]
public static void InitializeOnLaunchAutopilotMethod()
{
// initialize code for your game.
}
Async methods are also supported.
[InitializeOnLaunchAutopilot]
private static async UniTask InitializeOnLaunchAutopilotMethodAsync()
{
// initialize code for your game.
}
Note
You can specify callback order with argument. Callbacks with lower values are called before ones with higher values.
Note
The autopilot launch process is performed with RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.AfterSceneLoad)
(default for RuntimeInitializeOnLoadMethod
).
Also, RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.SubsystemRegistration)
implements the initialization process for Configurable Enter Play Mode.
Annotations are defined to control Anjin operations.
Use the DeNA.Anjin.Annotations
assembly by adding it to the Assembly Definition References.
Please note that this will be included in the release build due to the way it works.
Note
Even if the annotations assembly is removed from the release build, the link to the annotation component will remain Scenes and Prefabs in the asset bundle built. Therefore, a warning log will be output during instantiate. To avoid this, annotations assembly are included in release builds.
The GameObject
to which this component is attached avoids manipulation by the UGUIMonkeyAgent
.
When a Button
to which this component is attached appears, the EmergencyExitAgent
will immediately attempt to click on it.
It is intended to be attached to buttons that are irregular in the execution of the test scenario, such as the "return to title screen" button due to a communication error or a daybreak.
It can run autopilot on the players (real devices) if Anjin is included in player builds.
Warning
This feature is experimental. And Anjin itself is designed without much consideration for memory allocation, so please be careful when using it for performance testing.
Set a custom scripting symbols. See below:
Manual: Custom scripting symbols
Any necessary configuration files must be placed in the Resources
folder to be included in the player build.
It can use IPreprocessBuildWithReport
and IPostprocessBuildWithReport
to insert processing into the player build.
Built-in File Logger and Agent screenshots are output under Application.persistentDataPath
when running the player.
This path is included in iCloud backup by default, so exclude it if not required.
UnityEngine.iOS.Device.SetNoBackupFlag(Application.persistentDataPath);
e.g., Launch autopilot with UnityDebugSheet:
AddButton("Launch autopilot", clicked: () =>
{
_drawerController.SetStateWithAnimation(DrawerState.Min); // close debug menu before launch autopilot
#if UNITY_EDITOR
const string Path = "Assets/Path/To/AutopilotSettings.asset";
#else
const string Path = "Path/To/AutopilotSettings"; // relative path from Resources and without the .asset extension.
#endif
Launcher.LaunchAutopilotAsync(Path).Forget();
});
Note
Launcher.LaunchAutopilotAsync
can also use an overload that passes an AutopilotSettings
instance as an argument.
It can also launch it using command line arguments, in addition to launching it from the debug menu. To execute from the command line, specify the following arguments.
$(ROM) -LAUNCH_AUTOPILOT_SETTINGS Path/To/AutopilotSettings
ROM
is the path to the player build executable file.-LAUNCH_AUTOPILOT_SETTINGS
is the path to the settings file (AutopilotSettings) you want to run. Path is relative to theResources
folder and without the.asset
extension.
It can specify the same arguments as when running in the editor.
Note
Built-in File Logger and Agent screenshots are output under Application.persistentDataPath
when running the player.
See below for each platform:
Application.persistentDataPath
Note
Android has a different way of specifying command line arguments. See below:
Manual: Specify Android Player command-line arguments
The AutopilotState.asset
that persists in Anjin's execution state may be in an incorrect state.
Open it in the inspector and click the Reset button.
If this does not solve the problem, try deleting AutopilotState.asset
.
The following compilation error has been reported in some cases:
[CompilerError] Argument 1:.
Cannot convert from 'DeNA.Anjin.Reporters.SlackReporter.CoroutineRunner' to 'System.Threading.CancellationToken'
Compiler Error at Library\PackageCache\[email protected]\Runtime\Reporters\SlackReporter.cs:66 column 53
This compile error occurs when using UniTask v2.3.0 or older.
The UniTask.WaitForEndOfFrame(MonoBehaviour)
API was implemented in UniTask v2.3.1
Usually, the correct version of UniTask is installed if you follow the installation instructions.
However, if UniTask is placed as an embedded package in the project, it will be prioritized.
Find out why you have an old UniTask installed.
e.g., Check the contents of Packages/packages-lock.json generated by the Unity editor; use your IDE's code definition jump function to check where the source file of UniTask.WaitForEndOfFrame()
is.
For example, the following warning message may be output log.
Slack settings in AutopilotSettings has been obsolete.
Please delete the value using Debug Mode in the Inspector window. And create a SlackReporter asset file.
Even if you have already migrated to a new setting method (SlackReporter
in the above example), you are warned that values remain in obsolete fields.
You can edit obsolete fields by opening the settings file in the Inspector window and switching to Debug Mode.
For information on how to use the Inspector window, see the Unity manual:
Manual: Working in the Inspector
MIT License
Open an issue or create a pull request.
Be grateful if you could label the pull request as enhancement
, bug
, chore
, and documentation
. See PR Labeler settings for automatically labeling from the branch name.
The general policy for accepting new features is as follows:
- All built-in features can be easily configured in the Unity Editor's Inspector window.
- Avoid adding features to the
Autopilot
class as much as possible, and consider expanding it with Agents, etc. - Refrain from adding non-general-purpose Agents, Loggers, and Reporters. Consider publishing them on your blog or Gist or placing them in Samples.
The following feature requests/ pull requests will be rejected because they go against the concept of Anjin:
- Ability to run multiple test scenarios (AutopilotSettings) in succession. Use the feature to run from Play Mode tests.
- Ability to specify "Start Scene" in AutopilotSettings. We recommend creating a test scenario that can be run from any Scene.
Add this repository as a submodule to the Packages/ directory in your project.
Run the command below:
git submodule add https://github.com/dena/Anjin.git Packages/com.dena.anjin
Warning
Required install packages for running tests (when adding to the testables
in package.json), as follows:
- Unity Test Framework package v1.3.4 or later
- Test Helper package v0.7.2 or later
Generate a temporary project and run tests on each Unity version from the command line.
make create_project
UNITY_VERSION=2019.4.40f1 make -k test
Run Actions > Create release pull request > Run workflow and merge created pull request. (Or bump version in package.json on default branch)
Then, Will do the release process automatically by Release workflow. And after tagged, OpenUPM retrieves the tag and updates it.
Do NOT manually operation the following operations:
- Create release tag
- Publish draft releases