Skip to content

Latest commit

 

History

History
929 lines (707 loc) · 27 KB

naming.md

File metadata and controls

929 lines (707 loc) · 27 KB
Raw

File Naming

Naming determines the file name for the .received. resulting .verified. files.

The format is

{Directory}/{TestClassName}.{TestMethodName}_{Parameters}_{UniqueFor1}_{UniqueFor2}_{UniqueForX}.verified.{extension}

Directory

The directory that contains the test.

The path provided can be absolute or relative to the directory that contains the test.

Instance

var settings = new VerifySettings();
settings.UseDirectory("CustomDirectory");
await Verify("valueUseDirectory", settings);

snippet source | anchor

Fluent

await Verify("valueUseDirectoryFluent")
    .UseDirectory("CustomDirectory");

snippet source | anchor

Will result in CustomDirectory/TypeName.MethodName.verified.txt.

TestClassName

The class name that contains the test. A custom test name can be used via UseTypeName:

Instance

var settings = new VerifySettings();
settings.UseTypeName("CustomTypeName");
await Verify("valueUseTypeName", settings);

snippet source | anchor

Fluent

await Verify("valueUseTypeNameFluent")
    .UseTypeName("CustomTypeName");

snippet source | anchor

Will result in CustomTypeName.MethodName.verified.txt.

TestMethodName

The test method name. A custom test name can be used via UseMethodName:

var settings = new VerifySettings();
settings.UseMethodName("CustomMethodName");
await Verify("valueUseMethodName", settings);

snippet source | anchor

Will result in TestClass.CustomMethodName.verified.txt.

Fluent

await Verify("valueUseMethodNameFluent")
    .UseMethodName("CustomMethodNameFluent");

snippet source | anchor

Will result in TestClass.CustomMethodNameFluent.verified.txt.

Multiple calls to Verify

UseMethodName can also be used to allow multiple calls to Verify in the same method:

[Fact]
public Task MultipleCalls() =>
    Task.WhenAll(
        Verify("Value1MultipleCalls")
            .UseMethodName("MultipleCalls_1"),
        Verify("Value1MultipleCalls")
            .UseMethodName("MultipleCalls_2"));

snippet source | anchor

UseFileName

To fully control the {TestClassName}.{TestMethodName}_{Parameters} parts of the file use UseFileName:

Instance

var settings = new VerifySettings();
settings.UseFileName("CustomFileName");
await Verify("valueUseFileName", settings);

snippet source | anchor

Will result in CustomFileName.verified.txt.

Fluent

await Verify("valueUseFileNameFluent")
    .UseFileName("CustomFileNameFluent");

snippet source | anchor

Will result in UseFileNameFluent.verified.txt.

Compatibility:

  • Not compatible with UseTypeName, UseMethodName, or UseParameters. An exception will be thrown if they are combined.
  • Can be used in combination with UseDirectory.
  • Can be used in combination with UniqueFor*.

Parameters

See Parameterised Tests.

UniqueFor

UniqueFor* allows for one or more delimiters to be added to the file name.

NUnit

[TestFixture]
public class UniqueForSample
{
    [Test]
    public Task Runtime()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntime();
        return Verify("value", settings);
    }

    [Test]
    public Task RuntimeFluent() =>
        Verify("value")
            .UniqueForRuntime();

    [Test]
    public Task AssemblyConfiguration()
    {
        var settings = new VerifySettings();
        settings.UniqueForAssemblyConfiguration();
        return Verify("value", settings);
    }

    [Test]
    public Task AssemblyConfigurationFluent() =>
        Verify("value")
            .UniqueForAssemblyConfiguration();

    [Test]
    public Task RuntimeAndVersion()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntimeAndVersion();
        return Verify("value", settings);
    }

    [Test]
    public Task RuntimeAndVersionFluent() =>
        Verify("value")
            .UniqueForRuntimeAndVersion();

    [Test]
    public Task Architecture()
    {
        var settings = new VerifySettings();
        settings.UniqueForArchitecture();
        return Verify("value", settings);
    }

    [Test]
    public Task ArchitectureFluent() =>
        Verify("value")
            .UniqueForArchitecture();

    [Test]
    public Task OSPlatform()
    {
        var settings = new VerifySettings();
        settings.UniqueForOSPlatform();
        return Verify("value", settings);
    }

    [Test]
    public Task OSPlatformFluent() =>
        Verify("value")
            .UniqueForOSPlatform();
}

snippet source | anchor

XUnit

public class UniqueForSample
{
    [Fact]
    public Task Runtime()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntime();
        return Verify("value", settings);
    }

    [Fact]
    public Task RuntimeFluent() =>
        Verify("value")
            .UniqueForRuntime();

    [Fact]
    public Task RuntimeAndVersion()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntimeAndVersion();
        return Verify("value", settings);
    }

    [Fact]
    public Task AssemblyConfiguration()
    {
        var settings = new VerifySettings();
        settings.UniqueForAssemblyConfiguration();
        return Verify("value", settings);
    }

    [Fact]
    public Task AssemblyConfigurationFluent() =>
        Verify("value")
            .UniqueForAssemblyConfiguration();

    [Fact]
    public Task Architecture()
    {
        var settings = new VerifySettings();
        settings.UniqueForArchitecture();
        return Verify("value", settings);
    }

    [Fact]
    public Task ArchitectureFluent() =>
        Verify("value")
            .UniqueForArchitecture();

    [Fact]
    public Task OSPlatform()
    {
        var settings = new VerifySettings();
        settings.UniqueForOSPlatform();
        return Verify("value", settings);
    }

    [Fact]
    public Task OSPlatformFluent() =>
        Verify("value")
            .UniqueForOSPlatform();
}

snippet source | anchor

Fixie

public class UniqueForSample
{
    public Task Runtime()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntime();
        return Verify("value", settings);
    }

    public Task RuntimeFluent() =>
        Verify("value")
            .UniqueForRuntime();

    public Task AssemblyConfiguration()
    {
        var settings = new VerifySettings();
        settings.UniqueForAssemblyConfiguration();
        return Verify("value", settings);
    }

    public Task AssemblyConfigurationFluent() =>
        Verify("value")
            .UniqueForAssemblyConfiguration();

    public Task RuntimeAndVersion()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntimeAndVersion();
        return Verify("value", settings);
    }

    public Task RuntimeAndVersionFluent() =>
        Verify("value")
            .UniqueForRuntimeAndVersion();

    public Task Architecture()
    {
        var settings = new VerifySettings();
        settings.UniqueForArchitecture();
        return Verify("value", settings);
    }

    public Task ArchitectureFluent() =>
        Verify("value")
            .UniqueForArchitecture();

    public Task OSPlatform()
    {
        var settings = new VerifySettings();
        settings.UniqueForOSPlatform();
        return Verify("value", settings);
    }

    public Task OSPlatformFluent() =>
        Verify("value")
            .UniqueForOSPlatform();
}

snippet source | anchor

MSTest

[TestClass]
public partial class UniqueForSample
{
    [TestMethod]
    public Task Runtime()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntime();
        return Verify("value", settings);
    }

    [TestMethod]
    public Task RuntimeFluent() =>
        Verify("value")
            .UniqueForRuntime();

    [TestMethod]
    public Task RuntimeAndVersion()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntimeAndVersion();
        return Verify("value", settings);
    }

    [TestMethod]
    public Task RuntimeAndVersionFluent() =>
        Verify("value")
            .UniqueForRuntimeAndVersion();

    [TestMethod]
    public Task AssemblyConfiguration()
    {
        var settings = new VerifySettings();
        settings.UniqueForAssemblyConfiguration();
        return Verify("value", settings);
    }

    [TestMethod]
    public Task AssemblyConfigurationFluent() =>
        Verify("value")
            .UniqueForAssemblyConfiguration();

    [TestMethod]
    public Task Architecture()
    {
        var settings = new VerifySettings();
        settings.UniqueForArchitecture();
        return Verify("value", settings);
    }

    [TestMethod]
    public Task ArchitectureFluent() =>
        Verify("value")
            .UniqueForArchitecture();

    [TestMethod]
    public Task OSPlatform()
    {
        var settings = new VerifySettings();
        settings.UniqueForOSPlatform();
        return Verify("value", settings);
    }

    [TestMethod]
    public Task OSPlatformFluent() =>
        Verify("value")
            .UniqueForOSPlatform();
}

snippet source | anchor

Expecto

[<Tests>]
let uniqueTests =
    testTask "unique" {
        let settings = VerifySettings()
        settings.UniqueForRuntime()
        do! Verifier.Verify("unique", "value", settings).ToTask()
    }

snippet source | anchor

TUnit

public class UniqueForSample
{
    [Test]
    public Task Runtime()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntime();
        return Verify("value", settings);
    }

    [Test]
    public Task RuntimeFluent() =>
        Verify("value")
            .UniqueForRuntime();

    [Test]
    public Task AssemblyConfiguration()
    {
        var settings = new VerifySettings();
        settings.UniqueForAssemblyConfiguration();
        return Verify("value", settings);
    }

    [Test]
    public Task AssemblyConfigurationFluent() =>
        Verify("value")
            .UniqueForAssemblyConfiguration();

    [Test]
    public Task RuntimeAndVersion()
    {
        var settings = new VerifySettings();
        settings.UniqueForRuntimeAndVersion();
        return Verify("value", settings);
    }

    [Test]
    public Task RuntimeAndVersionFluent() =>
        Verify("value")
            .UniqueForRuntimeAndVersion();

    [Test]
    public Task Architecture()
    {
        var settings = new VerifySettings();
        settings.UniqueForArchitecture();
        return Verify("value", settings);
    }

    [Test]
    public Task ArchitectureFluent() =>
        Verify("value")
            .UniqueForArchitecture();

    [Test]
    public Task OSPlatform()
    {
        var settings = new VerifySettings();
        settings.UniqueForOSPlatform();
        return Verify("value", settings);
    }

    [Test]
    public Task OSPlatformFluent() =>
        Verify("value")
            .UniqueForOSPlatform();
}

snippet source | anchor

Result

For a project executed on both x64 and x86 that targets

<TargetFrameworks>netcoreapp3.0;net48</TargetFrameworks>

Will result in the following files being produced:

UniqueForSample.Runtime.Core.verified.txt
UniqueForSample.Runtime.Net.verified.txt
UniqueForSample.RuntimeAndVersion.Core3_0.verified.txt
UniqueForSample.RuntimeAndVersion.Net4_8.verified.txt
UniqueForSample.Architecture.X86.verified.txt
UniqueForSample.Architecture.X64.verified.txt

Extension

The default file extension is .txt. So the resulting verified file will be TestClass.TestMethod.verified.txt.

It can be overridden at two levels:

  • Method: Change the extension for the current test method.
  • Class: Change the extension all verifications in all test methods for a test class.

Usage:

public class ExtensionSample
{
    [Fact]
    public Task AtMethod() =>
        Verify(
            target: """
                    <note>
                      <to>Joe</to>
                      <from>Kim</from>
                      <heading>Reminder</heading>
                    </note>
                    """,
            extension: "xml");

    [Fact]
    public Task AtMethodFluent() =>
        Verify(
            target: """
                    <note>
                      <to>Joe</to>
                      <from>Kim</from>
                      <heading>Reminder</heading>
                    </note>
                    """,
            extension: "xml");
}

snippet source | anchor

Result in:

<note>
  <to>Joe</to>
  <from>Kim</from>
  <heading>Reminder</heading>
</note>

snippet source | anchor

NamerRuntimeAndVersion

To access the current Namer Runtime or RuntimeAndVersion strings use:

Debug.WriteLine(Namer.Runtime);
Debug.WriteLine(Namer.RuntimeAndVersion);

snippet source | anchor

DerivePathInfo

DerivePathInfo allows the storage directory of .verified. files to be customized based on the current context. The contextual parameters are parameters passed are as follows:

  • sourceFile: The full path to the file that the test existed in at compile time.
  • projectDirectory: The directory that the project existed in at compile time.
  • type: The class the test method exists in.
  • method: The test method.

Return null to any of the values to use the standard behavior. The returned path can be relative to the directory sourceFile exists in.

DerivePathInfo can also be useful when deriving the storage directory on a build server

For example to place all .verified. files in a {ProjectDirectory}\Snapshots the following could be used:

Xunit

Verifier.DerivePathInfo(
    (sourceFile, projectDirectory, type, method) => new(
        directory: Path.Combine(projectDirectory, "Snapshots"),
        typeName: type.Name,
        methodName: method.Name));

snippet source | anchor

NUnit

Verifier.DerivePathInfo(
    (sourceFile, projectDirectory, type, method) => new(
        directory: Path.Combine(projectDirectory, "Snapshots"),
        typeName: type.Name,
        methodName: method.Name));

snippet source | anchor

MSTest

Verifier.DerivePathInfo(
    (sourceFile, projectDirectory, type, method) => new(
        directory: Path.Combine(projectDirectory, "Snapshots"),
        typeName: type.Name,
        methodName: method.Name));

snippet source | anchor

Expecto

Verifier.DerivePathInfo(
    (sourceFile, projectDirectory, type, method) => new(
        directory: Path.Combine(projectDirectory, "Snapshots"),
        typeName: type,
        methodName: method));

snippet source | anchor

As a nuget

A DerivePathInfo convention can be shipped as a NuGet, for example Spectre.Verify.Extensions which adds an attribute driven file naming convention to Verify.

Default DerivePathInfo

public static PathInfo DeriveDefault(
    string sourceFile,
    string projectDirectory,
    Type type,
    MethodInfo method) =>
    new(
        directory: IoHelpers.ResolveDirectoryFromSourceFile(sourceFile),
        typeName: type.NameWithParent(),
        methodName: method.Name);

snippet source | anchor

Where NameWithParent is

public static string NameWithParent(this Type type)
{
    if (type.IsNested)
    {
        return $"{type.ReflectedType!.Name}.{type.Name}";
    }

    return type.Name;
}

snippet source | anchor

Any path calculated in DerivePathInfo should be fully qualified to remove the inconsistency of the current directory.

UseProjectRelativeDirectory

Verifier.UseProjectRelativeDirectory is a wrapper around DerivePathInfo that stores all .verified. files in a directory relative to the project directory.

For example to place all .verified. files in a {ProjectDirectory}\Snapshots the following could be used:

Verifier.UseProjectRelativeDirectory("Snapshots");

UseSourceFileRelativeDirectory

Verifier.UseSourceFileRelativeDirectory is a wrapper around DerivePathInfo that stores all .verified. files in a directory relative to the source file directory.

For example to place all .verified. files in a {SourceFileDirectory}\Snapshots the following could be used:

Verifier.UseSourceFileRelativeDirectory("Snapshots");

DisableRequireUniquePrefix

Snapshot file names have to be unique. If a duplicate name is used, then an exception will be throw. This is mostly caused by a conflicting combination of Verifier.DerivePathInfo(), UseMethodName.UseDirectory(), UseMethodName.UseTypeName(), and UseMethodName.UseMethodName(). If that's not the case, and having multiple identical prefixes is acceptable, then call VerifierSettings.DisableRequireUniquePrefix() to disable this uniqueness validation

UseUniqueDirectory

An alternative to the "unique file name in the current test directory".

This approach uses "a unique directory in the current test directory".

Useful when many test produce many files, and it is desirable to have them grouped in a directory.

The file format is:

{CurrentDirectory}/{TestClassName}.{TestMethodName}_{Parameters}_{UniqueFor1}_{UniqueFor2}_{UniqueForX}/{targetName}.verified.{extension}


var settings = new VerifySettings();
settings.UseUniqueDirectory();
await Verify("TheValue", settings);

snippet source | anchor

UseSplitModeForUniqueDirectory

UseSplitModeForUniqueDirectory is a global option that changes the behavior of all UseUniqueDirectory uses.

The received and verified are split and each exist in their own directory.

The file format is:

For received files:

{CurrentDirectory}/{TestClassName}.{TestMethodName}_{Parameters}_{UniqueFor1}_{UniqueFor2}_{UniqueForX}.received/{targetName}.{extension}

For verified files:

{CurrentDirectory}/{TestClassName}.{TestMethodName}_{Parameters}_{UniqueFor1}_{UniqueFor2}_{UniqueForX}.verified/{targetName}.{extension}


public static class ModuleInitializer
{
    [ModuleInitializer]
    public static void Init() =>
        VerifierSettings.UseSplitModeForUniqueDirectory();
}

snippet source | anchor

Also exclude *.received/ from source control.

eg. add the following to .gitignore

*.received/

Received and multi-targeting

When a test project uses more than one TargetFrameworks (eg <TargetFrameworks>net48;net7.0</TargetFrameworks>) the runtime and version will be always be added as a uniqueness to the received file. This prevents file locking contention when the tests from both target framework run in parallel.

Orphaned verified files

One problem with Verify is there is currently no way to track or clean up orphaned verified files.

Scenario

Given the following test

[TestFixture]
public class MyFixture
{
    [Test]
    public Task MyTest1() => Verify("Value");
}

The resulting verified file will be MyFixture.MyTest1.verified.txt

Now the test is changed to

[TestFixture]
public class MyFixture
{
    [Test]
    public Task Test1() => Verify("Value");
}

The new resulting verified file will be MyFixture.Test1.verified.txt.

The old file, MyFixture.MyTest1.verified.txt, will be now orphaned and never be cleaned up.

Mitigation

For small renames, with resulting small number of orphaned files, the recommended approach is to manually rename the verified files. Or alternatively:

  • Delete the orphaned *.verified.* files.
  • Run the test(s)
  • Accept all changes using one of the Snapshot management approaches.

In some scenarios it may be necessary to clean up many orphaned files. For example from a rename of test fixture with many tests, or a test with many parameter permutations. In this case the delete can be performed by DiffEngine Tray - Purge verified files feature.