Skip to content

BenchmarkDotNet should warn if there are no benchmarks #2712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
richardcocks opened this issue Mar 30, 2025 · 5 comments · May be fixed by #2718
Open

BenchmarkDotNet should warn if there are no benchmarks #2712

richardcocks opened this issue Mar 30, 2025 · 5 comments · May be fixed by #2718

Comments

@richardcocks
Copy link

I've got a small benchmark that I was about to open a bug report because it appeared to "crash" during Benchmark Validation.

It shows // Validating benchmarks: and then the process exits without any error or warning.

I'm an idiot, and it turns out I forgot to annotate any methods at all with [Benchmark].

A warning that "No benchmarks found" would help idiots like me realise their mistake without thinking that validation is crashing.

@CodingwithKetan
Copy link

@timcassell, can I work on this issue?

@timcassell
Copy link
Collaborator

Yes

@CodingwithKetan
Copy link

Thank you @timcassell !

@AvishaiDotan
Copy link
Contributor

@CodingwithKetan Have you had a chance to make any progress on this?

It looks like checking whether supportedBenchmarks.Length == 0 could be a good way to handle this. If there are no supported benchmarks, logging an informative message and returning a summary array might be sufficient. Reference:

compositeLogger.WriteLineInfo("// Validating benchmarks:");

if (supportedBenchmarks.Length == 0)
{
    compositeLogger.WriteLineError("// No benchmarks have been found");
    return new[] { Summary.ValidationFailed(title, resultsFolderPath, logFilePath,
        ImmutableArray.Create(new ValidationError(true, "No benchmarks have been found"))) };
}

If you're not actively working on this, I’d be happy to pick it up — just let me know.

@CodingwithKetan
Copy link

@AvishaiDotan, as you got solution, so you can work on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants