Tests with DataProvider attribute throw error

[PAXANDDOS]

Q	A
PHPUnit version	10.0.16
PHP version	8.2.3
Installation Method	Composer

Summary

When a test is using DataProvider attribute, it throws an error during execution.

Current behavior

Following error is thrown during execution:

/var/www/recon # vendor/bin/phpunit tests/Unit/BadNumberTest.php

An error occurred inside PHPUnit.

Message:  Attribute class "PHPUnit\Framework\Attributes\DataProvider" not found
Location: /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/AttributeParser.php:310

#0 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/AttributeParser.php(310): ReflectionAttribute->newInstance()
#1 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/ParserChain.php(48): PHPUnit\Metadata\Parser\AttributeParser->forMethod('Tests\\Unit\\BadN...', 'testUsage')     
#2 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/CachingParser.php(57): PHPUnit\Metadata\Parser\ParserChain->forMethod('Tests\\Unit\\BadN...', 'testUsage')       
#3 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Api/DataProvider.php(57): PHPUnit\Metadata\Parser\CachingParser->forMethod('Tests\\Unit\\BadN...', 'testUsage')
#4 /var/www/recon/vendor/phpunit/phpunit/src/Framework/TestBuilder.php(38): PHPUnit\Metadata\Api\DataProvider->providedData('Tests\\Unit\\BadN...', 'testUsage')
#5 /var/www/recon/vendor/phpunit/phpunit/src/Framework/TestSuite.php(490): PHPUnit\Framework\TestBuilder->build(Object(ReflectionClass), 'testUsage')
#6 /var/www/recon/vendor/phpunit/phpunit/src/Framework/TestSuite.php(131): PHPUnit\Framework\TestSuite->addTestMethod(Object(ReflectionClass), Object(ReflectionMethod))      
#7 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Configuration/TestSuiteBuilder.php(96): PHPUnit\Framework\TestSuite::fromClassReflector(Object(ReflectionClass))
#8 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Configuration/TestSuiteBuilder.php(46): PHPUnit\TextUI\Configuration\TestSuiteBuilder->testSuiteFromPath('/var/www/recon/...', Array)
#9 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Application.php(303): PHPUnit\TextUI\Configuration\TestSuiteBuilder->build(Object(PHPUnit\TextUI\Configuration\Configuration))
#10 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Application.php(99): PHPUnit\TextUI\Application->buildTestSuite(Object(PHPUnit\TextUI\Configuration\Configuration))      
#11 /var/www/recon/vendor/phpunit/phpunit/phpunit(90): PHPUnit\TextUI\Application->run(Array)
#12 /var/www/recon/vendor/bin/phpunit(123): include('/var/www/recon/...')
#13 {main}

How to reproduce

Minimal reproduction test:

<?php

declare(strict_types=1);

namespace Tests\Unit;

use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

final class BadNumberTest extends TestCase
{
    #[DataProvider('myProvider')]
    public function testUsage(string $expected, string $input): void
    {
        $this->assertEmpty('');
    }

    public static function myProvider(): array
    {
        return [['', '']];
    }
}

If we remove the attribute and function arguments, test will work just fine.

Expected behavior

Test is executed

Composer

PHPUnit is not wrapped in a framework, I just use packages from Symfony and CakePHP.

composer info | sort

cakephp/cache                      4.4.11           Easy to use Caching library with support for multiple caching backends
cakephp/chronos                    2.3.2            A simple API extension for DateTime.
cakephp/collection                 4.4.11           Work easily with arrays and iterators by having a battery of utility traversal methods
cakephp/core                       4.4.11           CakePHP Framework Core classes
cakephp/database                   4.4.11           Flexible and powerful Database abstraction library with a familiar PDO-like API
cakephp/datasource                 4.4.11           Provides connection managing and traits for Entities and Queries that can be reused for different datastores
cakephp/event                      4.4.11           CakePHP event dispatcher library that helps implementing the observer pattern
cakephp/i18n                       4.4.11           CakePHP Internationalization library with support for messages translation and dates and numbers localization
cakephp/log                        4.4.11           CakePHP logging library with support for multiple different streams
cakephp/orm                        4.4.11           CakePHP ORM - Provides a flexible and powerful ORM implementing a data-mapper pattern.
cakephp/utility                    4.4.11           CakePHP Utility classes such as Inflector, String, Hash, and Security
cakephp/validation                 4.4.11           CakePHP Validation library
composer/pcre                      3.1.0            PCRE wrapping library that offers type-safe preg_* replacements.
composer/semver                    3.3.2            Semver library that offers utilities, version constraint parsing and validation.
composer/xdebug-handler            3.0.3            Restarts a process without Xdebug.
doctrine/annotations               2.0.1            Docblock Annotations Parser
doctrine/inflector                 2.0.6            PHP Doctrine Inflector is a small library that can perform string manipulations with regard to upper/lowercase and ...    
doctrine/lexer                     3.0.0            PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers.
dragonmantank/cron-expression      v3.3.2           CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
egulias/email-validator            4.0.1            A library for validating emails against several RFCs
ezyang/htmlpurifier                v4.16.0          Standards compliant HTML filter written in PHP
francerz/enum                      v0.1.1           PHP Enumeration tool
francerz/php-power-data            v0.1.28          PHP data structures for complex data handling
francerz/sql-builder               v0.5.10          SQL Builder
friendsofphp/php-cs-fixer          v3.15.1          A tool to automatically fix PHP code style
gregwar/gnuplot                    v1.0.2           GnuPlot library
guzzlehttp/guzzle                  7.5.0            Guzzle is a PHP HTTP client library
guzzlehttp/promises                1.5.2            Guzzle promises library
guzzlehttp/psr7                    2.4.4            PSR-7 message implementation that also provides common utility methods
illuminate/collections             v10.3.3          The Illuminate Collections package.
illuminate/conditionable           v10.3.3          The Illuminate Conditionable package.
illuminate/container               v10.3.3          The Illuminate Container package.
illuminate/contracts               v10.3.3          The Illuminate Contracts package.
illuminate/macroable               v10.3.3          The Illuminate Macroable package.
illuminate/support                 v10.3.3          The Illuminate Support package.
league/csv                         9.9.0            CSV data manipulation made easy in PHP
lorisleiva/cron-translator         v0.4.2           Makes CRON expressions human-readable
maennchen/zipstream-php            v2.4.0           ZipStream is a library for dynamically streaming dynamic zip files from PHP without writing to the disk at all on t...    
markbaker/complex                  3.0.2            PHP Class for working with complex numbers
markbaker/matrix                   3.0.1            PHP Class for working with matrices
mjphaynes/php-resque               v4.x-dev 48c81ea Redis backed library for creating background jobs and processing them later.
monolog/monolog                    2.9.1            Sends your logs to files, sockets, inboxes, databases and various web services
myclabs/deep-copy                  1.11.1           Create deep copies (clones) of your objects
myclabs/php-enum                   1.8.4            PHP Enum implementation
napp/salesforce-api                1.0.7            A library for interacting with the Salesforce REST API and managing the OAuth flow
nesbot/carbon                      2.66.0           An API extension for DateTime that supports 281 different languages.
nikic/php-parser                   v4.15.4          A PHP parser written in PHP
phar-io/manifest                   2.0.3            Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                    3.2.1            Library for handling version information and constraints
phpoffice/phpspreadsheet           1.28.0           PHPSpreadsheet - Read, Create and Write Spreadsheet documents in PHP - Spreadsheet engine
phpspec/php-diff                   v1.1.3           A comprehensive library for generating differences between two hashable objects (strings or arrays).
phpstan/phpstan                    1.10.6           PHPStan - PHP Static Analysis Tool
phpunit/php-code-coverage          10.0.2           Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          4.0.1            FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                4.0.0            Invoke callables with a timeout
phpunit/php-text-template          3.0.0            Simple template engine.
phpunit/php-timer                  6.0.0            Utility class for timing
phpunit/phpunit                    10.0.16          The PHP Unit Testing framework.
predis/predis                      v1.1.10          Flexible and feature-complete Redis client for PHP and HHVM
psr/cache                          3.0.0            Common interface for caching libraries
psr/container                      2.0.2            Common Container Interface (PHP FIG PSR-11)
psr/event-dispatcher               1.0.0            Standard interfaces for event handling.
psr/http-client                    1.0.1            Common interface for HTTP clients
psr/http-factory                   1.0.1            Common interfaces for PSR-7 HTTP message factories
psr/http-message                   1.0.1            Common interface for HTTP messages
psr/log                            2.0.0            Common interface for logging libraries
psr/simple-cache                   2.0.0            Common interfaces for simple caching
ralouphie/getallheaders            3.0.3            A polyfill for getallheaders.
robmorgan/phinx                    0.13.4           Phinx makes it ridiculously easy to manage the database migrations for your PHP app.
sebastian/cli-parser               2.0.0            Library for parsing CLI options
sebastian/code-unit                2.0.0            Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 3.0.0            Looks up which function or method a line of code belongs to
sebastian/comparator               5.0.0            Provides the functionality to compare PHP values for equality
sebastian/complexity               3.0.0            Library for calculating the complexity of PHP code units
sebastian/diff                     5.0.0            Diff implementation
sebastian/environment              6.0.0            Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 5.0.0            Provides the functionality to export PHP variables for visualization
sebastian/global-state             6.0.0            Snapshotting of global state
sebastian/lines-of-code            2.0.0            Library for counting the lines of code in PHP source code
sebastian/object-enumerator        5.0.0            Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector         3.0.0            Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context        5.0.0            Provides functionality to recursively process PHP variables
sebastian/type                     4.0.0            Collection of value objects that represent the types of the PHP type system
sebastian/version                  4.0.1            Library that helps with managing the version number of Git-hosted PHP projects
symfony/config                     v6.2.7           Helps you find, load, combine, autofill and validate configuration values of any kind
symfony/console                    v6.2.7           Eases the creation of beautiful and testable command line interfaces
symfony/deprecation-contracts      v3.2.1           A generic function and convention to trigger deprecation notices
symfony/dotenv                     v6.2.7           Registers environment variables from a .env file
symfony/error-handler              v6.2.7           Provides tools to manage errors and ease debugging PHP code
symfony/event-dispatcher           v6.2.7           Provides tools that allow your application components to communicate with each other by dispatching events and list...    
symfony/event-dispatcher-contracts v3.2.1           Generic abstractions related to dispatching event
symfony/filesystem                 v6.2.7           Provides basic utilities for the filesystem
symfony/finder                     v6.2.7           Finds files and directories via an intuitive fluent interface
symfony/google-mailer              v6.2.7           Symfony Google Mailer Bridge
symfony/http-foundation            v6.2.7           Defines an object-oriented layer for the HTTP specification
symfony/http-kernel                v6.2.7           Provides a structured process for converting a Request into a Response
symfony/mailer                     v6.2.7           Helps sending emails
symfony/mime                       v6.2.7           Allows manipulating MIME messages
symfony/options-resolver           v6.2.7           Provides an improved replacement for the array_replace PHP function
symfony/polyfill-ctype             v1.27.0          Symfony polyfill for ctype functions
symfony/polyfill-intl-grapheme     v1.27.0          Symfony polyfill for intl's grapheme_* functions
symfony/polyfill-intl-idn          v1.27.0          Symfony polyfill for intl's idn_to_ascii and idn_to_utf8 functions
symfony/polyfill-intl-normalizer   v1.27.0          Symfony polyfill for intl's Normalizer class and related functions
symfony/polyfill-mbstring          v1.27.0          Symfony polyfill for the Mbstring extension
symfony/polyfill-php72             v1.27.0          Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions
symfony/polyfill-php80             v1.27.0          Symfony polyfill backporting some PHP 8.0+ features to lower PHP versions
symfony/polyfill-php81             v1.27.0          Symfony polyfill backporting some PHP 8.1+ features to lower PHP versions
symfony/process                    v6.2.7           Executes commands in sub-processes
symfony/routing                    v6.2.7           Maps an HTTP request to a set of configuration variables
symfony/service-contracts          v3.2.1           Generic abstractions related to writing services
symfony/stopwatch                  v6.2.7           Provides a way to profile code
symfony/string                     v6.2.7           Provides an object-oriented API to strings and deals with bytes, UTF-8 code points and grapheme clusters in a unifi...    
symfony/translation                v6.2.7           Provides tools to internationalize your application
symfony/translation-contracts      v3.2.1           Generic abstractions related to translation
symfony/twig-bridge                v6.2.7           Provides integration for Twig with various Symfony components
symfony/var-dumper                 v6.2.7           Provides mechanisms for walking through any arbitrary PHP variable
symfony/yaml                       v6.2.7           Loads and dumps YAML files
theseer/tokenizer                  1.2.1            A small library for converting tokenized PHP source code into XML and potentially other formats
troydavisson/phrets                2.6.3            RETS library in PHP
twig/twig                          v3.5.1           Twig, the flexible, fast, and secure template language for PHP
voku/portable-ascii                2.0.1            Portable ASCII library - performance optimized (ascii) string functions for php.
webmozart/assert                   1.11.0           Assertions to validate method input/output with nice error messages.
wyndow/fuzzywuzzy                  v0.6.0           Fuzzy string matching based on FuzzyWuzzy from Seatgeek

[sebastianbergmann]

I cannot reproduce this:

$ cat composer.json                     
{
    "require-dev": {
        "phpunit/phpunit": "^10.0"
    }
}

$ composer info                         
myclabs/deep-copy                  1.11.1  Create deep copies (clones) of your objects
nikic/php-parser                   v4.15.4 A PHP parser written in PHP
phar-io/manifest                   2.0.3   Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                    3.2.1   Library for handling version information and constraints
phpunit/php-code-coverage          10.0.2  Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          4.0.1   FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                4.0.0   Invoke callables with a timeout
phpunit/php-text-template          3.0.0   Simple template engine.
phpunit/php-timer                  6.0.0   Utility class for timing
phpunit/phpunit                    10.0.16 The PHP Unit Testing framework.
sebastian/cli-parser               2.0.0   Library for parsing CLI options
sebastian/code-unit                2.0.0   Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 3.0.0   Looks up which function or method a line of code belongs to
sebastian/comparator               5.0.0   Provides the functionality to compare PHP values for equality
sebastian/complexity               3.0.0   Library for calculating the complexity of PHP code units
sebastian/diff                     5.0.0   Diff implementation
sebastian/environment              6.0.0   Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 5.0.0   Provides the functionality to export PHP variables for visualization
sebastian/global-state             6.0.0   Snapshotting of global state
sebastian/lines-of-code            2.0.0   Library for counting the lines of code in PHP source code
sebastian/object-enumerator        5.0.0   Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector         3.0.0   Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context        5.0.0   Provides functionality to recursively process PHP variables
sebastian/type                     4.0.0   Collection of value objects that represent the types of the PHP type system
sebastian/version                  4.0.1   Library that helps with managing the version number of Git-hosted PHP projects
theseer/tokenizer                  1.2.1   A small library for converting tokenized PHP source code into XML and potentially other formats

$ cat BadNumberTest.php 
<?php

declare(strict_types=1);

namespace Tests\Unit;

use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

final class BadNumberTest extends TestCase
{
    #[DataProvider('myProvider')]
    public function testUsage(string $expected, string $input): void
    {
        $this->assertEmpty('');
    }

    public static function myProvider(): array
    {
        return [['', '']];
    }
}

$ ./vendor/bin/phpunit BadNumberTest.php                      
PHPUnit 10.0.16 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.3

.                                                                   1 / 1 (100%)

Time: 00:00.008, Memory: 4.00 MB

OK (1 test, 1 assertion)

[PAXANDDOS]

Thanks for looking into this @sebastianbergmann!
This is so weird... I was about to send you a repository and the docker image with the error but it disappeared for whatever reason. Probably has to do with the other installed extension, because they were the only thing I removed from the Dockerfile while testing.
Unfortunately don't have much time so I will return to that later!

[PAXANDDOS]

@sebastianbergmann
I guess I found the issue! And it was quite unexpected...
So for development, I use Windows 11 and Docker w/ WSL2 and this error happens when I have my working directory mounted via volume from my PC to the container, and when I don't volume - everything works smoothly.
This might be related to the DataProvider attribute only, or to any other, I haven't checked yet.

Code

see my repository

download code phpunit-issue-master.zip

To reproduce:

I have a docker image uploaded to Docker Hub or you can build it manually:

1. docker build -t phpunit-issue .
2. docker run -it --rm -v ${PWD}:/var/www/phpunit phpunit-issue sh

Without -v ${PWD}:/var/www/phpunit option set:

1. docker run -it --rm paxanddos/phpunit-issue sh
2. vendor/bin/phpunit
3. See that everything works fine

With -v ${PWD}:/var/www/phpunit set:

1. cd to the repository folder
2. docker run -it --rm -v ${PWD}:/var/www/phpunit paxanddos/phpunit-issue sh
3. composer install (mounted folder w/o vendor)
4. vendor/bin/phpunit
5. See that error is thrown

Strange that none of my other packages throw errors like that (if it's related to all attributes), PHPUnit 10 seems to be the only one with such an issue. I also have attributes used it that project, and they work just fine. Can't think of a reason...

[Echron]

@PAXANDDOS I'm having the same issue (Win 11 - Docker WSL2). I'm wondering if it's an issue in PHPUnit or in Composer. I've created a tests/bootstrap.php file (the directory tests contain my tests) with this content, I can force register the classes (it's not working still because I haven't added all classes).

<?php

/** @var \Composer\Autoload\ClassLoader $classLoader */
$classLoader = require 'vendor/autoload.php';

$classLoader->addClassMap([
    \PHPUnit\Metadata\Metadata::class             => __DIR__ . '/../vendor/phpunit/phpunit/src/Metadata/Metadata.php',
    \PHPUnit\Metadata\Api\DataProvider::class     => __DIR__ . '/../vendor/phpunit/phpunit/src/Metadata/Api/DataProvider.php',
    \PHPUnit\Metadata\Parser\Registry::class      => __DIR__ . '/../vendor/phpunit/phpunit/src/Metadata/Parser/Registry.php',
    \PHPUnit\Metadata\Parser\CachingParser::class => __DIR__ . '/../vendor/phpunit/phpunit/src/Metadata/Parser/CachingParser.php',
    \PHPUnit\Metadata\Parser\Parser::class        => __DIR__ . '/../vendor/phpunit/phpunit/src/Metadata/Parser/Parser.php',
    \PHPUnit\Metadata\Parser\ParserChain::class   => __DIR__ . '/../vendor/phpunit/phpunit/src/Metadata/Parser/ParserChain.php',
]);

$classLoader->register();

But as the error now shows other classes that are not found in that directory, I started looking into Composer issues and found this:

composer/composer#8808 (comment)

Stating that there is a bug in WSL2 where the RecursiveDirectoryIterator returns an incomplete list... which, according to PHP, is a WSL2 bug: microsoft/WSL#5074

So this is not a PHPUnit bug, and for now there is no clear/easy solution to apply

[PAXANDDOS]

@Echron Thanks for the investigation! You definitely have shed light on this. As I can see on the WSL issue, it is mostly breaking PHP applications, which is a pity.

And I guess the workaround for this issue would be.. run tests when the directory is not volumed.

[jsanahuja]

I had the same issue, updating composer from version 2.1.6 to 2.6.5 and running again composer dump-autoload -o solved it