Better infinite loop inference, some PHP 8.1 support and a bunch of fixes

Added

• (#6398) array_is_list() signature (@orklah)
• (#6386, #6392) Psalm now understands that while(true) is an infinite loop (@orklah)
• (#6403, #6421) PHP 8.1 signatures for ftp functions (@weirdan)

Fixes

• (#6305, #6310) SimpleXMLIterator signatures no longer depend on runtime PHP version (@weirdan)
• (#6309, #6311) Some assertions generated by in_array() were causing crashes (@weirdan)
• (#6312) PhpToken::getAll() was renamed to PhpToken::tokenize() (@TysonAndre)
• (#6312) fdiv() argument names were corrected to match names used by PHP (@TysonAndre)
• (#6312) ReflectionClass::getAttributes(), ReflectionClassConstant::getAttributes(), ReflectionFunctionAbstract::getAttributes(), ReflectionParameter::getAttributes(), ReflectionProperty::getAtributes(), ReflectionUnionType::getTypes() and ReflectionNamedType::isBuiltin() signatures were missing (@TysonAndre)
• (#6318, #6320) socket_select()'s tv_sec parameter is nullable (@Forceu)
• (#6323) Another case of crashes due to in_array() assertions (@boesing)
• (#6324) str_split() and mb_str_split() return require positive-int and str_split() returns non-empty array (@VincentLanglet)
• (#6328) Corrected message for calls from root namespace to internal methods in a different namespace (@bdsl)
• (#6325, #6327) Interface aliases were not recognized (@boesing)
• (#4738) is_subclass_of() requires a class-string for its parent parameter (@BenMorel)
• (#5356, #6344) UnusedSuppression was sometimes reported in ignored folders (@orklah)
• (#6343, #6346) @psalm-trace was reporting keys that were unset (@orklah)
• (#5096, #6321) Constants accessed through object instance (e.g. $obj::CONST) were not inferred (@orklah)
• (#6359) OuterIterator::getInnerIterator() always return Iterator (@drupol)
• (#6367, #6376) Variables used in backticks (`ls $file`) were reported as unused (@orklah)
• (#6358, #6375) openssl_x509_parse() and openssl_x509_read() signatures were outdated (@ThomasLandauer)
• (#6338, #6339) array_walk() and array_walk_recursive() accept object (and will trigger RawObjectIteration when given one) (@niconoe-, @weirdan)
• (#6330, #6335) Alias chains created with class_alias() (A -> B -> C) were not understood (@boesing)
• (#6387, #6390) Type params can now be extracted from Generator argument passed to Iterator parameter (@orklah)
• (#6388, #6391) empty() could not be forbidden via forbiddenFunctions (@ro0NL)
• (#6333, #6317) in_array() did not produce correct assertions with non-literal types (@TysonAndre)
• (#6384, #6420) Variables reassigned in both branches of ternary operator were not correctly inferred (@orklah)

Docs

• (#6409) Added a guide on how to edit callmaps (@weirdan)
• (#6400) Added a section on taint analysis limitations (@craigfrancis)

Just a bunch of fixes + callmap tests

Fixes

• (#6211, #6215) Class strings used in attributes were not properly qualified (@weirdan)
• (#6217, #6219) mb_str_split() signature was missing (@bitwise-operators)
• (#6231) Fixed language server crash on PHP 8 (@tm1000)
• (#6234, #6238) UnusedForeachValue was incorrectly flagged as fixable (@orklah)
• (#6234, #6244) date() now accepts null as timestamp when targeting PHP 8 (@masonmcelvain)
• (#3573, #6242) UnnecessaryVarAnnotation issuea are now suppressable (@orklah)
• (#6228, #6281, #6229) Prevented remote schema file download during config validation (@gmessier)
• (#4952, #6251) Psalm was outputting (incorrect) cached errors when error level was overridden with a cli parameter (@TysonAndre)
• (#6257, #6260) Psalm incorrectly assumed imagejpeg() to return string when path was specified as null (@MauricioFauth)
• (psalm/phar#9, #6267) psalm/phar releases were not created automatically (@weirdan)
• (#6261, #6272) Result of a sum of arrays was not inferred correctly, leading to possibly mixed keys (@orklah)
• (#5501, #6253) Some suppressions were incorrectly reported as unused (@orklah)
• (#6266, #6273) PHP_VERSION, DIRECTORY_SEPARATOR, PATH_SEPARATOR and PHP_EOL are now always considered non-empty (@VincentLanglet)
• (#6263, #6290) Psalm was incorrectly inferring result of splats in constant arrays (@weirdan)
• (#6289, #6290) Psalm was incorrectly inferring values for integer array keys in constant arrays (@weirdan)
• (#6220, #6233) in_array() now informs needle type based on haystack type (@supersmile2009)
• (#6268) Improved debug_backtrace() return type (@vudaltsov)
• (#6301, #6302) Phar-packaged language server did not analyze code on change (@thomasbley)
• (#3660, #6270) Psalm sometimes reported type coverage exceeding 100% (@muglug)

Internal changes

• (#6240) Literal arithmetic refactoring (@orklah)
• (#6252) Literal float methods for Union (@boesing)
• (#6247) Tests for callmap and deltas (@bitwise-operators)
• (#6279) Constant array inference refactoring (@weirdan)

Typos

• (#6248, #6291) @TysonAndre and @zonuexe fixed some typos

Improve preloading and fix template bounds checks

Fixes an issue introduced in 4.9.0: #6212

Also improves Psalm's knowledge when some stub files are loaded (#5126, #5626)

Fixed TIntRange bounds types

TIntRange changes

The types for int range bounds were sub-optimal. This release fixes this. Formally this is a BC break, for plugins. However, given the short time that passed since TIntRange introduction we believe it should be OK to release it as a patch release.

Fixes

• (#6208) Fixed bounds types for TIntRange (@orklah)

First community release

Features

• (#5974, #5980) Ability to turn off inferPropertyTypesFromConstructor (thanks @weirdan)
• (#6008, #6010) LSP server now runs analysis on open and save (thanks @tm1000)
• (#6013, #6014, #6055) LSP server now understands didChangeWatchedFiles notification (thanks @tm1000)
• (#6058, #6059) Support for xml:base attribute (useful for modular configs) (thanks @turnabout)
• (#6060) FunctionReturnTypeProviderEvent and MethodReturnTypeProviderEvent now provide access to the statement via getStmt() method (thanks @VincentLanglet)
• (#6106) Added support for interface-string<FooInterface> (currently as an alias to class-string<FooInterface>) (thanks @muglug)
• (#5512) Added option to enable Shepherd reporting with environment variables (PSALM_SHEPHERD and PSALM_SHEPHERD_HOST) (thanks @ngmy)
• (#6117, #6130) Psalm now flags static access to non-static property (and vice versa) (thanks @weirdan)
• (#6155) Added support for PHP 8.1 native readonly flag on properties (thanks @muglug)
• (#6110, #6142) Now you may configure trigger_error() behavior (thanks @orklah)
• (#6152, #6181) Psalm now understands that the only descendants of DateTimeInterface are DateTime and DateTimeImmutable (thanks @Jack97)
• (#6207) Preliminary support for int ranges (e.g. int<min, 25>, int<2, 8> etc) (thanks @orklah)

Fixes

• (#5981, #5992) Marked more built-in functions as impure (thanks @samsonasik)
• (#5964) Fixed RdKafka\ProducerTopic::producev() (thanks @danog)
• (#5975, #5989) Values used in throw are no longer considered unused (thanks @weirdan)
• (#5979, #6000) Fixed newline-related issues when using baseline cross-platform (Windows/everything else) (thanks @bdsl)
• (#5997, #6007) Fixed errors during LSP shutdown sequence (thanks @tm1000)
• (#6011, #6012) Fixed psalm-language-server crash when invoked with --version argument (thanks @tm1000)
• (#6005, #6019) Correctly resolve self and static in callable arrays passed to array_reduce() (thanks @elnoro)
• (#3287, #6036) Fixed mbstring functions signatures (they changed in PHP 8) (thanks @mpesari)
• (#6038, #6039) Generic part of the array shape is no longer erased by array_map() (thanks @weirdan)
• (#6027, #6041) Fixed inferred value for __FUNCTION__ constant used in methods (thanks @orklah)
• (#6040) DateTime::format() and DateTimeImmutable::format() return values on PHP 8 (thanks @j4nr6n)
• (#6062) Do not interpret psalm.xml in --config psalm.xml as a file to check (thanks @simPod)
• (#4663, #5378, #6063, #6064) Fixed parsing of negative int literals (thanks @ElisDN)
• (#6050, #6054) Fixed crash due to internal structures getting out of sync (thanks @caugner)
• (#6016, #6069) Improve str_word_count() signature (thanks @elnoro)
• (#5584) Added ReflectionProperty::hasType() and ReflectionProperty::getType() stubs (thanks @franmomu)
• (#5971, #6068) Fixed plugin crash due to missing ClassLikeStorage (thanks @caugner)
• (#5608) Ignore string keys in callable arrays passed to preg_replace_callback() (thanks @paxal)
• (#6066, #6072) Tightened constraints on generic type parameter inference that might lead to runtime errors (thanks @muglug)
• (#6061, #6081) Analyze array keys before values (thanks @muglug)
• (#6080, #6082) join() signature is synced to implode() (thanks @weirdan)
• (#5790, #6083) Emit InvalidArgument instead of PossiblyFalseArgument when the argument is certainly false (thanks @orklah)
• (#5957, #6084) Fixed false-positive UnusedVariable issues on values used in bitwise OR and NOT (thanks @elnoro)
• (#6088, #6089) Fixed crash on closed resource used in switch (thanks @orklah)
• (#6049, #6094) Fixed inference of non-empty-string returned from str_pad() and str_repeat() (thanks @weirdan)
• (#6104) Fixed return type for strtoupper() (thanks @staabm)
• (#6116) Fixed return type for radius_send_request() (thanks @fkooman)
• (#2616, #6107) Fixed return type for get_defined_constants() (thanks @orklah)
• (#6119, #6121) Allowed zero for all bitmask types (thanks @weirdan)
• (#6122, #6123) Fixed path to be relative in CodeClimate reports (thanks @caugner)
• (#6118, #6127) Deprecated properties are now reported even when they are read with $this->prop (thanks @weirdan)
• (#3421) Fixed useless empty cache folder being created (thanks @JoshuaBehrens)
• (#3903) Now Psalm emits RedundantIdentityWithTrue for !== false as it did for === true before (thanks @greg0ire)
• (#6091, #6099) Fixed signatures for XmlWriter methods (thanks @aboks)
• (#6111, #6129) Prevented false-positive UnusedVariable issues on global vars (thanks @orklah)
• (#5736, #6131) bindtextdomain() is no longer considered pure (thanks @orklah)
• (#4344, #5663, #5639, #5955, #3272, #6133) Allow more types for magic properties defined with @property (thanks @weirdan)
• (#6146) Fixed XML report generation when issues contain taint trace (thanks @ndench)
• (#6145, #6150) Fixed false-positive UnusedVariable when value is used as operand of unary plus and minus (thanks @weirdan)
• (#6149) Fixed signatures of fputcsv(), SplFileObject::fputcsv() and SplTempFileObject::fputcsv() (thanks @weirdan)
• (#6168, #6169) Arithmetic operations on numeric types are now flagged in strictBinaryOperands mode (thanks @orklah)
• (#6160, #6170) Fixed array_slice() return type when used on generic parameters (thanks @Jack97)
• (#6166) Fixed false positives on incrementing a numeric-string (thanks @orklah)
• (#6172) Fixed crash on unbalanced parentheses in docblock types (thanks @Jack97)
• (#6167) Fixed result type of non-div arithmetic operations on numeric and int|float (thanks @orklah)
• (#6177, #6178) Fixed return type of shell_exec() (thanks @VincentLanglet)
• (#6128, #6187) Fixed docblock parameter being reported as nullable when it was actually the native type that was nullable (thanks @orklah)
• (#6136, #6188) Psalm no longer considers print(), exit(string) and die(string) pure (thanks @Jack97)
• (#6186, #6192) Fixed return type of array_replace() and array_replace_recursive() (thanks @orklah)
• (#6195, #6196) Fixed return type of exec() (thanks @weirdan)
• (#6197, #4871) Fixed false positive ArgumentTypeCoercion for class-strings passed as attribute constructor parameters (thanks @orklah)
• (#6199, #6206) Fixed return types for get_headers() and get_meta_tags() (thanks @Jack97)

Docs

• (#6038) Updated Sublime LSP instructions (thanks @DannyJJK)
• (#6047) Added example for asserting method returns (thanks @simonhammes)
• (#6198) Clarified that taint analysis is best run when Psalm detects no regular errors (thanks @mmcev106)

Typos

@antonioeatgoat and @orklah fixed some typos (#6033, #6185, #6205)

Fix baseline functionality

Release 4.8.0 introduced a bug in baseline generation. This release should fix it!

Detect more unused code

Features

New literal-string type

Inspired by the is_literal RFC we've added a new literal-string type.

The type will be most useful to annotate functions and methods that take SQL. In those methods you generally don't want any code that is not part of your app, e.g.

<?php

/** @param literal-string $sql */
function execute_sql(string $sql, array $params = []): void { }

$id = (string) ($_GET['id'] ?? '');

// passes type checks
execute_sql(
    'SELECT * FROM `foo` WHERE `id` = :id',
    [':id' => $id]
);

// fails
execute_sql(
    'SELECT * FROM `foo` WHERE `id` = "' .$id . '"'
);

Psalm’s taint analysis can also help detect this general class of issues, but a literal-string type allows the type-checker to provide guarantees much earlier in the process.

Read more in the documentation.

More specific taint analysis for unescaped quotes

htmlentities can be used to strip some harmful characters in strings, but not all of them (by default).

Psalm has a new issue TaintedTextWithQuotes to help detect strings that might not have HTML tags but can have harmful Javascript.

Detect unused return values

When running with the --find-unused-code flag, Psalm already detects a lot of unnecessary code, including unused public and private methods, unused properties and unused variables.

Now Psalm will also flag unused return values — where a function returns something, but nowhere that calls the given function actually uses the returned value.

See the documentation for UnusedReturnValue and PossiblyUnusedReturnValue for more information.

@var/@param mixup

@weirdan added a new issue that's emitted when using a @var docblock where @param is expected (#5845)

Plugins can declare custom scanners & analyzers

You've been able to set custom scanners and analyzers in your config, but now plugins can do this too — thanks @ohader (#5883)

Unused foreach values

@weirdan added a separate UnusedForeachVariable to unused variable detection to prevent Psalm incorrectly flagging unused foreach vars as fixable (#5932)

Bugfixes

• Taint analysis: @ohader added better tainting for var_dump and print_r output (#5827)
• Taint analysis: @ohader ensured more errors are emitted after successful taint error found (#5832)
• @weirdan ensured that errors are emitted for @internal calls when calling constructors (#5843)
• @orklah fixed Psalter to add literal return type output (#5844)
• @weirdan fixed Psalm signature for ReflectionClass::getConstants (#5847)
• @orklah fixed an issue where Psalter was using the wrong PHP version (#5855)
• @kesselb improved the signature of mb_convert_encoding (#5862)
• @franmomu improved Mongodb signatures (#5864)
• @klimick fixed some bugs with generic assertions (#5879, #5888)
• fixed a bunch of issues around the use of assignment operators, which are now treated more like assignments
• @weirdan restricted the types for spl_autoload functions (#5885)
• @weirdan fixed method return types on DomNode classes (#5895)
• @weirdan improved the signature for sscanf (#5901)
• @weirdan prevented a crash when class constant references a missing class (#5902)
• @christeredvartsen updated signatures for SessionHandlerInterface (#5904)
• fixed inference of non-terminating switch statements (#5911)
• @BafS fixed an uncaught TypeError with some problematic bit shifts (#5921)
• Prevented a crash when dealing with non-UTF-8 strings (#5945)
• @orklah allowed Psalter to fix RedundantCast (#5948)
• @orklah prevented an infinite loop when a class constant referenced itself (#5951)
• @danog fixed rdkafka signatures (#5950)

Deprecations

• @weirdan formally deprecated allowCoercionFromStringToClassConst (#5897)
• @weirdan formally deprecated legacy API hook interfaces (#5898)
• @weirdan deprecated some methods (#5956)

Bugfixes galore

Improved unused property detection

Psalm emits UnusedProperty and PossiblyUnusedProperty issues when properties are declared but never referenced. Previously Psalm would count a property assignment as a reference to a property, but this has now been fixed so a property is treated as unused unless it is explicitly read from (#5810). As before, UnusedProperty is emitted for private properties, while PossiblyUnusedProperty is emitted for public and protected properties.

More ParamNameMismatch strictness

When I originally implemented ParamNameMismatch I ignored methods with only one param, thinking that people would not call methods with a single param using named arguments, but in retrospect this was a little too liberal. @VincentLanglet has restricted this appropriately (#5732)

Other bugfixes

• Improved negation of complex conditionals containing assignments (#5685)
• combining scalar and empty-scalar should result in scalar (#5696)
• Inherit class-level suppressions when checking methods (#5687, #3357) - thanks @weirdan
• infer never types in more situations (#5777) — thanks @weirdan
• prevent notice for some invalid docblocks (#5709) - thanks @weirdan
• prefer @psalm-template over @phpstan-template (#5713) - thanks @weirdan
• add JSON schema link for SARIF output (#5718) - thanks @weirdan
• unpack can return false — thanks @robchett
• improved array_splice inference (#5738) — thanks @orklah
• combining never and empty should result in never(#5756)
• prevent OOM when using array_merge inside a loop (#4910)
• prevent crash when templates cannot be resolved (#5073)
• all opcache_* functions are impure (#5761)
• prevent OOM on very large conditionals (#5327)
• improved inference after a nullsafe method call has been made (#5771) - thanks @orklah
• prevent crash with class-string-map usage (#5434)
• improved inference of array_fill (#5770) — thanks @olsavmic
• fixed bug extending a mixed param (#5763) — thanks @orklah
• added link to GitHub Actions report format (#5759) — thanks @sjparkinson
• call methods with proper params (#5768)
• preserve type after redundant cast to int (#5773) — thanks @orklah
• improved return type of htmlspecialchars and strlen (#5785, #5793) — thanks @still-dreaming-1 and @orklah
• Psalm Refactor prevent trying to move synthetically-produced parts of the AST (#5780) — thanks @pawel-slowik
• narrow possible types when two variables are compared (#5774) — thanks @orklah
• Catch nullref inside __invoke arguments (#5809)
• allow ''| at the start of a union type in docblocks (#5814) — thanks @orklah
• add stubs for mongodb (#5813) — thanks @franmomu
• improve generated assertions for >= 0 (#5815) — thanks @orklah
• prevent incorrect templated array types (#5799)
• transform bad array offsets where necessary (#5817) — thanks @orklah

4.7.2: Always run legacy plugins first

• Includes a small change to run legacy plugins for a given function/method/class/property earlier than newer plugins
• @VincentLanglet fixed a bug where empty arrays were not permissible for some complex array types (#5677)

Fix potential templated static issue

Prevent type unsoundness in static generic

@gharlan pointed out that return new static can be unsound on generic classes (#5383).

This release fixes that unsoundness, emitting an UnsafeGenericInstantiation issue in some instances. The docs have more information.

Other bugfixes

• @weirdan fixed an issue with instanceof being flagged after a class_exists check (#5498)
• array_values and iterator_to_array should now respect nested templates (#5506)
• @weirdan added a prohibition for @template annotations on closures (#5499)
• @weirdan marked more functions as impure (#5524)
• Improved error message when calling isset(self::$some_property) when the property has a non-nullable type, with help from @weirdan (#5489)
• @vudaltsov marked flock and iterator_to_array as impure
• Ensure function and methods are marked as used inside throw expressions (#5540, #5545)
• Ensure nullsafe methods are assumed used (#5542)
• @AndrolGenhald improved analysis of concats (#5544)
• @weirdan improved the return type for get_object_vars() (#5557)
• @weirdan added a coercion warning when coercing object to object{foo: string} (#5566)
• @weirdan added support for arrays of objects in array_column (#5567)
• fix assigned-in-conditional edge-case (#5578, #5640)
• @AndrolGenhald added a lot of improvements to class constant resolution (#5591)
• @AndrolGenhald fixed method visibility false-negative (#5595)
• @orklah allowed numeric to be falsy (#5598)
• @AndrolGenhald improved int/float comparison for conditional types, allowing a fix for range types (#5601)
• @AndrolGenhald fixed a race condition with cache directory setup (#5603)
• @orklah added numeric - int = string|float calculations (#5611)
• @samsonasik marked json_last_error as impure
• @weirdan forbidded rejecting named arguments when the parent method supports them (#5627)
• @Jean85 added compatibility for xdebug handler 2.0
• @VincentLanglet added support for @phpstan-import-type as an alias for @psalm-import-type (#5648)
• @orklah added missing documentation for @no-named-arguments (#5654)
• Use more accurate type combination rules when converting templated union (#5652)
• Prevent crash when reporting issue on virtual arg (#5662)
• Use mixed type for $foo->$bar, preventing UnusedVariable in some situations (#5518)
• Ensure @var docblocks don’t override a variable's by-reference property (#5517)