Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Parser and Statement #540

Merged
merged 2 commits into from
Jan 20, 2024
Merged

Refactor Parser and Statement #540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f1dc58d
Split up Parser::KEYWORD_PARSERS
kamil-tekiela Jan 16, 2024
fbaf831
Improve documentation of getClauses()
kamil-tekiela Jan 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions phpstan-baseline.neon
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -540,16 +540,6 @@ parameters:
count: 1
path: src/Statement.php

-
message: "#^Offset 'class' does not exist on array\\{\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Array2d', field\\: 'values'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ArrayObj', field\\: 'partition'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Condition', field\\: 'having'\\|'where'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Expression', field\\: 'table', options\\: array\\{parseField\\: 'table'\\}\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ExpressionArray', field\\: 'expr'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ExpressionArray', field\\: 'fields'\\|'tables', options\\: array\\{parseField\\: 'table'\\}\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ExpressionArray', field\\: 'from', options\\: array\\{field\\: 'table'\\}\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\FunctionCall', field\\: 'call'\\|'procedure'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\GroupKeyword', field\\: 'group'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\IndexHint', field\\: 'index_hints'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\IntoKeyword', field\\: 'into'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\JoinKeyword', field\\: 'join'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Limit', field\\: 'limit'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\OptionsArray', field\\: 'endOptions'\\|'groupOptions'\\|'options'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\OrderKeyword', field\\: 'order'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\RenameOperation', field\\: 'renames'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\SetOperation', field\\: 'set'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\UnionKeyword', field\\: 'union'\\}\\.$#"
count: 1
path: src/Statement.php

-
message: "#^Offset 'field' does not exist on array\\{\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Array2d', field\\: 'values'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ArrayObj', field\\: 'partition'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Condition', field\\: 'having'\\|'where'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Expression', field\\: 'table', options\\: array\\{parseField\\: 'table'\\}\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ExpressionArray', field\\: 'expr'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ExpressionArray', field\\: 'fields'\\|'tables', options\\: array\\{parseField\\: 'table'\\}\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\ExpressionArray', field\\: 'from', options\\: array\\{field\\: 'table'\\}\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\FunctionCall', field\\: 'call'\\|'procedure'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\GroupKeyword', field\\: 'group'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\IndexHint', field\\: 'index_hints'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\IntoKeyword', field\\: 'into'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\JoinKeyword', field\\: 'join'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\Limit', field\\: 'limit'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\OptionsArray', field\\: 'endOptions'\\|'groupOptions'\\|'options'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\OrderKeyword', field\\: 'order'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\RenameOperation', field\\: 'renames'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\SetOperation', field\\: 'set'\\}\\|array\\{class\\: 'PhpMyAdmin\\\\\\\\SqlParser\\\\\\\\Components\\\\\\\\UnionKeyword', field\\: 'union'\\}\\.$#"
count: 1
path: src/Statement.php

-
message: "#^Property PhpMyAdmin\\\\SqlParser\\\\Statement\\:\\:\\$options \\(PhpMyAdmin\\\\SqlParser\\\\Components\\\\OptionsArray\\|null\\) does not accept mixed\\.$#"
count: 1
Expand Down
6 changes: 0 additions & 6 deletions psalm-baseline.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -797,12 +797,6 @@
<MixedOperand>
<code><![CDATA[$this->$field->build()]]></code>
</MixedOperand>
<PossiblyUndefinedArrayOffset>
<code><![CDATA[Parser::KEYWORD_PARSERS[$name]['class']]]></code>
<code><![CDATA[Parser::KEYWORD_PARSERS[$name]['field']]]></code>
<code><![CDATA[Parser::KEYWORD_PARSERS[$tokenValue]['class']]]></code>
<code><![CDATA[Parser::KEYWORD_PARSERS[$tokenValue]['field']]]></code>
</PossiblyUndefinedArrayOffset>
<PossiblyUnusedReturnValue>
<code>bool</code>
</PossiblyUnusedReturnValue>
Expand Down
5 changes: 0 additions & 5 deletions src/Parser.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,11 +115,6 @@
* Array of classes that are used in parsing SQL components.
*/
public const KEYWORD_PARSERS = [
// This is not a proper keyword and was added here to help the
// formatter.
'PARTITION BY' => [],
'SUBPARTITION BY' => [],

// This is not a proper keyword and was added here to help the
// builder.
'_OPTIONS' => [
Expand Down Expand Up @@ -375,14 +370,14 @@
*/
public function __construct(string|UtfString|TokensList|null $list = null, bool $strict = false)
{
if (Context::$keywords === []) {

Check warning on line 373 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "Identical":

--- Original
+++ New
@@ @@
      */
     public function __construct(string|UtfString|TokensList|null $list = null, bool $strict = false)
     {
-        if (Context::$keywords === []) {
+        if (Context::$keywords !== []) {
             Context::load();
         }
         if (is_string($list) || $list instanceof UtfString) {
Context::load();
}

if (is_string($list) || ($list instanceof UtfString)) {

Check warning on line 377 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "InstanceOf_":

--- Original
+++ New
@@ @@
         if (Context::$keywords === []) {
             Context::load();
         }
-        if (is_string($list) || $list instanceof UtfString) {
+        if (is_string($list) || false) {
             $lexer = new Lexer($list, $strict);
             $this->list = $lexer->list;
         } elseif ($list instanceof TokensList) {
$lexer = new Lexer($list, $strict);
$this->list = $lexer->list;
} elseif ($list instanceof TokensList) {

Check warning on line 380 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "InstanceOf_":

--- Original
+++ New
@@ @@
         if (is_string($list) || $list instanceof UtfString) {
             $lexer = new Lexer($list, $strict);
             $this->list = $lexer->list;
-        } elseif ($list instanceof TokensList) {
+        } elseif (true) {
             $this->list = $list;
         }
         $this->strict = $strict;
$this->list = $list;
}

Expand Down Expand Up @@ -435,7 +430,7 @@

// `DELIMITER` is not an actual statement and it requires
// special handling.
if (($token->type === TokenType::None) && (strtoupper($token->token) === 'DELIMITER')) {

Check warning on line 433 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "UnwrapStrToUpper":

--- Original
+++ New
@@ @@
             $token = $list->tokens[$list->idx];
             // `DELIMITER` is not an actual statement and it requires
             // special handling.
-            if ($token->type === TokenType::None && strtoupper($token->token) === 'DELIMITER') {
+            if ($token->type === TokenType::None && $token->token === 'DELIMITER') {
                 // Skipping to the end of this statement.
                 $list->getNextOfType(TokenType::Delimiter);
                 $prevLastIdx = $list->idx;
// Skipping to the end of this statement.
$list->getNextOfType(TokenType::Delimiter);
$prevLastIdx = $list->idx;
Expand Down Expand Up @@ -532,8 +527,8 @@
// Handles unions.
if (
! empty($unionType)
&& ($lastStatement instanceof SelectStatement)

Check warning on line 530 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "InstanceOf_":

--- Original
+++ New
@@ @@
             $statement->last = $list->idx;
             $prevLastIdx = $list->idx;
             // Handles unions.
-            if (!empty($unionType) && $lastStatement instanceof SelectStatement && $statement instanceof SelectStatement) {
+            if (!empty($unionType) && true && $statement instanceof SelectStatement) {
                 /*
                  * This SELECT statement.
                  *
&& ($statement instanceof SelectStatement)

Check warning on line 531 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "InstanceOf_":

--- Original
+++ New
@@ @@
             $statement->last = $list->idx;
             $prevLastIdx = $list->idx;
             // Handles unions.
-            if (!empty($unionType) && $lastStatement instanceof SelectStatement && $statement instanceof SelectStatement) {
+            if (!empty($unionType) && $lastStatement instanceof SelectStatement && true) {
                 /*
                  * This SELECT statement.
                  *
) {
/*
* This SELECT statement.
Expand Down Expand Up @@ -565,7 +560,7 @@
$unionType = false;

// Validate clause order
$statement->validateClauseOrder($this, $list);

Check warning on line 563 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "MethodCallRemoval":

--- Original
+++ New
@@ @@
                 // union ends.
                 $lastStatement->last = $statement->last;
                 $unionType = false;
-                // Validate clause order
-                $statement->validateClauseOrder($this, $list);
+                
                 continue;
             }
             // Handles transactions.
continue;
}

Expand All @@ -591,7 +586,7 @@
}

// Validate clause order
$statement->validateClauseOrder($this, $list);

Check warning on line 589 in src/Parser.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "MethodCallRemoval":

--- Original
+++ New
@@ @@
                     }
                     $lastTransaction = null;
                 }
-                // Validate clause order
-                $statement->validateClauseOrder($this, $list);
+                
                 continue;
             }
             // Validate clause order
continue;
}

Expand Down
51 changes: 12 additions & 39 deletions src/Statement.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,20 +48,15 @@
*/
public static array $statementOptions = [];

protected const ADD_CLAUSE = 1;
protected const ADD_KEYWORD = 2;

/**
* The clauses of this statement, in order.
*
* The value attributed to each clause is used by the builder and it may
* have one of the following values:
*
* - 1 = 01 - add the clause only
* - 2 = 10 - add the keyword
* - 3 = 11 - add both the keyword and the clause
*
* @var array<string, array<int, int|string>>
* @psalm-var array<string, array{non-empty-string, (1|2|3)}>
* @var array<string, array{non-empty-string, int-mask-of<self::ADD_*>}>
*/
public static array $clauses = [];

Check failure on line 59 in src/Statement.php

View workflow job for this annotation

GitHub Actions / lint-docs 

The hint on "clauses" at @var is invalid: "array<string, array{non-empty-string, int-mask-of<self::ADD_*>}>" on "PhpMyAdmin\SqlParser\Statement::$clauses"

/**
* The options of this query.
Expand All @@ -86,7 +81,7 @@
*/
public function __construct(Parser|null $parser = null, TokensList|null $list = null)
{
if (($parser === null) || ($list === null)) {

Check warning on line 84 in src/Statement.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "LogicalOr":

--- Original
+++ New
@@ @@
      */
     public function __construct(Parser|null $parser = null, TokensList|null $list = null)
     {
-        if ($parser === null || $list === null) {
+        if ($parser === null && $list === null) {
             return;
         }
         $this->parse($parser, $list);
return;
}

Expand Down Expand Up @@ -115,29 +110,7 @@
*/
$built = [];

/**
* Statement's clauses.
*/
$clauses = $this->getClauses();

foreach ($clauses as $clause) {
/**
* The name of the clause.
*/
$name = $clause[0];

/**
* The type of the clause.
*
* @see Statement::$clauses
*/
$type = $clause[1];

/**
* The builder (parser) of this clause.
*/
$class = Parser::KEYWORD_PARSERS[$name]['class'];

foreach ($this->getClauses() as [$name, $type]) {
/**
* The name of the field that is used as source for the builder.
* Same field is used to store the result of parsing.
Expand All @@ -150,7 +123,7 @@
}

// Checking if this field was already built.
if ($type & 1) {
if ($type & self::ADD_CLAUSE) {
if (! empty($built[$field])) {
continue;
}
Expand All @@ -159,16 +132,17 @@
}

// Checking if the name of the clause should be added.
if ($type & 2) {
if ($type & self::ADD_KEYWORD) {
$query = trim($query) . ' ' . $name;

Check warning on line 136 in src/Statement.php

View workflow job for this annotation

GitHub Actions / Mutation tests with PHP 8.1 

Escaped Mutant for Mutator "UnwrapTrim":

--- Original
+++ New
@@ @@
             }
             // Checking if the name of the clause should be added.
             if ($type & self::ADD_KEYWORD) {
-                $query = trim($query) . ' ' . $name;
+                $query = $query . ' ' . $name;
             }
             // Checking if the result of the builder should be added.
             if (!($type & self::ADD_CLAUSE)) {
}

// Checking if the result of the builder should be added.
if (! ($type & 1)) {
if (! ($type & self::ADD_CLAUSE)) {
continue;
}

if (is_array($this->$field)) {
$class = Parser::KEYWORD_PARSERS[$name]['class'];
$query = trim($query) . ' ' . $class::buildAll($this->$field);
} else {
$query = trim($query) . ' ' . $this->$field->build();
Expand Down Expand Up @@ -286,7 +260,7 @@
$options = [];

// Looking for duplicated clauses.
if (! empty(Parser::KEYWORD_PARSERS[$token->value]) || ! empty(Parser::STATEMENT_PARSERS[$token->value])) {
if (isset(Parser::KEYWORD_PARSERS[$token->value]) || ! empty(Parser::STATEMENT_PARSERS[$token->value])) {
if (! empty($parsedClauses[$token->value])) {
$parser->error('This type of clause was previously parsed.', $token);
break;
Expand All @@ -300,7 +274,7 @@
// but it might be the beginning of a statement of truncate,
// so let the value use the keyword field for truncate type.
$tokenValue = in_array($token->keyword, ['TRUNCATE']) ? $token->keyword : $token->value;
if (! empty(Parser::KEYWORD_PARSERS[$tokenValue]) && $list->idx < $list->count) {
if (isset(Parser::KEYWORD_PARSERS[$tokenValue]) && $list->idx < $list->count) {
$class = Parser::KEYWORD_PARSERS[$tokenValue]['class'];
$field = Parser::KEYWORD_PARSERS[$tokenValue]['field'];
if (! empty(Parser::KEYWORD_PARSERS[$tokenValue]['options'])) {
Expand Down Expand Up @@ -419,10 +393,9 @@
/**
* Gets the clauses of this statement.
*
* @return array<string, array<int, int|string>>
* @psalm-return array<string, array{non-empty-string, (1|2|3)}>
* @return array<string, array{non-empty-string, int-mask-of<Statement::ADD_*>}>
*/
public function getClauses(): array

Check failure on line 398 in src/Statement.php

View workflow job for this annotation

GitHub Actions / lint-docs 

The hint on "getClauses" at @return is invalid: "array<string, array{non-empty-string, int-mask-of<Statement::ADD_*>}>" on "PhpMyAdmin\SqlParser\Statement::getClauses"
{
return static::$clauses;
}
Expand Down
19 changes: 9 additions & 10 deletions src/Statements/DeleteStatement.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,42 +62,41 @@
*
* @see Statement::$clauses
*
* @var array<string, array<int, int|string>>
* @psalm-var array<string, array{non-empty-string, (1|2|3)}>
* @var array<string, array{non-empty-string, int-mask-of<self::ADD_*>}>
*/
public static array $clauses = [

Check failure on line 67 in src/Statements/DeleteStatement.php

View workflow job for this annotation

GitHub Actions / lint-docs 

The hint on "clauses" at @var is invalid: "array<string, array{non-empty-string, int-mask-of<self::ADD_*>}>" on "PhpMyAdmin\SqlParser\Statements\DeleteStatement::$clauses"
'DELETE' => [
'DELETE',
2,
Statement::ADD_KEYWORD,
],
// Used for options.
'_OPTIONS' => [
'_OPTIONS',
1,
Statement::ADD_CLAUSE,
],
'FROM' => [
'FROM',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
'PARTITION' => [
'PARTITION',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
'USING' => [
'USING',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
'WHERE' => [
'WHERE',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
'ORDER BY' => [
'ORDER BY',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
'LIMIT' => [
'LIMIT',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
];

Expand Down
11 changes: 5 additions & 6 deletions src/Statements/DropStatement.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,27 +42,26 @@
*
* @see Statement::$clauses
*
* @var array<string, array<int, int|string>>
* @psalm-var array<string, array{non-empty-string, (1|2|3)}>
* @var array<string, array{non-empty-string, int-mask-of<self::ADD_*>}>
*/
public static array $clauses = [

Check failure on line 47 in src/Statements/DropStatement.php

View workflow job for this annotation

GitHub Actions / lint-docs 

The hint on "clauses" at @var is invalid: "array<string, array{non-empty-string, int-mask-of<self::ADD_*>}>" on "PhpMyAdmin\SqlParser\Statements\DropStatement::$clauses"
'DROP' => [
'DROP',
2,
Statement::ADD_KEYWORD,
],
// Used for options.
'_OPTIONS' => [
'_OPTIONS',
1,
Statement::ADD_CLAUSE,
],
// Used for select expressions.
'DROP_' => [
'DROP',
1,
Statement::ADD_CLAUSE,
],
'ON' => [
'ON',
3,
Statement::ADD_CLAUSE | Statement::ADD_KEYWORD,
],
];

Expand Down
Loading
Loading