Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when store a big data to cache #1268

Closed
hunteryun opened this issue Mar 7, 2022 · 10 comments · Fixed by #1364
Closed

Error when store a big data to cache #1268

hunteryun opened this issue Mar 7, 2022 · 10 comments · Fixed by #1364

Comments

@hunteryun
Copy link

hunteryun commented Mar 7, 2022

image

image

sorry, I just want try to report this issue.

@colorfield
Copy link

Can be reproduced when the schema definition reaches a certain size. Indeed parse https://github.com/drupal-graphql/graphql/blob/8.x-4.x/src/Plugin/GraphQL/Schema/SdlSchemaPluginBase.php#L177 will produce a DocumentNode that is set properly in the cache table, then $this->astCache->get($cid) will trigger

Warning: unserialize(): Maximum depth of 4096 exceeded. The depth limit can be changed using the max_depth unserialize() option or the unserialize_max_depth ini setting in Drupal\Core\Cache\DatabaseBackend->prepareItem() (line 167 of core/lib/Drupal/Core/Cache/DatabaseBackend.php).
Notice: unserialize(): Error at offset 731605 of 2667590 bytes in Drupal\Core\Cache\DatabaseBackend->prepareItem() (line 167 of core/lib/Drupal/Core/Cache/DatabaseBackend.php).

Using php 7.4.

@rigoucr
Copy link

rigoucr commented Mar 24, 2022

Hi , I'm also having the same issue , and yes, serialize functions reach their limit because of the complexity of the DocumentNode object , also I confirmed that the solution proposed by @hunteryun makes sense (at least to me), because later in the code execution flow the data the is coming from the Drupal caches is being parse by the build function , so it doesn't breaks anything , my only concern here is that the parser aside from building the DocumentNode objects it runs Syntax validations over the schema to be saved into the cache, so I think the parsing should be kept Just for validation, to avoid having wrong schemas saved into cache. Currently I don't know if the webonyx/raphql-php provides some functions to validate the schema without generating the DocumentNode object, but I think that It could be the ideal solution, use some sort of syntax validation function and then save the string as @hunteryun proposed .

@Leksat
Copy link
Contributor

Leksat commented Apr 28, 2022

I also got this when worked on a project.

I reverted the schema to its previous state, so it was big (1100+ lines), but it was still serializable. Then I did some measures comparing the schema-parse time versus the cache-get time.
Leksat@8273f63
Results:

cache: 0.023980140686035
parse: 0.026409149169922
parse/cache: 1.1012925034798

And it looks like there is almost no difference. At least in my case (PHP 7.4, SQLite).

With xdebug isn't that good

xdebug enabled, its port is not listened:

cache: 0.020208120346069
parse: 0.1002140045166
parse/cache: 4.9590957892377

xdebug enabled, phpstorm is listening to its port:

cache: 0.020011186599731
parse: 0.43809795379639
parse/cache: 21.892652472806

In the end I used a workaround to disable the schema caching by adding $this->inDevelopment = TRUE; to the getResolverRegistry method of my schema plugin.

But after all, does it make sense to cache the parsed schema if this brings almost zero benefit?

@darvanen
Copy link
Contributor

darvanen commented Jun 8, 2022

Perhaps we could just cache the raw text schema so it doesn't have to be fetched from files each time and parse that for each request?

@darvanen
Copy link
Contributor

darvanen commented Jun 8, 2022

The problem is that \GraphQL\Language\Token has self-references and the serializer can't deal with the endless tree that creates:

image

The documentation for seralize() specfically states that some things just aren't serlializable, so I propose we don't try to do it here.

@darvanen
Copy link
Contributor

darvanen commented Jun 9, 2022

Issue filed with webonyx/graphql:
webonyx/graphql-php#1164

@Leksat
Copy link
Contributor

Leksat commented Sep 23, 2023

TLDR: Can we use noLocation option? If yes, #1364 is ready.


My journey here 🙃

When I first met unserialize(): Maximum depth of 4096 exceeded, I used inDevelopment = TRUE trick. It worked.

Then I got another project with a bigger schema. I tried inDevelopment = TRUE, but GraphQL calls became extremely slow. So I went with ini_set('unserialize_max_depth', '8192'). It worked.

Now I'm on a project with an even bigger schema. The parsed AST can't be serialized, the serialize function fails with Segmentation fault (only locally, not in CI). I tried #1280. It worked.

However, the GraphQL performance dropped. GraphQL calls went almost two times slower.

Finally, I tried noLocation option suggested in webonyx/graphql-php#1164 (comment). It worked just fine! I tried it on multiple projects mentioned above. No segfaults, no maxdepth issues. There was even a tiny performance improvement.

I was expecting that this option will do some harm, like error messages will have no location information. Yet the error messages look just as before. So probably it's safe to use noLocation. Is it?

@pmelab
Copy link
Contributor

pmelab commented Sep 23, 2023

Maybe we could bind that to inDevelopment? Code locations might help at some point during development, but I think its safe to assume they are not required in production.

@Leksat
Copy link
Contributor

Leksat commented Sep 24, 2023

We have tests for errors

$this->assertSame([
'errors' => [
[
'message' => 'Internal server error',
'extensions' => [
'category' => 'internal',
],
'locations' => [
[
'line' => 1,
'column' => 9,
],
],
'path' => [
'throwsException',
],
],
],
], json_decode($result->getContent(), TRUE));

They still work with noLocation.

@klausi
Copy link
Contributor

klausi commented Nov 11, 2023

Merged the PR, thanks a lot! I will push a small docs follow-up to document the why in code, as this noLocation option is not obvious.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants