-
Notifications
You must be signed in to change notification settings - Fork 315
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rollback back-end *node* engine support to LTS #1628
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Keeping applicable TLS restricted for eventual 12.x+ compatibility * Search query *(modelQuery)* on, at the very least, `about` field causes maximum CPU usage in *node*@12.0.0 through *node*@12.5.0 but not v11.x or v10.x. This causes severe lag for everyone up to observed sustained 20+ seconds on queries. * Restarting and exposing with transforming to secondary indexing for Script model as per all [*mongoose*/MongoDB documentation](https://mongoosejs.com/docs/guide.html#indexes). Use `syncIndexes` to maintain DB indexes. * Squash new deprecation warning for OpenUserJS#1516 NOTES: * This is going to be a major issue when v10.x is EOL'd if *node* and related dep issues *(mongoose, mongodb)* aren't resolved * For some reason *mongoose* is emitting the `index` event twice when used globally... unknown reason atm. Picking local disable in favor. Applies to OpenUserJS#1548 and post OpenUserJS#1603 with observed issue since then. Refs: * https://mongoosejs.com/docs/guide.html#indexes Dev startup: ``` console $ node app.js Starting application... Disabling GitHub `hooks` in unsecure mode S3rver initialized error: Error creating bucket. Bucket "openuserjs.org" already exists info: PUT /openuserjs.org 409 11ms - Default dev S3 bucket already exists MongoDB connection is opened Connected to MongoDB v3.6.12 GitHub client authenticated Index event triggered/trapped for Script model Script indexes: [ { v: 2, key: { _id: 1 }, name: '_id_', ns: 'openuserjs_devel.scripts' }, { v: 2, key: { _authorId: 1, flagged: 1, isLib: 1 }, name: '_authorId_1_flagged_1_isLib_1', ns: 'openuserjs_devel.scripts', background: true }, { v: 2, key: { installName: 1 }, name: 'installName_1', ns: 'openuserjs_devel.scripts', background: true }, { v: 2, key: { isLib: 1, author: 1, name: 1 }, name: 'isLib_1_author_1_name_1', ns: 'openuserjs_devel.scripts', background: true } ] ``` Recent db stats: ``` console > db.scripts.aggregate( [ { $indexStats: { } } ] ); { "name" : "_id_", "key" : { "_id" : 1 }, "host" : "<db>:<port>", "accesses" : { "ops" : NumberLong(12842), "since" : ISODate("2019-06-27T04:10:25.453Z") } } { "name" : "isLib_1_author_1_name_1", "key" : { "isLib" : 1, "author" : 1, "name" : 1 }, "host" : "<db>:<port>", "accesses" : { "ops" : NumberLong(7348), "since" : ISODate("2019-06-27T12:26:13.574Z") } } { "name" : "installName_1", "key" : { "installName" : 1 }, "host" : "<db>:<port>", "accesses" : { "ops" : NumberLong(144052), "since" : ISODate("2019-06-27T04:10:25.453Z") } } { "name" : "_authorId_1_flagged_1_isLib_1", "key" : { "_authorId" : 1, "flagged" : 1, "isLib" : 1 }, "host" : "<db>:<port>", "accesses" : { "ops" : NumberLong(16691), "since" : ISODate("2019-06-27T04:10:25.453Z") } } ```
Martii
added
tracking upstream
Waiting, watching, wanting.
migration
Use this to indicate that it may apply to an existing or announced migration.
CODE
Some other Code related issue and it should clearly describe what it is affecting in a comment.
needs mitigation
Needs additional followup.
labels
Jun 28, 2019
Martii
added
needs testing
Anyone can add this but it is primarily there for the Assignee indicating that Testers are wanted.
security
Usually relates to something critical.
labels
Jun 28, 2019
Guhhh still present with multiple queries outside of myself... ughh... plan B |
Martii
added a commit
to Martii/OpenUserJS.org
that referenced
this pull request
Jun 28, 2019
* Limit a few more things to possibly curtail some 431's * Ease up on MongoDB having issues with script querying and CPU load. Limit to first copied block of characters in `about`. This amount may get smaller and is currently in flux. * Create new clipped searchable fields. I spent a lot of time to see if MongoDB/*mongoose* had a limiter on their indexer and couldn't find one so we get to do it manually. NOTE: * This will temporarily remove everyone's Summary but it will be manually migrated in the DB in a while. If you are impatient... edit your script *(no changes necessary)* and resave it... that will do the migration also. Post OpenUserJS#1628 and applies to OpenUserJS#1548
Merged
Martii
added a commit
that referenced
this pull request
Jun 28, 2019
* Limit a few more things to possibly curtail some 431's * Ease up on MongoDB having issues with script querying and CPU load. Limit to first copied block of characters in `about`. This amount may get smaller and is currently in flux. * Create new clipped searchable fields. I spent a lot of time to see if MongoDB/*mongoose* had a limiter on their indexer and couldn't find one so we get to do it manually. NOTE: * This will temporarily remove everyone's Summary but it will be manually migrated in the DB in a while. If you are impatient... edit your script *(no changes necessary)* and resave it... that will do the migration also. Post #1628 and applies to #1548 Auto-merge
Martii
added a commit
to Martii/OpenUserJS.org
that referenced
this pull request
Dec 4, 2019
* Please read their CHANGELOGs * Delete op retested * Post OpenUserJS#1628 partial rollback now that v12.x is LTS and no longer the issue in OpenUserJS#1548
Merged
Martii
added a commit
that referenced
this pull request
Dec 4, 2019
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
CODE
Some other Code related issue and it should clearly describe what it is affecting in a comment.
DB
Pertains inclusively to the Database operations.
migration
Use this to indicate that it may apply to an existing or announced migration.
needs mitigation
Needs additional followup.
needs testing
Anyone can add this but it is primarily there for the Assignee indicating that Testers are wanted.
security
Usually relates to something critical.
tracking upstream
Waiting, watching, wanting.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
about
field causes maximum CPU usage in node@12.0.0 through node@12.5.0 but not v11.x or v10.x. This causes severe lag for everyone up to observed sustained 20+ seconds on queries.syncIndexes
to maintain DB indexes.NOTES:
index
event twice when used globally... unknown reason atm. Picking local disable in favor.Applies to #1548 and post #1603 with observed issue since then.
Refs:
Dev startup:
Recent db stats: