Strange issue regarding code sync - allows scripts over the limit to be synced

[qsniyg]

I set up https://openuserjs.org/scripts/qsniyg/Image_Max_URL to have code sync with Github, and though I initially set it to sync from userscript_min.user.js (I created the script to import from Github, and selected that file), it appears to be syncing from userscript.user.js instead.

This isn't an issue for me, I only created userscript_min.user.js in order to fit within the 1MB limit (and even with pretty much the maximum compression without resorting to an obfuscation-like technique, it's still ~860KB), having it sync from an unminified version is definitely preferable. However, I believe this would theoretically allow any script to violate the site's limits (including, accidentally, my own), as well as the fact that it syncs from the wrong file, which I guess on a repository that contains multiple different userscripts, this could lead to issues.

When downloading the userscript now, it's ~3.3MB, which is well above the limit:

$ curl 'https://openuserjs.org/install/qsniyg/Image_Max_URL.user.js' | wc -c
3373819

[Martii]

OY! Long time boog. This section of code is going to need a massive rewrite at some point (see #1705). Got a patch forthcoming that ignores the push. As I've mentioned before the connection to GH only sends a response that we're going to process a list of scripts... so you'll still get a 200 response but it will silently ignore files that have too large of a byte count.

Thanks for the report.

Loosely related at #793

[qsniyg]

Thanks for replying!

Unfortunately though, this doesn't fix the main issue, which was that it synced from the wrong file (it synced from userscript.user.js, instead of the file I set it to sync to, which was userscript_min.user.js). Now it unfortunately just fails to sync altogether.

[Martii]

it synced from userscript.user.js

If I'm understanding that part correctly reimport your script using your min file. If that doesn't work then GH is doing something hinky with the messages that they send us. As long as your metadata blocks are consistent between the unabridged and abridged (min file)... it should fix any sync-age issues.

[qsniyg]

@Martii Thanks! The import worked fine, I guess I'll find out later if it works properly with the webhook :) If it does work, it's odd that it didn't before, really curious as to what the issue might have been.

[Martii]

I guess I'll find out later if it works properly with the webhook

Wait at least five minutes for GH's cache to expire... they do that for DoS protection on their end. Might be shorter but that's a good wait time between changes.

Thanks!

Welcome and rethanks for the report. Things should be consistent. Uploading/updating via importing your ~3MiB file took quite some time so the limits should still be applied with GH in all instances.

[Martii]

really curious as to what the issue might have been.

Pondering on this a bit more in the noggin. If your abridged and unabridged are in the same repo, since we are metadata blocks-centric and not (as much of) filename, GH may have sent the abridged first in their list and then the unabridged next in the list... thus was the bug I fixed with the limits. If the unabridged is below the limit then there is a 50% chance the abridged may make it to OUJS or the other 50% the abridged. Depends on their ordering at any specific time.

[sizzlemctwizzle]

The hook endpoint gets a json diff for each commit that occurs in a GH repo that has the webhook setup for the oujs hook endpoint. The endpoint ignores commits to any branch except master. It also only pays attention to changes to files that end in .js. Files that end in .user.js are handled differently than .js (my memory is hazy about how these are hadled). The filename is unimportant beyond that it ends in .user.js and if the username for the repo corresponds to a oujs user who has authenticated with GH, then the current source of the file (at HEAD in the master branch) is retrieved from GH. Then the userscript metadata block is parsed, and if the @name and @namespace combination match an existing script under this users account on oujs, the the new source replaces the existing on the site. It might seem complicated, but the intent is for it to be transparent with no need for configuration: like worrying about whether the file names are the same as what oujs generates. You could place the unminified version in a different branch than master, and even keep the "_min" in the filename on GH so that it is clear that it has been minified. It kind of makes since since the min version is what you want to publish, and the extra step of minification could be done in two commits (min plus rename) on top of the current unminified version when you merge that branch into master.

…

On Sun, Apr 12, 2020, 5:45 PM Marti Martz ***@***.***> wrote: really curious as to what the issue might have been. Pondering on this a bit more in the noggin. If your abridged and unabridged are in the same repo, since we are metadata blocks-centric and not filename, GH may have sent the abridged first in their list and then the unabridged next in the list... thus was the bug I fixed with the limits. If the unabridged is below the limit then there is a 50% chance the abridged may make it to OUJS *or* the other 50% the abridged. Depends on their ordering at any specific time. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1715 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAIXRQ46LIBV53TLH2F7O3RMJACNANCNFSM4MGP4TBA> .