Urge users to update Tampermonkey

[GreenLunar]

Use an information bar (a line that pops up from top of a page), similarly to popup blocking or accept cookies notifications, that would ask users to update to recent version of Tampermonkey.

Also, add a News entry about this occurrence, so other developers of User script engines would take this overload scenario into consideration.

@derjanb, please post your feedback on this matter.

[Martii]

Not liking the too busy check ay ;) What's weird is usage almost always raises around 5pm-8pm EST and 5am-8am EST... odd that it's almost symmetrical time wise.

There's no reliable form of identification though that I know of... unless something changed in Chromium/Chromes/Androids identification. e.g. someone needs to tell us how to fingerprint it reliably before we would do that.

[Martii]

@GreenLunar
I think it might be better to have a feed, either RSS or Atom for #914 (how is the search going over there btw) ... however...

@derjanb
I don't see a feed (text or icons) on tampermonkey.net for this... Does this exist? Thanks for another moment of your time.

[derjanb]

@Martii
Not at the moment, cause I went to bed 2 a.m. after our conversation (CEST) and on Sundays my wife, my daughter and I usually visit my mother-in-law. However, after lunch there is some time to work. :)
I'm focusing on new Opera and Safari releases at the moment. Do you see many requests by those users? How is the traffic doing in general?

[Martii]

@derjanb
Like clockwork two times a day we get hit now... I've kept some of it at bay with memory checks just recently and will do some more after some more pondering.

If you could get a RSS feed up at some point we'll definitely add you when #914 gets completed... no rush because we don't have that implemented just yet. LOL ;)

[derjanb]

There's no reliable form of identification though that I know of... unless something changed in Chromium/Chromes/Androids identification. e.g. someone needs to tell us how to fingerprint it reliably before we would do that.

Tampermonkey sends Accept: text/x-userscript-meta, */* at meta data requests (if at all ;)) and Accept: text/x-userscript, */* when it requests a script. I don't think other script handlers send the second one. So this could help you to identify TM users regardless of their user agent, but only when they're downloading a script.

[Martii]

text/x-userscript, */*

That's what we are being bombarded with... plus we might have an issue with text/x-userscript-meta, */* which I'll look into right now.

[derjanb]

And the user agents? It's all caused by Chrome browsers or also Opera and Safari?

[Martii]

Well Chrome's UA's are bit duplicative... so I see Safari for sure.

Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36

[Martii]

Here's an Opera

Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36 OPR/31.0.1889.174

[derjanb]

But it's hard to say whether this are normal or "faulty" requests, right?

[Martii]

Unlikely that it's normal as @TimidScript reported 70K installs... that's not normal.

[derjanb]

I'm aware that Chrome users cause a lot of requests at the moment and I guess that since TM is not doing a meta data request, but is downloading the complete script source, OUJS counts every update request as installation, but I might be wrong.

However, my question was whether the "faulty" requests are caused by other browsers as well. "Faulty" means repeated downloads for no reason. But I think this is very hard to say when the log files overflow.

[Martii]

The snapshot I grabbed in the terminal had thousands of Chrome starting UA's... when I get done splitting out the Accept header which has been extended by TM I'll see if that improves things with the meta routine routes. As far as requesting goes the scripts aren't that popular twice a day. ;)

[Martii]

K... now that I have dedicated time to answer things here.

OUJS counts every update request as installation, but I might be wrong.

See https://openuserjs.org/about/Frequently-Asked-Questions#q-does-openuserjs-org-have-meta- and https://openuserjs.org/about/Frequently-Asked-Questions#q-is-there-a-way-to-not-to-count-script-updates-with-this-sites-install-counter- and quacku's Q with my A at https://openuserjs.org/discuss/Huge_Number_of_Downloads#comment-153fcb25955 . I've had zero time to update our docs to reflect quacku's question.

was whether the "faulty" requests are caused by other browsers as well.

GM and GM Port do not have this issue... part of the issue is that we at OUJS didn't know that you were extending the Accept header e.g. from the original upstream spec is just text/x-userscript and nothing else... it's okay to extend as the Accept header has this option... but it was unexpected.

Bringing @JasonBarnabe into this to ensure he knows this too as a courtesy... just a FYI in case you didn't know and need to know.

But I think this is very hard to say when the log files overflow.

We don't log traffic like that... as @JasonBarnabe found out it killed his server... we only nab when we need to.

[Martii]

As far as an improvement with #962 and adding in the TM extension to the Accept header it didn't help... we're still at ~50% MEM (the limiting I set in place for serving scripts)

[derjanb]

One last question. How many different IPs do you see? I'd expect it to be a low number. But those IPs request the scripts again and again.

[Martii]

Need to find that in the request object first... just a few moments and I'll look at another snapshot.

[Martii]

More than 10... they whiz by very quickly... I would have to say that 90% of them are Chrome based UA's.

---

Largest recurrence is an IP in France. ;) OUI OUI!

[derjanb]

If you know the IPs you could try to add them to a iptables DROP rule. This means that no response packet is sent back to the source anymore and all requests should run into the TCP timeout. This might throttle the requests a little bit and may be worth a try.

[Martii]

How many different IPs do you see? I'd expect it to be a low number.

There are five main ones:

• JAPAN
• FRANCE
• INDONESIA
• INDONESIA
• HUNGARY

However there are hundreds of little tiny ones that pop in and out... I also noticed install requests from Chrome UA's picking other Accept types like image/png when attempting to get the source... which is definitely not right.

If you know the IPs you could try to add them to a iptables DROP rule.

I've asked @sizzlemctwizzle to educate me on what he's set up that I don't have knowledge of what he did... I can IP block here but that's not the the VPS. I'd also like to keep this in node for contributors understanding as well... if needed one of us can block IP's in the OS. (and yes I know the IPs now)

[Martii]

image/webp,image/,/;q=0.8 /install/open2chExtender/open2chAdBlock.user.js
image/png,image/;q=0.8,/;q=0.5 /install/open2chExtender/open2chAdBlock.user.js

This isn't normal... both from chrome based agents and multiple recurrences over hours.

[derjanb]

I agree, but it's caused by the script:

[Martii]

Well I'll filter that out eventually. Thanks for the tip... anyhow... I'm exhausted...bbi about 8ish hours. Feel free to look over things... I usually comment pretty well. Hope you publish a working meta check soon too. ;) :)

[Martii]

Some more new countries today

• RUSSIA
• SWEDEN
• NORWAY

[derjanb]

Hope you publish a working meta check soon too. ;) :)

@Martii The current beta version requests meta data (incl. x-userscript-meta) from the source URL (.user.js) if no @updateURL is set. This is a little different from other script hosting sites - they server meta data only at .meta.js URLs.

Is this OK for you or do you think it's better to use .meta.js URLs for OUJS based scripts as well?

[Martii]

if no @updateURL is set.

This part is confusing me this morning... so what happens if it is set? This FAQ entry is what we currently cover and some of the /install route (URI) pitfalls.

---

I would prefer not having an engine using the meta.json though that's typically used in userscripts if they really want it. e.g. keep the traffic to a minimum overall.

[derjanb]

if no @updateURL is set.

This part is confusing me this morning... so what happens if it is set?

Then it's used. What else should happen?

Do you still see issues caused by TM users with that faulty version? GreasyFork is back at a normal traffic level quite a while now. If not, then you could loosen the restriction a little bit, because it seems that even regular Tampermonkey users are affected by it. #999

[Martii]

Do you still see issues caused by TM users with that faulty version?

... and current release as you already referenced.

GreasyFork is back at a normal traffic level quite a while now.

He's probably not doing a deeper analysis of his data stream so that's not accurate... we still capture several clients that hit with a vengeance a couple of times a day but we quash them nicely so far. e.g. we are even better than before the ordeal.

Then it's used.

Good... it might be a good idea to send the header all the time if you know your extension wants to do an update check.

If not, then you could loosen the restriction a little bit

Absolutely not. Your update checks are minimum at 6 hours and if someone continually taps the install route they will be too many requested. We won't have a repeat of this. Until I know for sure that TM has a viable meta routine check we won't even consider "loosening" this DDoS prevention. Again TM isn't the only one with an issue here but everybody needs time to absorb what happened and what was revealed.

You asked me earlier elsewhere if other engines had issues... GM / GM Port has one on busy and VM Presto has one with installation... both projects have issues created.

[derjanb]

we won't even consider "loosening" this DDoS prevention

So an attempt to install a script, but then cancel the installation for whatever reason, just to feel certain that you want to install it 3 minutes later is a DDoS?

[Martii]

So an attempt to install a script, but then cancel the installation for whatever reason, just to feel certain that you want to install it 3 minutes later is a DDoS?

Correct... that's what viewing the source is in all the engines is (e.g. from whatever cache they use)... e.g. review the script first and decide if you want it installed. TM unveiled this exploit which is a good thing in part and research led into a few other arenas that have some repeat requests.

It's bad enough that I found several of our users gaming the install count intentionally... the known ones have been put to a stop although I doubt we'll ever trap all of them. I've done my best to balance inept practices with good practices on script installation. An inept practice is what you just mentioned about installing twice... take the time to decide... use the tools available... then install in the engine.

[derjanb]

Ok, this means two people at let's say at a company network are also doing a DDoS when both are trying to install the same script?

[Martii]

Then they will have to wait a little while in between requests... it's that simple.

[Martii]

@derjanb
What exactly is "externals" and why isn't @externals in your documentation at this moment? TIA

Ref:

[derjanb]

@require and @resource are externals.

[Martii]

@derjanb

@require and @resource are externals.

Thanks.

So "Always" uses the browser cache then with the update interval? If so what is the current default caching time (no additional extensions and no presumed cache disabling)?

[derjanb]

Yes it always utilizes the browser cache regardless of the update interval setting.

If so what is the current default caching time

Tampermonkey uses a default update interval of 7 days. The browser caching time depends on the headers sent along with the resource.

[Martii]

This is a given so we really don't need to urge them. Hopefully google will handle this.

Closing.