Breaking

Analysis of Rails XML Parameter Parsing Vulnerability

This post tries to give an overview about the background and impact of the new Rails XML parameter parsing vulnerability patched today.

The bug

The root cause of the vulnerability is Rails handling of formatted parameters. In addition to standard GET and POST parameter formats, Rails can handle multiple different data encodings inside the body of POST requests. By default JSON and XML are supported. While support for JSON is widely used in production, the XML functionality does not seem to be known by many Rails developers.

XML parameter parsing

The code responsible for parsing these different data types is shown below:

# actionpack/lib/action_dispatch/middleware/params_parser.rb 
....
DEFAULT_PARSERS = {
      Mime::XML => : xml_simple,
      Mime::JSON => :json
    }
....
def parse_formatted_parameters(env)
        ...
        when Proc
          strategy.call(request.raw_post)
        when : xml_simple, : xml_node
          data = Hash.from_xml(request.raw_post) || {}
          data.with_indifferent_access
        when :yaml
          YAML.load(request.raw_post)
        when :json
          data = ActiveSupport::JSON.decode(request.raw_post)
          data = {:_json => data} unless data.is_a?(Hash)
          data.with_indifferent_access
        else
          false
        end
...


While the Proc and :yaml cases fortunately are not reachable by default and JSON is of minor importance, the XML case calls Hash.from_xml() passing the raw POST body as argument. The code below is a simplified, showing only relevant parts of this code path.

def typecast_xml_value(value)
        case value.class.to_s
          when 'Hash'
            if value['type'] == 'array'
              ....
              ...
            elsif .. ||  (value["__content__"] && 
                 (value.keys.size == 1 ||value["__content__"].present?))
              content = value["__content__"]
              if parser = ActiveSupport::XmlMini::PARSING[value["type"]]
                parser.arity == 1 ? parser.call(content) : parser.call(content, value)
              else
                content
              end
            .....
            end
          when 'Array'
            value.map! { |i| typecast_xml_value(i) }
            value.length > 1 ? value : value.first
          when 'String'
            value
        end
      end

Typed XML

As can be seen, the code supports the deserialization of XML trees into different datatypes. While basic types like Arrays, Strings and Hashs can be handled pretty easily, the XML parser includes additional support for several other types in order to allow code like the snippet below to be parsed correctly.

<?xml version="1.0" encoding="UTF-8"?>
<hash>
<foo type="integer">1</foo>
<bar type="float">1.3</bar>
</hash>

This typing support helps applications which do not want to implement their own date or integer parsing code. It is implemented using the “type” attribute and the following code line:

if parser = ActiveSupport::XmlMini::PARSING[value["type"]]
parser.arity == 1 ? parser.call(content) : parser.call(content, value)

 

Unfortunately ActiveSupport::XmlMini::PARSING not only includes harmless types like Integers or Floats, but also two special ones which have a critical security impact: symbol and yaml.

Symbols and Rails

A Symbol is a special ruby type which normally gets created using the :name syntax. Symbols have several interesting properties which are out of the scope of this blogpost, but most importantly internal Rails functions rely on the assumption that no external user can inject symbols with malicious values. This assumption has already been attacked in several ways, most recently by joernchen roughly one month ago in CVE-2012-5664. Even with the coressponding patch, more attack vectors using malicious symbols probably exist. For this bug the impact is reduced because the call to data.with_indifferent_access in parse_formatted_parameters transforms all Symbol keys in the hashtable to Strings which makes the exploitation of CVE-2012-5664 more difficult. However, the vector still exists and and exploitation might still be feasible.

In the end, the injection of symbols is a serious security issue. Still the second type “yaml” is even more interesting.

YAML

YAML (YAML Ain’t Markup Language) is a data serialization format similar to JSON. Support for YAML inside XML is implemented using the following entry in ActiveSupport::XmlMini::Parsing:

"yaml" => Proc.new { |yaml| YAML::load(yaml) rescue yaml }

As you might remember YAML formatted parameters are not enabled by default in Rails due
to  YAML (or more specifically the YAML parsers  used by most scripting languages like e.g. Python or Ruby) not being designed to handle malicious user input. The YAML parser used by Ruby supports the serialization and deserialization of arbitrary data types. This includes Symbols (again!) and also arbitrary Objects.
The YAML string “”— !ruby/object:A\nfoo: 1\nbar: 1\n” will create an object instance of class A and set the object attributes @foo and @bar to the value 1. This means that an attacker can create instances of all classes defined in the targeted Rails application. This includes the basic ruby classes, all classes defined in the different activexy namespaces and even the ones used by lower level libraries like rack.  This opens a enormous attack surface as described in the next sections.

Object Injection Attacks

So we have the ability to insert arbitary object instances as parameters into a running Rails application. What’s the impact? Let’s take a look at other web stacks:

PHP

Object injection in PHP is possible using the unserialize() function
as described by Stefan Esser in 2009. Thanks to “magic methods” like __wakeup and __destruct which get called
every time an object is deserialized or garbage collected, command execution is possible in most non trivial and current PHP applications, as demonstrated by bugs in phpMyAdmin, Typo3, Piwik or CakePHP.

Java

A quite similar vulnerability was discovered by Johannes Dahse in the Apache Struts2 web framework. Summarizing the very detailed blog post: Remote command execution is possible.

Python

Finally, the pickle library used by python is easily attackable, because it includes class definitions in the serialized data format as demonstrated in this blog post by Nelson Elhage.

As these examples demonstrate object injection vulnerabilities can be exploited in many different ways. In the next section we show some potential attack vectors for Rails applications

Exploitation in Rails


Let’s recap the abilities that an attacker obtains due to the described YAML injection: He can create an object instance of any class defined in the application. Furthermore he can set all instance variables to arbitrary values and is not restricted by validatios in the initialize or setter methods.

class B
  def initialize()
    @code = "puts 4"
  end
  def foo()
    eval @code
end
end

Imagine the (rather stupid) example class B above, if we are able to call the foo() method on a deserialized object we can execute arbitrary code. For example the YAML string “— !ruby/object:B\ncode: puts 10\n” results in the execution of puts 10. Of course, most real world applications do not have such simple exploitation vectors but thanks to the power of ruby, Rails becomes an easy target:

Ruby does have a dynamic type system and a rich support for operator overloading.  Missing type checks and the assumption that all user passed values are Strings,Array or Hashes open many exploitation vectors. While we do have discovered multiple classes and code paths that can be used to inject arbitrary code, we will not publish details about them till the majority of sites have applied the patch. Instead we demonstrate how object injection can be used to execute SQL injection attacks against Rails 3 application using arel objects.

arel allows the creation of complex SQL queries using familiar ruby syntax. An in depth discussion of arel is out of the scope of this post but arel objects have an interesting property when passed as argument to one of Rails dynamic finder methods (find_by_*): The SQL code representing the arel object is directly copied into the SELECT statement created by the finder method, without applying any escaping:

irb(main):008:0> arel = Arel::Nodes::SqlLiteral.new("foo")
 => "foo"
irb(main):009:0> Blog.find_by_id(arel)
Blog Load (0.5ms) SELECT "blogs".* FROM "blogs" WHERE "blogs"."id" = foo LIMIT 1
SQLite3::SQLException: no such column: foo: SELECT "blogs".* FROM "blogs" WHERE "blogs"."id" = foo LIMIT 1

This is quite similiar to CVE-2012-5664 and would not be a problem under normal circumstances, but because we can create an Arel::Nodes::SqlLiteral object using YAML it becomes easily exploitable when a finder method is called like this:

XYZ.find_by_*(params[:x])

We can pass the string representing such an object as value of the x parameter inside a correctly formed XML post body. This will result in the creation of a SqlLiteral object and the injection of “SQLCODE” inside the executed SQL statement. We validated this attack against Rails 3.2.10 running the most recent version of the redmine project managment application and could extract arbitrary database entries as unauthenticated user. Note that the dynamic finder requirement is specific to this attack vector. Arbitrary code injection is still possible even when no dynamic finders are used.

 Summary

The discussed vulnerability is highly critical and allows code executions and SQL injections in all Rails applications that do not disable parsing of XML formatted parameters. Administrators should apply the published patch as soon as possible. If a timely update is not possible ActionDispatch::ParamsParser::DEFAULT_PARSERS should be modified to remove XML support.

 
Felix Wilhelm

Comments

  1. So every production Rails app has a major security vulnerability and you published exploit code an hour after it was announced?

    You’re a compelete asshole.

    1. @hey:

      Sir (as you’re probably male),

      nice to meet you. Haven’t talked to such a gentle person for a long time. Really.
      So you seem to have a certain attitude as for handling vulnerability disclosure. We have another one, apparently.
      In addition you seem to have a certain way of addressing people, with not-so-nice words. Didn’t your mother tell you otherwise? Anyhow, again our way is a different one.
      Last you seem to stand for your beliefs in public, by providing your full name, to strengthen your statements. Congratulations! You made a difference today.

      Best

      Enno, erey@ernw.de

  2. Thanks for the write up, it’s really useful in understanding the security risk and some of the rails internals. I have a few more questions on the topic, could I email you directly?

  3. 真是死也要拉一个垫背的。乱用cPickle和eval也能算bug?

    If you trust data you pickle’d in Python, you are doing it WRONG.

  4. Enno, anyone has the right to be aggressive on this. He may very well be a Rails app owner that knows that his assets are wide open to threats now.

    From your side, being in favor of (somehow) full disclosure, you should be open to strict criticism as fierce it is. I also support full disclosure but I do understand people that don’t.

    Anyway, that was an excellent post.

  5. From what i’ve see here is that versions below 2.0 should not be affected by this issue as they do not support xml parameters and don’t have any XMLMini implementations.

  6. This is a really great write up.

    As for complaints about it coming up too soon, this exploit has already been announced and patched. How long do you want people to wait to discuss the ramifications of the exploit? This information is useful. I am quite sure people who are interested in exploiting this vulnerability don’t need a blog post explaining how the exploit works and what possible attack vectors are. People interested in that sort of thing usually know what’s up.

  7. Enno and Felix,

    I think your missing the point – while you’ll get a lot of traffic from this post, your actually doing more harm than good by posting precise code examples of the exploit.

    I echo Nathan Broadbent – you should pull this post and let it simmer for a few days while production sites protect themselves from the exploits.

  8. Thanks for the detailed analysis!

    It is way more helpful in avoiding similar issues in the future than the hand waving reports that are more typical for this type of issue.

    Ignore the WWW Potty-Mouth League. 😉

    Patrick

  9. Why is Python Pickle included in the list of comparitive web stacks? Pickle is a serialization format, not a web stack. Furthermore it says right at the top of the Pickle documentation in a scary red box:

    Warning

    The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.

  10. @erey,

    while the tone of the first commenter is absolutely counter-productive, he is right in saying that your post sheds too many details, too early, on how to exploit this critical Rails vulnerability.

    On the other hand, you saved me one hour of investigation while reaching for a proof-of-concept to test that the emergency fix I installed was really working — but you maybe also saved a bad guy, who’s never touched Ruby before, 8 hours of analysis before he is able to break into a production Rails webapp.

    This is the eternal fight between Full-Disclosure and the so-called “Responsible-Disclosure” – we’re not going to solve it today 🙂 but I hope that I helped you in understand the goods of the point that “@hey” has unsuccessfully tried to make.

    Cheers,

    ~Marcello (vjt)

  11. I think this coming too early, you don’t need to be a ruby ninga to pull down sites, please pull down this post and repost a week after.

    gracias

  12. I got suspended from high school back in 99 for showing the IT admin that I had the ability to get into all networked storage devices through an apple exploit. I showed him so he would be impressed and show me how we both could fix it… I was such a nice boy. When I got back to school I paid that suspension forward by deleting and crashing as many files and services as possible. I then had to go before the district school board.

    The moral of the story: its ok to find problems, publish them! Then fix them! Bad stuff will happen but much much worse could happen if you try to demonize the people who find the exploits. Exploits are what make technology fun!

  13. This post does far more good than harm. Knowledge about a vulnerability AFTER it is widely known about is usually a good thing.

  14. Ignore those telling you to withhold detailed info relating to exploits. They are shortsighted.

    Individuals or groups with the resources & knowledge to do widespread or serious damage will unlikely be informed of such exploits via your site. Who gives a fuck about kiddies querying Google for a new Ruby exploit?

    If developers were more active in investigating & openly discussing vulnerabilities, a chaotic situation like this may have been preventable.

    Stick it, Ruby developers. It’s your own fault, retards.

  15. To put Enno’s points another way (hopefully usefully):

    1) If you really want to get someone who has no obligation to do what you say to do what you say, diplomacy and openness are your best bets. If you reject those routes, then it’s unlikely people will take your desire or ability to change their minds seriously.

    2) He’s pretty obviously smart enough to know what he’s doing and has his own opinion about it, so you’re going to need more than bluntly telling him he’s making a mistake to get him to change his mind.

    My own point:

    Someone who realizes (1) and (2) and doesn’t think they could get what they want even after putting in the work for it might resort to trying to piss someone off they disagree with by name-calling. It’s worth remembering that such people exist.

Comments are closed.