Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 13 Nov 2015 01:11:11 +0000 (UTC)
From: "Andrew Purtell (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12912376.1447294173000.57364.1447377071159@Atlassian.JIRA>
In-Reply-To: <JIRA.12912376.1447294173000@Atlassian.JIRA>
References: <JIRA.12912376.1447294173000@Atlassian.JIRA>
 <JIRA.12912376.1447294173671@arcas>
Subject: [jira] [Comment Edited] (HBASE-14799) Commons-collections object
 deserialization remote command execution vulnerability
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003329#comment-15003329 ] 

Andrew Purtell edited comment on HBASE-14799 at 11/13/15 1:10 AM:
------------------------------------------------------------------

I investigated the test failures and found some issues. 

The first is we never added efficient support for serializing our Pair type. We rely on generic object serialization for it. I fixed this problem. Unfortunately I cannot be 100% backwards compatible. We can't just whitelist Pair. A Pair can hold any other type of object. We get to see that we have a Pair, but not the types contained within until after deserialization, and that's too late. Therefore I've added a code for Pair and special case handling for it, like we do with List. Older peers will not understand this change. The APIs affected are HMasterInterface#getAlterStatus and HRegionInterface#bulkLoadHFiles. Sorry, cannot be helped and avoid risk of exploit. However, thankfully it's only two APIs that are not super commonly used. To reiterate, 100% compatibility won't be possible. If that is required, then we must close this as Wont Fix. 

I also discovered we are generically serializing the java.lang.* types in some cases. However we will handle the primitive types in a backwards compatible way if we simply unbox, so I do this where we can. Newer peers will be able to communicate with older peers without issue. If older peers elect send object-serialized primitives, though, newer peers will reject the message unless configured to accept legacy serialization. This is intended behavior.

I'm still working through 0.94 tests.


was (Author: apurtell):
I investigated the test failures and found some issues. 

The first is we never added efficient support for serializing our Pair type. We rely on generic object serialization for it. I fixed this problem. Unfortunately I cannot be 100% backwards compatible. We can't just whitelist Pair. A Pair can hold any other type of object. We get to see that we have a Pair, but not the types contained within until after deserialization, and that's too late. Therefore I've added a code for Pair and special case handling for it, like we do with List. Older peers will not understand this change. The APIs affected are HMasterInterface#getAlterStatus and HRegionInterface#bulkLoadHFiles. Sorry, cannot be helped and avoid risk of exploit. However, thankfully it's only two APIs that are not super commonly used. 

I also discovered we are generically serializing the java.lang.* types. However we will handle the primitive types in a backwards compatible way if we simply unbox, so I do this where we can. Newer peers will be able to communicate with older peers without issue. If older peers elect send object-serialized primitives, though, newer peers will reject the message unless configured to accept legacy serialization. This is intended behavior.

I'm still working through 0.94 tests.

> Commons-collections object deserialization remote command execution vulnerability 
> ----------------------------------------------------------------------------------
>
>                 Key: HBASE-14799
>                 URL: https://issues.apache.org/jira/browse/HBASE-14799
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Critical
>             Fix For: 0.94.28, 0.98.17
>
>         Attachments: HBASE-14799-0.94.patch, HBASE-14799-0.98.patch
>
>
> Read: http://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/
> TL;DR: If you have commons-collections on your classpath and accept and process Java object serialization data, then you probably have an exploitable remote command execution vulnerability. 
> 0.94 and earlier HBase releases are vulnerable because we might read in and rehydrate serialized Java objects out of RPC packet data in HbaseObjectWritable using ObjectInputStream#readObject (see https://hbase.apache.org/0.94/xref/org/apache/hadoop/hbase/io/HbaseObjectWritable.html#714) and we have commons-collections on the classpath on the server.
> 0.98 also carries some limited exposure to this problem through inclusion of backwards compatible deserialization code in HbaseObjectWritableFor96Migration. This is used by the 0.94-to-0.98 migration utility, and by the AccessController when reading permissions from the ACL table serialized in legacy format by 0.94. Unprivileged users cannot run the tool nor access the ACL table.
> Unprivileged users can however attack a 0.94 installation. An attacker might be able to use the method discussed on that blog post to capture valid HBase RPC payloads for 0.94 and prior versions, rewrite them to embed an exploit, and replay them to trigger a remote command execution with the privileges of the account under which the HBase RegionServer daemon is running.
> We need to make a patch release of 0.94 that changes HbaseObjectWritable to disallow processing of random Java object serializations. This will be a compatibility break that might affect old style coprocessors, which quite possibly may rely on this catch-all in HbaseObjectWritable for custom object (de)serialization. We can introduce a new configuration setting, "hbase.allow.legacy.object.serialization", defaulting to false.
> To be thorough, we can also use the new configuration setting  "hbase.allow.legacy.object.serialization" (defaulting to false) in 0.98 to prevent the AccessController from falling back to the vulnerable legacy code. This turns out to not affect the ability to migrate permissions because TablePermission implements Writable, which is safe, not Serializable. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)