hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com.INVALID>
Subject Re: [DISCUSS]: securing ASF Hadoop releases out of the box
Date Thu, 05 Jul 2018 20:09:22 GMT
I really, really like the approach of defaulting to only non-routeable
IPs allowed. it seems like a good tradeoff for complexity of
implementation, pain to reconfigure, and level of protection.

On Thu, Jul 5, 2018 at 2:25 PM, Todd Lipcon <todd@cloudera.com.invalid> wrote:
> The approach we took in Apache Kudu is that, if Kerberos hasn't been
> enabled, we default to a whitelist of subnets. The default whitelist is
>,,,, which
> matches the IANA "non-routeable IP" subnet list.
> In other words, out-of-the-box, you get a deployment that works fine within
> a typical LAN environment, but won't allow some remote hacker to locate
> your cluster and access your data. We thought this was a nice balance
> between "works out of the box without lots of configuration" and "decent
> security". In my opinion a "localhost-only by default" would be be overly
> restrictive since I'd usually be deploying on some datacenter or EC2
> machine and then trying to access it from a client on my laptop.
> We released this first a bit over a year ago if my memory serves me, and
> we've had relatively few complaints or questions about it. We also made
> sure that the error message that comes back to clients is pretty
> reasonable, indicating the specific configuration that is disallowing
> access, so if people hit the issue on upgrade they had a clear idea what is
> going on.
> Of course it's not foolproof, since as Eric says, you're still likely open
> to the entirety of your corporation, and you may not want that, but as he
> also pointed out, that might be true even if you enable Kerberos
> authentication.
> -Todd
> On Thu, Jul 5, 2018 at 11:38 AM, Eric Yang <eyang@hortonworks.com> wrote:
>> Hadoop default configuration aimed for user friendliness to increase
>> adoption, and security can be enabled one by one.  This approach is most
>> problematic to security because system can be compromised before all
>> security features are turned on.
>> Larry's proposal will add some safety to remind system admin if security
>> is disabled.  However, reducing the number of knobs on security configs are
>> likely required to make the system secure for the banner idea to work
>> without writing too much guessing logic to determine if UI is secured.
>> Penetration test can provide better insights of what hasn't been secured to
>> improve the next release.  Thankfully most Hadoop vendors have done this
>> work periodically to help the community secure Hadoop.
>> There are plenty of company advertised if you want security, use
>> Kerberos.  This statement is not entirely true.  Kerberos makes security
>> more difficult to crack for external parties, but it shouldn't be the only
>> method to secure Hadoop.  When the Kerberos environment is larger than
>> Hadoop cluster, anyone within Kerberos environment can access Hadoop
>> cluster freely without restriction.  In large scale enterprises or some
>> cloud vendors that sublet their resources, this might not be acceptable.
>> From my point of view, a secure Hadoop release must default all settings
>> to localhost only and allow users to add more hosts through authorized
>> white list of servers.  This will keep security perimeter in check.  All
>> wild card ACLs will need to be removed or default to current user/current
>> host only.  Proxy user/host ACL list must be enforced on http channels.
>> This is basically realigning the default configuration to single node
>> cluster or firewalled configuration.
>> Regards,
>> Eric
>> ´╗┐On 7/5/18, 8:24 AM, "larry mccay" <larry.mccay@gmail.com> wrote:
>>     Hi Steve -
>>     This is a long overdue DISCUSS thread!
>>     Perhaps the UIs can very visibly state (in red) "WARNING: UNSECURED UI
>>     ACCESS - OPEN TO COMPROMISE" - maybe even force a click through the
>> warning
>>     to get to the page like SSL exceptions in the browser do?
>>     Similar tactic for UI access without SSL?
>>     A new AuthenticationFilter can be added to the filter chains that
>> blocks
>>     API calls unless explicitly configured to be open and obvious log a
>> similar
>>     message?
>>     thanks,
>>     --larry
>>     On Wed, Jul 4, 2018 at 11:58 AM, Steve Loughran <
>> stevel@hortonworks.com>
>>     wrote:
>>     > Bitcoins are profitable enough to justify writing malware to run on
>> Hadoop
>>     > clusters & schedule mining jobs: there have been a couple of
>> incidents of
>>     > this in the wild, generally going in through no security, well known
>>     > passwords, open ports.
>>     >
>>     > Vendors of Hadoop-related products get to deal with their lockdown
>>     > themselves, which they often do by installing kerberos from the
>> outset,
>>     > making users make up their own password for admin accounts, etc.
>>     >
>>     > The ASF releases though: we just provide something insecure out the
>> box
>>     > and some docs saying "use kerberos if you want security"
>>     >
>>     > What we can do here?
>>     >
>>     > Some things to think about
>>     >
>>     > * docs explaining IN CAPITAL LETTERS why you need to lock down your
>>     > cluster to a private subnet or use Kerberos
>>     > * Anything which can be done to make Kerberos easier (?). I see
>> there are
>>     > some oustanding patches for HADOOP-12649 which need review, but what
>> else?
>>     >
>>     > Could we have Hadoop determine when it's coming up on an open
>> network and
>>     > start warning? And how?
>>     >
>>     > At the very least, single node hadoop should be locked down. You
>> shouldn't
>>     > have to bring up kerberos to run it like that. And for more
>> sophisticated
>>     > multinode deployments, should the scripts refuse to work without
>> kerberos
>>     > unless you pass in some argument like "--Dinsecure-clusters-
>> permitted"
>>     >
>>     > Any other ideas?
>>     >
>>     >
>>     > ------------------------------------------------------------
>> ---------
>>     > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>     > For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>     >
>>     >
> --
> Todd Lipcon
> Software Engineer, Cloudera


To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message