hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anu Engineer <aengin...@hortonworks.com>
Subject Re: [DISCUSS]: securing ASF Hadoop releases out of the box
Date Thu, 05 Jul 2018 21:15:18 GMT
+1, on the Non-Routable Idea. We like it so much that we added it to the Ozone roadmap.
https://issues.apache.org/jira/browse/HDDS-231

If there is consensus on bringing this to Hadoop in general, we can build this feature in
common.

--Anu


On 7/5/18, 1:09 PM, "Sean Busbey" <busbey@cloudera.com.INVALID> wrote:

    I really, really like the approach of defaulting to only non-routeable
    IPs allowed. it seems like a good tradeoff for complexity of
    implementation, pain to reconfigure, and level of protection.
    
    On Thu, Jul 5, 2018 at 2:25 PM, Todd Lipcon <todd@cloudera.com.invalid> wrote:
    > The approach we took in Apache Kudu is that, if Kerberos hasn't been
    > enabled, we default to a whitelist of subnets. The default whitelist is
    > 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,169.254.0.0/16 which
    > matches the IANA "non-routeable IP" subnet list.
    >
    > In other words, out-of-the-box, you get a deployment that works fine within
    > a typical LAN environment, but won't allow some remote hacker to locate
    > your cluster and access your data. We thought this was a nice balance
    > between "works out of the box without lots of configuration" and "decent
    > security". In my opinion a "localhost-only by default" would be be overly
    > restrictive since I'd usually be deploying on some datacenter or EC2
    > machine and then trying to access it from a client on my laptop.
    >
    > We released this first a bit over a year ago if my memory serves me, and
    > we've had relatively few complaints or questions about it. We also made
    > sure that the error message that comes back to clients is pretty
    > reasonable, indicating the specific configuration that is disallowing
    > access, so if people hit the issue on upgrade they had a clear idea what is
    > going on.
    >
    > Of course it's not foolproof, since as Eric says, you're still likely open
    > to the entirety of your corporation, and you may not want that, but as he
    > also pointed out, that might be true even if you enable Kerberos
    > authentication.
    >
    > -Todd
    >
    > On Thu, Jul 5, 2018 at 11:38 AM, Eric Yang <eyang@hortonworks.com> wrote:
    >
    >> Hadoop default configuration aimed for user friendliness to increase
    >> adoption, and security can be enabled one by one.  This approach is most
    >> problematic to security because system can be compromised before all
    >> security features are turned on.
    >> Larry's proposal will add some safety to remind system admin if security
    >> is disabled.  However, reducing the number of knobs on security configs are
    >> likely required to make the system secure for the banner idea to work
    >> without writing too much guessing logic to determine if UI is secured.
    >> Penetration test can provide better insights of what hasn't been secured to
    >> improve the next release.  Thankfully most Hadoop vendors have done this
    >> work periodically to help the community secure Hadoop.
    >>
    >> There are plenty of company advertised if you want security, use
    >> Kerberos.  This statement is not entirely true.  Kerberos makes security
    >> more difficult to crack for external parties, but it shouldn't be the only
    >> method to secure Hadoop.  When the Kerberos environment is larger than
    >> Hadoop cluster, anyone within Kerberos environment can access Hadoop
    >> cluster freely without restriction.  In large scale enterprises or some
    >> cloud vendors that sublet their resources, this might not be acceptable.
    >>
    >> From my point of view, a secure Hadoop release must default all settings
    >> to localhost only and allow users to add more hosts through authorized
    >> white list of servers.  This will keep security perimeter in check.  All
    >> wild card ACLs will need to be removed or default to current user/current
    >> host only.  Proxy user/host ACL list must be enforced on http channels.
    >> This is basically realigning the default configuration to single node
    >> cluster or firewalled configuration.
    >>
    >> Regards,
    >> Eric
    >>
    >> On 7/5/18, 8:24 AM, "larry mccay" <larry.mccay@gmail.com> wrote:
    >>
    >>     Hi Steve -
    >>
    >>     This is a long overdue DISCUSS thread!
    >>
    >>     Perhaps the UIs can very visibly state (in red) "WARNING: UNSECURED UI
    >>     ACCESS - OPEN TO COMPROMISE" - maybe even force a click through the
    >> warning
    >>     to get to the page like SSL exceptions in the browser do?
    >>     Similar tactic for UI access without SSL?
    >>     A new AuthenticationFilter can be added to the filter chains that
    >> blocks
    >>     API calls unless explicitly configured to be open and obvious log a
    >> similar
    >>     message?
    >>
    >>     thanks,
    >>
    >>     --larry
    >>
    >>
    >>
    >>
    >>     On Wed, Jul 4, 2018 at 11:58 AM, Steve Loughran <
    >> stevel@hortonworks.com>
    >>     wrote:
    >>
    >>     > Bitcoins are profitable enough to justify writing malware to run on
    >> Hadoop
    >>     > clusters & schedule mining jobs: there have been a couple of
    >> incidents of
    >>     > this in the wild, generally going in through no security, well known
    >>     > passwords, open ports.
    >>     >
    >>     > Vendors of Hadoop-related products get to deal with their lockdown
    >>     > themselves, which they often do by installing kerberos from the
    >> outset,
    >>     > making users make up their own password for admin accounts, etc.
    >>     >
    >>     > The ASF releases though: we just provide something insecure out the
    >> box
    >>     > and some docs saying "use kerberos if you want security"
    >>     >
    >>     > What we can do here?
    >>     >
    >>     > Some things to think about
    >>     >
    >>     > * docs explaining IN CAPITAL LETTERS why you need to lock down your
    >>     > cluster to a private subnet or use Kerberos
    >>     > * Anything which can be done to make Kerberos easier (?). I see
    >> there are
    >>     > some oustanding patches for HADOOP-12649 which need review, but what
    >> else?
    >>     >
    >>     > Could we have Hadoop determine when it's coming up on an open
    >> network and
    >>     > start warning? And how?
    >>     >
    >>     > At the very least, single node hadoop should be locked down. You
    >> shouldn't
    >>     > have to bring up kerberos to run it like that. And for more
    >> sophisticated
    >>     > multinode deployments, should the scripts refuse to work without
    >> kerberos
    >>     > unless you pass in some argument like "--Dinsecure-clusters-
    >> permitted"
    >>     >
    >>     > Any other ideas?
    >>     >
    >>     >
    >>     > ------------------------------------------------------------
    >> ---------
    >>     > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    >>     > For additional commands, e-mail: common-dev-help@hadoop.apache.org
    >>     >
    >>     >
    >>
    >>
    >>
    >
    >
    > --
    > Todd Lipcon
    > Software Engineer, Cloudera
    
    
    
    -- 
    busbey
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org
    
    


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Mime
View raw message