hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Blaze Spinnaker <blazespinna...@gmail.com>
Subject Re: Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster
Date Tue, 31 Oct 2017 04:22:10 GMT
Ravi,

The code and architecture is based on the Hadoop source code submitted
through the Yarn Client.    This is an issue for map reduce as well.  eg:
https://pravinchavan.wordpress.com/2013/04/25/223/

On Mon, Oct 30, 2017 at 1:15 PM, Ravi Prakash <ravihadoop@gmail.com> wrote:

> Hi Blaze!
>
> Thanks for digging into this. I'm sure security related features could use
> more attention. Tokens for one user should be isolated from other users.
> I'm sorry I don't know how spark uses them.
>
> Would this question be more appropriate on the spark mailing list?
> https://spark.apache.org/community.html
>
> Thanks
> Ravi
>
> On Mon, Oct 30, 2017 at 12:43 PM, Blaze Spinnaker <
> blazespinnaker@gmail.com> wrote:
>
>> I looked at this a bit more and I see a container_tokens file in spark
>> directory.   Does this contain the credentials where are added by
>> addCredentials?   Is this file accessible to the spark executors?
>>
>> It looks like just a clear text protobuf file.
>>
>> https://github.com/apache/hadoop/blob/82cb2a6497caa7c5e693aa
>> 41ad18e92f1c7eb16a/hadoop-common-project/hadoop-common/
>> src/main/java/org/apache/hadoop/security/Credentials.java#L221
>>
>> This means that anyone with access to the user can read credentials from
>> any other user.  Correct?
>>
>> On Mon, Oct 30, 2017 at 12:28 PM, Blaze Spinnaker <
>> blazespinnaker@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We are submitting critical UserGroupInformation credentials and wanted
>>> to know how these are protected in Spark Cluster.
>>>
>>> Questions:
>>>
>>> Are the credentials persisted to disk at any point?  If so, where?
>>> If they are persisted, are they encrypted? Or just obfuscated?  is the
>>> encryption key accessible?
>>> Are they only protected by file permissions?
>>>
>>> Are they only in memory?
>>>
>>> How would you securely propagate UGI / credentials to spark executors?
>>>
>>> Regards,
>>>
>>> Tim
>>>
>>
>>
>

Mime
View raw message