apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: how to increase lifetime of hdfs delegation tokens ?
Date Fri, 01 Jul 2016 15:56:09 GMT
Hi Raja,

Let me look at this and get back to you.

Thanks

On Thu, Jun 30, 2016 at 11:20 PM, Raja.Aravapalli <
Raja.Aravapalli@target.com> wrote:

>
> Can someone pls help me, how can I ensure, my apex application doesn’t
> fail after 7days…
>
> Thanks a lot.
>
>
> Regards,
> Raja.
>
> From: "Raja.Aravapalli" <Raja.Aravapalli@target.com>
> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
> Date: Thursday, June 30, 2016 at 6:06 AM
>
> To: "users@apex.apache.org" <users@apex.apache.org>
> Subject: Re: how to increase lifetime of hdfs delegation tokens ?
>
>
> Hi,
>
> I triggered my application by specifying properties, “
> dt.authentication.principal” & “dt.authentication.keytab” , BUT, did not
> specify the property “dt.authentication.store.keytab”.
>
> I also observed the keytab is copied to hdfs location
> “/user/<user>/datatorrent”. But, still my apex application failed after
> 7days!!!
>
> I am setting these properties in “properties.xml” file!
>
> How can I ensure my settings are working correct. Having waiting for 7days
> to learn its failure is a very tough thing. Hope there should be some other
> alternatives.
>
> Can someone pls help me fix this ….  Thanks a lot !!
>
>
> Regards,
> Raja.
>
> From: "Raja.Aravapalli" <Raja.Aravapalli@target.com>
> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
> Date: Monday, June 20, 2016 at 5:43 PM
> To: "users@apex.apache.org" <users@apex.apache.org>
> Subject: Re: how to increase lifetime of hdfs delegation tokens ?
>
>
> Sure Pramod. Please respond on this mail chain when you get to know..
>
> Thanks very much.
>
>
> Regards,
> Raja.
>
> From: Pramod Immaneni <pramod@datatorrent.com>
> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
> Date: Monday, June 20, 2016 at 4:54 PM
> To: "users@apex.apache.org" <users@apex.apache.org>
> Subject: Re: how to increase lifetime of hdfs delegation tokens ?
>
> Raja,
>
> I believe it would. I will check and get back to you but the easiest way
> for you to check is that the file should appear in HDFS under
> /user/<username>/datatorrent with the same filename as it is in your local
> filesystem.
>
> Thanks
>
> On Mon, Jun 20, 2016 at 2:40 PM, Raja.Aravapalli <
> Raja.Aravapalli@target.com> wrote:
>
>>
>> Thanks for the response Pramod.
>>
>> My quick question is, I see we should mention these properties in
>> dt-site.xml !! I am not sure about dt-site.xml, all I am using is only
>> properites.xml file, which I am using to pass some configuration to
>> application.
>> Can I set these in properties.xml file and it will still work ?
>>
>>
>> Regards,
>> Raja.
>>
>> From: Pramod Immaneni <pramod@datatorrent.com>
>> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
>> Date: Monday, June 20, 2016 at 4:32 PM
>>
>> To: "users@apex.apache.org" <users@apex.apache.org>
>> Subject: Re: how to increase lifetime of hdfs delegation tokens ?
>>
>> Hi Raja,
>>
>> Yes the keytab would be copied over to HDFS and reused for getting a new
>> token before the old one expires. By default it is 7 days. If it is
>> different in your cluster please set the
>> properties dt.resourcemanager.delegation.token.max-lifetime and
>> dt.namenode.delegation.token.max-lifetime in dt-site.xml. Also if you don't
>> the default keytab to be copied over into HDFS and reused you can specify
>> your own keytab file for fetching a new token by putting it in HDFS and
>> specifying the property dt.authentication.store.keytab.All this is
>> described in the document that Thomas sent over.
>>
>> Thanks
>>
>> On Mon, Jun 20, 2016 at 1:54 PM, Raja.Aravapalli <
>> Raja.Aravapalli@target.com> wrote:
>>
>>>
>>> Hi Thomas,
>>>
>>> To ensure auto renewal of delegation tokens life time, Can I use the the
>>> below properties in properties.xml file ?
>>>
>>> <property>
>>>             <name>dt.authentication.principal</name>
>>>             <value>kerberos-principal-of-user</value>
>>>     </property>
>>>     <property>
>>>             <name>dt.authentication.keytab</name>
>>>             <value>absolute-path-to-keytab-file</value>
>>>     </property>
>>>
>>>
>>> FYI,
>>> I am launching application from Apex CLI! And till this time I haven’t
>>> used the above properties when launching apex applications in our secure
>>> hadoop environment, still they worked fine without any issues, but failing
>>> after 7days!!
>>>
>>> If I set the above properties in properties.xml, will that do
>>> auto-renewal and run successfully without any issues of failing again due
>>> to delegation token lifetime expiry ??
>>>
>>> Please advise.
>>>
>>>
>>> Thanks a lot in advance.
>>>
>>>
>>> Regards,
>>> Raja.
>>>
>>> From: "Raja.Aravapalli" <Raja.Aravapalli@target.com>
>>> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
>>> Date: Sunday, June 19, 2016 at 3:30 PM
>>>
>>> To: "users@apex.apache.org" <users@apex.apache.org>
>>> Subject: Re: how to increase lifetime of hdfs delegation tokens ?
>>>
>>>
>>> Thanks a lot Thomas.
>>>
>>> Will take this as reference and test our application. Great!
>>>
>>>
>>> Regards,
>>> Raja.
>>>
>>> From: Thomas Weise <thomas.weise@gmail.com>
>>> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
>>> Date: Sunday, June 19, 2016 at 2:01 PM
>>> To: "users@apex.apache.org" <users@apex.apache.org>
>>> Subject: Re: how to increase lifetime of hdfs delegation tokens ?
>>>
>>> Token expiration working as expected!
>>>
>>> Please have a look on how to extend or refresh it:
>>>
>>>
>>> https://github.com/apache/apex-core/blob/master/docs/security.md#token-refresh
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>> On Sat, Jun 18, 2016 at 10:26 PM, Raja.Aravapalli <
>>> Raja.Aravapalli@target.com> wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> My Apex application failed exactly after running 7days in our
>>>> distributed hadoop environment, with delegation token expiry!!
>>>>
>>>> Can someone pls help me with details, on how I can increase the
>>>> delegation token time to lifetime or any other process running in parallel
>>>> to renew the tokens ?
>>>>
>>>> *Exception details below:*
>>>>
>>>> ERROR hdfs.DFSClient (DFSClient.java:closeAllFilesBeingWritten(954)) - Failed
to close inode 11111111
>>>> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 111111 for XXXXXX) is expired
>>>> 	at org.apache.hadoop.ipc.Client.call(Client.java:1427)
>>>> 	at org.apache.hadoop.ipc.Client.call(Client.java:1358)
>>>>
>>>>
>>>>
>>>> Thanks a lot in advance.
>>>>
>>>>
>>>> Regards,
>>>> Raja.
>>>>
>>>>
>>>
>>
>

Mime
View raw message