hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Loïc Chanel <loic.cha...@telecomnancy.net>
Subject Re: Computation timeout
Date Thu, 30 Jul 2015 07:31:46 GMT
Rats, I think I just figured it out.
#2 Is NEGATIVE 3000, right ? I set it to positive yesterday.
As for #1, I think it is the default value, so I am not sure I have to set
it.

Can you confirm that there is a typo on the name of your properties
(missing last letter) and that is not the actual name of the properties ?

I'll try again and keep you informed


Loïc CHANEL
Engineering student at TELECOM Nancy
Trainee at Worldline - Villeurbanne

2015-07-29 20:15 GMT+02:00 Xuefu Zhang <xzhang@cloudera.com>:

> this works for me:
> In hive-site.xml:
>   1. hive.server2.session.check.interva=3000;
>   2. hive.server2.idle.operation.timeou=-30000;
> restart HiveServer2.
>
> at beeline, I do "analyze table X compute statistics for columns", which
> takes longer than 30s. it was aborted by HS2 because of above settings. I
> guess it didn't work for you because you didn't have #1.
>
> --Xuefu
>
> On Wed, Jul 29, 2015 at 9:23 AM, Loïc Chanel <loic.chanel@telecomnancy.net
> > wrote:
>
>> I don't think your solution works, as after more than 4 minutes I could
>> still see logs of my job showing that it was running.
>> Do you have a way to check that even if the job was running, it was not
>> being killed by Hive ?
>> Or another solution ?
>>
>> Thanks for your help,
>>
>>
>> Loïc
>>
>> Loïc CHANEL
>> Engineering student at TELECOM Nancy
>> Trainee at Worldline - Villeurbanne
>>
>> 2015-07-29 16:26 GMT+02:00 Loïc Chanel <loic.chanel@telecomnancy.net>:
>>
>>> Yes, I set it to negative 60.
>>>
>>> It's not a problem if the session is killed. That's actually what I try
>>> to do, because I can't allow to a user to try to end an infinite request.
>>> Therefore I'll try your solution :)
>>>
>>> Thanks,
>>>
>>>
>>> Loïc
>>>
>>> Loïc CHANEL
>>> Engineering student at TELECOM Nancy
>>> Trainee at Worldline - Villeurbanne
>>>
>>> 2015-07-29 16:14 GMT+02:00 Xuefu Zhang <xzhang@cloudera.com>:
>>>
>>>> Okay. To confirm, you set it to negative 60s?
>>>>
>>>> The next thing you can try is to set
>>>> hive.server2.idle.session.timeou=60000 (60sec) and
>>>> hive.server2.idle.session.check.operation=false. I'm pretty sure this
>>>> works, but the user's session will be killed though.
>>>>
>>>> --Xuefu
>>>>
>>>> On Wed, Jul 29, 2015 at 7:02 AM, Loïc Chanel <
>>>> loic.chanel@telecomnancy.net> wrote:
>>>>
>>>>> I confirm : I just tried hive.server2.idle.operation.timeout setting
>>>>> it to -60 (seconds), but my veeeeeery slow job have not been killed.
The
>>>>> issue here is "what if another user come and try to submit a MapReduce
job
>>>>> but the cluster is stuck in an infinite loop ?".
>>>>>
>>>>> Do you or anyone else have another idea ?
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Loïc
>>>>>
>>>>> Loïc CHANEL
>>>>> Engineering student at TELECOM Nancy
>>>>> Trainee at Worldline - Villeurbanne
>>>>>
>>>>> 2015-07-29 15:34 GMT+02:00 Loïc Chanel <loic.chanel@telecomnancy.net>:
>>>>>
>>>>>> No, because I thought the idea of infinite operation was not very
>>>>>> compatible with the "idle" word (as the operation will not stop running),
>>>>>> but I'll try :-)
>>>>>> Thanks for the idea,
>>>>>>
>>>>>>
>>>>>> Loïc
>>>>>>
>>>>>> Loïc CHANEL
>>>>>> Engineering student at TELECOM Nancy
>>>>>> Trainee at Worldline - Villeurbanne
>>>>>>
>>>>>> 2015-07-29 15:27 GMT+02:00 Xuefu Zhang <xzhang@cloudera.com>:
>>>>>>
>>>>>>> Have you tried hive.server2.idle.operation.timeout?
>>>>>>>
>>>>>>> --Xuefu
>>>>>>>
>>>>>>> On Wed, Jul 29, 2015 at 5:52 AM, Loïc Chanel <
>>>>>>> loic.chanel@telecomnancy.net> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> As I'm trying to build a secured and multi-tenant Hadoop
cluster
>>>>>>>> with Hive, I am desperately trying to set a timeout to Hive
requests.
>>>>>>>> My idea is that some users can make mistakes such as a join
with
>>>>>>>> wrong keys, and therefore start an infinite loop believing
that they are
>>>>>>>> just launching a very heavy job. Therefore, I'd like to set
a limit to the
>>>>>>>> time a request should take, in order to kill the job automatically
if it
>>>>>>>> exceeds it.
>>>>>>>>
>>>>>>>> As such a notion cannot be set directly in YARN, I saw that
>>>>>>>> MapReduce2 provides with its own native timeout property,
and I would like
>>>>>>>> to know if Hive provides with the same property someway.
>>>>>>>>
>>>>>>>> Did anyone heard about such a thing ?
>>>>>>>>
>>>>>>>> Thanks in advance for your help,
>>>>>>>>
>>>>>>>>
>>>>>>>> Loïc
>>>>>>>>
>>>>>>>> Loïc CHANEL
>>>>>>>> Engineering student at TELECOM Nancy
>>>>>>>> Trainee at Worldline - Villeurbanne
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message