hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravi teja <raviort...@gmail.com>
Subject Re: Quota for rogue ad-hoc queries
Date Fri, 09 Sep 2016 08:26:08 GMT
Hi,

I am trying to add this feature in hive ( HIVE-11735
<https://issues.apache.org/jira/browse/HIVE-11735> ).
But hit a road block while setting the quota during session folder creation
as the quota can be only set by super user in HDFS.
Any thoughts how to avoid this issue?

Thanks,
Ravi

On Fri, Sep 2, 2016 at 2:35 PM, ravi teja <raviorteja@gmail.com> wrote:

> Hi Gopal,
>
> We are using MR not Tez.
> I feel since the adhoc queries data output size is something we can
> determine, rather than the time the job takes, I was wondering more from
> output size/number of rows quota.
>
> Thanks,
> Ravi
>
> On Fri, Sep 2, 2016 at 2:57 AM, Gopal Vijayaraghavan <gopalv@apache.org>
> wrote:
>
>>
>> > Are there any other ways?
>>
>> Are you running Tez?
>>
>> Tez heartbeats counters back to the AppMaster every few seconds, so the
>> AppMaster has an accurate (but delayed) count of HDFS_BYTES_WRITTEN.
>>
>> Cheers,
>> Gopal
>>
>>
>>
>>
>>
>>
>>
>>
>

Mime
View raw message