hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MiaoMiao <>
Subject Re: Percentile calculation
Date Tue, 02 Oct 2012 03:10:39 GMT
More info, please.

On Mon, Oct 1, 2012 at 4:50 PM, Mayank Bansal
<> wrote:
> Hi,
> I am trying to run the hive udf percentile, I am trying to run it on a
> column with something around 116 million unique values.
> The maximum space that I can give to the reducer is 12 GB, the job keeps on
> failing due to java heap space error.
> Is there a way to optimize this, so that I don’t encounter this error?
> Or any other suggestion or solution which could help me out?
> Thanks,
> Mayank
> ________________________________
> This email message may contain proprietary, private and confidential
> information. The information transmitted is intended only for the person(s)
> or entities to which it is addressed. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance upon,
> this information by persons or entities other than the intended recipient is
> prohibited and may be illegal. If you received this in error, please contact
> the sender and delete the message from your system.
> Mu Sigma takes all reasonable steps to ensure that its electronic
> communications are free from viruses. However, given Internet accessibility,
> the Company cannot accept liability for any virus introduced by this e-mail
> or any attachment and you are advised to use up-to-date virus checking
> software.

View raw message