incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <rajesh.balamo...@gmail.com>
Subject Re: Out of Memory Error - Plz suggest if a JIRA needs to be created
Date Tue, 14 Aug 2012 00:24:55 GMT
Thanks Rohini, we are planning to apply this patch on same lines.

~Rajesh.B

On Tue, Aug 14, 2012 at 12:24 AM, Rohini Palaniswamy <
rohini.aditya@gmail.com> wrote:

> Internally in Yahoo we use a patched library of thrift in our deployments
> to solve this issue. Need to get hive to move to the libthrift 0.9 that has
> the fix for THRIFT-1468<https://issues.apache.org/jira/browse/THRIFT-1468>
>
> Regards,
> Rohini
>
>
> On Wed, Aug 8, 2012 at 8:21 PM, Rajesh Balamohan <
> rajesh.balamohan@gmail.com> wrote:
>
>> Thanks for the pointers Alan. I believe I need to apply both Hive-3008
>> and the thrift-1468.
>>
>> Analyzing heapdump points 2 GB of space occupied by TSaslServerTransport.
>> I do not see TUGIContainingTransport occupying memory.
>>
>> ~Rajesh.B
>>
>>
>> On Thu, Aug 9, 2012 at 12:25 AM, Alan Gates <gates@hortonworks.com>wrote:
>>
>>> This looks like it may be related to HIVE-3008.  I've tested back
>>> porting this to Hive 0.9.  I don't know how easy it would be to backport it
>>> to Hive 0.8.
>>>
>>> Alan.
>>>
>>> On Aug 8, 2012, at 7:33 AM, Rajesh Balamohan wrote:
>>>
>>> > I took the heapdump and analyzed it. It stems from
>>> TsaslServerTransport WeakHahsMap.
>>> >
>>> > I gues, its related to
>>> https://issues.apache.org/jira/browse/THRIFT-1468.
>>> >
>>> > Are there any fixes or workarounds for this issue in hive 0.8 itself
>>> (as we wouldn't be able to move to Hive 0.9 immediately).
>>> >
>>> > Any pointers would be of great help.
>>> >
>>> > ~Rajesh.B
>>> >
>>> > On Wed, Aug 8, 2012 at 7:56 PM, Rajesh Balamohan <
>>> rajesh.balamohan@gmail.com> wrote:
>>> > Hi All,
>>> >
>>> > I am currently using HCatalog 0.4 with PIG 0.9.2.
>>> >
>>> > After 3 or 4 days, I encountered PIG exceptions and digging deep I
>>> found that it tries to connect to HCatalog server and gets socket read
>>> timeout.
>>> >
>>> > Looking at the JVM process of Hcatalog server suggest OOM. I have
>>> attached the jstat details for further reference.
>>> >
>>> > At this point in time, I am not sure which usecase causes this OOM. I
>>> would post more details as and when I find. Should I raise a JIRA for
>>> this?. Plz suggest.
>>> >
>>> > [rajesh@machineXYZ hcatalog]$ sudo  /usr/java/jdk1.6.0_25/bin/jstat
>>> -gcutil 7279 250 9
>>> >   S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
>>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.64  99.99  59.76  33781  478.507  6095 2358.672
>>> 2837.179
>>> >   0.00   0.00  71.64  99.99  59.76  33781  478.507  6096 2358.712
>>> 2837.219
>>> >   0.00   0.00  71.64  99.99  59.76  33781  478.507  6097 2358.743
>>> 2837.249
>>> >
>>> > --
>>> > ~Rajesh.B
>>> >
>>> >
>>> >
>>> > --
>>> > ~Rajesh.B
>>>
>>>
>>
>>
>> --
>> ~Rajesh.B
>>
>
>


-- 
~Rajesh.B

Mime
View raw message