incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohini Palaniswamy <rohini.adi...@gmail.com>
Subject Re: Out of Memory Error - Plz suggest if a JIRA needs to be created
Date Mon, 13 Aug 2012 18:54:15 GMT
Internally in Yahoo we use a patched library of thrift in our deployments
to solve this issue. Need to get hive to move to the libthrift 0.9 that has
the fix for THRIFT-1468 <https://issues.apache.org/jira/browse/THRIFT-1468>

Regards,
Rohini

On Wed, Aug 8, 2012 at 8:21 PM, Rajesh Balamohan <rajesh.balamohan@gmail.com
> wrote:

> Thanks for the pointers Alan. I believe I need to apply both Hive-3008 and
> the thrift-1468.
>
> Analyzing heapdump points 2 GB of space occupied by TSaslServerTransport.
> I do not see TUGIContainingTransport occupying memory.
>
> ~Rajesh.B
>
>
> On Thu, Aug 9, 2012 at 12:25 AM, Alan Gates <gates@hortonworks.com> wrote:
>
>> This looks like it may be related to HIVE-3008.  I've tested back porting
>> this to Hive 0.9.  I don't know how easy it would be to backport it to Hive
>> 0.8.
>>
>> Alan.
>>
>> On Aug 8, 2012, at 7:33 AM, Rajesh Balamohan wrote:
>>
>> > I took the heapdump and analyzed it. It stems from TsaslServerTransport
>> WeakHahsMap.
>> >
>> > I gues, its related to
>> https://issues.apache.org/jira/browse/THRIFT-1468.
>> >
>> > Are there any fixes or workarounds for this issue in hive 0.8 itself
>> (as we wouldn't be able to move to Hive 0.9 immediately).
>> >
>> > Any pointers would be of great help.
>> >
>> > ~Rajesh.B
>> >
>> > On Wed, Aug 8, 2012 at 7:56 PM, Rajesh Balamohan <
>> rajesh.balamohan@gmail.com> wrote:
>> > Hi All,
>> >
>> > I am currently using HCatalog 0.4 with PIG 0.9.2.
>> >
>> > After 3 or 4 days, I encountered PIG exceptions and digging deep I
>> found that it tries to connect to HCatalog server and gets socket read
>> timeout.
>> >
>> > Looking at the JVM process of Hcatalog server suggest OOM. I have
>> attached the jstat details for further reference.
>> >
>> > At this point in time, I am not sure which usecase causes this OOM. I
>> would post more details as and when I find. Should I raise a JIRA for
>> this?. Plz suggest.
>> >
>> > [rajesh@machineXYZ hcatalog]$ sudo  /usr/java/jdk1.6.0_25/bin/jstat
>> -gcutil 7279 250 9
>> >   S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.38  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.64  99.99  59.76  33781  478.507  6095 2358.672
>> 2837.179
>> >   0.00   0.00  71.64  99.99  59.76  33781  478.507  6096 2358.712
>> 2837.219
>> >   0.00   0.00  71.64  99.99  59.76  33781  478.507  6097 2358.743
>> 2837.249
>> >
>> > --
>> > ~Rajesh.B
>> >
>> >
>> >
>> > --
>> > ~Rajesh.B
>>
>>
>
>
> --
> ~Rajesh.B
>

Mime
View raw message