hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Huy Phan <dac...@gmail.com>
Subject Re: libhdfs with FileSystem cache issue can causes to memory leak ?
Date Tue, 13 Oct 2009 17:40:08 GMT
Hi Eli,
    You're right that the problem is resolved in 0.20 with function 
newInstance(), unfortunately my system's running on Hadoop 0.18.3 and 
i'm still looking for a way to patch this version without affecting the 
current system.

Huy Phan

Eli Collins wrote:
> Hey Huy,
> What version of hadoop are you using?  I think HADOOP-4655 may have
> resolved the issue you're seeing but I think is only in 20 and later.
> Thanks,
> Eli
> On Mon, Oct 12, 2009 at 8:52 PM, Huy Phan <dachuy@gmail.com> wrote:
>> Hi All,
>>    I'm writing a multi-thread application using libhdfs in C, a known issue
>> of HDFS is that the FileSystem API caches FileSystem handles and always
>> returned the same FileSystem handle when called from different threads. It
>> means even though I called hdfsConnect for many times, I should not call
>> hdfsDisconnect in any single thread.
>> This may lead to memory leak on system, do you know any workaround for this
>> issue ?

View raw message