hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jason hadoop <jason.had...@gmail.com>
Subject Re: du command hangs the machine
Date Fri, 01 May 2009 21:27:57 GMT
There is a particular problem with XFS that I have only seen on PAE kernels
that can cause XFS deadlocks, on top of the long duration du issue.

We never had a solution but it only seemed to happen on the machines with
the PAE kernels (32 bit kernels with larger than 4gig of physical ram in the
machine).


On Fri, May 1, 2009 at 1:33 PM, Owen O'Malley <omalley@apache.org> wrote:

>
> On May 1, 2009, at 1:13 PM, Anshuman Sachdeva wrote:
>
>  Hi All,
>>       My engineers have seen this issue. After every few days the node
>> hangs. we are using xfs file system and when ever hadoop runs "du" command
>> and the data on the file system is really high and it some how locks the
>> directory and we end up rebooting the machine.
>>
>> A help or guidance from any one will be helpful
>>
>
> Probably the best way to solve the problem is:
>
> https://issues.apache.org/jira/browse/HADOOP-4998
>
> It would be great to have native libraries that provide the functionality
> we need without forking at all.
>
> -- Owen
>



-- 
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message