Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:date:from:user-agent:mime-version:to:subject:
	references:in-reply-to:content-type:content-transfer-encoding;
	b=RQRJeWZo5nCLiwCvnHhAaMHDF9i3a1hXSKZh4iqKbAqO4gglwjWNXGqbQFPdM63Z
Message-ID: <498A2582.605@yahoo-inc.com>
Date: Wed, 04 Feb 2009 15:32:18 -0800
From: Raghu Angadi <rangadi@yahoo-inc.com>
User-Agent: Thunderbird 2.0.0.19 (Windows/20081209)
MIME-Version: 1.0
To: core-user@hadoop.apache.org
Subject: Re: problem with completion notification from block movement
References: <1233334799.5164.80.camel@awol.kleinpaste.org>
	 <314098690902011758u2ac19fbev8fd851757eb6bcff@mail.gmail.com>
 <1233599004.16154.130.camel@awol.kleinpaste.org>
In-Reply-To: <1233599004.16154.130.camel@awol.kleinpaste.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Karl Kleinpaste wrote:
> On Sun, 2009-02-01 at 17:58 -0800, jason hadoop wrote:
>> The Datanode's use multiple threads with locking and one of the
>> assumptions is that the block report (1ce per hour by default) takes
>> little time. The datanode will pause while the block report is running
>> and if it happens to take a while weird things start to happen.
> 
> Thank you for responding, this is very informative for us.
> 
> Having looked through the source code with a co-worker regarding
> periodic scan and then checking the logs once again, we find that we are
> finding reports of this sort:
> 
> BlockReport of 1158499 blocks got processed in 308860 msecs
> BlockReport of 1159840 blocks got processed in 237925 msecs
> BlockReport of 1161274 blocks got processed in 177853 msecs
> BlockReport of 1162408 blocks got processed in 285094 msecs
> BlockReport of 1164194 blocks got processed in 184478 msecs
> BlockReport of 1165673 blocks got processed in 226401 msecs
> 
> The 3rd of these exactly straddles the particular example timeline I
> discussed in my original email about this question.  I suspect I'll find
> more of the same as I look through other related errors.

You could ask for "complete fix" in 
https://issues.apache.org/jira/browse/HADOOP-4584 . I don't think 
current patch there fixes your problem.

Raghu.

> --karl
>