Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 651EAF9F8 for ; Mon, 25 Mar 2013 16:30:50 +0000 (UTC) Received: (qmail 66871 invoked by uid 500); 25 Mar 2013 16:30:45 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 66667 invoked by uid 500); 25 Mar 2013 16:30:45 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 66654 invoked by uid 99); 25 Mar 2013 16:30:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Mar 2013 16:30:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of itretyakov@griddynamics.com designates 209.85.215.53 as permitted sender) Received: from [209.85.215.53] (HELO mail-la0-f53.google.com) (209.85.215.53) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Mar 2013 16:30:41 +0000 Received: by mail-la0-f53.google.com with SMTP id fr10so11542275lab.26 for ; Mon, 25 Mar 2013 09:30:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=griddynamics.com; s=google; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=HhsNh/5MGPsxKDNRzLdaVjizcSgOhOdzPKr0Uku+YU8=; b=NHI/0ZvzukN5HKPZv15KASzRdwPDgxRg6sG4f0O1pfsSZArRLC5yYyNyL1nMIQpEfl 3ncx6WAzYH1IOrMeJgXI9HWxs9/Gs0o90TmolObb8c8K7pDwrft1Ff2t9LwlFNG6q5WH W9aRkmEzGVyaEmW/4ef6OAnin77rO4zH0Sick= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type:x-gm-message-state; bh=HhsNh/5MGPsxKDNRzLdaVjizcSgOhOdzPKr0Uku+YU8=; b=CCw29EUQ/ehw3ZaPuzCAt01zuNzOS/9BYd44i9DF4BXx56WFpPQM2qVQGLMaGS9qOl dciDpfOL2LE9EFhCDuN2Ga/dVPjhq2tJjAHtURyXPJOB7bUcrYk3pA51IMYVQ1X/gA0x XXNVuDexJjCrtyrgcj95GGUttnzcDaqRzEP1aFwfUHsOPW6/OiHT4sXve/XHGXwXSEJ8 dnSLTqMPz3qkCWgIIuK9/2KTE6BcXDL+b/UrpoR8RaTdTzYJ0xBDvvxCy6tLl+hq1tjb A50ZmLIrMkTQifWZ1rV0L/at5PSxjFUIB0brpkZQWOWACmYlp3w0zb4D8xBKoiodCO1V TfQQ== X-Received: by 10.112.10.102 with SMTP id h6mr6133858lbb.75.1364229019355; Mon, 25 Mar 2013 09:30:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.114.26.165 with HTTP; Mon, 25 Mar 2013 09:29:59 -0700 (PDT) In-Reply-To: References: From: Ivan Tretyakov Date: Mon, 25 Mar 2013 20:29:59 +0400 Message-ID: Subject: Re: DataNode heartbeat average time peaks To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=e0cb4efe305017706104d8c252a3 X-Gm-Message-State: ALoCoQnM7AdWKwOixxy/OLh8zDxlXNFd/k9Htv6QD+6l1U1CQZa9Xk/Z9OtQZStITQdttUtGHyQe X-Virus-Checked: Checked by ClamAV on apache.org --e0cb4efe305017706104d8c252a3 Content-Type: text/plain; charset=ISO-8859-1 Thanks Harsh! My image size is about 3.1 Gb. Yes, I think feature from HDFS-1457 is what I need, but unfortunately it is not available in version of hadoop we use. What kind of risks pose by these peaks. On Mon, Mar 25, 2013 at 7:31 PM, Harsh J wrote: > What's your fsimage size? If its too high you would want to control > the checkpoint transfer bandwidth to not affect the load at the NN. > This is available via the JIRA HDFS-1457. > > > On Mon, Mar 25, 2013 at 8:13 PM, Ivan Tretyakov > wrote: > > Hi! > > > > We see DataNode heartbeat average time peaks in Ganglia up to 20-70 > seconds > > while SecondaryNameNode performs checkpointing. > > See attached screenshots please. > > > > I would like to clarify if it is Ok, or not. And what kind of > consequences > > and risks it could bring up. > > > > -- > > Best Regards > > Ivan Tretyakov > > > > -- > Harsh J > -- Best Regards Ivan Tretyakov --e0cb4efe305017706104d8c252a3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks Harsh!

My image size is about 3.1 Gb.
Y= es, I think feature from HDFS-1457 is what I need, but=A0unfortunately=A0it= is not available in version of hadoop we use.

Wha= t kind of risks pose by these peaks.=A0


On Mon, Mar 25, 2013 at = 7:31 PM, Harsh J <harsh@cloudera.com> wrote:
What's your fsimage size? If its too high you would want to control
the checkpoint transfer bandwidth to not affect the load at the NN.
This is available via the JIRA HDFS-1457.


On Mon, Mar 25, 2013 at 8:13 PM, Ivan Tretyakov
<itretyakov@griddynamics.= com> wrote:
> Hi!
>
> We see DataNode heartbeat average time peaks in Ganglia up to 20-70 se= conds
> while SecondaryNameNode performs checkpointing.
> See attached screenshots please.
>
> I would like to clarify if it is Ok, or not. And what kind of conseque= nces
> and risks it could bring up.
>
> --
> Best Regards
> Ivan Tretyakov



--
Harsh J



--
Best Re= gards
Ivan Tretyakov

--e0cb4efe305017706104d8c252a3--