Return-Path: Delivered-To: apmail-hbase-user-archive@www.apache.org Received: (qmail 5962 invoked from network); 27 Dec 2010 18:05:48 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 27 Dec 2010 18:05:48 -0000 Received: (qmail 35760 invoked by uid 500); 27 Dec 2010 18:05:46 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 35730 invoked by uid 500); 27 Dec 2010 18:05:46 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 35722 invoked by uid 99); 27 Dec 2010 18:05:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Dec 2010 18:05:46 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates 209.85.161.41 as permitted sender) Received: from [209.85.161.41] (HELO mail-fx0-f41.google.com) (209.85.161.41) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Dec 2010 18:05:41 +0000 Received: by fxm12 with SMTP id 12so2786935fxm.14 for ; Mon, 27 Dec 2010 10:05:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=UkU7QrNNkg8pDdCV2HMvGh4tLao2QHqATrVzulGv0ME=; b=LeGtUG+e2Qyup4MZyz6PDnC4+e3aufZkuAWhKsV/0Ce5Ms0LAgErXeVuam9ba3oFl7 Nk3Z/oZOxvKc0Sahv5eWiYJwfRLY1gUdAAV/+Tulf1mcNG6hJ0y+7PqWs86oS3zlWiXG zxDGv6kIRpoq2RHZuBB5VqS5vP3bPy4oESB74= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=bqZUakSs7pA9QqxjaGsHiK6mMuiNthqFrEHnusbmETwCS3xWj3Xv+li1Ah8bcmK9kV FozHCDDB8VqwLHoN8NA9kdpUQxE5gRfM4Wpirb83xKKCZFD96JVmtUZ2EQgK8UmKsQIz vDAY2HqEdqlpHu0s9iScQ5OrV7mY7PuLnEqf8= MIME-Version: 1.0 Received: by 10.223.89.142 with SMTP id e14mr6174123fam.143.1293473041343; Mon, 27 Dec 2010 10:04:01 -0800 (PST) Sender: saint.ack@gmail.com Received: by 10.223.83.9 with HTTP; Mon, 27 Dec 2010 10:04:01 -0800 (PST) In-Reply-To: <815456.29760.qm@web130103.mail.mud.yahoo.com> References: <815456.29760.qm@web130103.mail.mud.yahoo.com> Date: Mon, 27 Dec 2010 10:04:01 -0800 X-Google-Sender-Auth: 84OkvMXgoU3unnsCJ-REz27FaZ4 Message-ID: Subject: Re: RS self-abort when DNs are down From: Stack To: user@hbase.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Hey Otis: Yeah, we're a bit crass when it comes to dealing with exceptions that come up out of HDFS. We'll just abort the server rather than try fancy footwork to get around the outage. HBASE-2183 is about doing a better job of riding over HDFS outage. St.Ack On Sat, Dec 25, 2010 at 11:11 AM, Otis Gospodnetic wrote: > Hello, > > Is this normal: > > 2010-12-25 18:59:48,689 ERROR org.apache.hadoop.hdfs.DFSClient: Exception > closing file /hbase/.logs/example.com,60020,1293204828665/10.208.42.97%3A > 60020.1293302073168 : java.io.IOException: All datanodes 127.0.0.1:50010 = are > bad. Aborting... > java.io.IOException: All datanodes 127.0.0.1:50010 are bad. Aborting... > > =A0I understand the dependency on DN(s), but why completely self-abort? > > In this particular case I restarted the DNs (really 1 of 1 of them total)= , which > automatically "killed" the HBase RS, which I didn't expect. > > Why not just refuse any new Puts until DNs come back? > > Thanks, > Otis > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBas= e > Hadoop ecosystem search :: http://search-hadoop.com/ > >