Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Date: Mon, 1 Dec 2008 18:42:17 -0800 (PST)
From: Andrew Purtell <apurtell@apache.org>
Reply-To: apurtell@apache.org
Subject: Re: Bulk import question.
To: hbase-user@hadoop.apache.org, "Edward J. Yoon" <edwardyoon@apache.org>
In-Reply-To: <eb4706e0812011826wc495982kfe2b4837705a0ca4@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Message-ID: <823889.72529.qm@web65510.mail.ac4.yahoo.com>

Edward,

You are running with insufficient resources -- too little CPU
for your task and too little disk for your data. 

If you are running a mapreduce task and DFS runs out of space
for the temporary files, then you indeed should expect
aberrant job status from the Hadoop job framework, for
example such things as completion status running backwards.

I do agree that under these circumstances HBase daemons
should fail more gracefully, by entering some kind of
degraded read only mode, if DFS is not totally dead. I 
suspect this is already on a to do list somewhere, and I
vaguely recall a jira filed on that topic.

   - Andy


> From: Edward J. Yoon <edwardyoon@apache.org>
> Subject: Re: Bulk import question.
> To: hbase-user@hadoop.apache.org, apurtell@apache.org
> Date: Monday, December 1, 2008, 6:26 PM
> It was by 'Datanode DiskOutOfSpaceException'. But, I
> think daemons should not dead.
> 
> On Wed, Nov 26, 2008 at 1:08 PM, Edward J. Yoon
> <edwardyoon@apache.org> wrote:
> > Hmm. It often occurs to me. I'll check the logs.
> >
> > On Fri, Nov 21, 2008 at 9:46 AM, Andrew Purtell
> <apurtell@yahoo.com> wrote:
> > > I think a 2 node cluster is simply too small for
> > > the full load of everything.
> > >