hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jl...@streamy.com>
Subject RE: configuration fo map reduce
Date Tue, 25 Nov 2008 16:43:48 GMT
Yes it would be very useful to have more information.

Regarding your configuration, the most common setup is to put your Namenode,
HMaster, and JobTracker on a single node (this node becomes the only single
point of failure).

You would then have three matching servers running datanode, regionserver,
tasktracker.

One very important factor is the configuration of your tasktrackers.  What
do you have set as maximum maps/reduces per tasktracker?  (in hadoop conf,
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum).  With the hbase master co-hosted
with a regionserver, datanode, tasktracker, as well as a running MR job, it
would not be surprising to see something go wrong.  With 4 cores, if you had
that server setup with all four on one server, you'd only want a single map
or single reduce running at any one time to prevent starvation.

As JD said, do you have any logs?  Please turn your hbase logging to DEBUG
(in log4j.properties) and send us the logs from the crash.  You also might
change your configuration as above and rerun again with DEBUG.

JG


> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
> Daniel Cryans
> Sent: Tuesday, November 25, 2008 7:38 AM
> To: hbase-user@hadoop.apache.org
> Subject: Re: configuration fo map reduce
> 
> What kind of crash? Do you have a stack trace from the logs? What kind
> of MR
> job was that (source, size, etc)? What's your current configuration
> (*-site.xml)?
> 
> Without any of these it would be hard to assert anything.
> 
> J-D
> 
> 2008/11/25 ROL <dmkd@rol.ru>
> 
> > Hi.
> >
> > I have hbase master crashed after BulkImport mr job;
> >
> > is it possible that's because my configuration is bad? Can you
> > suggest configuration for map reduce jobs with that:
> >
> > -hadoop 0.18.2
> > -hbase 0.18.1
> >
> > 4 nodes:
> >
> > 4CPU, 4Gb RAM
> >
> >
> > my current configuration:
> >
> > 1 node : namenode, datanode, tasktracker, hbase region
> > 2 node : jobtracker, datanode, tasktracker, hbase region
> > 3 node : hbase master, datanode, tasktracker, hbase rehion
> > 4 node : datanode, tasktracker, hbase region
> >
> >
> >
> > --
> > С уважением,
> >  ROL                          mailto:dmkd@rol.ru
> >
> >


Mime
View raw message