Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of fundatureanu.sever@gmail.com
 designates 209.85.216.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAPQV63U+PSbgChQKpGg6+JhKUfZJ--ub0rjmQO9VxYUXp77s0Q@mail.gmail.com>
References: 
 <CAO8KN1U-9N9xZMPWsrhLrNK=NkXy14NahY2+OpA63eCzvbDuZQ@mail.gmail.com>
	<CA+RK=_A6pcEN5SjL_F9Fyjoj9QRKB1k+p-ppeAQGLo5HvdazMQ@mail.gmail.com>
	<CAO8KN1UC4h3YPGi=nCPhftXbc5q_bG5Bd3BA8DCG72xS_y2+zw@mail.gmail.com>
	<CAO8KN1U0aJhS7hcg5Dbz6kF3JYJeuTN9i6-AQCGttvHQCCTHiQ@mail.gmail.com>
	<CAPQV63U+PSbgChQKpGg6+JhKUfZJ--ub0rjmQO9VxYUXp77s0Q@mail.gmail.com>
Date: Mon, 2 Jul 2012 20:58:42 +0200
Message-ID: 
 <CAO8KN1WivSH0p71RNrQxszoKzM15NYcJgs1sZxAsWJFjr87NpQ@mail.gmail.com>
Subject: Re: HBase bulk load through co-processors
From: Sever Fundatureanu <fundatureanu.sever@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=20cf30363d3ff1cdf404c3dd62e9

--20cf30363d3ff1cdf404c3dd62e9
Content-Type: text/plain; charset=ISO-8859-1

I agree that increasing the timeout is not the best option, I will work
both on better balancing the load and maybe doing it in increments like you
suggested. However for now I want a quick fix to the problem.

Just to see if I understand this right: a zookeeper node redirects my
client to a region server node and then my client talk directly to this
region server; now the timeout happens on the client while talking to the
RS right? It expects some kind of confirmation and it times out.. if this
is the case how can I increase this timeout? I only found in the
documentation "zookeeper.session.timeout" which is the timeout between
zookeeper and HBase.

Thanks,
Sever

On Mon, Jul 2, 2012 at 8:19 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Hi Sever,
>
> It seems one of the nodes in your cluster is overwhelmed with the load
> you are giving him.
>
> So IMO, you have two options here:
> First, you can try to reduce the load. I mean, split the bulk in
> multiple smaller bulks and load them one by one to give the time to
> your cluster to dispatch it correctly.
> Second, you can inscreade the timeone from 60s to 120s. But you might
> face the same issue with 120s so  I really recommand the fist option.
>
> JM
>
> 2012/7/2, Sever Fundatureanu <fundatureanu.sever@gmail.com>:
> > Can someone please help me with this?
> >
> > Thanks,
> > Sever
> >
> > On Tue, Jun 26, 2012 at 8:14 PM, Sever Fundatureanu <
> > fundatureanu.sever@gmail.com> wrote:
> >
> >> My keys are built of 4  8-byte Ids. I am currently doing the load with
> MR
> >> but I get a timeout when doing the loadIncrementalFiles call:
> >>
> >> 12/06/24 21:29:01 ERROR mapreduce.LoadIncrementalHFiles: Encountered
> >> unrecoverable error from region server
> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> >> attempts=10, exceptions:
> >> Sun Jun 24 21:29:01 CEST 2012,
> >> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$3@4699ecf9,
> >> java.net.SocketTimeoutException: Call to das3002.cm.cluster/
> >> 10.141.0.79:60020
> >> failed on socket timeout exception: java.net.SocketTimeoutException:
> >> 60000
> >> millis timeout while waiting for channel to be ready for read. ch :
> >> java.nio.channels.SocketChannel[co
> >> nnected local=/10.141.0.254:43240 remote=das3002.cm.cluster/
> >> 10.141.0.79:60020]
> >>
> >>         at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1345)
> >>         at
> >>
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.tryAtomicRegionLoad(LoadIncrementalHFiles.java:487)
> >>         at
> >>
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$1.call(LoadIncrementalHFiles.java:275)
> >>         at
> >>
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$1.call(LoadIncrementalHFiles.java:273)
> >>         at
> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>         at java.lang.Thread.run(Thread.java:662)
> >> 12/06/24 21:30:52 ERROR mapreduce.LoadIncrementalHFiles: Encountered
> >> unrecoverable error from region server
> >>
> >> Is there a way in which I can increase the timeout period?
> >>
> >> Thank you,
> >>
> >> On Tue, Jun 26, 2012 at 7:05 PM, Andrew Purtell
> >> <apurtell@apache.org>wrote:
> >>
> >>> On Tue, Jun 26, 2012 at 9:56 AM, Sever Fundatureanu
> >>> <fundatureanu.sever@gmail.com> wrote:
> >>> > I have to bulkload 6 tables which contain the same information but
> >>> > with
> >>> a
> >>> > different order to cover all possible access patterns. Would it be a
> >>> good
> >>> > idea to do only one load and use co-processors to populate the other
> >>> > tables, instead of doing the traditional MR bulkload which would
> >>> require 6
> >>> > separate jobs?
> >>>
> >>> Without knowing more than you've said, it seems better to use MR to
> >>> build all input.
> >>>
> >>> Best regards,
> >>>
> >>>    - Andy
> >>>
> >>> Problems worthy of attack prove their worth by hitting back. - Piet
> >>> Hein (via Tom White)
> >>>
> >>
> >>
> >>
> >> --
> >> Sever Fundatureanu
> >>
> >> Vrije Universiteit Amsterdam
> >> E-mail: fundatureanu.sever@gmail.com
> >>
> >
> >
> >
> > --
> > Sever Fundatureanu
> >
> > Vrije Universiteit Amsterdam
> > E-mail: fundatureanu.sever@gmail.com
> >
>


-- 
Sever Fundatureanu

Vrije Universiteit Amsterdam
E-mail: fundatureanu.sever@gmail.com

--20cf30363d3ff1cdf404c3dd62e9--