hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HBASE 3904
Date Tue, 21 Jun 2011 01:20:09 GMT
Here is code from HBaseClient:
      private void handleTimeout(SocketTimeoutException e) throws
IOException {
        if (shouldCloseConnection.get() || !running.get() ||
            remoteId.rpcTimeout > 0) {
          throw e;
        }

BTW I think the following javadoc change is better than my first proposal

If the client receives socket timeout exception, that implies master hasn't
done populating .META. table with new region info.
Meaning HConnection.isTableAvailable() may return true prematurely.
If you don't want to handle the extra complexity, please use createTable()
which waits for all the regions to come online.

Thanks

On Mon, Jun 20, 2011 at 5:40 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Shouldn't the ipc client ping be keeping the IPC alive?
>
> On Mon, Jun 20, 2011 at 5:38 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > Can we wrap up 3904 with patch v5 and the following addition to javadoc
> of
> > createTableAsync() ?
> >
> > If the client receives socket timeout exception, that implies master
> hasn't
> > done populating .META. table with new region info.
> > Meaning HConnection.isTableAvailable() may return true prematurely.
> >
> > On Mon, Jun 20, 2011 at 3:54 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
> >
> >> Well in that case you know how many regions you expect, we could catch
> >> the socket exception and then check .META. until it's done or return
> >> the call right away and keep checking .META.
> >>
> >> J-D
> >>
> >> On Mon, Jun 20, 2011 at 3:41 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> > Vidhyashankar did encounter socket timeout.
> >> > I think even with HBASE-4010 implemented, 70k region table creation
> would
> >> > still produce socket timeout.
> >> >
> >> > On Mon, Jun 20, 2011 at 3:30 PM, Jean-Daniel Cryans <
> jdcryans@apache.org
> >> >wrote:
> >> >
> >> >> Could be a socket timeout, was able to get one with just 200 regions,
> >> >> then calling isTableAvailable would trigger the case that was
> >> >> described.
> >> >>
> >> >> J-D
> >> >>
> >> >> On Mon, Jun 20, 2011 at 3:28 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
> >> >> > J-D:
> >> >> > What's your assessment of why isTableAvailable() was inaccurate
for
> a
> >> >> table
> >> >> > with 70k regions ?
> >> >> >
> >> >> > Thanks
> >> >> >
> >> >> > On Mon, Jun 20, 2011 at 3:01 PM, Jean-Daniel Cryans <
> >> jdcryans@apache.org
> >> >> >wrote:
> >> >> >
> >> >> >> That's not how it works, whether the createTable call is sync
or
> not
> >> >> >> the regions are first created in .META. before the call can
> return.
> >> >> >>
> >> >> >> And looking at that code, it seems it's highly inefficient.
We
> should
> >> >> >> not change the enable flag for every region, we should batch
put
> all
> >> >> >> the .META. rows and we should not open a new HLog per new
region.
> >> >> >>
> >> >> >> J-D
> >> >> >>
> >> >> >> On Mon, Jun 20, 2011 at 2:55 PM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> >> >> >> > From Vidhyashankar @ 20/May/11 05:53:
> >> >> >> > Hence there might be a case when all regions are indeed
fully
> >> assigned
> >> >> in
> >> >> >> > META but it is just that the master is yet to populate
META with
> >> the
> >> >> rest
> >> >> >> of
> >> >> >> > the regions.
> >> >> >> >
> >> >> >> > The above means scanning .META. alone wouldn't guarantee
that
> all
> >> the
> >> >> >> > regions of the table are assigned.
> >> >> >> >
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message