hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HBASE 3904
Date Mon, 20 Jun 2011 21:55:58 GMT
>From Vidhyashankar @ 20/May/11 05:53:
Hence there might be a case when all regions are indeed fully assigned in
META but it is just that the master is yet to populate META with the rest of
the regions.

The above means scanning .META. alone wouldn't guarantee that all the
regions of the table are assigned.

On Mon, Jun 20, 2011 at 2:05 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Why not the following pseudo-code for isTableAvailable:
>
> scanMeta(table, Set<RegionId> online, Set<RegionId> offline) {
>  scan meta, inserting into the above sets any regions found for the
> table in question
> }
>
> Set<RegionId> everSeenOnline = new Set();
> while (true) {
>  Set<RegionId> online = new Set();
>  Set<RegionId> offline = new Set();
>  scanMeta(table, online, offline);
>  everSeenOnline.addAll(online);
>  Set<RegionId> stillWaiting = Sets.setDifference(offline, everSeenOnline)
>  if (stillWaiting.isEmpty()) return;
>  sleep;
> }
>
> i.e we wait until we have no offline regions that we've never seen online
>
> On Mon, Jun 20, 2011 at 10:27 AM, Stack <stack@duboce.net> wrote:
> > Yeah, its not intuitive but failing your passing isTableAvailable the
> > knowledge it needs, then its going to have to do heuristics to figure
> > table deploy.  We can work on what these will look like -- region is
> > open but not split, no region opened in last N minutes or seconds --
> > but I think it will always be an imprecise especially when you are up
> > in Vidhya's 70k regions range.
> >
> > St.Ack
> >
> > On Sun, Jun 19, 2011 at 9:02 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> I put patch v5 on 3904.
> >> When user calls HBaseAdmin.createTableAsync() to create the table,
> >> HCM.isTableAvailable() is still vulnerable because currently it doesn't
> >> known the desired number of regions.
> >> We can add a new parameter to HCM.isTableAvailable() but
> >> 1. HConnection needs to be modified to accommodate this change
> >> 2. I personally think it is not intuitive.
> >>
> >> Comments are welcome.
> >>
> >> On Fri, Jun 17, 2011 at 8:48 AM, Stack <stack@duboce.net> wrote:
> >>
> >>> On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>> > HBA.createTable() takes too long to execute. Client receives Socket
> >>> timeout
> >>> > exception.
> >>>
> >>> This is because the create is running in the master?  Can we have
> >>> waiting happen client-side over in HBaseAdmin?
> >>>
> >>> > Client calls HCM.isTableAvailable() which would report inaccurate
> status.
> >>> > I want to get other developers' opinion on whether
> HCM.isTableAvailable()
> >>> > should be accurate.
> >>> >
> >>>
> >>> Do we need two forms?  An isTableAvailable that will looks at table
> >>> and if all regions are online reports true and then another
> >>> isTableAvailable where you know how many regions there are supposed to
> >>> be in the table and it returns true if that count of regions is open
> >>> and deployed?
> >>>
> >>> St.Ack
> >>>
> >>
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message