hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HBASE 3904
Date Mon, 20 Jun 2011 21:30:37 GMT
Can we simplify the following with this:
1st iteration of meta scanning, calculate numRegs = number of offline
regions + number of online regions (for underlying table)
subsequent iterations of meta scanning, if number of online regions (for
underlying table) reaches numRegs, break.

Cheers

On Mon, Jun 20, 2011 at 2:05 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Why not the following pseudo-code for isTableAvailable:
>
> scanMeta(table, Set<RegionId> online, Set<RegionId> offline) {
>  scan meta, inserting into the above sets any regions found for the
> table in question
> }
>
> Set<RegionId> everSeenOnline = new Set();
> while (true) {
>  Set<RegionId> online = new Set();
>  Set<RegionId> offline = new Set();
>  scanMeta(table, online, offline);
>  everSeenOnline.addAll(online);
>  Set<RegionId> stillWaiting = Sets.setDifference(offline, everSeenOnline)
>  if (stillWaiting.isEmpty()) return;
>  sleep;
> }
>
> i.e we wait until we have no offline regions that we've never seen online
>
> On Mon, Jun 20, 2011 at 10:27 AM, Stack <stack@duboce.net> wrote:
> > Yeah, its not intuitive but failing your passing isTableAvailable the
> > knowledge it needs, then its going to have to do heuristics to figure
> > table deploy.  We can work on what these will look like -- region is
> > open but not split, no region opened in last N minutes or seconds --
> > but I think it will always be an imprecise especially when you are up
> > in Vidhya's 70k regions range.
> >
> > St.Ack
> >
> > On Sun, Jun 19, 2011 at 9:02 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> I put patch v5 on 3904.
> >> When user calls HBaseAdmin.createTableAsync() to create the table,
> >> HCM.isTableAvailable() is still vulnerable because currently it doesn't
> >> known the desired number of regions.
> >> We can add a new parameter to HCM.isTableAvailable() but
> >> 1. HConnection needs to be modified to accommodate this change
> >> 2. I personally think it is not intuitive.
> >>
> >> Comments are welcome.
> >>
> >> On Fri, Jun 17, 2011 at 8:48 AM, Stack <stack@duboce.net> wrote:
> >>
> >>> On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>> > HBA.createTable() takes too long to execute. Client receives Socket
> >>> timeout
> >>> > exception.
> >>>
> >>> This is because the create is running in the master?  Can we have
> >>> waiting happen client-side over in HBaseAdmin?
> >>>
> >>> > Client calls HCM.isTableAvailable() which would report inaccurate
> status.
> >>> > I want to get other developers' opinion on whether
> HCM.isTableAvailable()
> >>> > should be accurate.
> >>> >
> >>>
> >>> Do we need two forms?  An isTableAvailable that will looks at table
> >>> and if all regions are online reports true and then another
> >>> isTableAvailable where you know how many regions there are supposed to
> >>> be in the table and it returns true if that count of regions is open
> >>> and deployed?
> >>>
> >>> St.Ack
> >>>
> >>
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message