Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 74.125.83.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=K2ParyVeKxiDn8538JM+B/ydeWlAqdiTcDY4YwrluVvIO9uEMmKLfgv1GkbXRuCvqj
         h5KV0PkZtTt/jaV0HJIEBhlJRZypLIvQjX3xEZowBi24rYgZ4tF5hGCPcgHI5KIptcgK
         jVqRSWTaj67OmY/nyhmzHl4J5aC8WU9PfZ63k=
MIME-Version: 1.0
In-Reply-To: <BANLkTimj1DxwR2q_BrzYL2OtO2EWvyeJRg@mail.gmail.com>
References: <BANLkTiks9WM7OhNY1UKARgaNdSXU+9VRVQ@mail.gmail.com>
	<BANLkTikiWOPZjkau7XTu326JHods=diaZw@mail.gmail.com>
	<BANLkTimkfpuOav4K0BE=CXy4AeJVubtbRg@mail.gmail.com>
	<BANLkTimB3HKKNbDEmx9jbH4KdU7qvrSnjw@mail.gmail.com>
	<BANLkTin7zZZ4m2P9GZt+EvuincxypJ8Ctg@mail.gmail.com>
	<BANLkTin6bDBMaC-53cuNEWWbXC++JMSbYg@mail.gmail.com>
	<BANLkTikJrdFBbWhxHEhS5RWgFt7Ug-JZVw@mail.gmail.com>
	<BANLkTi=GcxR-SCVWrJy8D2NAEfL0u-3DKg@mail.gmail.com>
	<BANLkTin3tfsubDEjFK2H3PJc_rW57aM4tg@mail.gmail.com>
	<BANLkTimj1DxwR2q_BrzYL2OtO2EWvyeJRg@mail.gmail.com>
Date: Mon, 20 Jun 2011 14:30:37 -0700
Message-ID: <BANLkTikw-m+0Q_cMtH_J1oia+4KFhPA-4A@mail.gmail.com>
Subject: Re: HBASE 3904
From: Ted Yu <yuzhihong@gmail.com>
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=bcaec521628f3a5f1e04a62b7281

--bcaec521628f3a5f1e04a62b7281
Content-Type: text/plain; charset=ISO-8859-1

Can we simplify the following with this:
1st iteration of meta scanning, calculate numRegs = number of offline
regions + number of online regions (for underlying table)
subsequent iterations of meta scanning, if number of online regions (for
underlying table) reaches numRegs, break.

Cheers

On Mon, Jun 20, 2011 at 2:05 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Why not the following pseudo-code for isTableAvailable:
>
> scanMeta(table, Set<RegionId> online, Set<RegionId> offline) {
>  scan meta, inserting into the above sets any regions found for the
> table in question
> }
>
> Set<RegionId> everSeenOnline = new Set();
> while (true) {
>  Set<RegionId> online = new Set();
>  Set<RegionId> offline = new Set();
>  scanMeta(table, online, offline);
>  everSeenOnline.addAll(online);
>  Set<RegionId> stillWaiting = Sets.setDifference(offline, everSeenOnline)
>  if (stillWaiting.isEmpty()) return;
>  sleep;
> }
>
> i.e we wait until we have no offline regions that we've never seen online
>
> On Mon, Jun 20, 2011 at 10:27 AM, Stack <stack@duboce.net> wrote:
> > Yeah, its not intuitive but failing your passing isTableAvailable the
> > knowledge it needs, then its going to have to do heuristics to figure
> > table deploy.  We can work on what these will look like -- region is
> > open but not split, no region opened in last N minutes or seconds --
> > but I think it will always be an imprecise especially when you are up
> > in Vidhya's 70k regions range.
> >
> > St.Ack
> >
> > On Sun, Jun 19, 2011 at 9:02 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> I put patch v5 on 3904.
> >> When user calls HBaseAdmin.createTableAsync() to create the table,
> >> HCM.isTableAvailable() is still vulnerable because currently it doesn't
> >> known the desired number of regions.
> >> We can add a new parameter to HCM.isTableAvailable() but
> >> 1. HConnection needs to be modified to accommodate this change
> >> 2. I personally think it is not intuitive.
> >>
> >> Comments are welcome.
> >>
> >> On Fri, Jun 17, 2011 at 8:48 AM, Stack <stack@duboce.net> wrote:
> >>
> >>> On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>> > HBA.createTable() takes too long to execute. Client receives Socket
> >>> timeout
> >>> > exception.
> >>>
> >>> This is because the create is running in the master?  Can we have
> >>> waiting happen client-side over in HBaseAdmin?
> >>>
> >>> > Client calls HCM.isTableAvailable() which would report inaccurate
> status.
> >>> > I want to get other developers' opinion on whether
> HCM.isTableAvailable()
> >>> > should be accurate.
> >>> >
> >>>
> >>> Do we need two forms?  An isTableAvailable that will looks at table
> >>> and if all regions are online reports true and then another
> >>> isTableAvailable where you know how many regions there are supposed to
> >>> be in the table and it returns true if that count of regions is open
> >>> and deployed?
> >>>
> >>> St.Ack
> >>>
> >>
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

--bcaec521628f3a5f1e04a62b7281--