hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: HBASE 3904
Date Mon, 20 Jun 2011 21:05:22 GMT
Why not the following pseudo-code for isTableAvailable:

scanMeta(table, Set<RegionId> online, Set<RegionId> offline) {
  scan meta, inserting into the above sets any regions found for the
table in question
}

Set<RegionId> everSeenOnline = new Set();
while (true) {
  Set<RegionId> online = new Set();
  Set<RegionId> offline = new Set();
  scanMeta(table, online, offline);
  everSeenOnline.addAll(online);
  Set<RegionId> stillWaiting = Sets.setDifference(offline, everSeenOnline)
  if (stillWaiting.isEmpty()) return;
  sleep;
}

i.e we wait until we have no offline regions that we've never seen online

On Mon, Jun 20, 2011 at 10:27 AM, Stack <stack@duboce.net> wrote:
> Yeah, its not intuitive but failing your passing isTableAvailable the
> knowledge it needs, then its going to have to do heuristics to figure
> table deploy.  We can work on what these will look like -- region is
> open but not split, no region opened in last N minutes or seconds --
> but I think it will always be an imprecise especially when you are up
> in Vidhya's 70k regions range.
>
> St.Ack
>
> On Sun, Jun 19, 2011 at 9:02 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> I put patch v5 on 3904.
>> When user calls HBaseAdmin.createTableAsync() to create the table,
>> HCM.isTableAvailable() is still vulnerable because currently it doesn't
>> known the desired number of regions.
>> We can add a new parameter to HCM.isTableAvailable() but
>> 1. HConnection needs to be modified to accommodate this change
>> 2. I personally think it is not intuitive.
>>
>> Comments are welcome.
>>
>> On Fri, Jun 17, 2011 at 8:48 AM, Stack <stack@duboce.net> wrote:
>>
>>> On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> > HBA.createTable() takes too long to execute. Client receives Socket
>>> timeout
>>> > exception.
>>>
>>> This is because the create is running in the master?  Can we have
>>> waiting happen client-side over in HBaseAdmin?
>>>
>>> > Client calls HCM.isTableAvailable() which would report inaccurate status.
>>> > I want to get other developers' opinion on whether HCM.isTableAvailable()
>>> > should be accurate.
>>> >
>>>
>>> Do we need two forms?  An isTableAvailable that will looks at table
>>> and if all regions are online reports true and then another
>>> isTableAvailable where you know how many regions there are supposed to
>>> be in the table and it returns true if that count of regions is open
>>> and deployed?
>>>
>>> St.Ack
>>>
>>
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message