hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kannan Muthukkaruppan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6752) On region server failure, serve writes and timeranged reads during the log split
Date Mon, 10 Sep 2012 22:56:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452549#comment-13452549
] 

Kannan Muthukkaruppan commented on HBASE-6752:
----------------------------------------------

There might be a bunch of nitty gritties to be ironed out-- but being able to take writes
nearly all the time would be a very nice win. So big +1 for exploring this effort. Will throw
out a few things that come to mind:

* We do want the old edits to come in the correct order of sequence ids (i.e. be considered
older than the newer puts that arrive when the region is in recovery mode, correct)? So, we
somehow need to cheaply find the correct sequence id to use for the new puts. It needs to
be bigger than sequence ids for all the edits for that region in the log files. So maybe all
that's needed here is to open recover the latest log file, and scan it to find the last sequence
id?

* Picking a winner among duplicates in two files relies on using sequence id of the HFile
as a tie-break. And therefore, today, compactions always pick  a dense subrange of files order
by sequence ids. That is if we have HFiles a, b, c, d, e sorted by sequence id, we might compact
a,b,c or c,d,e but never say a,d,e. With this new scheme, we should take care that we don't
violate this property. The old data should correctly be recovered into HFiles with the correct
sequence id.. and even if newer data has been flushed before the recovery is complete we shouldn't
compact those newer files with older HFiles given that some new files are supposed to come
in between (after recovery).




                
> On region server failure, serve writes and timeranged reads during the log split
> --------------------------------------------------------------------------------
>
>                 Key: HBASE-6752
>                 URL: https://issues.apache.org/jira/browse/HBASE-6752
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.96.0
>            Reporter: nkeywal
>            Priority: Minor
>
> Opening for write on failure would mean:
> - Assign the region to a new regionserver. It marks the region as recovering
>   -- specific exception returned to the client when we cannot server.
>   -- allow them to know where they stand. The exception can include some time information
(failure stated on: ...)
>   -- allow them to go immediately on the right regionserver, instead of retrying or calling
the region holding meta to get the new address
>      => save network calls, lower the load on meta.
> - Do the split as today. Priority is given to region server holding the new regions
>   -- help to share the load balancing code: the split is done by region server considered
as available for new regions
>   -- help locality (the recovered edits are available on the region server) => lower
the network usage
> - When the split is finished, we're done as of today
> - while the split is progressing, the region server can
>  -- serve writes
>    --- that's useful for all application that need to write but not read immediately:
>    --- whatever logs events to analyze them later
>    --- opentsdb is a perfect example.   
>  -- serve reads if they have a compatible time range. For heavily used tables, it could
be an help, because:
>    --- we can expect to have a few minutes of data only (as it's loaded)
>    --- the heaviest queries, often accepts a few -or more- minutes delay. 
> Some "What if":
> 1) the split fails
> => Retry until it works. As today. Just that we serves writes. We need to know (as
today) that the region has not recovered if we fail again.
> 2) the regionserver fails during the split
> => As 1 and as of today/
> 3) the regionserver fails after the split but before the state change to fully available.
> => New assign. More logs to split (the ones already dones and the new ones).
> 4) the assignment fails
> => Retry until it works. As today.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message