hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1421) When a region server dies, its log must be distributed to all the new servers that have been assigned its regions
Date Mon, 04 Jun 2007 21:47:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501381

Jim Kellerman commented on HADOOP-1421:

It does not make sense to consolidate old log files when a
HRegionServer starts as it has not been assigned any regions yet.

If a HRegionServer is going down in a controlled fashion, then after
closing all the HRegions it is serving, it should close the HLog and
then split it on a per region basis and store the log file in the
region directory.

It should then report to the master that it is shutting down and
provide the master with a list of regions it was serving so the master
can quickly reassign them.

When a HRegionServer is assigned a region, it should look in the
region directory for the old log file, and apply it if it exists (this
is already a part of starting a HRegion, but we currently do not have
the HLog in the right place).

If a HRegionServer crashes, the master is notified by the lease
timeout. Since the HRegionServer crashed, the master needs to scan all
the regions to find what the region server was serving, and split the
log files.

> When a region server dies, its log must be distributed to all the new servers that have
been assigned its regions
> -----------------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-1421
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1421
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: contrib/hbase
>            Reporter: Jim Kellerman
>            Assignee: Jim Kellerman
> When a region server dies, the master reallocates all the regions it was serving to other
region servers. In order for the regions to be recovered properly, the log from the old region
server must be split up into chunks that contain only the commits for a single region.
> Ideally, if a region server is assigned more than one of the old servers regions, the
log for that server would contain all the commits for all the regions it has been assigned.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message