hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10347) HRegionInfo changes for adding replicaId and MetaEditor/MetaReader changes for region replicas
Date Tue, 04 Feb 2014 07:28:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13890462#comment-13890462
] 

Hadoop QA commented on HBASE-10347:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12626822/hbase-10347_redo_v5.patch
  against trunk revision .
  ATTACHMENT ID: 12626822

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 17 new or modified
tests.

    {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

    {color:green}+1 hadoop1.1{color}.  The patch compiles against the hadoop 1.1 profile.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 5 warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 1 new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +   * info:seqnumDuringOpen_<replicaId> => contains seqNum (in binary long form)
for the region at the time
+   * info:splitA              => contains a serialized HRI for the first daughter region
if the region
+   * info:splitB              => contains a serialized HRI for the second daughter region
if the region
+   * info:mergeB              => contains a serialized HRI for the second parent region
if the region
+  private static final Pattern SERVER_COLUMN_PATTERN = Pattern.compile("^server(_[0-9a-fA-F]{4})?$");
+    return HRegionInfo.parseFromOrNull(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength());
+            hConnection.updateCachedLocations(this.tableName, resultsForRS.getKey(), row.getRow(),
result, server);
+        return ((HConnectionImplementation)connection).metaCache.getNumberOfCachedRegionLocations(tableName);
+      updateCachedLocations(tableName, source == null ? null : source.getRegionInfo().getRegionName()
+  private ConcurrentSkipListMap<byte[], RegionLocations> getTableLocations(final TableName
tableName) {

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.coprocessor.TestMasterObserver
                  org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
                  org.apache.hadoop.hbase.regionserver.TestRegionServerNoMaster
                  org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
                  org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
                  org.apache.hadoop.hbase.regionserver.TestEncryptionRandomKeying
                  org.apache.hadoop.hbase.security.access.TestTablePermissions
                  org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat
                  org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
                  org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
                  org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2
                  org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
                  org.apache.hadoop.hbase.client.TestMultiParallel
                  org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
                  org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
                  org.apache.hadoop.hbase.client.TestTableSnapshotScanner
                  org.apache.hadoop.hbase.client.TestMetaScanner
                  org.apache.hadoop.hbase.client.TestAdmin
                  org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster
                  org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
                  org.apache.hadoop.hbase.master.TestRollingRestart
                  org.apache.hadoop.hbase.TestFullLogReconstruction
                  org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook
                  org.apache.hadoop.hbase.coprocessor.TestBigDecimalColumnInterpreter
                  org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
                  org.apache.hadoop.hbase.util.TestHBaseFsckEncryption
                  org.apache.hadoop.hbase.TestGlobalMemStoreSize
                  org.apache.hadoop.hbase.master.TestMasterTransitions
                  org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
                  org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
                  org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
                  org.apache.hadoop.hbase.TestNamespace
                  org.apache.hadoop.hbase.master.TestMasterFailover
                  org.apache.hadoop.hbase.master.TestTableLockManager
                  org.apache.hadoop.hbase.client.TestHCM
                  org.apache.hadoop.hbase.regionserver.TestCompactionState
                  org.apache.hadoop.hbase.client.TestFromClientSide
                  org.apache.hadoop.hbase.rest.TestTableResource
                  org.apache.hadoop.hbase.client.TestFromClientSide3
                  org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
                  org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1
                  org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
                  org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation
                  org.apache.hadoop.hbase.master.TestAssignmentManager
                  org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
                  org.apache.hadoop.hbase.master.TestDistributedLogSplitting
                  org.apache.hadoop.hbase.regionserver.TestTags
                  org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
                  org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics
                  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
                  org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
                  org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
                  org.apache.hadoop.hbase.master.TestMaster
                  org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
                  org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
                  org.apache.hadoop.hbase.migration.TestNamespaceUpgrade
                  org.apache.hadoop.hbase.util.TestHBaseFsck
                  org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8587//console

This message is automatically generated.

> HRegionInfo changes for adding replicaId and MetaEditor/MetaReader changes for region
replicas
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-10347
>                 URL: https://issues.apache.org/jira/browse/HBASE-10347
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Region Assignment
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 0.99.0
>
>         Attachments: hbase-10347_redo_v4.patch, hbase-10347_redo_v5.patch
>
>
> As per parent jira, the cleanest way to add region replicas we think is to actually create
one more region per replica per primary region. So for example, if a table has 10 regions
with replication = 3, the table would indeed be created with 30 regions. These regions will
be handled and assigned individually for AM purposes. 
> We can add replicaId to HRegionInfo to indicate the replicaId, and use this to differentiate
different replicas of the same region. So, primary replica would have replicaId = 0, and the
others will have replicaId > 0. 
> These replicas will share the same regionId prefix, but differ in an appended replicaId.
The primary will not contain the replicaId so that no changes would be needed for existing
tables. 
> In meta, the replica regions are kept in the same row as the primary ( so for above example,
there will be 10 rows in meta). The servers for the replicas are kept in columns like "server+replicaId".




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message