hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-7987) Snapshot Manifest file instead of multiple empty files
Date Mon, 04 Mar 2013 09:37:14 GMT
Matteo Bertozzi created HBASE-7987:
--------------------------------------

             Summary: Snapshot Manifest file instead of multiple empty files
                 Key: HBASE-7987
                 URL: https://issues.apache.org/jira/browse/HBASE-7987
             Project: HBase
          Issue Type: Improvement
          Components: snapshots
            Reporter: Matteo Bertozzi
            Priority: Minor


Currently taking a snapshot means creating one empty file for each file in the source table
directory, plus copying the .regioninfo file for each region, the table descriptor file and
a snapshotInfo file.

during the restore or snapshot verification we traverse the filesystem (fs.listStatus()) to
find the snapshot files, and we open the .regioninfo files to get the information.

to avoid hammering the NameNode and having lots of empty files, we can use a manifest file
that contains the list of files and information that we need.
To keep the RS parallelism that we have, each RS can write its own manifest.

{code}
message SnapshotDescriptor {
  required string name;
  optional string table;
  optional int64 creationTime;
  optional Type type;
  optional int32 version;
}

message SnapshotRegionManifest {
  required RegionInfo regionInfo;
  repeated FamilyFiles familyFiles;

  message StoreFile {
    required string name;
    optional Reference reference;
  }

  message FamilyFiles {
    required bytes familyName;
    repeated StoreFile storeFiles;
  }
}
{code}

{code}
/hbase/.snapshot/<snapshotName>
/hbase/.snapshot/<snapshotName>/snapshotInfo
/hbase/.snapshot/<snapshotName>/<tableName>
/hbase/.snapshot/<snapshotName>/<tableName>/tableInfo
/hbase/.snapshot/<snapshotName>/<tableName>/regionManifest(.n)
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message