accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] keith-turner commented on a change in pull request #575: Add splits to table at time of table creation #573
Date Wed, 05 Sep 2018 17:42:47 GMT
keith-turner commented on a change in pull request #575: Add splits to table at time of table
creation #573
URL: https://github.com/apache/accumulo/pull/575#discussion_r215355339
 
 

 ##########
 File path: server/master/src/main/java/org/apache/accumulo/master/tableOps/CreateDir.java
 ##########
 @@ -47,5 +58,21 @@ public void undo(long tid, Master master) throws Exception {
     VolumeManager fs = master.getFileSystem();
     fs.deleteRecursively(new Path(tableInfo.dir));
 
+    if (tableInfo.initialSplitSize > 0) {
+      SortedSet<Text> dirInfo = Utils
+          .getSortedSetFromFile(master.getInputStream(tableInfo.splitDirsFile), false);
+      for (Text dirname : dirInfo) {
+        fs.deleteRecursively(new Path(dirname.toString()));
+      }
+    }
+  }
+
+  private void createTabletDirectories(VolumeManager fs, SortedSet<Text> dirInfo)
+      throws IOException {
+
+    for (Text dir : dirInfo) {
+      if (!fs.mkdirs(new Path(dir.toString())))
 
 Review comment:
    I wish HDFS had async or batch operations.  Could possibly use a thread pool here to speed
up the case of many dirs. I think the HDFS client will batch request from multiple threads
into a single RPC to namenode. This could be a follow on issue. Since multiple places in Accumulo
code need to create or delete lots of files, could build these high level operations into
volume manager.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message