From notifications-return-52352-archive-asf-public=cust-asf.ponee.io@accumulo.apache.org Fri Sep 20 17:25:04 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id ED91E180608 for ; Fri, 20 Sep 2019 19:25:03 +0200 (CEST) Received: (qmail 47072 invoked by uid 500); 20 Sep 2019 17:25:03 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 47057 invoked by uid 99); 20 Sep 2019 17:25:03 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Sep 2019 17:25:03 +0000 From: GitBox To: notifications@accumulo.apache.org Subject: [GitHub] [accumulo] keith-turner commented on a change in pull request #1366: Fix #1365 2.1 Upgrade processing for #1043 ~del Message-ID: <156900030314.3818.13185839217969771329.gitbox@gitbox.apache.org> Date: Fri, 20 Sep 2019 17:25:03 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit keith-turner commented on a change in pull request #1366: Fix #1365 2.1 Upgrade processing for #1043 ~del URL: https://github.com/apache/accumulo/pull/1366#discussion_r326720507 ########## File path: server/master/src/main/java/org/apache/accumulo/master/upgrade/Upgrader9to10.java ########## @@ -352,4 +372,87 @@ MetadataTime computeRootTabletTime(ServerContext context, Collection goo } } + static public void upgradeFileDeletes(ServerContext ctx, Ample.DataLevel level) { + + String tableName = level.metaTable(); + AccumuloClient c = ctx; + + // find all deletes + try (BatchWriter writer = c.createBatchWriter(tableName, new BatchWriterConfig())) { + String continuePoint = ""; + boolean stillDeletes = true; + + while (stillDeletes) { + List deletes = new ArrayList<>(); + log.info("looking for candidates"); + stillDeletes = getOldCandidates(ctx, tableName, continuePoint, deletes); + log.info("found {} deletes to upgrade", deletes.size()); + for (String olddelete : deletes) { + // create new formatted delete + writer.addMutation(upgradeDeleteMutation(olddelete)); + } + writer.flush(); + + // if nothing thrown then we're good so mark all deleted + for (String olddelete : deletes) { + writer.addMutation(deleteOldDeleteMutation(olddelete)); + } + writer.flush(); + + // give it some time for memory to clean itself up if needed + sleepUninterruptibly(5, TimeUnit.SECONDS); + continuePoint = deletes.get(deletes.size() - 1); Review comment: There may be situations where `deletes.size()` is empty and this will throw an exception. Not sure about this, but it may make the code simpler to refactor this code as described in #1351 to remove the continue point. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services