Return-Path: X-Original-To: apmail-apex-commits-archive@minotaur.apache.org Delivered-To: apmail-apex-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E260818C11 for ; Wed, 27 Apr 2016 02:22:31 +0000 (UTC) Received: (qmail 11054 invoked by uid 500); 27 Apr 2016 02:22:31 -0000 Delivered-To: apmail-apex-commits-archive@apex.apache.org Received: (qmail 11018 invoked by uid 500); 27 Apr 2016 02:22:31 -0000 Mailing-List: contact commits-help@apex.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@apex.incubator.apache.org Delivered-To: mailing list commits@apex.incubator.apache.org Received: (qmail 11009 invoked by uid 99); 27 Apr 2016 02:22:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Apr 2016 02:22:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 4CE0BC0DB2 for ; Wed, 27 Apr 2016 02:22:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.221 X-Spam-Level: X-Spam-Status: No, score=-3.221 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id y4UghQ8Q40mt for ; Wed, 27 Apr 2016 02:22:29 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with SMTP id 667AD5F24D for ; Wed, 27 Apr 2016 02:22:28 +0000 (UTC) Received: (qmail 10905 invoked by uid 99); 27 Apr 2016 02:22:27 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Apr 2016 02:22:27 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 74AE4DFF8A; Wed, 27 Apr 2016 02:22:27 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: thw@apache.org To: commits@apex.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-apex-core git commit: APEXCORE-445 - Race condition in AsynFSStorageAgent.save() Date: Wed, 27 Apr 2016 02:22:27 +0000 (UTC) Repository: incubator-apex-core Updated Branches: refs/heads/release-3.2 a9ab4d373 -> f2fa255e1 APEXCORE-445 - Race condition in AsynFSStorageAgent.save() Project: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/commit/f2fa255e Tree: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/tree/f2fa255e Diff: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/diff/f2fa255e Branch: refs/heads/release-3.2 Commit: f2fa255e1def44472721ed61aea536d8224a0abd Parents: a9ab4d3 Author: Vlad Rozov Authored: Tue Apr 26 15:04:20 2016 -0700 Committer: Vlad Rozov Committed: Tue Apr 26 18:09:55 2016 -0700 ---------------------------------------------------------------------- .../common/util/AsyncFSStorageAgent.java | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-apex-core/blob/f2fa255e/common/src/main/java/com/datatorrent/common/util/AsyncFSStorageAgent.java ---------------------------------------------------------------------- diff --git a/common/src/main/java/com/datatorrent/common/util/AsyncFSStorageAgent.java b/common/src/main/java/com/datatorrent/common/util/AsyncFSStorageAgent.java index d0f00fa..a584dfe 100644 --- a/common/src/main/java/com/datatorrent/common/util/AsyncFSStorageAgent.java +++ b/common/src/main/java/com/datatorrent/common/util/AsyncFSStorageAgent.java @@ -47,7 +47,6 @@ public class AsyncFSStorageAgent extends FSStorageAgent { super(); conf = null; - localBasePath = null; } public AsyncFSStorageAgent(String path, Configuration conf) @@ -63,20 +62,25 @@ public class AsyncFSStorageAgent extends FSStorageAgent public AsyncFSStorageAgent(String localBasePath, String path, Configuration conf) { this(path, conf); + this.localBasePath = localBasePath; } @Override public void save(final Object object, final int operatorId, final long windowId) throws IOException { - // save() is only called by one thread in the worker container so the following is okay - if (this.localBasePath == null) { - this.localBasePath = Files.createTempDirectory("chkp").toString(); - logger.info("using {} as the basepath for checkpointing.", this.localBasePath); - } if (syncCheckpoint) { super.save(object, operatorId, windowId); return; } + + if (localBasePath == null) { + synchronized (this) { + if (localBasePath == null) { + localBasePath = Files.createTempDirectory("chkp").toString(); + logger.info("using {} as the basepath for checkpointing.", localBasePath); + } + } + } String operatorIdStr = String.valueOf(operatorId); File directory = new File(localBasePath, operatorIdStr); if (!directory.exists()) {