Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 20 Apr 2016 06:09:25 +0000 (UTC)
From: "Vinayakumar B (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12955754.1459774914000.275415.1461132565587@Atlassian.JIRA>
In-Reply-To: <JIRA.12955754.1459774914000@Atlassian.JIRA>
References: <JIRA.12955754.1459774914000@Atlassian.JIRA>
 <JIRA.12955754.1459774914776@arcas>
Subject: [jira] [Commented] (HDFS-10256) Use GenericTestUtils.getTestDir
 method in tests for temporary directories
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-10256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15249340#comment-15249340 ] 

Vinayakumar B commented on HDFS-10256:
--------------------------------------

bq. MiniDFSCluster#shutdown() registers base_dir to be deleted on shutdown. If this gets slow, the next test JVM will start to run before the shutdown hook completes. But forcing every test to call shutdown(true) can slowdown things. Instead, each instance should get a random base_dir, so that the deletion through shutdown hook and the subsequent new test setup can overlap.
I have investigated a bit on this. Registering a dir to be deleted during shutdown will not actually delete the directory unless its empty. I have tried to register all files/directories recursively to be deleted on shutdown. But this will sometimes keeps the empty directories.
Instead, I have used unique base directory (testclassname.methodname) for each MiniDFSCluster instance and FileUtils.fullyDelete()}} during shutdown itself, instead of registering for deleteOnExit() in MiniDFSCluster#shutdown().
All tests which actually uses MiniDfsCluster will delete its basedir. Apart from this, there are some more tests, which creates directories on its own ( using some Random String), and these may not get deleted after test run.

At the end of test run, there was ~300MB of data created in "hadoop-hdfs/target/test/data" directory, which is way less than ~6GB reported by Chris.


So I am not sure why {{base_dir.deleteOnExit()}} was used during {{MiniDfsCluster#shutdown()}}, which will not actually delete anything because base_dir will not be empty.

> Use GenericTestUtils.getTestDir method in tests for temporary directories
> -------------------------------------------------------------------------
>
>                 Key: HDFS-10256
>                 URL: https://issues.apache.org/jira/browse/HDFS-10256
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: build, test
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)