hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5916) Standardize fall-back value of test.build.data for testing directories
Date Tue, 26 May 2009 19:38:45 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jakob Homan updated HADOOP-5916:
--------------------------------

    Description: 
Currently when the test.build.data property is not set, as happens when run through some configurations
of Eclipse, the fall-back value varies wildly.  Most calls set to /tmp, which is not good
as it is beyond the scope of the ant clean task and thus will not be deleted.  Others default
to "." which can drop the test files right in the current directory.  Speaking with Konstanin,
it seems the correct location should be "build/test/data" to ensure any files that are created
are within the scope of Ant's clean command.

Attached is the current variation in this setting.

  was:
Currently when the test.build.data property is not set, as happens when run through some configurations
of Eclipse, the fall-back value varies wildly.  Most calls set to /tmp, which is not good
as it is beyond the scope of the ant clean task and thus will not be deleted.  Others default
to "." which can drop the test files right in the current directory.  Speaking with Konstanin,
it seems the correct location should be "build/test/data" to ensure any files that are created
are within the scope of Ant's clean command.

Current variation of setting the test.build.data:
{noformat}
./contrib/fairscheduler/src/test/org/apache/hadoop/mapred/TestFairScheduler.java:  final static
String TEST_DIR = new File(System.getProperty("test.build.data",
./contrib/sqoop/src/java/org/apache/hadoop/sqoop/ImportOptions.java:    String tmpDir = System.getProperty("test.build.data",
"/tmp/");
./contrib/sqoop/src/test/org/apache/hadoop/sqoop/testutil/ImportJobTestCase.java:    String
tmpDir = System.getProperty("test.build.data", "/tmp/");
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestAutoInputFormat.java:    Path
dir = new Path(System.getProperty("test.build.data", ".") + "/mapred");
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestMultipleArchiveFiles.java:  
   "-jobconf", "stream.tmpdir=" + System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestMultipleCachefiles.java:    
     "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp"),
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestRawBytesStreaming.java:     
"-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp"),
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamAggregate.java:      "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamDataProtocol.java:    
 "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamedMerge.java:      "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java:      "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingAsDifferentUser.java:
           "stream.tmpdir=" + System.getProperty("test.build.data", "/tmp") };
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingBadRecords.java:   
  "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingBadRecords.java:   
  "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingCombiner.java:     
"-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingEmptyInpNonemptyOut.java:
     "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingExitStatus.java:   
  "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java:      "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingKeyValue.java:     
"-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingSeparator.java:    
 "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp"),
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingStderr.java:      "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamReduceNone.java:      "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestSymLink.java:          "-jobconf",
"stream.tmpdir="+System.getProperty("test.build.data","/tmp"),
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestTypedBytesStreaming.java:   
  "-jobconf", "stream.tmpdir="+System.getProperty("test.build.data","/tmp"),
./contrib/streaming/src/test/org/apache/hadoop/streaming/TestUlimit.java:                
  System.getProperty("test.build.data","/tmp")
./contrib/streaming/src/test/org/apache/hadoop/streaming/UtilTest.java:    antTestDir_ = System.getProperty("test.build.data",
userDir_);
./contrib/streaming/src/test/org/apache/hadoop/typedbytes/TestIO.java:    this.tmpfile = new
File(System.getProperty("test.build.data", "/tmp"),
./examples/org/apache/hadoop/examples/DBCountPageView.java:        System.getProperty("test.build.data",".")
+ "/URLAccess");
./test/core/org/apache/hadoop/filecache/TestDistributedCache.java:    new Path(System.getProperty("test.build.data","/tmp/cachebasedir"))
./test/core/org/apache/hadoop/filecache/TestDistributedCache.java:    System.getProperty("test.build.data",
"/tmp/distributedcache");
./test/core/org/apache/hadoop/fs/kfs/TestKosmosFileSystem.java:        baseDir = new Path(System.getProperty("test.build.data",
"/tmp" ) +
./test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java:    = System.getProperty("test.build.data","build/test/data/work-dir/localfs");
./test/core/org/apache/hadoop/fs/TestDFVariations.java:      super(new File(System.getProperty("test.build.data","/tmp")),
0L);
./test/core/org/apache/hadoop/fs/TestDU.java:      System.getProperty("test.build.data","/tmp"),
"dutmp");
./test/core/org/apache/hadoop/fs/TestGetFileBlockLocations.java:      System.getProperty("test.build.data",
"/tmp/testGetFileBlockLocations");
./test/core/org/apache/hadoop/fs/TestLocalFileSystem.java:    = System.getProperty("test.build.data","build/test/data/work-dir/localfs");
./test/core/org/apache/hadoop/fs/TestTrash.java:    new Path(new File(System.getProperty("test.build.data","/tmp")
./test/core/org/apache/hadoop/fs/TestTruncatedInputBug.java:    new Path(System.getProperty("test.build.data","/tmp"))
./test/core/org/apache/hadoop/io/TestArrayFile.java:    System.getProperty("test.build.data",".")
+ "/test.array";
./test/core/org/apache/hadoop/io/TestBloomMapFile.java:    Path dirName = new Path(System.getProperty("test.build.data",".")
+
./test/core/org/apache/hadoop/io/TestMapFile.java:    Path dirName = new Path(System.getProperty("test.build.data",".")
+
./test/core/org/apache/hadoop/io/TestMapFile.java:    Path dirName = new Path(System.getProperty("test.build.data",".")
+
./test/core/org/apache/hadoop/io/TestMapFile.java:    Path dirName = new Path(System.getProperty("test.build.data",".")
+
./test/core/org/apache/hadoop/io/TestSequenceFileSerialization.java:    Path file = new Path(System.getProperty("test.build.data",".")
+
./test/core/org/apache/hadoop/io/TestSetFile.java:    System.getProperty("test.build.data",".")
+ "/test.set";
./test/hdfs/org/apache/hadoop/fs/loadGenerator/TestLoadGenerator.java:    System.getProperty("test.build.data","build/test/data");
./test/hdfs/org/apache/hadoop/fs/loadGenerator/TestLoadGenerator.java:    final String SCRIPT_TEST_DIR
= new File(System.getProperty("test.build.data",
./test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java:    return System.getProperty("test.build.data",
"build/test/data") + "/dfs/";
./test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java:      File baseDir = new
File(System.getProperty("test.build.data",
./test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java:    File baseDir = new File(System.getProperty("test.build.data",
./test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestNameEditsConfigs.java:      System.getProperty("test.build.data",
"build/test/data"), "dfs/");
./test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java:    String baseDir = System.getProperty("test.build.data",
"/tmp");
./test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStorageRestore.java:    String baseDir
= System.getProperty("test.build.data",  "build/test/data");
./test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java:    new Path(System.getProperty("test.build.data","/tmp"))
./test/hdfs/org/apache/hadoop/hdfs/TestDFSStartupVersions.java:                          
                    System.getProperty("test.build.data","/tmp").toString().replace(' ', '+'));
./test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java:    String dataDir = System.getProperty("test.build.data",
"build/test/data");
./test/hdfs/org/apache/hadoop/hdfs/TestFileStatus.java:    new Path(System.getProperty("test.build.data","/tmp"))
./test/hdfs/org/apache/hadoop/hdfs/TestFSInputChecker.java:    String dir = System.getProperty("test.build.data",
".");
./test/hdfs/org/apache/hadoop/hdfs/TestHDFSServerPorts.java:    return System.getProperty("test.build.data",
"build/test/data");
./test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestDelimitedImageVisitor.java:
 private static String ROOT = System.getProperty("test.build.data","/tmp");
./test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java: 
private static String ROOT = System.getProperty("test.build.data","/tmp");
./test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java:      System.getProperty("test.build.data","/tmp").replace('
', '+'));
./test/hdfs-with-mr/org/apache/hadoop/fs/DFSCIOTest.java:  private static String TEST_ROOT_DIR
= System.getProperty("test.build.data","/benchmarks/DFSCIOTest");
./test/hdfs-with-mr/org/apache/hadoop/fs/DistributedFSCheck.java:  private static Path TEST_ROOT_DIR
= new Path(System.getProperty("test.build.data","/benchmarks/DistributedFSCheck"));
./test/hdfs-with-mr/org/apache/hadoop/fs/TestCopyFiles.java:    new Path(System.getProperty("test.build.data","/tmp"))
./test/hdfs-with-mr/org/apache/hadoop/fs/TestDFSIO.java:  private static String TEST_ROOT_DIR
= System.getProperty("test.build.data","/benchmarks/TestDFSIO");
./test/hdfs-with-mr/org/apache/hadoop/fs/TestFileSystem.java:  private static String ROOT
= System.getProperty("test.build.data","fs_test");
./test/hdfs-with-mr/org/apache/hadoop/io/TestSequenceFileMergeProgress.java:    Path dir =
new Path(System.getProperty("test.build.data",".") + "/mapred");
./test/hdfs-with-mr/org/apache/hadoop/tools/TestDistCh.java:    new Path(System.getProperty("test.build.data","/tmp")
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:    Path file = new Path(System.getProperty("test.build.data",".")+"/test.seq");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:      new Path(System.getProperty("test.build.data",".")+"/test.rc.seq");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:      new Path(System.getProperty("test.build.data",".")+"/test.bc.seq");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:    Path file = new Path(System.getProperty("test.build.data",".")+"/test.seq.metadata");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:      new Path(System.getProperty("test.build.data",".")+"/test.sorted.seq.metadata");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:      new Path(System.getProperty("test.build.data",".")+"/test.rc.seq.metadata");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:      new Path(System.getProperty("test.build.data",".")+"/test.bc.seq.metadata");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:    Path path1 = new Path(System.getProperty("test.build.data",".")+"/test1.seq");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:    Path path2 = new Path(System.getProperty("test.build.data",".")+"/test2.seq");
./test/mapred/org/apache/hadoop/io/TestSequenceFile.java:    Path path = new Path(System.getProperty("test.build.data",".")+"/broken.seq");
./test/mapred/org/apache/hadoop/mapred/jobcontrol/TestJobControl.java:    Path rootDataDir
= new Path(System.getProperty("test.build.data", "."), "TestJobControlData");
./test/mapred/org/apache/hadoop/mapred/jobcontrol/TestLocalJobControl.java:    Path rootDataDir
= new Path(System.getProperty("test.build.data", "."),
./test/mapred/org/apache/hadoop/mapred/join/TestWrappedRecordReaderClassloader.java:    Path
testdir = new Path(System.getProperty("test.build.data", "/tmp"))
./test/mapred/org/apache/hadoop/mapred/lib/TestChainMapReduce.java:      String localPathRoot
= System.getProperty("test.build.data", "/tmp")
./test/mapred/org/apache/hadoop/mapred/lib/TestChainMapReduce.java:      String localPathRoot
= System.getProperty("test.build.data", "/tmp")
./test/mapred/org/apache/hadoop/mapred/lib/TestLineInputFormat.java:    new Path(new Path(System.getProperty("test.build.data",
"."), "data"),
./test/mapred/org/apache/hadoop/mapred/lib/TestMultipleOutputs.java:      String localPathRoot
= System.getProperty("test.build.data", "/tmp")
./test/mapred/org/apache/hadoop/mapred/lib/TestMultithreadedMapRunner.java:      String localPathRoot
= System.getProperty("test.build.data", "/tmp")
./test/mapred/org/apache/hadoop/mapred/MRBench.java:    new Path(System.getProperty("test.build.data","/benchmarks/MRBench"));
./test/mapred/org/apache/hadoop/mapred/MRCaching.java:    String TEST_ROOT_DIR = new Path(System.getProperty("test.build.data","/tmp"))
./test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java:      String localPathRoot
= System.getProperty("test.build.data","/tmp")
./test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java:      String localPathRoot
= System.getProperty("test.build.data","/tmp")
./test/mapred/org/apache/hadoop/mapred/TestCommandLineJobSubmission.java:  File buildDir =
new File(System.getProperty("test.build.data", "/tmp"));
./test/mapred/org/apache/hadoop/mapred/TestEmptyJob.java:      new File(System.getProperty("test.build.data",
"/tmp")).toURI()
./test/mapred/org/apache/hadoop/mapred/TestFileInputFormatPathFilter.java:      new Path(new
Path(System.getProperty("test.build.data", "."), "data"),
./test/mapred/org/apache/hadoop/mapred/TestFileOutputCommitter.java:     System.getProperty("test.build.data",
"."), "output");
./test/mapred/org/apache/hadoop/mapred/TestFileOutputFormat.java:      String localPathRoot
= System.getProperty("test.build.data", "/tmp")
./test/mapred/org/apache/hadoop/mapred/TestIndexCache.java:    Path p = new Path(System.getProperty("test.build.data",
"/tmp"),
./test/mapred/org/apache/hadoop/mapred/TestIndexCache.java:    Path p = new Path(System.getProperty("test.build.data",
"/tmp"),
./test/mapred/org/apache/hadoop/mapred/TestJobHistoryParsing.java:    Path historyDir = new
Path(System.getProperty("test.build.data", "."), 
./test/mapred/org/apache/hadoop/mapred/TestJobHistoryVersion.java:    new Path(System.getProperty("test.build.data",
"/tmp"), 
./test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java:    new Path(System.getProperty("test.build.data","/tmp"),
"jip-testing");
./test/mapred/org/apache/hadoop/mapred/TestJobQueueInformation.java:    new Path(System.getProperty("test.build.data","/tmp"),

./test/mapred/org/apache/hadoop/mapred/TestJobTrackerRestart.java:    new Path(System.getProperty("test.build.data","/tmp"),

./test/mapred/org/apache/hadoop/mapred/TestJobTrackerSafeMode.java:    new Path(System.getProperty("test.build.data",
"/tmp"), "jt-safemode");
./test/mapred/org/apache/hadoop/mapred/TestKeyValueTextInputFormat.java:    new Path(new Path(System.getProperty("test.build.data",
"."), "data"),
./test/mapred/org/apache/hadoop/mapred/TestMapRed.java:        System.getProperty("test.build.data","/tmp")).makeQualified(fs);
./test/mapred/org/apache/hadoop/mapred/TestMapRed.java:        System.getProperty("test.build.data","/tmp")).makeQualified(fs);
./test/mapred/org/apache/hadoop/mapred/TestMiniMRLocalFS.java:    new File(System.getProperty("test.build.data","/tmp"))
./test/mapred/org/apache/hadoop/mapred/TestMiniMRMapRedDebugScript.java:    String TEST_ROOT_DIR
= new Path(System.getProperty("test.build.data",
./test/mapred/org/apache/hadoop/mapred/TestMiniMRTaskTempDir.java:    String TEST_ROOT_DIR
= new Path(System.getProperty("test.build.data",
./test/mapred/org/apache/hadoop/mapred/TestMiniMRWithDFS.java:        new File(System.getProperty("test.build.data","/tmp"))
./test/mapred/org/apache/hadoop/mapred/TestMultiFileInputFormat.java:    Path dir = new Path(System.getProperty("test.build.data",".")
+ "/mapred");
./test/mapred/org/apache/hadoop/mapred/TestMultipleTextOutputFormat.java:                
     new Path(System.getProperty("test.build.data", "."), 
./test/mapred/org/apache/hadoop/mapred/TestRecoveryManager.java:    new Path(System.getProperty("test.build.data",
"/tmp"), 
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileAsBinaryInputFormat.java:    Path dir
= new Path(System.getProperty("test.build.data",".") + "/mapred");
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java:      new
Path(new Path(new Path(System.getProperty("test.build.data",".")), 
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java:    Path
dir = new Path(System.getProperty("test.build.data",".") + "/mapred");
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java:    Path
outputdir = new Path(System.getProperty("test.build.data",".") 
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileAsTextInputFormat.java:    Path dir
= new Path(System.getProperty("test.build.data",".") + "/mapred");
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileInputFilter.java:  private static final
Path inDir = new Path(System.getProperty("test.build.data",".") + "/mapred");
./test/mapred/org/apache/hadoop/mapred/TestSequenceFileInputFormat.java:    Path dir = new
Path(System.getProperty("test.build.data",".") + "/mapred");
./test/mapred/org/apache/hadoop/mapred/TestSpilledRecordsCounter.java:    String TEST_ROOT_DIR
= new Path(System.getProperty("test.build.data",
./test/mapred/org/apache/hadoop/mapred/TestTaskFail.java:    String TEST_ROOT_DIR = new Path(System.getProperty("test.build.data",
./test/mapred/org/apache/hadoop/mapred/TestTextInputFormat.java:    new Path(new Path(System.getProperty("test.build.data",
"."), "data"),
./test/mapred/org/apache/hadoop/mapred/TestTextOutputFormat.java:                      new
Path(System.getProperty("test.build.data", "."), 
./test/mapred/org/apache/hadoop/mapred/ThreadedMapBenchmark.java:    new Path(System.getProperty("test.build.data",

./test/mapred/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java:  
 final Path testdir = new Path(System.getProperty("test.build.data", "/tmp")
./test/mapred/org/apache/hadoop/mapreduce/TestMapReduceLocal.java:    new Path(System.getProperty("test.build.data","/tmp"));
./test/mapred/org/apache/hadoop/mapreduce/TestValueIterReset.java:    new Path(System.getProperty("test.build.data","/tmp"));
./test/mapred/org/apache/hadoop/record/TestRecordWritable.java:    Path dir = new Path(System.getProperty("test.build.data",".")
+ "/mapred");
{noformat}


Deleted overly long list of offending statements.  Will attach as a file.

> Standardize fall-back value of test.build.data for testing directories
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-5916
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5916
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>            Reporter: Jakob Homan
>         Attachments: test.build.data.txt
>
>
> Currently when the test.build.data property is not set, as happens when run through some
configurations of Eclipse, the fall-back value varies wildly.  Most calls set to /tmp, which
is not good as it is beyond the scope of the ant clean task and thus will not be deleted.
 Others default to "." which can drop the test files right in the current directory.  Speaking
with Konstanin, it seems the correct location should be "build/test/data" to ensure any files
that are created are within the scope of Ant's clean command.
> Attached is the current variation in this setting.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message