hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15578) GridmixTestUtils uses the wrong staging directory in windows
Date Tue, 03 Jul 2018 01:50:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530672#comment-16530672
] 

Íñigo Goiri commented on HADOOP-15578:
--------------------------------------

The error would be:
{code}
Exception in thread "StressJobFactory" java.lang.IllegalArgumentException: Pathname /D:/hadoop-2.9/hadoop-tools/hadoop-gridmix/target/tmp/mapred/staging
from D:/hadoop-2.9/hadoop-tools/hadoop-gridmix/target/tmp/mapred/staging is not a valid DFS
filename.
	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:221)
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1248)
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1245)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1262)
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1237)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2216)
	at org.apache.hadoop.mapred.gridmix.GridmixTestUtils.createHomeAndStagingDirectory(GridmixTestUtils.java:115)
	at org.apache.hadoop.mapred.gridmix.DebugJobProducer$MockJob.getUser(DebugJobProducer.java:217)
	at org.apache.hadoop.mapred.gridmix.JobFactory$FilterJobStory.getUser(JobFactory.java:147)
	at org.apache.hadoop.mapred.gridmix.StressJobFactory$StressReaderThread.run(StressJobFactory.java:210)
{code}

[~surmountian], do you see any other way to fix this issue?/


> GridmixTestUtils uses the wrong staging directory in windows
> ------------------------------------------------------------
>
>                 Key: HADOOP-15578
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15578
>             Project: Hadoop Common
>          Issue Type: Test
>            Reporter: Íñigo Goiri
>            Priority: Major
>         Attachments: HADOOP-15578.000.patch
>
>
> {{GridmixTestUtils#createHomeAndStagingDirectory}} gets the staging area from the configuration
key {{mapreduce.jobtracker.staging.root.dir}}. This variable depends on {{hadoop.tmp.dir}}
which in Windows is set to a local Windows folder. When the test tries to create the path
in HDFS it gets an error because the path is not compliant.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message