hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14118) move jets3t into a dependency on hadoop-aws JAR
Date Fri, 24 Feb 2017 22:16:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15883617#comment-15883617
] 

Steve Loughran commented on HADOOP-14118:
-----------------------------------------

+1


I normally turn off the s3n stuff unless I touch any of the FS contract tests. I turned them
on here and first all the tests failed (endpoint was S3 frankfurt, see). Turning to s3 ireland
on, two failures. Looks like consistency, though the second one is weirder. I'd have expected
that test to create something in the root path, but clearly it isn't. Nothing to do with this
patch though



{code}
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.751 sec - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesDiskBlocks
Running org.apache.hadoop.fs.s3native.ITestJets3tNativeS3FileSystemContract
Tests run: 52, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 60.125 sec <<< FAILURE!
- in org.apache.hadoop.fs.s3native.ITestJets3tNativeS3FileSystemContract
testListStatusForRoot(org.apache.hadoop.fs.s3native.ITestJets3tNativeS3FileSystemContract)
 Time elapsed: 0.299 sec  <<< FAILURE!
junit.framework.AssertionFailedError: Root directory is not empty;  expected:<0> but
was:<1>
	at junit.framework.Assert.fail(Assert.java:57)
	at junit.framework.Assert.failNotEquals(Assert.java:329)
	at junit.framework.Assert.assertEquals(Assert.java:78)
	at junit.framework.Assert.assertEquals(Assert.java:234)
	at junit.framework.TestCase.assertEquals(TestCase.java:401)
	at org.apache.hadoop.fs.s3native.NativeS3FileSystemContractBaseTest.testListStatusForRoot(NativeS3FileSystemContractBaseTest.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at junit.framework.TestCase.runBare(TestCase.java:141)
	at junit.framework.TestResult$1.protect(TestResult.java:122)
	at junit.framework.TestResult.runProtected(TestResult.java:142)
	at junit.framework.TestResult.run(TestResult.java:125)
	at junit.framework.TestCase.run(TestCase.java:129)
	at junit.framework.TestSuite.runTest(TestSuite.java:255)
	at junit.framework.TestSuite.run(TestSuite.java:250)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

testLSRootDir(org.apache.hadoop.fs.s3native.ITestJets3tNativeS3FileSystemContract)  Time elapsed:
1.274 sec  <<< ERROR!
java.io.FileNotFoundException: File s3n://hwdev-steve-ireland-new/Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/test/data/62zuTzHEJZ
does not exist.
	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.listStatus(NativeS3FileSystem.java:603)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1823)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1865)
	at org.apache.hadoop.fs.FileSystem$4.<init>(FileSystem.java:2027)
	at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:2026)
	at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:2009)
	at org.apache.hadoop.fs.FileSystem$5.handleFileStat(FileSystem.java:2167)
	at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:2144)
	at org.apache.hadoop.fs.FileSystemContractBaseTest.assertListFilesFinds(FileSystemContractBaseTest.java:755)
	at org.apache.hadoop.fs.FileSystemContractBaseTest.testLSRootDir(FileSystemContractBaseTest.java:740)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at junit.framework.TestCase.runBare(TestCase.java:141)
	at junit.framework.TestResult$1.protect(TestResult.java:122)
	at junit.framework.TestResult.runProtected(TestResult.java:142)
	at junit.framework.TestResult.run(TestResult.java:125)
	at junit.framework.TestCase.run(TestCase.java:129)
	at junit.framework.TestSuite.runTest(TestSuite.java:255)
	at junit.framework.TestSuite.run(TestSuite.java:250)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)


{code}

> move jets3t into a dependency on hadoop-aws JAR
> -----------------------------------------------
>
>                 Key: HADOOP-14118
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14118
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: build, fs/s3
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Akira Ajisaka
>         Attachments: HADOOP-14118.01.patch
>
>
> hadoop-common still declares a dependency on jets3t, which allows downstream projects
to pick it up. But as they can't get s3n to work without the hadoop-aws JAR, it's hard to
see how much use this is.
> I propose: moving it to a dependency of hadoop-aws JAR alone.
> Marking as incompatible as it will be in the specific situation
> * downstream maven build doesn't pull in hadoop-aws
> * command line doesnt have hadoop-aws JAR dependencies on CP and still wants to use jets3t.
Hard to imagine how this arises.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message