bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rui Gao <r...@yahoo-corp.jp>
Subject Proposal of adding HDFS Erasure Coding test cases, and resource unpack problem with HDFS smoke test.
Date Mon, 16 May 2016 11:19:38 GMT

Hi all

Nice to meet you all. I am Gao, an engineer working in Japan. And I have got two issues about
Bigtop which I want to share.

The first, I want to proposal that we add HDFS Erasure Coding(HDFS-7285 , HDFS-8031) test
cases in to Bigtop HDFS smoke tests. This could be very helpful when users want to test HDFS
EC(Erasure Coding) related functions before they use Erasure Coding in production clusters.
I wonder if there are engineers also interested in developing Erasure Coding test cases, so
we could gather a team to work together? I have added a basic demo test case “TestECPolicy.groovy”,
and send it in attached patch.

The second, when I am trying to reuse HDFS test cases by simply copying them and setting EC
policy to TESTDIR, like I copied “TestPut.groovy” and make some minor changes to make
a new EC test case “TestPutEC.groovy”. This test case failed dur to unpack resource errors.
Then I try to run the original “TestPut.groovy”, some how it failed with the same reason
too. Could you give some tips to resolve these problem? Thank you very much.

Looking forward to your reply.
Gao.


Following is the detailed failure log of the second issue.

TestPut.groovy:
After build with “pom.xml” and “clean package -Pnative,dist -Dtar -DskipTests” command,
“install-hadoop” and “bigtop-tests:smoke-tests:hdfs:test -Psmoke.tests –info” grandle
tasks was ran, then I got following error log in line 53(  JarContent.unpackJarContainer(TestPut.class,
".", null);).


Gradle Test Executor 2 started executing tests.
Gradle Test Executor 2 finished executing tests.

org.apache.bigtop.itest.hadoop.hdfs.TestPut > classMethod FAILED
    java.io.IOException: Class TestPut doesn't belong to any jar file in the classpath
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
        at org.apache.bigtop.itest.JarContent.unpackJarContainer(JarContent.groovy:181)
        at org.apache.bigtop.itest.JarContent$unpackJarContainer.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
        at org.apache.bigtop.itest.hadoop.hdfs.TestPut.setUp(TestPut.groovy:53)

1 test completed, 1 failed






TestPutEC.groovy:
After build with “pom.xml” and “clean package -Pnative,dist -Dtar -DskipTests” command,
“install-hadoop” and “bigtop-tests:smoke-tests:hdfs-ec:test -Psmoke.tests –info”
grandle tasks was ran, then I got following error log in line 53(  JarContent.unpackJarContainer(TestPutEC.class,
".", null);). While “TestECPolicy.groovy” was passed without problem.



Gradle Test Executor 2 started executing tests.
Gradle Test Executor 2 finished executing tests.

org.apache.bigtop.itest.hadoop.hdfs.TestPutEC > classMethod FAILED
    java.io.IOException: Class TestPutEC doesn't belong to any jar file in the classpath
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
        at org.apache.bigtop.itest.JarContent.unpackJarContainer(JarContent.groovy:181)
        at org.apache.bigtop.itest.JarContent$unpackJarContainer.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
        at org.apache.bigtop.itest.hadoop.hdfs.TestPutEC.setUp(TestPutEC.groovy:53)

2 tests completed, 1 failed


Mime
View raw message