hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karim Awara <karim.aw...@kaust.edu.sa>
Subject Re: MiniDFSCluster setup
Date Sun, 15 Dec 2013 15:45:50 GMT
Attached below. So What I did was after downloading hadoop src. I ran

$ mvn install -DskipTests$ mvn eclipse:eclipse -DdownloadSources=true
-DdownloadJavadocs=true


And imported both Hadoop Common and HDFS projects into eclipse.








java.lang.ExceptionInInitializerError
    at
org.apache.hadoop.hdfs.server.common.Storage.getBuildVersion(Storage.java:1026)
    at
org.apache.hadoop.hdfs.server.protocol.NamespaceInfo.<init>(NamespaceInfo.java:58)
    at
org.apache.hadoop.hdfs.server.namenode.NNStorage.newNamespaceInfo(NNStorage.java:550)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:144)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:837)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:266)
    at
org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:122)
    at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:775)
    at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:642)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:334)
    at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:316)
    at
org.apache.hadoop.hdfs.TestWriteRead.initJunitModeTest(TestWriteRead.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
    at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.NullPointerException
    at java.util.Properties$LineReader.readLine(Properties.java:435)
    at java.util.Properties.load0(Properties.java:354)
    at java.util.Properties.load(Properties.java:342)
    at org.apache.hadoop.util.VersionInfo.<init>(VersionInfo.java:51)
    at org.apache.hadoop.util.VersionInfo.<clinit>(VersionInfo.java:97)
    ... 36 more

java.lang.NullPointerException
    at org.apache.hadoop.hdfs.TestWriteRead.shutdown(TestWriteRead.java:89)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)





--
Best Regards,
Karim Ahmed Awara


On Sun, Dec 15, 2013 at 5:59 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Can you show us the full stack trace ?
> In Eclipse, was there any project shown with a red bang or red cross ?
>
> Cheers
>
>
> On Sun, Dec 15, 2013 at 2:26 AM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>
>>
>> It tells me java.lang.ExceptionInitializerError at
>> org.apache.....hdfs.server.common.Storage.getBuildVersion!
>>
>> I feel I am missing some parameters?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>>
>> On Sun, Dec 15, 2013 at 5:08 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> You can use the following command to generate .project files for Eclipse
>>> (at the root of your workspace):
>>> mvn clean package -DskipTests eclipse:eclipse
>>>
>>> When you import hadoop, call sub-projects would be imported.
>>>
>>> I was able to run TestWriteRead in Eclipse successfully.
>>>
>>> Cheers
>>>
>>>
>>> On Sat, Dec 14, 2013 at 4:56 PM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>>>
>>>>
>>>> Running the test from Maven through commandline works fine. But  I am
>>>> using eclipse. And it generates problem if I try to run the test as Junit,
>>>> as if eclipse is not aware of any of the conf parameters or args. Can
>>>> someone point to me a detailed source where it explains how to run Junit
>>>> through Eclipse for hadoop 2.2.x?
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Karim Ahmed Awara
>>>>
>>>>
>>>> On Sun, Dec 15, 2013 at 2:55 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>
>>>>> If you search under hadoop-hdfs-project/hadoop-hdfs/src/test, you
>>>>> would see a lot of tests which use MiniDFSCluster
>>>>> e.g.
>>>>>     cluster = new MiniDFSCluster.Builder(conf).numDataNodes(3).build();
>>>>>
>>>>> hadoop-hdfs-project/hadoop-hdfs/src/test//java/org/apache/hadoop/hdfs/TestWriteRead.java
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>> On Sat, Dec 14, 2013 at 2:43 PM, Karim Awara <karim.awara@kaust.edu.sa
>>>>> > wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I read I can use MiniDFSCluster to set my own tests in case I modify
>>>>>> hadop source code. I have built hadoop 2.2. However, I can't find
any
>>>>>> source on how to get the MiniDFSCluster working. Can someone point
a link
>>>>>> to me that helps?
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Karim Ahmed Awara
>>>>>>
>>>>>> ------------------------------
>>>>>> This message and its contents, including attachments are intended
>>>>>> solely for the original recipient. If you are not the intended recipient
or
>>>>>> have received this message in error, please notify me immediately
and
>>>>>> delete this message from your computer system. Any unauthorized use
or
>>>>>> distribution is prohibited. Please consider the environment before
printing
>>>>>> this email.
>>>>>
>>>>>
>>>>>
>>>>
>>>> ------------------------------
>>>> This message and its contents, including attachments are intended
>>>> solely for the original recipient. If you are not the intended recipient
or
>>>> have received this message in error, please notify me immediately and
>>>> delete this message from your computer system. Any unauthorized use or
>>>> distribution is prohibited. Please consider the environment before printing
>>>> this email.
>>>>
>>>
>>>
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>
>

-- 

------------------------------
This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

Mime
View raw message