hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Yang" <richardy...@richardyang.net>
Subject RE: RE: Re: Hadoop for perl
Date Sat, 24 Mar 2007 17:58:22 GMT
hello,

when you do 
bin/hadoop dfs -ls myfs
it should say
1 item(s) found
LICENSE.txt  <r 1>

if you have no problem moving files/folder to and away from HDFS, this should indicate HDFS
dfs is functioning normally. then,
how about
bin/hadoop jar hadoop-*-examples.jar wordcount myfs/LICENSE.txt out



and the following comman

bin/hadoop jar hadoop-*-examples.jar wordcount ./myfs/LICENSE.txt ./myFS/out 

I have already executed ./bin/start-all.sh and I have formatted ./myFS/


Richard Yang <richardyang@richardyang.net> ??????: 
By the error message, it looks like 'workingdirectory' is out of touch.  Is
HDFS dfs portion functioning normally??

Best Regards
 
Richard Yang
richardyang@richardyang.net
kusanagiyang@gmail.com
 
 
-----Original Message-----
From: Andreas Papadakis [mailto:atsia2003@yahoo.gr] 
Sent: Friday, March 23, 2007 1:18 PM
To: hadoop-user@lucene.apache.org
Subject: Re: Re: Hadoop for perl

Is there any other way to use hadoop with perl?

I read that I can use it with python(using jython)... I tried the example
but jython cannot find the module org.apache.hadoop.fs...(I dont know much
about java). Anyone knows where can I find that module and where should I
place it??

Also, I tried the java example and I get too many errors....
I used the command
bin/hadoop jar hadoop-*-examples.jar wordcount  mydfs/LICENSE.txt mydfs/a

but I get the following errors:

java.lang.RuntimeException: java.net.SocketTimeoutException: timed out
waiting for rpc response
        at
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:247)
        at org.apache.hadoop.mapred.JobConf.setInputPath(JobConf.java:150)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:137)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver
.java:71)
        at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
        at
org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
response
        at org.apache.hadoop.ipc.Client.call(Client.java:469)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
        at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
        at org.apache.hadoop.dfs.DFSClient.(DFSClient.java:106)
        at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem
.java:65)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:160)
        at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:91)
        at
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:243)

Anyone can help me???
Thank you

Erik Hetzner  ??????: At Fri, 23 Mar 2007 20:58:38
+0100,
Maximilian Sch??fmann  wrote:
>
> > I want to ask if there is an implementation of Hadoop for perl...or
> > something similar to Hadoop for python
>
> Well, there is one for Ruby - at least the MapReduce part: Starfish
>
> http://rufy.com/starfish/doc/

Starfish isn???t (or wasn???t last time I
looked) really much like
MapReduce at all.

best,
Erik Hetzner


   
---------------------------------
 ?????????????? Yahoo!
 ?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ???
???????? ?????? ????????? ???? ??? ??????????? ????????? 
 http://login.yahoo.com/config/mail?.intl=gr 




 		
---------------------------------
 ?????????????? Yahoo!
 ?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ??? ???????? ?????? ?????????
???? ??? ??????????? ????????? 
 http://login.yahoo.com/config/mail?.intl=gr 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message