hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Wang" <ryanwang.em...@gmail.com>
Subject what is the datanode's default port?
Date Thu, 29 Nov 2007 14:32:41 GMT
Hi,
  I wrote a short Program on Hadoop Reading and Writing. I want to read a
local file and write it to the hadoop.
 The code is below:

public class HadoopDFSFileReadWrite {

 public static void main(String[] argv) throws IOException {
  Configuration conf = new Configuration();
  FileSystem fs = FileSystem.get(conf);
  FileSystem dfs = null;
  try {
   dfs = FileSystem.get(new URI("hdfs://root:hadoop@node04:50010"), conf);
      //?????  can not connect to datanode Exception Below:
/*
Exception in thread "main" java.net.SocketTimeoutException: timed out
waiting for rpc response
at org.apache.hadoop.ipc.Client.call(Client.java:484)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184)
at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:269)
at org.apache.hadoop.dfs.DFSClient.createNamenode(DFSClient.java:147)
at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:161)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(
DistributedFileSystem.java:65)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:159)
at readWrite.HadoopDFSFileReadWrite.main(HadoopDFSFileReadWrite.java:50)
*/
  } catch (URISyntaxException e1) {
   // TODO Auto-generated catch block
   e1.printStackTrace();
  }
  if (argv.length != 2)
   usage();

  // Hadoop DFS deals with Path
  Path inFile = new Path(argv[0]);
  Path outFile = new Path(argv[1]);

  // Check if input/output are valid
  if (!fs.exists(inFile))
   printAndExit("Input file not found");
  if (!fs.isFile(inFile))
   printAndExit("Input should be a file");
  if (dfs.exists(outFile))
   printAndExit("Output already exists");

  // Read from and write to new file
  FSDataInputStream in = fs.open(inFile);
  FSDataOutputStream out = dfs.create(outFile);
  byte buffer[] = new byte[256];
  try {
   System.out.println("Start...");
   int bytesRead = 0;
   while ((bytesRead = in.read(buffer)) > 0) {
    out.write(buffer, 0, bytesRead);
   }
   System.out.println("End.....");
  } catch (IOException e) {
   System.out.println("Error while copying file");
  } finally {
   in.close();
   out.close();
  }
 }
}



I wonder whether the rpc port is wrong? or there are other problems?
or Someone can give me a example to write to the hdfs?

Thanks,
Ryan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message