hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xeonmailinglist-gmail <xeonmailingl...@gmail.com>
Subject how to merge sequence files?
Date Mon, 27 Apr 2015 21:36:03 GMT
Hi,

I have several directories that contain several files of 
|SequenceFileOutputFormat| with |org.apache.hadoop.io.Text| as key and 
value. I want to merge all these files into one.

I have looked to the join example [1], but it is not working. How I 
merge SequenceFiles?

|+ /root/Programs/hadoop/bin/hadoop jar /root/Programs/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar
join -inFormat org.apache.hadoop.mapred.SequenceFileInputFormat -outFormat org.apache.hadoop.mapred.SequenceFileOutputFormat
-outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text /wiki-output,/wiki-output2
/aggregate-output
15/04/27 17:34:47 INFO client.RMProxy: Connecting to ResourceManager at /172.16.100.11:8040
java.lang.ClassCastException: class org.apache.hadoop.mapred.SequenceFileOutputFormat
         at java.lang.Class.asSubclass(Class.java:3168)
         at org.apache.hadoop.examples.Join.run(Join.java:112)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
         at org.apache.hadoop.examples.Join.main(Join.java:173)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:622)
         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
         at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
         at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:622)
         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
|

—

​

Mime
View raw message