hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mich Talebzadeh" <m...@peridale.co.uk>
Subject RE: How to create 3GB of compressed sequence file?
Date Sat, 04 Apr 2015 18:53:01 GMT


Do you mean you want to end up with a 3GB file in compressed sequence format?


If you have an RDBMS you can use Sqoop to import the data in sequence format. For example,
this one imports table hddtester.t from Oracle as sequence file


    sqoop import \

        --connect "jdbc:oracle:thin:@rhes564:1521:mydb" --username hddtester -P \

                -query "select * from hddtester.t where \

                 \$CONDITIONS" \

                                       --split-by object_id \

                                      --as-sequencefile \

                                      --target-dir "/work/t" \

                                  --num-mappers 1


Now if I want to import it as a compressed sequential file I get an error


Error: java.lang.IllegalArgumentException: SequenceFile doesn't work with GzipCodec without
native-hadoop code!


Which I am trying to resolve




Mich Talebzadeh




Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache


NOTE: The information in this email is proprietary and confidential. This message is for the
designated recipient only, if you are not the intended recipient, you should destroy it immediately.
Any information in this message shall not be understood as given or endorsed by Peridale Ltd,
its subsidiaries or their employees, unless expressly so stated. It is the responsibility
of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd,
its subsidiaries nor their employees accept any responsibility.


-----Original Message-----
From: xeonmailinglist-gmail [mailto:xeonmailinglist@gmail.com] 
Sent: 04 April 2015 16:16
To: user@hadoop.apache.org
Subject: How to create 3GB of compressed sequence file?





I want to create 3GB of compressed sequence file in Hadoop. How can I do this?






View raw message