hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Gonzalez <zlgonza...@yahoo.com>
Subject Re: Need command to compress the files
Date Thu, 30 Jul 2015 00:37:09 GMT
I think you can pick the compression algorithm when using sqoop - either 
deflate or snappy when specifying the --compress option.
Is that what you were asking?

Thanks,
Ron

On 07/29/2015 03:40 PM, Ted Yu wrote:
> You can use the following command to see options for gzip:
> gzip -h
>
> For snappy, see:
> https://github.com/kubo/snzip
> https://code.google.com/p/snappy/issues/detail?id=34
>
> FYI
>
> On Wed, Jul 29, 2015 at 3:34 PM, SP <sajidmca@gmail.com 
> <mailto:sajidmca@gmail.com>> wrote:
>
>     Hi All,
>
>     I am working on comparing different compression ratios.
>
>     I have these files in AVRO format. How can I compress them using
>     snappy or gzip.
>
>     -rw-r--r--   3 hdfs supergroup 3080866838 2015-07-29 18:16
>     /tmp/fact_splitby_date_id/part-m-00000.avro
>     -rw-r--r--   3 hdfs supergroup 3021258762 2015-07-29 18:15
>     /tmp/fact_splitby_date_id/part-m-00001.avro
>     -rw-r--r--   3 hdfs supergroup 3164101762 2015-07-29 18:17
>     /tmp/fact_splitby_date_id/part-m-00002.avro
>     -rw-r--r--   3 hdfs supergroup 3251578205 2015-07-29 18:16
>     /tmp/fact_splitby_date_id/part-m-00003.avro
>
>
>
>
>     Thanks
>     Sp
>
>


Mime
View raw message