Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Content-Transfer-Encoding: 7bit
Content-Type: multipart/alternative;
	boundary=Apple-Mail-03E965B7-AB8D-4539-A83E-F60A04AD55C8
From: Hadoop User <kjayapal17@gmail.com>
Mime-Version: 1.0 (1.0)
Date: Wed, 29 Jul 2015 18:36:22 -0700
Message-Id: <197C0850-58BD-493F-99CC-8799D94DE5C6@gmail.com>
Subject: Re: Need command to compress the files
References: 
 <CAGSm+-tuQxpundw0OLzj_1q3W6j2VmoMDLh14qv7=cA8x9h5Dg@mail.gmail.com>
 <CALte62zexy2nBiUzzpMzF6Y8fy4md87tTN9FZHC-A1nVHihxqA@mail.gmail.com>
 <55B971B5.8040603@yahoo.com>
In-Reply-To: <55B971B5.8040603@yahoo.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>


--Apple-Mail-03E965B7-AB8D-4539-A83E-F60A04AD55C8
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

I already have the data in HDFS. I want to test compression ratio with gzip a=
nd snappy.

Thanks=20
Sajid

Sent from my iPhone

> On Jul 29, 2015, at 5:37 PM, Ron Gonzalez <zlgonzalez@yahoo.com> wrote:
>=20
> I think you can pick the compression algorithm when using sqoop - either d=
eflate or snappy when specifying the --compress option.
> Is that what you were asking?
>=20
> Thanks,
> Ron
>=20
>> On 07/29/2015 03:40 PM, Ted Yu wrote:
>> You can use the following command to see options for gzip:
>> gzip -h
>>=20
>> For snappy, see:
>> https://github.com/kubo/snzip
>> https://code.google.com/p/snappy/issues/detail?id=3D34
>>=20
>> FYI
>>=20
>>> On Wed, Jul 29, 2015 at 3:34 PM, SP <sajidmca@gmail.com> wrote:
>>> Hi All,
>>>=20
>>> I am working on comparing different compression ratios.=20
>>>=20
>>> I have these files in AVRO format. How can I compress them using snappy o=
r gzip.
>>>=20
>>> -rw-r--r--   3 hdfs supergroup 3080866838 2015-07-29 18:16 /tmp/fact_spl=
itby_date_id/part-m-00000.avro
>>> -rw-r--r--   3 hdfs supergroup 3021258762 2015-07-29 18:15 /tmp/fact_spl=
itby_date_id/part-m-00001.avro
>>> -rw-r--r--   3 hdfs supergroup 3164101762 2015-07-29 18:17 /tmp/fact_spl=
itby_date_id/part-m-00002.avro
>>> -rw-r--r--   3 hdfs supergroup 3251578205 2015-07-29 18:16 /tmp/fact_spl=
itby_date_id/part-m-00003.avro
>>>=20
>>>=20
>>>=20
>>>=20
>>> Thanks
>>> Sp

--Apple-Mail-03E965B7-AB8D-4539-A83E-F60A04AD55C8
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: 7bit

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div><span></span></div><div><div>I already have the data in HDFS. I want to test compression ratio with gzip and snappy.</div><div><br></div><div>Thanks&nbsp;</div><div>Sajid<br><br>Sent from my iPhone</div><div><br>On Jul 29, 2015, at 5:37 PM, Ron Gonzalez &lt;<a href="mailto:zlgonzalez@yahoo.com">zlgonzalez@yahoo.com</a>&gt; wrote:<br><br></div><blockquote type="cite"><div>
  
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  
  
    I think you can pick the compression algorithm when using sqoop -
    either deflate or snappy when specifying the --compress option.<br>
    Is that what you were asking?<br>
    <br>
    Thanks,<br>
    Ron<br>
    <br>
    <div class="moz-cite-prefix">On 07/29/2015 03:40 PM, Ted Yu wrote:<br>
    </div>
    <blockquote cite="mid:CALte62zexy2nBiUzzpMzF6Y8fy4md87tTN9FZHC-A1nVHihxqA@mail.gmail.com" type="cite">
      <div dir="ltr">You can use the following command to see options
        for gzip:<br>
        <div>gzip -h<br>
        </div>
        <div><br>
        </div>
        <div>For snappy, see:</div>
        <div><a moz-do-not-send="true" href="https://github.com/kubo/snzip">https://github.com/kubo/snzip</a><br>
        </div>
        <div><a moz-do-not-send="true" href="https://code.google.com/p/snappy/issues/detail?id=34">https://code.google.com/p/snappy/issues/detail?id=34</a></div>
        <div><br>
        </div>
        <div>FYI</div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Wed, Jul 29, 2015 at 3:34 PM, SP <span dir="ltr">&lt;<a moz-do-not-send="true" href="mailto:sajidmca@gmail.com" target="_blank">sajidmca@gmail.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">Hi All,
              <div><br>
              </div>
              <div>I am working on comparing different compression
                ratios.&nbsp;</div>
              <div><br>
              </div>
              <div>I have these files in AVRO format. How can I compress
                them using snappy or gzip.</div>
              <div><br>
              </div>
              <div>
                <div>-rw-r--r-- &nbsp; 3 hdfs supergroup 3080866838
                  2015-07-29 18:16
                  /tmp/fact_splitby_date_id/part-m-00000.avro</div>
                <div>-rw-r--r-- &nbsp; 3 hdfs supergroup 3021258762
                  2015-07-29 18:15
                  /tmp/fact_splitby_date_id/part-m-00001.avro</div>
                <div>-rw-r--r-- &nbsp; 3 hdfs supergroup 3164101762
                  2015-07-29 18:17
                  /tmp/fact_splitby_date_id/part-m-00002.avro</div>
                <div>-rw-r--r-- &nbsp; 3 hdfs supergroup 3251578205
                  2015-07-29 18:16
                  /tmp/fact_splitby_date_id/part-m-00003.avro</div>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div>Thanks</div>
              <span class="HOEnZb"><font color="#888888">
                  <div>Sp</div>
                </font></span></div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  

</div></blockquote>
</div></body></html>
--Apple-Mail-03E965B7-AB8D-4539-A83E-F60A04AD55C8--