avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <scottca...@apache.org>
Subject Re: Avro + Snappy changing blocksize of snappy compression
Date Wed, 18 Apr 2012 21:18:08 GMT
Try a range from smaller block sizes (4k) and up.  256K is a larger block
size than many compression codecs are sensitive to.

Also for reference, try it with the deflate codec at a couple different
compression levels -- 1, 3, 5, and 7 should show a trend with respect to
block size.  As the compression level increases, the compressor can take
advantage of larger blocks.

In the deflate/gzip case that I have explored heavily, the effectiveness
of the block size also varies significantly depending on the
characteristics of the data being compressed.

(note: gzip uses deflate compression)

On 4/18/12 1:33 PM, "snikhil0" <snikhil@telenav.com> wrote:

>I had tried the sync Interval as well and I get the same results: meaning
>change in final avro data file.
>View this message in context:
>Sent from the Avro - Users mailing list archive at Nabble.com.

View raw message