hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Nanda <>
Subject Re: Re: bz2 Splits.
Date Tue, 28 Jul 2009 05:02:19 GMT
> The 1600MB number looks like record-level compression. Are you sure
> you've turned on block compression?

Here's the exact snippet from my shell script. Do I have to set these
configuration parameters directly in the hadoop configuration file:

    ${HIVE_COMMAND} -e "set hive.exec.compress.output=true; set
io.seqfile.compression.type=BLOCK; set; set
io.seqfile.compress.blocksize=50000000; insert overwrite table
raw_compressed partition(dt='${D}') select line from raw where dt='${D}'"


View raw message