hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish Shenoy <ashe...@instartlogic.com>
Subject Set the ORC file HDFS block size to 64MB
Date Thu, 30 Jul 2015 00:22:51 GMT
Hi,

I am using the OrcFile.createWriter() function to get an ORC writer to
write ORC files. However, upon inspecting the ORC files in HDFS, I notice
that they have a block size of 256MB. Now, I would like to set it to 64MB
instead.

I digged around the net and tried a few options:
- I set the "dfs.block.size" config property to 64MB and passed it to the
createWriter() call via OrcFile.writerOptions(config)
- I set the "hive.exec.orc.default.block.size" to 64MB in hive-site.xml
file via ambari and restarted hive.

Neither of these approaches worked for me. I still see that my files have a
block size of 256MB which corresponds to the default setting for the block
size for ORC files. How can I set the block size of ORC files to 64MB ?

Thanks,
Ashish

Mime
View raw message