hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KayVajj <>
Subject Basic question regarding calculating number of reducers
Date Fri, 27 Jun 2014 07:07:43 GMT

I have a basic question regarding the calculation of the number of reducers
in hive. I know that is computed as <Total-Input-Size>/<Bytes-Per-Reducer>.

In case of compressed files it is not clear whether total input size is
calculated when compressed or decompressed. Doesn't it make a significant
difference if calculated when compressed. I have tried checking the API but
it looks like it is getting the file sizes from the Namenode using the
getFileInfo API call. This tells me that it calculated when compressed.


View raw message