hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijay <>
Subject Re: How to load a web log file (text format) to Hive with compression
Date Tue, 12 Nov 2013 04:25:49 GMT
Hive has built-in support for gzipped text files so you don't need to do
anything special with them (meaning you just load them like regular files).
However, there is one important limitation with this approach these files
are not splittable. Depending on the size of your log files and how many of
them you have perday that may not be a problem. The other, slightly more
complicated approach is to use sequence files, which support splittable
compression. You can find more details here:

On Mon, Nov 11, 2013 at 5:09 PM, Raj Hadoop <> wrote:

> Hi,
> I have a web log files (text format). I want to load these files to a Hive
> table in compressed format. How do I do it ?
> Should I compress the text file (using any Linux utilities) and then
> create the Hive table?
> Can any one provide me the Hive syntax for loading the compressed file?
> Thanks,
> Raj

View raw message