hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopal Vijayaraghavan <>
Subject Re: HDFS small files to Sequence file using Hive
Date Fri, 23 Sep 2016 23:16:47 GMT

> Is there a way to create an external table on a directory, extract 'key' as file name
and 'value' as file content and write to a sequence file table?

Do you care that it is a sequence file?

The HDFS HAR format was invented for this particular problem, check if the "hadoop archive"
command works for you and offers a filesystem abstraction.

Otherwise, there's always the old Mahout "seqdirectory" job, which is great if you have like
.jpg files and want to pack them for HDFS to handle better (like GPS tiles).


View raw message