hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Larry Ogrodnek <>
Subject feature request: allow external tables with different partition directory structure
Date Mon, 12 Oct 2009 17:32:21 GMT
This was actually a previous feature request by someone else:

The implemented solution was to allow the location to be specified in
the "alter table .. add partition" command.

This has worked pretty well for us, however with the recent addition
of hive support in amazon's elastic map-reduce, it would be really
convenient if we could set it up so that our data in s3 could be
auto-discovered without having to add partitions... (it's of the
format bucket/data-source/YYYYMMDD[HH]).

Amazon has support for simple key/val substitution in scripts, but
since there's no kind of iteration/looping available, it's not really
useful if you need to add a couple of weeks worth of partitions as
part of your script....

If we could somehow define the partition directory layout, it would
really simplify our hive scripting with elastic mapreduce...


View raw message