hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cam Bazz <camb...@gmail.com>
Subject Re: for each partition
Date Wed, 09 Feb 2011 13:49:44 GMT
Well, I designed my dataflow to work incrementally based on
partitions. But I have a number of datafiles now,
and for the first run, I have to for example:

insert overwrite table item_view_aggregate partition
(date_hour=2011310116) select iv.sid, count(*) from item_view iv where
iv.date_hour='2011310116' group by iv.sid;

..

I have to repeat this for each partition. so I need something like a
for loop, or do it with an external program.

best regards,
-c.b.

On Wed, Feb 9, 2011 at 7:33 AM, Christopher, Pat
<patrick.christopher@hp.com> wrote:
> If you want to operate over all partitions in a table you don't need to specify the partitions
at all.  Run your query and enjoy!
>
> If you want to specify the partition mapping of the output dataset from a query, I think
you can derive that value on a per row basis like so:
>
>  Partition=substr(dateval,1,14)
>
> Assuming dateval is a field in your table and partition is your partition field.
>
> Lemme know if you need more or if it doesn't work.  I'll check it out tomorrow at work.
>
> Pat
>
>
>
> -- Sent from my Palm Pre
>
> ________________________________
> On Feb 8, 2011 8:57 PM, Cam Bazz <cambazz@gmail.com> wrote:
>
> Hello,
>
> How can I do some process for each partition in some other table.
>
> for example lets say table A has partitions 1,2,3
>
> I want to be able to say
>
> for each partition in A do {
> select * from A where partition is ? into some othertable where partition is ?
> }
>
> Best Regards,
> C.B.
>

Mime
View raw message