Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (athena.apache.org: domain of njain@fb.com designates
 66.220.144.132 as permitted sender)
From: Namit Jain <njain@fb.com>
To: "user@hive.apache.org" <user@hive.apache.org>
Subject: Re: for each partition
Thread-Topic: for each partition
Thread-Index: AcvIFexjayeFrvrRDUmM+rTjPvZyaQABPfETACIXuAD//6TlAA==
Date: Wed, 9 Feb 2011 16:23:30 +0000
Message-ID: <C977FF4D.1E67A%njain@fb.com>
In-Reply-To: <AANLkTimMKcpwzBeiBg+hOi5=oTgtOCraE2OEgnWZs_0O@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.0.0.100825
Content-Type: text/plain; charset="us-ascii"
Content-ID: <FFB0CBBAF5A9064592CB9969F1E66499@fb.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

You can use dynamic partitioning:


insert overwrite table item_view_aggregate partition
(date_hour) select iv.sid, count(*), date_hour from item_view iv where
(iv.date_hour=3D'2011310116' or date_hour=3D''' or date_hour=3D'.....)
group by iv.sid, date_hour;


On 2/9/11 5:49 AM, "Cam Bazz" <cambazz@gmail.com> wrote:

>Well, I designed my dataflow to work incrementally based on
>partitions. But I have a number of datafiles now,
>and for the first run, I have to for example:
>
>insert overwrite table item_view_aggregate partition
>(date_hour=3D2011310116) select iv.sid, count(*) from item_view iv where
>iv.date_hour=3D'2011310116' group by iv.sid;
>
>..
>
>I have to repeat this for each partition. so I need something like a
>for loop, or do it with an external program.
>
>best regards,
>-c.b.
>
>On Wed, Feb 9, 2011 at 7:33 AM, Christopher, Pat
><patrick.christopher@hp.com> wrote:
>> If you want to operate over all partitions in a table you don't need to
>>specify the partitions at all.  Run your query and enjoy!
>>
>> If you want to specify the partition mapping of the output dataset from
>>a query, I think you can derive that value on a per row basis like so:
>>
>>  Partition=3Dsubstr(dateval,1,14)
>>
>> Assuming dateval is a field in your table and partition is your
>>partition field.
>>
>> Lemme know if you need more or if it doesn't work.  I'll check it out
>>tomorrow at work.
>>
>> Pat
>>
>>
>>
>> -- Sent from my Palm Pre
>>
>> ________________________________
>> On Feb 8, 2011 8:57 PM, Cam Bazz <cambazz@gmail.com> wrote:
>>
>> Hello,
>>
>> How can I do some process for each partition in some other table.
>>
>> for example lets say table A has partitions 1,2,3
>>
>> I want to be able to say
>>
>> for each partition in A do {
>> select * from A where partition is ? into some othertable where
>>partition is ?
>> }
>>
>> Best Regards,
>> C.B.
>>