Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
From: Aurora Skarra-Gallagher <aurora@yahoo-inc.com>
To: "user@hive.apache.org" <user@hive.apache.org>,
        "Christopher, Pat"
	<patrick.christopher@hp.com>
CC: Steven Wong <swong@netflix.com>
Date: Fri, 11 Mar 2011 16:44:19 -0800
Subject: Re: UDAF documentation
Thread-Topic: UDAF documentation
Thread-Index: AcvgTp/5cQBz2h2HQaKJi1hUxcyJAQ==
Message-ID: <AF912632-1523-4821-BD82-CE354814BE37@yahoo-inc.com>
References: 
 <FB0065ADBE630748A6D362671F0DE4E725A937D2CC@GVW0432EXB.americas.hpqcorp.net>
 <AANLkTi=t=WD-6bO=0ra9n7CJ3r0S_uQGAFcoDhMucWrf@mail.gmail.com>
 <4F6B25AFFFCAFE44B6259A412D5F9B102DED87E0@ExchMBX104.netflix.com>
 <FB0065ADBE630748A6D362671F0DE4E725A93EEE34@GVW0432EXB.americas.hpqcorp.net>
 <E357C6E8-64BF-4D5A-BC77-759A9175AB11@yahoo-inc.com>
 <FB0065ADBE630748A6D362671F0DE4E725A93EF174@GVW0432EXB.americas.hpqcorp.net>
In-Reply-To: 
 <FB0065ADBE630748A6D362671F0DE4E725A93EF174@GVW0432EXB.americas.hpqcorp.net>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi,

Did you actually call those functions directly from your unit tests? I'm lo=
oking for examples of that working, but all I see reference to are tests to=
 make sure the query produces the expected output (rather than directly tes=
ting the UDAF).

-Aurora

On Mar 11, 2011, at 3:44 PM, Christopher, Pat wrote:

> Awesome, awesome.  That's what I had pieced together from Steve and Ed's =
emails.  Glad to get confirmation on it.
>=20
> Its also what I did for my unit testing.  I also called everything with n=
ull arguments to make sure those got handled gracefully.
>=20
> Pat
>=20
> -----Original Message-----
> From: Aurora Skarra-Gallagher [mailto:aurora@yahoo-inc.com]=20
> Sent: Friday, March 11, 2011 3:40 PM
> To: user@hive.apache.org
> Cc: Steven Wong
> Subject: Re: UDAF documentation
>=20
> Hadoop: The Definitive Guide has a good section on this. Chapter 12: Hive=
: User Defined Functions. It has a diagram that shows how things are called=
 and when. The example I'm looking at shows this sequence:
>=20
> (first instance)
> init()
> iterate(1)
> iterate(2)
> iterate(3)
> terminatePartial()
>=20
> (second instance)
> init()
> iterate(4)
> iterate(2)
> terminatePartial()
>=20
> (then)
> init()
> merge(3)
> merge(4)
> terminate()
>=20
> The UDAF being described is a max integer function, hence the merge endin=
g up with the highest integer from each instance.
>=20
> -Aurora
>=20
> On Mar 11, 2011, at 9:54 AM, Christopher, Pat wrote:
>=20
>> Ahh, perfect.  The docs don't agree terribly well but the case study is =
great.  The context for when merge() gets called was not clear to me.
>>=20
>> Thanks guys!
>>=20
>> Pat
>>=20
>> -----Original Message-----
>> From: Steven Wong [mailto:swong@netflix.com]=20
>> Sent: Thursday, March 10, 2011 6:24 PM
>> To: user@hive.apache.org
>> Cc: Christopher, Pat
>> Subject: RE: UDAF documentation
>>=20
>> Take a look at http://wiki.apache.org/hadoop/Hive/GenericUDAFCaseStudy, =
in case you haven't found it already.
>>=20
>>=20
>> -----Original Message-----
>> From: Edward Capriolo [mailto:edlinuxguru@gmail.com]=20
>> Sent: Thursday, March 10, 2011 6:18 PM
>> To: user@hive.apache.org
>> Cc: Christopher, Pat
>> Subject: Re: UDAF documentation
>>=20
>> On Thu, Mar 10, 2011 at 8:27 PM, Christopher, Pat
>> <patrick.christopher@hp.com> wrote:
>>> Hi Guys,
>>>=20
>>> I'm writing a UDAF to run against hive 0.5 or hive 0.7.  The documentat=
ion I
>>> can find says to implement UDAFEvaluator and ensure that you implement
>>> init() , aggregate() and evaluate().  However, all of the examples I ca=
n
>>> find implement init(), iterate(), merge(), terminatePartial() and
>>> terminate().
>>>=20
>>>=20
>>>=20
>>> What's the difference and where I can find the documentation on how to =
write
>>> a UDAF?
>>>=20
>>>=20
>>>=20
>>> Thanks,
>>>=20
>>> Pat
>>=20
>> At time the documentation may lag behind the code. I would checkout
>> the hive source code for the version you are working with and base
>> your work on other already existing UDAF's that are similar.
>>=20
>> Edward
>>=20
>=20