Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (athena.apache.org: domain of john.meagher@gmail.com
 designates 209.85.215.54 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAKb7DdhC7jUmmjwZC53BBOu=ajh889Yx75P55nQAY5ZxWtKmzw@mail.gmail.com>
References: 
 <CAKb7DdhC7jUmmjwZC53BBOu=ajh889Yx75P55nQAY5ZxWtKmzw@mail.gmail.com>
Date: Wed, 12 Feb 2014 09:41:14 -0500
Message-ID: 
 <CAKb7DdgjjOLcVsKXEnnhezp+Jt-vu422sTvx1-0juKQKRNk7Og@mail.gmail.com>
Subject: Re: Can data be passed to the final mode init call in a UDAF?
From: John Meagher <john.meagher@gmail.com>
To: user@hive.apache.org
Content-Type: text/plain; charset=ISO-8859-1

The code I'm having problems with is
https://github.com/livingsocial/HiveSwarm/blob/WIP_scripting/src/main/java/com/livingsocial/hive/udf/ScriptedUDAF.java.

There is a FIXME comment at the problem area where the return type for
the init method can not be determined.

On Mon, Feb 10, 2014 at 4:26 PM, John Meagher <john.meagher@gmail.com> wrote:
> I'm working on a UDAF that takes in a constant string that defines
> what the final output of the UDAF will be.  In the mode=PARTIAL1 call
> to the init function all the parameters are available and the constant
> can be read so the output ObjectInspector can be built.  I haven't
> found a way to pass this same constant data through so it is available
> to the mode=FINAL call to init.
>
> Is there a way to pass constant parameters through the UDAF so it is
> available in the mode=FINAL init call?
>
> I have tried having the PARTIAL1 init call return a
> StructObjectInspector containing constant object inspectors with the
> data, but when that is read during the FINAL call the incoming
> ObjectInspectors are no longer constants.
>
> I've looked for ways to access the JobConf object for the job to try
> to pass this parameter along, but I can't find a way to access that
> from Hive.  I can pass the data along with the data coming from
> terminatePartial, but that data is not available until merge is called
> and init in FINAL mode may be called without another invocation of
> init.
>
> Note: this is with Hive 0.10.  While I'd like to get this working
> under 0.10 I'd like to know if this is a feature available in newer
> releases.
>
> Thanks,
> John