hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Meagher <john.meag...@gmail.com>
Subject Can data be passed to the final mode init call in a UDAF?
Date Mon, 10 Feb 2014 21:26:33 GMT
I'm working on a UDAF that takes in a constant string that defines
what the final output of the UDAF will be.  In the mode=PARTIAL1 call
to the init function all the parameters are available and the constant
can be read so the output ObjectInspector can be built.  I haven't
found a way to pass this same constant data through so it is available
to the mode=FINAL call to init.

Is there a way to pass constant parameters through the UDAF so it is
available in the mode=FINAL init call?

I have tried having the PARTIAL1 init call return a
StructObjectInspector containing constant object inspectors with the
data, but when that is read during the FINAL call the incoming
ObjectInspectors are no longer constants.

I've looked for ways to access the JobConf object for the job to try
to pass this parameter along, but I can't find a way to access that
from Hive.  I can pass the data along with the data coming from
terminatePartial, but that data is not available until merge is called
and init in FINAL mode may be called without another invocation of
init.

Note: this is with Hive 0.10.  While I'd like to get this working
under 0.10 I'd like to know if this is a feature available in newer
releases.

Thanks,
John

Mime
View raw message