hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Meagher <>
Subject Re: Using Reflect: A thread for ideas
Date Tue, 19 Feb 2013 15:03:42 GMT
Another option for this functionality would be to use the Java scripting
API.  The basic structure of the call would be...

select script( scriptLanguage, scriptToRun, args... )

I haven't seen that in Hive, but something similar is available for Pig.
 Documentation for that is available on  There's also a
variation in Jira

On Wed, Feb 13, 2013 at 11:38 PM, John Omernik <> wrote:

> I stumbled across the little documented reflect function today. I've
> always known about it, but java scares me if it's not in a cup so I didn't
> dig.  Well today I dug, and found an awesome use case for reflect (for me)
> and wanted to share.  I also thought it would be nice to validate some
> thoughts I had on reflect, and how we could possibly share ideas on reflect
> so that folks could get more use out of this great feature of hive.
> Here's my example: A simple URL decode function:
> select url, reflect('', 'decode', url, 'utf-8') as
> decoded_url from logs
> Basically I am using the decode function of the class.
>  Pretty awesome, works great, no files to distribute either.  Even works
> through JDBC!
> Ok that being said, I realized now that the function I am trying to call
> has to return data in a simple data type.  For example, I struggle to come
> up with a simple reflect() for making an Hex MD5 out of a string because
> the built in function return an object, which have methods that can return
> what I am looking for. Which is great, but then I have to compile java
> code, distribute a jar, and then run the code. I am looking for simple like
> the URLDecoding function.
> I love this reflect feature, but I think it's probably underutilized due
> to the perceived usability issues for beginners.  So that leads me to my
> next thought. What if we brain storm here handy functions in Java that are
> not included in the standard hive language, that make the transition to
> hive well using the reflect function and the show an example of it's use? I
> went first with my URLDecode, and obviously will be looking for more, but
> have you seen some examples that we neat and worked well for you? Can you
> share?
> Perhaps if we get enough examples we could roll some of these into a wiki
> page on the hive wiki that folks can use to get over the "perceived"
> complexity of using java reflect?
> Thanks to those who have worked hard to implement features like this, it
> is truly awesome.

View raw message