hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliot West <tea...@gmail.com>
Subject Re: Interrogating a uniontype
Date Thu, 24 Nov 2016 17:07:24 GMT
Thanks for your reply. We just did something similar to this, implementing
a UDF that returns a struct containing a field per union subtype. This data
structure is then easily navigated using standard HQL operators. I'm
surprised such a feature doesn't already exist.

On Wed, 23 Nov 2016 at 21:50, Per Ullberg <per.ullberg@klarna.com> wrote:

> Could you write a UDF that parses it and returns a json object. From there
> you can use the standard json support in Hive. I did something similar for
> Erlang structs about 3 years ago. I actually kept them on file and wrote a
> serde that exposed them as json objects.
>
> regards
> /Pelle
>
> On Wed, Nov 23, 2016 at 6:40 PM, Elliot West <teabot@gmail.com> wrote:
>
> Ah, I see that this can't be done with an array as there is no type common
> to all union indexes. Perhaps a struct with one field per indexed type?
>
> On Wed, 23 Nov 2016 at 17:29, Elliot West <teabot@gmail.com> wrote:
>
> Can anyone recommend a good approach for interrogating uniontype values in
> HQL? I note that the documentation states that the support for such types
> is limited to 'look-at-only' which I assume to mean that I may only dump
> out the value in its entirety, and extract sub-elements. Using the example
> below, is there anyway I can get to field 'b' of union index 3 to extract
> only the value 5?
>
> {0:1}
> {1:2.0}
> {2:["three","four"]}
> {3:{"a":5,"b":"five"}}
>
>
> If not possible with HQL, would it be possible to implement a UDF that can
> explode the type into something more navigable, like an array, struct, or
> map?
>
> Example when exploded as array:
>
>
> [1,null,null,null]
> [null,2.0,null,null]
> [null,null,["three","four"],null]
> [null,null,null,{"a":5,"b":"five"}]
>
> Has anyone done this?
>
> Thanks,
>
> Elliot.
>
>
>
>
> --
>
> *Per Ullberg*
> Data Vault Tech Lead
> Odin Uppsala
> +46 701612693 <+46+701612693>
>
> Klarna AB (publ)
> Sveavägen 46, 111 34 Stockholm
> Tel: +46 8 120 120 00 <+46812012000>
> Reg no: 556737-0431
> klarna.com
>
>

Mime
View raw message