hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Namit Jain <nj...@facebook.com>
Subject RE: getting the field types of a query result
Date Mon, 29 Jun 2009 19:03:15 GMT
Had a discussion with Raghu/Zheng offline.
Basically, genFileSinkPlan is not passing the type information - that will be added, and then
LazySerDe will support serializing into json even if the types are provided


From: Ashish Thusoo [mailto:athusoo@facebook.com]
Sent: Monday, June 29, 2009 11:06 AM
To: hive-user@hadoop.apache.org
Subject: RE: getting the field types of a query result

Not sure what the issue is here, but SemanticAnalyzer does do inferencing of types including
that for udfs. I think this is not exposed out through an API, but that can be easily added.

Ashish

________________________________
From: Prasad Chakka [mailto:pchakka@facebook.com]
Sent: Monday, June 29, 2009 10:47 AM
To: hive-user@hadoop.apache.org
Subject: Re: getting the field types of a query result
SemanticAnalyzer should set the correct schema of the output result during compilation of
the query. Doing the way suggested below is not the right way. If SemanticAnalyzer is not
doing this correct thing then we should fix it.

________________________________
From: David Lerman <dlerman@videoegg.com>
Reply-To: <hive-user@hadoop.apache.org>
Date: Mon, 29 Jun 2009 08:34:25 -0700
To: <hive-user@hadoop.apache.org>
Subject: Re: getting the field types of a query result

If I'm following's Min's Jira, the challenge is that I would need to parse
the query, find the selected fields and look up their type via metaserver,
then look at all the UDF's to determine their output type given the inputs
-- which is a bit redundant since we already did all that work in executing
the query.  Min, have you settled on an approach for this in the JDBC
driver?


> From: He Yongqiang <heyongqiang@software.ict.ac.cn>
> Date: Mon, 29 Jun 2009 04:33:59 -0700
> To: <hive-user@hadoop.apache.org>
> Subject: Re: getting the field types of a query result
>
> If I understanding correctly, I think Prasad means is that the type info of
> each column is stored in Hive metadata. And you can fetch that information by
> HiveServer or JDBC client (and you need to set up a remote hive metaserver).
>
> Yongqiang
>
>On 09-6-29 下午6:42, "Min Zhou" <coderplay@gmail.com> wrote:
>> Hi,
>>
>> I've came across the same problem when developing jdbc for
>> hive(https://issues.apache.org/jira/browse/HIVE-576). it had nothing to do
>> with HiveServer and jdbc. I thought currently there is no? good way solving
>> it. SenmanticAnalyzer needs to get the result's type returned by udf/udaf for
>> building the schema you mentioned. we should also consider the user defined
>> type .
>>
>> Regards,
>> Min



On 6/29/09 11:27 AM, "hive-user-help@hadoop.apache.org"
<hive-user-help@hadoop.apache.org> wrote:

>
>
> hive-user Digest of: get
>
> Topics (messages 947 through 950):
>
> getting the field types of a query result
>         947 by: David Lerman
>         948 by: Prasad Chakka
>         949 by: Min Zhou
>         950 by: He Yongqiang
>
> Administrivia:
>
>
> --- Administrative commands for the hive-user list ---
>
> I can handle administrative requests automatically. Please
> do not send them to the list address! Instead, send
> your message to the correct command address:
>
> To subscribe to the list, send a message to:
>    <hive-user-subscribe@hadoop.apache.org>
>
> To remove your address from the list, send a message to:
>    <hive-user-unsubscribe@hadoop.apache.org>
>
> Send mail to the following for info and FAQ for this list:
>    <hive-user-info@hadoop.apache.org>
>    <hive-user-faq@hadoop.apache.org>
>
> Similar addresses exist for the digest list:
>    <hive-user-digest-subscribe@hadoop.apache.org>
>    <hive-user-digest-unsubscribe@hadoop.apache.org>
>
> To get messages 123 through 145 (a maximum of 100 per request), mail:
>    <hive-user-get.123_145@hadoop.apache.org>
>
> To get an index with subject and author for messages 123-456 , mail:
>    <hive-user-index.123_456@hadoop.apache.org>
>
> They are always returned as sets of 100, max 2000 per request,
> so you'll actually get 100-499.
>
> To receive all messages with the same subject as message 12345,
> send a short message to:
>    <hive-user-thread.12345@hadoop.apache.org>
>
> The messages should contain one line or word of text to avoid being
> treated as sp@m, but I will ignore their content.
> Only the ADDRESS you send to is important.
>
> You can start a subscription for an alternate address,
> for example "john@host.domain", just add a hyphen and your
> address (with '=' instead of '@') after the command word:
> <hive-user-subscribe-john=host.domain@hadoop.apache.org>
>
> To stop subscription for this address, mail:
> <hive-user-unsubscribe-john=host.domain@hadoop.apache.org>
>
> In both cases, I'll send a confirmation message to that address. When
> you receive it, simply reply to it to complete your subscription.
>
> If despite following these instructions, you do not get the
> desired results, please contact my owner at
> hive-user-owner@hadoop.apache.org. Please be patient, my owner is a
> lot slower than I am ;-)
>
> --- Enclosed is a copy of the request I received.
>
> Return-Path: <dlerman@videoegg.com>
> Received: (qmail 3099 invoked by uid 99); 29 Jun 2009 15:27:40 -0000
> Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
>     by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jun 2009 15:27:40 +0000
> X-ASF-Spam-Status: No, hits=-0.0 required=10.0
>         tests=SPF_PASS
> X-Spam-Check-By: apache.org
> Received-SPF: pass (nike.apache.org: local policy)
> Received: from [8.8.14.102] (HELO smtp001.evlta.videoegg.com) (8.8.14.102)
>     by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jun 2009 15:27:31 +0000
> X-IronPort-AV: E=Sophos;i="4.42,309,1243839600";
>    d="scan'208";a="2277316"
> Received: from troca1.evlta.ad.videoegg.com ([10.11.12.11])
>   by smtp002.evlta.videoegg.com with ESMTP; 29 Jun 2009 08:27:10 -0700
> Received: from troca1.evlta.ad.videoegg.com ([10.11.12.11]) by
>  troca1.evlta.ad.videoegg.com ([10.11.12.11]) with mapi; Mon, 29 Jun 2009
>  08:25:32 -0700
> From: David Lerman <dlerman@videoegg.com>
> To: "hive-user-get@hadoop.apache.org" <hive-user-get@hadoop.apache.org>
> Date: Mon, 29 Jun 2009 08:27:07 -0700
> Subject: <no subject>
> Thread-Topic: <no subject>
> Thread-Index: Acn4zg/ffeKshPhRWEmMgyYFzoSarw==
> Message-ID: <C66E558B.11D9F%dlerman@videoegg.com<C66E558B.11D9F%25dlerman@videoegg.com>>
> Accept-Language: en-US
> Content-Language: en
> X-MS-Has-Attach:
> X-MS-TNEF-Correlator:
> acceptlanguage: en-US
> Content-Type: text/plain; charset="iso-8859-1"
> Content-Transfer-Encoding: quoted-printable
> MIME-Version: 1.0
> X-Virus-Checked: Checked by ClamAV on apache.org
>
>
>
>
> ----------------------------------------------------------------------
>


Mime
View raw message