hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <shouvanik.hal...@accenture.com>
Subject RE: Issue with creating HIVE metadata for a HBASE table with 2000 + columns
Date Mon, 10 Jun 2013 07:33:41 GMT
Hi,

"if your values for the column on postgres are more than 4000 chars then I would recommend
you alter your postgres meta data table to have a bigger limit."

How to do the above?

Actually, while looking at the logs, I found out that /usr/lib/hive/scripts/metastore/upgrade/postgres
location, a file resides of the name -> hive-schema-0.10.0.postgres.sql.

Inside, the file,
CREATE TABLE "SERDE_PARAMS" (
    "SERDE_ID" bigint NOT NULL,
    "PARAM_KEY" character varying(256) NOT NULL,
    "PARAM_VALUE" character varying(4000) DEFAULT NULL::character varying
);

What are the steps to do the same? Please help

Regards,
Shouvanik


From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Monday, June 10, 2013 12:58 PM
To: user@hive.apache.org
Subject: Re: Issue with creating HIVE metadata for a HBASE table with 2000 + columns

I just took a look at the error stack and hive schema definition.

>From error, it looks like you are hitting the char length limit on postgres schema table
SERDE_PARAMS.

if your values for the column on postgres are more than 4000 chars then I would recommend
you alter your postgres meta data table to have a bigger limit.

On Mon, Jun 10, 2013 at 12:51 PM, Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
wrote:
Can you share your hive version?


On Sat, Jun 8, 2013 at 12:03 AM, Stephen Sprague <spragues@gmail.com<mailto:spragues@gmail.com>>
wrote:
I would venture to say if you haven't got a reply nobody particularly has anything useful
to add.

you have a meta data error there. the error message shows you the table name. You're going
to have to dig in there on your own and start with a some trivial example and work your way
up to where it fails.
just showing us an error message is like playing a game of guess which number i'm thinking
of.  Show us something that does work; then show us something slightly different that doesn't
work and we might have a chance at proposing some ideas.

On Fri, Jun 7, 2013 at 1:50 AM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
Hi,

Can anybody please reply. I thought this forum is ACTIVE?

-----Original Message-----
From: Haldar, Shouvanik
Sent: Thursday, June 06, 2013 2:08 PM
To: 'user@hive.apache.org<mailto:user@hive.apache.org>'
Subject: Issue with creating HIVE metadata for a HBASE table with 2000 + columns

Hi,

I have a HBASE table with 2000 + columns. I have to create a HIVE metadata. But am facing
issue while creating the HIVE table

FAILED: Error in metadata: MetaException(message:javax.jdo.JDODataStoreException: Put request
failed : INSERT INTO "SERDE_PARAMS" ("PARAM_VALUE","SERDE_ID","PARAM_KEY") VALUES (?,?,?)
NestedThrowables:
org.datanucleus.store.mapped.exceptions.MappedDatastoreException: INSERT INTO "SERDE_PARAMS"
("PARAM_VALUE","SERDE_ID","PARAM_KEY") VALUES (?,?,?) )
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask


Please help!

Regards,
Shouvanik

-----Original Message-----
From: user-help@hive.apache.org<mailto:user-help@hive.apache.org> [mailto:user-help@hive.apache.org<mailto:user-help@hive.apache.org>]
Sent: Thursday, June 06, 2013 12:42 PM
To: Haldar, Shouvanik
Subject: WELCOME to user@hive.apache.org<mailto:user@hive.apache.org>

Hi! This is the ezmlm program. I'm managing the user@hive.apache.org<mailto:user@hive.apache.org>
mailing list.

Acknowledgment: I have added the address

   shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>

to the user mailing list.

Welcome to user@hive.apache.org<mailto:user@hive.apache.org>!

Please save this message so that you know the address you are subscribed under, in case you
later want to unsubscribe or change your subscription address.


--- Administrative commands for the user list ---

I can handle administrative requests automatically. Please do not send them to the list address!
Instead, send your message to the correct command address:

To subscribe to the list, send a message to:
   <user-subscribe@hive.apache.org<mailto:user-subscribe@hive.apache.org>>

To remove your address from the list, send a message to:
   <user-unsubscribe@hive.apache.org<mailto:user-unsubscribe@hive.apache.org>>

Send mail to the following for info and FAQ for this list:
   <user-info@hive.apache.org<mailto:user-info@hive.apache.org>>
   <user-faq@hive.apache.org<mailto:user-faq@hive.apache.org>>

Similar addresses exist for the digest list:
   <user-digest-subscribe@hive.apache.org<mailto:user-digest-subscribe@hive.apache.org>>
   <user-digest-unsubscribe@hive.apache.org<mailto:user-digest-unsubscribe@hive.apache.org>>

To get messages 123 through 145 (a maximum of 100 per request), mail:
   <user-get.123_145@hive.apache.org<mailto:user-get.123_145@hive.apache.org>>

To get an index with subject and author for messages 123-456 , mail:
   <user-index.123_456@hive.apache.org<mailto:user-index.123_456@hive.apache.org>>

They are always returned as sets of 100, max 2000 per request, so you'll actually get 100-499.

To receive all messages with the same subject as message 12345, send a short message to:
   <user-thread.12345@hive.apache.org<mailto:user-thread.12345@hive.apache.org>>

The messages should contain one line or word of text to avoid being treated as sp@m, but I
will ignore their content.
Only the ADDRESS you send to is important.

You can start a subscription for an alternate address, for example "john@host.domain<mailto:john@host.domain>",
just add a hyphen and your address (with '=' instead of '@') after the command word:
<user-subscribe-john=host.domain@hive.apache.org<mailto:host.domain@hive.apache.org>>

To stop subscription for this address, mail:
<user-unsubscribe-john=host.domain@hive.apache.org<mailto:host.domain@hive.apache.org>>

In both cases, I'll send a confirmation message to that address. When you receive it, simply
reply to it to complete your subscription.

If despite following these instructions, you do not get the desired results, please contact
my owner at user-owner@hive.apache.org<mailto:user-owner@hive.apache.org>. Please be
patient, my owner is a lot slower than I am ;-)

--- Enclosed is a copy of the request I received.

Return-Path: <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
Received: (qmail 39420 invoked by uid 99); 6 Jun 2013 07:11:56 -0000
Received: from athena.apache.org<http://athena.apache.org> (HELO athena.apache.org<http://athena.apache.org>)
(140.211.11.136)
    by apache.org<http://apache.org> (qpsmtpd/0.29) with ESMTP; Thu, 06 Jun 2013 07:11:56
+0000
X-ASF-Spam-Status: No, hits=-0.7 required=5.0
        tests=RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org<http://apache.org>
Received-SPF: pass (athena.apache.org<http://athena.apache.org>: domain of shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>
designates 216.32.180.187 as permitted sender)
Received: from [216.32.180.187] (HELO co1outboundpool.messaging.microsoft.com<http://co1outboundpool.messaging.microsoft.com>)
(216.32.180.187)
    by apache.org<http://apache.org> (qpsmtpd/0.29) with ESMTP; Thu, 06 Jun 2013 07:11:52
+0000
Received: from mail209-co1-R.bigfish.com<http://mail209-co1-R.bigfish.com> (10.243.78.241)
by  CO1EHSOBE016.bigfish.com<http://CO1EHSOBE016.bigfish.com> (10.243.66.79) with Microsoft
SMTP Server id  14.1.225.23; Thu, 6 Jun 2013 07:11:31 +0000
Received: from mail209-co1 (localhost [127.0.0.1])      by
 mail209-co1-R.bigfish.com<http://mail209-co1-R.bigfish.com> (Postfix) with ESMTP id
91A202401C7  for
 <user-sc.1370502547.pbccgcglodompekokpie-shouvanik.haldar=accenture.com@hive.apache.org<mailto:accenture.com@hive.apache.org>>;
 Thu,  6 Jun 2013 07:11:31 +0000 (UTC)
X-Forefront-Antispam-Report: CIP:170.252.43.196<tel:170.252.43.196>;KIP:(null);UIP:(null);IPV:NLI;H:amrxe3001.dir.svc.accenture.com<http://amrxe3001.dir.svc.accenture.com>;RD:amrxe3001.accenture.com<http://amrxe3001.accenture.com>;EFVD:NLI
X-SpamScore: -2
X-BigFish: VPS-2(z5687k56bfiz9371I542Izz1f42h1ee6h1de0h1fdah1202h1e76h1d1ah1d2ah1fc6hzz8275ch17326ah186M18602eh8275bh8275dh1cd15fiz2fh2a8h668h839h944hd25hf0ah1220h1288h12a5h12a9h12bdh137ah13b6h1441h1504h1537h153bh15d0h162dh1631h1758h18e1h1946h19b5h1b0ah1d0ch1d2eh1d3fh1dfeh1dffh1e10h1e1dh1155h)
Received-SPF: pass (mail209-co1: domain of accenture.com<http://accenture.com> designates
170.252.43.196<tel:170.252.43.196> as permitted sender) client-ip=170.252.43.196<tel:170.252.43.196>;
envelope-from=shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>;
helo=amrxe3001.dir.svc.accenture.com<http://amrxe3001.dir.svc.accenture.com> ;ccenture.com<http://ccenture.com>
;
Received: from mail209-co1 (localhost.localdomain [127.0.0.1]) by mail209-co1
 (MessageSwitch) id 1370502635940071_26153; Thu,  6 Jun 2013 07:10:35 +0000
 (UTC)
Received: from CO1EHSMHS019.bigfish.com<http://CO1EHSMHS019.bigfish.com> (unknown [10.243.78.251])
      by
 mail209-co1.bigfish.com<http://mail209-co1.bigfish.com> (Postfix) with ESMTP id 62070A80090
   for
 <user-sc.1370502547.pbccgcglodompekokpie-shouvanik.haldar=accenture.com@hive.apache.org<mailto:accenture.com@hive.apache.org>>;
 Thu,  6 Jun 2013 07:10:28 +0000 (UTC)
Received: from amrxe3001.dir.svc.accenture.com<http://amrxe3001.dir.svc.accenture.com>
(170.252.43.196<tel:%28170.252.43.196>) by  CO1EHSMHS019.bigfish.com<http://CO1EHSMHS019.bigfish.com>
(10.243.66.29) with Microsoft SMTP Server (TLS) id  14.1.225.23; Thu, 6 Jun 2013 07:10:28
+0000
Received: from AMRXH3006.dir.svc.accenture.com<http://AMRXH3006.dir.svc.accenture.com>
(10.63.34.50) by  amrxe3001.accenture.com<http://amrxe3001.accenture.com> (10.63.35.191)
with Microsoft SMTP Server id  8.3.298.1; Thu, 6 Jun 2013 03:11:10 -0400
Received: from email.accenture.com<http://email.accenture.com> (170.248.175.71<tel:%28170.248.175.71>)
by  AMRXH3006.dir.svc.accenture.com<http://AMRXH3006.dir.svc.accenture.com> (10.63.34.76)
with Microsoft SMTP Server
 (TLS) id 8.3.298.1; Thu, 6 Jun 2013 03:11:52 -0400
Received: from 048-CH1MPN2-281.048d.mgd.msft.net<http://048-CH1MPN2-281.048d.mgd.msft.net>
([169.254.1.183]) by  048-CH1MMR1-032.048d.mgd.msft.net<http://048-CH1MMR1-032.048d.mgd.msft.net>
([170.248.175.71]) with mapi id  14.02.0328.011; Thu, 6 Jun 2013 07:11:23 +0000
From: <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
To:
        <user-sc.1370502547.pbccgcglodompekokpie-shouvanik.haldar=accenture.com@hive.apache.org<mailto:accenture.com@hive.apache.org>>
Subject: RE: confirm subscribe to user@hive.apache.org<mailto:user@hive.apache.org>
Thread-Topic: confirm subscribe to user@hive.apache.org<mailto:user@hive.apache.org>
Thread-Index: AQHOYoTMTXy1e1YHhkilKi/plTgpAJkoRLBQ
Date: Thu, 6 Jun 2013 07:10:07 +0000
Message-ID: <D4A61AEE32B48844B37951EFDBC6C21B16FADC73@048-CH1MPN2-281.048d.mgd.msft.net<mailto:D4A61AEE32B48844B37951EFDBC6C21B16FADC73@048-CH1MPN2-281.048d.mgd.msft.net>>
References: <1370502547.37782.ezmlm@hive.apache.org<mailto:1370502547.37782.ezmlm@hive.apache.org>>
In-Reply-To: <1370502547.37782.ezmlm@hive.apache.org<mailto:1370502547.37782.ezmlm@hive.apache.org>>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.13.37.130]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: accenture.com<http://accenture.com>
X-Virus-Checked: Checked by ClamAV on apache.org<http://apache.org>



This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.

Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy.

______________________________________________________________________________________

www.accenture.com<http://www.accenture.com>




--
Nitin Pawar



--
Nitin Pawar

Mime
View raw message