hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kirk True <k...@mustardgrain.com>
Subject Re: millions of columns in Hive
Date Thu, 22 Dec 2011 18:49:43 GMT
IIRC, the names and types of the columns are kept in a *_PARAMS table in 
the metastore. At one point (depending on your DB) this was only ~4000 
characters and thus posed a limitation on the number of columns one 
could use.

I believe there was work to turn some of the *_PARAMS table columns into 
CLOBs (or equivalent).

Kirk

On 12/20/11 11:08 AM, ameet chaubal wrote:
> Hi all,
>
> is there a limitation in Hive to create a large number of columns, say 
> a few million or so? If one is using MySQL for metastore, where would 
> this limitation come from, if so?
>
> thanks,
> Sincerely,
>
> Ameet

Mime
View raw message