hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivram Mani <shivram.m...@gmail.com>
Subject Re: [DISCUSS] Hive Complex types in \d
Date Mon, 25 Apr 2016 21:16:23 GMT
Option 2 is definitely a no for me as it breaks uniformity. We can't have
different cases display different results for widely used commands such as
\d

Option 1 is preferable, but even here I wouldn't recommend introducing
another column. Can't we instead leverage the description column
(pg_catalog.col_description)
Refer to http://www.postgresql.org/docs/9.1/static/sql-comment.html about
this attribute. Built in function col_description exists to return this
info.

On Mon, Apr 25, 2016 at 1:35 PM, Noa Horn <noapjgr@gmail.com> wrote:

> I vote for option 1.
> I think \d should match the information displayed for other tables.
> \d+ can be more flexible and contain more information that doesn't apply to
> all objects.
> For example, for a view the view statement will only be displayed with \d+.
>
>
>
> On 22 April 2016 at 16:19, Ting(Goden) Yao <tyao@pivotal.io> wrote:
>
> > i prefer option2 for both \d and \d+ we should tell users source types
> and
> > HAWQ types.
> > This is more meaningful to them.
> >
> >
> > On Fri, Apr 22, 2016 at 2:18 PM Oleksandr Diachenko <
> odiachenko@pivotal.io
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I am working on https://issues.apache.org/jira/browse/HAWQ-703 and
> > wanted
> > > to get some opinions.
> > >
> > > As for now HAWQ supports \d for Hive tables with primitive types
> > only(int,
> > > boolean, date etc).
> > > If user wants to describe Hive table which has some complex
> types(array,
> > > map, struct, uniontype) error occurs.
> > >
> > > *Example:*
> > >
> > > User have Hive table:
> > >
> > > *hive> describe hive_small_data;*
> > >
> > > *OK*
> > >
> > > *s1                  string                                  *
> > >
> > > *s2                  string                                  *
> > >
> > > *n1                  int                                     *
> > >
> > > *d1                  double*
> > > User is able to describe this table in psql:
> > >
> > > *# \d hcatalog.default.hive_small_data*
> > >
> > > *PXF Hive Table "default.hive_small_data"*
> > >
> > > * Column |  Type  *
> > >
> > > *--------+--------*
> > >
> > > * s1     | text*
> > >
> > > * s2     | text*
> > >
> > > * n1     | int4*
> > >
> > > * d1     | float8*
> > > Thus psql shows column and types mapped to HAWQ's types(string -> text,
> > int
> > > -> int4).
> > >
> > > The goal in HAWQ-703 - to be able to query Hive tables with complex
> > > columns, represent those complex types as TEXT. Long term goal will be
> > > mapping HIve's complex types to HAWQ's types when applicable.
> > >
> > > *Disclaimer*:
> > >
> > > Proposed changes will affect only Hive tables, behavior for all other
> > > objects will remain the same.
> > >
> > > *The problems:*
> > >
> > >    1. Not all Hive complex types could be mapped to HAWQ types. For
> > example
> > >    - uniontype, there is no corresponding HAWQ type.
> > >    2. Assuming fact that we do mapping Hive types to HAWQ types,
> > displaying
> > >    only HAWQ type in \d might be not enough for user and even
> confusing.
> > >
> > >
> > > *Possible options:*
> > >
> > >    1. \d behavior for Hive tables remains the same, column Type shows
> > HAWQ
> > >    type, \d+ shows additional column, Source Type, where Type=HAWQ
> type,
> > >    Source Type=Hive type.
> > >       -
> > >
> > >       *# \d hcatalog.default.reg_collections;*
> > >
> > >       *PXF Hive Table "default.reg_collections"*
> > >
> > >       * Column |  Type  *
> > >
> > >       *------------+--------*
> > >
> > >       * s1          | text*
> > >
> > >       * f1           | float4*
> > >
> > >       * a1          | text*
> > >
> > >       * m1         | text*
> > >
> > >       * sr1         | text*
> > >       -
> > >
> > >       *# \d+ hcatalog.default.reg_collections;*
> > >
> > >       *PXF Hive Table "default.reg_collections"*
> > >
> > >       * Column |  Type  | Source Type                          |*
> > >
> > >       *------------+---------+--------------------------------------+*
> > >
> > >       * s1          | text     |  string
> > >        |*
> > >
> > >       * f1           | float4  |  float
> > >        |*
> > >
> > >       * a1          | text     |  array<string>
> |*
> > >
> > >       * m1         | text     |  map<strung, float>                |*
> > >
> > >       * sr1         | text     |  struct<a:string,b:string,c:int> |*
> > >       2. \d shows three columns (Column, Type, Source Type), where
> > >    Type=HAWQ type, Source Type=Hive type.
> > >       -
> > >
> > >       *# \d+ hcatalog.default.reg_collections;*
> > >
> > >       *PXF Hive Table "default.reg_collections"*
> > >
> > >       * Column |  Type  | Source Type                          |*
> > >
> > >       *------------+---------+--------------------------------------+*
> > >
> > >       * s1          | text     |  string
> > >         |*
> > >
> > >       * f1           | float4  |  float
> > >         |*
> > >
> > >       * a1          | text     |  array<string>
> >  |*
> > >
> > >       * m1         | text     |  map<strung, float>                |*
> > >
> > >       * sr1         | text     |  struct<a:string,b:string,c:int> |*
> > >       3. \d shows Column, Type, where Type=Hive type.
> > >
> > >
> > >    -
> > >
> > >       *# \d hcatalog.default.reg_collections;*
> > >
> > >       *PXF Hive Table "default.reg_collections"*
> > >
> > >       * Column |    Type                                   |*
> > >
> > >       *------------+--------------------------------------+*
> > >
> > >       * s1          | string                                     |*
> > >
> > >       * f1           | float                                       |*
> > >
> > >       * a1          | array<string>                         |*
> > >
> > >       * m1         | map<strung, float>                 |*
> > >
> > >       * sr1         |  struct<a:string,b:string,c:int> |*
> > >
> > > I would prefer option 1.
> > > Any thoughts/opinions?
> > >
> > >
> > > Regards, Alex.
> > >
> >
>
>
>
> --
> אין דנים אדם בשעת קצת צערו
> http://www.nrg.co.il/online/1/ART1/765/515.html
>



-- 
shivram mani

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message