hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Notes from the last Hive Contributors Meeting
Date Thu, 15 Jul 2010 20:32:10 GMT
On Thu, Jul 15, 2010 at 2:02 PM, John Sichi <jsichi@facebook.com> wrote:
> Create/Drop View
>
> Note: View support is only available starting in Hive 0.6.
> I added this caveat when I added the CREATE/DROP view section to the DDL
> page.  For the most part, we've been following this convention, and I think
> we should keep it up whatever the final decision on docs is (copy from wiki
> to xdocs during release, or maintain only in xdocs).
> Personally, I find wiki much friendlier in general, but the wiki software
> (MoinMoin) we are using leaves a lot to be desired compared to mediawiki or
> Confluence.
> JVS
> On Jul 15, 2010, at 7:30 AM, Edward Capriolo wrote:
>
> So a user is running hive and reads the wiki, and says "Wow we have
> view support, let me try this" This fails because views are only in
> trunk. This gives people a general bad impression about hive because
> they expect trunk features, because they have no authoritative
> documentation on THEIR VERSION. Users can be fickle and if they hit
> incorrect documentation they start to get the impression the software
> is "buggy" suddenly they start questioning everything and bringing
> every problem to the hive administrator because even though they wrote
> a query wrong their first instinct is to "blame hive".
>
>
>
>
> I find editing xdocs EASIER then working with wiki. Wiki is great and
> all but in my travels I have to work on 5 different wiki's they all
> are slightly different in what they support and their mark up. We
> should be able to commit xdoc patches without full unit tests. Keeping
> the xdoc up to date should not be an issue because we should simply
> not accept a patch that changes/adds functionality without some xdoc.
>
> Another issue right now is there are features that are NOT documented
> anywhere. When a user asks about those features I have to send them to
> Jira tickets, often times the ticket will have a long back and forth
> where the feature is debated, or sometimes just a patch, you never see
> the full syntax, it can be very confusing,I often end up telling them
> to dig through a .q file inside a patch to figure out what this
> feature is and how to use it. While most people are good about
> updating the wiki we know that things tend to fall though the cracks.
>
> I think there is still a place for wiki, free form, multi-person
> planning, etc but I do not think a mature software product can every
> have authoritative documentation in a wiki.
>
>



On Thu, Jul 15, 2010 at 2:02 PM, John Sichi <jsichi@facebook.com> wrote:
> Create/Drop View
>
> Note: View support is only available starting in Hive 0.6.
> I added this caveat when I added the CREATE/DROP view section to the DDL
> page.  For the most part, we've been following this convention, and I think
> we should keep it up whatever the final decision on docs is (copy from wiki
> to xdocs during release, or maintain only in xdocs).
> Personally, I find wiki much friendlier in general, but the wiki software
> (MoinMoin) we are using leaves a lot to be desired compared to mediawiki or
> Confluence.
> JVS
> On Jul 15, 2010, at 7:30 AM, Edward Capriolo wrote:
>
> So a user is running hive and reads the wiki, and says "Wow we have
> view support, let me try this" This fails because views are only in
> trunk. This gives people a general bad impression about hive because
> they expect trunk features, because they have no authoritative
> documentation on THEIR VERSION. Users can be fickle and if they hit
> incorrect documentation they start to get the impression the software
> is "buggy" suddenly they start questioning everything and bringing
> every problem to the hive administrator because even though they wrote
> a query wrong their first instinct is to "blame hive".
>
>
>
>
> I find editing xdocs EASIER then working with wiki. Wiki is great and
> all but in my travels I have to work on 5 different wiki's they all
> are slightly different in what they support and their mark up. We
> should be able to commit xdoc patches without full unit tests. Keeping
> the xdoc up to date should not be an issue because we should simply
> not accept a patch that changes/adds functionality without some xdoc.
>
> Another issue right now is there are features that are NOT documented
> anywhere. When a user asks about those features I have to send them to
> Jira tickets, often times the ticket will have a long back and forth
> where the feature is debated, or sometimes just a patch, you never see
> the full syntax, it can be very confusing,I often end up telling them
> to dig through a .q file inside a patch to figure out what this
> feature is and how to use it. While most people are good about
> updating the wiki we know that things tend to fall though the cracks.
>
> I think there is still a place for wiki, free form, multi-person
> planning, etc but I do not think a mature software product can every
> have authoritative documentation in a wiki.
>
>

John,

I was not trying to pick on your because of the view example, I only
mentioned views because it was a 6.0 feature off the top of my head. I
was just making a general point. You bring up another point.

Take something like this:

CREATE [EXTERNAL] TABLE [IF NOT EXISTS] table_name
  [(col_name data_type [COMMENT col_comment], ...)]
  [COMMENT table_comment]
  [PARTITIONED BY (col_name data_type [COMMENT col_comment], ...)]
  [CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name
[ASC|DESC], ...)] INTO num_buckets BUCKETS]
  [
   [ROW FORMAT row_format] [STORED AS file_format]
   | STORED BY 'storage.handler.class.name' [ WITH SERDEPROPERTIES
(...) ]  (Note:  only available starting with 0.6.0)
  ]
  [STORED AS file_format]
  [LOCATION hdfs_path]
  [TBLPROPERTIES (property_name=property_value, ...)]  (Note:  only
available starting with 0.6.0)
  [AS select_statement]  (Note: this feature is only available
starting with 0.5.0.)

CREATE [EXTERNAL] TABLE [IF NOT EXISTS] table_name
  LIKE existing_table_name
  [LOCATION hdfs_path]

I can imagine in two years when we have possibly four releases and
several version dependant features. The language manual is going to
have multiple Caveats, that will be less clear.

Also this is a fixable and anecdotal problem, but the wiki is slow. It
feels like it has been that way for months now.

[edward@ec ~]$ wget http://wiki.apache.org/hadoop/Hive/LanguageManual/Explain
--2010-07-15 16:27:36--
http://wiki.apache.org/hadoop/Hive/LanguageManual/Explain
    [    <=>
                                     ] 21,497      11.1K/s   in 1.9s

2010-07-15 16:27:41 (11.1 KB/s) - “Explain” saved [21497]


Edward

Mime
View raw message