hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vaibhav Gumashta (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-7224) Set incremental printing to true by default in Beeline
Date Tue, 05 Jul 2016 17:31:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362826#comment-15362826
] 

Vaibhav Gumashta edited comment on HIVE-7224 at 7/5/16 5:30 PM:
----------------------------------------------------------------

[~stakiar] There seems to be an issue with the column width estimation when incremental printing
is enabled by default. An example (output of describe table):
With incremental true:
{code}
| search_engine   | string          |                 |
| exclude_hit     | string          |                 |
| hier1           | string          |                 |
| hier2           | string          |                 |
| hier3           | string          |                 |
| hier4           | string          |                 |
| hier5           | string          |                 |
| browser         | string          |                 |
| post_browser_height | string          |                 |
| post_browser_width | string          |                 |
| post_cookies    | string          |                 |
| post_java_enabled | string          |                 |
| post_persistent_cookie | string          |                 |
| color           | string          |                 |
| connection_type | string          |                 |
| country         | string          |                 |
| domain          | string          |                 |
| post_t_time_info | string          |                 |
| javascript      | string          |                 |
| language        | string          |                 |
| os              | string          |                 |
| plugins         | string          |                 |
| resolution      | string          |                 |
| last_hit_time_gmt | string          |                 |
| first_hit_time_gmt | string          |                 |
| visit_start_time_gmt | string          |                 |
| last_purchase_time_gmt | string          |                 |
+-----------------+-----------------+-----------------+--+
|    col_name     |    data_type    |     comment     |
+-----------------+-----------------+-----------------+--+
| last_purchase_num | string          |                 |
| first_hit_page_url | string          |                 |
| first_hit_pagename | string          |                 |
| visit_start_page_url | string          |                 |
| visit_start_pagename | string          |                 |
| first_hit_referrer | string          |                 |
| visit_referrer  | string          |                 |
| visit_search_engine | string          |                 |
| visit_num       | string          |                 |
| visit_page_num  | string          |                 |
| prev_page       | string          |                 |
| geo_city        | string          |                 |
| geo_country     | string          |                 |
| geo_region      | string          |                 |
| duplicate_purchase | string          |                 |
{code}

With incremental false:
{code}
| search_engine            | string                |                       |
| exclude_hit              | string                |                       |
| hier1                    | string                |                       |
| hier2                    | string                |                       |
| hier3                    | string                |                       |
| hier4                    | string                |                       |
| hier5                    | string                |                       |
| browser                  | string                |                       |
| post_browser_height      | string                |                       |
| post_browser_width       | string                |                       |
| post_cookies             | string                |                       |
| post_java_enabled        | string                |                       |
| post_persistent_cookie   | string                |                       |
| color                    | string                |                       |
| connection_type          | string                |                       |
| country                  | string                |                       |
| domain                   | string                |                       |
| post_t_time_info         | string                |                       |
| javascript               | string                |                       |
| language                 | string                |                       |
| os                       | string                |                       |
| plugins                  | string                |                       |
| resolution               | string                |                       |
| last_hit_time_gmt        | string                |                       |
| first_hit_time_gmt       | string                |                       |
| visit_start_time_gmt     | string                |                       |
| last_purchase_time_gmt   | string                |                       |
+--------------------------+-----------------------+-----------------------+--+
|         col_name         |       data_type       |        comment        |
+--------------------------+-----------------------+-----------------------+--+
| last_purchase_num        | string                |                       |
| first_hit_page_url       | string                |                       |
| first_hit_pagename       | string                |                       |
| visit_start_page_url     | string                |                       |
| visit_start_pagename     | string                |                       |
| first_hit_referrer       | string                |                       |
| visit_referrer           | string                |                       |
| visit_search_engine      | string                |                       |
| visit_num                | string                |                       |
| visit_page_num           | string                |                       |
| prev_page                | string                |                       |
| geo_city                 | string                |                       |
| geo_country              | string                |                       |
| geo_region               | string                |                       |
| duplicate_purchase       | string                |                       |
{code}

I think we'll need to fix this before we can get this in.


was (Author: vgumashta):
[~stakiar] There seems to be an issue with the column width estimation when incremental printing
is enabled by default. An example:
With incremental true:
{code}
| search_engine   | string          |                 |
| exclude_hit     | string          |                 |
| hier1           | string          |                 |
| hier2           | string          |                 |
| hier3           | string          |                 |
| hier4           | string          |                 |
| hier5           | string          |                 |
| browser         | string          |                 |
| post_browser_height | string          |                 |
| post_browser_width | string          |                 |
| post_cookies    | string          |                 |
| post_java_enabled | string          |                 |
| post_persistent_cookie | string          |                 |
| color           | string          |                 |
| connection_type | string          |                 |
| country         | string          |                 |
| domain          | string          |                 |
| post_t_time_info | string          |                 |
| javascript      | string          |                 |
| language        | string          |                 |
| os              | string          |                 |
| plugins         | string          |                 |
| resolution      | string          |                 |
| last_hit_time_gmt | string          |                 |
| first_hit_time_gmt | string          |                 |
| visit_start_time_gmt | string          |                 |
| last_purchase_time_gmt | string          |                 |
+-----------------+-----------------+-----------------+--+
|    col_name     |    data_type    |     comment     |
+-----------------+-----------------+-----------------+--+
| last_purchase_num | string          |                 |
| first_hit_page_url | string          |                 |
| first_hit_pagename | string          |                 |
| visit_start_page_url | string          |                 |
| visit_start_pagename | string          |                 |
| first_hit_referrer | string          |                 |
| visit_referrer  | string          |                 |
| visit_search_engine | string          |                 |
| visit_num       | string          |                 |
| visit_page_num  | string          |                 |
| prev_page       | string          |                 |
| geo_city        | string          |                 |
| geo_country     | string          |                 |
| geo_region      | string          |                 |
| duplicate_purchase | string          |                 |
{code}

With incremental false:
{code}
| search_engine            | string                |                       |
| exclude_hit              | string                |                       |
| hier1                    | string                |                       |
| hier2                    | string                |                       |
| hier3                    | string                |                       |
| hier4                    | string                |                       |
| hier5                    | string                |                       |
| browser                  | string                |                       |
| post_browser_height      | string                |                       |
| post_browser_width       | string                |                       |
| post_cookies             | string                |                       |
| post_java_enabled        | string                |                       |
| post_persistent_cookie   | string                |                       |
| color                    | string                |                       |
| connection_type          | string                |                       |
| country                  | string                |                       |
| domain                   | string                |                       |
| post_t_time_info         | string                |                       |
| javascript               | string                |                       |
| language                 | string                |                       |
| os                       | string                |                       |
| plugins                  | string                |                       |
| resolution               | string                |                       |
| last_hit_time_gmt        | string                |                       |
| first_hit_time_gmt       | string                |                       |
| visit_start_time_gmt     | string                |                       |
| last_purchase_time_gmt   | string                |                       |
+--------------------------+-----------------------+-----------------------+--+
|         col_name         |       data_type       |        comment        |
+--------------------------+-----------------------+-----------------------+--+
| last_purchase_num        | string                |                       |
| first_hit_page_url       | string                |                       |
| first_hit_pagename       | string                |                       |
| visit_start_page_url     | string                |                       |
| visit_start_pagename     | string                |                       |
| first_hit_referrer       | string                |                       |
| visit_referrer           | string                |                       |
| visit_search_engine      | string                |                       |
| visit_num                | string                |                       |
| visit_page_num           | string                |                       |
| prev_page                | string                |                       |
| geo_city                 | string                |                       |
| geo_country              | string                |                       |
| geo_region               | string                |                       |
| duplicate_purchase       | string                |                       |
{code}

I think we'll need to fix this before we can get this in.

> Set incremental printing to true by default in Beeline
> ------------------------------------------------------
>
>                 Key: HIVE-7224
>                 URL: https://issues.apache.org/jira/browse/HIVE-7224
>             Project: Hive
>          Issue Type: Bug
>          Components: Clients, JDBC
>    Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0
>            Reporter: Vaibhav Gumashta
>            Assignee: Sahil Takiar
>         Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, HIVE-7224.3.patch
>
>
> See HIVE-7221.
> By default beeline tries to buffer the entire output relation before printing it on stdout.
This can cause OOM when the output relation is large. However, beeline has the option of incremental
prints. We should keep that as the default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message