hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15437) avro tables join fails when - tbl join tbl_postfix
Date Wed, 04 Jan 2017 09:30:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797759#comment-15797759
] 

ASF GitHub Bot commented on HIVE-15437:
---------------------------------------

GitHub user anishek opened a pull request:

    https://github.com/apache/hive/pull/127

    HIVE-15437 Progress Bar on Beeline client

    Have a common strategy to rendering the in place updates from both the hive cli and beeline.
Various summary updates once the tez job is completed are no longer rendered with fancy colors.
There is a possible condition where the logRunnable thread requests progress update from the
server before the session state is updated with relevant object (TezJobMonitor in this case)
to provide information. In this case no progress bar will be displayed. 


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/anishek/hive master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/hive/pull/127.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #127
    
----
commit f432b068dd843d5093da2ba138382e3fe27c6b72
Author: anishek <anishek@gmail.com>
Date:   2016-12-26T06:39:37Z

    HIVE-15437 Progress Bar on Beeline client
    
    Initial set of changes to add a remote api to get progress bar as a serialized object
    beeline client side changes to allow printing information

commit 9cb1a3a6b12be8f6b50aafcb13b6958e4a449f7b
Author: anishek <anishek@gmail.com>
Date:   2016-12-26T08:57:12Z

    HIVE-15437 Progress Bar on Beeline client
    
    DDL's are not trying to show progress bar and no call to server is made if operation handle
not available

commit 1b32f38afc78762921376af8662cc9b663e859e1
Author: anishek <anishek@gmail.com>
Date:   2016-12-28T08:31:23Z

    HIVE-15437 Progress Bar on Beeline client
    
    correcting  divide by zero

commit d33341a95b617af887b2a48578833fd4564ffdd3
Author: anishek <anishek@gmail.com>
Date:   2016-12-30T09:47:05Z

    HIVE-15437 Progress Bar on Beeline client
    
    adding ability to show the progress bar within tez job monitor

commit e50e71f5ac85c14f51d2b312c7f3a7b44ac57c7b
Author: anishek <anishek@gmail.com>
Date:   2017-01-03T09:20:48Z

    HIVE-15437 Progress Bar on Beeline client
    
    Ability to print correct vertex status, beeline not having any configuration for inplace
updates its on server side, where if server side is off then beeline is on, and stateful progress
bar print on hive cli

commit 30ada9d6857995f564a4d1852778894c049b4051
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T06:43:46Z

    HIVE-15437 Progress Bar on Beeline client
    
    Providing PrintStream to be used for rendering, with sleep at the beginning when updating
logs we are trying to make sure that the session sate is setup correctly by the execute call
so we can get the correct progress bar information from server.

commit bc39731a24d72675eb04d918d31339a58891a740
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T08:16:57Z

    HIVE-15437 Progress Bar on Beeline client
    
    only returning null from progressStatus if query execution is complete or failed  else
throw an exception

commit a8e212a565e17f38e479a68b958de81d2670d6fd
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T08:55:36Z

    HIVE-15437 Progress Bar on Beeline client
    
    state maintained on client side to render.  trying to render logs at the end when execution
is over.

commit f4c6dbdc50142b77a1776b7c021a100fe7ca8ab7
Author: anishek <anishek@gmail.com>
Date:   2016-12-26T06:39:37Z

    HIVE-15437 Progress Bar on Beeline client
    
    Initial set of changes to add a remote api to get progress bar as a serialized object
    beeline client side changes to allow printing information

commit e6e57a56e13da6ad0e72d742f5f46480486bb41f
Author: anishek <anishek@gmail.com>
Date:   2016-12-26T08:57:12Z

    HIVE-15437 Progress Bar on Beeline client
    
    DDL's are not trying to show progress bar and no call to server is made if operation handle
not available

commit 8538919c2c69432cf78135e147eda858ae12d152
Author: anishek <anishek@gmail.com>
Date:   2016-12-28T08:31:23Z

    HIVE-15437 Progress Bar on Beeline client
    
    correcting  divide by zero

commit 3850384f61feb28aa1de12e5f97c0f63adcd0656
Author: anishek <anishek@gmail.com>
Date:   2016-12-30T09:47:05Z

    HIVE-15437 Progress Bar on Beeline client
    
    adding ability to show the progress bar within tez job monitor

commit 774e6d911a3173e1e5729fb860cec8dab5883e2c
Author: anishek <anishek@gmail.com>
Date:   2017-01-03T09:20:48Z

    HIVE-15437 Progress Bar on Beeline client
    
    Ability to print correct vertex status, beeline not having any configuration for inplace
updates its on server side, where if server side is off then beeline is on, and stateful progress
bar print on hive cli

commit 87ba333c66b67436858ac148784466f2572d561a
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T06:43:46Z

    HIVE-15437 Progress Bar on Beeline client
    
    Providing PrintStream to be used for rendering, with sleep at the beginning when updating
logs we are trying to make sure that the session sate is setup correctly by the execute call
so we can get the correct progress bar information from server.

commit 91c1cdbc86e923070286ebb9ca605577cb59022f
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T08:16:57Z

    HIVE-15437 Progress Bar on Beeline client
    
    only returning null from progressStatus if query execution is complete or failed  else
throw an exception

commit d6af821c1fb87b3ac924e96c12b567394720acf3
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T08:55:36Z

    HIVE-15437 Progress Bar on Beeline client
    
    state maintained on client side to render.  trying to render logs at the end when execution
is over.

commit c5780e01406f7c88a1fcde59992f9e3f210f688c
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T09:16:02Z

    Merge branch 'master' of github.com:anishek/hive

commit 1ee8512be193b50a3e4f19632765393aae65c6f1
Author: anishek <anishek@gmail.com>
Date:   2017-01-04T09:25:55Z

    HIVE-15437 Progress Bar on Beeline client
    
    renaming method

----


> avro tables join fails when - tbl join tbl_postfix
> --------------------------------------------------
>
>                 Key: HIVE-15437
>                 URL: https://issues.apache.org/jira/browse/HIVE-15437
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Yongzhi Chen
>            Assignee: Yongzhi Chen
>             Fix For: 2.2.0
>
>         Attachments: HIVE-15437.1.patch
>
>
> The following queries return good results:
> select * from table1 where col1=key1; 
> select * from table1_1 where col1=key1; 
> When join them together, it gets following error:
> {noformat}
> Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long, expecting
union
>         at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> {noformat}
> The two avro tables both is defined by using avro schema, and the first table's name
is the second table name's prefix. 
> Note that this happens when single map is reading input from both tables. If map-join
is used with MR execution engine, or if Tez execution engine is used, this issue is not seen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message