drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4035) NPE seen on Functional test run using JDK 8
Date Tue, 01 Dec 2015 19:06:10 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034372#comment-15034372
] 

Khurram Faraaz commented on DRILL-4035:
---------------------------------------

With JDK8 (openjdk version "1.8.0_65") and assertions enabled and latest Drill 1.4 RPM (git.commit.id.abbrev=ff76078)
several Hive Functional tests fail due to the same NPE. 
In one of the comments above a possible Fix was suggested.

{noformat}
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_65]
        at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_65]
        at org.apache.drill.exec.store.hive.HiveRecordReader.setup(HiveRecordReader.java:242)
~[drill-storage-hive-core-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        ... 27 common frames omitted
Caused by: java.lang.NullPointerException: null
        at hive.parquet.format.converter.ParquetMetadataConverter.fromParquetStatistics(ParquetMetadataConverter.java:247)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at hive.parquet.format.converter.ParquetMetadataConverter.fromParquetMetadata(ParquetMetadataConverter.java:368)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at hive.parquet.format.converter.ParquetMetadataConverter.readParquetMetadata(ParquetMetadataConverter.java:346)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at hive.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:296) ~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at hive.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:254) ~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.getSplit(ParquetRecordReaderWrapper.java:200)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:79)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:66)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:72)
~[drill-hive-exec-shaded-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.drill.exec.store.hive.HiveRecordReader.init(HiveRecordReader.java:219)
~[drill-storage-hive-core-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.drill.exec.store.hive.HiveRecordReader.access$000(HiveRecordReader.java:64)
~[drill-storage-hive-core-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.drill.exec.store.hive.HiveRecordReader$1.call(HiveRecordReader.java:235)
~[drill-storage-hive-core-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.drill.exec.store.hive.HiveRecordReader$1.call(HiveRecordReader.java:232)
~[drill-storage-hive-core-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at org.apache.drill.exec.ops.OperatorContextImpl$1$1.run(OperatorContextImpl.java:158)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_65]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_65]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
~[hadoop-common-2.7.0-mapr-1506.jar:na]
        at org.apache.drill.exec.ops.OperatorContextImpl$1.call(OperatorContextImpl.java:155)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_65]
{noformat}

> NPE seen on Functional test run using JDK 8
> -------------------------------------------
>
>                 Key: DRILL-4035
>                 URL: https://issues.apache.org/jira/browse/DRILL-4035
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 1.3.0
>         Environment: 4 node cluster CentOS
>            Reporter: Khurram Faraaz
>            Assignee: Deneche A. Hakim
>             Fix For: 1.4.0
>
>
> I am seeing an NPE in the Functional test run using JDK8 and Drill 1.3
> Failing test is : Functional/partition_pruning/hive/parquet/dynamic_hier_intint/data/parquetCount1.q
> select count(*) from hive.dynamic_partitions.lineitem_parquet_partitioned_hive_hier_intint;
> {code}
> Drill version was, git.commit.id=e4b94a78
> root@centos drill-1.3.0]# java -version
> openjdk version "1.8.0_65"
> OpenJDK Runtime Environment (build 1.8.0_65-b17)
> OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)
> [root@centos drill-1.3.0]# javac -version
> javac 1.8.0_65
> {code}
> {code}
> 2015-11-05 01:37:45 INFO  DrillTestJdbc:76 - running test /root/public_framework/drill-test-framework/framework/resources/Functional/window_functions/last_val/lastValFn_9.q
981260622
> 2015-11-05 01:37:45 INFO  DrillResultSetImpl$ResultsListener:1470 - [#137] Query failed:
> oadd.org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: cefc7238-a646-4f9a-b4f2-0bd102efe393 on centos-01.qa.lab:31010]
>         at oadd.org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>         at oadd.org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:110)
>         at oadd.org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>         at oadd.org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>         at oadd.org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:61)
>         at oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:233)
>         at oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:205)
>         at oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>         at oadd.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>         at oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>         at oadd.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>         at oadd.io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>         at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>         at oadd.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
>         at oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
>         at oadd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
>         at oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
>         at oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
>         at oadd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
>         at oadd.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
>         at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message