Return-Path: X-Original-To: apmail-drill-issues-archive@minotaur.apache.org Delivered-To: apmail-drill-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C55B6C8CF for ; Wed, 12 Nov 2014 19:52:58 +0000 (UTC) Received: (qmail 61740 invoked by uid 500); 12 Nov 2014 19:52:58 -0000 Delivered-To: apmail-drill-issues-archive@drill.apache.org Received: (qmail 61632 invoked by uid 500); 12 Nov 2014 19:52:58 -0000 Mailing-List: contact issues-help@drill.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.incubator.apache.org Delivered-To: mailing list issues@drill.incubator.apache.org Received: (qmail 61571 invoked by uid 99); 12 Nov 2014 19:52:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Nov 2014 19:52:58 +0000 X-ASF-Spam-Status: No, hits=-2000.6 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 12 Nov 2014 19:52:56 +0000 Received: (qmail 58545 invoked by uid 99); 12 Nov 2014 19:52:36 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Nov 2014 19:52:36 +0000 Date: Wed, 12 Nov 2014 19:52:36 +0000 (UTC) From: "Rahul Challapalli (JIRA)" To: issues@drill.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (DRILL-1652) flatten causing the sqlline to hang indefinitely when dealing with 0.1 GB data MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/DRILL-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208521#comment-14208521 ] Rahul Challapalli edited comment on DRILL-1652 at 11/12/14 7:52 PM: -------------------------------------------------------------------- The below flatten query looks good {code} select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`; {code} However if we apply flatten on 'events' the query succeeds a few times but fails most of the times with the below error Query: {code} select uid, flatten(events) from `json_kvgenflatten/transactions.json`; {code} Error: {code} ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in pipeline. Closing channel between local /10.10.100.191:31010 and remote /10.10.100.190:45973 java.io.IOException: Connection timed out at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_51] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.7.0_51] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.7.0_51] at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.7.0_51] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) ~[na:1.7.0_51] at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final] at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final] at io.netty.buffer.WrappedByteBuf.writeBytes(WrappedByteBuf.java:641) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final] at io.netty.buffer.UnsafeDirectLittleEndian.writeBytes(UnsafeDirectLittleEndian.java:25) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:4.0.24.Final] at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:225) ~[netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) ~[netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) [netty-common-4.0.24.Final.jar:4.0.24.Final] at java.lang.Thread.run(Thread.java:744) {code} was (Author: rkins): The below flatten query looks good {code} select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`; {code} However if we apply flatten on 'events' the query succeeds a few times but fails most of the times with the below error Query: {code} select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`; {code} Error: {code} ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in pipeline. Closing channel between local /10.10.100.191:31010 and remote /10.10.100.190:45973 java.io.IOException: Connection timed out at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_51] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.7.0_51] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.7.0_51] at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.7.0_51] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) ~[na:1.7.0_51] at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final] at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final] at io.netty.buffer.WrappedByteBuf.writeBytes(WrappedByteBuf.java:641) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final] at io.netty.buffer.UnsafeDirectLittleEndian.writeBytes(UnsafeDirectLittleEndian.java:25) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:4.0.24.Final] at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:225) ~[netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) ~[netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-transport-4.0.24.Final.jar:4.0.24.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) [netty-common-4.0.24.Final.jar:4.0.24.Final] at java.lang.Thread.run(Thread.java:744) {code} > flatten causing the sqlline to hang indefinitely when dealing with 0.1 GB data > ------------------------------------------------------------------------------ > > Key: DRILL-1652 > URL: https://issues.apache.org/jira/browse/DRILL-1652 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill, Storage - JSON > Reporter: Rahul Challapalli > Attachments: error.log, select_star_error.log > > > git.commit.id.abbrev=60aa446 > I ran the below test against the private branch of Jason which has some patches for bugs related to flatten which are not yet merged into the master. > I used the below json record and copied it over 100000 times > {code} > { > "uid" : 1, > "events" : [ > { "evnt_id":"e1", "campaign_id":"c1", "event_name":"e1_name", "event_time":1000000}, > { "evnt_id":"e2", "campaign_id":"c1", "event_name":"e2_name", "event_time":2000000}, > { "evnt_id":"e3", "campaign_id":"c1", "event_name":"e3_name", "event_time":3000000}, > { "evnt_id":"e4", "campaign_id":"c1", "event_name":"e4_name", "event_time":4000000}, > { "evnt_id":"e5", "campaign_id":"c2", "event_name":"e5_name", "event_time":5000000}, > { "evnt_id":"e6", "campaign_id":"c1", "event_name":"e6_name", "event_time":6000000}, > { "evnt_id":"e7", "campaign_id":"c1", "event_name":"e7_name", "event_time":7000000}, > { "evnt_id":"e8", "campaign_id":"c2", "event_name":"e8_name", "event_time":8000000}, > { "evnt_id":"e9", "campaign_id":"c2", "event_name":"e9_name", "event_time":9000000} > ], > "transactions" : [ > { "trans_id":"t1", "amount":100, "trans_time":7777777}, > { "trans_id":"t2", "amount":100, "trans_time":8888888} > ] > } > {code} > The below query which applies a flatten function hangs after returning a few thousand records > {code} > select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`; > {code} > Select * from the file also fails after displaying a bunch of records > {code} > select * from `json_kvgenflatten/transactions.json`; > {code} > The below query works > {code} > select uid from `json_kvgenflatten/transactions.json`; > {code} > I attached the error logs for the 2 failing queries -- This message was sent by Atlassian JIRA (v6.3.4#6332)