drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1652) flatten causing the sqlline to hang indefinitely when dealing with 0.1 GB data
Date Wed, 12 Nov 2014 19:38:33 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208521#comment-14208521
] 

Rahul Challapalli commented on DRILL-1652:
------------------------------------------

The below flatten query looks good
{code}
 select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`;
{code}

However if we apply flatten on 'events' the query succeeds a few times but fails most of the
times with the below error
Query:
{code}
select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`;
{code}

Error:
{code}
ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in pipeline.  Closing channel between
local /10.10.100.191:31010 and remote /10.10.100.190:45973
java.io.IOException: Connection timed out
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_51]
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.7.0_51]
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.7.0_51]
        at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.7.0_51]
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) ~[na:1.7.0_51]
        at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.buffer.WrappedByteBuf.writeBytes(WrappedByteBuf.java:641) ~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.buffer.UnsafeDirectLittleEndian.writeBytes(UnsafeDirectLittleEndian.java:25)
~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:4.0.24.Final]
        at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:225)
~[netty-transport-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
~[netty-transport-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-transport-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[netty-transport-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-transport-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-transport-4.0.24.Final.jar:4.0.24.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
[netty-common-4.0.24.Final.jar:4.0.24.Final]
        at java.lang.Thread.run(Thread.java:744)
{code}

> flatten causing the sqlline to hang indefinitely when dealing with 0.1 GB data
> ------------------------------------------------------------------------------
>
>                 Key: DRILL-1652
>                 URL: https://issues.apache.org/jira/browse/DRILL-1652
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Functions - Drill, Storage - JSON
>            Reporter: Rahul Challapalli
>         Attachments: error.log, select_star_error.log
>
>
> git.commit.id.abbrev=60aa446
> I ran the below test against the private branch of Jason which has some patches for bugs
related to flatten which are not yet merged into the master.
> I used the below json record and copied it over 100000 times
> {code}
> {
>     "uid" : 1,
>     "events" : [
>         { "evnt_id":"e1", "campaign_id":"c1", "event_name":"e1_name", "event_time":1000000},
>         { "evnt_id":"e2", "campaign_id":"c1", "event_name":"e2_name", "event_time":2000000},
>         { "evnt_id":"e3", "campaign_id":"c1", "event_name":"e3_name", "event_time":3000000},
>         { "evnt_id":"e4", "campaign_id":"c1", "event_name":"e4_name", "event_time":4000000},
>         { "evnt_id":"e5", "campaign_id":"c2", "event_name":"e5_name", "event_time":5000000},
>         { "evnt_id":"e6", "campaign_id":"c1", "event_name":"e6_name", "event_time":6000000},
>         { "evnt_id":"e7", "campaign_id":"c1", "event_name":"e7_name", "event_time":7000000},
>         { "evnt_id":"e8", "campaign_id":"c2", "event_name":"e8_name", "event_time":8000000},
>         { "evnt_id":"e9", "campaign_id":"c2", "event_name":"e9_name", "event_time":9000000}
>     ],
>    "transactions" : [
>         { "trans_id":"t1", "amount":100, "trans_time":7777777},
>         { "trans_id":"t2", "amount":100, "trans_time":8888888}
>     ]
> }
> {code}
> The below query which applies a flatten function hangs after returning a few thousand
records
> {code}
>  select uid, flatten(transactions) from `json_kvgenflatten/transactions.json`;
> {code}
> Select * from the file also fails after displaying a bunch of records
> {code}
> select * from `json_kvgenflatten/transactions.json`;
> {code}
> The below query works
> {code}
> select uid from `json_kvgenflatten/transactions.json`;
> {code}
> I attached the error logs for the 2 failing queries



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message