drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudheesh Katkam (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-2666) Query against json profile with large number of fragments fails
Date Wed, 08 Apr 2015 20:53:13 GMT

     [ https://issues.apache.org/jira/browse/DRILL-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sudheesh Katkam updated DRILL-2666:
-----------------------------------
    Attachment: small_cancelled_query2.json

This does not anything to do with the query profile containing large number of fragments.

This query on a smaller data set fails
{code}
SELECT FLATTEN(t0.fragmentprofile) col FROM dfs.`small_cancelled_query2.json` t0;
{code}
The "doubleValue" value in one of the operator profiles' metrics is an integer, which is incorrect.
This is because the cancelled_query2.json profile is from "Full JSON Profile" in the profile
page. This profile is subject to JavaScript [weirdness | http://stackoverflow.com/questions/5520399/how-to-prevent-removing-decimal-point-when-parsing-json]
that removes the decimal point.

So the "real" query profile that should be used is in the tmp directory mentioned in the corresponding
storage plugin.

In any case, drill should not have failed with  _You tried to write a Float8 type when you
are using a ValueWriter of type NullableBigIntWriterImpl_. But there is already a [JIRA |
https://issues.apache.org/jira/browse/DRILL-1460] for this.


> Query against json profile with large number of fragments fails
> ---------------------------------------------------------------
>
>                 Key: DRILL-2666
>                 URL: https://issues.apache.org/jira/browse/DRILL-2666
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 0.8.0
>            Reporter: Krystal
>            Assignee: Sudheesh Katkam
>             Fix For: 0.9.0
>
>         Attachments: cancelled_query2.json.gz, drill2666.log, small_cancelled_query2.json
>
>
> git.commit.id=417ad38f7239c6dc1ff0421cbd5a2358d1437c87
> I have a json profile that contains about 300 fragments.  When I try to run queries against
it, the query would fail with "You tried to do a batch data read operation when you were in
a state of STOP" error.  Below is an example:
> {code}
> 0: jdbc:drill:schema=dfs.hbase> select t3.majorId from (select t2.majorFragmentId
majorId,t2.minorFragmentProfile.minorFragmentId minorId,flatten(t2.minorFragmentProfile.operatorProfile)
oProfile from (select t1.col.majorFragmentId majorFragmentId,flatten(t1.col.minorFragmentProfile)
minorFragmentProfile from (select flatten(t.fragmentprofile) col from `dfs.drillTestDir`.`/ui_profiles/cancelled_query2.json`t)
t1) t2) t3 group by t3.majorId,t3.oProfile.operatorId,t3.oProfile.operatorType order by t3.majorId,t3.oProfile.operatorId;
> Query failed: RemoteRpcException: Failure while running fragment., You tried to do a
batch data read operation when you were in a state of STOP.  You can only do this type of
operation when you are in a state of OK or OK_NEW_SCHEMA. [ 4de479a5-ce93-4a3f-89b4-3aa55481869c
on qa-node114.qa.lab:31010 ]
> [ 4de479a5-ce93-4a3f-89b4-3aa55481869c on qa-node114.qa.lab:31010 ]
> {code}
> For json profile files that contain smaller number of fragments, the queries run fine.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message