Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Wed, 8 Apr 2015 20:53:13 +0000 (UTC)
From: "Sudheesh Katkam (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12787760.1427998418000.31527.1428526393506@Atlassian.JIRA>
In-Reply-To: <JIRA.12787760.1427998418000@Atlassian.JIRA>
References: <JIRA.12787760.1427998418000@Atlassian.JIRA>
 <JIRA.12787760.1427998418038@arcas>
Subject: [jira] [Updated] (DRILL-2666) Query against json profile with large
 number of fragments fails
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/DRILL-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sudheesh Katkam updated DRILL-2666:
-----------------------------------
    Attachment: small_cancelled_query2.json

This does not anything to do with the query profile containing large number of fragments.

This query on a smaller data set fails
{code}
SELECT FLATTEN(t0.fragmentprofile) col FROM dfs.`small_cancelled_query2.json` t0;
{code}
The "doubleValue" value in one of the operator profiles' metrics is an integer, which is incorrect. This is because the cancelled_query2.json profile is from "Full JSON Profile" in the profile page. This profile is subject to JavaScript [weirdness | http://stackoverflow.com/questions/5520399/how-to-prevent-removing-decimal-point-when-parsing-json] that removes the decimal point.

So the "real" query profile that should be used is in the tmp directory mentioned in the corresponding storage plugin.

In any case, drill should not have failed with  _You tried to write a Float8 type when you are using a ValueWriter of type NullableBigIntWriterImpl_. But there is already a [JIRA | https://issues.apache.org/jira/browse/DRILL-1460] for this.


> Query against json profile with large number of fragments fails
> ---------------------------------------------------------------
>
>                 Key: DRILL-2666
>                 URL: https://issues.apache.org/jira/browse/DRILL-2666
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 0.8.0
>            Reporter: Krystal
>            Assignee: Sudheesh Katkam
>             Fix For: 0.9.0
>
>         Attachments: cancelled_query2.json.gz, drill2666.log, small_cancelled_query2.json
>
>
> git.commit.id=417ad38f7239c6dc1ff0421cbd5a2358d1437c87
> I have a json profile that contains about 300 fragments.  When I try to run queries against it, the query would fail with "You tried to do a batch data read operation when you were in a state of STOP" error.  Below is an example:
> {code}
> 0: jdbc:drill:schema=dfs.hbase> select t3.majorId from (select t2.majorFragmentId majorId,t2.minorFragmentProfile.minorFragmentId minorId,flatten(t2.minorFragmentProfile.operatorProfile) oProfile from (select t1.col.majorFragmentId majorFragmentId,flatten(t1.col.minorFragmentProfile) minorFragmentProfile from (select flatten(t.fragmentprofile) col from `dfs.drillTestDir`.`/ui_profiles/cancelled_query2.json`t) t1) t2) t3 group by t3.majorId,t3.oProfile.operatorId,t3.oProfile.operatorType order by t3.majorId,t3.oProfile.operatorId;
> Query failed: RemoteRpcException: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP.  You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ 4de479a5-ce93-4a3f-89b4-3aa55481869c on qa-node114.qa.lab:31010 ]
> [ 4de479a5-ce93-4a3f-89b4-3aa55481869c on qa-node114.qa.lab:31010 ]
> {code}
> For json profile files that contain smaller number of fragments, the queries run fine.  


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)