drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3846) Metadata Caching : A count(*) query took more time with the cache in place
Date Mon, 28 Sep 2015 21:08:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14933994#comment-14933994
] 

Rahul Challapalli commented on DRILL-3846:
------------------------------------------

I cannot attach the data set as it is larger than the allowed size limit. Reach out to me
if you need more information.

> Metadata Caching : A count(*) query took more time with the cache in place
> --------------------------------------------------------------------------
>
>                 Key: DRILL-3846
>                 URL: https://issues.apache.org/jira/browse/DRILL-3846
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Metadata
>            Reporter: Rahul Challapalli
>             Fix For: 1.2.0
>
>
> git.commit.id.abbrev=3c89b30
> I have a folder with 10k complex files. The generated cache file is around 486 MB. The
below numbers indicate that we regressed in terms of performance when we generated the metadata
cache
> {code}
> 0: jdbc:drill:zk=10.10.100.190:5181> select count(*) from `complex_sparse_50000files`;
> +----------+
> |  EXPR$0  |
> +----------+
> | 1000000  |
> +----------+
> 1 row selected (30.835 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> refresh table metadata `complex_sparse_50000files`;
> +-------+---------------------------------------------------------------------+
> |  ok   |                               summary                               |
> +-------+---------------------------------------------------------------------+
> | true  | Successfully updated metadata for table complex_sparse_50000files.  |
> +-------+---------------------------------------------------------------------+
> 1 row selected (10.69 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> select count(*) from `complex_sparse_50000files`;
> +----------+
> |  EXPR$0  |
> +----------+
> | 1000000  |
> +----------+
> 1 row selected (47.614 seconds)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message