drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-1055) Implement ProducerConsumer operator
Date Sun, 22 Jun 2014 00:49:24 GMT
Steven Phillips created DRILL-1055:

             Summary: Implement ProducerConsumer operator
                 Key: DRILL-1055
                 URL: https://issues.apache.org/jira/browse/DRILL-1055
             Project: Apache Drill
          Issue Type: Bug
            Reporter: Steven Phillips

Currently, a minor fragment is run in a single thread, and parallelization is achieved by
running multiple minor fragments. There are cases where we are limited in how much we can
parallelize the leaf fragments due to the limited number of files/row groups. This can result
in under-utilization of resources.

This operator will allow the scan operator to run in parallel with the downstream operators
in the leaf fragments, thus increasing cluster utilization, and reducing the duration of the
leaf fragments in certain cases.

This message was sent by Atlassian JIRA

View raw message