carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From manishgupta88 <...@git.apache.org>
Subject [GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...
Date Mon, 28 Nov 2016 10:27:44 GMT
GitHub user manishgupta88 opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/362

    [CARBONDATA-459] Block distribution is wrong in case of dynamic allocation=true

    Problem: Block distribution is wrong in case of dynamic allocation=true
    
    Analysis: In case when dynamic allocation is true and configured max executors are more
than the initial executors then carbon is not able to request the max number of executors
configured. Due to this resources are getting under utilized and case when number of blocks
increases, the distribution of blocks is limited to the number of nodes and the number of
tasks launched are less. This leads to under utilization of resources and hence impacts the
query and load performance.
    
    Fix: Request for starting the maximum number of configured executors in case dynamic allocation
is true.
    
    Impact area: Query and data load flow performance due to under utilization of resources.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/manishgupta88/incubator-carbondata dynamic_allocation_block_distribution

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/362.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #362
    
----
commit 0019dfc08ef589d75c45feecd8b559bca0b35f08
Author: manishgupta88 <tomanishgupta18@gmail.com>
Date:   2016-11-28T10:07:11Z

    Problem: Block distribution is wrong in case of dynamic allocation=true
    
    Analysis: In case when dynamic allocation is true and configured max executors are more
than the initial executors then carbon is not able to request the max number of executors
configured. Due to this resources are getting under utilized and case when number of blocks
increases, the distribution of blocks is limited to the number of nodes and the number of
tasks launched are less. This leads to under utilization of resources and hence impacts the
query and load performance.
    
    Fix: Request for starting the maximum number of configured executors in case dynamic allocation
is true.
    
    Impact area: Query and data load flow performance due to under utilization of resources.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message