hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yun Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-19226) Limit the reduce tasks number of incremental load
Date Sun, 12 Nov 2017 01:13:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yun Zhao updated HBASE-19226:
    Release Note: Add a configuration item `hbase.mapreduce.hfileoutputformat.reduces.ratio`,
it is used to adjust the reduce tasks number for the MapReduce job when incremental load into
a table´╝îwhich is the ratio of the regions number between 0 and 1.

> Limit the reduce tasks number of incremental load
> -------------------------------------------------
>                 Key: HBASE-19226
>                 URL: https://issues.apache.org/jira/browse/HBASE-19226
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Yun Zhao
>            Assignee: Yun Zhao
>            Priority: Minor
>         Attachments: HBASE-19226.master.001.patch, HBASE-19226.master.002.patch
> When using MapReduce job to perform an incremental load into a table,the number of reduce
tasks is the current number of regions. If there are too many regions, will lead to network+disk
I/O is too large, affecting the real-time request.
> Need to use a configuration to set a number or ratio?
> Limit running reduce tasks since [https://issues.apache.org/jira/browse/MAPREDUCE-5583],
the old version can only be set reduce number.

This message was sent by Atlassian JIRA

View raw message