hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12596) bulkload needs to follow locality
Date Fri, 03 Jul 2015 03:40:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612803#comment-14612803
] 

Ted Yu commented on HBASE-12596:
--------------------------------

{code}
212	            if (null == tableName || tableName.isEmpty()) {
213	              LOG.warn("table name is null, so use default writer");
{code}
When would the above happen ?
Table name is set in configureIncrementalLoad(), right ?
{code}
218	                Connection connection = ConnectionFactory.createConnection(conf);
219	                RegionLocator locator = connection.getRegionLocator(TableName.valueOf(tableName));
{code}
You can use try-with-resources.
{code}
231	              if (null == loc) {
232	                LOG.warn("failed to get region location, so use default writer");
{code}
Should the log level be at TRACE ?


> bulkload needs to follow locality
> ---------------------------------
>
>                 Key: HBASE-12596
>                 URL: https://issues.apache.org/jira/browse/HBASE-12596
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile, regionserver
>    Affects Versions: 0.98.8
>         Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 0.98.14
>
>         Attachments: HBASE-12596-0.98-v1.patch, HBASE-12596-master-v1.patch, HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded;
2. Move these HFiles to the right hdfs directory. However, the locality could be loss during
the first step. Why not just write the HFiles directly into the right place? We can do this
easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need
to call it in HFileOutputFormat's getNewWriter().
> This feature is disabled by default, and we could use 'hbase.bulkload.locality.sensitive.enabled=true'
to enable it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message