hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15069) Unify HFile Writer and Reader creation patterns
Date Wed, 06 Jan 2016 23:24:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086482#comment-15086482

Jonathan Hsieh commented on HBASE-15069:

FSUtils.getCurrentFileSystem(conf) is a static so it only really helps us if we are debugging
by modifying code and recompile testing.  That method also eventually [1] uses getFileSystem
from Path which calls FileSystem.get(...)[2]
You can mock or do dependency injection of an FS at the ReaderBuilder level  using ReaderBuilder.withFileSystem(FileSystem).

I'll look into replacing the FileSystem.get() in the constructors with FSUtils.getCFS().

Looks like I missed a few removable lines of code in some of the tests; i will clean that
up too.

[1] https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java#L1526
[2] https://github.com/apache/hadoop-common/blob/branch-2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java#L294

> Unify HFile Writer and Reader creation patterns
> -----------------------------------------------
>                 Key: HBASE-15069
>                 URL: https://issues.apache.org/jira/browse/HBASE-15069
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile
>    Affects Versions: 2.0.0
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>             Fix For: 2.0.0
>         Attachments: hbase-15069.patch, hbase-15069.v2.patch, hbase-15069.v3.patch
> There are a plethora of different static methods sprinkled through out HStoreFile and
HFile, and many tests that have extraneous calls to 'new CacheConfig(conf)' or essentially
extraneous FileSystem arguments threaded through out the code.
> This patch forces all creation to go through HFile Reader and Writer Builders, eliminates
all static Builder constructors, and limits the exposure Reader/Writers .  It also forces
all HFile writer uses outside of the o.a.h.h.io.hfile package to use the StoreFile writers

This message was sent by Atlassian JIRA

View raw message