hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vasu Mariyala (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10541) Make the file system properties customizable per table/column family
Date Thu, 20 Feb 2014 01:05:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906405#comment-13906405

Vasu Mariyala commented on HBASE-10541:

Thanks [~stack] for the review.

The patch is intended to work for the file systems which honor the properties passed to it
using the method

  public FSDataOutputStream create(Path file, FsPermission permission,
      boolean overwrite, int bufferSize,
      short replication, long blockSize, Progressable progress)

Some of the file system implementations like FTPFileSystem, S3FileSystem don't honor the parameters
that don't make sense to them. In this example, they don't honor the replication. WebHdfsFileSystem
honors all these parameters.

Based on going through the code, the existing BLOCKSIZE of the HColumnDescriptor indicates
the size of the Blocks in HFile (Data block, meta block) while the FS_BLOCKSIZE is the block
size used by the file system while storing this HFile. Please correct me.

Added more documentation to the rev3.patch

> Make the file system properties customizable per table/column family
> --------------------------------------------------------------------
>                 Key: HBASE-10541
>                 URL: https://issues.apache.org/jira/browse/HBASE-10541
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Vasu Mariyala
>            Assignee: Vasu Mariyala
>         Attachments: trunk-HBASE-10541-rev1.patch, trunk-HBASE-10541-rev2.patch, trunk-HBASE-10541-rev3.patch,
> The file system properties like replication (the number of nodes to which the hfile needs
to be replicated), block size need to be customizable per table/column family. This is important
especially in the testing scenarios or for test tables where we don't want the hfile to be
replicated 3 times.

This message was sent by Atlassian JIRA

View raw message