hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9345) Erasure Coding: create dummy coder and schema
Date Fri, 24 Mar 2017 05:46:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939854#comment-15939854
] 

SammiChen commented on HDFS-9345:
---------------------------------

Some thoughts so far. The motivation of this JIRA is to help end user tune performance of
EC by skip encoding calculation and decoding calculation(?). So without EC calculation, the
left factors that impact the performance is data block number, parity block number and cell
size. So both dummy schema and dummy EC policy are required.  And for each real EC policy,
it's better to have a corresponding dummy policy. 

> Erasure Coding: create dummy coder and schema
> ---------------------------------------------
>
>                 Key: HDFS-9345
>                 URL: https://issues.apache.org/jira/browse/HDFS-9345
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Rui Li
>            Assignee: SammiChen
>              Labels: hdfs-ec-3.0-nice-to-have
>
> We  can create dummy coder which does no computation and simply returns zero bytes. Similarly,
we can create a test-only schema with no parity blocks.
> Such coder and schema can be used to isolate the performance issue to HDFS-side logic
instead of codec, which would be useful when tuning performance of EC.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message