hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-1094) Performance test infrastructure : skip writing user data on Datanode
Date Thu, 22 Aug 2019 20:22:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913668#comment-16913668

Anu Engineer commented on HDDS-1094:

Thanks for the explanation and updating the Jira descption [~arp] . It makes sense why we
need this. [~sdeka]  Please go ahead with the patch, thank you for addressing this issue.

> Performance test infrastructure : skip writing user data on Datanode
> --------------------------------------------------------------------
>                 Key: HDDS-1094
>                 URL: https://issues.apache.org/jira/browse/HDDS-1094
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>          Components: Ozone Datanode
>            Reporter: Supratim Deka
>            Assignee: Supratim Deka
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
> Goal:
> It can be useful to exercise the IO and control paths in Ozone for simulated large datasets
without having huge disk capacity at hand. For example, this will allow us to get things like
container reports and incremental container reports, while not needing huge cluster capacity.
The [SimulatedFsDataset|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java]
does something similar in HDFS. It has been an invaluable tool to simulate large data stores.

This message was sent by Atlassian Jira

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message