hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-1094) Performance test infrastructure : skip writing user data on Datanode
Date Thu, 22 Aug 2019 20:22:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913667#comment-16913667

Arpit Agarwal commented on HDDS-1094:

Thanks for the feedback [~anu]. Edited the Jira description based on your comments.

> Performance test infrastructure : skip writing user data on Datanode
> --------------------------------------------------------------------
>                 Key: HDDS-1094
>                 URL: https://issues.apache.org/jira/browse/HDDS-1094
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>          Components: Ozone Datanode
>            Reporter: Supratim Deka
>            Assignee: Supratim Deka
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
> Goal:
> It can be useful to exercise the IO and control paths in Ozone for simulated large datasets
without having huge disk capacity at hand. For example, this will allow us to get things like
container reports and incremental container reports, while not needing huge cluster capacity.
The [SimulatedFsDataset|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java]
does something similar in HDFS. It has been an invaluable tool to simulate large data stores.

This message was sent by Atlassian Jira

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message