hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads
Date Thu, 15 Feb 2018 16:56:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365915#comment-16365915
] 

Anu Engineer commented on HDFS-12345:
-------------------------------------

{quote}I am curious if you can comment at all on one thing - I remember seeing a HDP blog
post about using some of the containerization support in newer versions of YARN to run HDFS
on YARN
{quote}
I know [~vinodkv] has been working on this, tagging him so he can comment. It would be really
cool to run Dynamometer (at least the Datanodes) inside a container. Personally, I think we
should explore this as future extensions to Dynamometer. That is, a work we can do after we
get this tool into Hadoop.

> Scale testing HDFS NameNode with real metadata and workloads
> ------------------------------------------------------------
>
>                 Key: HDFS-12345
>                 URL: https://issues.apache.org/jira/browse/HDFS-12345
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode, test
>            Reporter: Zhe Zhang
>            Assignee: Erik Krogen
>            Priority: Major
>
> Dynamometer has now been open sourced on our [GitHub page|https://github.com/linkedin/dynamometer].
Read more at our [recent blog post|https://engineering.linkedin.com/blog/2018/02/dynamometer--scale-testing-hdfs-on-minimal-hardware-with-maximum].
> To encourage getting the tool into the open for others to use as quickly as possible,
we went through our standard open sourcing process of releasing on GitHub. However we are
interested in the possibility of donating this to Apache as part of Hadoop itself and would
appreciate feedback on whether or not this is something that would be supported by the community.
> Also of note, previous [discussions on the dev mail lists|http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201707.mbox/%3c98FCEFFA-FAFF-4CF1-A14D-4FAAB6567F31@gmail.com%3e]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message