hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)
Date Thu, 28 Feb 2019 22:35:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780994#comment-16780994
] 

Erik Krogen commented on HDFS-12345:
------------------------------------

Hm. I wonder if it even makes sense for MiniDFSCluster to depend on JUnit... Anyway, I solved
this problem in the GitHub repo by specifying JUnit as a [compile-time dependency|https://github.com/xkrogen/dynamometer/blob/ekrogen-hadoop-3-support/dynamometer-infra/build.gradle#L8].
This works even for the tarball since the tarball is set to [include all of the non-Hadoop
runtime dependencies|https://github.com/linkedin/dynamometer/blob/master/build.gradle#L100].
I'm not sure how to achieve the equivalent in Maven...

> Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)
> --------------------------------------------------------------------------
>
>                 Key: HDFS-12345
>                 URL: https://issues.apache.org/jira/browse/HDFS-12345
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode, test
>            Reporter: Zhe Zhang
>            Assignee: Erik Krogen
>            Priority: Major
>         Attachments: HDFS-12345.000.patch, HDFS-12345.001.patch
>
>
> Dynamometer has now been open sourced on our [GitHub page|https://github.com/linkedin/dynamometer].
Read more at our [recent blog post|https://engineering.linkedin.com/blog/2018/02/dynamometer--scale-testing-hdfs-on-minimal-hardware-with-maximum].
> To encourage getting the tool into the open for others to use as quickly as possible,
we went through our standard open sourcing process of releasing on GitHub. However we are
interested in the possibility of donating this to Apache as part of Hadoop itself and would
appreciate feedback on whether or not this is something that would be supported by the community.
> Also of note, previous [discussions on the dev mail lists|http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201707.mbox/%3c98FCEFFA-FAFF-4CF1-A14D-4FAAB6567F31@gmail.com%3e]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message