hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siyao Meng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)
Date Thu, 28 Feb 2019 21:56:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780957#comment-16780957

Siyao Meng commented on HDFS-12345:

We might have a little dependency issue here on JUnit.
I noticed that in the trunk (3.3.0), junit jar has been removed from the Hadoop tarball at
some point (it was under ./share/hadoop/mapreduce/lib/junit-4.11.jar in Hadoop 3.2.1 and before).
Since dynamometer-infra uses MiniDFSCluster that depends on JUnit, we won't be able to run
dynamometer-infra jar in trunk. It can still be compiled, just not included in the distribution
I've tried to add junit jar back under somewhere in the tarball like ./share/hadoop/tools/lib/
but no luck so far. Question: is maven "copy-dependencies" the right way to do it? Need some
help here.

> Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)
> --------------------------------------------------------------------------
>                 Key: HDFS-12345
>                 URL: https://issues.apache.org/jira/browse/HDFS-12345
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode, test
>            Reporter: Zhe Zhang
>            Assignee: Erik Krogen
>            Priority: Major
>         Attachments: HDFS-12345.000.patch, HDFS-12345.001.patch
> Dynamometer has now been open sourced on our [GitHub page|https://github.com/linkedin/dynamometer].
Read more at our [recent blog post|https://engineering.linkedin.com/blog/2018/02/dynamometer--scale-testing-hdfs-on-minimal-hardware-with-maximum].
> To encourage getting the tool into the open for others to use as quickly as possible,
we went through our standard open sourcing process of releasing on GitHub. However we are
interested in the possibility of donating this to Apache as part of Hadoop itself and would
appreciate feedback on whether or not this is something that would be supported by the community.
> Also of note, previous [discussions on the dev mail lists|http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201707.mbox/%3c98FCEFFA-FAFF-4CF1-A14D-4FAAB6567F31@gmail.com%3e]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message