hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8520) Patch for PPC64 block size
Date Mon, 08 Jun 2015 18:43:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577631#comment-14577631

Andrew Wang commented on HDFS-8520:

Hey Tony, thanks for working on this. I hit Submit Patch since we do need to get a Jenkins
run. It'd help also if you based you patch on trunk rather than branch-2.7.

The NoMlock manipulator is meant for test environments, and it doesn't actually call mlock.
Not related to Windows, just so we don't have to worry about mlock ulimits being set.

What issues do you run into when running the tests on PPC64? I'm guessing PageRounder, since
it's page size isn't affected by setting the NoMlock manipulator. We should probably combine
these two classes, so the PageRounder is also correctly updated when setting the cache manipulator.
This would probably let us avoid the test changes.

If you want to get serious about not regressing PPC64, seems like we should also add some
tests that relate to page size. Having a LargePageCacheManipulator or something would be one

> Patch for PPC64 block size
> --------------------------
>                 Key: HDFS-8520
>                 URL: https://issues.apache.org/jira/browse/HDFS-8520
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>         Environment: RHEL 7.1 /PPC64
>            Reporter: Tony Reix
>            Assignee: Tony Reix
>             Fix For: 2.7.1
>         Attachments: HDFS-8520.patch
> The attached patch enables Hadoop to work on PPC64.
> That deals with SystemPageSize and BloclSize , which are not 4096 on PPC64.
> There are changes in 3 files:
> - hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
> - hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
> - hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCacheDirectives.java
> where 4096 is replaced by getOperatingSystemPageSize() or by using PAGE_SIZE
> The patch has been built on branch-2.7 .

This message was sent by Atlassian JIRA

View raw message