hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10114) TestHdfsNativeCodeLoader should reside in hadoop-common instead of hadoop-hdfs
Date Mon, 25 Nov 2013 19:04:36 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831783#comment-13831783

Colin Patrick McCabe commented on HADOOP-10114:

We did this deliberately to detect misbuilds, where {{libhadoop.so}} did not exist.  Running
the tests with partial coverage is the exact opposite of what we want.

There is code in {{test-patch.sh}} designed to address this scenario.  If it's not working,
we should fix it, not rip out native test coverage.  See:

### Run the tests
runTests () {
  echo ""
  echo ""
  echo "======================================================================"
  echo "======================================================================"
  echo "    Running tests."
  echo "======================================================================"
  echo "======================================================================"
  echo ""
  echo ""

  # If we are building hadoop-hdfs-project, we must build the native component
  # of hadoop-common-project first.  In order to accomplish this, we move the
  # hadoop-hdfs subprojects to the end of the list so that common will come
  # first.
  # Of course, we may not be building hadoop-common at all-- in this case, we
  # explicitly insert a mvn compile -Pnative of common, to ensure that the
  # native libraries show up where we need them.
  for module in $modules; do

> TestHdfsNativeCodeLoader should reside in hadoop-common instead of hadoop-hdfs
> ------------------------------------------------------------------------------
>                 Key: HADOOP-10114
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10114
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HADOOP-10114.000.patch
> TestHdfsNativeCodeLoader tests whether Java is able to load the native library libhadoop.so.
> However, it is the hadoop-common project, rather than the hadoop-hdfs project that creates
this library during the build. Therefore this unit test will complain that it cannot find
the library if Jenkins does not rebuild hadoop-common.
> HDFS-3987 is an example that hits this bug.The patch touches hadoop-auth and hadoop-hdfs.
Jenkins decides it won't built hadoop-common, thus the unit test fails simply complaining
about it cannot find libhadoop.so.
> The log of the build can be found at https://builds.apache.org/job/PreCommit-HDFS-Build/5470/consoleFull
> We can avoid this problem by moving the unit test to hadoop-common, or we can remove
this unit test since TestNativeCodeLoader in hadoop-common has already covered the same case.

This message was sent by Atlassian JIRA

View raw message