hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12257) rework build tool support; add gradle; add scala
Date Wed, 09 Sep 2015 12:36:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736771#comment-14736771
] 

Sean Busbey commented on HADOOP-12257:
--------------------------------------

+1. I'm fine with any of the below getting fixed on commit.


{code}
 # Plug-ins
 
-test-patch allows one to add to its basic feature set via plug-ins.  There is a directory
called test-patch.d off of the directory where test-patch.sh lives.  Inside this directory
one may place some bash shell fragments that, if setup with proper functions, will allow for
test-patch to call it as necessary.
+test-patch allows one to add to its basic feature set via plug-ins.  There is a directory
called test-patch.d off of the directory where test-patch.sh lives.  Inside this directory
one may place some bash shell fragments that, if setup with proper functions, will allow for
test-patch to call it as necessary.  See other documentation for the documentation covering
specific plug-in types.
{code}

the plug-ins can also be provided by a user via command line, which I think is worth mentioning.
I'd be fine with fixing this in a follow on  since it looks like this isn't a part you actually
changed.

"See the following sections for the documentation covering specific plug-in types" would read
better.

{code}
-## Test Plug-ins
+## Generic Plug Functions
 
{code}

"Generic Plug-in Functions" or "Common Plug-in Functions"

{code}
-Every test plugin must have one line in order to be recognized:
+Every plugin must have one line in order to be recognized, usually an 'add' statement.  Test
plugins, for example, have this statement:
 {code}

sometime we use plugin and sometimes we use plug-in. we should pick one.

{code}
 ```bash
 add_plugin <pluginname>
 ```
 
-This function call registers the `pluginname` so that test-patch knows that it exists.  This
plug-in name also acts as the key to the custom functions that you can define. For example:
+This function call registers the `pluginname` so that test-patch knows that it exists.  This
plug-in type also acts as the key to the custom functions that you can define. For example:
 {code}

What does "This plug-in type" mean here? I think maybe it's referring to the type of function
used to register the plug-in? If so, something like "The custom functions a plug-in can define
will also vary by the type of plugin. Continuing with our example of a test plugin:"

{code}
+## Compile Cycle (Branch)
+
+Compilation is done twice with four steps each time:
+
{code}

"When compilation must be done, we follow these four steps:" (so we don't imply that compilation
must be done twice in the branch-compile phase)

{code}
+Now that the patch has been applied, the compile cycle is repeated, but now with the patch
applied. This is where a lot of 'after' checks are performed.
{code}

"Now that the patch has been applied the steps to compile we outlined in the compilation (branch)
phase are repeated but now with"

{code}
+# Ant Specific
+
+## Command Arguments
+
+test-patch always passes -noinput to Ant.  This forces ant to be non-interactive.
+
+# Gradle Specific
+
+The gradle plugin always rebuilds the gradlew file and uses gradlew as the method to execute
commands.
+
{code}


Can we define a test-patch property for each of these, similar to how we give maven users
a profile activation?


{code}
 +# Maven Specific
+
+## Command Arguments
+
+test-patch always passes --batch-mode to maven to force it into non-interactive mode.  Additionally,
some tests will also force -fae in order to get all of messages/errors during that mode. 
It *does not* pass -DskipTests.  Additional arguments should be handled via the personality.
+
{code}

how about: "Additional arguments should be handled via the personality; notably we *do not*
by default pass -DskipTests to compilation phases."

{code}
+  # if we requested offline, pass that to mvn
+  if [[ ${OFFLINE} == "true" ]]; then
+    ANT_ARGS=("${ANT_ARGS[@]}" -Doffline=)
+  fi
+}
{code}

pass that to ant.

{code}
+function javac_initialize
+{
+  local i
+  local jdkdir
+  local tmplist
+
+  if [[ -z ${JAVA_HOME:-} ]]; then
+    case ${OSTYPE} in
+      Darwin)
+        if [[ -z "${JAVA_HOME}" ]]; then
+          if [[ -x /usr/libexec/java_home ]]; then
+            JAVA_HOME="$(/usr/libexec/java_home)"
+            export JAVA_HOME
+          else
+            export JAVA_HOME=/Library/Java/Home
+          fi
+        fi
+      ;;
+      *)
+        yetus_error "WARNING: JAVA_HOME not defined. Disabling java tests."
+        delete_test javac
+        delete_test javadoc
+        return 1
+      ;;
+    esac
+  fi
{code}

Worth a check for /usr/java/default or /usr/java/latest as a fallback for e.g. CentOs / RHEL?

{code}
+function maven_modules_worker
+{
+  declare repostatus=$1
+  declare tst=$2
+
+  # shellcheck disable=SC2034
+  UNSUPPORTED_TEST=false
+
+  case ${tst} in
+    findbugs)
+      modules_workers "${repostatus}" findbugs test-compile findbugs:findbugs -DskipTests=true
+    ;;
+    compile)
+      modules_workers "${repostatus}" compile clean test-compile -DskipTests=true
+    ;;
+    distclean)
+      modules_workers "${repostatus}" distclean clean -DskipTests=true
+    ;;
+    javadoc)
+      modules_workers "${repostatus}" javadoc clean javadoc:javadoc -DskipTests=true
+    ;;
+    scaladoc)
+      modules_workers "${repostatus}" scaladoc clean scala:doc -DskipTests=true
+    ;;
+    unit)
+      modules_workers "${repostatus}" unit clean test -fae
+    ;;
+    *)
+      # shellcheck disable=SC2034
+      UNSUPPORTED_TEST=true
+      if [[ ${repostatus} = patch ]]; then
+        add_footer_table "${tst}" "not supported by the ${BUILDTOOL} plugin"
+      fi
+      yetus_error "WARNING: ${tst} is unsupported by ${BUILDTOOL}"
+      return 1
+    ;;
+  esac
+}
{code}

I thought in the docs we said we didn't pass in -DskipTests? Was that just in some specific
category?

{code}
-## @description  Import content from test-patch.d and optionally
-## @description  from user provided plugin directory
-## @audience     private
+## @description  Execute the static analysis test cycle.
+## @description  This will callout to _precompile, compile, and _postcompile
+## @audience     public
 ## @stability    evolving
 ## @replaceable  no
-function importplugins
+## @param        branch|patch
+## @return       0 on success
+## @return       1 on failure
+function cycle
 {
{code}

If we call this "compile_cycle" it'll line up better with what the documentation calls it.
I don't know if that gets confusing since the steps  in the cycle include compile and pre/post
compile. I think it would be fine.

> rework build tool support; add gradle; add scala
> ------------------------------------------------
>
>                 Key: HADOOP-12257
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12257
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: yetus
>    Affects Versions: HADOOP-12111
>            Reporter: Allen Wittenauer
>            Assignee: Allen Wittenauer
>         Attachments: HADOOP-12257.HADOOP-12111.00.patch, HADOOP-12257.HADOOP-12111.01.patch,
HADOOP-12257.HADOOP-12111.02.patch, HADOOP-12257.HADOOP-12111.03.patch, HADOOP-12257.HADOOP-12111.04.patch,
HADOOP-12257.HADOOP-12111.05.patch
>
>
> We need to rework build tool support to be pluggable as well as add gradle support so
that we cover more of the ecosystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message