hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Masatake Iwasaki <iwasak...@oss.nttdata.co.jp>
Subject Re: [VOTE] Merge feature branch HADOOP-12930
Date Sun, 15 May 2016 04:04:33 GMT
Hi Allen,

I had a problem when I was testing HADOOP-12930 branch. (built by mvn 
package -Pdist)

Single argument is ok.::

   [iwasakims@centos7 HADOOP-12930]$ bin/hadoop hello foo

Error on multiple arguments.  ::

   [iwasakims@centos7 HADOOP-12930]$ bin/hadoop hello foo bar
   bin/hadoop: line 200: foo bar: syntax error in expression (error 
token is "bar")
   Error: Could not find or load main class

   [iwasakims@centos7 HADOOP-12930]$ bin/hadoop distcp /foo /bar
   bin/hadoop: line 200: /foo /bar: syntax error: operand expected 
(error token is "/foo /bar")
   Error: Could not find or load main class

The fix of HADOOP-13120 seems to be related.::

   diff --git a/hadoop-common-project/hadoop-common/src/main/bin/hadoop 
   index 61fdc2e..0f6982b 100755
   --- a/hadoop-common-project/hadoop-common/src/main/bin/hadoop
   +++ b/hadoop-common-project/hadoop-common/src/main/bin/hadoop
   @@ -197,6 +197,7 @@ shift

    if declare -f hadoop_subcommand_"${HADOOP_SUBCMD}" >/dev/null 2>&1; then
   +  hadoop_debug "Calling dynamically: 
hadoop_subcommand_${HADOOP_SUBCMD} ${HADOOP_SUBCMD_ARGS[$*]}"
      "hadoop_subcommand_${HADOOP_SUBCMD}" "${HADOOP_SUBCMD_ARGS[@]}"
      hadoopcmd_case "${HADOOP_SUBCMD}" "${HADOOP_SUBCMD_ARGS[@]}"

Masatake Iwasaki

On 5/15/16 02:33, Allen Wittenauer wrote:
> This vote closes in 2 days and the only response has been from a non-committer and one
of the 137 other committers on the project…. it’d be great if some others could take a
> Thanks!
>> On May 12, 2016, at 6:07 PM, Andrew Wang <andrew.wang@cloudera.com> wrote:
>> +1. I looked at the patches on the branch, wasn't too bad to review. As
>> Allen said, there's some code movement, assorted other nice doc and shell
>> fixups.
>> Found one extra typo, which I added to HADOOP-13129.
>> Best,
>> Andrew
>> On Wed, May 11, 2016 at 1:14 AM, Sean Busbey <busbey@cloudera.com> wrote:
>>> +1 (non-binding)
>>> reviewed everything, filed an additional subtask for a very trivial
>>> typo in the docs. should be fine to make a full issue after close and
>>> then fix.
>>> tried merging locally, tried running through new shell tests (both
>>> with and without bats installed), tried making an example custom
>>> command (valid and malformed). everything looks great.
>>> On Mon, May 9, 2016 at 1:26 PM, Allen Wittenauer <aw@apache.org> wrote:
>>>>         Hey gang!
>>>>         I’d like to call a vote to run for 7 days (ending May 16 at
>>> 13:30 PT) to merge the HADOOP-12930 feature branch into trunk. This branch
>>> was developed exclusively by me as per the discussion two months ago as a
>>> way to make what would be a rather large patch hopefully easier to review.
>>> The vast majority of the branch is code movement in the same file,
>>> additional license headers, maven assembly hooks for distribution, and
>>> variable renames. Not a whole lot of new code, but a big diff file
>>> none-the-less.
>>>>         This branch modifies the ‘hadoop’, ‘hdfs’, ‘mapred’,
and ‘yarn’
>>> commands to allow for subcommands to be added or modified at runtime.  This
>>> allows for individual users or entire sites to tweak the execution
>>> environment to suit their local needs.  For example, it has been a practice
>>> for some locations to change the distcp jar out for a custom one.  Using
>>> this functionality, it is possible that the ‘hadoop distcp’ command could
>>> run the local version without overwriting the bundled jar and for existing
>>> documentation (read: results from Internet searches) to work as written
>>> without modification. This has the potential to be a huge win, especially
>>> for:
>>>>                 * advanced end users looking to supplement the Apache
>>> Hadoop experience
>>>>                 * operations teams that may be able to leverage existing
>>> documentation without having to remain local “exception” docs
>>>>                 * development groups wanting an easy way to trial
>>> experimental features
>>>>         Additionally, this branch includes the following, related
>>> changes:
>>>>                 * Adds the first unit tests for the ‘hadoop’ command
>>>>                 * Adds the infrastructure for hdfs script testing and
>>> the first unit test for the ‘hdfs’ command
>>>>                 * Modifies the hadoop-tools components to be dynamic
>>> rather than hard coded
>>>>                 * Renames the shell profiles for hdfs, mapred, and yarn
>>> to be consistent with other bundled profiles, including the ones introduced
>>> in this branch
>>>>         Documentation, including a ‘hello world’-style example, is in
>>> the UnixShellGuide markdown file.  (Of course!)
>>>>          I am at ApacheCon this week if anyone wants to discuss in-depth.
>>>>         Thanks!
>>>> P.S.,
>>>>         There are still two open sub-tasks.  These are blocked by other
>>> issues so that we may add unit testing to the shell code in those
>>> respective areas.  I’ll covert to full issues after HADOOP-12930 is closed.
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>> --
>>> busbey
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message