Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 74.125.82.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62yh+ZECXr4WoPmBYVKi1osnoQaAftPE4_kDZ9oEixQ2Rg@mail.gmail.com>
References: 
 <CALte62ws3g561gh864Lizho+B9W6agQERkvG1yp5knQyxxjKQw@mail.gmail.com>
	<CADY20s7Nu9kYi=1x32S6t6686_1pn4yDu-oO9p3u0Yh8K2S=pw@mail.gmail.com>
	<CALte62wPSfO0ieR3Rc9w59ouLxJ0HpTw-4n0=JkAvK3BLmMABQ@mail.gmail.com>
	<CALte62wsH=jEGHK9OJb1XcC9dey-3eAYTLT4Qpp4tg3owDLgcg@mail.gmail.com>
	<CABFz-zjdkSsn=Q9W6qtKp5=uh+Ep54in15jnUkNWtniz1TU=OA@mail.gmail.com>
	<CALte62yh+ZECXr4WoPmBYVKi1osnoQaAftPE4_kDZ9oEixQ2Rg@mail.gmail.com>
Date: Sat, 19 May 2012 18:35:20 -0700
Message-ID: 
 <CALte62yEh-pZnWN+y=ELE6veVnzxj4ptADqytQ0k0GhyJem3pg@mail.gmail.com>
Subject: Re: ClassNotFoundException: org.apache.commons.io.FileUtils
From: Ted Yu <yuzhihong@gmail.com>
To: mapreduce-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=f46d0443060a7212db04c06dccd0

--f46d0443060a7212db04c06dccd0
Content-Type: text/plain; charset=ISO-8859-1

I don't see 1.0.4 in Fix Versions dropdown list.

Personally this defect should at least be documented in 1.0.3 release
notes. Users would have to package commons-io 2.1 themselves so that
map/reduce jobs can succeed.

On Sat, May 19, 2012 at 6:25 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> HADOOP-8417 has been filed.
>
>
> On Sat, May 19, 2012 at 5:59 PM, Luke Lu <llu@vicaya.com> wrote:
>
>> Looks like this is caused by HADOOP-6963, which introduced commons-io
>> 2.1 in ivy.xml but forgot to update the hadoop-core-pom-template.xml.
>> The old version (1.3.2) of the commons-io is probably an existing
>> transient dependency. Please file a blocker/critical bug for
>> 1.0.4/1.1.0. The workaround would be explicitly include commons-io 2.1
>> in downstream projects, but it's not the right thing to do.
>>
>> __Luke
>>
>> On Thu, May 17, 2012 at 9:37 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > I changed hadoop version to 1.0.2 and the test passed.
>> >
>> > I would tend to say 1.0.3 had a regression in this regard.
>> >
>> > Cheers
>> >
>> > On Thu, May 17, 2012 at 8:11 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >
>> >> Correction to my first email:
>> >> I refreshed my HBase 0.92 workspace and I could see the same test
>> failure.
>> >>
>> >> Under ~/.m2/repository/org/apache/commons/commons-io, I still see 1.3.2
>> >>
>> >> I ran 'mvn dependency:tree' under 0.92 and don't see commons-io in the
>> >> output.
>> >>
>> >> Do we need to include commons-io explicitly in our pom.xml ?
>> >>
>> >> For Todd's question:
>> >>
>> >> LM-SJN-00713032:205-hadoop zhihyu$ svn info
>> >> Path: .
>> >> URL: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1
>> >> Repository Root: https://svn.apache.org/repos/asf
>> >> Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
>> >> Revision: 1339961
>> >> Node Kind: directory
>> >> Schedule: normal
>> >> Last Changed Author: mattf
>> >> Last Changed Rev: 1339945
>> >> Last Changed Date: 2012-05-17 18:25:57 -0700 (Thu, 17 May 2012)
>> >>
>> >> The line I referenced in my first email came from
>> ivy/libraries.properties
>> >>
>> >>
>> >> On Thu, May 17, 2012 at 8:06 PM, Todd Lipcon <todd@cloudera.com>
>> wrote:
>> >>
>> >>> I believe we upgraded to commons-io 3.0 in 1.0.3. Did you check the
>> 1.0.3
>> >>> pom?
>> >>>
>> >>> -Todd
>> >>>
>> >>> On Thu, May 17, 2012 at 8:01 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >>> > Hi,
>> >>> > HBase 0.92 upgraded hadoop to 1.0.3 today.
>> >>> > We started seeing some test failures which we didn't see before.
>> e.g.:
>> >>> >
>> >>>
>> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/410/testReport/org.apache.hadoop.hbase.mapred/TestTableMapReduce/testMultiRegionTable/
>> >>> > Here is the interesting part:
>> >>> >
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> >>> org.apache.commons.io.FileUtils
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> >>> >        ... 15 more
>> >>> >
>> >>> > In hadoop 1.0, I see:
>> >>> > commons-io.version=2.1
>> >>> >
>> >>> > HBase doesn't specify commons-io version.
>> >>> >
>> >>> > When I run the same test on MacBook, I don't see the test fail.
>> >>> >
>> >>> > commons-io in local mvn repo is 1.3.2
>> >>> >
>> >>> > If someone can give me hint on why the unit test failed, that would
>> be
>> >>> > great.
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Todd Lipcon
>> >>> Software Engineer, Cloudera
>> >>>
>> >>
>> >>
>>
>
>

--f46d0443060a7212db04c06dccd0--