hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@alumni.cmu.edu>
Subject Re: Switching to Java 7
Date Mon, 08 Dec 2014 19:58:50 GMT
On Mon, Dec 8, 2014 at 7:46 AM, Steve Loughran <stevel@hortonworks.com> wrote:
> On 8 December 2014 at 14:58, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> Looks like there was still OutOfMemoryError :
>>
>>
>> https://builds.apache.org/job/Hadoop-Hdfs-trunk/1964/testReport/junit/org.apache.hadoop.hdfs.server.namenode.snapshot/TestRenameWithSnapshots/testRenameDirAcrossSnapshottableDirs/
>>
>
> Well, I'm going to ignore that for now as it's a java 8 problem, surfacing
> this weekend once the builds were actually switched to Java 8. memory size
> tuning can continue.
>
> I have now committed the Java 7+ only patch to branch-2 and up: new code
> does not have to worry about java 6 compatibility unless they plan to
> backport to Java 2.6 or earlier. Having written some Java 7 code, the <>
> constructor for typed classes are a convenience, the multiple-catch entries
> more useful, as they eliminate duplicate code in exception handling.
>
> Getting this patch in has revealed that the Jenkins builds of hadoop are
> (a) a bit of a mess and (b) prone to race conditions related to the m2
> repository if >1 project builds simultaneously. The way the nightly builds
> are staggered means this doesn't usually surface, but it may show up during
> precommit/postcommit builds.

It would be nice if we could have a separate .m2 directory per test executor.

It seems like that would eliminate these race conditions once and for
all, at the cost of storing a few extra jars (proportional to the # of
simultaneous executors)

best,
Colin


>
> The switch to Java 7 as the underlying JDK appears to be triggering
> failures, these are things that the projects themselves are going to have
> to look at.
>
>
> This then, is where we are with builds right now. This is not a consequence
> of the changes to the POM; this list predates that patch. This is Jenkins
> running Hadoop builds and tests with Java 7u55
>
>
> *Working: *
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-branch2/
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk/
>
> *failing tests*
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Common-2-Build/
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk/
> https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-YARN-Build/
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Mapreduce-trunk/
>
> *failing tests on Java 8 (may include OOM)*
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-common-trunk-Java8/
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk-Java8/
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Java8/
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/
>
>
> *failing with maven internal dependency problems*
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-Commit/
>
>
> *failing even though it appears to work in the logs*
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Common-trunk/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Mime
View raw message