hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: Branching 2.5
Date Sat, 02 Aug 2014 00:31:34 GMT
please document all the stuff was not in the wiki. including urls where to go (jenkins, nexus,
etc), goal is:somebody doing it for first time should not have to scratch his/head. 

thx

Alejandro
(phone typing)

> On Aug 1, 2014, at 17:25, Karthik Kambatla <kasha@cloudera.com> wrote:
> 
> Tom White helped me figure it out, and closed the Nexus repository for me.
> 
> Thanks Tom for helping and Stack for offering to help.
> 
> 
> On Fri, Aug 1, 2014 at 11:28 AM, Karthik Kambatla <kasha@cloudera.com>
> wrote:
> 
>> Folks,
>> 
>> I think we are very close to voting on RC0. Just wanted to check one
>> (hopefully) last thing.
>> 
>> I am unable to verify the signed maven artifacts are actually deployed. To
>> deploy the artifacts, I did the following and it looked like it ran fine.
>> 
>>   1. .m2/settings.xml -> server-id is apache.staging.https
>>   2. mvn deploy -Psign,src,dist -Dmaven.test.skip.exec=true
>>   -Dcontainer-executor.conf.dir=/etc/hadoop/conf
>>   -Dgpg.passphrase=<my-passphrase>
>> 
>> However, I don't see it here - https://repository.apache.org. How do I
>> verify this?
>> 
>> Thanks
>> Karthik
>> 
>> On Wed, Jul 30, 2014 at 4:30 PM, Karthik Kambatla <kasha@cloudera.com>
>> wrote:
>> 
>>> Thanks to Andrew's patch on HADOOP-10910, I am able to build an RC.
>>> 
>>> 
>>>> On Wed, Jul 30, 2014 at 1:59 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>> 
>>>> Adding builds@apache.org
>>>> 
>>>> Cheers
>>>> 
>>>> On Jul 30, 2014, at 12:52 AM, Andrew Wang <andrew.wang@cloudera.com>
>>>> wrote:
>>>> 
>>>>> Alright, dug around some more and I think it's that FINDBUGS_HOME is
>>>> not
>>>>> being set correctly. I downloaded and extracted Findbugs 1.3.9, pointed
>>>>> FINDBUGS_HOME at it, and the build worked after that. I don't know
>>>> what's
>>>>> up with the default maven build, it'd be great if someone could check.
>>>>> 
>>>>> Can someone with access to the build machines check this?
>>>>> 
>>>>> As a side note, I think 1.3.9 was released in 2009. It'd be nice to
>>>> catch
>>>>> up with the last 5 years of static analysis :)
>>>>> 
>>>>> 
>>>>> On Tue, Jul 29, 2014 at 11:36 PM, Andrew Wang <
>>>> andrew.wang@cloudera.com>
>>>>> wrote:
>>>>> 
>>>>>> I looked in the log, it also looks like findbugs is OOMing:
>>>>>> 
>>>>>>    [java] Exception in thread "main" java.lang.OutOfMemoryError:
GC
>>>> overhead limit exceeded
>>>>>>    [java]    at edu.umd.cs.findbugs.ba.Path.grow(Path.java:263)
>>>>>>    [java]    at edu.umd.cs.findbugs.ba.Path.copyFrom(Path.java:113)
>>>>>>    [java]    at edu.umd.cs.findbugs.ba.Path.duplicate(Path.java:103)
>>>>>>    [java]    at
>>>> edu.umd.cs.findbugs.ba.obl.State.duplicate(State.java:65)
>>>>>> 
>>>>>> 
>>>>>> This is quite possibly related, since there's an error at the end
like
>>>>>> this:
>>>>>> 
>>>>>> [ERROR] Failed to execute goal
>>>> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project
>>>> hadoop-hdfs: An Ant BuildException has occured: input file
>>>> /home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/findbugsXml.xml
>>>> does not exist
>>>>>> 
>>>>>> [ERROR] around Ant part ...<xslt
>>>>>> style="/home/jenkins/tools/findbugs/latest/src/xsl/default.xsl"
>>>> in="/home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/findbugsXml.xml"
>>>> out="/home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/site/findbugs.html"/>...
>>>>>> @ 44:368 in
>>>> /home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml
>>>>>> 
>>>>>> I'll try to figure out how to increase this, but if anyone else knows,
>>>>>> feel free to chime in.
>>>>>> 
>>>>>> 
>>>>>> On Tue, Jul 29, 2014 at 5:41 PM, Karthik Kambatla <kasha@cloudera.com
>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> Devs,
>>>>>>> 
>>>>>>> I created branch-2.5.0 and was trying to cut an RC, but ran into
>>>> issues
>>>>>>> with creating one. If anyone knows what is going on, please help
me
>>>> out. I
>>>>>>> ll continue looking into it otherwise.
>>>> https://builds.apache.org/job/HADOOP2_Release_Artifacts_Builder/24/console
>>>>>>> is the build that failed. It appears the issue is because it
can't
>>>> find
>>>>>>> Null.java. I run into the same issue locally as well, even with
>>>>>>> branch-2.4.1. So, I wonder if I should be doing anything else
to
>>>> create
>>>>>>> the
>>>>>>> RC instead?
>>>>>>> 
>>>>>>> Thanks
>>>>>>> Karthik
>>>>>>> 
>>>>>>> 
>>>>>>> On Sun, Jul 27, 2014 at 11:09 AM, Zhijie Shen <zshen@hortonworks.com
>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> I've just committed YARN-2247, which is the last 2.5 blocker
from
>>>> YARN.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Sat, Jul 26, 2014 at 5:02 AM, Karthik Kambatla <
>>>> kasha@cloudera.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> A quick update:
>>>>>>>>> 
>>>>>>>>> All remaining blockers are on the verge of getting committed.
Once
>>>>>>> that
>>>>>>>> is
>>>>>>>>> done, I plan to cut a branch for 2.5.0 and get an RC
out hopefully
>>>>>>> this
>>>>>>>>> coming Monday.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Fri, Jul 25, 2014 at 12:32 PM, Andrew Wang <
>>>>>>> andrew.wang@cloudera.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> One thing I forgot, the release note activities are
happening at
>>>>>>>>>> HADOOP-10821. If you have other things you'd like
to see
>>>> mentioned,
>>>>>>>> feel
>>>>>>>>>> free to leave a comment on the JIRA and I'll try
to include it.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Andrew
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Fri, Jul 25, 2014 at 12:28 PM, Andrew Wang <
>>>>>>>> andrew.wang@cloudera.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I just went through and fixed up the HDFS and
Common CHANGES.txt
>>>>>>> for
>>>>>>>>>> 2.5.0.
>>>>>>>>>>> 
>>>>>>>>>>> As a friendly reminder, please try to put things
under the
>>>> correct
>>>>>>>>>> section
>>>>>>>>>>> :) We have subsections for the xattr changes
in HDFS-2006 and
>>>>>>>>>> HADOOP-10514,
>>>>>>>>>>> and there were some unrelated JIRAs appended
to the end.
>>>>>>>>>>> 
>>>>>>>>>>> I'd also encourage committers to be more liberal
with their use
>>>> of
>>>>>>>> the
>>>>>>>>>> NEW
>>>>>>>>>>> FEATURES section. I'm helping Karthik write up
the 2.5 release
>>>>>>> notes,
>>>>>>>>> and
>>>>>>>>>>> I'm using NEW FEATURES to fill it out. When looking
through the
>>>>>>> JIRA
>>>>>>>>> list
>>>>>>>>>>> though, I decided to promote things like the
SNN/DN/JN webUI
>>>>>>>>>> improvements,
>>>>>>>>>>> the HCFS specification work, and OIV read-only
WebHDFS access to
>>>>>>> new
>>>>>>>>>>> features. One rule-of-thumb, if a feature required
an umbrella
>>>>>>> JIRA,
>>>>>>>>> put
>>>>>>>>>>> the umbrella under NEW FEATURES when it's resolved.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Andrew
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Jul 16, 2014 at 7:59 PM, Wangda Tan <wheeleast@gmail.com
>>>>> 
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Thanks Tsuyoshi for pointing me this,
>>>>>>>>>>>> 
>>>>>>>>>>>> Wangda
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Thu, Jul 17, 2014 at 10:30 AM, Tsuyoshi
OZAWA <
>>>>>>>>>>>> ozawa.tsuyoshi@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Wangda,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The following link is same link as Karthik
mentioned:
>>>> https://issues.apache.org/jira/browse/YARN-2247?jql=project%20in%20(Hadoop%2C%20HDFS%2C%20YARN%2C%20%22Hadoop%20Map%2FReduce%22)%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%2Fs%22%20%3D%202.5.0%20AND%20priority%20in%20(Blocker)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Or, please access to http://goo.gl/FX3iWp
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> - Tsuyoshi
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Jul 17, 2014 at 10:55 AM, Zhijie
Shen <
>>>>>>>>> zshen@hortonworks.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> I raised YARN-2247 as the blocker
of 2.5.0.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Thu, Jul 17, 2014 at 9:42 AM,
Wangda Tan <
>>>>>>>> wheeleast@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi Karthik,
>>>>>>>>>>>>>>> I found I cannot access the filter:
http://s.apache.org/vJg
>>>>>>> .
>>>>>>>>> Could
>>>>>>>>>>>> you
>>>>>>>>>>>>>>> please check its permission?
I'd like to know if there's any
>>>>>>>>>> related
>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>> to me. :)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Wangda
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Thu, Jul 17, 2014 at 5:54
AM, Karthik Kambatla <
>>>>>>>>>>>> kasha@cloudera.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> We are down to 4 blockers
and looks like they are all
>>>>>>>> actively
>>>>>>>>>>>> being
>>>>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>> on. Please reconsider marking
new JIRAs as blockers.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>> Karthik
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> PS: I moved out a couple
of JIRAs that didn't seem like
>>>>>>> true
>>>>>>>>>>>> blockers
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> 2.6.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 11:43
AM, Karthik Kambatla <
>>>>>>>>>>>> kasha@cloudera.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Folks,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> We have 10 blockers for
2.5. Can the people working on
>>>>>>> them
>>>>>>>>>>>> revisit
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> see if they are really
blockers. If they are, can we
>>>>>>> try to
>>>>>>>>> get
>>>>>>>>>>>>> them in
>>>>>>>>>>>>>>>>> soon? It would be nice
to get an RC out the end of this
>>>>>>>> week
>>>>>>>>> or
>>>>>>>>>>>> at
>>>>>>>>>>>>>>> least
>>>>>>>>>>>>>>>>> early next week?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>> Karthik
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Jul 2, 2014 at
11:32 PM, Karthik Kambatla <
>>>>>>>>>>>>> kasha@cloudera.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I just
>>>>>>>>>>>>>>>>>> 1. moved non-blocker
2.5 JIRAs to 2.6.
>>>>>>>>>>>>>>>>>> 2. created branch-2.5
and added sections for 2.6.0 in
>>>>>>> all
>>>>>>>>>>>>> CHANGES.txt
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> trunk and branch-2.
>>>>>>>>>>>>>>>>>> 3. Will create branch-2.5.0
when we are ready to
>>>>>>> create an
>>>>>>>>> RC
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> There are 11 pending
blockers for 2.5.0:
>>>>>>>>>>>> http://s.apache.org/vJg
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Committers - please
exercise caution when merging to
>>>>>>>>>> branch-2.5
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> target non-blockers
preferably to 2.6
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Jul 2, 2014
at 10:24 PM, Karthik Kambatla <
>>>>>>>>>>>>> kasha@cloudera.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Committers,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I am working
on branching 2.5. Will send an update as
>>>>>>>> soon
>>>>>>>>> as
>>>>>>>>>>>> I am
>>>>>>>>>>>>>>> done
>>>>>>>>>>>>>>>>>>> branching.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Zhijie Shen
>>>>>>>>>>>>>> Hortonworks Inc.
>>>>>>>>>>>>>> http://hortonworks.com/
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>>>>>>>>> NOTICE: This message is intended
for the use of the
>>>>>>> individual
>>>>>>>> or
>>>>>>>>>>>> entity
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> which it is addressed and may contain
information that is
>>>>>>>>>>>> confidential,
>>>>>>>>>>>>>> privileged and exempt from disclosure
under applicable law.
>>>>>>> If
>>>>>>>> the
>>>>>>>>>>>> reader
>>>>>>>>>>>>>> of this message is not the intended
recipient, you are hereby
>>>>>>>>>> notified
>>>>>>>>>>>>> that
>>>>>>>>>>>>>> any printing, copying, dissemination,
distribution,
>>>>>>> disclosure
>>>>>>>> or
>>>>>>>>>>>>>> forwarding of this communication
is strictly prohibited. If
>>>>>>> you
>>>>>>>>> have
>>>>>>>>>>>>>> received this communication in error,
please contact the
>>>>>>> sender
>>>>>>>>>>>>> immediately
>>>>>>>>>>>>>> and delete it from your system. Thank
You.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> - Tsuyoshi
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Zhijie Shen
>>>>>>>> Hortonworks Inc.
>>>>>>>> http://hortonworks.com/
>>>>>>>> 
>>>>>>>> --
>>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>>> NOTICE: This message is intended for the use of the individual
or
>>>>>>> entity to
>>>>>>>> which it is addressed and may contain information that is
>>>> confidential,
>>>>>>>> privileged and exempt from disclosure under applicable law.
If the
>>>>>>> reader
>>>>>>>> of this message is not the intended recipient, you are hereby
>>>> notified
>>>>>>> that
>>>>>>>> any printing, copying, dissemination, distribution, disclosure
or
>>>>>>>> forwarding of this communication is strictly prohibited.
If you have
>>>>>>>> received this communication in error, please contact the
sender
>>>>>>> immediately
>>>>>>>> and delete it from your system. Thank You.
>> 

Mime
View raw message