hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Foley <mfo...@hortonworks.com>
Subject Re: [Vote] Merge branch-trunk-win to trunk
Date Sat, 02 Mar 2013 20:32:30 GMT
Konstantin,
I would like to explore what it would take to remove this perceived
impediment -- although I reserve the right to argue that this is not
pre-requisite to merging the cross-platform support patch.

If we implemented full "test-patch" support for Windows on trunk, would
that fulfill both your items #1 and #2?  Please note that:
a) Pushing the "Patch Available" button in Jira shall cause a pre-commit
build to start within, I believe, 20 minutes.
b) That build keeps logs for both java build and unit tests for several
days, that are accessible to all viewers.

So, does this provide sufficient on-demand support that we don't have to
implement a whole new on-demand VM support structure of some sort for #2
(which would be an extraordinary and impractical demand)?

Thanks,
--Matt


On Fri, Mar 1, 2013 at 12:18 PM, Konstantin Shvachko
<shv.hadoop@gmail.com>wrote:

> -1
> We should have a CI infrastructure in place before we can commit to
> supporting Windows platform.
>
> Eric is right Win/Cygwin was supported since day one.
> I had a Windows box under my desk running nightly builds back in 2006-07.
> People were irritated but I was filing windows bugs until 0.22 release.
> Times changing and I am glad to see wider support for Win platform.
>
> But in order to make it work you guys need to put the CI process in place
>
> 1. windows jenkins build: could be nightly or PreCommit.
> - Nightly would mean that changes can be committed to trunk based on
> linux PreCommit build. And people will file bugs if the change broke
> Windows nightly build.
> - PreCommit-win build will mean automatic reporting failed tests to
> respective jira blocking commits the same way as it is now with linux
> PreCommit builds.
> We should discuss which way is more efficient for developers.
>
> 2. On-demand-windows Jenkins build.
> I see it as a build to which I can attach my patch and the build will
> run my changes on a dedicated windows box.
> That way people can test their changes without having personal windows
> nodes.
>
> I think this is the minimal set of requirement for us to be able to
> commit to the new platform.
> Right now I see only one windows related build
> https://builds.apache.org/view/Hadoop/job/Hadoop-1-win/
> Which was failing since Sept 8, 2012 and did not run in the last month.
>
> Thanks,
> --Konst
>
> On Thu, Feb 28, 2013 at 8:47 PM, Eric Baldeschwieler
> <eric14@hortonworks.com> wrote:
> > +1 (non-binding)
> >
> > A few of observations:
> >
> > - Windows has actually been a supported platform for Hadoop since 0.1 .
>  Doug championed supporting windows then and we've continued to do it with
> varying vigor over time.  To my knowledge we've never made a decision to
> drop windows support.  The change here is improving our support and
> dropping the requirement of cigwin.  We had Nutch windows users on the list
> in 2006 and we've been supporting windows FS requirements since inception.
> >
> > - A little pragmatism will go a long way.  As a community we've got to
> stay committed to keeping hadoop simple (so it does work on many platforms)
> and extending it to take advantage of key emerging OS/hardware features,
> such as containers, new FSs, virtualization, flash ...  We should all plan
> to let new features & optimizations emerge that don't work everywhere, if
> they are compelling and central to hadoop's mission of being THE best
> fabric for storing and processing big data.
> >
> > - A UI project like KDE has to deal with the MANY differences between
> windows and linux UI APIs.  Hadoop faces no such complex challenge and
> hence can be maintained from a single codeline IMO.  It is mostly
> abstracted from the OS APIs via Java and our design choices.  Where it is
> not we can continue to add plugable abstractions.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message