hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <shv.had...@gmail.com>
Subject Re: [Vote] Merge branch-trunk-win to trunk
Date Sun, 03 Mar 2013 20:16:26 GMT
Didn't I explain in details what I am asking for?

Thanks,
--Konst

On Sun, Mar 3, 2013 at 11:08 AM, Matt Foley <mfoley@hortonworks.com> wrote:
> Hi Konstantin,
> I'd like to point out two things:
> First, I already committed in this thread (email of Thu, Feb 28, 2013 at
> 6:01 PM) to providing CI for Windows builds.  So please stop acting like I'm
> resisting this idea or something.
> Second, you didn't answer my question, you just kvetched about the phrasing.
> So I ask again:
>
> Will providing full "test-patch" integration (pre-commit build and unit test
> triggered by Jira "Patch Available" state) satisfy your request for
> functionality #1 and #2?  Yes or no, please.
>
> Thanks,
> --Matt
>
>
> On Sat, Mar 2, 2013 at 7:32 PM, Konstantin Shvachko <shv.hadoop@gmail.com>
> wrote:
>>
>> Hi Matt,
>>
>> On Sat, Mar 2, 2013 at 12:32 PM, Matt Foley <mfoley@hortonworks.com>
>> wrote:
>> > Konstantin,
>> > I would like to explore what it would take to remove this perceived
>> > impediment --
>>
>> Glad you decided to explore. Thank you.
>>
>> > although I reserve the right to argue that this is not
>> > pre-requisite to merging the cross-platform support patch.
>>
>> It's your right indeed. So as mine to question what the platform
>> support means for you, which I believe remained unclear.
>> I do not impede the change as you should have noticed. My requirement
>> comes from my perception of the support, which means to me exactly two
>> things:
>> 1. The ability to recognise the code is broken for the platform
>> 2. The ability to test new patches on the platform
>> The latter is problematic, as many noticed in this thread, for those
>> whose customary environment does not include Windows.
>>
>> > If we implemented full "test-patch" support for Windows on trunk, would
>> > that
>> > fulfill both your items #1 and #2?  Please note that:
>> > a) Pushing the "Patch Available" button in Jira shall cause a pre-commit
>> > build to start within, I believe, 20 minutes.
>> > b) That build keeps logs for both java build and unit tests for several
>> > days, that are accessible to all viewers.
>>
>> In item #1 I mostly asking for the nightly build, which is simpler
>> than "test-patch". The latter would be ideal from the platform support
>> viewpoint, but it is for the community to decide if we want to add
>> extra +3 hours to the build.
>> Nightly build in my understanding is triggered by the timer rather
>> than by Jira's "submit patch" button.  On Jenkins build configuration
>> you can specify it under "Build periodically".
>>
>> > So, does this provide sufficient on-demand support that we don't have to
>> > implement a whole new on-demand VM support structure of some sort for #2
>> > (which would be an extraordinary and impractical demand)?
>>
>> I did not mention VMs. Item #2 means a build, which runs "test-patch"
>> target with the file specified by a user (instead of a jira
>> attachment).
>> When user clicks "Build Now" link a box is displayed where the user
>> can enter the file path containing the patch. This can be specified in
>> the Build Configuration under "This build is parameterized" by
>> choosing AddParameter / FileParameter. The build can run on the same
>> Windows machine as the nightly build.
>> Such build will let people test their patches for Windows on Jenkins
>> if they don't posses a license for the right version of Windows.
>> I hope this will not turn into extraordinary or impractical effort.
>>
>> Thanks,
>> --Konst
>>
>> > Thanks,
>> > --Matt
>> >
>> >
>> > On Fri, Mar 1, 2013 at 12:18 PM, Konstantin Shvachko
>> > <shv.hadoop@gmail.com>
>> > wrote:
>> >>
>> >> -1
>> >> We should have a CI infrastructure in place before we can commit to
>> >> supporting Windows platform.
>> >>
>> >> Eric is right Win/Cygwin was supported since day one.
>> >> I had a Windows box under my desk running nightly builds back in
>> >> 2006-07.
>> >> People were irritated but I was filing windows bugs until 0.22 release.
>> >> Times changing and I am glad to see wider support for Win platform.
>> >>
>> >> But in order to make it work you guys need to put the CI process in
>> >> place
>> >>
>> >> 1. windows jenkins build: could be nightly or PreCommit.
>> >> - Nightly would mean that changes can be committed to trunk based on
>> >> linux PreCommit build. And people will file bugs if the change broke
>> >> Windows nightly build.
>> >> - PreCommit-win build will mean automatic reporting failed tests to
>> >> respective jira blocking commits the same way as it is now with linux
>> >> PreCommit builds.
>> >> We should discuss which way is more efficient for developers.
>> >>
>> >> 2. On-demand-windows Jenkins build.
>> >> I see it as a build to which I can attach my patch and the build will
>> >> run my changes on a dedicated windows box.
>> >> That way people can test their changes without having personal windows
>> >> nodes.
>> >>
>> >> I think this is the minimal set of requirement for us to be able to
>> >> commit to the new platform.
>> >> Right now I see only one windows related build
>> >> https://builds.apache.org/view/Hadoop/job/Hadoop-1-win/
>> >> Which was failing since Sept 8, 2012 and did not run in the last month.
>> >>
>> >> Thanks,
>> >> --Konst
>> >>
>> >> On Thu, Feb 28, 2013 at 8:47 PM, Eric Baldeschwieler
>> >> <eric14@hortonworks.com> wrote:
>> >> > +1 (non-binding)
>> >> >
>> >> > A few of observations:
>> >> >
>> >> > - Windows has actually been a supported platform for Hadoop since 0.1
>> >> > .
>> >> > Doug championed supporting windows then and we've continued to do it
>> >> > with
>> >> > varying vigor over time.  To my knowledge we've never made a decision
>> >> > to
>> >> > drop windows support.  The change here is improving our support and
>> >> > dropping
>> >> > the requirement of cigwin.  We had Nutch windows users on the list
in
>> >> > 2006
>> >> > and we've been supporting windows FS requirements since inception.
>> >> >
>> >> > - A little pragmatism will go a long way.  As a community we've got
>> >> > to
>> >> > stay committed to keeping hadoop simple (so it does work on many
>> >> > platforms)
>> >> > and extending it to take advantage of key emerging OS/hardware
>> >> > features,
>> >> > such as containers, new FSs, virtualization, flash ...  We should all
>> >> > plan
>> >> > to let new features & optimizations emerge that don't work
>> >> > everywhere, if
>> >> > they are compelling and central to hadoop's mission of being THE best
>> >> > fabric
>> >> > for storing and processing big data.
>> >> >
>> >> > - A UI project like KDE has to deal with the MANY differences between
>> >> > windows and linux UI APIs.  Hadoop faces no such complex challenge
>> >> > and hence
>> >> > can be maintained from a single codeline IMO.  It is mostly
>> >> > abstracted from
>> >> > the OS APIs via Java and our design choices.  Where it is not we can
>> >> > continue to add plugable abstractions.
>> >> >
>> >
>> >
>
>

Mime
View raw message