hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: i686 support
Date Mon, 20 Jun 2016 19:48:50 GMT
I'm still in information-gathering mode right now... hoping somebody can
tell me what the *expected* behavior of this module's build is supposed to
be, so I can see exactly where there's a discrepancy and what the right
direction is. I can't file a JIRA saying it's not doing what it's supposed
to do, if I don't know what it's supposed to do. All I know right now is
that some portion of the build breaks if you upgrade the version of the
maven-jar-plugin.

I *think* the intent is simply to build the main jar earlier in the
lifecycle (vs. creating a second jar). If somebody can confirm that's true,
then I'll have more information to file the JIRA with.

On Mon, Jun 20, 2016 at 3:33 PM Chris Nauroth <cnauroth@hortonworks.com>
wrote:

> Christopher, thank you for the follow-up information.  Please feel free to
> file an Apache JIRA in the HADOOP project with your findings.
>
> --Chris Nauroth
>
> From: Christopher <ctubbsii@apache.org>
> Date: Monday, June 20, 2016 at 12:26 PM
>
> To: Chris Nauroth <cnauroth@hortonworks.com>, "user@hadoop.apache.org" <
> user@hadoop.apache.org>
> Subject: Re: i686 support
>
> As it turns out, it looks like it's because the version of
> maven-jar-plugin in Fedora was upgraded, and hadoop's build takes advantage
> of the fact that older maven-jar-plugin versions don't notice clobbering
> artifacts with altered ones. This is probably going to bite Hadoop (if not
> already) when upgrading to ASF parent pom 18, which specifies the newest
> maven-jar-plugin.
>
> The solution, which was suggested to me on the Fedora lists, is to skip
> the default-jar execution of the maven-jar-plugin, so that only the
> customized execution would run:
>
> <execution>
>   <id>default-jar</id>
>   <phase>skip</phase>
> </execution>
>
> Unfortunately, maven-jar-plugin doesn't seem to provide a proper "skip"
> flag (all plugins should!), so one has got to hack the execution by
> instructing it to run on a non-existing phase.
>
> Alternatively, Hadoop should really override the "default-jar" execution
> with its own configuration (which as far as I can tell, it just makes the
> jar build earlier in the lifecycle; side note: this is the wrong way to
> go... the "unit test" described in the comment, which requires the main
> jar, should probably be an "integration test").
>
> Thoughts?
>
> (I'm not sure why the build worked for me fine in x86_64, vs. i686, but it
> appears to be a fluke... maybe I had an older version of maven-jar-plugin
> installed already?)
>
> On Fri, Jun 17, 2016 at 5:45 PM Christopher <ctubbsii@apache.org> wrote:
>
>> It's possible it's a problem with the Fedora-packaged Maven or the jar
>> plugin. However, I was able to do a mock build (mock in Fedora uses
>> containers) to build it fine on x86_64. I've been given a hint as to how to
>> tell mock to use a different arch, but haven't yet had a chance to try it.
>>
>> Mainly, I just want to make sure it's supported upstream. If it's not,
>> and never will be, then I can use ExcludeArch in the RPM spec. Otherwise,
>> I've gotta try to figure out why it's broken.
>>
>> On Thu, Jun 16, 2016 at 4:31 PM Chris Nauroth <cnauroth@hortonworks.com>
>> wrote:
>>
>>> Thank you for sharing that.
>>>
>>> I was expecting to see some kind of failure in building our C codebase,
>>> where a 32-bit vs. 64-bit portability problem is likely to happen.
>>> Instead, that's a failure in Maven's handling of a jar dependency.  I tried
>>> building from the 2.4.0 source, and I couldn't reproduce the problem.
>>>
>>> Is it possible that this is a problem specific to the Maven version
>>> running your build?  The BUILDING.txt file in the root of the source tree
>>> has details on what tools and what versions are required for the build.
>>>
>>> --Chris Nauroth
>>>
>>> From: Christopher <ctubbsii@apache.org>
>>> Date: Thursday, June 16, 2016 at 12:48 PM
>>> To: Chris Nauroth <cnauroth@hortonworks.com>, "user@hadoop.apache.org"
<
>>> user@hadoop.apache.org>
>>> Subject: Re: i686 support
>>>
>>> Well, the error I got was:
>>>
>>> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.0.1:jar
(default-jar) on project hadoop-yarn-applications-distributedshell: You have to use a classifier
to attach supplemental artifacts to the project instead of replacing them. -> [Help 1]
>>>
>>>
>>> https://kojipkgs.fedoraproject.org//work/tasks/1712/14521712/build.log
>>>
>>> I'm not sure if this is a Fedora-specific problem (due to its dependency
>>> convergence and the specific version of the maven-jar-plugin, for example),
>>> or if it's an upstream problem. This was version 2.4.1. Right now, I'm just
>>> trying to stabilize the currently packaged 2.4 version, before I begin
>>> moving towards an upgrade path for users to migrate to a newer version
>>> (2.6, probably). It's just that I'm a bit overwhelmed with Hadoop's build
>>> and quantity of components.
>>>
>>> Not sure where to begin looking.
>>>
>>> On Thu, Jun 16, 2016 at 3:42 PM Chris Nauroth <cnauroth@hortonworks.com>
>>> wrote:
>>>
>>>> Hello Christopher,
>>>>
>>>> While 64-bit is certainly the common case at this point, we also have
>>>> produced 32-bit builds in the past.  I'm not aware of any conscious choice
>>>> by the community to halt 32-bit support, so if that's not working, then it
>>>> might be a bug.  Do you have more details on the build failure?  That might
>>>> help us confirm if it's a bug, and if so, move to tracking it in an Apache
>>>> JIRA.
>>>>
>>>> --Chris Nauroth
>>>>
>>>> From: Christopher <ctubbsii@apache.org>
>>>> Date: Thursday, June 16, 2016 at 12:35 PM
>>>> To: "user@hadoop.apache.org" <user@hadoop.apache.org>
>>>> Subject: i686 support
>>>>
>>>> Hi,
>>>>
>>>> Is i686 a supported arch by the upstream Hadoop developers? I'm trying
>>>> to package hadoop RPMs for Fedora (reviving somebody else's previous
>>>> efforts), and can't get things to build on i686. Is this normal? Does
>>>> upstream care about 32-bit users? What about release testing on i686?
>>>>
>>>> I normally wouldn't care about 32-bit, but Fedora packaging standards
>>>> encourage me to get it to work for all core architectures supported by
>>>> Fedora.
>>>>
>>>> Has anybody else done any 32-bit packaging that could help me
>>>> troubleshoot? I don't even own 32-bit hardware. (Even better if you want
to
>>>> help co-maintain Hadoop in Fedora.. that would be awesome).
>>>>
>>>>

Mime
View raw message