hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ray Chiang <rchi...@apache.org>
Subject Re: [DISCUSS] A final minor release off branch-2?
Date Wed, 15 Nov 2017 21:16:20 GMT
On 11/15/17 10:34 AM, Andrew Wang wrote:
> Hi Junping,
>
> On Wed, Nov 15, 2017 at 1:37 AM, Junping Du <jdu@hortonworks.com> wrote:
>> 3. Beside incompatibilities, there is also possible to have performance
>> regressions (lower throughput, higher latency, slower job running, bigger
>> memory footprint or even memory leaking, etc.) for new hadoop releases.
>> While the performance impact of migration (if any) could be neglectable to
>> some users, other users could be very sensitive and wish to roll back if it
>> happens on their production cluster.
>>
>> Yes, bugs exist. I won't claim that 3.0.0 is bug-free. All new releases
> can potentially introduce new bugs.
>
> However, I don't think rollback is the solution. In my experience, users
> rarely rollback since it's so disruptive and causes data loss. It's much
> more common that they patch and upgrade. With that in mind, I'd rather we
> spend our effort on making 3.0.x high-quality vs. making it easier to
> rollback.
>
> The root of my concern in announcing a "bridge release" is that it
> discourages users from upgrading to 3.0.0 until a bridge release is out. I
> strongly believe the level of quality provided by 3.0.0 is at least equal
> to new 2.x minor releases, given our extended testing and integration
> process, and we don't have bridge releases for 2.x.
>
> This is why I asked for a list of known issues with 2.x -> 3.0 upgrades,
> that would necessitate a bridge release. Arun raised a concern about NM
> rollback. Are there any other *known* issues?
>

While going over the JACC report as part of YARN-6142, I filed 
HADOOP-14534, MAPREDUCE-6902, and YARN-6717 to document the major issues 
that I ran across.  I think we found one or two other JIRAs which we 
marked as incompatible as part of this investigation.

The protobuf changes should be forward compatible going from 2.8.0 to 3.0.0.

YARN-6798 should fix the NM state store versioning when upgrading from 
2.9.0 to 3.0.0.  2.8.0 to 3.0.0 could have an issue if the relevant 
features are enabled. (queued containers, work-preserving NM restart 
w/AMRMProxy).

-Ray


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message