helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: Helix V2 0.7.0
Date Mon, 24 Feb 2014 09:46:08 GMT
Hi,

I'm also in favor of improving the API even though that involves breaking backward compatibility.

it looks like the 0.7.0 version captures concepts and usage much better.

As Json pointed out, most compatibility issues should be detected by swapping libs and fixing
compilation errors.

But that also means a few issues might remain. In particular, if there are semantic changes,
it would be good to have 
them pointed out somewhere (small wiki page or something like that, with a mapping old concepts/API
-> new concepts/API).

Thanks!

Matthieu

On Feb 23, 2014, at 02:20 , Zhen Zhang <nehzgnahz@gmail.com> wrote:

> +1 for helix-api package that contains public interface only and decouples
> from implementation. The api package should includes Accessor, HelixAdmin,
> HelixManager, HelixConnection, MessagingService, etc. We can start by
> listing all the necessary interfaces. I am not sure if we have a clear
> separation between API and SPI. For example, we are using Accessor and
> MessagingService as both API and SPI. Nevertheless, we need to stick to the
> interfaces inside Helix also. Current Helix code base has many places where
> we use concrete implementations instead of interfaces.
> 
> For backward compatibility, I agree with Kishore that as long as we are
> compatible with physical model, most of the compatibility issues should be
> solved at compile time. We might have to keep the old interfaces for
> current uses, but need to clean them up. For example, we can't get rid of
> HelixManager totally and enforce people to switch to the new
> HelixConnectiont/HelixParticipant, but we should remove/change confusing
> api's. Users should be able to easily fix the compatibility issues at
> compile time.
> 
> Thanks,
> Jason
> 
> 
> 
> 
> On Sat, Feb 22, 2014 at 11:48 AM, <bob@schulman.com> wrote:
> 
>> Here's how I'd evaluate what to do, consider these 4 sub-problems:
>> 
>> 1. Ease of learning
>> - are the APIs accessible for a first-timer?  Like it or not, that's an
>> important factor in getting wider adoption.  Necessary but not sufficient.
>> - a separate helix-api package is a good way to keep it as simple as
>> possible, because then you only have to understand that to build a system
>> using Helix
>> - also, a new package gives us an opportunity to change terminology if
>> that will help new users
>> 
>> 2. Functionality
>> - can Helix do what I want?
>> - as long as the APIs, say in a helix-api package don't hide things a
>> system developer needs, this isn't a tradeoff
>> - careful terminology might help here
>> 
>> 3. Maintainability/Extensibility
>> - for a Helix developer, is it possible to add functionality?  That only
>> matters if there are gaps in (2), it's hard to know if that will keep
>> coming up as more use-cases emerge
>> - for a Helix committer, is the cost of maintenance too high, particularly
>> to maintain backward compatibility
>> 
>> 4. Backward compatibility
>> - we don't want to leave early adopters out in the cold
>> 
>> Given that framework for evaluation, the helix-api package looks like the
>> way to go.  I'm not quite sure how the old branch would be retired, other
>> than polling users and making a judgment call.  There are more people using
>> Helix than we know about, as I continue to run into people using it that we
>> didn't know about.
>> 
>> The key is to get the APIs right, easier said than done.  That's my $0.02.
>> 
>> Thanks,
>> Bob
>> 
>> 
>> Good suggestion, I think Vinayak had suggestions on similar lines but I
>>> think he suggestion writing high level api's on top of existing api's, as
>>> opposed to changing the  changing the underlying implementation. What that
>>> means is we leave the existing api's as is but add high level api. What
>>> this means is we can keep the core as is but add helix-api module that
>>> provides high level interfaces. We can re-use existing implementation
>>> under
>>> the hood. I like the idea of creating a separate module helix-api that is
>>> decoupled from implementation. This will enforce us to have no dependency
>>> on zookeeper in api and allows one to plug other forms of storage. For
>>> example many usecases dont need the level of consistency we get from
>>> Zookeeper.
>>> 
>>> Looks like we might have to make decisions on a case by case basis.
>>> 
>>> Can we make a decision of having a helix-api package.
>>> 
>>> thanks,
>>> Kishore G
>>> 
>>> 
>>> On Sat, Feb 22, 2014 at 10:09 AM, Kanak Biscuitwala <kanak.b@hotmail.com
>>>> wrote:
>>> 
>>> The biggest problem is that we pretty much have to backport bug fixes
>>>> forever if we completely break compatibility. Here's what I think we
>>>> could
>>>> do:
>>>> 
>>>> 1) Remove APIs that we're confident no one uses or can use (the alerting
>>>> stuff)
>>>> 2) Keep the interfaces for the most common currently used APIs
>>>> (HelixManagerFactory, HelixManager, HelixAdmin, ClusterSetup, command
>>>> line,
>>>> REST), but use our new implementations underneath (HelixConnection,
>>>> HelixController, HelixParticipant, HelixSpectator, HelixAdministrator).
>>>> This means there will be breakages for people who used the implementation
>>>> classes directly, but the changes would be minor.
>>>> 3) Provide adapters between common new and old APIs
>>>> (HelixConnectionAdapter is an example)
>>>> 4) Maintain the model package as-is
>>>> 
>>>> This way, there's a clear transition path from one branch to another and
>>>> we can eventually end support for the old branch.
>>>> ________________________________
>>>>> Date: Sat, 22 Feb 2014 07:33:10 -0800
>>>>> Subject: Helix V2 0.7.0
>>>>> From: g.kishore@gmail.com
>>>>> To: dev@helix.apache.org; user@helix.apache.org
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> We have made lot of good changes in 0.7.0 and improved the api's.
>>>>> However, I think it is not easy/intuitive for a new user. One of the
>>>>> problems with 0.7.0 code is that we tried to maintain complete backward
>>>>> compatibility with respect to both logical api's and physical layout
on
>>>>> zookeeper.
>>>>> 
>>>>> Trying to maintain the logical api backward compatibility has
>>>>> definitely caused some pain and did not allow us to do the right thing
>>>>> in 0.7.0 and it has made our code base huge. The reasoning here is -
>>>>> when we built Helix ((almost 3 years back), we did not anticipate Helix
>>>>> being used in other systems. So our main focus was minimal code and to
>>>>> make sure it works for the use case we had. We did not gather much
>>>>> feedback from users.
>>>>> 
>>>>> However, we are seeing the usage grow and while everyone agrees that
>>>>> the high level concepts are good, it is apparent that api's are making
>>>>> people shy away from Helix. I would even say some of the terminologies
>>>>> are confusing until you spend quite some time with Helix.
>>>>> 
>>>>> I want to see what others think about this.
>>>>> 
>>>>> We have two options going forward
>>>>> 
>>>>> Option1: Continue to maintain backward compatibility and improving the
>>>> api's
>>>>> Option2: Break the api compatibility and call it Helix V2. We redesign
>>>>> our api's and make it more intuitive and easier/flexible to use.
>>>>> 
>>>>> I think the core functionality and design is great and don't see much
>>>>> change needed in the architecture (Do let us know if you think we need
>>>>> any change). What is lacking is documentation and a simple set of api's
>>>>> that are intuitive.
>>>>> 
>>>>> While Option 1 is great for existing users, I prefer Option2. We will
>>>>> redesign the 0.7.0 api's without maintaining backward compatibility.
>>>>> Lot of work has already been done in 0.7.0, so we are not that far.
>>>>> This also gives chance to the community to contribute and provide
>>>>> suggestions/feedback/ideas.
>>>>> 
>>>>> For existing users, we will continue to maintain 0.6.2 and continue to
>>>>> make critical bug fixes. But no new features will be added.
>>>>> 
>>>>> Thoughts ?
>>>>> 
>>>>> thanks,
>>>>> Kishore G
>>>> 
>>>> 
>>> 
>> 
>> 
>> 


Mime
View raw message