Return-Path: X-Original-To: apmail-ignite-dev-archive@minotaur.apache.org Delivered-To: apmail-ignite-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AD50C175AD for ; Wed, 7 Oct 2015 10:46:26 +0000 (UTC) Received: (qmail 73745 invoked by uid 500); 7 Oct 2015 10:46:26 -0000 Delivered-To: apmail-ignite-dev-archive@ignite.apache.org Received: (qmail 73696 invoked by uid 500); 7 Oct 2015 10:46:26 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 73685 invoked by uid 99); 7 Oct 2015 10:46:26 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Oct 2015 10:46:26 +0000 Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com [209.85.217.171]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id AA3341A006D for ; Wed, 7 Oct 2015 10:46:25 +0000 (UTC) Received: by lbbwt4 with SMTP id wt4so7382149lbb.1 for ; Wed, 07 Oct 2015 03:46:23 -0700 (PDT) X-Gm-Message-State: ALoCoQkVG5L/+bjdlk8RfNfcC9uFAfCzcX4Zuk17IAaxsy0VRlfSwdgwgdCzqnH15bfx183oNJgs MIME-Version: 1.0 X-Received: by 10.112.63.135 with SMTP id g7mr202904lbs.16.1444214783541; Wed, 07 Oct 2015 03:46:23 -0700 (PDT) Received: by 10.114.23.66 with HTTP; Wed, 7 Oct 2015 03:46:23 -0700 (PDT) In-Reply-To: References:

Date: Wed, 7 Oct 2015 13:46:23 +0300 Message-ID: Subject: Re: Cluster group affinity From: Yakov Zhdanov To: dev@ignite.apache.org Content-Type: multipart/alternative; boundary=001a11c3ea422723b50521817567 --001a11c3ea422723b50521817567 Content-Type: text/plain; charset=UTF-8 Andrey, probably it is, but I am not sure if I have ever had a thought for mentioned scenario. I think you should get existing implementation and use it like this. Make sure to cache assignments as this may be quiet expensive operation. Ignite ignite = Ignition.start(cfg); ClusterGroup group = ignite.cluster().forPredicate(new Predicate()); final List snapshot = new ArrayList<>(group.nodes()); RendezvousAffinityFunction aff = new RendezvousAffinityFunction(); List> parts = aff.assignPartitions(new AffinityFunctionContext() { @Nullable @Override public List previousAssignment(int part) { return null; } @Override public int backups() { return 0; } @Override public List currentTopologySnapshot() { return snapshot; } @Override public AffinityTopologyVersion currentTopologyVersion() { return null; } @Nullable @Override public DiscoveryEvent discoveryEvent() { return null; } }); // Picking node. ClusterNode node = parts.get(aff.partition(key)).get(0); --Yakov 2015-10-07 11:39 GMT+03:00 Andrey Kornev : > Dmitriy, > > This approach would definitely work, if it wasn't for the fact that the > cluster groups in my case are created dynamically and may include any > combination of nodes in the cluster (where the number of combinations grows > exponentially with the number of nodes in the cluster). I don't think it's > practical to create that many caches. > > I still can't get why the affinity function can't be applied to an > arbitrary cluster group, and why it must necessarily be a cache. Isn't the > cache affinity just a special case of the cluster group affinity defined as > ClusterGroup.forCache()? > > Thanks > Andrey > > > From: dsetrakyan@apache.org > > Date: Tue, 6 Oct 2015 12:07:39 -0700 > > Subject: Re: Cluster group affinity > > To: dev@ignite.apache.org > > > > On Tue, Oct 6, 2015 at 8:46 AM, Andrey Kornev > > wrote: > > > > > Thanks, Andrey! This definitely helps. > > > > > > It's just that implementing such a simple feature in the "user space" > > > feels awkward and requires intimate knowledge of fairly low-level > details > > > of how things work in the current version. > > > > > > Just curios, how about providing an override for Ignite.affinity() > method > > > that ClusterGroup? Is there something fundamentally wrong about > calculating > > > the affinity for an arbitrary collection of nodes (such as a > ClusterGroup > > > is)? > > > > > > > Affinity is usually associated with data. In your case you have no data, > > but you still need keys to be always mapped to the same node. How about > > creating an empty cache and using standard cache API for determining the > > affinity for a key? > > > > > > > Regards > > > Andrey > > > > > > > Date: Tue, 6 Oct 2015 18:12:48 +0300 > > > > Subject: Re: Cluster group affinity > > > > From: agura@gridgain.com > > > > To: dev@ignite.apache.org > > > > > > > > Andrey, > > > > > > > > > > > > > 1) I'm expected to return an instance of the internal class > > > > > AffinityTopologyVersion. > > > > > > > > > > > > If you are talking about > AffinityContextFunction.currentTopologyVersion > > > > method then for now this method is nowhere uses. But it make sense to > > > > return non null value in order to avoid problems in the future. > > > > > > > > 2) the consequences of returning null from > > > > > AffinityFunctionContext.previousAssignment and > > > > > AffinityFunctionContext.discoveryEvent methods (because I can't > > > provide any > > > > > meaningful implementation for them) are not clear. > > > > > > > > > > > > > Both methods declared as @Nullable, so affinity function developer > should > > > > correctly handle this cases. In Ignite only FairAffinityFunction uses > > > these > > > > methods. FairAffinityFunction tries to obtain left node Id from > event of > > > > EventType.EVT_NODE_LEFT or EventType.EVT_NODE_FAILED type. It needs > to > > > > exclude this node assignment from previous assignments. So if your > > > cluster > > > > group lost node you can return EVT_NODE_LEFT discovery event with Id > of > > > > lost node from discoveryEvent method and assignments for previous > cluster > > > > group state from previousAssignment method. > > > > > > > > RendezvousAffinityFunction uses only currentTopologySnapshot() and > > > > backups() methods of AffinityFunctionContext interface. > > > > > > > > > > > > On Tue, Oct 6, 2015 at 5:07 PM, Andrey Kornev < > andrewkornev@hotmail.com> > > > > wrote: > > > > > > > > > Andrey, thanks! > > > > > > > > > > But a "properly formed AffinityFunctionContext" is the problem: > > > > > 1) I'm expected to return an instance of the internal class > > > > > AffinityTopologyVersion. > > > > > 2) the consequences of returning null from > > > > > AffinityFunctionContext.previousAssignment and > > > > > AffinityFunctionContext.discoveryEvent methods (because I can't > > > provide any > > > > > meaningful implementation for them) are not clear. > > > > > > > > > > Please advise. > > > > > > > > > > Thanks > > > > > Andrey > > > > > > > > > > > Date: Tue, 6 Oct 2015 16:43:10 +0300 > > > > > > Subject: Re: Cluster group affinity > > > > > > From: agura@gridgain.com > > > > > > To: dev@ignite.apache.org > > > > > > > > > > > > Andrey, > > > > > > > > > > > > See AffinityFunction.assignPartitions method. It returns > assignment > > > list > > > > > as > > > > > > List> where index of element in returned list > > > > > corresponds > > > > > > to partition number. Assignment for each partition represented as > > > list of > > > > > > nodes where primary node is always the first. So you can use > existing > > > > > > affinity functions for you case just passing properly formed > > > > > > AffinityFunctionContext to assignPartitions method. > > > > > > > > > > > > On Tue, Oct 6, 2015 at 4:25 PM, Andrey Kornev < > > > andrewkornev@hotmail.com> > > > > > > wrote: > > > > > > > > > > > > > Dmitriy, > > > > > > > > > > > > > > The affinity function only maps a key to a partition id and it > > > doesn't > > > > > > > seem to provide a way to map the partition id to a cluster > node. So > > > > > I'm a > > > > > > > little bit confused right now. > > > > > > > > > > > > > > Could you please clarify? > > > > > > > > > > > > > > Thanks a lot > > > > > > > Andrey > > > > > > > > > > > > > > > From: dsetrakyan@apache.org > > > > > > > > Date: Mon, 5 Oct 2015 09:53:25 -0700 > > > > > > > > Subject: Re: Cluster group affinity > > > > > > > > To: dev@ignite.apache.org > > > > > > > > > > > > > > > > On Mon, Oct 5, 2015 at 2:28 AM, Andrey Kornev < > > > > > andrewkornev@hotmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > I have a user-defined cluster group and I'd like to be > able to > > > > > > > > > consistently pick the same node in the group for a given > key. > > > > > > > Essentially, > > > > > > > > > what I want is a cluster group affinity that is not > associated > > > > > with any > > > > > > > > > cache. How can I do it? > > > > > > > > > > > > > > > > > > > > > > > > > Andrey, perhaps you could just take our affinity function and > > > use it > > > > > > > > directly, no? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > Andrey > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Andrey Gura > > > > > > GridGain Systems, Inc. > > > > > > www.gridgain.com > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Andrey Gura > > > > GridGain Systems, Inc. > > > > www.gridgain.com > > > > > > > > --001a11c3ea422723b50521817567--