ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: Changing public IgniteCompute API to improve changes in 5037 ticket
Date Fri, 21 Jul 2017 20:42:22 GMT
Maxim,

The issue is that it's currently assumed to support job mapping, but it
actually doesn't. However, I agree that AffinityKeyMapped annotation
doesn't fit the use case well. Let's fix documentation and JavaDoc then.

As for the proposed API, it's overcomplicated, took me 15 minutes to
understand what it does :)

What is the use case for which current affinityRun/Call API doesn't work?

-Val

On Fri, Jul 21, 2017 at 5:57 AM, Kozlov Maxim <dreamx.max@gmail.com> wrote:

> Valentin,
>
> The author of tiket wants to see to provide some API allows to map
> ComputeJobs to partitions or keys. If we use @AffinityKeyMapped then you
> need to enter the cache name parameter, I think this is not convenient for
> the user. Therefore, I propose to extend the existing API.
> Having consulted with Anton V. decided to make a separate interface
> ReducibleTask, which will allow us to have different map logic at each
> inheritor.
>
> Old method, allows to map to node
> public interface ComputeTask<T, R> extends ReducibleTask<R> {
>     @Nullable public Map<? extends ComputeJob, ClusterNode>
> map(List<ClusterNode> subgrid, @Nullable T arg) throws IgniteException;
> }
>
> Brand new method with mapping to partitions, which solves topology change
> issues.
> public interface AffinityComputeTask<T, R> extends ReducibleTask<R> {
>     @Nullable public Map<? extends ComputeJob, Integer> map(@NotnullString
> cacheName, List<Integer> partIds, @Nullable T arg) throws IgniteException;
> }
>
> public interface ReducibleTask<R> extends Serializable {
>     public ComputeJobResultPolicy result(ComputeJobResult res,
> List<ComputeJobResult> rcvd) throws IgniteException;
>
>     @Nullable public R reduce(List<ComputeJobResult> results) throws
> IgniteException;
> }
>
> We also need to implement AffinityComputeTaskAdapter and
> AffinityComputeTaskSplitAdapter, for implementation by default. It is
> right?
>
> In the IgniteCompute add:
>
> @IgniteAsyncSupported
> public <T, R> R affinityExecute(Class<? extends AffinityComputeTask<T, R>>
> taskCls, List<Integer> partIds, @Nullable T arg) throws IgniteException;
> @IgniteAsyncSupported
> public <T, R> R affinityExecute(AffinityComputeTask<T, R> task,
> List<Integer> partIds, @Nullable T arg) throws IgniteException;
>
> public <T, R> ComputeTaskFuture<R> affinityExecuteAsync(Class<? extends
> AffinityComputeTask<T, R>> taskCls, List<Integer> partIds, @Nullable T
arg)
> throws IgniteException;
> public <T, R> ComputeTaskFuture<R> affinityExecuteAsync(AffinityComputeTask<T,
> R> task, List<Integer> partIds, @Nullable T arg) throws IgniteException;
>
>
> How do you like this idea or do you insist that you need to use
> @AffinityKeyMapped to solve the problem?
>
>
> > 13 июля 2017 г., в 6:36, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> написал(а):
> >
> > Hi Max,
> >
> > This ticket doesn't assume any API changes, it's about broken
> > functionality. I would start with checking what tests we have
> > for @AffinityKeyMapped and creating missing one. From what I understand
> > functionality is broken completely or almost completely, so I guess
> testing
> > coverage is very weak there.
> >
> > -Val
> >
> > On Wed, Jul 12, 2017 at 4:27 PM, Kozlov Maxim <dreamx.max@gmail.com>
> wrote:
> >
> >> Igniters,
> >>
> >> jira: https://issues.apache.org/jira/browse/IGNITE-5037 <
> >> https://issues.apache.org/jira/browse/IGNITE-5037>
> >> How do you look to solve this ticket by adding two methods to the public
> >> IgniteCompute API?
> >>
> >> @IgniteAsyncSupported
> >> public void affinityRun(@NotNull Collection<String> cacheNames,
> >> Collection<Object> keys, IgniteRunnable job)
> >>    throws IgniteException;
> >>
> >> @IgniteAsyncSupported
> >> public <R> R affinityCall(@NotNull Collection<String> cacheNames,
> >> Collection<Object> keys, IgniteCallable<R> job)
> >>    throws IgniteException;
> >>
> >> There is also a question of how to act when changing the topology during
> >> the execution of the job.
> >> 1) complete with an exception;
> >> 2) stop execution and wait until the topology is rebuilt and continue
> >> execution;
> >>
> >> I think the second way, do you think?
> >>
> >> --
> >> Best Regards,
> >> Max K.
> >>
> >>
> >>
> >>
> >>
>
> --
> Best Regards,
> Max K.
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message