ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: Thin client: compute support
Date Thu, 21 Nov 2019 12:36:51 GMT
Igor,

Yes, we can make it work with some additional support on the server:
- Include Platform code along with task name
- Invoke Platform when code is not JAVA
- Add a way to invoke tasks by name in C++ and .NET

On Thu, Nov 21, 2019 at 2:55 PM Igor Sapego <isapego@apache.org> wrote:

> Pavel,
>
> The proposed solution won't work for PHP, Python, Node.js,
> but it will work for C++ and .NET, isn't it? We will just have
> to deploy C++/.NET code in closter (just as in Java).
>
> Best Regards,
> Igor
>
>
> On Thu, Nov 21, 2019 at 12:59 PM Aleksandr Shapkin <lexwert@gmail.com>
> wrote:
>
> > Folks,
> >
> >
> >
> > I started to implement the ClusterGroup API.
> >
> >
> >
> > There are two tickets at the moment:
> >
> > - .NET thin client: introduce Cluster API [1]
> >
> > - .NET Thin Client: introduce ClusterGroup API [2]
> >
> >
> >
> > The first one introduces the cluster API and is merged to master.
> >
> > The second ticket is in progress and allows clients to query and filter
> > server nodes.
> >
> >
> >
> > [1] - https://issues.apache.org/jira/browse/IGNITE-11709
> >
> > [2] - https://issues.apache.org/jira/browse/IGNITE-12385
> >
> > чт, 21 нояб. 2019 г. в 12:48, Denis Garus <garus.d.g@gmail.com>:
> >
> > > Hello, Sergey!
> > >
> > >
> > > >> 3. Ignite doesn't have roles/authorization functionality for now.
> > >
> > >
> > > I can't agree with you.
> > >
> > > We already have authorization functionality in Ignite and for a thin
> > client
> > > too [1].
> > >
> > > But, compute support for a thin client requires some additional efforts
> > to
> > > get an appropriate SecurityContext on a remote node.
> > >
> > > The list of tasks allowed for subjects, including thin clients, is the
> > area
> > > of responsibility of GridSecurityProcessor [2].
> > >
> > >
> > >
> > >    1.
> > >
> > >
> >
> org.apache.ignite.internal.processors.security.client.ThinClientPermissionCheckTest
> > >    2.
> > org.apache.ignite.internal.processors.security.GridSecurityProcessor
> > >
> > >
> > > чт, 21 нояб. 2019 г. в 12:41, Pavel Tupitsyn <ptupitsyn@apache.org>:
> > >
> > > > Good points, Sergey.
> > > > Maybe you are right, and Java-based compute without peer deployment
> is
> > a
> > > > good first step for thin clients.
> > > >
> > > > On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov <skozlov@gridgain.com
> >
> > > > wrote:
> > > >
> > > > > Hi Pavel
> > > > >
> > > > > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn <
> > ptupitsyn@apache.org>
> > > > > wrote:
> > > > >
> > > > > > 1. I believe that Cluster operations for Thin Client protocol
are
> > > > already
> > > > > > in the works
> > > > > > by Alexandr Shapkin. Can't find the ticket though.
> > > > > > Alexandr, can you please confirm and attach the ticket number?
> > > > > >
> > > > > > 2. Proposed changes will work only for Java tasks that are
> already
> > > > > deployed
> > > > > > on server nodes.
> > > > > > This is mostly useless for other thin clients we have (Python,
> PHP,
> > > > .NET,
> > > > > > C++).
> > > > > >
> > > > >
> > > > > I don't guess so. The task (execution) is a way to implement own
> > layer
> > > > for
> > > > > the thin client application.
> > > > >
> > > > >
> > > > > > We should think of a way to make this useful for all clients.
> > > > > > For example, we may allow sending tasks in some scripting
> language
> > > like
> > > > > > Javascript.
> > > > > > Thoughts?
> > > > > >
> > > > >
> > > > > The arbitrary code execution from a remote client must be protected
> > > > > from malicious code.
> > > > > I don't know how it could be designed but without that we open the
> > hole
> > > > to
> > > > > kill cluster.
> > > > >
> > > > >
> > > > > >
> > > > > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov <
> > skozlov@gridgain.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Alex
> > > > > > >
> > > > > > > The idea is great. But I have some concerns that probably
> should
> > be
> > > > > taken
> > > > > > > into account for design:
> > > > > > >
> > > > > > >    1. We need to have the ability to stop a task execution,
> smth
> > > like
> > > > > > >    OP_COMPUTE_CANCEL_TASK  operation (client to server)
> > > > > > >    2. What's about task execution timeout? It may help
to the
> > > cluster
> > > > > > >    survival for buggy tasks
> > > > > > >    3. Ignite doesn't have roles/authorization functionality
for
> > > now.
> > > > > But
> > > > > > a
> > > > > > >    task is the risky operation for cluster (for security
> > reasons).
> > > > > Could
> > > > > > we
> > > > > > >    add for Ignite configuration new options:
> > > > > > >       - Explicit turning on for compute task support for
thin
> > > > protocol
> > > > > > >       (disabled by default) for whole cluster
> > > > > > >       - Explicit turning on for compute task support for
a node
> > > > > > >       - The list of task names (classes) allowed to execute
by
> > thin
> > > > > > client.
> > > > > > >    4. Support the labeling for task that may help to
> investigate
> > > > issues
> > > > > > on
> > > > > > >    cluster (the idea from IEP-34 [1])
> > > > > > >
> > > > > > > 1.
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov <
> > > > > plehanov.alex@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello, Igniters!
> > > > > > > >
> > > > > > > > I have plans to start implementation of Compute interface
for
> > > > Ignite
> > > > > > thin
> > > > > > > > client and want to discuss features that should be
> implemented.
> > > > > > > >
> > > > > > > > We already have Compute implementation for binary-rest
> clients
> > > > > > > > (GridClientCompute), which have the following functionality:
> > > > > > > > - Filtering cluster nodes (projection) for compute
> > > > > > > > - Executing task by the name
> > > > > > > >
> > > > > > > > I think we can implement this functionality in a thin
client
> as
> > > > well.
> > > > > > > >
> > > > > > > > First of all, we need some operation types to request
a list
> of
> > > all
> > > > > > > > available nodes and probably node attributes (by a
list of
> > > nodes).
> > > > > Node
> > > > > > > > attributes will be helpful if we will decide to implement
> > analog
> > > of
> > > > > > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate
> methods
> > > in
> > > > > the
> > > > > > > thin
> > > > > > > > client. Perhaps they can be requested lazily.
> > > > > > > >
> > > > > > > > From the protocol point of view there will be two
new
> > operations:
> > > > > > > >
> > > > > > > > OP_CLUSTER_GET_NODES
> > > > > > > > Request: empty
> > > > > > > > Response: long topologyVersion, int minorTopologyVersion,
int
> > > > > > nodesCount,
> > > > > > > > for each node set of node fields (UUID nodeId, Object
or
> String
> > > > > > > > consistentId, long order, etc)
> > > > > > > >
> > > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES
> > > > > > > > Request: int nodesCount, for each node: UUID nodeId
> > > > > > > > Response: int nodesCount, for each node: int attributesCount,
> > for
> > > > > each
> > > > > > > node
> > > > > > > > attribute: String name, Object value
> > > > > > > >
> > > > > > > > To execute tasks we need something like these methods
in the
> > > client
> > > > > > API:
> > > > > > > > Object execute(String task, Object arg)
> > > > > > > > Future<Object> executeAsync(String task, Object
arg)
> > > > > > > > Object affinityExecute(String task, String cache,
Object key,
> > > > Object
> > > > > > arg)
> > > > > > > > Future<Object> affinityExecuteAsync(String task,
String
> cache,
> > > > Object
> > > > > > > key,
> > > > > > > > Object arg)
> > > > > > > >
> > > > > > > > Which can be mapped to protocol operations:
> > > > > > > >
> > > > > > > > OP_COMPUTE_EXECUTE_TASK
> > > > > > > > Request: UUID nodeId, String taskName, Object arg
> > > > > > > > Response: Object result
> > > > > > > >
> > > > > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY
> > > > > > > > Request: String cacheName, Object key, String taskName,
> Object
> > > arg
> > > > > > > > Response: Object result
> > > > > > > >
> > > > > > > > The second operation is needed because we sometimes
can't
> > > calculate
> > > > > and
> > > > > > > > connect to affinity node on the client-side (affinity
> awareness
> > > can
> > > > > be
> > > > > > > > disabled, custom affinity function can be used or
there can
> be
> > no
> > > > > > > > connection between client and affinity node), but
we can make
> > > best
> > > > > > effort
> > > > > > > > to send request to target node if affinity awareness
is
> > enabled.
> > > > > > > >
> > > > > > > > Currently, on the server-side requests always processed
> > > > synchronously
> > > > > > and
> > > > > > > > responses are sent right after request was processed.
To
> > execute
> > > > long
> > > > > > > tasks
> > > > > > > > async we should whether change this logic or introduce
some
> > kind
> > > > > > two-way
> > > > > > > > communication between client and server (now only
one-way
> > > requests
> > > > > from
> > > > > > > > client to server are allowed).
> > > > > > > >
> > > > > > > > Two-way communication can also be useful in the future
if we
> > will
> > > > > send
> > > > > > > some
> > > > > > > > server-side generated events to clients.
> > > > > > > >
> > > > > > > > In case of two-way communication there can be new
operations
> > > > > > introduced:
> > > > > > > >
> > > > > > > > OP_COMPUTE_EXECUTE_TASK (from client to server)
> > > > > > > > Request: UUID nodeId, String taskName, Object arg
> > > > > > > > Response: long taskId
> > > > > > > >
> > > > > > > > OP_COMPUTE_TASK_FINISHED (from server to client)
> > > > > > > > Request: taskId, Object result
> > > > > > > > Response: empty
> > > > > > > >
> > > > > > > > The same for affinity requests.
> > > > > > > >
> > > > > > > > Also, we can implement not only execute task operation,
but
> > some
> > > > > other
> > > > > > > > operations from IgniteCompute (broadcast, run, call),
but it
> > will
> > > > be
> > > > > > > useful
> > > > > > > > only for java thin client. And even with java thin
client we
> > > should
> > > > > > > whether
> > > > > > > > implement peer-class-loading for thin clients (this
also
> > requires
> > > > > > two-way
> > > > > > > > client-server communication) or put classes with executed
> > > closures
> > > > to
> > > > > > the
> > > > > > > > server locally.
> > > > > > > >
> > > > > > > > What do you think about proposed protocol changes?
> > > > > > > > Do we need two-way requests between client and server?
> > > > > > > > Do we need support of compute methods other than "execute
> > task"?
> > > > > > > > What do you think about peer-class-loading for thin
clients?
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Sergey Kozlov
> > > > > > > GridGain Systems
> > > > > > > www.gridgain.com
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sergey Kozlov
> > > > > GridGain Systems
> > > > > www.gridgain.com
> > > > >
> > > >
> > >
> >
> >
> > --
> > Alex.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message