ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakov Zhdanov <yzhda...@apache.org>
Subject Re: Remote job execution in C++
Date Mon, 26 Oct 2015 15:23:57 GMT
Agree, I am talking about some future releases.

--Yakov

2015-10-26 14:06 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:

> Job load without cluster restart is advanced feature. I think we should
> make basic API work first.
>
> On Mon, Oct 26, 2015 at 12:16 PM, Yakov Zhdanov <yzhdanov@apache.org>
> wrote:
>
> > I think suggested approach should work.
> >
> > Any idea on how to add dynamic jobs loading (I would like it is added in
> > one of nearest releases)? I.e. I copy new library to each server then
> load
> > it with Ignite API and have new jobs available without restarting
> cluster.
> >
> > --Yakov
> >
> > 2015-10-20 18:46 GMT+03:00 Igor Sapego <isapego@gridgain.com>:
> >
> > > Hello Guys,
> > >
> > > Recently I've been working on the IGNITE-1680: CPP: Implement basic API
> > for
> > > remote job execution https://issues.apache.org/jira/browse/IGNITE-1680
> .
> > >
> > > I would like to discuss the approach I came up with.
> > >
> > > In my current approach I use existing portable mechanism to serialize
> the
> > > job object so we can send it to the remote node. However in C++ unlike
> in
> > > Java or C# we do not have any out-of-box way to instantiate objects of
> > type
> > > which is unknown on the compile time. Still we have some runtime type
> > > information provided by the portable layer (e.g. we have type id). So
> we
> > > need to implement solution to deserialize and invoke a job object and
> > > serialize result of this invocation. To put it simple we are going to
> > need
> > > something like this:
> > >
> > > bool InvokeRemoteJob(int32_t jobTypeId,
> > >                      PortableReaderImpl& jobReader,
> > >                      PortableWriterImpl& resultWriter);
> > >
> > > As we can not operate with type during runtime in C++ it is proposed to
> > > make user to provide us with such kind of function. We can define three
> > > simple macros so user can to provide us with the list of the jobs like
> > > this:
> > >
> > > IGNITE_REMOTE_JOB_LIST_BEGIN
> > > IGNITE_REMOTE_JOB_DECLARE(JobType1, ReturnType1)
> > > IGNITE_REMOTE_JOB_DECLARE(JobType2, ReturnType2)
> > > ...
> > > IGNITE_REMOTE_JOB_DECLARE(JobTypeN, ReturnTypeN)
> > > IGNITE_REMOTE_JOB_LIST_END
> > >
> > > This will expand to something like that:
> > >
> > > extern "C" IGNITE_IMPORT_EXPORT
> > > bool InvokeRemoteJob(int32_t jobTypeId,
> > > PortableReaderImpl& jobReader,
> > > PortableWriterImpl& resultWriter)
> > > {
> > > if (jobTypeId == PortableType<JobType1>::GetTypeId())
> > > {
> > > JobType1 job(jobReader.ReadTopObject<JobType1>());
> > > ReturnType1 call_res = job.call();
> > > resultWriter.WriteTopObject(call_res);
> > > return true;
> > > }
> > > ...
> > > return false;
> > > }
> > >
> > > Every module (dynamic library or executable) that has jobs that could
> be
> > > invoked remotely is going to need to have list of the remote jobs
> > defined.
> > > Then while we are loading modules we can lookup InvokeRemoteJob
> function
> > > for every module and store them so we can invoke jobs later.
> > >
> > > I have implemented proof of concept for this approach and proved it
> > > working. Now I would like to hear what do you people think about it so
> > > please provide your comments if any.
> > >
> > > Thanks
> > >
> > > Best Regards,
> > > Igor
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message