ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Remote job execution in C++
Date Mon, 26 Oct 2015 11:06:59 GMT
Job load without cluster restart is advanced feature. I think we should
make basic API work first.

On Mon, Oct 26, 2015 at 12:16 PM, Yakov Zhdanov <yzhdanov@apache.org> wrote:

> I think suggested approach should work.
>
> Any idea on how to add dynamic jobs loading (I would like it is added in
> one of nearest releases)? I.e. I copy new library to each server then load
> it with Ignite API and have new jobs available without restarting cluster.
>
> --Yakov
>
> 2015-10-20 18:46 GMT+03:00 Igor Sapego <isapego@gridgain.com>:
>
> > Hello Guys,
> >
> > Recently I've been working on the IGNITE-1680: CPP: Implement basic API
> for
> > remote job execution https://issues.apache.org/jira/browse/IGNITE-1680.
> >
> > I would like to discuss the approach I came up with.
> >
> > In my current approach I use existing portable mechanism to serialize the
> > job object so we can send it to the remote node. However in C++ unlike in
> > Java or C# we do not have any out-of-box way to instantiate objects of
> type
> > which is unknown on the compile time. Still we have some runtime type
> > information provided by the portable layer (e.g. we have type id). So we
> > need to implement solution to deserialize and invoke a job object and
> > serialize result of this invocation. To put it simple we are going to
> need
> > something like this:
> >
> > bool InvokeRemoteJob(int32_t jobTypeId,
> >                      PortableReaderImpl& jobReader,
> >                      PortableWriterImpl& resultWriter);
> >
> > As we can not operate with type during runtime in C++ it is proposed to
> > make user to provide us with such kind of function. We can define three
> > simple macros so user can to provide us with the list of the jobs like
> > this:
> >
> > IGNITE_REMOTE_JOB_LIST_BEGIN
> > IGNITE_REMOTE_JOB_DECLARE(JobType1, ReturnType1)
> > IGNITE_REMOTE_JOB_DECLARE(JobType2, ReturnType2)
> > ...
> > IGNITE_REMOTE_JOB_DECLARE(JobTypeN, ReturnTypeN)
> > IGNITE_REMOTE_JOB_LIST_END
> >
> > This will expand to something like that:
> >
> > extern "C" IGNITE_IMPORT_EXPORT
> > bool InvokeRemoteJob(int32_t jobTypeId,
> > PortableReaderImpl& jobReader,
> > PortableWriterImpl& resultWriter)
> > {
> > if (jobTypeId == PortableType<JobType1>::GetTypeId())
> > {
> > JobType1 job(jobReader.ReadTopObject<JobType1>());
> > ReturnType1 call_res = job.call();
> > resultWriter.WriteTopObject(call_res);
> > return true;
> > }
> > ...
> > return false;
> > }
> >
> > Every module (dynamic library or executable) that has jobs that could be
> > invoked remotely is going to need to have list of the remote jobs
> defined.
> > Then while we are loading modules we can lookup InvokeRemoteJob function
> > for every module and store them so we can invoke jobs later.
> >
> > I have implemented proof of concept for this approach and proved it
> > working. Now I would like to hear what do you people think about it so
> > please provide your comments if any.
> >
> > Thanks
> >
> > Best Regards,
> > Igor
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message