ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakov Zhdanov <yzhda...@apache.org>
Subject Re: Remote job execution in C++
Date Mon, 26 Oct 2015 09:16:29 GMT
I think suggested approach should work.

Any idea on how to add dynamic jobs loading (I would like it is added in
one of nearest releases)? I.e. I copy new library to each server then load
it with Ignite API and have new jobs available without restarting cluster.

--Yakov

2015-10-20 18:46 GMT+03:00 Igor Sapego <isapego@gridgain.com>:

> Hello Guys,
>
> Recently I've been working on the IGNITE-1680: CPP: Implement basic API for
> remote job execution https://issues.apache.org/jira/browse/IGNITE-1680.
>
> I would like to discuss the approach I came up with.
>
> In my current approach I use existing portable mechanism to serialize the
> job object so we can send it to the remote node. However in C++ unlike in
> Java or C# we do not have any out-of-box way to instantiate objects of type
> which is unknown on the compile time. Still we have some runtime type
> information provided by the portable layer (e.g. we have type id). So we
> need to implement solution to deserialize and invoke a job object and
> serialize result of this invocation. To put it simple we are going to need
> something like this:
>
> bool InvokeRemoteJob(int32_t jobTypeId,
>                      PortableReaderImpl& jobReader,
>                      PortableWriterImpl& resultWriter);
>
> As we can not operate with type during runtime in C++ it is proposed to
> make user to provide us with such kind of function. We can define three
> simple macros so user can to provide us with the list of the jobs like
> this:
>
> IGNITE_REMOTE_JOB_LIST_BEGIN
> IGNITE_REMOTE_JOB_DECLARE(JobType1, ReturnType1)
> IGNITE_REMOTE_JOB_DECLARE(JobType2, ReturnType2)
> ...
> IGNITE_REMOTE_JOB_DECLARE(JobTypeN, ReturnTypeN)
> IGNITE_REMOTE_JOB_LIST_END
>
> This will expand to something like that:
>
> extern "C" IGNITE_IMPORT_EXPORT
> bool InvokeRemoteJob(int32_t jobTypeId,
> PortableReaderImpl& jobReader,
> PortableWriterImpl& resultWriter)
> {
> if (jobTypeId == PortableType<JobType1>::GetTypeId())
> {
> JobType1 job(jobReader.ReadTopObject<JobType1>());
> ReturnType1 call_res = job.call();
> resultWriter.WriteTopObject(call_res);
> return true;
> }
> ...
> return false;
> }
>
> Every module (dynamic library or executable) that has jobs that could be
> invoked remotely is going to need to have list of the remote jobs defined.
> Then while we are loading modules we can lookup InvokeRemoteJob function
> for every module and store them so we can invoke jobs later.
>
> I have implemented proof of concept for this approach and proved it
> working. Now I would like to hear what do you people think about it so
> please provide your comments if any.
>
> Thanks
>
> Best Regards,
> Igor
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message