ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <jaroslav.s...@accenture.com>
Subject RE: ?
Date Fri, 16 Oct 2015 13:59:50 GMT
Hello Ivan

Many thanks for great news. We start to prototype ☺

From: Ivan Veselovskiy [mailto:iveselovskiy@gridgain.com]
Sent: Thursday, October 15, 2015 7:47 PM
To: user@ignite.apache.org
Subject: Re: ?

Hi, Jaroslav,
In my understanding, with Ignite the answers to all the questions are in general positive.
Possibly one of the most challenging tasks there is to adjust cache affinity function to achieve
best data/data & data/compute collocation to gain the best performance.
Documentation on affinity collocation can be found there: https://apacheignite.readme.io/docs/affinity-collocation

On Thu, Oct 15, 2015 at 7:30 PM, <jaroslav.saxa@accenture.com<mailto:jaroslav.saxa@accenture.com>>
Hello Ignite users,

Let me share my exaltation from things you’re doing.
Let me explain my problem/challenge – sorry I had not time to deeply dive into prototyping
yet and I have to make decision very quickly so I use this way to ask general  questions which
could help me to decide whether to include Ignite into short list for following prototyping
of Poc for our solution.

I really appreciate if anybody could respond  to my maybe un-competent queries (Q).

Our challenge:

•       We have files “BLOBs” which represents binary vectors.

•       We’d like to load them into Ignite data grid as distributed key values c

•       We are up to implement capability where the service [cid:image001.png@01D1082B.B8BE7300]
 (our service layer) is getting binary vector (in the same format as stored in file system
-> Ignite data grid) and we need to

o   perform “search” in our gallery which vector is most close to vector on the input.
To calculate the distance (or probability of match) between 2 vectors we have encapsulated
in external lib(jar) which basically expose 1:1 match (vector1, vector2) capability. So we
need to perform 1:1 match over all vectors we have stored. As the number of vectors in the
galley is huge and number of requests to compare is also impressive – this is tack for memory

•       What’s more, there is service [cid:image002.png@01D1082B.B8BE7300] which continuously
adds new records into our gallery

Can you please confirm that following setup/approach with Ignite would work?

•       Will we  be able to load distributed data [cid:image003.png@01D1082B.B8BE7300] from
file storage into grid nodes in distributed manner?

•       Will we be able to from service [cid:image001.png@01D1082B.B8BE7300] trigger on
every grid node computation function [cid:image004.png@01D1082B.B8BE7300]  which will :

o   Go over all data (vectors) which are distributed on the current node [cid:image005.png@01D1082B.B8BE7300]

o   Call external lib be called from computation function [cid:image004.png@01D1082B.B8BE7300]

o   Can we pass parameter to computation function [cid:image004.png@01D1082B.B8BE7300]

o   Can computation function be deployed on nodes w/o needing to broadcast it from service
(computation trigger) ?

o   Is there any support from ignite to run computation function [cid:image004.png@01D1082B.B8BE7300]
in parallel on certain node?

•       We will be able to update data on computation nodes from service [cid:image002.png@01D1082B.B8BE7300]
(add new records), so new requests from service [cid:image001.png@01D1082B.B8BE7300] will
be able to process updated gallery of vectors in memory

Thank you so MUCH


Jaroslav Saxa
Accenture – Technology Architecture
Plynarenska 7/C, 829 01 Bratislava
Mobile: +421-904-706113
Email: jaroslav.saxa@accenture.com<mailto:jaroslav.saxa@accenture.com>


This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy.


View raw message