ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vkulichenko <valentin.kuliche...@gmail.com>
Subject Re: Implementing a distributed crawler
Date Wed, 03 Aug 2016 03:13:05 GMT
As I already mentioned, I would use compute grid for this. As for the back
pressure, you can limit the number of jobs executed at the same time and
define different scheduling strategies [1]. The queue just looks like an
unnecessary piece here.

[1] https://apacheignite.readme.io/docs/job-scheduling

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Implementing-a-distributed-crawler-tp6654p6689.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message