ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vkulichenko <valentin.kuliche...@gmail.com>
Subject Re: Job scheduling in apache ignite for single node system
Date Mon, 28 Nov 2016 21:23:51 GMT
Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscribe@ignite.apache.org and follow simple instructions in
the reply.


pra527vin wrote
> We have a system to crawl a data from 400 different websites .To do this
> task we have created 400 processes for each site to crawl........when we
> run program after starting about 80 threads there is memory overflow
> .......I want to know if we can use Job scheduling mechanism of Ignite to
> schedule job on the FIFO basis for this system?if yes how should I use for
> the processes I have created by own?

I would agree with Dmitry, what is the reason of doing this in separate
processes? Why not start several Ignite nodes and use Compute Grid [1] for
scaling out the computation?

[1] https://apacheignite.readme.io/docs/compute-grid

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Job-scheduling-in-apache-ignite-for-single-node-system-tp9191p9235.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message