incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Algermissen <jan.algermis...@nordsc.com>
Subject Re: Using C* and CAS to coordinate workers
Date Fri, 04 Apr 2014 16:48:44 GMT
Hi DuyHai,


On 04 Apr 2014, at 13:58, DuyHai Doan <doanduyhai@gmail.com> wrote:

> @Jan
> 
>  This subject of distributed workers & queues has been discussed in the mailing list
many times.

Sorry + thanks.

Unfortunately, I do not want to use C* as a queue, but to coordinate workers that page through
an (XML) data feed of events every N seconds.

Let me try again:

- I have N instances of the same system, replicated to ensure work is
  being done despite failure of instances
- the instances are master less and know nothing about each other. Given
  them an integer ID isn’r really possible and the number of instances isn’t
  really known
- there is a schedule, controlling how often the feed is read, say once every Minute
- the schedule might change by way of an administrator of the ‘feed polling’
- the worker instances check for work every, e.g. 10 secs
- once a worker starts, it checks whether there is work to do (the schedule aspect) and if
so,
  starts polling the feed until the last event has been reached.
- During that time, no other worker must poll the feed
- once the working worker is done it saves the timestamp or ID of the last seen event and
sets the next schedule

- the processing of the events might take much longer than the schedule intervals

I hope this explains more, what I am up to. Maybe I can adapt your suggestion, I just do not
see how.

Jan



> Basically one implementation can be:
> 
> 1) p data providers, c data consumers
> 2) create partitions (physical rows) of arbitrary number of columns (let's say 10 000,
not too big though). Partition key = bucket number (#b)
> 3) assign an integer id (pId) to each provider, same for each consumer (cId)
> 4) each provider can only write messages in bucket number such that #b mod p = pId mod
p
> 5) once the provider reaches 10 000 messages per bucket, it switches to the next one
with new #b = old #b + p
> 6) the consumers follow the same rule for bucket switching
> 
> Example:
> 
>  p = 5, c = 3
> 
>  - p1 writes messages into buckets {1,6,11,16...} // 1, 1+5, 1+5+5, ....
>  - p2 writes messages into buckets {2,7,12,17...} // 2, 2+5, 2+5+5,...
>  - p3 writes messages into buckets {3,8,13,18...}
>  - p4 writes messages into buckets {4,9,14,19...}
>  - p5 writes messages into buckets {5,10,15,20...} 
> 
>  - c1 consumes messages from buckets {1,4,7,10...} // 1, 1+3, 1+3+3...
>  - c2 consumes messages from buckets {2,5,8,11...}
>  - c1 consumes messages from buckets {3,6,9,12...}
> 
> Of course, consumers can not re-put messages into the bucket otherwise the counting (10
000 elements/bucket) is screwed
> 
> Alternatively, you can insert messages with TTL to automatically expired "consumed buckets"
after a while, saving you the hassle to clean up old buckets to reclaim disk space.
> 
> 
>  There are other implementations based on distributed lock using C* C.A.S also but the
above algorithm do not requires any lock.
> 
> Regards
> 
>  Duy Hai DOAN
> 
> 
> 
> 
> 
> On Fri, Apr 4, 2014 at 12:47 PM, prem yadav <ipremyadav@gmail.com> wrote:
> Oh ok. I thought you did not have a cassandra cluster already. Sorry about that.
> 
> 
> On Fri, Apr 4, 2014 at 11:42 AM, Jan Algermissen <jan.algermissen@nordsc.com> wrote:
> 
> On 04 Apr 2014, at 11:18, prem yadav <ipremyadav@gmail.com> wrote:
> 
>> Though cassandra can work but to me it looks like you could use a persistent queue
for example (rabbitMQ) to implement this. All your workers can subscribe to a queue.
>> In fact, why not just MySQL?
> 
> Hey, I have got a C* cluster that can (potentially) do CAS.
> 
> Why would I set up a MySQL cluster to solve that problem?
> 
> And yeah, I could use a queue or redis or whatnot, but I want to avoid yet another moving
part :-)
> 
> Jan
> 
> 
>> 
>> 
>> On Thu, Apr 3, 2014 at 11:44 PM, Jan Algermissen <jan.algermissen@nordsc.com>
wrote:
>> Hi,
>> 
>> maybe someone knows a nice solution to the following problem:
>> 
>> I have N worker processes that are intentionally masterless and do not know about
each other - they are stateless and independent instances of a given service system.
>> 
>> These workers need to poll an event feed, say about every 10 seconds and persist
a state after processing the polled events so the next worker knows where to continue processing
events.
>> 
>> I would like to use C*’s CAS feature to coordinate the workers and protect the
shared state (a row or cell in  a C* key space, too).
>> 
>> Has anybody done something similar and can suggest a ‘clever’ data model design
and interaction?
>> 
>> 
>> 
>> Jan
>> 
> 
> 
> 


Mime
View raw message