incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davide Simoncelli <Davide.Simonce...@neclab.eu>
Subject RE: Adapters and S4 cluster
Date Tue, 12 Jun 2012 07:18:47 GMT
On Monday, June 11, 2012 06:23:13 PM Matthieu Morel wrote:

> That looks like a key mismatch, you should check key/stream/events specifications are
consistent. That could explain the event being received but not redispatched to any PE.


My configuration for the PE entry point looks like this:


<bean id="cdrCatcher" class="test.s4.EntryPointPE">

<property name="dispatcher" ref="dispatcher"/>

<property name="keys">

<list>

<value>RawEvents id</value>

</list>

</property>

</bean>


And here is the partition config:


<bean id="rawEventsPartitioner" class="io.s4.dispatcher.partitioner.DefaultPartitioner">

<property name="streamNames">

<list>

<value>RawEvents</value>

</list>

</property>

<property name="hashKey">

<list>

<value>id</value>

</list>

</property>

<property name="hasher" ref="hasher" />

<property name="debug" value="true" />

</bean>


All events on RawEvents stream should be routed to one of EntryPointPE instances depending
on the value of the key. Is there something wrong?



> I assume you are using S4 0.3, have you tried the following: http://docs.s4.io/manual/client_adapter.html#sending-events-into-s4-cluster?


I'm using version 0.3 and I tried that.

I see that bytesFromEventWrapper method implementation in GenericJsonClientStub class doesn't
serialize the 'keys' field. Then how can the adapter dispatch events depending on keys to
EntryPointPE instances?

Is that ok?


> Otherwise, you may want to follow the design pattern where keyless events are received
as input, and where keyless PEs are used to partition the input stream: http://docs.s4.io/manual/overview.html#special-types-of-pes


> Note that to achieve that with S4 piper it's quite clear: you would use a KeyFinder in
the adapter in order to identify keys and partition events when sending them to the S4 app.
Regards, Matthieu

Thank you


- Davide

Mime
View raw message