incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Wordcount test case
Date Sat, 22 Oct 2011 00:45:16 GMT
Works for me now, will merge it tonight.

I have couple of things before i merge. Would be nice if i get some ideas
before i merge.

The Cluster class takes the hosts and ports when its created and uses the
number of nodes as the partitions. This is not what we will have when we
move to distributed mode.

   - We will not know all the nodes during start up.
   - Number of nodes is not number of partitions, we need a separate field
   to keep the number partitions. I added this to be part of Topology.

The Assignment.assign is called only once and the code assumes that it will
always own the partition. I think we need a slightly different mechanism
where we check with assignment every time we process a request or add a
listener (similar to TopologyChangeListener) when there is change in
partition ownership.

I haven't thought much about the communication mode at this point. I know
there is discussion about having different mode at sending and receiving or
on per stream basis. It will be difficult to get that in for this release.
Will take that in account when working on the next release of making s4 as a
release.

thanks,
Kishore G


On Fri, Oct 21, 2011 at 2:25 PM, Leo Neumeyer <leoneumeyer@gmail.com> wrote:

> Kishore, I merged Matthieu's changes, please try the latest commit.
>
> -leo
>
>
> On Fri, Oct 21, 2011 at 12:40 PM, Leo Neumeyer <leoneumeyer@gmail.com>
> wrote:
> > Yes, ignore that test case, it will be fixed once I merge Matthieu's code
> later today. Please wait until then.
> >
> > -leo
> >
> > On Oct 21, 2011, at 10:22 AM, Matthieu Morel wrote:
> >
> >> Hi Kishore,
> >>
> >> I think this is probably due to Leo's commit:
> >>
> https://github.com/leoneu/s4-piper/commit/da4cdd79d1061ab4b579641273c8f21bf94030fe
> >>
> >> I just issued a pull request for Leo, that takes my previous changes
> rebased
> >> on top of the package renaming commit. Maybe you should sync with Leo
> (i.e.
> >> wait for him to integrate the pull request), or you could sync with this
> >> branch: https://github.com/matthieumorel/s4-piper/tree/s4-12
> >>
> >> Matthieu
> >>
> >> On Fri, Oct 21, 2011 at 7:15 PM, kishore g <g.kishore@gmail.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> I am finding this test case failing in s4-piper
> >>>
> >>> read: doobie
> >>> 10:01:37.652 [words seen stream] ERROR OverloadDispatcher1801 - Cannot
> >>> dispatch event of type [io.s4.wordcount.WordSeenEvent] to PE of type
> >>> [io.s4.wordcount.WordCounterPE] : no matching onEvent method found
> >>> 10:01:37.652 [words seen stream] ERROR OverloadDispatcher1801 - Cannot
> >>> dispatch event of type [io.s4.wordcount.WordSeenEvent] to PE of type
> >>> [io.s4.wordcount.WordCounterPE] : no matching onEvent method found
> >>>
> >>> Is there something I am missing.
> >>>
> >>> thanks,
> >>> Kishore G
> >>>
> >
> >
>
>
>
> --
>
> -leo
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message