storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Perrot <gper...@ubikod.com>
Subject Re: spout instance id - can be used for partitioning?
Date Thu, 09 Jan 2014 16:47:39 GMT
Hi, yes you can do that, DRPCSpout uses the following code from the
open(Map,TopologyContext,SpoutOutputCollector) method:

int numTasks =
context.getComponentTasks(context.getThisComponentId()).size();  // number
of this spout tasks
int index = context.getThisTaskIndex();
                            // this task index

2014/1/9 Alexander S. Klimov <alexklim@microsoft.com>

>  Hi guys,
>
>
>
> From a given topic in Kafka I can read messages from multiple partitions.
> Let’s say we have 100 partitions.
>
>
>
> Can this load be evenly distributed across 10 spouts reading from Kafka.
> If spout instance has persistent id – I can use hash function to understand
> which part of partitions should given spout instance read.
>
>
>
> Do we have spout instance id notion accessible in Storm Java API? Even
> simple number from 1 to 10 would work.
>
>
>
> Thanks,
> Alex
>

Mime
View raw message