samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dotan Patrich <dot...@fortscale.com>
Subject Mutiple tasks running in a single container
Date Tue, 25 Nov 2014 06:58:16 GMT
Hi,

We run a topology that contains multiple tasks and plan to add more to it
in the near future. However, one of the key design issues that I
considering is how granular should each samza task should be: on the one
hand have granular tasks helps integrating them at different parts of the
topology, however on the other hand each task has it's own basic JVM memory
requirement that restrict how many tasks a machine can host.

One thing I noticed in the documentation is that each samza container can
host several tasks?
"The SamzaContainer is responsible for managing the startup, execution, and
shutdown of one or more StreamTask
<http://samza.incubator.apache.org/learn/documentation/0.7.0/api/overview.html>
 instances"

I thought this could be some sort of workaround to the memory concerns I
have (assuming cpu consumption of the streaming task will work out ok).
Can anyone share how to host several tasks in a single container? Are those
only tasks instances for different partitions or can it be different tasks
all together?

Thanks,
Dotan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message