Mailing-List: contact dev-help@samza.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@samza.incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
Date: Tue, 25 Nov 2014 08:58:16 +0200
Message-ID: 
 <CA+g=GHZQA3WqA3GF6Uc2-MPrTRg2DRmeFx211wWJKU8rFoGM4g@mail.gmail.com>
Subject: Mutiple tasks running in a single container
From: Dotan Patrich <dotanp@fortscale.com>
To: dev@samza.incubator.apache.org
Content-Type: multipart/alternative; boundary=047d7b2e3d687725400508a96f45

--047d7b2e3d687725400508a96f45
Content-Type: text/plain; charset=UTF-8

Hi,

We run a topology that contains multiple tasks and plan to add more to it
in the near future. However, one of the key design issues that I
considering is how granular should each samza task should be: on the one
hand have granular tasks helps integrating them at different parts of the
topology, however on the other hand each task has it's own basic JVM memory
requirement that restrict how many tasks a machine can host.

One thing I noticed in the documentation is that each samza container can
host several tasks?
"The SamzaContainer is responsible for managing the startup, execution, and
shutdown of one or more StreamTask
<http://samza.incubator.apache.org/learn/documentation/0.7.0/api/overview.html>
 instances"

I thought this could be some sort of workaround to the memory concerns I
have (assuming cpu consumption of the streaming task will work out ok).
Can anyone share how to host several tasks in a single container? Are those
only tasks instances for different partitions or can it be different tasks
all together?

Thanks,
Dotan

--047d7b2e3d687725400508a96f45--