samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan <danharve...@gmail.com>
Subject Store changelog
Date Thu, 02 Apr 2015 17:30:16 GMT
Hi all,

We're just starting out using Samza to process streams we've already got in
Kafka. Some of the jobs we've written are using the per task KV store which
are being persisted to a changelog topic in Kafka. As you need a different
changelog topic per task we are wondering how people are dealing with
ensuring that each task's store has a separate changelog.

I think we could define multiple stores in the properties file, then pick
the correct one for each task index. But that seems quite a verbose way to
go about that?

If Samza could use a prefix in the properties file then generate a topic
name for each task it would simplify using that. Maybe there's something
I'm missing from this?

Thanks,
Dan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message