samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xinyuiscool <...@git.apache.org>
Subject [GitHub] samza pull request #297: SAMZA-1417: Clear and recreate intermediate and met...
Date Tue, 19 Sep 2017 17:07:11 GMT
GitHub user xinyuiscool opened a pull request:

    https://github.com/apache/samza/pull/297

    SAMZA-1417: Clear and recreate intermediate and metadata streams for batch processing

    For each run of a batch application, we need to clear the internal streams from the previous
run and recreate new ones. This patch introduces the following:
    1) isBatch flag to StreamSpec
    2) app.mode (BATCH/STREAM) in the application config
    3) app.runId and use it to generate the internal topics for each run.
    
    run.id generation is not addressed in this pr. There will be another patch to resolve
it for both yarn and standalone. For now, this patch only works for yarn.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/xinyuiscool/samza SAMZA-1417

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/297.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #297
    
----

----


---

Mime
View raw message