samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navina Ramesh" <nram...@linkedin.com>
Subject Re: Review Request 35241: refactoring the code for coordinator stream writer
Date Wed, 17 Jun 2015 00:25:11 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35241/#review88174
-----------------------------------------------------------



samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamWriter.java
(line 41)
<https://reviews.apache.org/r/35241/#comment140582>

    Can you add a .sh wrapper using run-job to actually run the job from command line?
    Take a look at samza-shell/src/main/bash/checkpoint-tool.sh. You can follow a similar
pattern to run the CoordinatorStreamWriter class.



samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamWriter.java
(line 61)
<https://reviews.apache.org/r/35241/#comment140581>

    Do we really need this check? There are no other components starting the same write thread.



samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamWriter.java
(line 125)
<https://reviews.apache.org/r/35241/#comment140580>

    I was thinking about how a continuous input is more useful than a one-time command. I
think it is safer to expose / allow writing only 1 config change at a time. 
    This will make input validation simpler and also, avoid the job-coordinator to react to
all config changes at the same time. 
    
    Can you change this to input only 1 config key/value pair at a time ?



samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamSystemFactory.java
(line 60)
<https://reviews.apache.org/r/35241/#comment140583>

    nit: spacing


- Navina Ramesh


On June 9, 2015, 11:53 p.m., Shadi A. Noghabi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35241/
> -----------------------------------------------------------
> 
> (Updated June 9, 2015, 11:53 p.m.)
> 
> 
> Review request for samza, Yi Pan (Data Infrastructure), Navina Ramesh, and Naveen Somasundaram.
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> In order to be able to change configurations while a job is running, a tool for writing
a message to the coordinator stream is needed. This code targets creating such a tool that
can write messages to the coordinator stream after the bootstrap of the job. This code is
related to the SAMZA-704 JIRA.
> 
> To run the code use the folowing command:
> 
> <path to samza deployment>/bin/run-class.sh org.apache.samza.coordinator.stream.CoordinatorStreamWriter
--config-factory=<config factory> --config-path=<path to config file of a job>
> 
> 
> Diffs
> -----
> 
>   checkstyle/import-control.xml 3374f0c432e61ac4cda275377604cfd481f0cddf 
>   samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamMessage.java
6c1e488d00d8593d59c89b57e673e0b6b90fd7d2 
>   samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamWriter.java
PRE-CREATION 
>   samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamSystemFactory.java
647cadb3a4e51bec8204197d77ad35a6b29afcec 
>   samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamSystemProducer.java
68e32554c18f443565284b807f43f4a5b05afbce 
>   samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamWriter.java
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/35241/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Shadi A. Noghabi
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message