samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navina Ramesh <nram...@linkedin.com>
Subject Re: Review Request 45601: SAMZA-886 document rack awareness impact on host affinity
Date Tue, 12 Apr 2016 20:40:55 GMT


> On April 12, 2016, 6:49 p.m., Boris Shkolnik wrote:
> > docs/learn/documentation/versioned/yarn/yarn-host-affinity.md, line 106
> > <https://reviews.apache.org/r/45601/diff/1/?file=1322576#file1322576line106>
> >
> >     Please expand 'mapping'. It is not clear from the context without reading the
SAMZA-886.
> 
> Jake Maes wrote:
>     I don't want to duplicate the hadoop documentation that is linked in the first sentence,
so can you please specify what is unclear?
>     
>     Would it help if I said "DNSToSwitchMapping"?
> 
> Navina Ramesh wrote:
>     I think the gap is due to the fact that we don't have any documentation on tuning
Rack Awarness for Yarn in the Samza website. So, those who are new to Samza will have a tough
time digging out the information from the Yarn documentation.
>     1. You can replace "mapping" with "DNSToSwitchMapping" and also, explain what info
this mapping maintains, what will happen if the mapping is not configured etc.
>     2. The documentation should also clearly indicate whether RackAwarness is a pre-requisite
for correct behavior of host-affinity. 
>     
>     Can you please incorporate the above points? Thanks!
> 
> Jake Maes wrote:
>     > I think the gap is due to the fact that we don't have any documentation on tuning
Rack Awarness for Yarn in the Samza website.
>     Why should we? Rack awareness currently only impacts Samza if it's broken. So, I'm
not sure what "tuning" would entail. Should we similarly document every other config that
has no impact on Samza?
>     
>     > You can replace "mapping" with "DNSToSwitchMapping" and also, explain what info
this mapping maintains, what will happen if the mapping is not configured etc.
>     I'll expand on how the mapping impacts Samza, but I don't think it makes sense to
restate the hadoop documentation that explains the property. We are not asking users to configure
it, we are asking that if they do, they make sure it's robust. If they choose to implement
it, it is reasonable to assume that they are familiar with the feature. 
>     
>     > The documentation should also clearly indicate whether RackAwarness is a pre-requisite
for correct behavior of host-affinity.
>     Fair enough.

Yarn for Samza definitely deserves a page on the Samza website that we still have on our backlog.
That said, I don't imply we duplicate docs from other websites. It willbe a nightmare keeping
track of the changes. 

I think we both agree that we have to document how a config/feature on a dependent component
impact Samza.


- Navina


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45601/#review128501
-----------------------------------------------------------


On April 1, 2016, 7:49 p.m., Jake Maes wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45601/
> -----------------------------------------------------------
> 
> (Updated April 1, 2016, 7:49 p.m.)
> 
> 
> Review request for samza, Navina Ramesh, Jagadish Venkatraman, and Yi Pan (Data Infrastructure).
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> SAMZA-886 document rack awareness impact on host affinity
> 
> 
> Diffs
> -----
> 
>   docs/learn/documentation/versioned/yarn/yarn-host-affinity.md 1d9c29e7ee7fd19fabba55111b5088cc5d36f1fe

> 
> Diff: https://reviews.apache.org/r/45601/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jake Maes
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message