aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zameer Manji <zma...@apache.org>
Subject Re: Review Request 51513: Add support for ETags in the Aurora API.
Date Wed, 31 Aug 2016 01:32:27 GMT


> On Aug. 30, 2016, 12:37 a.m., Stephan Erb wrote:
> > Were you able to test this on a cluster with concurrent modifications? Right now,
I am not confident this is working as most ouf our thrift responses contain unordered sequences.
I would therefore suspect that we get a different hash each time, even though the content
itself has not changed.
> > 
> > Considering it is working in some cases, do we have to do something to get some
speedups for the Aurora UI as well?
> 
> Zameer Manji wrote:
>     Could you describe what you mean by 'concurrent modifications'?
>     
>     I understand your concern that it doesn't work, but empirical interaction in vagrant
via `curl` shows the same ETag is set for the same data. I will investigate further and construct
a test case that demonstrates this.
>     
>     I'm not sure how the UI works, but yes we would need to do something to store the
ETag in a response and send that back if we want a `304` response.
> 
> Stephan Erb wrote:
>     I can imagine that whenever we modify our in-memory task store in a way triggers
a resizing of its unordered data structures, the result might be ordered differently.
>     
>     For example, when we cache a call `getTasksWithoutConfig(myQuery)` returning 100
instances, the order of instances might change if we schedule 1000 completely unrelated instances
not even matched by our query.
> 
> Zameer Manji wrote:
>     You are correct, in some cases a write unrelated to the data we are trying to retrieve
will uncessarily alter the ETag. However in cases where there has been no write the ETag is
constant.
>     
>     I guess this shows there is a flaw in this hashing method, however I'm unaware of
another method of computing an ETag without altering storage or changing how we use thrift.
>     
>     Do you think these flaws hinder this change?

Actually after some experimentation, I have eliminated most of these instances where the ETag
changes by having the API layer sort the results by the natural comparator (provided by thrift
generated code).

For example in `getTasksWithoutConfigs` if I sort the tasks, then the Etag is constant between
reboots and other writes done.

I'm willing to write e2e tests that ensure this behaviour if that's what's required.


- Zameer


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51513/#review147270
-----------------------------------------------------------


On Aug. 29, 2016, 6:12 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51513/
> -----------------------------------------------------------
> 
> (Updated Aug. 29, 2016, 6:12 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Bugs: AURORA-1757
>     https://issues.apache.org/jira/browse/AURORA-1757
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch enhances the Aurora API by producing an `ETag` header for each API
> request. It also consumes etag values in API requests via the `If-None-Match`
> header and produces a HTTP 304 if the request would produce a body with the same
> etag value.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md 1819eaa20cf5014228643a1e120316d646cc2824 
>   src/main/java/org/apache/aurora/scheduler/http/api/TContentAwareServlet.java 1634cb88ac09c778c5bb277ca902f4ca35dd6c9d

>   src/test/java/org/apache/aurora/scheduler/http/api/ApiIT.java 0a3ff05586c87e0ab2cc20470e99b5dd609f7039

> 
> Diff: https://reviews.apache.org/r/51513/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message