flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From StephanEwen <...@git.apache.org>
Subject [GitHub] flink issue #5601: [FLINK-8818][yarn/s3][tests] harden YarnFileStageTest upl...
Date Mon, 05 Mar 2018 16:52:48 GMT
Github user StephanEwen commented on the issue:

    https://github.com/apache/flink/pull/5601
  
    S3 is actually strongly consistent when reading newly created objects, just not in listing
or renaming objects (files).
    
    The test seems to actually use reads of full paths, so wondering why there is a failure
in the first place.
    
    If there is an issue that the Yarn upload code relies on eventually consistent operations,
then fixing the test by retries may disguise the actual issue. If there is no eventually consistent
operation, then this should not be necessary in the first place. I fear this change may be
down a tricky path...
    
    Can you explain/double check why the failure happened and why the retry is necessary to
stabilize the test, but the actual Yarn code is not affected by this?


---

Mime
View raw message