hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4716) TimelineClient to implement Flushable; propagate to writer
Date Mon, 22 Feb 2016 12:33:18 GMT

     [ https://issues.apache.org/jira/browse/YARN-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated YARN-4716:
    Target Version/s: 2.8.0

> TimelineClient to implement Flushable; propagate to writer
> ----------------------------------------------------------
>                 Key: YARN-4716
>                 URL: https://issues.apache.org/jira/browse/YARN-4716
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
> I need a {{flush()}} operation in the timeline client.  Knowing the lifecycle of my app,
I do explicitly want to flush up events at certain points (app start event, app end, without
relying on an async flush client)
> Right now, in tests, the time it takes for an event to propagate is: flush delay + scan
delay, you are looking at 2 seconds minimum, per test case, with less deterministic outcomes.
> In production, those big app lifecycle events are so important my client code currently
explicitly flushes my own event queue, and expects them to reach the destination
> With the filesystem writer, I've lost those durability guarantees.
> Implementing {{Flushable.flush()}} would let my app tell the timeline client to write;
it'd be a no-op on the http client, but for the filesystem writer (or any other async writer),
it'd be expected to force a write to the durable medium.

This message was sent by Atlassian JIRA

View raw message