hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4716) TimelineClient to implement Flushable; propagate to writer
Date Mon, 22 Feb 2016 13:39:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15156929#comment-15156929
] 

Junping Du commented on YARN-4716:
----------------------------------

+1 to add a flush() API to TimelineClient especially we add a cache layer in ATS v1.5. 
CC [~gtCarrera9].

> TimelineClient to implement Flushable; propagate to writer
> ----------------------------------------------------------
>
>                 Key: YARN-4716
>                 URL: https://issues.apache.org/jira/browse/YARN-4716
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
>
> I need a {{flush()}} operation in the timeline client.  Knowing the lifecycle of my app,
I do explicitly want to flush up events at certain points (app start event, app end, without
relying on an async flush client)
> Right now, in tests, the time it takes for an event to propagate is: flush delay + scan
delay, you are looking at 2 seconds minimum, per test case, with less deterministic outcomes.
> In production, those big app lifecycle events are so important my client code currently
explicitly flushes my own event queue, and expects them to reach the destination
> With the filesystem writer, I've lost those durability guarantees.
> Implementing {{Flushable.flush()}} would let my app tell the timeline client to write;
it'd be a no-op on the http client, but for the filesystem writer (or any other async writer),
it'd be expected to force a write to the durable medium.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message