crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-271) Get the Counters for a job immediately upon Job completion
Date Sun, 29 Sep 2013 02:40:23 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Wills updated CRUNCH-271:
------------------------------

    Attachment: CRUNCH-271b.patch

And a slight improvement to the null handling in the PipelineResult class.

> Get the Counters for a job immediately upon Job completion
> ----------------------------------------------------------
>
>                 Key: CRUNCH-271
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-271
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.8.0
>
>         Attachments: CRUNCH-271b.patch, CRUNCH-271.patch
>
>
> It's possible for the Counters for a Job that runs in the early phase of a Crunch pipeline
to no longer be available in the JobTracker at the end of the last pipeline run, when we go
to ask the JT for the Counters for all of the jobs that were executed during the pipeline.
Therefore, we should cache the Counters for each job in the CrunchControlledJob object at
the time each Job completes, and then use the cached Counters in the pipeline stage reports.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message