Return-Path: 
 <mapreduce-issues-return-1225-apmail-hadoop-mapreduce-issues-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org
Received: (qmail 16164 invoked from network); 23 Jul 2009 10:30:33 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 23 Jul 2009 10:30:33 -0000
Received: (qmail 93317 invoked by uid 500); 23 Jul 2009 10:31:39 -0000
Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org
Received: (qmail 93269 invoked by uid 500); 23 Jul 2009 10:31:39 -0000
Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-issues@hadoop.apache.org>
List-Id: <mapreduce-issues.hadoop.apache.org>
Reply-To: mapreduce-issues@hadoop.apache.org
Delivered-To: mailing list mapreduce-issues@hadoop.apache.org
Received: (qmail 93244 invoked by uid 99); 23 Jul 2009 10:31:39 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jul 2009 10:31:39 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=10.0
	tests=ALL_TRUSTED
X-Spam-Check-By: apache.org
Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jul 2009 10:31:35 +0000
Received: from brutus (localhost [127.0.0.1])
	by brutus.apache.org (Postfix) with ESMTP id D6856234C004
	for <mapreduce-issues@hadoop.apache.org>;
 Thu, 23 Jul 2009 03:31:14 -0700 (PDT)
Message-ID: <1390072837.1248345074842.JavaMail.jira@brutus>
Date: Thu, 23 Jul 2009 03:31:14 -0700 (PDT)
From: "Sharad Agarwal (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-786) Jobtracker history should be
 written aysnchronously to the filesystem
In-Reply-To: <606275945.1248254774798.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
X-Virus-Checked: Checked by ClamAV on apache.org


    [ https://issues.apache.org/jira/browse/MAPREDUCE-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734542#action_12734542 ] 

Sharad Agarwal commented on MAPREDUCE-786:
------------------------------------------

I think we should cleanup the way history is written. Providing cleaner API to plugin the async. Perhaps a separate JIRA for it. Something like:
{code}
public interface Event {
  public Map<String, String> getData();
}

public interface EventReader {
   public Iterator<Event> read();
   public void close();
}

public interface EventWriter {
  public void write(Event event);
  public void flush();
  public void close();
}
{code}

Async writer should be able to wrap to an EventWriter object:
{code}
class AsyncEventWriter implements EventWriter {
  AsyncEventWriter(EventWriter writer){
  }
}
{code}

> Jobtracker history should be written aysnchronously to the filesystem
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-786
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-786
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>
> Jobtracker lock is held while writing the history events. This makes the jobtracker slow on flushes, especially when history is written to HDFS. History events should be written asynchronously to avoid this problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.