hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1003) Proposal to batch commits to edits log.
Date Mon, 04 Jun 2007 17:37:28 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501291
] 

Raghu Angadi commented on HADOOP-1003:
--------------------------------------

No. Oviously, locking around FlushAndSync to protect around two other small variables does
not make sense.
We should lock around FlushAndSync for a good reason, I think.

If you think lock around FlushAndSyc does not block anything then fine. If the lock is only
to protect those variables, then you can just lock around them. FlushAndSync() is obviosly
costlier, which is what this Jira wants to avoid (of course the current patch already reduces
that overhead).

I am not saying it is wrong. SInce I only looked at the patch briefly, I want to know why
locking is done certain way.


> Proposal to batch commits to edits log.
> ---------------------------------------
>
>                 Key: HADOOP-1003
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1003
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Raghu Angadi
>            Assignee: dhruba borthakur
>         Attachments: editLogSync.patch
>
>
> Right now most expensive namenode operations are that require commits to edits log. e.g.
creating a file, deleting, renaming etc. Most of the time is spent in fsync() of edits file
(multiple fsync() calls in the case of multiple image directories). During this time whole
namesystem is under lock and even non-mutating operations like open() are blocked.
> On a local filesystem, each fsync could take in the order of milliseconds. My understanding
is that guarantee namenode provides is that edits log is synced before replying to the client.
Without any changes to  current locking structure, I was thinking of the following for batching
multiple edits : 
>      a) a facility in RPC Server to postpone responding to a particular call (communication
with ThreadLocals may be). This is strictly not required but without it, number operations
batched would be limited to number of IPC threads.
>      b) Another Server thread that waits for pending commits to be synced and replies
back to clients. 
>      c)  fsync manager that periodically syncs the edit log and informs waiting RPCs.
The sync thread can dynamically decide to wait longer or shorter based on the load so that
we don't increase the latency when namenode is lightly loaded. Event simple policy of 'sync
if there are any mutations' will also work but that might reduce the hard disk life.
>  
> All the synchronization between these threads is a bit complicated but it can be stable.
My main concern is whether the guarantee we are providing enough for namenode operation. I
think it is enough.  
> In terms of throughput, number of creates a namenode can do should be on the same range
as number of opens it can do.
>  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message