hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kannan Muthukkaruppan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2353) HBASE-2283 removed bulk sync optimization for multi-row puts
Date Wed, 24 Mar 2010 01:20:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848996#action_12848996
] 

Kannan Muthukkaruppan commented on HBASE-2353:
----------------------------------------------

<<< The first fix to this is to bring back deferred log flush. I have a forthcoming
patch. >>>

How are the numbers looking with the posted patch (of not syncing if deferred log flush is
set?).

<<<Given this, what makes the most sense? It seems like hlog.append() then syncFs()
of all the puts, THEN memstore mutate is the way to go. In HRS.put our protection from 'over
memory' is this call :>>>

Are you talking about how to improve things for batch puts in the default case (i.e. when
deferred log flush is not set?). Is so, can you refer to my update on <22/Mar/10 05:47
PM> -- do you plan to lock all the rows for the entire duration? Or reget the locks before
doing the memstore mutates? If you reget the row locks, I think you'll introduce other correctness
issues.






> HBASE-2283 removed bulk sync optimization for multi-row puts
> ------------------------------------------------------------
>
>                 Key: HBASE-2353
>                 URL: https://issues.apache.org/jira/browse/HBASE-2353
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: ryan rawson
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2353-deferred.txt
>
>
> previously to HBASE-2283 we used to call flush/sync once per put(Put[]) call (ie: batch
of commits).  Now we do for every row.  
> This makes bulk uploads slower if you are using WAL.  Is there an acceptable solution
to achieve both safety and performance by bulk-sync'ing puts?  Or would this not work in face
of atomic guarantees?
> discuss!

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message