hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
Date Wed, 22 May 2013 20:28:16 GMT
Hi Himanshu,

I have a follow up question on these metrics. I see that both these metrics
basically diff against the timestamp of the WALEdit. The timestamp on the
WALEdit is generated by the region server in source cluster. So if there is
some clock skew (even 1 second) on the slave cluster, the
ageOfLastAppliedEdit could be somewhat off. On the other hand, I see that
ageOfLastShippedEdit would use the same clock (unless we have region server
failures) for calculating the time on the WALEdit and the time by which it
has been shipped. Also since the shipping is synchronous, we are sure that
the WALEdit has truly been applied on the sink cluster.

Is that correct ?

Varun


On Wed, May 22, 2013 at 11:55 AM, Demai Ni <nidmgg@gmail.com> wrote:

> Himanshu,
>
> Thanks a lot. Your suggestion saved me a lot of time if have to figure it
> out myself
>
> Demai on the run
>
> On May 22, 2013, at 11:00 AM, Himanshu Vashishtha <hv.csuoa@gmail.com>
> wrote:
>
> > Since the source metrics would not be available in case the master
> cluster
> > is down,
> > your best bet would be "sink.ageOfLastAppliedOp" at the slave cluster.
> > Its value is the timestamp of the last WALEdit that is applied to the
> > slave.
> >
> >
> > On Wed, May 22, 2013 at 10:02 AM, N Dm <nidmgg@gmail.com> wrote:
> >
> >> Stack and Himanshu,
> >>
> >> many thanks for putting this together.  Particularly  the info from
> >> Replication Metrics, whichs help a lot to understand the replication.
> >>
> >> A quick question on source.ageOfLastShippedOp. In the case that master
> >> cluster down, the user will like to know what the delta between master
> and
> >> slave. In another word, to find out the portion of mutation that already
> >> commit on Master, but haven't  replicated yet. Is
> source.ageOfLastShippedOp
> >> the right place to look at? will it be available even with HBASE down?
> >>
> >> Again thanks for the doc and appreciate it.
> >>
> >> Demai
> >>
> >>
> >> On Tue, May 21, 2013 at 10:39 PM, stack (JIRA) <jira@apache.org> wrote:
> >>
> >>>
> >>>     [
> >>>
> >>
> https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> ]
> >>>
> >>> stack resolved HBASE-8591.
> >>> --------------------------
> >>>
> >>>       Resolution: Fixed
> >>>    Fix Version/s: 0.98.0
> >>>     Hadoop Flags: Reviewed
> >>>
> >>> Committed to trunk.  Thanks Himanshu (And Chris for review)
> >>>
> >>>> Doc Improvement: Replication blog
> >>>> ---------------------------------
> >>>>
> >>>>                Key: HBASE-8591
> >>>>                URL: https://issues.apache.org/jira/browse/HBASE-8591
> >>>>            Project: HBase
> >>>>         Issue Type: Task
> >>>>         Components: documentation, Replication
> >>>>   Affects Versions: 0.95.0
> >>>>           Reporter: Himanshu Vashishtha
> >>>>           Assignee: Himanshu Vashishtha
> >>>>            Fix For: 0.98.0
> >>>>
> >>>>        Attachments: HBase-8591.patch, HBase-8591-v1.patch
> >>>>
> >>>>
> >>>> Add a section for source level metrics and some truth about table
> batch
> >>> ops at sink.
> >>>
> >>> --
> >>> This message is automatically generated by JIRA.
> >>> If you think it was sent incorrectly, please contact your JIRA
> >>> administrators
> >>> For more information on JIRA, see:
> >> http://www.atlassian.com/software/jira
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message