Mailing-List: contact activemq-dev-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-dev@geronimo.apache.org
Received-SPF: pass (asf.osuosl.org: domain of sridharkomandur@gmail.com
 designates 66.249.92.206 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:references;
        b=UZVbZjOsRMNpPJaTNcY2ZEy+eYLpy88KoZ/+QwPHJw8ZrOMkWWIGjl/vEdVoycu6czUpbhMhMSUTF2Dv6ER17YwNArRT4SCPU4K7knk+yS6kT8M3z4PrAuD7JOpNW3Y06HKYBEYSRQC5wnA5c4ujqzPYxetC8kybN9i7v9LsiSM=
Message-ID: <ae7ffdb0603081128u1670f0a0y5ff3a9f2ad825f43@mail.gmail.com>
Date: Wed, 8 Mar 2006 11:28:02 -0800
From: "Sridhar Komandur" <anon2005@komandur.com>
Sender: sridharkomandur@gmail.com
To: activemq-dev@geronimo.apache.org
Subject: Re: improve master/slave topology
In-Reply-To: 
 <4CAF5EC6A35347429158CE18A07247F309896CB6@exch-van01.intl.businessobjects.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_4219_28621592.1141846082842"
References: 
 <4CAF5EC6A35347429158CE18A07247F309896CB6@exch-van01.intl.businessobjects.com>

------=_Part_4219_28621592.1141846082842
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On 3/8/06, Ning Li <Ning.Li@businessobjects.com> wrote:
>
> Bulk synch is a good idea, I think we can find a way to do it in current
> system, like create a topic and every message comes in will be sent to
> that topic, when the secondary comes up, it can pull those messages. Or
> we can find other ways to do it.


Yes, an internally created (persisted) queue at the primary
to store stuff when the secondary is not in sight.  When the secondary come=
s
up
it drains from that subject ? Sounds like a good idea to me.


One difficulty is we cannot pause the primary broker, it is hard for the
> secondary to catch up with both the historic and ongoing messages, I
> think there is a timing issue in it. I guess that is why James
> recommended pausing the primary broker.
>
> I am not sure if we can find a way to do both dynamic synch and bulk
> synch at the same time in the current system that will be great.


It can be done - we need a notion of ordering among all the messages (comin=
g
from both dynamic as well as bulk synch). This ordering can be provided by
the message arrival  time stamp  at the primary.

Once we do this it is a matter of inserting the incoming messages (without
worrying about the source) to the same target store. We can even have the
bulk synch proceed in a lazy fashion - a background task at the primary (an=
d
possibly at the secondary) for a couple of reasons:
 - latest messages are more relevant/important
-  latest messages could in fact be retransmissions of the old, so it is ok
to process the old messeges later for recovery purposes

Regards
- Sridhar

Thanks.
>
> Ning
> -----Original Message-----
> From: sridharkomandur@gmail.com [mailto:sridharkomandur@gmail.com] On
> Behalf Of Sridhar Komandur
> Sent: Wednesday, March 08, 2006 9:59 AM
> To: activemq-dev@geronimo.apache.org
> Subject: Re: improve master/slave topology
>
> I like the idea of broker-broker synchronization. One of the  issues to
> resolve is how reliable this synch activity needs to be ? A
> transactional
> approach is too heavy weight for the common case.
>
> I think a middle ground based on TCP may be good enough. We can divide
> the
> synchronization into two phases:
> - dynamic synch : messages are sent to the partner on an ongoing basis
> - bulk synch: a new secondary comes up and its state needs to be brought
> up
> to par with primary
>
> Thanks
> Regards
> - Sridhar
>
> On 3/6/06, Ning Li <Ning.Li@businessobjects.com> wrote:
> >
> > Hi,
> >
> > This is a continued discussion about dynamically reintroduce the
> master
> > after a failure, the original discussion is here.
> >
> > http://forums.activemq.org/posts/list/468.page#1653
> >
> > James idea about pausing the slave and synchronize two DBs is better
> > than stopping the slave and doing a manual sync. But I doubt this is
> > acceptable to us, as in real production environment, we won't be able
> to
> > pause the only message broker unless for a really short interval (I
> > guess have to less than one minute otherwise the end user will notice
> > it).
> >
> > Maybe a broker-broker synchronization protocol is the ultimate
> solution,
> > just we are not sure how to get there. Any recommendation or
> > suggestions?
> >
> >
> > Thanks
> >
> > Ning
> >
>

------=_Part_4219_28621592.1141846082842--