geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barry Oglesby <bogle...@pivotal.io>
Subject Re: Odg: Restart gateway-receiver
Date Fri, 22 Nov 2019 21:35:13 GMT
> If we don't send any event, the connection will be closed after some time
as connection is inactive.

Are you seeing this behavior? I don't think this is true by default.

AbstractRemoteGatewaySender.initProxy sets these fields on the PoolFactory:

pf.setReadTimeout(this.socketReadTimeout);
pf.setIdleTimeout(connectionIdleTimeOut);

By default, socketReadTimeout is 0 (no timeout), and connectionIdleTimeOut
is -1 (disabled).

Each Event Processor thread will have its own connection to the remote site:

Event Processor for GatewaySender_ny.1:
GatewaySenderEventRemoteDispatcher.initializeConnection connection=Pooled
Connection to 127.0.0.1:5452: Connection[127.0.0.1:5452]@306907760
Event Processor for GatewaySender_ny.2:
GatewaySenderEventRemoteDispatcher.initializeConnection connection=Pooled
Connection to 127.0.0.1:5452: Connection[127.0.0.1:5452]@608855137
Event Processor for GatewaySender_ny.0:
GatewaySenderEventRemoteDispatcher.initializeConnection connection=Pooled
Connection to 127.0.0.1:5452: Connection[127.0.0.1:5452]@950613560
Event Processor for GatewaySender_ny.4:
GatewaySenderEventRemoteDispatcher.initializeConnection connection=Pooled
Connection to 127.0.0.1:5452: Connection[127.0.0.1:5452]@1005378489
Event Processor for GatewaySender_ny.3:
GatewaySenderEventRemoteDispatcher.initializeConnection connection=Pooled
Connection to 127.0.0.1:5452: Connection[127.0.0.1:5452]@629246640

There will be one Event Processor thread for each dispatcher thread (5 by
default).

There aren't any good public ways to monitor the connections other than JMX.

One way to monitor these connections is with ConnectionStats on the sender
side.

You can do that with vsd: ClientStats -> GatewaySenderStats -> connections

You can also do it with code like:

private int getConnectionCount(String gatewaySenderId) {
  AbstractGatewaySender sender = (AbstractGatewaySender)
cache.getGatewaySender(gatewaySenderId);
  int totalConnections = 0;
  if (sender != null) {
    for (ConnectionStats connectionStats :
sender.getProxy().getEndpointManager().getAllStats().values()) {
      totalConnections += connectionStats.getConnections();
    }
    System.out.println("Sender=" + gatewaySenderId + "; connectionCount=" +
totalConnections);
  }
  return totalConnections;
}

You can also dump whether the dispatcher is connected like:

private void dumpConnected(String gatewaySenderId) {
  AbstractGatewaySender sender = (AbstractGatewaySender)
cache.getGatewaySender(gatewaySenderId);
  if (sender.isParallel()) {
    ConcurrentParallelGatewaySenderEventProcessor concurrentProcessor =
(ConcurrentParallelGatewaySenderEventProcessor) sender.getEventProcessor();
    for (ParallelGatewaySenderEventProcessor processor :
concurrentProcessor.getProcessors()) {
      System.out.println("Processor=" + processor + "; isConnected=" +
processor.getDispatcher().isConnectedToRemote());
    }
  } else {
    ConcurrentSerialGatewaySenderEventProcessor concurrentProcessor =
(ConcurrentSerialGatewaySenderEventProcessor) sender.getEventProcessor();
    List<SerialGatewaySenderEventProcessor> processors =
concurrentProcessor.getProcessors();
    for (SerialGatewaySenderEventProcessor processor :
concurrentProcessor.getProcessors()) {
      System.out.println("Processor=" + processor + "; isConnected=" +
processor.getDispatcher().isConnectedToRemote());
    }
  }
}

The isConnectedToRemote method does:

return connection != null && !connection.isDestroyed();

Thanks,
Barry Oglesby



On Thu, Nov 21, 2019 at 11:15 PM Mario Kevo <mario.kevo@est.tech> wrote:

> Hi,
>
> @Barry Oglesby <boglesby@pivotal.io>, thanks for the clarification.
>
> If we don't send any event, the connection will be closed after some time
> as connection is inactive.
> Is it possible to somehow monitor from the app if the replication is
> established to get information if there is a some problem with network or
> it is just closed due to inactivity?
> Can we monitor the replication on some other way than looking
> "isConnected" state on JMX?
>
> BR,
> Mario
> ------------------------------
> *Ċ alje:* Barry Oglesby <boglesby@pivotal.io>
> *Poslano:* 14. studenog 2019. 18:29
> *Prima:* dev@geode.apache.org <dev@geode.apache.org>
> *Predmet:* Re: Restart gateway-receiver
>
> Mario,
>
> Thats the current behavior. When the sender is initially started, it
> attempts to connect to the receiver. If it does not connect, it won't retry
> until there is a batch of events to send. Look for callers of
> GatewaySenderEventRemoteDispatcher.initializeConnection to see the
> behavior. It could be changed to have a task to retry lost connections, but
> generally there are events in the queue, so the connection is
> re-established pretty quickly by the event processor thread.
>
> Thanks,
> Barry Oglesby
>
>
>
> On Wed, Nov 13, 2019 at 4:55 AM Mario Kevo <mario.kevo@est.tech> wrote:
>
> > Hi geode dev,
> >
> > After creating gateways senders and receivers between two geode clusters
> > replications is established. After restart gateway receiver, sender will
> > not connect to it until we send some entry from sender to receiver.
> > Is this a normal behavior or a bug?
> > Should geode have some mechanism for checking if connection is
> established
> > no matter if entry is sent or not?
> >
> > BR,
> > Mario
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message