Mailing-List: contact user-help@flume.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flume.apache.org
MIME-Version: 1.0
In-Reply-To: <CAMQCvxqOMPSAh_ukKr9NLMYU+-2bsaiBVUv=TqLKkTC+j=eE=Q@mail.gmail.com>
References: <CAMQCvxqOMPSAh_ukKr9NLMYU+-2bsaiBVUv=TqLKkTC+j=eE=Q@mail.gmail.com>
From: "Marco B." <marcusbu@gmail.com>
Date: Tue, 26 Jul 2016 12:05:27 +0200
Message-ID: <CAMQCvxrnT0=1vuCb=ahGGC+a12aqS2nTsrnpdR=gHcut6+SS1w@mail.gmail.com>
Subject: Re: issue reading from Tibco EMS queue - pendingMsgs too high
To: user@flume.apache.org
Content-Type: multipart/alternative; boundary=94eb2c0554184aac610538870a04
archived-at: Tue, 26 Jul 2016 10:05:34 -0000

--94eb2c0554184aac610538870a04
Content-Type: text/plain; charset=UTF-8

Hello,

I would like to confirm that the problem was solved when I set
AUTO_ACKNOWLEDGE in the session - I made a mistake, but I have just found
out that it was due to something else.

Hopefully this will help someone else in the future.

Kind regards,
Marco

2016-07-22 16:27 GMT+02:00 Marco B. <marcusbu@gmail.com>:

> Hello everyone,
>
> I have configured Flume to read messages from a Tibco EMS queue with the
> following configuration:
>
> a.sources.jms.type = jms
> a.sources.jms.initialContextFactory =
> com.tibco.tibjms.naming.TibjmsInitialContextFactory
> a.sources.jms.connectionFactory = QueueConnectionFactory
> a.sources.jms.providerURL = tcp://host:7222
> a.sources.jms.destinationName = <queue_name>
> a.sources.jms.destinationType = queue
> a.sources.jms.channels = mem-ch
>
> This seems to work pretty fine, except that the Tibco EMS queue has a very
> high number of pending messages. My understanding is that no ACKs are sent
> back, and at a first glance this hypothesis has seemed to be confirmed by
> the fact that in the file
>
> https://github.com/apache/flume/blob/trunk/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSMessageConsumer.java
>
> the session is created as follows
>
> session = connection.createSession(true, Session.SESSION_TRANSACTED);
>
> whereas in this specific case it should be AUTO_ACKNOWLEDGE. However,
> after changing the flume-jms-source.jar file and doing a simple experiment
> with a small bunch of messages, I have got the the same results:
> pendingMsgs is equal to the amount of messages actually sent - I say
> actually, because Flume has successfully consumed the messages, but Tibco
> doesn't see that. So, in the end I have got same situation as before.
>
> Does anybody know where I should have a look at? I don't have access to
> Tibco EMS settings - as this is part of another system - but I may ask the
> admin to provide me some settings, if necessary.
>
> I would really appreciate your help.
>
> Thanks a lot in advance.
>
> Kind regards,
> Marco
>

--94eb2c0554184aac610538870a04
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div>Hello,<br><br></div>I would like to co=
nfirm that the problem was solved when I set AUTO_ACKNOWLEDGE in the sessio=
n - I made a mistake, but I have just found out that it was due to somethin=
g else.<br><br></div>Hopefully this will help someone else in the future.<b=
r><br></div>Kind regards,<br></div>Marco<br></div><div class=3D"gmail_extra=
"><br><div class=3D"gmail_quote">2016-07-22 16:27 GMT+02:00 Marco B. <span =
dir=3D"ltr">&lt;<a href=3D"mailto:marcusbu@gmail.com" target=3D"_blank">mar=
cusbu@gmail.com</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=
=3D"ltr"><div><div><div>Hello everyone,<br><br></div>I have configured Flum=
e to read messages from a Tibco EMS queue with the following configuration:=
<br><br><div style=3D"margin-left:40px">a.sources.jms.type =3D jms<br>a.sou=
rces.jms.initialContextFactory =3D com.tibco.tibjms.naming.TibjmsInitialCon=
textFactory<br>a.sources.jms.connectionFactory =3D QueueConnectionFactory<b=
r>a.sources.jms.providerURL =3D tcp://host:7222<br>a.sources.jms.destinatio=
nName =3D &lt;queue_name&gt;<br>a.sources.jms.destinationType =3D queue<br>=
a.sources.jms.channels =3D mem-ch<br></div><br></div>This seems to work pre=
tty fine, except that the Tibco EMS queue has a very high number of pending=
 messages.  My understanding is that no ACKs are sent back, and at a first =
glance this hypothesis has seemed to be confirmed by the fact that in the f=
ile<br><a href=3D"https://github.com/apache/flume/blob/trunk/flume-ng-sourc=
es/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSMessageCon=
sumer.java" target=3D"_blank">https://github.com/apache/flume/blob/trunk/fl=
ume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/J=
MSMessageConsumer.java</a><br></div><div><br>the session is created as foll=
ows<br><br><div style=3D"margin-left:40px">      session <span>=3D</span> c=
onnection<span>.</span>createSession(<span>true</span>, <span>Session</span=
><span><span>.</span>SESSION_TRANSACTED</span>);<br></div></div><div><br></=
div><div>whereas in this specific case it should be AUTO_ACKNOWLEDGE. Howev=
er, after changing the flume-jms-source.jar file and doing a simple experim=
ent with a small bunch of messages, I have got the the same=20
results: pendingMsgs is equal to the amount of messages actually sent - I
 say actually, because Flume has successfully consumed the messages, but Ti=
bco doesn&#39;t see that. So, in the end I have got same situation as befor=
e.<br><br></div><div>Does anybody know where I should have a look at? I don=
&#39;t have access to Tibco EMS settings - as this is part of another syste=
m - but I may ask the admin to provide me some settings, if necessary.<br><=
br>I would really appreciate your help. <br><br></div><div>Thanks a lot in =
advance.<br><br></div><div>Kind regards,<br></div><div>Marco<br></div></div=
>
</blockquote></div><br></div>

--94eb2c0554184aac610538870a04--