uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mihaela M <mmihaela1...@yahoo.com>
Subject Re: uima-as 2.3.1 - java.io.IOException: Frame size of 147 MB larger than max allowed 100 MB
Date Thu, 13 Feb 2014 12:33:01 GMT
I don't know why it doesn't work because I have explicitly set the brokerUrl to the deployment
descriptors, ran the UIMA_Service with argument -defaultBrokerURL and set also the system
property -DdefaultBrokerURL. All were set with value: tcp://127.0.0.1:61616?wireFormat.maxInactivityDuration=0&wireFormat.maxFrameSize=209715200&jms.useCompression=true.

The pipeline has the following components: one main aggregate with a remote RDF Consumer and
one remote aggregate. The remote aggregate has one remote primitive annotator and one local
primitive annotator. 
I don't know exactly how many connections are created for the services to communicate with
each other but in the log file of each of the components I noticed that there is ONLY ONE
connection which is built with default frame size of 100 MB instead of 200 MB as the others.

In the aggregators I see a line like the following:

02/13/2014 05:36:37.081 [DEBUG] [main] org.apache.activemq.transport.WireFormatNegotiator
- Sending: WireFormatInfo { version=9, properties={MaxFrameSize=104857600, CacheSize=1024,
CacheEnabled=true, SizePrefixDisabled=false, MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
02/13/2014 05:36:37.082 [TRACE] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.tcp.TcpTransport
- TCP consumer thread for tcp://localhost/127.0.0.1:61616 starting
02/13/2014 05:36:37.083 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.InactivityMonitor
- Using min of local: WireFormatInfo { version=9, properties={MaxFrameSize=104857600, CacheSize=1024,
CacheEnabled=true, SizePrefixDisabled=false, MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
and remote: WireFormatInfo { version=9, properties={CacheSize=1024, MaxFrameSize=209715200,
CacheEnabled=true, SizePrefixDisabled=false, TcpNoDelayEnabled=true, MaxInactivityDurationInitalDelay=10000,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
02/13/2014 05:36:37.083 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.WireFormatNegotiator
- Received WireFormat: WireFormatInfo { version=9, properties={CacheSize=1024, MaxFrameSize=209715200,
CacheEnabled=true, SizePrefixDisabled=false, TcpNoDelayEnabled=true, MaxInactivityDurationInitalDelay=10000,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
02/13/2014 05:36:37.083 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.WireFormatNegotiator
- tcp://localhost/127.0.0.1:61616 before negotiation: OpenWireFormat{version=9, cacheEnabled=false,
stackTraceEnabled=false, tightEncodingEnabled=false, sizePrefixDisabled=false, maxFrameSize=104857600}
02/13/2014 05:36:37.083 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.WireFormatNegotiator
- tcp://localhost/127.0.0.1:61616 after negotiation: OpenWireFormat{version=9, cacheEnabled=true,
stackTraceEnabled=true, tightEncodingEnabled=true, sizePrefixDisabled=false, maxFrameSize=104857600}


and in the primitive annotator I see the following:

02/13/2014 05:36:37.122 [DEBUG] [primitive_input_msgLsnrCntnr_TokenizerQueue_1_getMeta - Thread
- 18] org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer - Received
message of type [class org.apache.activemq.command.ActiveMQTextMessage] from consumer [ActiveMQMessageConsumer
{ value=ID:PCCS000630WS-56652-1392287793445-1:2:1:1, started=true }] of session [ActiveMQSession
{id=ID:PCCS000630WS-56652-1392287793445-1:2:1,started=true}]
02/13/2014 05:36:37.143 [DEBUG][primitive_input_msgLsnrCntnr_TokenizerQueue_1_getMeta - Thread
- 18] org.apache.activemq.transport.WireFormatNegotiator - Sending: WireFormatInfo { version=9,
properties={MaxFrameSize=104857600, CacheSize=1024, CacheEnabled=true, SizePrefixDisabled=false,
MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true, MaxInactivityDuration=0, TightEncodingEnabled=true,
StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
02/13/2014 05:36:37.146 [TRACE] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.tcp.TcpTransport
- TCP consumer thread for tcp://localhost/127.0.0.1:61616 starting
02/13/2014 05:36:37.146 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.InactivityMonitor
- Using min of local: WireFormatInfo { version=9, properties={MaxFrameSize=104857600, CacheSize=1024,
CacheEnabled=true, SizePrefixDisabled=false, MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
and remote: WireFormatInfo { version=9, properties={CacheSize=1024, MaxFrameSize=209715200,
CacheEnabled=true, SizePrefixDisabled=false, TcpNoDelayEnabled=true, MaxInactivityDurationInitalDelay=10000,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
02/13/2014 05:36:37.147 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.WireFormatNegotiator
- Received WireFormat: WireFormatInfo { version=9, properties={CacheSize=1024, MaxFrameSize=209715200,
CacheEnabled=true, SizePrefixDisabled=false, TcpNoDelayEnabled=true, MaxInactivityDurationInitalDelay=10000,
MaxInactivityDuration=0, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
02/13/2014 05:36:37.147 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.WireFormatNegotiator
- tcp://localhost/127.0.0.1:61616 before negotiation: OpenWireFormat{version=9, cacheEnabled=false,
stackTraceEnabled=false, tightEncodingEnabled=false, sizePrefixDisabled=false, maxFrameSize=104857600}
02/13/2014 05:36:37.147 [DEBUG] [ActiveMQ Transport: tcp://localhost/127.0.0.1:61616] org.apache.activemq.transport.WireFormatNegotiator
- tcp://localhost/127.0.0.1:61616 after negotiation: OpenWireFormat{version=9, cacheEnabled=true,
stackTraceEnabled=true, tightEncodingEnabled=true, sizePrefixDisabled=false, maxFrameSize=104857600}

I have logged also the name of the thread and it seems that, at least for the remote primitive
annotator, not the main thread starts negotiating using the default maxFrameSize of 100 MB.
The thread name contains "meta". Could it be the thread used for checking the state of the
component? Is it possible that this connection is not using the brokerUrl passed, but the
default one?

Any feedback is appreciated. 

Thanks,
Mihaela




On Wednesday, February 12, 2014 4:43 PM, Jaroslaw Cwiklik <uimaee@gmail.com> wrote:
 
It seems like the ActimeMQ documentation (http://activemq.apache.org/configuring-wire-formats.html)
is wrong with respect to the default maxFrameSize being MAX_LONG. I checked ActiveMQ source
code and the default is 100 MB:

publicfinalclassOpenWireFormatimplementsWireFormat{publicstaticfinalintDEFAULT_VERSION=CommandTypes.PROTOCOL_STORE_VERSION;publicstaticfinalintDEFAULT_WIRE_VERSION=CommandTypes.PROTOCOL_VERSION;publicstaticfinalintDEFAULT_MAX_FRAME_SIZE=100*1024*1024;//100
MB   <-------------------------------------staticfinalbyteNULL_TYPE=CommandTypes.NULL;privatestaticfinalintMARSHAL_CACHE_SIZE=Short.MAX_VALUE/2;privatestaticfinalintMARSHAL_CACHE_FREE_SPACE=100;

The UIMA-AS doesnt set this value so the default is being used unless overriden. It seems
to me that
either your service or a client is not overriding the default. Please check your deployment
descriptors to make sure
that you changing the default in the brokerURL. 

Jerry



On Wed, Feb 12, 2014 at 9:21 AM, Mihaela M <mmihaela1985@yahoo.com> wrote:

Hello,
>
>I have upgraded uima-as to version 2.4.2 but I still encounter an issue with the wireFormat.maxFrameSize
setting for the ActiveMQ broker.
>1. I have updated the configuration for transport connector in activemq.xml file:
><transportConnectors>
>            <transportConnector name="openwire" uri="tcp://127.0.0.1:61616?wireFormat.maxInactivityDuration=0&wireFormat.maxFrameSize=209715200&jms.useCompression=true"/>
></transportConnectors>
>2. I have set the brokerURL attribute in uima-as deployment descriptors to value: "tcp://127.0.0.1:61616?wireFormat.maxInactivityDuration=0&wireFormat.maxFrameSize=209715200&jms.useCompression=true"
>3. I have set the TRACE level for logger org.apache.activemq.transport
>
>After performing all the above settings I noticed that when I started the pipeline, for
each remote delegate, multiple negotiations are performed by org.apache.activemq.transport.WireFormatNegotiator.
All use the maxFrameSize of 200 MB that I specified, except one negotiation that is done using
maxFrameSize of 100 MB.
>I do not understand from where does come this limitation of 100 MB. Does exist in the
UIMA client? By default I saw that ActiveMQ is using MAX_LONG for maxFrameSize so I really
don't know from where does come this 100 MB setting for maxFrameSize.
>
>Does anyone have an idea why is happening this? Could somebody tell me a starting point
for looking in the uima code?
>
>
>On the other hand does anybody know whether there are some limitations when using the
"binary" serializer for remote delegates instead of "xmi" serializer? I found in one jira
issue (https://issues.apache.org/jira/browse/UIMA-1196) that for the "binary" serializer is
mandatory that all uima AS services use a common type system. Is this still an issue in uima-as
2.4.2?
>
>Thank you!
>Mihaela
>
>
>
>
>
>On Monday, January 27, 2014 4:30 PM, Eddie Epstein <eaepstein@gmail.com> wrote:
>
>On Thu, Jan 23, 2014 at 9:28 AM, Thomas Ginter <thomas.ginter@utah.edu>wrote:
>
>> It is likely then that your expansion is happening after the remote
>> service is called or else is not yet big enough to be over the 100MB limit.
>>
>
>Also note that by default UIMA-AS [Java] services use a delta-CAS
>interface. Only changes to the CAS
>are returned from a service.
>
>Besides deleting unnecessary FS from the final CAS to be returned, another
>option to consider is to use compression on JMS messages:
>jms.useCompression=true
>This decoration can be added to the broker configuration file,
>   $UIMA_HOME/amq/conf/activemq-nojournal.xml
>
>as
>   <transportConnector name="openwire" uri="tcp://
>0.0.0.0:61616?jms.useCompression=true"/>
>which will cause messages in all queues to be compressed.
>
>Eddie
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message