ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vasilisa Sidorova (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-1722) Many examples fail if they are running with JdkMarshaller in the cluster with different vendors and versions jvm
Date Mon, 19 Oct 2015 16:17:05 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963547#comment-14963547
] 

Vasilisa  Sidorova commented on IGNITE-1722:
--------------------------------------------

# ScalaCacheQueryExample get exception on the java8 node in the cluster:
{noformat}
[19:11:07,968][ERROR][tcp-disco-msg-worker-#2%null][TcpDiscoverySpi] Failed to unmarshal discovery
custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given class loader
for unmarshalling (make sure same versions of all classes are available on all nodes or enable
peer-class-loading): sun.misc.Launcher$AppClassLoader@27c170f0
	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:108)
	at org.apache.ignite.marshaller.AbstractMarshaller.unmarshal(AbstractMarshaller.java:68)
	at org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:78)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:4340)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:4299)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2125)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:5382)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: org.apache.ignite.scalar.examples.Organization
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8137)
	at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:54)
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613)
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
	at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1484)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1334)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1707)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
	at java.util.ArrayList.readObject(ArrayList.java:791)
	at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:102)
	... 7 more
{noformat} 
# Isn't reproducible if there are one version and different vendors java in the cluster
# Yes, this is bad cluster but most of examples work fine in this case 

> Many examples fail if they are running with JdkMarshaller in the cluster with different
vendors and versions jvm
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-1722
>                 URL: https://issues.apache.org/jira/browse/IGNITE-1722
>             Project: Ignite
>          Issue Type: Bug
>          Components: general
>    Affects Versions: 1.5
>         Environment: Ubuntu 14.04, community 1.5.0 build #319
>            Reporter: Vasilisa  Sidorova
>            Assignee: Andrey Gura
>             Fix For: 1.5
>
>
> ENVIRONMENT
> --------------------------------------------
> ibm-java-7 -version: 
> {noformat}
> java version "1.7.0"
> Java(TM) SE Runtime Environment (build pxa6470_27sr3fp10-20150708_01(SR3 FP10))
> IBM J9 VM (build 2.7, JRE 1.7.0 Linux amd64-64 Compressed References 20150630_255653
(JIT enabled, AOT enabled)
> J9VM - R27_Java727_SR3_20150630_2236_B255653
> JIT  - tr.r13.java_20150623_94888.01
> GC   - R27_Java727_SR3_20150630_2236_B255653_CMPRSS
> J9CL - 20150630_255653)
> JCL - 20150628_01 based on Oracle jdk7u85-b15
> {noformat}
> oracle-java-7 -version:
> {noformat}
> java version "1.7.0_80" 
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15) 
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> {noformat}
> ---------------------------------------------
> DESCRIPTION
> ---------------------------------------------
> When default optimizedMarshaller is changed into JdkMarshaller in example-ignite.xml
and cluster with different vendors and versions jvm is started then most of Compute examples
(ClusterGroupExample, CreditRiskExample, ComputeAsyncExample, ComputeBroadcastExample, ComputeCallableExample,
ComputeClosureExample, ComputeReducerExample, ComputeRunnableExample), CacheAffinityExample,
CacheContinuousQueryExample, IgniteAtomicLongExample, IgniteExecutorServiceExample, EventsExample
fail during running in this cluster
> ---------------------------------------------
> STEPS FOR REPRODUCE
> ---------------------------------------------
> # Build examples project in IDE (for example, for the ibm-java-7 jvm)
> # In the example-ignite.xml change property from:
> {noformat}
> <property name="marshaller">
>             <bean class="org.apache.ignite.marshaller.optimized.OptimizedMarshaller">
-->
>                 <!-- Set to false to allow non-serializable objects in examples, default
is true. -->
>                 <!-- <property name="requireSerializable" value="false"/>
>             </bean>
>         </property>
> {noformat}
> to
> {noformat}
> <property name="marshaller">
>         <bean class="org.apache.ignite.marshaller.jdk.JdkMarshaller"/>
>     </property>
> {noformat}
> # Run ExampleNodeStartup on the ibm-java-7 jvm 
> # In the same cluster run second ExampleNodeStartup (or corresponding node from command
line) on the, for example, oracle-java-7 jvm
> # For example, run CreditRiskExample on the ibm-java-7 jvm
> ---------------------------------------------
> ACTUAL RESULT
> ---------------------------------------------
> Example is failed with exception:
> {noformat}
> [18:32:50,486][ERROR][ignite-#19%sys-null%][GridTaskWorker] Failed to obtain remote job
result policy for result from ComputeTask.result(..) method (will fail the whole task): GridJobResultImpl
[job=C2 [], sib=GridJobSiblingImpl [sesId=4e238417051-f0bdd2e0-2e1b-40c9-b395-5adeea3abb80,
jobId=6e238417051-b4bcdc8a-d015-4add-a605-63dff152b030, nodeId=b4bcdc8a-d015-4add-a605-63dff152b030,
isJobDone=false], jobCtx=GridJobContextImpl [jobId=6e238417051-b4bcdc8a-d015-4add-a605-63dff152b030,
timeoutObj=null, attrs={}], node=TcpDiscoveryNode [id=b4bcdc8a-d015-4add-a605-63dff152b030,
addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47501], discPort=47501, order=4, intOrder=3, lastExchangeTime=1445009569431,
loc=false, ver=1.5.0#20151012-sha1:6844370d, isClient=false], ex=class o.a.i.IgniteException:
Failed to deserialize object with given class loader: sun.misc.Launcher$AppClassLoader@501d5ebc,
hasRes=true, isCancelled=false, isOccupied=true]
> class org.apache.ignite.IgniteException: Remote job threw user exception (override or
implement ComputeTask.result(..) method if you would like to have automatic failover for this
exception).
> 	at org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101)
> 	at org.apache.ignite.internal.processors.closure.GridClosureProcessor$T3.result(GridClosureProcessor.java:1246)
> 	at org.apache.ignite.internal.processors.task.GridTaskWorker$3.apply(GridTaskWorker.java:903)
> 	at org.apache.ignite.internal.processors.task.GridTaskWorker$3.apply(GridTaskWorker.java:896)
> 	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6403)
> 	at org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:896)
> 	at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:792)
> 	at org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:995)
> 	at org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1219)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:811)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:106)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:774)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
> 	at java.lang.Thread.run(Thread.java:809)
> Caused by: class org.apache.ignite.IgniteException: Failed to deserialize object with
given class loader: sun.misc.Launcher$AppClassLoader@501d5ebc
> 	at org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:424)
> 	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1094)
> 	at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1776)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:811)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:106)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:774)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object
with given class loader: sun.misc.Launcher$AppClassLoader@501d5ebc
> 	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:105)
> 	at org.apache.ignite.marshaller.AbstractMarshaller.unmarshal(AbstractMarshaller.java:68)
> 	at org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:409)
> 	... 8 more
> Caused by: java.io.InvalidClassException: org.apache.ignite.examples.computegrid.montecarlo.CreditRiskExample$2;
local class incompatible: stream classdesc serialVersionUID = 1895801581034216497, local class
serialVersionUID = 9086725335444863242
> 	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 	at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.readExternal(GridClosureProcessor.java:1808)
> 	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:102)
> 	... 10 more
> >>> Finished calculating portfolio risk [risk=499581.9554895319, time=2728ms]
> [18:32:53] Ignite node stopped OK [uptime=00:00:02:841]
> Exception in thread "main" class org.apache.ignite.IgniteException: Failed to deserialize
object with given class loader: sun.misc.Launcher$AppClassLoader@501d5ebc
> 	at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:881)
> 	at org.apache.ignite.internal.IgniteComputeImpl.call(IgniteComputeImpl.java:408)
> 	at org.apache.ignite.examples.computegrid.montecarlo.CreditRiskExample.main(CreditRiskExample.java:85)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
> 	at java.lang.reflect.Method.invoke(Method.java:620)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object
with given class loader: sun.misc.Launcher$AppClassLoader@501d5ebc
> 	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:105)
> 	at org.apache.ignite.marshaller.AbstractMarshaller.unmarshal(AbstractMarshaller.java:68)
> 	at org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:409)
> 	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1094)
> 	at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1776)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:811)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:106)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:774)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.InvalidClassException: org.apache.ignite.examples.computegrid.montecarlo.CreditRiskExample$2;
local class incompatible: stream classdesc serialVersionUID = 1895801581034216497, local class
serialVersionUID = 9086725335444863242
> 	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 	at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.readExternal(GridClosureProcessor.java:1808)
> 	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:102)
> 	... 10 more
> {noformat}
> There is exception in the remote (second) node console:
> {noformat}
> [18:32:50,466][ERROR][ignite-#7%pub-null%][GridJobWorker] Failed to initialize job [jobId=6e238417051-b4bcdc8a-d015-4add-a605-63dff152b030,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=o.a.i.examples.computegrid.montecarlo.CreditRiskExample$2,
dep=GridDeployment [ts=1445009556695, depMode=SHARED, clsLdr=sun.misc.Launcher$AppClassLoader@501d5ebc,
clsLdrId=35ff7417051-b4bcdc8a-d015-4add-a605-63dff152b030, userVer=0, loc=true, sampleClsName=java.util.HashMap,
pendingUndeploy=false, undeployed=false, usage=1], taskClsName=o.a.i.examples.computegrid.montecarlo.CreditRiskExample$2,
sesId=4e238417051-f0bdd2e0-2e1b-40c9-b395-5adeea3abb80, startTime=1445009570408, endTime=9223372036854775807,
taskNodeId=f0bdd2e0-2e1b-40c9-b395-5adeea3abb80, clsLdr=sun.misc.Launcher$AppClassLoader@501d5ebc,
closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=f0bdd2e0-2e1b-40c9-b395-5adeea3abb80,
mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1445009570454,
endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]], jobId=6e238417051-b4bcdc8a-d015-4add-a605-63dff152b030]]
> class org.apache.ignite.IgniteCheckedException: Failed to deserialize object with given
class loader: sun.misc.Launcher$AppClassLoader@501d5ebc
> 	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:105)
> 	at org.apache.ignite.marshaller.AbstractMarshaller.unmarshal(AbstractMarshaller.java:68)
> 	at org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:409)
> 	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1094)
> 	at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1776)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:811)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:106)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:774)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.InvalidClassException: org.apache.ignite.examples.computegrid.montecarlo.CreditRiskExample$2;
local class incompatible: stream classdesc serialVersionUID = 1895801581034216497, local class
serialVersionUID = 9086725335444863242
> 	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 	at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.readExternal(GridClosureProcessor.java:1808)
> 	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:102)
> 	... 10 more
> {noformat}
> ---------------------------------------------
> EXPECTED RESULT
> ---------------------------------------------
> Example is passed without any exceptions
> --------------------------------------------
> ADDITIONAL INFO
> --------------------------------------------
> # Isn't reproducible for optimized and portable marshallers
> # Isn't reproducible for cluster were all jvm are identical
> # Reproducible, at least, for clusters with next list of jvm: 
> ## oracle-java-7 + ibm-java-7
> ## oracle-java-7 + openjdk-7
> ## oracle-java-7 + oracle-java-8
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message