axis-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tammy Dugan <tdu...@regenstrief.org>
Subject Re: [AXIOM] How to serialize axiom document without building objects in memory
Date Thu, 28 Jun 2007 19:44:06 GMT
With a little work, I was able to get streaming soap body xml output with input coming from
a database to work for very large files (200MB). 
Thanks, Ryan. However, now I want to be able to stream dynamically created xml. Here is most
of the code from my service:

    	StringDOM dom = new StringDOM();
    	PipedOutputStream output = new PipedOutputStream();
    	final StringDocument variableListDoc = dom.createDocument(output);
    	InputStream inputStream = new PipedInputStream(output);
    	OMFactory factory = OMAbstractFactory.getOMFactory();
    	OMElement resultElement = XMLUtil.parseXMLFromStreamOM(inputStream,"getQueryVariableListResponse",true,factory);
    	final StringElement resultXML = (StringElement) variableListDoc.createElement(RESPONSE_XML_NODENAME);
	Thread thread = new Thread(new Runnable()
		{
			public void run()
			{
				try
				{
					Connection con = db.openConnection();
					StringElement result = (StringElement) qdb.readQueryVariableList(con, db, variableListDoc);
					resultXML.appendChild(result);
					result.closeBuffer();
				} catch (Exception e)
				{
					InitializationServlet.error("", e);
				}
			}
		});
		
		thread.start();
		
		return resultElement;


StringDom is a DOM implementation one of our developers wrote that streams the content of
a node added through an appendChild call, thus very little is ever held in memory. 
When the above service runs, it is buffering the content of resultElement in memory and not
streaming it to the response. I copied the exact same code to a local test case and ran serializeAndConsume
on resultElement. 
When I did this, everything streamed just fine. I am using a Custom OMDataSource to create
resultElement as an OMSourcedElementImpl to facilitate the streaming. When I ran the local
example that streamed, 
the following method was called in my custom data source:

public void serialize(OutputStream output, OMOutputFormat format) throws XMLStreamException
	{
		System.out.println("CustomDataSource serialized(output,format)");
		try
		{
			org.regenstrief.util.Util.bufferedReadWrite(this.data, output, false);
		} catch (IOException e)
		{
			throw new XMLStreamException(e);
		}
	}

where bufferedReadWrite just reads the bytes from an input stream and writes them to an output
stream in chunks. The serialization method called when I executed the service was the following:

public void serialize(XMLStreamWriter xmlWriter) throws XMLStreamException
	{
		System.out.println("CustomDataSource serialized(xmlWriter)");
		XMLStreamReader reader = null;

		try
		{
			StreamingOMSerializer serializer = new StreamingOMSerializer();
			reader =  StAXUtils.createXMLStreamReader(this.data);
			serializer.serialize(reader, xmlWriter);// OutOfMemory error here
		} catch (Throwable e)
		{
				InitializationServlet.error("", e);
		}
	}

This method would not stream the data to the output. Is there something about StreamingOMSerializer
that it is hanging while the xml is getting dynamically built? This same method works just
fine with
an InputStream from a database so why doesn't it work for a PipedInputStream?

Thanks,

Tammy


>I've had some luck getting this to work by implementing an OMDataSource that
>doesn't start writing XML until it is passed a writer and then using it to
>create an OMSourcedElementImpl.  It sort of depends on the semantics of what
>you are doing, and you have to be careful as to what methods get called on
>the OMSourcedElementImpl (it doesn't seem to take much to cause the whole
>thing to inflate - try to avoid doing anything that causes the getReader()
>method to get called before you are ready to start sending stuff out the
>door), but we've managed to get true streaming to work using Axis2/Axiom,
>which saves a lot of transient memory.

-- 
Tammy Dugan
Computer Programmer

Regenstrief Institute, Inc.
Medical Informatics
Health Information and Translational Sciences (HITS) Building
410 West 10th Street, Suite 2000
Indianapolis, IN 46202
Main: 317.423.5500
Fax: 317.423.5695
IU campus mail address: HS, 2000

(317) 423 - 5541

Confidentiality Notice: The contents of this message and any files transmitted with it may
contain confidential and/or privileged information and are intended solely for the use of
the named addressee(s). Additionally, the information contained herein may have been disclosed
to you from medical records with confidentiality protected by federal and state laws. Federal
regulations and State laws prohibit you from making further disclosure of such information
without the specific written consent of the person to whom the information pertains or as
otherwise permitted by such regulations. A general authorization for the release of medical
or other information is not sufficient for this purpose.
 
If you have received this message in error, please notify the sender by return e-mail and
delete the original message. Any retention, disclosure, copying, distribution or use of this
information by anyone other than the intended recipient is strictly prohibited.



---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Mime
View raw message