jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jukka Zitting" <jukka.zitt...@gmail.com>
Subject Re: JackRabbit performance tests report
Date Fri, 01 Dec 2006 09:04:50 GMT
Hi,

On 12/1/06, Stefan Gruszczynski <stefan.gruszczynski@gmail.com> wrote:
> Recently you wrote about custom importer - could you give me an example? The
> thing I need mostly in JackRabbit is an ability to import large XML files
> with low memory usage.

See below for an (undocumented) example I quickly wrote in one
project. It implements a basic XML-to-JCR mapping that's roughly like
the standard document view, but lacks all advanced features like node
type handling, references, etc.

The essential part in achieving huge imports is the following snippet
that saves the pending changes every now and then:

    count++;
    if (count % 100 == 0) {
        node.getSession().save();
    }

Note that this only works for content (nt:unstructured) that allows
partial saves. Otherwise you might easily get an exception when trying
to save when all required children of a node have not yet been added.
Supporting such cases requires adding extra logic to the importer.

BR,

Jukka Zitting

    class CustomHandler extends DefaultHandler {

        private StringBuffer buffer = new StringBuffer();

        private int count = 0;

        private Node node;

        private Node current;

        public CustomHandler(Node node, Node current) {
            this.node = node;
            this.current = current;
        }

        public Node getCurrent() {
            return node;
        }

        public void startElement(String uri, String localName, String
qName, Attributes attributes) throws SAXException {
            try {
                if (buffer.length() > 0) {
                    Node text = current.addNode("jcr:xmltext");
                    text.setProperty("jcr:xmlcharacters", buffer.toString());
                    buffer.setLength(0);
                }

                current = (current == null) ? node : current.addNode(qName);
                for (int i = 0; i < attributes.getLength(); i++) {
                    current.setProperty(attributes.getQName(i),
attributes.getValue(i));
                }
            } catch (RepositoryException e) {
                throw new SAXException(e.getMessage(), e);
            }
        }

        public void endElement(String uri, String localName, String
qName) throws SAXException {
            try {
                if (buffer.length() > 0) {
                    if (current.getNodes().hasNext()) {
                        Node text = current.addNode("jcr:xmltext");
                        text.setProperty("jcr:xmlcharacters",
buffer.toString());
                    } else {
                        current.setProperty("jcr:xmlcharacters",
buffer.toString());
                    }
                    buffer.setLength(0);
                }

                node = current;
                current = current.getParent();
                count++;
                if (count % 100 == 0) {
                    node.getSession().save();
                }
            } catch (RepositoryException e) {
                throw new SAXException(e.getMessage(), e);
            }
        }

        public void characters(char[] ch, int start, int length) {
            buffer.append(ch, start, length);
        }

    }

Mime
View raw message