Mailing-List: contact dev-help@poi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "POI Developers List" <dev@poi.apache.org>
Received-SPF: pass (athena.apache.org: local policy)
Date: Tue, 16 Dec 2014 09:31:24 +0000 (GMT)
From: Nick Burch <apache@gagravarr.org>
To: POI Developers List <dev@poi.apache.org>
Subject: Re: Using MapDB to reduce memory footprint of shared strings table
 in SXSSF
In-Reply-To: 
 <CAG-GydZCiggHGirsz=3+BSXHrnvu0n8pNnGzp_8CPu1-HL2KPA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1412160927250.1059@urchin.earth.li>
References: 
 <CAG-GydbW0koHQ4=0zC0fVqzV=dXaeSRHnZo9VQCZi8YbMwWOmg@mail.gmail.com>
 <alpine.DEB.2.02.1412151344340.1059@urchin.earth.li>
 <CAG-GydZCiggHGirsz=3+BSXHrnvu0n8pNnGzp_8CPu1-HL2KPA@mail.gmail.com>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

On Tue, 16 Dec 2014, Sumedh wrote:
> 1. For a quick win, is it possible to provide a hook so that we can plug 
> in an overridden implementation of SharedStringTable class? As far as I 
> saw, there is no clean pluggability available right now (but I have very 
> little understanding of POI codebase).

We'd need to tweak things to allow that. However, is working at the CTRst 
level going to be good for you with MapDB or similar? Will serialising 
then deserialising those cause you lots of problems / overhead? Would 
there be a better "thing" to pass back and forth between XSSF / SXSSF / 
SAX code for a shared string?

(There has been discussion lately about trying to avoid the amount of 
xmlbeans objects on public interfaces, so that a switch to something like 
jaxp could be done later if we want to, so this is one case when we can 
consider it)

> 2. If that works well, we can explore using MapDB as one of the options 
> to be used natively after considering all the other factors (like 
> licensing and size)...or may be some other smaller library focused only 
> on this aspect, or Alex's homegrown code. :)
>
> BTW, MapDB is free as speech and free as beer under Apache License 2.0
> <https://github.com/jankotek/MapDB/blob/master/doc/license.txt>. :)
> - https://github.com/jankotek/MapDB/blob/master/license.txt

And small too, so I don't see any major issues with making it an option 
for people wanting lower memory but higher IO reading, assuming we can't 
find a better one (eg from Alex or Lucene!)

Nick

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org