poi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumedh <sumedh.inam...@gmail.com>
Subject Re: Using MapDB to reduce memory footprint of shared strings table in SXSSF
Date Tue, 16 Dec 2014 10:56:08 GMT
We'll send you what we have done, it's showing good trade-off performance
(use memory if available, else use disk).

On Tue, Dec 16, 2014 at 3:01 PM, Nick Burch <apache@gagravarr.org> wrote:
>
> On Tue, 16 Dec 2014, Sumedh wrote:
>
>> 1. For a quick win, is it possible to provide a hook so that we can plug
>> in an overridden implementation of SharedStringTable class? As far as I
>> saw, there is no clean pluggability available right now (but I have very
>> little understanding of POI codebase).
>>
>
> We'd need to tweak things to allow that. However, is working at the CTRst
> level going to be good for you with MapDB or similar? Will serialising then
> deserialising those cause you lots of problems / overhead? Would there be a
> better "thing" to pass back and forth between XSSF / SXSSF / SAX code for a
> shared string?
>
> (There has been discussion lately about trying to avoid the amount of
> xmlbeans objects on public interfaces, so that a switch to something like
> jaxp could be done later if we want to, so this is one case when we can
> consider it)
>
>  2. If that works well, we can explore using MapDB as one of the options
>> to be used natively after considering all the other factors (like licensing
>> and size)...or may be some other smaller library focused only on this
>> aspect, or Alex's homegrown code. :)
>>
>> BTW, MapDB is free as speech and free as beer under Apache License 2.0
>> <https://github.com/jankotek/MapDB/blob/master/doc/license.txt>. :)
>> - https://github.com/jankotek/MapDB/blob/master/license.txt
>>
>
> And small too, so I don't see any major issues with making it an option
> for people wanting lower memory but higher IO reading, assuming we can't
> find a better one (eg from Alex or Lucene!)
>
>
> Nick
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
> For additional commands, e-mail: dev-help@poi.apache.org
>
>

-- 
Cheers,
Sumedh
http://www.linkedin.com/in/sumedhinamdar
Ph: +91 - 95610 99125

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message