We'll be measuring that soon, so I should be able to report on that.  We've got some pretty large views (last I looked, ~1M docs with some views ~350GB).  My general observation tho by just plain wall clock estimation, is that indexing a large number of docs is significantly faster.  The trickle-in doc indexing may not be as efficient.

I considered some sort of server model for the design. It could be forked at some point to work like that - I considered using some curl-like interface to servlet to do this, I just wasn't sure which would be worse, the overhead from JVM bootstrap or the dynamic classloader.  I suspected, as my past experience with this type of solution, is that the dynamic classloader has a bit higher overhead, making the JVM bootstrap more negligible.

I've not seen Jackson before.. I'll take a look. Thanks!  Maybe I can support both.


Jim Klo
Senior Software Engineer
Center for Software Engineering
SRI International

On Feb 13, 2012, at 9:25 PM, Dan Everton wrote:

Looks interesting. I'm curious if you see any performance difference by
writing your views in Java? I imagine it could be significantly faster but
if the JVM gets started on each view server call it's going to bypass any
HotSpot optimisation.

Also, you might want to consider switching to Jackson from the org.json
library for JSON handling as Jackson is significantly (like 10x or more)
faster for handling JSON data which may be significant during view
generation.

Cheers,
Dan