So I am doing a lot of random reads and I am getting only 1-2 lookups per second where I used to get thousands per second.  I did an HPROF and found that the root cause relates to the Sun http protocol handler not handling InputStream.skip() correctly.  So every time it looks up a block it reads the whole file across the network.  

Anyone have any suggestions on how to improve this?  Maybe use the Apache HTTP Client instead of sun.net?

Here’s a sample stack (Derby 10.8):

TRACE 373064:
    java.net.SocketInputStream.socketRead0(SocketInputStream.java:Unknown line)
    java.net.SocketInputStream.read(SocketInputStream.java:129)
    java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
    java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
    java.io.BufferedInputStream.read(BufferedInputStream.java:317)
    sun.net.www.MeteredStream.read(MeteredStream.java:116)
    java.io.FilterInputStream.read(FilterInputStream.java:116)
    sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2672)
    sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.skip(HttpURLConnection.java:2719)
    org.apache.derby.iapi.services.io.InputStreamUtil.skipPersistent(InputStreamUtil.java:169)
    org.apache.derby.iapi.services.io.InputStreamUtil.skipFully(InputStreamUtil.java:147)
    org.apache.derby.impl.store.raw.data.InputStreamContainer.readPositionedPage(InputStreamContainer.java:208)
    org.apache.derby.impl.store.raw.data.InputStreamContainer.readPage(InputStreamContainer.java:186)
    org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:673)
    org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:193)
    org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295)
    org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2541)
    org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2591)
    org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319)
    org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.latchPage(OpenConglomerate.java:308)
    org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.fetch(GenericConglomerateController.java:263)
    org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCore(IndexRowToBaseRowResultSet.java:389)
    org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478)
    org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:430)
    org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:374)
    com.icrossing.collection.ipinfo.derby.DerbyQuery.run(DerbyQuery.java:90)
    java.lang.Thread.run(Thread.java:680)

iCrossing Privileged and Confidential Information
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information of iCrossing. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.