jena-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Castagna <castagna.li...@googlemail.com>
Subject Re: ConcurrentModificationException with Fuseki/TDB
Date Wed, 09 Feb 2011 12:12:13 GMT
Hi Andy,
comments inline.

Andy Seaborne wrote:
> New SNAPSHOT built.
> 
>  > SPARQL_Query.executeForm
> 
> It's a SPARQL Update set by POST in an HTML form.
> 
> What does the log say?
> 
> I've found potential bug and added some Fuseki-level internal checking. 
>  While it works for me, I haven't run your script.  Also, Andreas tells 
> me that the BSBM tools have changed a lot recently.

Is Andreas on jena-users mailing list?

> Paolo - could you run your tests with the new snapshot with a product 
> count of 14092 (5M), please?

I think everything is ok, now (i.e. I am using Fuseki and BSBM from trunk
since the script I am using [1] is checking out those from SVN):

----
if [ ! -d "$BSBM_ROOT_PATH/bsbmtools" ]; then
echo "Downloading and compiling Berlin SPARQL Benchmark (BSBM)..."
     cd $BSBM_ROOT_PATH
     svn co https://bsbmtools.svn.sourceforge.net/svnroot/bsbmtools/trunk bsbmtools
     cd $BSBM_ROOT_PATH/bsbmtools
     ant jar
     echo "done."
fi

[...]

if [ ! -d "$BSBM_ROOT_PATH/fuseki" ]; then
echo "Downloading and compiling Fuseki..."
     cd $BSBM_ROOT_PATH
     svn co http://jena.svn.sourceforge.net/svnroot/jena/Fuseki/trunk fuseki
     cd $BSBM_ROOT_PATH/fuseki
     mvn package
     echo "done."
fi
----

BSBM runs fine, now.

I am using:

----
BSBM_SCALE_FACTOR=14092
BSBM_CONCURRENT_CLIENTS=4
----

I get these results running it on my laptop:

----
Scale factor:           14092
Number of warmup runs:  32
Number of clients:      4
Seed:                   808080
Number of query mix runs (without warmups): 128 times
min/max Querymix runtime: 0.3574s / 1.2071s
Total runtime (sum):    96.598 seconds
Total actual runtime:   27.153 seconds
QMpH:                   16970.74 query mixes per hour
CQET:                   0.75467 seconds average runtime of query mix
CQET (geom.):           0.73416 seconds geometric mean runtime of query mix
----

No exceptions.

I can see what you have done [2], however I am not sure if I understand
correctly this: is TDB now throwing a ConcurrentModificationException if
MRSW locking is not used properly? Just double checking...

As always, thanks for the speed of fixing things when problems come up.
I would have liked to help more on this... but by the time I was still
searching for the bug, you had a fix for it. :-)

Paolo

  [1] https://github.com/castagna/bsbm-automated
  [2] svn diff -r 8474:8497


> 
> 
>     Andy
> 
> (Overall, I'll glad the new checks are showing up these things because 
> it's all accidents waiting to happen silently)
> 
> On 08/02/11 19:55, Andy Seaborne wrote:
>> Even concurrently sent updates should work with Fuseki because it
>> applies locking.
>>
>> The latest BSBM does send updates.
>>
>> I need to investigate ...
>>
>> Andy
>>
>> On 08/02/11 19:22, Paolo Castagna wrote:
>>> Hi Benson,
>>> thank you, you raised a good point, actually.
>>>
>>> I need to make sure the BSBM benchmark is not sending any SPARQL Update.
>>> It seems to me I got the exception just sending SPARQL queries.
>>>
>>> However, I'll double check this.
>>>
>>> Paolo
>>>
>>> Benson Margulies wrote:
>>>> Paolo,
>>>>
>>>> I think it's a feature. Between 0.8.9 and trunk, TDB got much more
>>>> sensitive to modifying anything at all while any iterator was live.
>>>>
>>>>
>>>> On Tue, Feb 8, 2011 at 1:13 PM, Paolo Castagna
>>>> <castagna.lists@googlemail.com> wrote:
>>>>> Hi,
>>>>> today, I have been playing with the BSBM (i.e. trying to automate as
>>>>> much as
>>>>> possible).
>>>>> I got a (reasonable?) bash script:
>>>>> https://github.com/castagna/bsbm-automated
>>>>>
>>>>> If I use just one client everything is fine.
>>>>>
>>>>> However, when I try to run BSBM against Fuseki with more than one
>>>>> client
>>>>> (i.e. BSBM uses multiple threads to run queries) I get this exception
>>>>> on the
>>>>> Fuseki log:
>>>>>
>>>>> java.util.ConcurrentModificationException: Reader = 0, Writer = 2
>>>>> at
>>>>> com.hp.hpl.jena.tdb.sys.ConcurrencyPolicyMRSW.policyError(ConcurrencyPolicyMRSW.java:127)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> com.hp.hpl.jena.tdb.sys.ConcurrencyPolicyMRSW.policyError(ConcurrencyPolicyMRSW.java:122)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> com.hp.hpl.jena.tdb.sys.ConcurrencyPolicyMRSW.checkConcurrency(ConcurrencyPolicyMRSW.java:62)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> com.hp.hpl.jena.tdb.sys.ConcurrencyPolicyMRSW.startUpdate(ConcurrencyPolicyMRSW.java:46)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.startWrite(NodeTupleTableConcrete.java:54)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.sync(NodeTupleTableConcrete.java:237)

>>>>>
>>>>>
>>>>>
>>>>> at com.hp.hpl.jena.tdb.store.TripleTable.sync(TripleTable.java:89)
>>>>> at
>>>>> com.hp.hpl.jena.tdb.store.DatasetGraphTDB.sync(DatasetGraphTDB.java:262)

>>>>>
>>>>>
>>>>> at com.hp.hpl.jena.sparql.SystemARQ.sync(SystemARQ.java:47)
>>>>> at org.openjena.fuseki.servlets.HttpAction.sync(HttpAction.java:61)
>>>>> at
>>>>> org.openjena.fuseki.servlets.HttpAction.endWrite(HttpAction.java:39)
>>>>> at
>>>>> org.openjena.fuseki.servlets.SPARQL_Query.execute(SPARQL_Query.java:226)

>>>>>
>>>>>
>>>>> at
>>>>> org.openjena.fuseki.servlets.SPARQL_Query.executeForm(SPARQL_Query.java:187)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.openjena.fuseki.servlets.SPARQL_Query.perform(SPARQL_Query.java:87)

>>>>>
>>>>> at
>>>>> org.openjena.fuseki.servlets.SPARQL_ServletBase.doCommon(SPARQL_ServletBase.java:76)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.openjena.fuseki.servlets.SPARQL_Query.doGet(SPARQL_Query.java:68)
>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>>> at
>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:534)
>>>>> at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:475)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:921)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:856)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)

>>>>>
>>>>>
>>>>>
>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:352)
>>>>> at
>>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1052)

>>>>>
>>>>>
>>>>>
>>>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:590)
>>>>> at
>>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:212)
>>>>> at
>>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426)

>>>>>
>>>>> at
>>>>> org.eclipse.jetty.server.nio.BlockingChannelConnector$BlockingChannelEndPoint.run(BlockingChannelConnector.java:292)

>>>>>
>>>>>
>>>>>
>>>>> at
>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450)

>>>>>
>>>>>
>>>>>
>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>>
>>>>> I have been using TDB (from trunk) and Fuseki (from trunk):
>>>>>
>>>>> Jena: VERSION: 2.6.4
>>>>> Jena: BUILD_DATE: 2010-12-12T16:56:15+0000
>>>>> ARQ: VERSION: 2.8.8-SNAPSHOT
>>>>> ARQ: BUILD_DATE: 2011-02-04T18:04:22+0000
>>>>> TDB: VERSION: 0.8.10-SNAPSHOT
>>>>> TDB: BUILD_DATE: 2011-02-07T13:25:50+0000
>>>>> Fuseki: VERSION: 0.2.0-SNAPSHOT
>>>>> Fuseki: BUILD_DATE: 2011-02-08T12:43:18+0000
>>>>>
>>>>> I have tried to do the same with Joseki v3.4.3 (I was not able to
>>>>> check it
>>>>> out using CVS as anonymous user, bah...)
>>>>>
>>>>> I know, I know...
>>>>>
>>>>> I am just asking, is it a known problem?
>>>>>
>>>>> Paolo
>>>>>


Mime
View raw message