incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jp>
Subject Re:Re: BulkLoader error with large data and fast harddrive
Date Sat, 18 Jun 2011 15:38:46 GMT
Hey Andy,

My entire program is run on one jvm as follows.

public static void main(String[] args) throws IOException{
  DatasetGraphTDB datasetGraph = TDBFactory.createDatasetGraph(tdbDir);

/* I saw the BulkLoader had two ways of loading data based on whether
the dataset existed already. I did two runs one with the following two
lines commented out to test both ways the BulkLoader runs. Hopefully
this had the desired effect. */
Triple(Node.createURI("urn:hello"), RDF.type.asNode(),

  InputStream inputStream = new FileInputStream(dbpediaData);

  BulkLoader bulkLoader = new BulkLoader();
  bulkLoader.loadDataset(datasetGraph, inputStream, true);

The data can be found here
I appended the ontology to end of file it can be found here

The tdbDir is an empty directory.
On my system the error starts occurring after about 2-3minutes and
8-12 million triples loaded.

Thanks for looking over this and please let me know if I can be of
further assistance.


On Jun 17, 2011 9:29 am, andy wrote:
>How does this fit with running:
>Triple(Node.createURI("urn:hello"), RDF.type.asNode(),
>Is the preload of one triple a separate JVM or the same JVM as the
>BulkLoader call - could you provide a single complete minimal example?
>In attempting to reconstruct this, I don't want to hide the problem by
>guessing how things are wired together.
>Also - exactly which dbpedia file are you loading (URL?) although I
>doubt the exact data is the cause here.

View raw message