jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rajesh <rajes...@skava.com>
Subject Jackrabbit Oak : How to configure SegmentNodeStoreService and S3DataStore so that it will delete the S3 file if we remove the node.
Date Sun, 18 Mar 2018 13:42:59 GMT
I am using jackrabbit oak version 1.8.2. I configured SegmentNodeService with
S3DataStore as a custom blob store. When I save the content in Session, the
file got saved in S3 bucket. When I remove the node,  file still remains in
the S3 bucket. 

Here is the code i am using for saving the file.


Node contentFolder = null;
if(!session.nodeExists("/content"))
{
	contentFolder = root.addNode("content", NT_FOLDER);
}
else
{
	contentFolder = root.getNode("content");
}

nfFile = contentFolder.addNode(file.getName(), NT_FILE);


Node content = nfFile.addNode(JCR_CONTENT, NT_RESOURCE);
Binary binary = session.getValueFactory().createBinary(new
FileInputStream(file));
content.setProperty(JCR_DATA, binary);
content.setProperty(JCR_MIMETYPE, Files.probeContentType(file.toPath()));
session.save();                              




Here is the code I am using to delete the node



Node node = session.getNode("/content");
node.remove();
session.save();




Here is the log I am getting while deleting the node.


18:06:57.599 [oak-ds-cache-scheduled-thread-1] INFO 
o.a.j.o.p.blob.UploadStagingCache - Starting purge of uploaded files
18:06:57.599 [oak-ds-cache-scheduled-thread-1] DEBUG
o.a.j.o.p.blob.UploadStagingCache - Trying to delete file
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\up
load\b2\45\14\b24514baa6e05d0321381d0e5d6bf06a5f4ab0c2c540d3574a5995860a8162ed]
18:06:57.599 [oak-ds-cache-scheduled-thread-1] INFO 
o.a.j.o.p.blob.DataStoreCacheUtils - Deleted file
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\upload\b2\
45\14\b24514baa6e05d0321381d0e5d6bf06a5f4ab0c2c540d3574a5995860a8162ed]
18:06:57.599 [oak-ds-cache-scheduled-thread-1] DEBUG
o.a.j.o.p.blob.DataStoreCacheUtils - Deleted directory
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\uploa
d\b2\45\14], [true]
18:06:57.599 [oak-ds-cache-scheduled-thread-1] DEBUG
o.a.j.o.p.blob.DataStoreCacheUtils - Deleted directory
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\uploa
d\b2\45], [true]
18:06:57.599 [oak-ds-cache-scheduled-thread-1] DEBUG
o.a.j.o.p.blob.DataStoreCacheUtils - Deleted directory
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\uploa
d\b2], [true]
18:06:57.615 [oak-ds-cache-scheduled-thread-1] DEBUG
o.a.j.o.p.blob.UploadStagingCache - deleted file
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\upload\b2\4
5\14\*b24514baa6e05d0321381d0e5d6bf06a5f4ab0c2c540d3574a5995860a8162ed*]
18:06:57.615 [oak-ds-cache-scheduled-thread-1] DEBUG
o.a.j.o.p.blob.UploadStagingCache - Cache
*[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\repository\datastore\upload\b2\45\14\b2
4514baa6e05d0321381d0e5d6bf06a5f4ab0c2c540d3574a5995860a8162ed] file deleted
for id [b24514baa6e05d0321381d0e5d6bf06a5f4ab0c2c540d3574a5995860a8162ed]*
18:06:57.615 [oak-ds-cache-scheduled-thread-1] INFO 
o.a.j.o.p.blob.UploadStagingCache - Finished removal of [1] files
18:06:58.985 [oak-scheduled-executor-8] DEBUG
o.a.j.o.p.index.AsyncIndexUpdate - [async] Running background index task
18:06:58.985 [oak-scheduled-executor-8] DEBUG
o.a.j.o.p.index.AsyncIndexUpdate - [async] No changes since last checkpoint;
skipping the index update
18:07:00.321 [TarMK flush
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\segmentstore]]
DEBUG o.a.j.oak.segment.file.TarRevisions - Head state did not change,
skipping flush
18:07:00.993 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Closed writer
18:07:00.993 [pool-4-thread-1] DEBUG o.a.j.o.p.b.datastore.BlobIdTracker -
File renamed true
18:07:00.993 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Process file renamed to
D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\blobids\blob-9c615d51-e040-499d-bd1d
-3ba4ccd61dd1.1521376420981.gen
18:07:00.993 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Created new process file and writer over
D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\blobids\blob-9c615d
51-e040-499d-bd1d-3ba4ccd61dd1.1521376620993.gen.process
18:07:01.008 [pool-4-thread-1] DEBUG o.a.j.o.p.b.datastore.BlobIdTracker -
Merged files into references
[D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\blobids\blob-9c615d51-e040-499
d-bd1d-3ba4ccd61dd1.1521376420981.gen]
18:07:01.008 [pool-4-thread-1] DEBUG o.a.j.o.p.b.datastore.BlobIdTracker -
Completed snapshot in [16]
18:07:02.483 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Added blob id metadata record in DataStore in [1481]
18:07:02.483 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Deleted blob record file after snapshot and upload
D:\workspace\jackrabbit_oak\jackrabbit-oak-client\oak\repository\blobids\b
lob-9c615d51-e040-499d-bd1d-3ba4ccd61dd1.refs
18:07:02.483 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Updated snapshot marker
18:07:02.483 [pool-4-thread-1] INFO  o.a.j.o.p.b.datastore.BlobIdTracker -
Finished taking snapshot

I have highlighted the logs that shows that blob file is deleted but it
still remains in the S3 bucket.


Here is the config file I am using


*segmentmk config*

{
  "org.apache.jackrabbit.oak.segment.SegmentNodeStoreService" : {
    "customBlobStore" : true,
    "blobGcMaxAgeInSecs" : 300,
    "blobTrackSnapshotIntervalInSecs" : 200,
    "compaction.progressLog" : "-2L"
  }
}


I set the  "blobGcMaxAgeInSecs" to 5 minutes. Which one should i configure
to remove the file after removing the node.

I tried using the trunk version also. Still no progress on this.

Please assist me to fix this issue.

Thanks.




--
Sent from: http://jackrabbit.510166.n4.nabble.com/Jackrabbit-Users-f510167.html

Mime
View raw message