jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Yuen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-4933) Create a data store implementation that integrates with Microsoft Azure Blob Storage
Date Mon, 04 Sep 2017 10:24:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152441#comment-16152441
] 

Nicolas Yuen commented on OAK-4933:
-----------------------------------

Hi,

Is the AzureDataStore implementation fully functional? I tried to use it with AEM6.3 + August
FP + Oak 1.6.3 + AzureDataStore Plugin (https://repo.adobe.com/nexus/content/groups/public/com/adobe/granite/com.adobe.granite.oak.azureblobconnector/1.6.3/)
and launched aem with java -jar cq-quickstart-6.3.0.jar -r crx3tar-nofds but it doesn't seem
functional

Here is my configuration file located in crx-quickstart/install with obfuscated accounts/key,
I escaped the last == to \=\=. Note that I tried both with access/secret and endpoint/SAS
token

accessKey="myaemblobaccount"
secretKey="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\=\="
secretKey="d5q5JXk1J0jSwxPSRG5i3skaD4aq4hAQpUhlsHyLVy/20V1/T0ywA5o3dqxSlTPXXgWWbz61679UlypaZWjNjQ\=\="
container="aem"
maxConnections="4"
maxErrorRetry="10"
socketTimeout="120000"

below is the exception I have at startup if I install the Cummulative pack + Oak through the
Package Manager and then put the azure blob connector + config in the install folder after
a shutdown:

1.09.2017 16:42:37.693 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak-blob-cloud-azure
BundleEvent RESOLVED
01.09.2017 16:42:37.694 *INFO* [FelixStartLevel] com.fasterxml.jackson.core.jackson-core BundleEvent
RESOLVED
01.09.2017 16:42:37.694 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak-blob-cloud-azure
BundleEvent STARTING
01.09.2017 16:42:37.694 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak-blob-cloud-azure
BundleEvent STARTED
01.09.2017 16:42:37.698 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak-blob-cloud-azure
Service [904, [org.osgi.service.cm.ManagedService]] ServiceEvent REGISTERED
01.09.2017 16:42:37.722 *ERROR* [FelixStartLevel] org.apache.jackrabbit.oak-core [org.apache.jackrabbit.oak.plugins.blob.ConsolidatedDataStoreCacheStats(349)]
The activate method has thrown an exception (java.lang.
NullPointerException)
java.lang.NullPointerException: null
	at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.getStats(AbstractSharedCachingDataStore.java:364)
	at org.apache.jackrabbit.oak.plugins.blob.ConsolidatedDataStoreCacheStats.activate(ConsolidatedDataStoreCacheStats.java:94)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.felix.scr.impl.inject.BaseMethod.invokeMethod(BaseMethod.java:224)
	at org.apache.felix.scr.impl.inject.BaseMethod.access$500(BaseMethod.java:39)
	at org.apache.felix.scr.impl.inject.BaseMethod$Resolved.invoke(BaseMethod.java:617)
	at org.apache.felix.scr.impl.inject.BaseMethod.invoke(BaseMethod.java:501)
	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:302)
	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:294)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:298)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:109)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:906)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:879)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:749)
	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:1012)
	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:968)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1215)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1136)
	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.trackAdding(ServiceTracker.java:945)
	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.track(ServiceTracker.java:881)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1167)
	at org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.serviceChanged(BundleComponentActivator.java:127)
	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:991)
	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:839)
	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:546)
	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4558)
	at org.apache.felix.framework.Felix.registerService(Felix.java:3550)
	at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:348)
	at org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AbstractAzureDataStoreService.createDataStore(AbstractAzureDataStoreService.java:53)
	at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:79)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.felix.scr.impl.inject.BaseMethod.invokeMethod(BaseMethod.java:224)
	at org.apache.felix.scr.impl.inject.BaseMethod.access$500(BaseMethod.java:39)
	at org.apache.felix.scr.impl.inject.BaseMethod$Resolved.invoke(BaseMethod.java:617)
	at org.apache.felix.scr.impl.inject.BaseMethod.invoke(BaseMethod.java:501)
	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:302)
	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:294)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:298)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:109)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:906)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:879)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:749)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:675)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:430)
	at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:657)
	at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:341)
	at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:390)
	at org.apache.felix.scr.impl.Activator.access$200(Activator.java:54)
	at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:265)
	at org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
	at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
	at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
	at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:916)
	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:835)
	at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:517)
	at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4542)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2173)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
	at java.lang.Thread.run(Thread.java:748)
01.09.2017 16:42:37.724 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak-blob-cloud-azure
Service [org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzureDataStore,905, [org.apache.jackrabbit.oak.plugins.blo
b.AbstractSharedCachingDataStore, org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore]]
ServiceEvent REGISTERED

If I work on a clean install with Oak 1.6.3 + Azure Blob + config on crx-quickstart/install
from a fresh -unpack instance half of the bundles do not start (no stack trace)


> Create a data store implementation that integrates with Microsoft Azure Blob Storage
> ------------------------------------------------------------------------------------
>
>                 Key: OAK-4933
>                 URL: https://issues.apache.org/jira/browse/OAK-4933
>             Project: Jackrabbit Oak
>          Issue Type: Epic
>          Components: blob, core
>            Reporter: Matt Ryan
>            Assignee: Amit Jain
>             Fix For: 1.7.0, 1.8, 1.6.2
>
>         Attachments: full-patch-1.6.patch, OAK-4933-v5.patch
>
>
> This epic proposes the creation of a new type of Oak data store, AzureDataStore, that
offers an integration between Oak and Microsoft Azure Blob Storage.  AzureDataStore would
be very similar in purpose and functionality to S3DataStore, with a different backend that
uses Azure Blob Storage instead of S3.
> Some initial exploration into this concept can be seen in my github here:  https://github.com/mattvryan/jackrabbit-oak/tree/azure-blob-store
> More info about Azure Blob Storage:
> * [Java SDK|https://azure.microsoft.com/en-us/documentation/articles/storage-java-how-to-use-blob-storage/]
> * [Javadoc|http://azure.github.io/azure-storage-java/]
> * [Source|https://github.com/azure/azure-storage-java] (GitHub) - Microsoft's Azure Storage
Java SDK is open source and Apache licensed
> * [Package info|https://mvnrepository.com/artifact/com.microsoft.azure/azure-storage]
on mvnrepository.com
> As I see it, the following work would be required:
> * Create an AzureDataStore class that extends CachingDataStore
> * Create a new backend for Azure Blob Storage
> * Comprehensive unit testing of the new data store and backend classes
> * Create test "mocks" for the necessary Azure Storage classes to facilitate unit testing
(they are all final classes and cannot be mocked directly)
> * Create SharedAzureDataStore class
> * Create AzureDataStoreService class
> * Implement similar JMX metrics as exist for S3DataStore
> * Combine and refactor existing Oak code with newly added code to make best reuse of
existing code, etc.
> * Integration testing with system configured with AzureDataStore, comparison w/ S3DataStore
in terms of performance and correctness
> * Modify Azure Storage SDK code to make it into a valid OSGi bundle, and submit these
changes back to that project (currently it is not an OSGi bundle and therefore currently has
to be embedded)
> List isn't purported to be comprehensive of course.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message