manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piergiorgio Lucidi <piergiorgioluc...@gmail.com>
Subject Re: CMIS Connector - Tests
Date Thu, 04 Aug 2011 08:10:01 GMT
I found a solution to create the right configuration for the CMIS Repository
Connector in the integration test code, and now it works ;)

Chemistry guys are supporting me about an issue that I have found in the
OpenCMIS InMemory Repository that I would like to use in the integration
test for Manifold.

Now I'm finishing the integration test implementation using the public
Alfresco CMIS server. Then I can start to test this new version of the
InMemory Repository with the bugfix provided by Jens. And I have to follow
some useful suggestions provided by Florian.

I'll let you know soon about all these tasks.

Piergiorgio

---------- Forwarded message ----------
From: Jens Hübel <jhuebel@opentext.com>
Date: 2011/8/4
Subject: RE: CMIS and Lucene
To: dev@chemistry.apache.org


Hi Piergiorgio,

you are right. I could reproduce the problem. There is a bug in the query
implementation of the InMemory server if you query for predefined properties
like cmis:name, cmis:createdBy, etc. :-(

I have fixed this, so hopefully this works for you now if you take the
latest build. I have created CMIS-413 for this (
https://issues.apache.org/jira/browse/CMIS-414). Please reopen if you still
have issues.

One more note: Your test code is quite fragile if you use a fixed name for
your test document/folder. This implementation does not allow multiple
objects with the same name in one folder. This means unless you restart the
server your code will work only once. I recommend you using a random UUID as
name or something like that. In the default configuration the InMemory
creates a tree of document and folders by default. So it might not be
necessary to create objects at all (use the Workbench to take a look at
that). If you really need this reproducible behavior there is an option to
run the server using the local binding in the same Java VM. Then you can
restart the server with each test. The InMemory JUnit tests use this
mechanism. I do not recommend this however, because you bypass the whole
protocol layer for AtomPub/SOAP. This leaves many issues undetected a client
may see in a real connection later (and of course you lose the option to
switch to another CMIS server just by changing configuration).

Jens


-----Original Message-----
From: Piergiorgio Lucidi [mailto:piergiorgiolucidi@gmail.com]
Sent: Mittwoch, 3. August 2011 14:48
To: dev@chemistry.apache.org
Subject: Re: CMIS and Lucene

Hi Jens,

here the code of my integration test that is used to create some content in
the InMemory Repository, the OpenCMIS server is running because it is
bootstrapped by Jetty / Maven. The CMIS Repository Connector works fine with
Alfresco 3.4d Community, but I would like to have the OpenCMIS server in the
test suite.

In the following snippet I started to initialize the test environment with a
new folder with a new content, here I don't have any problem, it works with
the InMemory Repository:

 private Session getCmisClientSession(){
>     // default factory implementation
>     SessionFactory factory = SessionFactoryImpl.newInstance();
>     Map<String, String> parameters = new HashMap<String, String>();
>     // user credentials
>     parameters.put(SessionParameter.USER, "dummyuser");
>     parameters.put(SessionParameter.PASSWORD, "dummysecret");
>     // connection settings
>     parameters.put(SessionParameter.ATOMPUB_URL, CMIS_ENDPOINT_TEST_SERVER
> );
>     parameters.put(SessionParameter.BINDING_TYPE, BindingType.ATOMPUB
> .value());
>     // create session
>     return factory.getRepositories(parameters).get(0).createSession();
>   }
>  @Before
>   public void createTestArea()
>     throws Exception
>   {
>     try
>     {
>       Session session = getCmisClientSession();
>       //creating a new folder
>       Folder root = session.getRootFolder();
>       Map<String, Object> folderProperties = new HashMap<String,
> Object>();
>       folderProperties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
>       folderProperties.put(PropertyIds.NAME, "testdata");
>
>       Folder newFolder = root.createFolder(folderProperties);
>       //create a new content in the folder
>       String name = "testdata1.txt";
>       // properties
>       // (minimal set: name and object type id)
>       Map<String, Object> contentProperties = new HashMap<String,
> Object>();
>       contentProperties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:document");
>       contentProperties.put(PropertyIds.NAME, name);
>
>       // content
>       byte[] content = "CMIS Testdata One".getBytes();
>       InputStream stream = new ByteArrayInputStream(content);
>       ContentStream contentStream = new ContentStreamImpl(name,
newBigInteger(content),
> "text/plain", stream);
>
>        // create a major version
>       Document newContent1 = newFolder.createDocument(contentProperties,
> contentStream, null);


But if I try to search the new content in the InMemory Repository, in the
same way I implemented in the CMIS Repository Connector:

 ItemIterable<QueryResult> results = session.query("SELECT * FROM
> cmis:folder WHERE cmis:name='testdata'", false);
>       for (QueryResult result : results) {
>         String id = result.getPropertyValueById(PropertyIds.OBJECT_ID);
>       }



It returns the following exception:

 org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeException: null
> at
>
org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.convertStatusCode(
> AbstractAtomPubService.java:450)
> at
>
org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.post(
> AbstractAtomPubService.java:568)
> at
>
org.apache.chemistry.opencmis.client.bindings.spi.atompub.DiscoveryServiceImpl.query(
> DiscoveryServiceImpl.java:141)
> at org.apache.chemistry.opencmis.client.runtime.SessionImpl$3.fetchPage(
> SessionImpl.java:557)
> at
>
org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getCurrentPage(
> AbstractIterator.java:132)
> at
>
org.apache.chemistry.opencmis.client.runtime.util.CollectionIterator.hasNext(
> CollectionIterator.java:48)
> at org.apache.manifoldcf.cmis_tests.APISanityTest.createTestArea(
> APISanityTest.java:139)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(
> ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(
> FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.RunBefores.evaluate(
> RunBefores.java:27)
> at org.junit.internal.runners.statements.RunAfters.evaluate(
> RunAfters.java:31)
> at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(
> BlockJUnit4ClassRunner.java:79)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:71)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:49)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(
> JUnit4TestReference.java:49)
> at org.eclipse.jdt.internal.junit.runner.TestExecution.run(
> TestExecution.java:38)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(
> RemoteTestRunner.java:467)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(
> RemoteTestRunner.java:683)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(
> RemoteTestRunner.java:390)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
> RemoteTestRunner.java:197)


Let me know if you have any hints!

Thank you again for your support.

Regards,

Piergiorgio

2011/8/3 Jens Hübel <jhuebel@opentext.com>

> Hi Pergiorio,
>
> do you have a code piece that I can take a look at? What is your client
> API? Do you connect to Chemistry client API
(chemistry-opencmis-client-api)?
> Or perhaps you can grab the query sent from the log file?
>
> Thanks Jens
>
>
> -----Original Message-----
> From: Piergiorgio Lucidi [mailto:piergiorgiolucidi@gmail.com]
> Sent: Mittwoch, 3. August 2011 13:22
> To: dev@chemistry.apache.org
> Subject: Re: CMIS and Lucene
>
> Hi Jens,
>
> I contributed to the Apache ManifoldCF (ex Lucene Connector Framework)
> project implementing the CMIS Connector using OpenCMIS 0.4.0.
>
> Now I'm working on integration test trying to use the OpenCMIS InMemory
> Repository that is available from Apache Chemistry ;)
>
> I have some problems because now I'm finishing my test implementation but
> it
> seems that I can't execute queries on the InMemory Repository, it returns
a
> null string in the convertStatusCode method.
>
> The CMIS Repository Connector that I implemented for ManifoldCF has a job
> parameter that is the CMIS Query, this parameter is used by agents to
> select
> all the contents that needs to be indexed on the Output Connector (for
> example Apache Solr).
> Without a query feature exposed by the InMemory Repository I can't test
the
> connector in the right way.
>
> I tried to use the latest version of the InMemory Repository
> (0.5.0-SNAPSHOT) but with the same result.
>
> Have you got any ideas to solve this problem?
> Thank you for your support.
>
> Regards,
> Piergiorgio
>
>
> 2011/8/3 Jens Hübel <jhuebel@opentext.com>
>
> > Hi Chemistries,
> >
> >
> >
> > is anyone of us already involved in the Lucene project?
> >
> >
> >
> > Reading this here
> > http://search-lucene.com/m/wI38e1K3BWJ&subj=Re+CMIS+Connector+Tests
> >
> >
> >
> > I think it would make sense to give them an introduction to what we
have.
> >
> >
> >
> > if I understand this correctly one of the options they discuss is to
> create
> > an InMemory mock server. It does not make much sense to duplicate what
we
> > already have (and if we need getContentChanges() for that I am happy to
> > implement this).
> >
> >
> >
> > Jens
> >
> >
> >
> >
>
>
> --
> Piergiorgio Lucidi
> Web: http://about.me/piergiorgiolucidi
>



--
Piergiorgio Lucidi
Web: http://about.me/piergiorgiolucidi



-- 
Piergiorgio Lucidi
Web: http://about.me/piergiorgiolucidi

2011/8/3 Piergiorgio Lucidi <piergiorgiolucidi@gmail.com>

> Yes, this is a very quick way to test my configuration code for the
> integration test ;)
> I'm going to fix this part, I saw what I need in the JSON retrieved by a
> GET call against the Manifold API service.
>
> I'll let you know tomorrow an update about this.
>
> Anyway we have a problem with the OpenCMIS InMemory server. It seems that
> there is a problem during the execution of CMIS queries, I notified the
> problem to the Chemistry guys (Jens Hubel and Florian Muller) and they are
> trying to reproduce the issue to solve our problem.
>
> I'm going on developing the code using the public Alfresco CMIS server
> exposed at the following address:
> http://cmis.alfresco.com
>
> Piergiorgio
>
>
> 2011/8/3 Karl Wright <daddywri@gmail.com>
>
>> Another good way to see exactly what you need to do is to call the API
>> to get configuration information for an existing connection.  Then,
>> use the toXML() method to convert to XML, or the toJSON() to get it as
>> JSON.  Either way you will see the structure.  BTW, ManifoldCF in
>> Action Chapter 3 covers this in great detail as well.
>>
>> Karl
>>
>> On Wed, Aug 3, 2011 at 12:50 PM, Karl Wright <daddywri@gmail.com> wrote:
>> > The ConfigParams class is, I believe, derived from the Configuration
>> > class.  So, you can create a ConfigParams object instead of a
>> > Configuration object if you want to use the API in the manner you
>> > describe.
>> >
>> > The reason your commented-out code doesn't work is because the
>> > setParameter() method isn't doing quite what you are expecting.  It's
>> > creating a node named "_PARAMETER_" with a "name" attribute and a
>> > value area, and you are creating nodes named by the parameter name.
>> >
>> > Karl
>> >
>> > On Wed, Aug 3, 2011 at 12:41 PM, Piergiorgio Lucidi
>> > <piergiorgiolucidi@gmail.com> wrote:
>> >> I'm trying to implement tests but I found a problem to set all the
>> needed
>> >> parameters to the CMIS Repository Connector that needs: an username, a
>> >> password and the endpoint (url).
>> >>
>> >> I need to know how to create the configuration nodes for the connector,
>> in
>> >> the connector code I managed the configuration parameters in this way
>> in the
>> >> processConfigurationPost method:
>> >>
>> >>  public String processConfigurationPost(IThreadContext threadContext,
>> >>>       IPostParameters variableContext, ConfigParams parameters)
>> >>>       throws ManifoldCFException {
>> >>>     String username =
>> variableContext.getParameter(CONFIG_PARAM_USERNAME);
>> >>>     if (StringUtils.isNotEmpty(username))
>> >>>       parameters.setParameter(CONFIG_PARAM_USERNAME, username);
>> >>>     String password =
>> variableContext.getParameter(CONFIG_PARAM_PASSWORD);
>> >>>     if (StringUtils.isNotEmpty(password))
>> >>>       parameters.setParameter(CONFIG_PARAM_PASSWORD, password);
>> >>>     String endpoint =
>> variableContext.getParameter(CONFIG_PARAM_ENDPOINT);
>> >>>     if (StringUtils.isNotEmpty(endpoint) && endpoint.length()
> 0)
>> >>>       parameters.setParameter(CONFIG_PARAM_ENDPOINT, endpoint);
>> >>>     String repositoryId = variableContext
>> >>>         .getParameter(CONFIG_PARAM_REPOSITORY_ID);
>> >>>     if (StringUtils.isNotEmpty(repositoryId))
>> >>>       parameters.setParameter(CONFIG_PARAM_REPOSITORY_ID,
>> repositoryId);
>> >>>     return null;
>> >>>   }
>> >>
>> >>
>> >> Now I have to setup the same parameters inside my test class
>> APISanityTest
>> >> that doesn't like the following snippet, it works only if CMIS
>> parameters
>> >> are commented as the following:
>> >>
>> >>  @Test
>> >>>   public void sanityCheck()
>> >>>     throws Exception
>> >>>   {
>> >>>     try
>> >>>     {
>> >>>       // Hey, we were able to install the file system connector etc.
>> >>>       // Now, create a local test job and run it.
>> >>>       IThreadContext tc = ThreadContextFactory.make();
>> >>>       int i;
>> >>>       IJobManager jobManager = JobManagerFactory.make(tc);
>> >>>       // Create a basic file system connection, and save it.
>> >>>       ConfigurationNode connectionObject;
>> >>>       ConfigurationNode child;
>> >>>       Configuration requestObject;
>> >>>       Configuration result;
>> >>>
>> >>>       connectionObject = new
>> ConfigurationNode("repositoryconnection");
>> >>>
>> >>>       child = new ConfigurationNode("name");
>> >>>       child.setValue("CMIS Connection");
>> >>>
>> connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>>
>> >>>       child = new ConfigurationNode("class_name");
>> >>>       child.setValue(
>> >>>
>> "org.apache.manifoldcf.crawler.connectors.cmis.CmisRepositoryConnector");
>> >>>
>> connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>>
>> >>>       child = new ConfigurationNode("description");
>> >>>       child.setValue("CMIS Connection");
>> >>>
>> connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>>       child = new ConfigurationNode("max_connections");
>> >>>       child.setValue("10");
>> >>>
>> connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>>
>> >>>       //setting the CMIS specific parameters
>> >>> //      child = new ConfigurationNode("username");
>> >>> //      child.setValue(CMIS_USERNAME);
>> >>> //
>>  connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>> //
>> >>> //      child = new ConfigurationNode("password");
>> >>> //      child.setValue(CMIS_PASSWORD);
>> >>> //
>>  connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>> //
>> >>> //      child = new ConfigurationNode("endpoint");
>> >>> //      child.setValue(CMIS_ENDPOINT_TEST_SERVER);
>> >>> //
>>  connectionObject.addChild(connectionObject.getChildCount(),child);
>> >>>       requestObject = new Configuration();
>> >>>       requestObject.addChild(0,connectionObject);
>> >>>
>> >>>       result = performAPIPutOperationViaNodes(
>> >>> "repositoryconnections/CMIS%20Connection",201,requestObject);
>> >>
>> >>
>> >> How can I set the username, password and endpoint for the CMIS
>> Repository
>> >> Connector parameters in this test class?
>> >>
>> >> Thank you.
>> >>
>> >> Piergiorgio
>> >>
>> >>
>> >> 2011/8/2 Karl Wright <daddywri@gmail.com>
>> >>
>> >>> Thanks for the status report.  I hope to see your patch soon!
>> >>>
>> >>> Also, FWIW, once the documentation is also done I'd like to consider
>> >>> solidifying the 0.3 release.  It's got a lot of good stuff in it and
I
>> >>> think as soon as we've finished off the new CMIS connector in all
>> >>> dimensions we should go ahead.  Thoughts, anyone?
>> >>>
>> >>> Karl
>> >>>
>> >>>
>> >>> On Tue, Aug 2, 2011 at 5:00 AM, Piergiorgio Lucidi
>> >>> <piergiorgiolucidi@gmail.com> wrote:
>> >>> > Yesterday I started to work on end-to-end integration test for
the
>> CMIS
>> >>> > Connector and now I have a full running OpenCMIS test server
>> integrated
>> >>> with
>> >>> > the ManifoldCF Maven build process.
>> >>> >
>> >>> > Now I have to implement:
>> >>> > - a setup method to create the test documents in the CMIS server
>> >>> > - a null output connector using the ManifoldCF api
>> >>> > - tests using the ManifoldCF api to create a mock configuration
>> against
>> >>> the
>> >>> > test CMIS server
>> >>> >
>> >>> > I'll let you know when it works.
>> >>> >
>> >>> > Regards,
>> >>> > Piergiorgio
>> >>> >
>> >>> > 2011/7/29 Piergiorgio Lucidi <piergiorgiolucidi@gmail.com>
>> >>> >
>> >>> >> Hi Karl,
>> >>> >>
>> >>> >> thank you for the details and as soon as I finish a first version
>> of
>> >>> >> integration and/or unit test I will create a new ticket in
the CMIS
>> >>> >> Component to release the patch.
>> >>> >>
>> >>> >> I hope to release this new patch soon.
>> >>> >> I'll let you know during the next week.
>> >>> >>
>> >>> >> Piergiorgio
>> >>> >>
>> >>> >>
>> >>> >> 2011/7/28 Karl Wright <daddywri@gmail.com>
>> >>> >>
>> >>> >>> The tests/filesystem/src/test/java area are end-to-end
tests
>> >>> >>> principally designed to test the filesystem connector.
 (They also
>> >>> >>> involve other connectors because, obviously, an end-to-end
test is
>> not
>> >>> >>> going going to work without them.  This is why they are
at the
>> root
>> >>> >>> level.)
>> >>> >>>
>> >>> >>> There are also unit tests you can use as models under
>> >>> >>> connectors/filesystem/connector/src/test/....  These tests
use
>> only
>> >>> >>> the filesystem connector, no others.
>> >>> >>>
>> >>> >>> If you would be so kind as to create a ticket to cover
your test
>> work,
>> >>> >>> that would be great.
>> >>> >>>
>> >>> >>> Also, another workitem I've been meaning to get to but
haven't
>> >>> >>> involves the conversion of a python-based HTML browser
and
>> Javascript
>> >>> >>> engine I wrote for MetaCarta into a Java equivalent.  This
would
>> allow
>> >>> >>> direct testing of the UI components of the framework and
>> connectors.
>> >>> >>> The python sources are in:
>> >>> >>>
>> >>> >>> legacy-tests/VirtualBrowser.py
>> >>> >>> and
>> >>> >>> legacy-tests/Javascript.py
>> >>> >>>
>> >>> >>> You may not have time to get that far for a while, but
I thought
>> you'd
>> >>> >>> want to know where my thoughts were headed. ;-)
>> >>> >>>
>> >>> >>> Karl
>> >>> >>>
>> >>> >>> On Thu, Jul 28, 2011 at 2:03 PM, Piergiorgio Lucidi
>> >>> >>> <piergiorgiolucidi@gmail.com> wrote:
>> >>> >>> > I'm starting to think a way to implement tests for
the CMIS
>> >>> Connector,
>> >>> >>> > probably I can use the Apache Chemistry (OpenCMIS)
local binding
>> to
>> >>> >>> create a
>> >>> >>> > mock of a CMIS Server (an InMemory repository), but
I have to
>> check
>> >>> this
>> >>> >>> > possibility because I'm not sure that all the features
are now
>> >>> >>> implemented.
>> >>> >>> >
>> >>> >>> > Anyway I saw some test classes for the filesystem
connector at
>> the
>> >>> >>> following
>> >>> >>> > package:
>> >>> >>> >
>> >>> >>> > tests/filesystem/src/test/java
>> >>> >>> >
>> >>> >>> > Could you please confirm that these classes could
be taken as an
>> >>> example
>> >>> >>> for
>> >>> >>> > tests?
>> >>> >>> > Or do I need to take a look at other classes?
>> >>> >>> >
>> >>> >>> > Thank you.
>> >>> >>> > Piergiorgio
>> >>> >>> >
>> >>> >>> > --
>> >>> >>> > Piergiorgio Lucidi
>> >>> >>> > Web: http://about.me/piergiorgiolucidi
>> >>> >>> >
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Piergiorgio Lucidi
>> >>> >> Web: http://about.me/piergiorgiolucidi
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Piergiorgio Lucidi
>> >>> > Web: http://about.me/piergiorgiolucidi
>> >>> >
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Piergiorgio Lucidi
>> >> Web: http://about.me/piergiorgiolucidi
>> >>
>> >
>>
>
>
>
> --
> Piergiorgio Lucidi
> Web: http://about.me/piergiorgiolucidi
>
>


-- 
Piergiorgio Lucidi
Web: http://about.me/piergiorgiolucidi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message