manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject RE: Connector 1003 CMIS
Date Sun, 24 Aug 2014 05:39:47 GMT
Ok, the problem is because you only get to write the seeding query.  The
query that fetches individual documents is hardwired.  I believe it is set
in opencmis in fact.

It may be possible to do this a different way, but it would require a
rewrite of the connector.  But I will think about it.

Karl

Sent from my Windows Phone
------------------------------
From: Karl Wright
Sent: 8/22/2014 11:05 AM
To: user@manifoldcf.apache.org
Subject: Re: Connector 1003 CMIS

Please be sure to run "ant make-core-deps" since that downloads the newest
OpenCMIS jars.

Karl



On Fri, Aug 22, 2014 at 11:03 AM, Baptiste Berthier <ba.berthier@gmail.com>
wrote:

> Thanks. I will try it
>
> Baptiste
>
> 2014-08-22 15:51 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>
>   Hi Baptiste,
>>
>> Trunk has been updated to use OpenCMIS 0.11, which may fix your problem.
>> Please give it a try.
>>
>> Thanks,
>> Karl
>>
>>
>>
>> On Fri, Aug 22, 2014 at 9:34 AM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>>  Hi Baptiste,
>>>
>>> The CMIS connector depends on Apache Chemistry as its client software.
>>> It is possible that either we are behind the release cycle, or there are
>>> bugs, in the version of Chemistry that we use.  There is a ticket open for
>>> upgrade; I will see if this can be done painlessly, and if so let's see if
>>> that helps.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Fri, Aug 22, 2014 at 9:26 AM, Baptiste Berthier <
>>> ba.berthier@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have tested multiple CMIS request. I attach a document to show the
>>>> results of my test cause I don't manage to index all internal metadata from
>>>> Alfresco and I don't know why.
>>>>
>>>> thanks for your help
>>>>
>>>> Baptiste
>>>>
>>>> 2014-08-21 13:46 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>
>>>>  Hi Baptiste,
>>>>>
>>>>> The CMIS connector passes the query intact to the CMIS
>>>>> implementation.  If it sees "*" as any of the fields in the select, it
>>>>> takes all metadata items it finds in the resultset.  If the "select *"
is
>>>>> somehow canceling the join, then try "select d.*,e.*,*".  If that doesn't
>>>>> work, then document.getProperties() is simply not returning everything.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Aug 21, 2014 at 7:15 AM, Baptiste Berthier <
>>>>> ba.berthier@gmail.com> wrote:
>>>>>
>>>>>> I tried it and the fields title or description aren't indexed. I
have
>>>>>> verified the result in Solr log message and there is not the title
field or
>>>>>> the description field while they appeared when I was testing the
query in
>>>>>> Chrome extension. I don't know why.
>>>>>>  Baptiste
>>>>>> 2014-08-21 12:25 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>>>
>>>>>>  Hi Baptiste,
>>>>>>>
>>>>>>> Instead of "d.*,e.*", try just "*".  The column selection in
the
>>>>>>> CMIS connector is not very smart.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 21, 2014 at 3:18 AM, Baptiste Berthier <
>>>>>>> ba.berthier@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> In order to get the field that I want (for example description,
>>>>>>>> title) with CMIS connector, I changed my CMIS request by
using a join. Here
>>>>>>>> is the CMIS request :
>>>>>>>>
>>>>>>>> SELECT d.*, e.* FROM cmis:document as d JOIN cmis:titled
e ON
>>>>>>>> d.cmis:objectId = e.cmis:objectId WHERE d.cmis:name LIKE
'doc.pdf'
>>>>>>>>
>>>>>>>> I have verified the result in Chrome and there is well the
content
>>>>>>>> of the description and title field. You can see it in the
attach document.
>>>>>>>> So I have written the request in MCF and launched crawling
but
>>>>>>>> nothing appears to be indexed in Solr. Do you think that
that MCF supports
>>>>>>>> CMIS request with join ?
>>>>>>>>
>>>>>>>> Thanks for your reply. It helps me a lot in my work.
>>>>>>>>
>>>>>>>> Baptiste
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-08-20 15:46 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>>>>>
>>>>>>>>   Hi Baptiste,
>>>>>>>>>
>>>>>>>>> The current Alfresco connector will only work on older
versions of
>>>>>>>>> Alfresco.  It will not work, for example, on Alfresco
One.  We are trying
>>>>>>>>> to field an alfresco connector based on the REST API,
but that API has not
>>>>>>>>> apparently stabilized yet.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Aug 20, 2014 at 9:41 AM, Baptiste Berthier <
>>>>>>>>> ba.berthier@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks a lot ! Indeed It works fine like that. To
avoid to modify
>>>>>>>>>> the Alfresco's CMIS implementation, do you think,
that I can index all the
>>>>>>>>>> metadata that I want, by using the Alfresco connector
instead of CMIS
>>>>>>>>>> connector ?
>>>>>>>>>>
>>>>>>>>>> Thanks a lot ! It works fine. I noticed that the
properties of
>>>>>>>>>> the document and certains Alfresco's metadata are
all indexes by Solr under
>>>>>>>>>> different field's name.
>>>>>>>>>>
>>>>>>>>>> Do you think, that it's possible to avoid to change
the
>>>>>>>>>> Alfresco's CMIS implementation by using the existing
Alfresco connector
>>>>>>>>>> instead of using CMIS connector ?
>>>>>>>>>>
>>>>>>>>>> Baptiste
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2014-08-20 12:06 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>   Hi Baptiste,
>>>>>>>>>>>
>>>>>>>>>>> All the metadata names provided by the Alfresco
CMIS
>>>>>>>>>>> implementation seem to be prefixed with "cmis:".
 For example,
>>>>>>>>>>> "cmis:createdBy" has a value of "admin".  If
you change your MCF mappings
>>>>>>>>>>> and/or Solr configuration to recognize these
attribute names, they will be
>>>>>>>>>>> indexed.
>>>>>>>>>>>
>>>>>>>>>>> It does not appear that description or title
are metadata that
>>>>>>>>>>> Alfresco's CMIS implementation includes.  ManifoldCF
cannot fix that
>>>>>>>>>>> because it is likely that it is Alfresco's implementation
that is
>>>>>>>>>>> incomplete.
>>>>>>>>>>>
>>>>>>>>>>> Karl
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Aug 20, 2014 at 5:39 AM, Baptiste Berthier
<
>>>>>>>>>>> ba.berthier@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> I attached my Solr log messages. It seems
that there is the id,
>>>>>>>>>>>> the name, the mime type as metadata but nothing
about the description or
>>>>>>>>>>>> the title that we could find in Alfresco.
I don't know why. I would like to
>>>>>>>>>>>> know what is the difference between Schema
section in List output connector
>>>>>>>>>>>> and the Solr field mapping when I configure
a job. It seems that the Solr
>>>>>>>>>>>> field mapping doesn't enable me to match
field from Alfresco with Solr
>>>>>>>>>>>> field.
>>>>>>>>>>>> I will be interesting in the MCF 1.7 RC2
when it will be
>>>>>>>>>>>> available.
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> Baptiste
>>>>>>>>>>>> 2014-08-19 13:27 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>   Hi Baptiste,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe the problem is in how you have
declared your Solr
>>>>>>>>>>>>> schema.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you look at the Solr log messages,
the INFO statements show
>>>>>>>>>>>>> the information that ManifoldCF is sending
to Solr.  If that includes
>>>>>>>>>>>>> everything you are expecting, then the
problem is on the Solr side.  If
>>>>>>>>>>>>> not, it is on the ManifoldCF side.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, I've recently fixed yet another
issue with the CMIS
>>>>>>>>>>>>> connector, so you will want MCF 1.7 RC2
when it is available.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 19, 2014 at 7:10 AM, Baptiste
Berthier <
>>>>>>>>>>>>> ba.berthier@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks a lot for the link that you
sent me last day. I
>>>>>>>>>>>>>> installed and tested MCF 1.7. I met
some problems. I try to explain them in
>>>>>>>>>>>>>> the attached document.
>>>>>>>>>>>>>> Do you have an idea about the problem
that I have ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Baptiste
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2014-08-12 8:11 GMT+02:00 Karl Wright
<daddywri@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>   Hi Baptiste,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can download MCF 1.7 RC0
from:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://people.apache.org/~mingchun/apache-manifoldcf-1.7-RC0/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please let me know what happens.
>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Aug 11, 2014 at 8:45
AM, Baptiste Berthier <
>>>>>>>>>>>>>>> ba.berthier@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am using Apache ManifoldCF
1.6.1 but if the version 1.7
>>>>>>>>>>>>>>>> is available in one or two
days. I can wait.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>> Baptiste
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2014-08-11 13:26 GMT+02:00
Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>   Hi Baptiste,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I strongly suggest waiting
for the first ManifoldCF 1.7
>>>>>>>>>>>>>>>>> release candidate to
be available for download.  This should be available
>>>>>>>>>>>>>>>>> within a day or two.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The alternative is that
I can attach a full patch to the
>>>>>>>>>>>>>>>>> ticket from whatever
version you are using to current trunk.  Please let me
>>>>>>>>>>>>>>>>> know what your version
is, first.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Aug 11, 2014
at 6:46 AM, Baptiste Berthier <
>>>>>>>>>>>>>>>>> ba.berthier@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks a lot. It
works fine ! I have just one last
>>>>>>>>>>>>>>>>>> question concerning
the CMIS connector used with Alfresco to get all
>>>>>>>>>>>>>>>>>> internal metadata
affected to a document by Alfresco. I used this kind of
>>>>>>>>>>>>>>>>>> cmis query : select
* from cmis:document where cmis:name in ('doc.pdf')
>>>>>>>>>>>>>>>>>> which doesn't work.
So you advised me to use trunk to commit my application
>>>>>>>>>>>>>>>>>> in order to be able
to use the query but I cannot use trunk because of the
>>>>>>>>>>>>>>>>>> internet restriction
in my enterprise.
>>>>>>>>>>>>>>>>>> So I'd like to know
if it exists another query expression
>>>>>>>>>>>>>>>>>> that enables me to
do what I want.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Baptiste
>>>>>>>>>>>>>>>>>> 2014-08-11 9:53 GMT+02:00
Karl Wright <daddywri@gmail.com
>>>>>>>>>>>>>>>>>> >:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>   Hi Baptiste,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>>>>>> http://manifoldcf.apache.org/release/trunk/en_US/how-to-build-and-deploy.html#Configuring+a+MySQL+database
>>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The instructions
are similar to configuration of
>>>>>>>>>>>>>>>>>>> PostgreSQL, except
that the properties.xml property
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.databaseimplementationclass
should be set to
>>>>>>>>>>>>>>>>>>> "org.apache.manifoldcf.core.database.DBInterfaceMySQL".
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Aug 11,
2014 at 3:05 AM, Baptiste Berthier <
>>>>>>>>>>>>>>>>>>> ba.berthier@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks for
your answer. I have just one question. I
>>>>>>>>>>>>>>>>>>>> have configured
ManifoldCF to run on PostgreSQL but now I would like to run
>>>>>>>>>>>>>>>>>>>> it on MySQL.
I found a lot of documentation to configure it On Postgresql
>>>>>>>>>>>>>>>>>>>> but nothing
about the eventual possibility to configure it on MySQL. Is it
>>>>>>>>>>>>>>>>>>>> possible
to configure ManifoldCF on Mysql ?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Baptiste
>>>>>>>>>>>>>>>>>>>> 2014-08-08
14:10 GMT+02:00 Karl Wright <
>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com>:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>  Hi Baptiste,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> The patch
is not complete; other changes need to be
>>>>>>>>>>>>>>>>>>>>> made
as well to fix the ticket.  But if you are using trunk, you do not
>>>>>>>>>>>>>>>>>>>>> need
to apply any patch; it's already committed.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri,
Aug 8, 2014 at 7:36 AM, Baptiste Berthier <
>>>>>>>>>>>>>>>>>>>>> ba.berthier@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I
follow the answer of Karl Wright mentionned at
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/i#browse/CONNECTORS-1002
>>>>>>>>>>>>>>>>>>>>>> with
this content :
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> So
I understand that the internal metadata does not
>>>>>>>>>>>>>>>>>>>>>> make
it through from Alfresco into CMIS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The
CMIS connector working against an Alfresco
>>>>>>>>>>>>>>>>>>>>>> repository
depends completely on Alfresco's implementation of CMIS. It is
>>>>>>>>>>>>>>>>>>>>>> not
clear how complete Alfresco's implementation is. The CMIS connector
>>>>>>>>>>>>>>>>>>>>>> code
seems like it would definitely pick up most properties, PROVIDED they
>>>>>>>>>>>>>>>>>>>>>> are
explicitly mentioned in the CMIS query you provide:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> //properties
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> List<Property<?>>
properties =
>>>>>>>>>>>>>>>>>>>>>> document.getProperties();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> String
id = StringUtils.EMPTY;
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> for
(Property<?> property : properties) {
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> String
propertyId = property.getId();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> if(CmisRepositoryConnectorUtils.existsInSelectClause(cmisQuery,
>>>>>>>>>>>>>>>>>>>>>> propertyId)){
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> if
(propertyId.endsWith(Constants.PARAM_OBJECT_ID))
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> id
= (String) property.getValue();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 
This is from roughly line 1129 of
>>>>>>>>>>>>>>>>>>>>>> CMISRepositoryConnector.java.
If you can't figure out what is going on, you
>>>>>>>>>>>>>>>>>>>>>> can
modify this code to add System.out.println() statements that should
>>>>>>>>>>>>>>>>>>>>>> shed
some light on the problem.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Please
also note that the CMIS connector may not be
>>>>>>>>>>>>>>>>>>>>>> resilient
against changes to the query string. To be sure, you may want to
>>>>>>>>>>>>>>>>>>>>>> delete
your old job and create a new job with the changed query string, to
>>>>>>>>>>>>>>>>>>>>>> be
sure everything that needs to be recrawled actually gets reprocessed. I
>>>>>>>>>>>>>>>>>>>>>> will
create a different ticket to cover that issue.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I
have modified the code in ManifoldCF but when I
>>>>>>>>>>>>>>>>>>>>>> compile
with ant in the folder apache-manifoldcf-1.6.1\connector\cmis, I
>>>>>>>>>>>>>>>>>>>>>> have
the following error in
>>>>>>>>>>>>>>>>>>>>>> CMISRepositoryConnector.java
:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> "error
: cannot find symbol
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> [javac]
>>>>>>>>>>>>>>>>>>>>>> if(CmisRepositoryConnectorUtils.existsInSelectClause(cmisQuery,
>>>>>>>>>>>>>>>>>>>>>> propertyId)){
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> [javac]
symbol : variable cmisQuery
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> [javac]
location : class CmisRepositoryConnector
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> [javac]
1 error
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Build
failed
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I
don't know what I have to do to correct this error
>>>>>>>>>>>>>>>>>>>>>> which
appears during compilation.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks
for your answer
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Baptiste
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message