manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Has anyone successfully populated Elastic Search properties from custom SharePoint Meta data fields?
Date Tue, 30 Aug 2016 14:21:43 GMT
Hi Mike,

If you are just trying things out you can use the single-process example
and it will make your life easier for experimentation.

As far as the ES plugin, there are two: one for ES up to 1.5, and one for
1.5 onward.  Which did you try to install?

Karl


On Tue, Aug 30, 2016 at 9:56 AM, Holtz, Michael (IT Consultant) <
MHoltz@consultantemail.com> wrote:

> Karl,
>
>
>
> You are correct.
>
>
>
> Before I setup zookeeper I would like to prove out a few use cases.
>
>
>
> 1.       Index custom meta data fields from SharePoint into the
> ElasticSearch index
>
> 2.       Full text indexing of SharePoint content in ElasticSearch
>
> 3.       SharePoint security information indexed in ElasticSearch
>
> 4.       ElasticSearch Plugin to enforce SharePoint security settings for
> search
>
>
>
> At this point I’ve only managed to see #2 and #3 work
>
>
>
> For #4 I was getting an error when installing the ElasticSearch plugin.
>
> I would like to see #1 work before I do anything else. Are there any
> example configurations that show how to do this?
>
>
>
>
>
> Thanks,
>
> Mike H.
>
>
>
> *From:* Karl Wright [mailto:daddywri@gmail.com]
> *Sent:* Tuesday, August 30, 2016 9:49 AM
>
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Has anyone successfully populated Elastic Search
> properties from custom SharePoint Meta data fields?
>
>
>
> Hi Mike,
>
>
>
> This sounds like you're having a lock problem.
>
>
>
> Let me guess: you are using the multiprocess example with file-based
> locking?  If so, you need to stop everything and do the "lock clean"
> procedure.  I would also strongly recommend that you switch to the
> zookeeper multiprocess example.
>
>
>
> Thanks,
>
> Karl
>
>
>
>
>
> On Tue, Aug 30, 2016 at 9:44 AM, Holtz, Michael (IT Consultant) <
> MHoltz@consultantemail.com> wrote:
>
> I currently cannot view the job. The UI is hanging when I click the “view”
> link for the job. I’m wondering if that has anything to do with the
> “Metadata Adjuster” I added?
>
>
>
> I did not schedule the job.
>
>
>
> At this point the job has finished running and the Document Status and
> Queue status reports. I do not see any errors reported there.
>
>
>
> Also I still do not see my custom meta data fields from SharePoint in the
> ElasticSearch index.
>
>
>
> We are hoping to use this in production, but at this point it is not
> looking too good.
>
>
>
> Thanks,
>
> Mike H.
>
>
>
> *From:* Karl Wright [mailto:daddywri@gmail.com]
> *Sent:* Tuesday, August 30, 2016 9:09 AM
>
>
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Has anyone successfully populated Elastic Search
> properties from custom SharePoint Meta data fields?
>
>
>
> It is also important to know whether you have your job set up as
> "continuous".  If so, it will run forever.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Aug 30, 2016 at 8:48 AM, Furkan KAMACI <furkankamaci@gmail.com>
> wrote:
>
> Hi Mike,
>
>
>
> Could you tell us what "Document Status" and "Queue Status" says as Karl
> mentioned?
>
>
>
> Kind Regards,
>
> Furkan KAMACI
>
>
>
> On Tue, Aug 30, 2016 at 3:39 PM, Holtz, Michael (IT Consultant) <
> MHoltz@consultantemail.com> wrote:
>
> I’m looking at manifoldcf.log and I see no errors.
>
>
>
> I have only 13 files and they are all very tiny < 15kb .docx files.
>
>
>
> Mike H.
>
>
>
> *From:* Karl Wright [mailto:daddywri@gmail.com]
> *Sent:* Monday, August 29, 2016 5:26 PM
>
>
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Has anyone successfully populated Elastic Search
> properties from custom SharePoint Meta data fields?
>
>
>
> You should be looking at the ManifoldCF.log file, and the examples have
> logging for errors at least as being the default.
>
>
>
> The only other possibility is that you have a very large document that is
> taking a long time to transfer/index.
>
>
>
> Also, you can generally see what MCF is up to using the "document status"
> and "queue status" reports.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Aug 29, 2016 at 5:22 PM, Holtz, Michael (IT Consultant) <
> MHoltz@consultantemail.com> wrote:
>
> Karl,
>
>
>
> I see no errors in the log.
>
>
>
> I probably need to configure the logging I think.
>
>
>
> Thanks,
>
> Mike H.
>
>
>
> *From:* Karl Wright [mailto:daddywri@gmail.com]
> *Sent:* Monday, August 29, 2016 5:20 PM
>
>
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Has anyone successfully populated Elastic Search
> properties from custom SharePoint Meta data fields?
>
>
>
> Hi Mike,
>
>
>
> Have a look in the log.  If there are errors there please send them to
> us.  Generally, "hangs" are due to MCF retrying documents that have errors.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Aug 29, 2016 at 5:18 PM, Holtz, Michael (IT Consultant) <
> MHoltz@consultantemail.com> wrote:
>
> Karl,
>
>
>
> Yes, it shows up. I include all meta-data.
>
>
>
> I’m no trying to use a MetaData adjuster like this
>
>
>
> Parameter
>
> field1                     “${Field 1}”
>
>
>
> And now I see “field1”=”” in Elastic Search.
>
>
>
> Also it seems each time I make a change the job will hang and I need to
> restart everything including ElasticSearch to get it to complete.
>
>
>
> Thanks,
>
> Mike H.
>
>
>
> *From:* Karl Wright [mailto:daddywri@gmail.com]
> *Sent:* Monday, August 29, 2016 5:13 PM
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Has anyone successfully populated Elastic Search
> properties from custom SharePoint Meta data fields?
>
>
>
> Hi Mike,
>
>
>
> If I were you I'd try to independently verify that the custom field shows
> up.  Do you see it appear in the pulldown for selecting metadata?  If not,
> it probably is not accessible through the inspection methods that give us
> access into SharePoint fields.
>
>
>
> If it appears there then we should look at what ES gets sent.  It's
> possible it appears but under a different field name.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Aug 29, 2016 at 5:01 PM, Holtz, Michael (IT Consultant) <
> MHoltz@consultantemail.com> wrote:
>
> Hi,
>
>
>
> I have created a custom content type with a custom field “Field 1”.
>
>
>
> I have configured the job to include all meta data.
>
>
>
> I do not see “Field 1” being populated in the Elastic Search index.
>
>
>
>
>
> Does anyone have any experience with this?
>
>
>
> Thanks in advance,
>
> Mike H
>
>
>
>
>
>
>
>
>
>
>
>
>

Mime
View raw message