manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Runtime dependency issues in manifoldcf connector
Date Thu, 06 Aug 2015 10:23:09 GMT
Hi Dileepa,

If you bundle together your connector and its dependencies in a single jar,
you must not override any dependencies that are in the core classloader
area or ManifoldCF will not work properly.  Nor will your connector.

MCF is built primarily by ant.  Look at any of the connectors we provide;
there is a build.xml.  You can copy one of these and modify it according to
your needs.  It should be relatively easy to do provided you have a full
transitive list of dependencies available.

Karl


On Thu, Aug 6, 2015 at 6:17 AM, Dileepa Jayakody <djayakody@zaizi.com>
wrote:

> Hi Karl,
>
> Thanks for your response.
>
> On Thu, Aug 6, 2015 at 3:36 PM, Karl Wright <daddywri@gmail.com> wrote:
>
> > Where are you putting the new jars?
> >
>
> > There are two different classloader levels in ManifoldCF: the core
> > classloader level, and the connector classloader level.  Jars that only
> > connectors depend on should be placed in the connector-lib directory.
> Can
> > you verify that all of your resteasy jars and their dependencies are in
> > that directory?
> >
> >
> I have bundled my connector with all the dependencies as a single jar using
> maven assembly plugin.
> I put that jar to connector-lib.
>
>
> > It's also the case that sometimes a specific technology cannot work with
> > the ManifoldCF classloader structure very well.  If that happens, we are
> > usually forced to move the jar and its dependencies to the core level.
> But
> > before you presume that, let's figure out what in fact you are doing.
> >
> > If you are trying to run your connector with only a maven build, I would
> > strongly encourage you not to do that.  Use the ant build instead, until
> > you get things straightened out.
> >
> > I didn't get this part clearly. Are you suggesting to use Ant to build my
> connector?
> Can you please elaborate?
>
>
> > Karl
> >
> > Thanks,
> Dileepa
>
> >
> > On Thu, Aug 6, 2015 at 5:11 AM, Dileepa Jayakody <djayakody@zaizi.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > I started developing a transformation connector to Apache ManifoldCF to
> > > enhance repo-documents by connecting to Apache Stanbol. In my connector
> > I'm
> > > using Apache Stanbol Client version 1.0-Snapshot [1] which is using
> > > RestEasy 3.0.8.Final as a dependency. When I invoke the Stanbol client
> > in a
> > > standalone java class (Main method) it works with excepted results, but
> > > when I use it in my ManifoldCF connector with all dependencies
> (including
> > > RestEasy 3.0.8 related dependencies: ) I keep getting below error in
> > > ManifoldCF runtime [2].
> > > I think it's due to some cyclic dependency issue with ManifoldCF and
> > > RestEasy 3.0.8 used in my component.
> > > I tried out this in both ManifoldCF 1.8 and 2.1 both get the same
> error.
> > >
> > > Can someone please advice on how I should resolve this dependency
> issue?
> > >
> > > [1] https://github.com/zaizi/apache-stanbol-client
> > >
> > > [2] ERROR 2015-08-05 16:07:48,401 (Worker thread '4') - Error enhancing
> > the
> > > document  :
> > > file:/Users/djayakody/Documents/zaizi/manifoldTest/simple/sample0.txt
> > > javax.ws.rs.ProcessingException: Unable to invoke request
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:287)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:407)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.client.jaxrs.internal.ClientInvocationBuilder.post(ClientInvocationBuilder.java:195)
> > >     at
> > >
> > >
> >
> org.apache.stanbol.client.rest.RestClientExecutor.post(RestClientExecutor.java:63)
> > >     at
> > >
> > >
> >
> org.apache.stanbol.client.enhancer.impl.EnhancerImpl.enhance(EnhancerImpl.java:74)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.transformation.stanbol.StanbolEnhancer.addOrReplaceDocumentWithException(StanbolEnhancer.java:251)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3221)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3072)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3466)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:324)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3221)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3072)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2706)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1503)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1468)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:404)
> > >     at
> > >
> > >
> >
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:379)
> > >
> > > *Caused by: javax.ws.rs.ProcessingException: could not find writer for
> > > content-type text/plain type: java.io.ByteArrayInputStream *   at
> > >
> > >
> >
> org.jboss.resteasy.core.interception.ClientWriterInterceptorContext.throwWriterNotFoundException(ClientWriterInterceptorContext.java:40)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.getWriter(AbstractWriterInterceptorContext.java:138)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:117)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.writeRequestBody(ClientInvocation.java:341)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.writeRequestBodyToOutputStream(ApacheHttpClient4Engine.java:558)
> > >     at
> > >
> > >
> >
> org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.buildEntity(ApacheHttpClient4Engine.java:524)
> > >
> > > Thanks,
> > > Dileepa
> > >
> > > --
> > >
> > > ------------------------------
> > > This message should be regarded as confidential. If you have received
> > this
> > > email in error please notify the sender and destroy it immediately.
> > > Statements of intent shall only become binding when confirmed in hard
> > copy
> > > by an authorised signatory.
> > >
> > > Zaizi Ltd is registered in England and Wales with the registration
> number
> > > 6440931. The Registered Office is Brook House, 229 Shepherds Bush Road,
> > > London W6 7AN.
> > >
> >
>
> --
>
> ------------------------------
> This message should be regarded as confidential. If you have received this
> email in error please notify the sender and destroy it immediately.
> Statements of intent shall only become binding when confirmed in hard copy
> by an authorised signatory.
>
> Zaizi Ltd is registered in England and Wales with the registration number
> 6440931. The Registered Office is Brook House, 229 Shepherds Bush Road,
> London W6 7AN.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message