manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rafa Haro" <rharoapa...@gmail.com>
Subject Re: Runtime dependency issues in manifoldcf connector
Date Thu, 06 Aug 2015 10:37:09 GMT
Hi Dileepa, 




At [1] you can find a version of the Stanbol Client with several additions and improvements
above the version you are using right now. Among other things, it is not packaging it with
all the dependencies in a single jar anymore. Anyway, you would still need to use ant for
a proper deployment within ManifoldCF. If you are using Maven for creating a shaded jar and
including it in the connector-lib folder for testing, you will probably have the same error
again. Also at [2] you can find a branch which is using jax-rs 1.1 with the jersey-client
implementation which probably does not present that problem, but it is still using a bad packaging
approach.




Cheers,

Rafa




[1] - https://github.com/rafaharo/apache-stanbol-client

[2] - https://github.com/rafaharo/apache-stanbol-client/tree/jaxrs-1.0

On Thu, Aug 6, 2015 at 12:23 PM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Dileepa,
> If you bundle together your connector and its dependencies in a single jar,
> you must not override any dependencies that are in the core classloader
> area or ManifoldCF will not work properly.  Nor will your connector.
> MCF is built primarily by ant.  Look at any of the connectors we provide;
> there is a build.xml.  You can copy one of these and modify it according to
> your needs.  It should be relatively easy to do provided you have a full
> transitive list of dependencies available.
> Karl
> On Thu, Aug 6, 2015 at 6:17 AM, Dileepa Jayakody <djayakody@zaizi.com>
> wrote:
>> Hi Karl,
>>
>> Thanks for your response.
>>
>> On Thu, Aug 6, 2015 at 3:36 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>> > Where are you putting the new jars?
>> >
>>
>> > There are two different classloader levels in ManifoldCF: the core
>> > classloader level, and the connector classloader level.  Jars that only
>> > connectors depend on should be placed in the connector-lib directory.
>> Can
>> > you verify that all of your resteasy jars and their dependencies are in
>> > that directory?
>> >
>> >
>> I have bundled my connector with all the dependencies as a single jar using
>> maven assembly plugin.
>> I put that jar to connector-lib.
>>
>>
>> > It's also the case that sometimes a specific technology cannot work with
>> > the ManifoldCF classloader structure very well.  If that happens, we are
>> > usually forced to move the jar and its dependencies to the core level.
>> But
>> > before you presume that, let's figure out what in fact you are doing.
>> >
>> > If you are trying to run your connector with only a maven build, I would
>> > strongly encourage you not to do that.  Use the ant build instead, until
>> > you get things straightened out.
>> >
>> > I didn't get this part clearly. Are you suggesting to use Ant to build my
>> connector?
>> Can you please elaborate?
>>
>>
>> > Karl
>> >
>> > Thanks,
>> Dileepa
>>
>> >
>> > On Thu, Aug 6, 2015 at 5:11 AM, Dileepa Jayakody <djayakody@zaizi.com>
>> > wrote:
>> >
>> > > Hi All,
>> > >
>> > > I started developing a transformation connector to Apache ManifoldCF to
>> > > enhance repo-documents by connecting to Apache Stanbol. In my connector
>> > I'm
>> > > using Apache Stanbol Client version 1.0-Snapshot [1] which is using
>> > > RestEasy 3.0.8.Final as a dependency. When I invoke the Stanbol client
>> > in a
>> > > standalone java class (Main method) it works with excepted results, but
>> > > when I use it in my ManifoldCF connector with all dependencies
>> (including
>> > > RestEasy 3.0.8 related dependencies: ) I keep getting below error in
>> > > ManifoldCF runtime [2].
>> > > I think it's due to some cyclic dependency issue with ManifoldCF and
>> > > RestEasy 3.0.8 used in my component.
>> > > I tried out this in both ManifoldCF 1.8 and 2.1 both get the same
>> error.
>> > >
>> > > Can someone please advice on how I should resolve this dependency
>> issue?
>> > >
>> > > [1] https://github.com/zaizi/apache-stanbol-client
>> > >
>> > > [2] ERROR 2015-08-05 16:07:48,401 (Worker thread '4') - Error enhancing
>> > the
>> > > document  :
>> > > file:/Users/djayakody/Documents/zaizi/manifoldTest/simple/sample0.txt
>> > > javax.ws.rs.ProcessingException: Unable to invoke request
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:287)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:407)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.client.jaxrs.internal.ClientInvocationBuilder.post(ClientInvocationBuilder.java:195)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.stanbol.client.rest.RestClientExecutor.post(RestClientExecutor.java:63)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.stanbol.client.enhancer.impl.EnhancerImpl.enhance(EnhancerImpl.java:74)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.transformation.stanbol.StanbolEnhancer.addOrReplaceDocumentWithException(StanbolEnhancer.java:251)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3221)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3072)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3466)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:324)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3221)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3072)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2706)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1503)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1468)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:404)
>> > >     at
>> > >
>> > >
>> >
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:379)
>> > >
>> > > *Caused by: javax.ws.rs.ProcessingException: could not find writer for
>> > > content-type text/plain type: java.io.ByteArrayInputStream *   at
>> > >
>> > >
>> >
>> org.jboss.resteasy.core.interception.ClientWriterInterceptorContext.throwWriterNotFoundException(ClientWriterInterceptorContext.java:40)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.getWriter(AbstractWriterInterceptorContext.java:138)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:117)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.writeRequestBody(ClientInvocation.java:341)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.writeRequestBodyToOutputStream(ApacheHttpClient4Engine.java:558)
>> > >     at
>> > >
>> > >
>> >
>> org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.buildEntity(ApacheHttpClient4Engine.java:524)
>> > >
>> > > Thanks,
>> > > Dileepa
>> > >
>> > > --
>> > >
>> > > ------------------------------
>> > > This message should be regarded as confidential. If you have received
>> > this
>> > > email in error please notify the sender and destroy it immediately.
>> > > Statements of intent shall only become binding when confirmed in hard
>> > copy
>> > > by an authorised signatory.
>> > >
>> > > Zaizi Ltd is registered in England and Wales with the registration
>> number
>> > > 6440931. The Registered Office is Brook House, 229 Shepherds Bush Road,
>> > > London W6 7AN.
>> > >
>> >
>>
>> --
>>
>> ------------------------------
>> This message should be regarded as confidential. If you have received this
>> email in error please notify the sender and destroy it immediately.
>> Statements of intent shall only become binding when confirmed in hard copy
>> by an authorised signatory.
>>
>> Zaizi Ltd is registered in England and Wales with the registration number
>> 6440931. The Registered Office is Brook House, 229 Shepherds Bush Road,
>> London W6 7AN.
>>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message