flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <till.rohrm...@gmail.com>
Subject Re: Lucene SPI class loading fails with shaded flink-connector-elasticsearch
Date Thu, 22 Mar 2018 16:42:41 GMT
Hi Manuel,

thanks for reporting this issue. It sounds to me like a bug we should fix.
I've pulled Gordon into the conversation since he will most likely know
more about the ElasticSearch connector shading.

Cheers,
Till

On Thu, Mar 22, 2018 at 5:09 PM, Haddadi Manuel <manuel.haddadi@gfi.fr>
wrote:

> Hello,
>
> When upgrading from flink-1.3.2 to flink-1.4.2, I faced this error on
> runtime of a Flink job :
>
> java.util.ServiceConfigurationError: An SPI class of type
> org.apache.lucene.codecs.PostingsFormat with classname
> org.apache.lucene.search.suggest.document.Completion50PostingsFormat does
> not exist, please fix the file 'META-INF/services/org.apache.lucene.codecs.PostingsFormat'
> in your classpath.
>
> I added lucene-suggest dependency and then I encountered this :
> java.lang.ClassCastException: class org.elasticsearch.search.
> suggest.completion2x.Completion090PostingsFormat
>
> The Flink job runs Lucene queries on a data stream which ends up in an
> Elasticsearch index.
>
> It seems to me that this exception is a side effect of shading
> flink-connector-elasticsearch-5 dependencies. Actually, the only solution
> I have found is to re-build flink-connector-elasticsearch-5 jar excluding
> META-INF/services/org.apache.lucene.codecs.*
>
> I would highly appreciate any opinion on this workaround. Could it have
> side effect ?
>
> Thanks. And by the way, congrats to all Flink contributors, this is a
> pretty good piece of technology !
>
> Regards,
>
> Manuel Haddadi
>
>

Mime
View raw message