flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haddadi Manuel <manuel.hadd...@gfi.fr>
Subject Lucene SPI class loading fails with shaded flink-connector-elasticsearch
Date Thu, 22 Mar 2018 16:09:04 GMT

When upgrading from flink-1.3.2 to flink-1.4.2, I faced this error on runtime of a Flink job

java.util.ServiceConfigurationError: An SPI class of type org.apache.lucene.codecs.PostingsFormat
with classname org.apache.lucene.search.suggest.document.Completion50PostingsFormat does not
exist, please fix the file 'META-INF/services/org.apache.lucene.codecs.PostingsFormat' in
your classpath.

I added lucene-suggest dependency and then I encountered this :
java.lang.ClassCastException: class org.elasticsearch.search.suggest.completion2x.Completion090PostingsFormat

The Flink job runs Lucene queries on a data stream which ends up in an Elasticsearch index.

It seems to me that this exception is a side effect of shading flink-connector-elasticsearch-5
dependencies. Actually, the only solution I have found is to re-build flink-connector-elasticsearch-5
jar excluding META-INF/services/org.apache.lucene.codecs.*

I would highly appreciate any opinion on this workaround. Could it have side effect ?

Thanks. And by the way, congrats to all Flink contributors, this is a pretty good piece of
technology !


Manuel Haddadi

View raw message