flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Elasticsearch connector and number of shards
Date Thu, 25 Aug 2016 14:40:17 GMT
I've just added a JIRA improvement ticket for this (
https://issues.apache.org/jira/browse/FLINK-4491).

Best,
Flavio

On Wed, Jul 20, 2016 at 4:21 PM, Maximilian Michels <mxm@apache.org> wrote:

> The connector doesn't cover this use case. Through the API you need to
> use the IndicesAdminClient:
> https://www.elastic.co/guide/en/elasticsearch/client/java-
> api/current/java-admin-indices.html
>
> Otherwise Elasticsearch creates an index with shards automatically. We
> could add support for configuring shards in the future.
>
> On Mon, Jul 18, 2016 at 11:42 AM, Flavio Pompermaier
> <pompermaier@okkam.it> wrote:
> > Indeed, we've tried with the parameter index.number_of_shards but it
> didn't
> > work so I fear that this parameter is not handled by the current
> > implementation..am I wrong?
> >
> > On Mon, Jul 18, 2016 at 11:37 AM, Ufuk Celebi <uce@apache.org> wrote:
> >>
> >> I've never used the Elasticsearch sink, but the docs say:
> >>
> >> "Note how a Map of Strings is used to configure the Sink. The
> >> configuration keys are documented in the Elasticsearch
> >> documentationhere. Especially important is the cluster.name parameter
> >> that must correspond to the name of your cluster."
> >>
> >> The config keys for index creation are found here:
> >>
> >> https://www.elastic.co/guide/en/elasticsearch/reference/
> current/indices-create-index.html
> >>
> >> Does this help? If not, maybe @Aljoscha can chime in here.
> >>
> >> – Ufuk
> >>
> >>
> >> On Mon, Jul 18, 2016 at 11:23 AM, Flavio Pompermaier
> >> <pompermaier@okkam.it> wrote:
> >> > Hi to all,
> >> >
> >> > we tried to use the streaming ES connector of Flink 1.1-SNAPSHOT and
> we
> >> > wanted to set the number of shards when creating a new index. Is that
> >> > possible at the moment?
> >> >
> >> > Best,
> >> > Flavio
> >
> >
>

Mime
View raw message