Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id EF5A2200B4A for ; Wed, 20 Jul 2016 16:21:48 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id EDA2F160A64; Wed, 20 Jul 2016 14:21:48 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 40A2F160A5B for ; Wed, 20 Jul 2016 16:21:48 +0200 (CEST) Received: (qmail 87157 invoked by uid 500); 20 Jul 2016 14:21:47 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 87148 invoked by uid 99); 20 Jul 2016 14:21:47 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Jul 2016 14:21:47 +0000 Received: from mail-vk0-f46.google.com (mail-vk0-f46.google.com [209.85.213.46]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 070EC1A0055 for ; Wed, 20 Jul 2016 14:21:46 +0000 (UTC) Received: by mail-vk0-f46.google.com with SMTP id x130so71214361vkc.0 for ; Wed, 20 Jul 2016 07:21:46 -0700 (PDT) X-Gm-Message-State: ALyK8tKeqAdEHFpRknCNXj44Z/BkiW6SnwoFQvZAb272iZXyihtN1BaR6hJzxJAHQoGAMCK3bl8IdkGXpN/DcyaL X-Received: by 10.31.170.200 with SMTP id t191mr23692721vke.131.1469024505872; Wed, 20 Jul 2016 07:21:45 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.53.15 with HTTP; Wed, 20 Jul 2016 07:21:26 -0700 (PDT) In-Reply-To: References: From: Maximilian Michels Date: Wed, 20 Jul 2016 16:21:26 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Elasticsearch connector and number of shards To: "user@flink.apache.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable archived-at: Wed, 20 Jul 2016 14:21:49 -0000 The connector doesn't cover this use case. Through the API you need to use the IndicesAdminClient: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-= admin-indices.html Otherwise Elasticsearch creates an index with shards automatically. We could add support for configuring shards in the future. On Mon, Jul 18, 2016 at 11:42 AM, Flavio Pompermaier wrote: > Indeed, we've tried with the parameter index.number_of_shards but it didn= 't > work so I fear that this parameter is not handled by the current > implementation..am I wrong? > > On Mon, Jul 18, 2016 at 11:37 AM, Ufuk Celebi wrote: >> >> I've never used the Elasticsearch sink, but the docs say: >> >> "Note how a Map of Strings is used to configure the Sink. The >> configuration keys are documented in the Elasticsearch >> documentationhere. Especially important is the cluster.name parameter >> that must correspond to the name of your cluster." >> >> The config keys for index creation are found here: >> >> https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-= create-index.html >> >> Does this help? If not, maybe @Aljoscha can chime in here. >> >> =E2=80=93 Ufuk >> >> >> On Mon, Jul 18, 2016 at 11:23 AM, Flavio Pompermaier >> wrote: >> > Hi to all, >> > >> > we tried to use the streaming ES connector of Flink 1.1-SNAPSHOT and w= e >> > wanted to set the number of shards when creating a new index. Is that >> > possible at the moment? >> > >> > Best, >> > Flavio > >