flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3034) Redis SInk Connector
Date Thu, 31 Mar 2016 18:56:25 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220454#comment-15220454
] 

ASF GitHub Bot commented on FLINK-3034:
---------------------------------------

Github user mjsax commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1813#discussion_r58108235
  
    --- Diff: docs/apis/streaming/connectors/redis.md ---
    @@ -0,0 +1,172 @@
    +---
    +title: "Redis Connector"
    +
    +# Sub-level navigation
    +sub-nav-group: streaming
    +sub-nav-parent: connectors
    +sub-nav-pos: 6
    +sub-nav-title: Redis
    +---
    +<!--
    +Licensed to the Apache Software Foundation (ASF) under one
    +or more contributor license agreements.  See the NOTICE file
    +distributed with this work for additional information
    +regarding copyright ownership.  The ASF licenses this file
    +to you under the Apache License, Version 2.0 (the
    +"License"); you may not use this file except in compliance
    +with the License.  You may obtain a copy of the License at
    +
    +  http://www.apache.org/licenses/LICENSE-2.0
    +
    +Unless required by applicable law or agreed to in writing,
    +software distributed under the License is distributed on an
    +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    +KIND, either express or implied.  See the License for the
    +specific language governing permissions and limitations
    +under the License.
    +-->
    +
    +This connector provides a Sink that can write to
    +[Redis](http://redis.io/) and also can publish data to [Redis PubSub](http://redis.io/topics/pubsub).
To use this connector, add the
    +following dependency to your project:
    +{% highlight xml %}
    +<dependency>
    +  <groupId>org.apache.flink</groupId>
    +  <artifactId>flink-connector-redis{{ site.scala_version_suffix }}</artifactId>
    +  <version>{{site.version }}</version>
    +</dependency>
    +{% endhighlight %}
    +
    +Note that the streaming connectors are currently not part of the binary distribution.
See linking with them for cluster execution [here]({{site.baseurl}}/apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution).
    +
    +#### Installing Redis
    +Follow the instructions from the [Redis download page](http://redis.io/download).
    +
    +#### Redis Sink
    +A class providing an interface for sending data to Redis. 
    +The sink can use three different methods for communicating with different type of Redis
environments:
    +1. Single Redis Server
    +2. Redis Cluster
    +3. Redis Sentinel
    +
    +This code shows how to create a sink that communicate to a single redis server:
    +
    +<div class="codetabs" markdown="1">
    +<div data-lang="java" markdown="1">
    +{% highlight java %}
    +public static class RedisExampleMapper implements RedisMapper<Tuple2<String, String>>{
    +
    +    @Override
    +    public RedisDataTypeDescription getDataTypeDescription() {
    +        return new RedisDataTypeDescription(RedisDataType.HASH, "HASH_NAME");
    +    }
    +
    +    @Override
    +    public String getKeyFromData(Tuple2<String, String> data) {
    +        return data.f0;
    +    }
    +
    +    @Override
    +    public String getValueFromData(Tuple2<String, String> data) {
    +        return data.f1;
    +    }
    +}
    +JedisPoolConfig conf = new JedisPoolConfig.Builder().setHost("127.0.0.1").build();
    +
    +DataStream<String> stream = ...;
    +stream.addSink(new RedisSink<Tuple2<String, String>>(conf, new RedisExampleMapper());
    +{% endhighlight %}
    +</div>
    +<div data-lang="scala" markdown="1">
    +{% highlight scala %}
    +class RedisExampleMapper extends RedisMapper[(String, String)]{
    +  override def getDataTypeDescription: RedisDataTypeDescription = {
    +    new RedisDataTypeDescription(RedisDataType.HASH, "HASH_NAME")
    +  }
    +
    +  override def getKeyFromData(data: (String, String)): String = data._1
    +
    +  override def getValueFromData(data: (String, String)): String = data._2
    +}
    +val conf = new JedisPoolConfig.Builder().setHost("127.0.0.1").build()
    +stream.addSink(new RedisSink[(String, String)](conf, new RedisExampleMapper))
    +{% endhighlight %}
    +</div>
    +</div>
    +
    +This example code does the same, but for Redis Cluster:
    +
    +<div class="codetabs" markdown="1">
    +<div data-lang="java" markdown="1">
    +{% highlight java %}
    +
    +JedisPoolConfig conf = new JedisClusterConfig.Builder()
    +    .setNodes(new HashSet<InetSocketAddress>(Arrays.asList(new InetSocketAddress(5601)))).build();
    +
    +DataStream<String> stream = ...;
    +stream.addSink(new RedisSink<Tuple2<String, String>>(conf, new RedisExampleMapper());
    +{% endhighlight %}
    +</div>
    +<div data-lang="scala" markdown="1">
    +{% highlight scala %}
    +val conf = new JedisClusterConfig.Builder().setNodes(...).build()
    +stream.addSink(new RedisSink[(String, String)](conf, new RedisExampleMapper))
    +{% endhighlight %}
    +</div>
    +</div>
    +
    +This example shows when the Redis environment is with Sentinels:
    +
    +<div class="codetabs" markdown="1">
    +<div data-lang="java" markdown="1">
    +{% highlight java %}
    +
    +JedisSentinelConfig conf = new JedisSentinelConfig.Builder().setMasterName("master").setSentinels(...).build();
    +
    +DataStream<String> stream = ...;
    +stream.addSink(new RedisSink<Tuple2<String, String>>(conf, new RedisExampleMapper());
    +{% endhighlight %}
    +</div>
    +<div data-lang="scala" markdown="1">
    +{% highlight scala %}
    +val conf = new JedisSentinelConfig.Builder().setMasterName("master").setSentinels(...).build()
    +stream.addSink(new RedisSink[(String, String)](conf, new RedisExampleMapper))
    +{% endhighlight %}
    +</div>
    +</div>
    +
    +This section gives a description of all the available Data types and what redis command
used for that.
    --- End diff --
    
    Typo: Data types => data types


> Redis SInk Connector
> --------------------
>
>                 Key: FLINK-3034
>                 URL: https://issues.apache.org/jira/browse/FLINK-3034
>             Project: Flink
>          Issue Type: New Feature
>          Components: Streaming Connectors
>            Reporter: Matthias J. Sax
>            Assignee: Subhankar Biswas
>            Priority: Minor
>
> Flink does not provide a sink connector for Redis.
> See FLINK-3033



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message