flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-827) Add Java example program with KeySelectors
Date Thu, 03 Jul 2014 12:37:24 GMT

    [ https://issues.apache.org/jira/browse/FLINK-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051388#comment-14051388
] 

ASF GitHub Bot commented on FLINK-827:
--------------------------------------

Github user rmetzger commented on a diff in the pull request:

    https://github.com/apache/incubator-flink/pull/45#discussion_r14510808
  
    --- Diff: stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/wordcount/WordCountKeySelector.java
---
    @@ -0,0 +1,178 @@
    +/***********************************************************************************************************************
    + *
    + * Copyright (C) 2010-2013 by the Stratosphere project (http://stratosphere.eu)
    + *
    + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software distributed under
the License is distributed on
    + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the
    + * specific language governing permissions and limitations under the License.
    + *
    + **********************************************************************************************************************/
    +package eu.stratosphere.example.java.wordcount;
    +
    +import eu.stratosphere.api.java.DataSet;
    +import eu.stratosphere.api.java.ExecutionEnvironment;
    +import eu.stratosphere.api.java.functions.FlatMapFunction;
    +import eu.stratosphere.api.java.functions.KeySelector;
    +import eu.stratosphere.api.java.functions.ReduceFunction;
    +import eu.stratosphere.util.Collector;
    +
    +
    +
    +/**
    + * Implements a "WordCount" program that computes a simple word occurrence histogram
    + * over hard coded examples or text files. This example demonstrates how to use KeySelectors,
ReduceFunction and FlatMapFunction.
    + */
    +@SuppressWarnings("serial")
    +public class WordCountKeySelector {
    +	
    +	/**
    +	 * Runs the WordCount program.
    +	 * 
    +	 * @param args Input and output file.
    +	 */
    +	public static void main(String[] args) throws Exception {
    +		// Check whether arguments are given and tell user how to use this example with files.
    +		if (args.length < 2) {
    +			System.out.println("You can specify: WordCountKeySelector <input path> <result
path>, in order to work with files.");
    +		}
    +		
    +		// Input and output path [optional]
    +		String inputPath = null;
    +		String outputPath = null;
    +		
    +		// Get the environment as starting point
    +		final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    +		
    +		// Read the text file from given input path or hard coded
    +		DataSet<String> text = null;
    +		if(args.length >= 1) {
    +			inputPath = args[0];
    +			text = env.readTextFile(inputPath);
    +		}
    +		else {
    +			System.out.println("No input file specified. Using hard coded example.");
    +			text = env.fromElements("To be", "or not to be", "or to be still", "and certainly
not to be not at all", "is that the question?");
    +		}
    +		
    +		// Split up the lines in pairs (2-tuples) containing: (word,1)
    +		DataSet<CustomizedWord> words = text.flatMap(new Tokenizer());
    +		
    +		// Group by the tuple field "0" and sum up tuple field "1". Create KeySelector to be
able to group CustomizedWord. Instantiate customized reduce function. 
    --- End diff --
    
    This comment and also the comment in line 62 is most likely a copy-paste error from our
other word count.


> Add Java example program with KeySelectors
> ------------------------------------------
>
>                 Key: FLINK-827
>                 URL: https://issues.apache.org/jira/browse/FLINK-827
>             Project: Flink
>          Issue Type: Bug
>            Reporter: GitHub Import
>            Assignee: Tobias
>              Labels: github-import
>             Fix For: pre-apache
>
>
> We need a Java example program that shows the use of `KeySelector` functions.
> Alternatively we could modify an existing program to do this.
> The new example program should follow the general layout of the other example programs,
i.e., run without parameters on default data, have the same code structure, etc.
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/827
> Created by: [fhueske|https://github.com/fhueske]
> Labels: documentation, java api, 
> Milestone: Release 0.6 (unplanned)
> Created at: Sat May 17 12:45:24 CEST 2014
> State: open



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message