flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rmetzger <...@git.apache.org>
Subject [GitHub] incubator-flink pull request: ISSUE #827 fix FLINK-827
Date Wed, 25 Jun 2014 17:43:34 GMT
Github user rmetzger commented on a diff in the pull request:

    https://github.com/apache/incubator-flink/pull/45#discussion_r14200819
  
    --- Diff: stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/wordcount/WordCountPLOJO.java
---
    @@ -0,0 +1,183 @@
    +/***********************************************************************************************************************
    + *
    + * Copyright (C) 2010-2013 by the Stratosphere project (http://stratosphere.eu)
    + *
    + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software distributed under
the License is distributed on
    + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the
    + * specific language governing permissions and limitations under the License.
    + *
    + **********************************************************************************************************************/
    +package eu.stratosphere.example.java.wordcount;
    +
    +import eu.stratosphere.api.java.DataSet;
    +import eu.stratosphere.api.java.ExecutionEnvironment;
    +import eu.stratosphere.api.java.functions.FlatMapFunction;
    +import eu.stratosphere.api.java.functions.KeySelector;
    +import eu.stratosphere.api.java.functions.ReduceFunction;
    +import eu.stratosphere.util.Collector;
    +
    +
    +
    +/**
    + * Implements a "WordCount" program that computes a simple word occurrence histogram
    + * over hard coded examples or text files. This example demonstrates how to use KeySelectors,
ReduceFunction and FlatMapFunction.
    + */
    +@SuppressWarnings("serial")
    +public class WordCountPLOJO {
    +	
    +	/**
    +	 * Runs the WordCount program.
    +	 * 
    +	 * @param args Input and output file.
    +	 */
    +	public static void main(String[] args) throws Exception {
    +		// Check whether arguments are given and tell user how to use this example with files.
    +		if (args.length < 2) {
    +			System.out.println("You can specify: WordCountPLOJO <input path> <result
path>, in order to work with files.");
    +		}
    +		
    +		// Input and output path [optional]
    +		String inputPath = null;
    +		String outputPath = null;
    +		
    +		// Get the environment as starting point
    +		final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    +		
    +		// Read the text file from given input path or hard coded
    +		DataSet<String> text = null;
    +		try {
    +			inputPath = args[0];
    +			env.readTextFile(inputPath);
    +		}
    +		catch(Exception e) {
    +			System.out.println("No input file specified. Using hard coded example.");
    +			text = env.fromElements("To be", "or not to be", "or to be still", "and certainly
not to be not at all", "is that the question?");
    +		}
    +		
    +		// Split up the lines in pairs (2-tuples) containing: (word,1)
    +		DataSet<CustomizedWord> words = text.flatMap(new Tokenizer());
    +		
    +		// Create KeySelector to be able to group CustomizedWord 
    +		CustomizedWordKeySelector keySelector = new CustomizedWordKeySelector();
    +		
    +		// Instantiate customized reduce function
    +		CustomizedWordReducer reducer = new CustomizedWordReducer();
    +		
    +		// Group by the tuple field "0" and sum up tuple field "1"
    +		DataSet<CustomizedWord> result = words.groupBy(keySelector).reduce(reducer);
    --- End diff --
    
    I think its nicer to do this "inline". Similar to the other wordcount example
    ```java
    words.groupBy(new CustomizedWordKeySelector())
      .reduce(new CustomizedWordReducer());
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message