flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Metzger (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1569) Object reuse mode is not working with KeySelector functions.
Date Tue, 17 Feb 2015 12:50:13 GMT
Robert Metzger created FLINK-1569:
-------------------------------------

             Summary: Object reuse mode is not working with KeySelector functions.
                 Key: FLINK-1569
                 URL: https://issues.apache.org/jira/browse/FLINK-1569
             Project: Flink
          Issue Type: Bug
          Components: Java API
    Affects Versions: 0.9
            Reporter: Robert Metzger


The following code works correctly when object reuse is switched off.
When switching it on, the results are wrong.
Using a string-based key selection (putting "name") works for both cases.
{code}
	@Test
	public void testWithAvroGenericSer() throws Exception {
		final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
	//	env.getConfig().enableObjectReuse();
		Path in = new Path(inFile.getAbsoluteFile().toURI());

		AvroInputFormat<User> users = new AvroInputFormat<User>(in, User.class);
		DataSet<User> usersDS = env.createInput(users);

		DataSet<Tuple2<String, Integer>> res = usersDS.groupBy(new KeySelector<User,
String>() {
			@Override
			public String getKey(User value) throws Exception {
				return String.valueOf(value.getName());
			}
		}).reduceGroup(new GroupReduceFunction<User, Tuple2<String, Integer>>() {
			@Override
			public void reduce(Iterable<User> values, Collector<Tuple2<String, Integer>>
out) throws Exception {
				for(User u : values) {
					out.collect(new Tuple2<String, Integer>(u.getName().toString(), 1));
				}
			}
		});

		res.writeAsText(resultPath);
		res.print();
		env.execute("Avro Key selection");


		expected = "(Charlie,1)\n(Alyssa,1)\n";
	}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message