flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tillrohrmann <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-3315] Fix Slot Sharing in Streaming API
Date Wed, 24 Feb 2016 11:38:45 GMT
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1641#discussion_r53926589
  
    --- Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/api/graph/SlotAllocationTest.java
---
    @@ -40,17 +48,142 @@ public void test() {
     			public boolean filter(Long value) { return false; }
     		};
     
    -		env.generateSequence(1, 10).filter(dummyFilter).isolateResources().filter(dummyFilter)
    -				.disableChaining().filter(dummyFilter).startNewResourceGroup().filter(dummyFilter)
    -				.startNewChain().print();
    +		env.generateSequence(1, 10)
    +			.filter(dummyFilter).slotSharingGroup("isolated")
    +			.filter(dummyFilter).slotSharingGroup("default").disableChaining()
    +			.filter(dummyFilter).slotSharingGroup("group 1")
    +			.filter(dummyFilter).startNewChain()
    +			.print().disableChaining();
    +
    +		// verify that a second pipeline does not inherit the groups from the first pipeline
    +		env.generateSequence(1, 10)
    +				.filter(dummyFilter).slotSharingGroup("isolated-2")
    +				.filter(dummyFilter).slotSharingGroup("default").disableChaining()
    +				.filter(dummyFilter).slotSharingGroup("group 2")
    +				.filter(dummyFilter).startNewChain()
    +				.print().disableChaining();
     
     		JobGraph jobGraph = env.getStreamGraph().getJobGraph();
     
     		List<JobVertex> vertices = jobGraph.getVerticesSortedTopologicallyFromSources();
     
    -		assertEquals(vertices.get(0).getSlotSharingGroup(), vertices.get(2).getSlotSharingGroup());
    +		assertEquals(vertices.get(0).getSlotSharingGroup(), vertices.get(3).getSlotSharingGroup());
    +		assertNotEquals(vertices.get(0).getSlotSharingGroup(), vertices.get(2).getSlotSharingGroup());
    +		assertNotEquals(vertices.get(3).getSlotSharingGroup(), vertices.get(4).getSlotSharingGroup());
    +		assertEquals(vertices.get(4).getSlotSharingGroup(), vertices.get(5).getSlotSharingGroup());
    +		assertEquals(vertices.get(5).getSlotSharingGroup(), vertices.get(6).getSlotSharingGroup());
    --- End diff --
    
    That's how I found the ordering out. Apparently first the sources appear and then the
rest of the individual pipelines. But here the first pipeline comes first. 
    
    Given that a topological ordering is partial and, thus, there are multiple linearization
of it, I think this test is bound to crash at some point. It is simply too tightly coupled
to the internal implementation of the `JobGraph`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message