apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bright chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2182) benchmark for spillable data structure and storage
Date Thu, 11 Aug 2016 22:22:23 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418038#comment-15418038
] 

bright chen commented on APEXMALHAR-2182:
-----------------------------------------

The unit test will be failed due to OutOfMemoryError after couple of minutes in my local machine
environment. And the average performance will be decrease rapid.

Here is the OutOfMemoryError stack trace:
java.lang.OutOfMemoryError: Java heap space
	at org.apache.apex.malhar.lib.utils.serde.SerdeListSlice.serialize(SerdeListSlice.java:70)
	at org.apache.apex.malhar.lib.utils.serde.SerdeListSlice.serialize(SerdeListSlice.java:37)
	at org.apache.apex.malhar.lib.state.spillable.SpillableByteMapImpl.endWindow(SpillableByteMapImpl.java:219)
	at org.apache.apex.malhar.lib.state.spillable.SpillableArrayListImpl.endWindow(SpillableArrayListImpl.java:318)
	at org.apache.apex.malhar.lib.state.spillable.SpillableByteArrayListMultimapImpl.endWindow(SpillableByteArrayListMultimapImpl.java:274)
	at org.apache.apex.malhar.lib.state.spillable.SpillableDSBenchmarkTest.testSpillableMutimap(SpillableDSBenchmarkTest.java:114)
	at org.apache.apex.malhar.lib.state.spillable.SpillableDSBenchmarkTest.testSpillableMutimap(SpillableDSBenchmarkTest.java:78)

Here is the log, which shows performance decreasing: 
2016-08-11 15:00:25,946 [main] WARN  util.NativeCodeLoader <clinit> - Unable to load
native-hadoop library for your platform... using builtin-java classes where applicable
2016-08-11 15:00:26,121 [main] INFO  spillable.SpillableDSBenchmarkTest testSpillableMutimap
- Spent 67 mills for 0 operation. average: 0
2016-08-11 15:00:26,121 [main] INFO  spillable.SpillableDSBenchmarkTest checkEnvironment -
freeMemory: 195M; allocatedMemory: 257M; maxMemory: 3817M
2016-08-11 15:01:26,055 [main] INFO  spillable.SpillableDSBenchmarkTest testSpillableMutimap
- Spent 60001 mills for 88948004 operation. average: 1482
2016-08-11 15:01:26,055 [main] INFO  spillable.SpillableDSBenchmarkTest checkEnvironment -
freeMemory: 842M; allocatedMemory: 2582M; maxMemory: 3817M
2016-08-11 15:02:26,055 [main] INFO  spillable.SpillableDSBenchmarkTest testSpillableMutimap
- Spent 120001 mills for 166005088 operation. average: 1383
2016-08-11 15:02:26,055 [main] INFO  spillable.SpillableDSBenchmarkTest checkEnvironment -
freeMemory: 417M; allocatedMemory: 3117M; maxMemory: 3817M
2016-08-11 15:03:26,493 [main] INFO  spillable.SpillableDSBenchmarkTest testSpillableMutimap
- Spent 180439 mills for 224000000 operation. average: 1241
2016-08-11 15:03:26,494 [main] INFO  spillable.SpillableDSBenchmarkTest checkEnvironment -
freeMemory: 614M; allocatedMemory: 3816M; maxMemory: 3817M
2016-08-11 15:04:26,572 [main] INFO  spillable.SpillableDSBenchmarkTest testSpillableMutimap
- Spent 240518 mills for 239125628 operation. average: 994
2016-08-11 15:04:26,572 [main] INFO  spillable.SpillableDSBenchmarkTest checkEnvironment -
freeMemory: 450M; allocatedMemory: 3816M; maxMemory: 3817M
2016-08-11 15:05:26,289 [main] INFO  spillable.SpillableDSBenchmarkTest testSpillableMutimap
- Spent 300235 mills for 241102872 operation. average: 803
2016-08-11 15:05:26,289 [main] INFO  spillable.SpillableDSBenchmarkTest checkEnvironment -
freeMemory: 423M; allocatedMemory: 3816M; maxMemory: 3817M


> benchmark for spillable data structure  and storage
> ---------------------------------------------------
>
>                 Key: APEXMALHAR-2182
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2182
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: bright chen
>            Assignee: bright chen
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message