Release 1.0.0 – Changelog and Known Issues

+ +

Changelog
- Sub-task
- Bug
- Improvement
- Task
- Test
- Wish
+
Known Issues

+ +

Changelog

+ +

Sub-task

+ +

[FLINK-7] - [GitHub] Enable Range Partitioner +
[FLINK-146] - Sorted output not working +
[FLINK-1982] - Remove dependencies on Record for Flink runtime and core +
[FLINK-2676] - Add abstraction for keyed window state +
[FLINK-2680] - Create a dedicated aligned-event time window operator +
[FLINK-2730] - Add CPU/Network utilization graphs to new web dashboard +
[FLINK-2732] - Add access to the TaskManagers' log file and out file in the web dashboard. +
[FLINK-2850] - Limit the types of jobs which can run in detached mode +
[FLINK-2853] - Apply JMH on MutableHashTablePerformanceBenchmark class. +
[FLINK-2869] - Apply JMH on IOManagerPerformanceBenchmark class. +
[FLINK-2889] - Apply JMH on LongSerializationSpeedBenchmark class +
[FLINK-2890] - Apply JMH on StringSerializationSpeedBenchmark class. +
[FLINK-2900] - Remove Record-API dependencies from Hadoop Compat module +
[FLINK-2901] - Several flink-test ITCases depend on Record API features +
[FLINK-2906] - Remove Record-API +
[FLINK-2919] - Apply JMH on FieldAccessMinibenchmark class. +
[FLINK-2920] - Apply JMH on KryoVersusAvroMinibenchmark class. +
[FLINK-2933] - Flink scala libraries exposed with maven should carry scala version +
[FLINK-2972] - Remove Twitter Chill dependency from flink-java module +
[FLINK-3057] - [py] Provide a way to pass information back to the plan process +
[FLINK-3070] - Create an asynchronous state handle interface +
[FLINK-3071] - Add asynchronous materialization thread +
[FLINK-3140] - NULL value data layout in Row Serializer/Comparator +
[FLINK-3195] - Restructure examples projects and package streaming examples +
[FLINK-3201] - Enhance Partitioned State Interface with State Types +
[FLINK-3208] - Rename Gelly vertex-centric model to scatter-gather +
[FLINK-3224] - The Streaming API does not call setInputType if a format implements InputTypeConfigurable +
[FLINK-3275] - [py] Add support for Dataset.setParallelism() +
[FLINK-3285] - Skip Maven deployment of flink-java8 +
[FLINK-3312] - Add convenience accessor methods for extended state interface +
[FLINK-3327] - Attach the ExecutionConfig to the JobGraph and make it accessible to the AbstractInvocable. +
[FLINK-3363] - JobManager does not shut down dedicated executor properly +
[FLINK-3365] - BlobLibraryCacheManager does not shutdown Timer thread +
[FLINK-3371] - Move TriggerCotext and TriggerResult to their own classes +
[FLINK-3384] - Create atomic closable queue for communication between Kafka Threads +
[FLINK-3388] - Expose task accumulators via JMX +
[FLINK-3401] - AscendingTimestampExtractor should not fail on order violation +
[FLINK-3437] - Fix UI router state for job plan +
[FLINK-3468] - Change Timestamp Extractors to not support negative timestamps +
[FLINK-3470] - EventTime WindowAssigners should error on negative timestamps +

+ +

Bug

+ +

[FLINK-1278] - Remove the Record special code paths +
[FLINK-1644] - WebClient dies when no ExecutionEnvironment in main method +
[FLINK-1989] - Sorting of POJO data set from TableEnv yields NotSerializableException +
[FLINK-2115] - TableAPI throws ExpressionException for "Dangling GroupBy operation" +
[FLINK-2263] - ExecutionGraph uses Thread.sleep to delay execution retries +
[FLINK-2315] - Hadoop Writables cannot exploit implementing NormalizableKey +
[FLINK-2348] - IterateExampleITCase failing +
[FLINK-2351] - Deprecate config builders in InputFormats and Output formats +
[FLINK-2369] - On Windows, in testFailingSortingDataSinkTask the temp file is not removed +
[FLINK-2392] - Instable test in flink-yarn-tests +
[FLINK-2443] - [CompactingHashTable] GSA Connected Components fails with NPE +
[FLINK-2491] - Operators are not participating in state checkpointing in some cases +
[FLINK-2504] - ExternalSortLargeRecordsITCase.testSortWithLongAndShortRecordsMixed failed spuriously +
[FLINK-2586] - Unstable Storm Compatibility Tests +
[FLINK-2608] - Arrays.asList(..) does not work with CollectionInputFormat +
[FLINK-2622] - Scala DataStream API does not have writeAsText method which supports WriteMode +
[FLINK-2624] - RabbitMQ source / sink should participate in checkpointing +
[FLINK-2662] - CompilerException: "Bug: Plan generation for Unions picked a ship strategy between binary plan operators." +
[FLINK-2671] - Instable Test StreamCheckpointNotifierITCase +
[FLINK-2674] - Rework windowing logic +
[FLINK-2695] - KafkaITCase.testConcurrentProducerConsumerTopology failed on Travis +
[FLINK-2719] - ProcessFailureStreamingRecoveryITCase>AbstractProcessFailureRecoveryTest.testTaskManagerProcessFailure failed on Travis +
[FLINK-2739] - Release script depends on the order of parent module information +
[FLINK-2747] - TypeExtractor does not correctly analyze Scala Immutables (AnyVal) +
[FLINK-2752] - Documentation is not easily differentiable from the Flink homepage +
[FLINK-2757] - DataSinkTaskTest fails on Windows +
[FLINK-2758] - TaskManagerRegistrationTest fails on Windows +
[FLINK-2759] - TaskManagerProcessReapingTest fails on Windows +
[FLINK-2769] - Web dashboard port not configurable on client side +
[FLINK-2771] - IterateTest.testSimpleIteration fails on Travis +
[FLINK-2797] - CLI: Missing option to submit jobs in detached mode +
[FLINK-2799] - Yarn tests cannot be executed with DEBUG log level +
[FLINK-2800] - kryo serialization problem +
[FLINK-2823] - YARN client should report a proper exception if Hadoop Env variables are not set +
[FLINK-2826] - transformed is modified in BroadcastVariableMaterialization#decrementReferenceInternal without proper locking +
[FLINK-2827] - Potential resource leak in TwitterSource#loadAuthenticationProperties() +
[FLINK-2832] - Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement +
[FLINK-2838] - Inconsistent use of URL and Path classes for resources +
[FLINK-2845] - TimestampITCase.testWatermarkPropagation +
[FLINK-2879] - Links in documentation are broken +
[FLINK-2913] - Close of ObjectOutputStream should be enclosed in finally block in FsStateBackend +
[FLINK-2914] - Missing break in ZooKeeperSubmittedJobGraphStore#SubmittedJobGraphsPathCacheListener#childEvent() +
[FLINK-2930] - ExecutionConfig execution retry delay not respected +
[FLINK-2934] - Remove placehoder pages in Web dashboard +
[FLINK-2937] - Typo in Quickstart->Scala API->Alternative Build Tools: SBT +
[FLINK-2938] - Streaming docs not in sync with latest state changes +
[FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic) +
[FLINK-2950] - Markdown presentation problem in SVM documentation +
[FLINK-2954] - Not able to pass custom environment variables in cluster to processes that spawning TaskManager +
[FLINK-2958] - StreamingJobGraphGenerator sets hard coded number execution retry +
[FLINK-2963] - Dependence on SerializationUtils#deserialize() should be avoided +
[FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs +
[FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster +
[FLINK-2979] - RollingSink does not work with Hadoop 2.7.1 +
[FLINK-2987] - Flink 0.10 fails to start on YARN 2.6.0 +
[FLINK-2989] - Job Cancel button doesn't work on Yarn +
[FLINK-2990] - Scala 2.11 build fails to start on YARN +
[FLINK-2992] - New Windowing code is using SerializationUtils with wrong classloader +
[FLINK-3000] - Add ShutdownHook to YARN CLI to prevent lingering sessions +
[FLINK-3005] - Commons-collections object deserialization remote command execution vulnerability +
[FLINK-3009] - Cannot build docs with Jekyll 3.0.0 +
[FLINK-3011] - Cannot cancel failing/restarting streaming job from the command line +
[FLINK-3013] - Incorrect package declaration in GellyScalaAPICompletenessTest.scala +
[FLINK-3019] - CLI does not list running/restarting jobs +
[FLINK-3020] - Local streaming execution: set number of task manager slots to the maximum parallelism +
[FLINK-3022] - Broken link 'Working With State' in Fault Tolerance Section of Stream Programming Guide +
[FLINK-3024] - TimestampExtractor Does not Work When returning Long.MIN_VALUE +
[FLINK-3025] - Flink Kafka consumer may get stuck due to Kafka/Zookeeper client bug +
[FLINK-3032] - Flink does not start on Hadoop 2.7.1 (HDP), due to class conflict +
[FLINK-3043] - Kafka Connector description in Streaming API guide is wrong/outdated +
[FLINK-3047] - Local batch execution: set number of task manager slots to the maximum parallelism +
[FLINK-3048] - DataSinkTaskTest.testCancelDataSinkTask +
[FLINK-3052] - Optimizer does not push properties out of bulk iterations +
[FLINK-3054] - Remove R (return) type variable from SerializationSchema +
[FLINK-3059] - Javadoc fix for DataSet.writeAsText() +
[FLINK-3061] - Kafka Consumer is not failing if broker is not available +
[FLINK-3062] - Kafka Producer is not failing if broker is not available/no partitions available +
[FLINK-3067] - Kafka source fails during checkpoint notifications with NPE +
[FLINK-3080] - Cannot union a data stream with a product of itself +
[FLINK-3081] - Kafka Periodic Offset Committer does not properly terminate on canceling +
[FLINK-3082] - Confusing error about ManualTimestampSourceFunction +
[FLINK-3087] - Table API do not support multi count in aggregation. +
[FLINK-3098] - Cast from Date to Long throw compile error. +
[FLINK-3100] - Signal handler prints error on normal shutdown of cluster +
[FLINK-3101] - Flink Kafka consumer crashes with NPE when it sees deleted record +
[FLINK-3103] - Remove synchronization in FsStateBackend#FsCheckpointStateOutputStream#close() +
[FLINK-3108] - JoinOperator's with() calls the wrong TypeExtractor method +
[FLINK-3117] - Storm Tick Tuples are not supported +
[FLINK-3118] - Check if MessageFunction implements ResultTypeQueryable +
[FLINK-3121] - Watermark forwarding does not work for sources not producing any data +
[FLINK-3125] - Web dashboard does not start when log files are not found +
[FLINK-3134] - Make YarnJobManager's allocate call asynchronous +
[FLINK-3136] - Scala Closure Cleaner uses wrong ASM import +
[FLINK-3138] - Method References are not supported as lambda expressions +
[FLINK-3143] - Update Clojure Cleaner's ASM references to ASM5 +
[FLINK-3144] - [storm] LocalCluster prints nothing without a configured logger +
[FLINK-3145] - Storm examples can't be run without flink-java as dependency +
[FLINK-3151] - YARN kills Flink TM containers due to memory overuse (outside heap/offheap) +
[FLINK-3156] - FlinkKafkaConsumer fails with NPE on notifyCheckpointComplete +
[FLINK-3157] - Web frontend json files contain author attribution +
[FLINK-3166] - The first program in ObjectReuseITCase has the wrong expected result, and it succeeds +
[FLINK-3171] - Consolidate zoo of wrapper classes for input/output-stream to data-input/output-view +
[FLINK-3173] - Bump org.apache.httpcomponents.httpclient version to 4.2.6 +
[FLINK-3175] - KafkaITCase.testOffsetAutocommitTest +
[FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned +
[FLINK-3180] - MemoryLogger does not log direct memory +
[FLINK-3185] - Silent failure during job graph recovery +
[FLINK-3188] - Deletes in Kafka source should be passed on to KeyedDeserializationSchema +
[FLINK-3189] - Error while parsing job arguments passed by CLI +
[FLINK-3196] - InputStream should be closed in EnvironmentInformation#getRevisionInformation() +
[FLINK-3197] - InputStream not closed in BinaryInputFormat#createStatistics +
[FLINK-3206] - Heap size for non-pre-allocated off-heap memory +
[FLINK-3218] - Merging Hadoop configurations overrides user parameters +
[FLINK-3220] - Flink does not start on Hortonworks Sandbox 2.3.2 due to missing class +
[FLINK-3236] - Flink user code classloader should have Flink classloader as parent classloader +
[FLINK-3240] - Remove or document DataStream(.global|.forward) +
[FLINK-3242] - User-specified StateBackend is not Respected if Checkpointing is Disabled +
[FLINK-3243] - Fix Interplay of TimeCharacteristic and Time Windows +
[FLINK-3247] - Kafka Connector unusable with quickstarts - shading issue +
[FLINK-3248] - RMQSource does not provide a constructor for credentials or other options +
[FLINK-3250] - Savepoint coordinator requires too strict parallelism match +
[FLINK-3251] - Checkpoint stats show ghost numbers +
[FLINK-3252] - Checkpoint stats only displayed after click on graph +
[FLINK-3254] - CombineFunction interface not respected +
[FLINK-3255] - Chaining behavior should not depend on parallelism +
[FLINK-3260] - ExecutionGraph gets stuck in state FAILING +
[FLINK-3261] - Tasks should eagerly report back when they cannot start a checkpoint +
[FLINK-3266] - LocalFlinkMiniCluster leaks resources when multiple jobs are submitted +
[FLINK-3267] - Disable reference tracking in Kryo fallback serializer +
[FLINK-3268] - Unstable test JobManagerSubmittedJobGraphsRecoveryITCase +
[FLINK-3271] - Using webhdfs in a flink topology throws classnotfound exception +
[FLINK-3274] - Prefix Kafka connector accumulators with unique id +
[FLINK-3280] - Wrong usage of Boolean.getBoolean() +
[FLINK-3281] - IndexOutOfBoundsException when range-partitioning empty DataSet +
[FLINK-3286] - Remove JDEB Debian Package code from flink-dist +
[FLINK-3287] - Flink Kafka Consumer fails due to Curator version conflict +
[FLINK-3289] - Double reference to flink-contrib +
[FLINK-3292] - Bug in flink-jdbc. Not all JDBC drivers supported +
[FLINK-3293] - Custom Application Name on YARN is ignored in deploy jobmanager mode +
[FLINK-3296] - DataStream.write*() methods are not flushing properly +
[FLINK-3300] - Concurrency Bug in Yarn JobManager +
[FLINK-3304] - AvroOutputFormat.setSchema() doesn't work in yarn-cluster mode +
[FLINK-3314] - Early cancel calls can cause Tasks to not cancel properly +
[FLINK-3322] - MemoryManager creates too much GC pressure with iterative jobs +
[FLINK-3328] - Incorrectly shaded dependencies in flink-runtime +
[FLINK-3333] - Documentation about object reuse should be improved +
[FLINK-3337] - mvn test fails on flink-runtime because curator classes not found +
[FLINK-3338] - Kafka deserialization issue - ClassNotFoundException +
[FLINK-3339] - Checkpointing NPE when using filterWithState +
[FLINK-3340] - Fix object juggling in drivers +
[FLINK-3341] - Kafka connector's 'auto.offset.reset' inconsistent with Kafka +
[FLINK-3342] - Operator checkpoint statistics state size overflow +
[FLINK-3350] - Increase timeouts on Travis Builds +
[FLINK-3351] - RocksDB Backend cannot determine correct local db path +
[FLINK-3352] - RocksDB Backend cannot determine correct hdfs path +
[FLINK-3353] - CSV-related tests may fail depending on locale +
[FLINK-3357] - Drop JobId.toShortString() +
[FLINK-3359] - Make RocksDB file copies asynchronous +
[FLINK-3361] - Wrong error messages for execution retry delay and akka ask pause config values +
[FLINK-3368] - Kafka 0.8 consumer fails to recover from broker shutdowns +
[FLINK-3369] - RemoteTransportException should be instance of CancelTaskException +
[FLINK-3373] - Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems +
[FLINK-3385] - Fix outer join skipping unprobed partitions +
[FLINK-3392] - Unprotected access to elements in ClosableBlockingQueue#size() +
[FLINK-3393] - ExternalProcessRunner wait to finish copying error stream +
[FLINK-3394] - Clear up the contract of MutableObjectIterator.next(reuse) +
[FLINK-3396] - Job submission Savepoint restore logic flawed +
[FLINK-3400] - RocksDB Backend does not work when not in Flink lib folder +
[FLINK-3412] - Remove implicit conversions JavaStream / ScalaStream +
[FLINK-3415] - TimestampExctractor accepts negative watermarks +
[FLINK-3416] - [py] .bat files fail when path contains spaces +
[FLINK-3420] - Remove "ReadTextFileWithValue" from StreamExecutionEnvironment +
[FLINK-3423] - ExternalProcessRunnerTest fails on Windows +
[FLINK-3424] - FileStateBackendtest.testStateOutputStream fails on windows +
[FLINK-3425] - DataSinkTaskTest.Failing[Sorting]DataSinkTask fails on Windows +
[FLINK-3426] - JobManagerLeader[Re]ElectionTest.testleaderElection fails on Windows +
[FLINK-3432] - ZookeeperOffsetHandlerTest fails on windows +
[FLINK-3434] - Return value from flinkYarnClient#deploy() should be checked against null +
[FLINK-3438] - ExternalProcessRunner fails to detect ClassNotFound exception because of locale settings +
[FLINK-3439] - Remove final Long.MAX_VALUE Watermark in StreamSource +
[FLINK-3440] - Kafka should also checkpoint partitions where no initial offset was retrieved +
[FLINK-3448] - WebRuntimeMonitor: Move initialization of handlers to start() method +
[FLINK-3449] - createInput swallows exception if TypeExtractor fails +
[FLINK-3450] - RocksDB Backed Window Fails with KryoSerializer +
[FLINK-3453] - Fix TaskManager logs exception when sampling backpressure while task completes +
[FLINK-3458] - Shading broken in flink-shaded-hadoop +
[FLINK-3478] - Flink serves arbitary files through the web interface +
[FLINK-3483] - Job graph visualization not working properly in OS X Chrome +
[FLINK-3485] - The SerializedListAccumulator value doest seem to be right +
[FLINK-3499] - Possible ghost references in ZooKeeper completed checkpoint store +
[FLINK-3511] - Flink library examples not runnable without adding dependencies +
[FLINK-3512] - Savepoint backend should not revert to "jobmanager" +
[FLINK-3513] - Fix interplay of automatic Operator UID and Changing name of WindowOperator +
[FLINK-3518] - Stale docs for quickstart setup +
[FLINK-3525] - Missing call to super#close() in TimestampsAndPeriodicWatermarksOperator#close() +
[FLINK-3526] - ProcessingTime Window Assigner and Trigger are broken +
[FLINK-3534] - Cancelling a running job can lead to restart instead of stopping +
[FLINK-3540] - Hadoop 2.6.3 build contains /com/google/common (guava) classes in flink-dist.jar +
[FLINK-3554] - Bounded sources should emit a Max Watermark when they are done +
[FLINK-3556] - Unneeded check in HA blob store configuration +
[FLINK-3562] - Update docs in the course of EventTimeSourceFunction removal +
[FLINK-3565] - FlinkKafkaConsumer does not work with Scala 2.11 +
[FLINK-3569] - Test cases fail due to Maven Shade plugin +

+ +

Improvement

+ +

[FLINK-8] - [GitHub] Implement automatic sample histogram building for Range Partitioning +
[FLINK-553] - Add getGroupKey() method to group-at-time operators +
[FLINK-573] - Clean-up MapOperators in optimizer +
[FLINK-734] - Integrate web job client into JobManager web interface +
[FLINK-987] - Extend TypeSerializers and -Comparators to work directly on Memory Segments +
[FLINK-1045] - Remove Combinable Annotation +
[FLINK-1228] - Add REST Interface to JobManager +
[FLINK-1240] - We cannot use sortGroup on a global reduce +
[FLINK-1513] - Remove GlobalConfiguration Singleton +
[FLINK-1666] - Clean-up Field Expression Code +
[FLINK-1702] - Authenticate via Kerberos from the client only +
[FLINK-1778] - Improve normalized keys in composite key case +
[FLINK-1903] - Joins where one side uses a field more than once don't work +
[FLINK-1947] - Make Avro and Tachyon test logging less verbose +
[FLINK-2017] - Add predefined required parameters to ParameterTool +
[FLINK-2021] - Rework examples to use ParameterTool +
[FLINK-2075] - Shade akka and protobuf dependencies away +
[FLINK-2185] - Rework semantics for .setSeed function of SVM +
[FLINK-2213] - Configure number of vcores +
[FLINK-2239] - print() on DataSet: stream results and print incrementally +
[FLINK-2342] - Add new fit operation and more tests for StandardScaler +
[FLINK-2344] - Deprecate/Remove the old Pact Pair type +
[FLINK-2380] - Allow to configure default FS for file inputs +
[FLINK-2399] - Fail when actor versions don't match +
[FLINK-2455] - Misleading I/O manager error log messages +
[FLINK-2488] - Expose attemptNumber in RuntimeContext +
[FLINK-2518] - Avoid predetermination of ports for network services +
[FLINK-2523] - Make task canceling interrupt interval configurable +
[FLINK-2524] - Add "getTaskNameWithSubtasks()" to RuntimeContext +
[FLINK-2559] - Fix Javadoc (e.g. Code Examples) +
[FLINK-2646] - Rich functions should provide a method "closeAfterFailure()" +
[FLINK-2667] - Rework configuration parameters +
[FLINK-2716] - Checksum method for DataSet and Graph +
[FLINK-2788] - Add type hint with TypeExtactor call on Hint Type +
[FLINK-2795] - Print JobExecutionResult for interactively invoked jobs +
[FLINK-2882] - Improve performance of string conversions +
[FLINK-2893] - Rename recovery configuration keys +
[FLINK-2895] - Duplicate immutable object creation +
[FLINK-2897] - Use distinct initial indices for OutputEmitter round-robin +
[FLINK-2898] - Invert Travis CI build order +
[FLINK-2902] - Web interface sort tasks newest first +
[FLINK-2904] - Web interface truncated task counts +
[FLINK-2932] - Flink quickstart docs should ask users to download from https, not http +
[FLINK-2936] - ClassCastException when using EventTimeSourceFunction in non-EventTime program +
[FLINK-2940] - Deploy multiple Scala versions for Maven artifacts +
[FLINK-2947] - Coloured Scala Shell +
[FLINK-2949] - Add method 'writeSequencefile' to DataSet +
[FLINK-2961] - Add support for basic type Date in Table API +
[FLINK-2962] - Cluster startup script refers to unused variable +
[FLINK-2966] - Improve the way job duration is reported on web frontend. +
[FLINK-2974] - Add periodic offset commit to Kafka Consumer if checkpointing is disabled +
[FLINK-2976] - Save and load checkpoints manually +
[FLINK-2981] - Update README for building docs +
[FLINK-2984] - Support lenient parsing of SVMLight input files +
[FLINK-2991] - Extend Window Operators to Allow Efficient Fold Operation +
[FLINK-2993] - Set default DelayBetweenExecutionRetries to 0 +
[FLINK-2994] - Client sysout logging does not report exceptions +
[FLINK-3003] - Add container allocation timeout to YARN CLI +
[FLINK-3017] - Broken 'Slots' link on Streaming Programming Guide +
[FLINK-3023] - Show Flink version + commit id for -SNAPSHOT versions in web frontend +
[FLINK-3028] - Cannot cancel restarting job via web frontend +
[FLINK-3030] - Enhance Dashboard to show Execution Attempts +
[FLINK-3039] - Trigger KeyValueState cannot be Scala Int +
[FLINK-3040] - Add docs describing how to configure State Backends +
[FLINK-3042] - Define a way to let types create their own TypeInformation +
[FLINK-3044] - In YARN mode, configure FsStateBackend by default. +
[FLINK-3045] - Properly expose the key of a kafka message +
[FLINK-3046] - Integrate the Either Java type with the TypeExtractor +
[FLINK-3049] - Move "Either" type to package "org.apache.flink.types" +
[FLINK-3050] - Add custom Exception type to suppress job restarts +
[FLINK-3051] - Define a maximum number of concurrent inflight checkpoints +
[FLINK-3055] - ExecutionVertex has duplicate method getParallelSubtaskIndex and getSubTaskIndex +
[FLINK-3056] - Show bytes sent/received as MBs/GB and so on in web interface +
[FLINK-3063] - [py] Remove combiner +
[FLINK-3069] - Make state materialization asynchronous +
[FLINK-3073] - Activate streaming mode by default +
[FLINK-3074] - Make ApplicationMaster/JobManager akka port configurable +
[FLINK-3076] - Display channel exchange mode in the plan visualizer +
[FLINK-3077] - Add "version" command to CliFrontend for showing the version of the installation +
[FLINK-3083] - Add docs how to configure streaming fault tolerance +
[FLINK-3084] - File State Backend should not write very small state into files +
[FLINK-3085] - Move State Backend Initialization from "registerInputOutput()" to "invoke()" +
[FLINK-3114] - Read cluster's default parallelism upon remote job submission +
[FLINK-3115] - Update Elasticsearch connector to 2.X +
[FLINK-3116] - Remove RecordOperator +
[FLINK-3120] - Set number of event loop thread and number of buffer pool arenas to same number +
[FLINK-3122] - Generalize value type in LabelPropagation +
[FLINK-3123] - Allow setting custom start-offsets for the Kafka consumer +
[FLINK-3124] - Introduce a TaskInfo object to better represent task name, index, attempt number etc. +
[FLINK-3127] - Measure backpressure in Flink jobs +
[FLINK-3131] - Expose checkpoint metrics +
[FLINK-3132] - Restructure streaming guide +
[FLINK-3133] - Introduce collect()/coun()/print() methods in DataStream API +
[FLINK-3135] - Add chainable driver for UNARY_NO_OP strategy +
[FLINK-3147] - HadoopOutputFormatBase should expose fields as protected +
[FLINK-3154] - Update Kryo version from 2.24.0 to 3.0.3 +
[FLINK-3160] - Aggregate operator statistics by TaskManager +
[FLINK-3161] - Externalize cluster start-up and tear-down when available +
[FLINK-3169] - Drop type from Record Data Model +
[FLINK-3172] - Specify jobmanager port in HA mode +
[FLINK-3174] - Add merging WindowAssigner +
[FLINK-3176] - Window Apply Website Example +
[FLINK-3178] - Make Closing Behavior of Window Operators Configurable +
[FLINK-3181] - The vertex-centric SSSP example and library method send unnecessary messages during the first superstep +
[FLINK-3194] - Remove web client +
[FLINK-3198] - Rename Grouping.getDataSet() method and add JavaDocs +
[FLINK-3200] - Use Partitioned State Abstraction in WindowOperator +
[FLINK-3209] - Remove Unused ProcessingTime, EventTime and AbstractTime +
[FLINK-3213] - Union of two streams with different parallelism should adjust parallelism automatically +
[FLINK-3219] - Implement DataSet.count using a single operator +
[FLINK-3232] - Add option to eagerly deploy channels +
[FLINK-3233] - PartitionOperator does not support expression keys on atomic types +
[FLINK-3234] - SortPartition does not support KeySelectorFunctions +
[FLINK-3235] - Drop Flink-on-Tez code +
[FLINK-3244] - Add log messages to savepoint coordinator restore +
[FLINK-3246] - Consolidate maven project names with *-parent suffix +
[FLINK-3249] - Wrong "unknown partition state/input gate" error messages +
[FLINK-3258] - Merge AbstractInvokable's registerInputOutput and invoke +
[FLINK-3259] - Redirect programming guides to new layout +
[FLINK-3262] - Remove fuzzy versioning from Bower dependencies +
[FLINK-3265] - RabbitMQ Source not threadsafe: ConcurrentModificationException +
[FLINK-3270] - Add example for reading and writing to Kafka +
[FLINK-3273] - Remove Scala dependency from flink-streaming-java +
[FLINK-3276] - Move runtime parts of flink-streaming-java to flink-runtime +
[FLINK-3278] - Add Partitioned State Backend Based on RocksDB +
[FLINK-3290] - [py] Generalize OperationInfo transfer +
[FLINK-3299] - Remove ApplicationID from Environment +
[FLINK-3303] - Move all non-batch specific classes in flink-java to flink-core +
[FLINK-3305] - Remove JavaKaffee serializer util +
[FLINK-3306] - Auto type registration at Kryo is buggy +
[FLINK-3308] - [py] Remove debug mode +
[FLINK-3309] - [py] Resolve maven warnings +
[FLINK-3310] - Add back pressure statistics to web frontend +
[FLINK-3334] - Change default log4j.properties Conversion pattern to include year-month-day in the timestamp +
[FLINK-3336] - Add Semi-Rebalance Data Shipping for DataStream +
[FLINK-3347] - TaskManager ActorSystems need to restart themselves in case they notice quarantine +
[FLINK-3348] - taskmanager.memory.off-heap missing bc documentation +
[FLINK-3355] - Allow passing RocksDB Option to RocksDBStateBackend +
[FLINK-3358] - Create constructors for FsStateBackend in RocksDBBackens +
[FLINK-3364] - Don't initialize SavepointStore in JobManager constructor +
[FLINK-3375] - Allow Watermark Generation in the Kafka Source +
[FLINK-3376] - Add an illustration of Event Time and Watermarks to the docs +
[FLINK-3378] - Consolidate TestingCluster and FokableFlinkMiniCluster +
[FLINK-3379] - Refactor TimestampExtractor +
[FLINK-3386] - Kafka consumers are not properly respecting the "auto.offset.reset" behavior +
[FLINK-3389] - Add Pre-defined Options settings for RocksDB State backend +
[FLINK-3395] - Polishing the web UI +
[FLINK-3402] - Refactor Common Parts of Stream/Batch Documentation +
[FLINK-3413] - Remove implicit Seq to DataStream conversion +
[FLINK-3419] - Drop partitionByHash from DataStream +
[FLINK-3421] - Remove all unused ClassTag context bounds in the Streaming Scala API +
[FLINK-3422] - Scramble HashPartitioner hashes +
[FLINK-3430] - Remove "no POJO" warning in TypeAnalyzer +
[FLINK-3435] - Change interplay of Ingestion Time and Event Time +
[FLINK-3436] - Remove ComplexIntegrationITCase +
[FLINK-3455] - Bump Kafka 0.9 connector dependency to Kafka 0.9.0.1 +
[FLINK-3459] - Make build SBT compatible +
[FLINK-3460] - Make flink-streaming-connectors' flink dependencies provided +
[FLINK-3461] - Remove duplicate condition check in ZooKeeperLeaderElectionService +
[FLINK-3469] - Improve documentation for grouping keys +
[FLINK-3509] - Update Hadoop versions in release script and on travis to the latest minor version +
[FLINK-3520] - Periodic watermark operator should emit current watermark in close() +
[FLINK-3521] - Make Iterable part of method signature for WindowFunction +
[FLINK-3522] - Storm examples "PrintSampleStream" throws an error if called without arguments +
[FLINK-3527] - Scala DataStream has no transform method +
[FLINK-3528] - Add Incremental Fold for Non-Keyed Window Operator +
[FLINK-3535] - Decrease logging verbosity of StackTraceSampleCoordinator +
[FLINK-3536] - Make clearer distinction between event time and processing time +
[FLINK-3538] - DataStream join API does not enforce consistent usage +
[FLINK-3548] - Remove unnecessary generic parameter from SingleOutputStreamOperator +
[FLINK-3559] - Don't print pid file check if no active PID +

+ +

New Feature +

[FLINK-1723] - Add cross validation for model evaluation +
[FLINK-2157] - Create evaluation framework for ML library +
[FLINK-2390] - Replace iteration timeout with algorithm for detecting termination +
[FLINK-2411] - Add basic graph summarization algorithm +
[FLINK-2435] - Add support for custom CSV field parsers +
[FLINK-2720] - Add Storm-CountMetric in flink-stormcompatibility +
[FLINK-2728] - Add missing features to new web dashboard +
[FLINK-2871] - Add OuterJoin strategy with HashTable on outer side +
[FLINK-2905] - Add intersect method to Graph class +
[FLINK-2951] - Add Union operator to Table API. +
[FLINK-2955] - Add operations introduction in Table API page. +
[FLINK-2978] - Integrate web submission interface into the new dashboard +
[FLINK-2996] - Add config entry to define BlobServer port +
[FLINK-3001] - Add Support for Java 8 Optional type +
[FLINK-3002] - Add an EitherType to the Java API +
[FLINK-3058] - Add Kafka consumer for new 0.9.0.0 Kafka API +
[FLINK-3093] - Introduce annotations for interface stability +
[FLINK-3102] - Allow reading from multiple topics with one FlinkKafkaConsumer +
[FLINK-3170] - Expose task manager metrics via JMX +
[FLINK-3192] - Add explain support to print ast and sql physical execution plan. +

+ +

Task

+ +

[FLINK-1681] - Remove the old Record API +
[FLINK-2973] - Add flink-benchmark with compliant licenses again +
[FLINK-3112] - Remove unused RecordModelPostPass class +
[FLINK-3113] - Remove unused global order methods from GenericDataSinkBase +
[FLINK-3155] - Update Flink docker version to latest stable Flink version +
[FLINK-3186] - Deprecate DataSink.sortLocalOutput() methods +
[FLINK-3366] - Rename @Experimental annotation to @PublicEvolving +
[FLINK-3367] - Annotate all user-facing API classes with @Public or @PublicEvolving +
[FLINK-3484] - Add setSlotSharingGroup documentation +
[FLINK-3490] - Bump Chill version to 0.7.4 +

+ +

Test

+ +

[FLINK-2480] - Improving tests coverage for org.apache.flink.streaming.api +
[FLINK-2573] - Add Kerberos test case +
[FLINK-2803] - Add test case for Flink's memory allocation +
[FLINK-3517] - Number of job and task managers not checked in scripts +

+ +

Wish

+ +

[FLINK-2429] - Remove the "enableCheckpointing()" without interval variant +
[FLINK-2995] - Set default number of retries to larger than 0 +
[FLINK-3377] - Remove final flag from ResultPartitionWriter class +

+ +

Known Issues

+ +

The mvn clean verify command does not succeed for the source release, because of issues with the Maven shade plugin. This has been resolved in the release-1.0 branch.
FLINK-3578 - Scala DataStream API does not support Rich Window Functions.

+ + +