Release 1.0.0 – Changelog and Known Issues

- -

Changelog
- Sub-task
- Bug
- Improvement
- Task
- Test
- Wish
-
Known Issues

- -

Changelog

- -

Sub-task

- -

[FLINK-7] - [GitHub] Enable Range Partitioner -
[FLINK-146] - Sorted output not working -
[FLINK-1982] - Remove dependencies on Record for Flink runtime and core -
[FLINK-2676] - Add abstraction for keyed window state -
[FLINK-2680] - Create a dedicated aligned-event time window operator -
[FLINK-2730] - Add CPU/Network utilization graphs to new web dashboard -
[FLINK-2732] - Add access to the TaskManagers' log file and out file in the web dashboard. -
[FLINK-2850] - Limit the types of jobs which can run in detached mode -
[FLINK-2853] - Apply JMH on MutableHashTablePerformanceBenchmark class. -
[FLINK-2869] - Apply JMH on IOManagerPerformanceBenchmark class. -
[FLINK-2889] - Apply JMH on LongSerializationSpeedBenchmark class -
[FLINK-2890] - Apply JMH on StringSerializationSpeedBenchmark class. -
[FLINK-2900] - Remove Record-API dependencies from Hadoop Compat module -
[FLINK-2901] - Several flink-test ITCases depend on Record API features -
[FLINK-2906] - Remove Record-API -
[FLINK-2919] - Apply JMH on FieldAccessMinibenchmark class. -
[FLINK-2920] - Apply JMH on KryoVersusAvroMinibenchmark class. -
[FLINK-2933] - Flink scala libraries exposed with maven should carry scala version -
[FLINK-2972] - Remove Twitter Chill dependency from flink-java module -
[FLINK-3057] - [py] Provide a way to pass information back to the plan process -
[FLINK-3070] - Create an asynchronous state handle interface -
[FLINK-3071] - Add asynchronous materialization thread -
[FLINK-3140] - NULL value data layout in Row Serializer/Comparator -
[FLINK-3195] - Restructure examples projects and package streaming examples -
[FLINK-3201] - Enhance Partitioned State Interface with State Types -
[FLINK-3208] - Rename Gelly vertex-centric model to scatter-gather -
[FLINK-3224] - The Streaming API does not call setInputType if a format implements InputTypeConfigurable -
[FLINK-3275] - [py] Add support for Dataset.setParallelism() -
[FLINK-3285] - Skip Maven deployment of flink-java8 -
[FLINK-3312] - Add convenience accessor methods for extended state interface -
[FLINK-3327] - Attach the ExecutionConfig to the JobGraph and make it accessible to the AbstractInvocable. -
[FLINK-3363] - JobManager does not shut down dedicated executor properly -
[FLINK-3365] - BlobLibraryCacheManager does not shutdown Timer thread -
[FLINK-3371] - Move TriggerCotext and TriggerResult to their own classes -
[FLINK-3384] - Create atomic closable queue for communication between Kafka Threads -
[FLINK-3388] - Expose task accumulators via JMX -
[FLINK-3401] - AscendingTimestampExtractor should not fail on order violation -
[FLINK-3437] - Fix UI router state for job plan -
[FLINK-3468] - Change Timestamp Extractors to not support negative timestamps -
[FLINK-3470] - EventTime WindowAssigners should error on negative timestamps -

- -

Bug

- -

[FLINK-1278] - Remove the Record special code paths -
[FLINK-1644] - WebClient dies when no ExecutionEnvironment in main method -
[FLINK-1989] - Sorting of POJO data set from TableEnv yields NotSerializableException -
[FLINK-2115] - TableAPI throws ExpressionException for "Dangling GroupBy operation" -
[FLINK-2263] - ExecutionGraph uses Thread.sleep to delay execution retries -
[FLINK-2315] - Hadoop Writables cannot exploit implementing NormalizableKey -
[FLINK-2348] - IterateExampleITCase failing -
[FLINK-2351] - Deprecate config builders in InputFormats and Output formats -
[FLINK-2369] - On Windows, in testFailingSortingDataSinkTask the temp file is not removed -
[FLINK-2392] - Instable test in flink-yarn-tests -
[FLINK-2443] - [CompactingHashTable] GSA Connected Components fails with NPE -
[FLINK-2491] - Operators are not participating in state checkpointing in some cases -
[FLINK-2504] - ExternalSortLargeRecordsITCase.testSortWithLongAndShortRecordsMixed failed spuriously -
[FLINK-2586] - Unstable Storm Compatibility Tests -
[FLINK-2608] - Arrays.asList(..) does not work with CollectionInputFormat -
[FLINK-2622] - Scala DataStream API does not have writeAsText method which supports WriteMode -
[FLINK-2624] - RabbitMQ source / sink should participate in checkpointing -
[FLINK-2662] - CompilerException: "Bug: Plan generation for Unions picked a ship strategy between binary plan operators." -
[FLINK-2671] - Instable Test StreamCheckpointNotifierITCase -
[FLINK-2674] - Rework windowing logic -
[FLINK-2695] - KafkaITCase.testConcurrentProducerConsumerTopology failed on Travis -
[FLINK-2719] - ProcessFailureStreamingRecoveryITCase>AbstractProcessFailureRecoveryTest.testTaskManagerProcessFailure failed on Travis -
[FLINK-2739] - Release script depends on the order of parent module information -
[FLINK-2747] - TypeExtractor does not correctly analyze Scala Immutables (AnyVal) -
[FLINK-2752] - Documentation is not easily differentiable from the Flink homepage -
[FLINK-2757] - DataSinkTaskTest fails on Windows -
[FLINK-2758] - TaskManagerRegistrationTest fails on Windows -
[FLINK-2759] - TaskManagerProcessReapingTest fails on Windows -
[FLINK-2769] - Web dashboard port not configurable on client side -
[FLINK-2771] - IterateTest.testSimpleIteration fails on Travis -
[FLINK-2797] - CLI: Missing option to submit jobs in detached mode -
[FLINK-2799] - Yarn tests cannot be executed with DEBUG log level -
[FLINK-2800] - kryo serialization problem -
[FLINK-2823] - YARN client should report a proper exception if Hadoop Env variables are not set -
[FLINK-2826] - transformed is modified in BroadcastVariableMaterialization#decrementReferenceInternal without proper locking -
[FLINK-2827] - Potential resource leak in TwitterSource#loadAuthenticationProperties() -
[FLINK-2832] - Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement -
[FLINK-2838] - Inconsistent use of URL and Path classes for resources -
[FLINK-2845] - TimestampITCase.testWatermarkPropagation -
[FLINK-2879] - Links in documentation are broken -
[FLINK-2913] - Close of ObjectOutputStream should be enclosed in finally block in FsStateBackend -
[FLINK-2914] - Missing break in ZooKeeperSubmittedJobGraphStore#SubmittedJobGraphsPathCacheListener#childEvent() -
[FLINK-2930] - ExecutionConfig execution retry delay not respected -
[FLINK-2934] - Remove placehoder pages in Web dashboard -
[FLINK-2937] - Typo in Quickstart->Scala API->Alternative Build Tools: SBT -
[FLINK-2938] - Streaming docs not in sync with latest state changes -
[FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic) -
[FLINK-2950] - Markdown presentation problem in SVM documentation -
[FLINK-2954] - Not able to pass custom environment variables in cluster to processes that spawning TaskManager -
[FLINK-2958] - StreamingJobGraphGenerator sets hard coded number execution retry -
[FLINK-2963] - Dependence on SerializationUtils#deserialize() should be avoided -
[FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs -
[FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster -
[FLINK-2979] - RollingSink does not work with Hadoop 2.7.1 -
[FLINK-2987] - Flink 0.10 fails to start on YARN 2.6.0 -
[FLINK-2989] - Job Cancel button doesn't work on Yarn -
[FLINK-2990] - Scala 2.11 build fails to start on YARN -
[FLINK-2992] - New Windowing code is using SerializationUtils with wrong classloader -
[FLINK-3000] - Add ShutdownHook to YARN CLI to prevent lingering sessions -
[FLINK-3005] - Commons-collections object deserialization remote command execution vulnerability -
[FLINK-3009] - Cannot build docs with Jekyll 3.0.0 -
[FLINK-3011] - Cannot cancel failing/restarting streaming job from the command line -
[FLINK-3013] - Incorrect package declaration in GellyScalaAPICompletenessTest.scala -
[FLINK-3019] - CLI does not list running/restarting jobs -
[FLINK-3020] - Local streaming execution: set number of task manager slots to the maximum parallelism -
[FLINK-3022] - Broken link 'Working With State' in Fault Tolerance Section of Stream Programming Guide -
[FLINK-3024] - TimestampExtractor Does not Work When returning Long.MIN_VALUE -
[FLINK-3025] - Flink Kafka consumer may get stuck due to Kafka/Zookeeper client bug -
[FLINK-3032] - Flink does not start on Hadoop 2.7.1 (HDP), due to class conflict -
[FLINK-3043] - Kafka Connector description in Streaming API guide is wrong/outdated -
[FLINK-3047] - Local batch execution: set number of task manager slots to the maximum parallelism -
[FLINK-3048] - DataSinkTaskTest.testCancelDataSinkTask -
[FLINK-3052] - Optimizer does not push properties out of bulk iterations -
[FLINK-3054] - Remove R (return) type variable from SerializationSchema -
[FLINK-3059] - Javadoc fix for DataSet.writeAsText() -
[FLINK-3061] - Kafka Consumer is not failing if broker is not available -
[FLINK-3062] - Kafka Producer is not failing if broker is not available/no partitions available -
[FLINK-3067] - Kafka source fails during checkpoint notifications with NPE -
[FLINK-3080] - Cannot union a data stream with a product of itself -
[FLINK-3081] - Kafka Periodic Offset Committer does not properly terminate on canceling -
[FLINK-3082] - Confusing error about ManualTimestampSourceFunction -
[FLINK-3087] - Table API do not support multi count in aggregation. -
[FLINK-3098] - Cast from Date to Long throw compile error. -
[FLINK-3100] - Signal handler prints error on normal shutdown of cluster -
[FLINK-3101] - Flink Kafka consumer crashes with NPE when it sees deleted record -
[FLINK-3103] - Remove synchronization in FsStateBackend#FsCheckpointStateOutputStream#close() -
[FLINK-3108] - JoinOperator's with() calls the wrong TypeExtractor method -
[FLINK-3117] - Storm Tick Tuples are not supported -
[FLINK-3118] - Check if MessageFunction implements ResultTypeQueryable -
[FLINK-3121] - Watermark forwarding does not work for sources not producing any data -
[FLINK-3125] - Web dashboard does not start when log files are not found -
[FLINK-3134] - Make YarnJobManager's allocate call asynchronous -
[FLINK-3136] - Scala Closure Cleaner uses wrong ASM import -
[FLINK-3138] - Method References are not supported as lambda expressions -
[FLINK-3143] - Update Clojure Cleaner's ASM references to ASM5 -
[FLINK-3144] - [storm] LocalCluster prints nothing without a configured logger -
[FLINK-3145] - Storm examples can't be run without flink-java as dependency -
[FLINK-3151] - YARN kills Flink TM containers due to memory overuse (outside heap/offheap) -
[FLINK-3156] - FlinkKafkaConsumer fails with NPE on notifyCheckpointComplete -
[FLINK-3157] - Web frontend json files contain author attribution -
[FLINK-3166] - The first program in ObjectReuseITCase has the wrong expected result, and it succeeds -
[FLINK-3171] - Consolidate zoo of wrapper classes for input/output-stream to data-input/output-view -
[FLINK-3173] - Bump org.apache.httpcomponents.httpclient version to 4.2.6 -
[FLINK-3175] - KafkaITCase.testOffsetAutocommitTest -
[FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned -
[FLINK-3180] - MemoryLogger does not log direct memory -
[FLINK-3185] - Silent failure during job graph recovery -
[FLINK-3188] - Deletes in Kafka source should be passed on to KeyedDeserializationSchema -
[FLINK-3189] - Error while parsing job arguments passed by CLI -
[FLINK-3196] - InputStream should be closed in EnvironmentInformation#getRevisionInformation() -
[FLINK-3197] - InputStream not closed in BinaryInputFormat#createStatistics -
[FLINK-3206] - Heap size for non-pre-allocated off-heap memory -
[FLINK-3218] - Merging Hadoop configurations overrides user parameters -
[FLINK-3220] - Flink does not start on Hortonworks Sandbox 2.3.2 due to missing class -
[FLINK-3236] - Flink user code classloader should have Flink classloader as parent classloader -
[FLINK-3240] - Remove or document DataStream(.global|.forward) -
[FLINK-3242] - User-specified StateBackend is not Respected if Checkpointing is Disabled -
[FLINK-3243] - Fix Interplay of TimeCharacteristic and Time Windows -
[FLINK-3247] - Kafka Connector unusable with quickstarts - shading issue -
[FLINK-3248] - RMQSource does not provide a constructor for credentials or other options -
[FLINK-3250] - Savepoint coordinator requires too strict parallelism match -
[FLINK-3251] - Checkpoint stats show ghost numbers -
[FLINK-3252] - Checkpoint stats only displayed after click on graph -
[FLINK-3254] - CombineFunction interface not respected -
[FLINK-3255] - Chaining behavior should not depend on parallelism -
[FLINK-3260] - ExecutionGraph gets stuck in state FAILING -
[FLINK-3261] - Tasks should eagerly report back when they cannot start a checkpoint -
[FLINK-3266] - LocalFlinkMiniCluster leaks resources when multiple jobs are submitted -
[FLINK-3267] - Disable reference tracking in Kryo fallback serializer -
[FLINK-3268] - Unstable test JobManagerSubmittedJobGraphsRecoveryITCase -
[FLINK-3271] - Using webhdfs in a flink topology throws classnotfound exception -
[FLINK-3274] - Prefix Kafka connector accumulators with unique id -
[FLINK-3280] - Wrong usage of Boolean.getBoolean() -
[FLINK-3281] - IndexOutOfBoundsException when range-partitioning empty DataSet -
[FLINK-3286] - Remove JDEB Debian Package code from flink-dist -
[FLINK-3287] - Flink Kafka Consumer fails due to Curator version conflict -
[FLINK-3289] - Double reference to flink-contrib -
[FLINK-3292] - Bug in flink-jdbc. Not all JDBC drivers supported -
[FLINK-3293] - Custom Application Name on YARN is ignored in deploy jobmanager mode -
[FLINK-3296] - DataStream.write*() methods are not flushing properly -
[FLINK-3300] - Concurrency Bug in Yarn JobManager -
[FLINK-3304] - AvroOutputFormat.setSchema() doesn't work in yarn-cluster mode -
[FLINK-3314] - Early cancel calls can cause Tasks to not cancel properly -
[FLINK-3322] - MemoryManager creates too much GC pressure with iterative jobs -
[FLINK-3328] - Incorrectly shaded dependencies in flink-runtime -
[FLINK-3333] - Documentation about object reuse should be improved -
[FLINK-3337] - mvn test fails on flink-runtime because curator classes not found -
[FLINK-3338] - Kafka deserialization issue - ClassNotFoundException -
[FLINK-3339] - Checkpointing NPE when using filterWithState -
[FLINK-3340] - Fix object juggling in drivers -
[FLINK-3341] - Kafka connector's 'auto.offset.reset' inconsistent with Kafka -
[FLINK-3342] - Operator checkpoint statistics state size overflow -
[FLINK-3350] - Increase timeouts on Travis Builds -
[FLINK-3351] - RocksDB Backend cannot determine correct local db path -
[FLINK-3352] - RocksDB Backend cannot determine correct hdfs path -
[FLINK-3353] - CSV-related tests may fail depending on locale -
[FLINK-3357] - Drop JobId.toShortString() -
[FLINK-3359] - Make RocksDB file copies asynchronous -
[FLINK-3361] - Wrong error messages for execution retry delay and akka ask pause config values -
[FLINK-3368] - Kafka 0.8 consumer fails to recover from broker shutdowns -
[FLINK-3369] - RemoteTransportException should be instance of CancelTaskException -
[FLINK-3373] - Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems -
[FLINK-3385] - Fix outer join skipping unprobed partitions -
[FLINK-3392] - Unprotected access to elements in ClosableBlockingQueue#size() -
[FLINK-3393] - ExternalProcessRunner wait to finish copying error stream -
[FLINK-3394] - Clear up the contract of MutableObjectIterator.next(reuse) -
[FLINK-3396] - Job submission Savepoint restore logic flawed -
[FLINK-3400] - RocksDB Backend does not work when not in Flink lib folder -
[FLINK-3412] - Remove implicit conversions JavaStream / ScalaStream -
[FLINK-3415] - TimestampExctractor accepts negative watermarks -
[FLINK-3416] - [py] .bat files fail when path contains spaces -
[FLINK-3420] - Remove "ReadTextFileWithValue" from StreamExecutionEnvironment -
[FLINK-3423] - ExternalProcessRunnerTest fails on Windows -
[FLINK-3424] - FileStateBackendtest.testStateOutputStream fails on windows -
[FLINK-3425] - DataSinkTaskTest.Failing[Sorting]DataSinkTask fails on Windows -
[FLINK-3426] - JobManagerLeader[Re]ElectionTest.testleaderElection fails on Windows -
[FLINK-3432] - ZookeeperOffsetHandlerTest fails on windows -
[FLINK-3434] - Return value from flinkYarnClient#deploy() should be checked against null -
[FLINK-3438] - ExternalProcessRunner fails to detect ClassNotFound exception because of locale settings -
[FLINK-3439] - Remove final Long.MAX_VALUE Watermark in StreamSource -
[FLINK-3440] - Kafka should also checkpoint partitions where no initial offset was retrieved -
[FLINK-3448] - WebRuntimeMonitor: Move initialization of handlers to start() method -
[FLINK-3449] - createInput swallows exception if TypeExtractor fails -
[FLINK-3450] - RocksDB Backed Window Fails with KryoSerializer -
[FLINK-3453] - Fix TaskManager logs exception when sampling backpressure while task completes -
[FLINK-3458] - Shading broken in flink-shaded-hadoop -
[FLINK-3478] - Flink serves arbitary files through the web interface -
[FLINK-3483] - Job graph visualization not working properly in OS X Chrome -
[FLINK-3485] - The SerializedListAccumulator value doest seem to be right -
[FLINK-3499] - Possible ghost references in ZooKeeper completed checkpoint store -
[FLINK-3511] - Flink library examples not runnable without adding dependencies -
[FLINK-3512] - Savepoint backend should not revert to "jobmanager" -
[FLINK-3513] - Fix interplay of automatic Operator UID and Changing name of WindowOperator -
[FLINK-3518] - Stale docs for quickstart setup -
[FLINK-3525] - Missing call to super#close() in TimestampsAndPeriodicWatermarksOperator#close() -
[FLINK-3526] - ProcessingTime Window Assigner and Trigger are broken -
[FLINK-3534] - Cancelling a running job can lead to restart instead of stopping -
[FLINK-3540] - Hadoop 2.6.3 build contains /com/google/common (guava) classes in flink-dist.jar -
[FLINK-3554] - Bounded sources should emit a Max Watermark when they are done -
[FLINK-3556] - Unneeded check in HA blob store configuration -
[FLINK-3562] - Update docs in the course of EventTimeSourceFunction removal -
[FLINK-3565] - FlinkKafkaConsumer does not work with Scala 2.11 -
[FLINK-3569] - Test cases fail due to Maven Shade plugin -

- -

Improvement

- -

[FLINK-8] - [GitHub] Implement automatic sample histogram building for Range Partitioning -
[FLINK-553] - Add getGroupKey() method to group-at-time operators -
[FLINK-573] - Clean-up MapOperators in optimizer -
[FLINK-734] - Integrate web job client into JobManager web interface -
[FLINK-987] - Extend TypeSerializers and -Comparators to work directly on Memory Segments -
[FLINK-1045] - Remove Combinable Annotation -
[FLINK-1228] - Add REST Interface to JobManager -
[FLINK-1240] - We cannot use sortGroup on a global reduce -
[FLINK-1513] - Remove GlobalConfiguration Singleton -
[FLINK-1666] - Clean-up Field Expression Code -
[FLINK-1702] - Authenticate via Kerberos from the client only -
[FLINK-1778] - Improve normalized keys in composite key case -
[FLINK-1903] - Joins where one side uses a field more than once don't work -
[FLINK-1947] - Make Avro and Tachyon test logging less verbose -
[FLINK-2017] - Add predefined required parameters to ParameterTool -
[FLINK-2021] - Rework examples to use ParameterTool -
[FLINK-2075] - Shade akka and protobuf dependencies away -
[FLINK-2185] - Rework semantics for .setSeed function of SVM -
[FLINK-2213] - Configure number of vcores -
[FLINK-2239] - print() on DataSet: stream results and print incrementally -
[FLINK-2342] - Add new fit operation and more tests for StandardScaler -
[FLINK-2344] - Deprecate/Remove the old Pact Pair type -
[FLINK-2380] - Allow to configure default FS for file inputs -
[FLINK-2399] - Fail when actor versions don't match -
[FLINK-2455] - Misleading I/O manager error log messages -
[FLINK-2488] - Expose attemptNumber in RuntimeContext -
[FLINK-2518] - Avoid predetermination of ports for network services -
[FLINK-2523] - Make task canceling interrupt interval configurable -
[FLINK-2524] - Add "getTaskNameWithSubtasks()" to RuntimeContext -
[FLINK-2559] - Fix Javadoc (e.g. Code Examples) -
[FLINK-2646] - Rich functions should provide a method "closeAfterFailure()" -
[FLINK-2667] - Rework configuration parameters -
[FLINK-2716] - Checksum method for DataSet and Graph -
[FLINK-2788] - Add type hint with TypeExtactor call on Hint Type -
[FLINK-2795] - Print JobExecutionResult for interactively invoked jobs -
[FLINK-2882] - Improve performance of string conversions -
[FLINK-2893] - Rename recovery configuration keys -
[FLINK-2895] - Duplicate immutable object creation -
[FLINK-2897] - Use distinct initial indices for OutputEmitter round-robin -
[FLINK-2898] - Invert Travis CI build order -
[FLINK-2902] - Web interface sort tasks newest first -
[FLINK-2904] - Web interface truncated task counts -
[FLINK-2932] - Flink quickstart docs should ask users to download from https, not http -
[FLINK-2936] - ClassCastException when using EventTimeSourceFunction in non-EventTime program -
[FLINK-2940] - Deploy multiple Scala versions for Maven artifacts -
[FLINK-2947] - Coloured Scala Shell -
[FLINK-2949] - Add method 'writeSequencefile' to DataSet -
[FLINK-2961] - Add support for basic type Date in Table API -
[FLINK-2962] - Cluster startup script refers to unused variable -
[FLINK-2966] - Improve the way job duration is reported on web frontend. -
[FLINK-2974] - Add periodic offset commit to Kafka Consumer if checkpointing is disabled -
[FLINK-2976] - Save and load checkpoints manually -
[FLINK-2981] - Update README for building docs -
[FLINK-2984] - Support lenient parsing of SVMLight input files -
[FLINK-2991] - Extend Window Operators to Allow Efficient Fold Operation -
[FLINK-2993] - Set default DelayBetweenExecutionRetries to 0 -
[FLINK-2994] - Client sysout logging does not report exceptions -
[FLINK-3003] - Add container allocation timeout to YARN CLI -
[FLINK-3017] - Broken 'Slots' link on Streaming Programming Guide -
[FLINK-3023] - Show Flink version + commit id for -SNAPSHOT versions in web frontend -
[FLINK-3028] - Cannot cancel restarting job via web frontend -
[FLINK-3030] - Enhance Dashboard to show Execution Attempts -
[FLINK-3039] - Trigger KeyValueState cannot be Scala Int -
[FLINK-3040] - Add docs describing how to configure State Backends -
[FLINK-3042] - Define a way to let types create their own TypeInformation -
[FLINK-3044] - In YARN mode, configure FsStateBackend by default. -
[FLINK-3045] - Properly expose the key of a kafka message -
[FLINK-3046] - Integrate the Either Java type with the TypeExtractor -
[FLINK-3049] - Move "Either" type to package "org.apache.flink.types" -
[FLINK-3050] - Add custom Exception type to suppress job restarts -
[FLINK-3051] - Define a maximum number of concurrent inflight checkpoints -
[FLINK-3055] - ExecutionVertex has duplicate method getParallelSubtaskIndex and getSubTaskIndex -
[FLINK-3056] - Show bytes sent/received as MBs/GB and so on in web interface -
[FLINK-3063] - [py] Remove combiner -
[FLINK-3069] - Make state materialization asynchronous -
[FLINK-3073] - Activate streaming mode by default -
[FLINK-3074] - Make ApplicationMaster/JobManager akka port configurable -
[FLINK-3076] - Display channel exchange mode in the plan visualizer -
[FLINK-3077] - Add "version" command to CliFrontend for showing the version of the installation -
[FLINK-3083] - Add docs how to configure streaming fault tolerance -
[FLINK-3084] - File State Backend should not write very small state into files -
[FLINK-3085] - Move State Backend Initialization from "registerInputOutput()" to "invoke()" -
[FLINK-3114] - Read cluster's default parallelism upon remote job submission -
[FLINK-3115] - Update Elasticsearch connector to 2.X -
[FLINK-3116] - Remove RecordOperator -
[FLINK-3120] - Set number of event loop thread and number of buffer pool arenas to same number -
[FLINK-3122] - Generalize value type in LabelPropagation -
[FLINK-3123] - Allow setting custom start-offsets for the Kafka consumer -
[FLINK-3124] - Introduce a TaskInfo object to better represent task name, index, attempt number etc. -
[FLINK-3127] - Measure backpressure in Flink jobs -
[FLINK-3131] - Expose checkpoint metrics -
[FLINK-3132] - Restructure streaming guide -
[FLINK-3133] - Introduce collect()/coun()/print() methods in DataStream API -
[FLINK-3135] - Add chainable driver for UNARY_NO_OP strategy -
[FLINK-3147] - HadoopOutputFormatBase should expose fields as protected -
[FLINK-3154] - Update Kryo version from 2.24.0 to 3.0.3 -
[FLINK-3160] - Aggregate operator statistics by TaskManager -
[FLINK-3161] - Externalize cluster start-up and tear-down when available -
[FLINK-3169] - Drop type from Record Data Model -
[FLINK-3172] - Specify jobmanager port in HA mode -
[FLINK-3174] - Add merging WindowAssigner -
[FLINK-3176] - Window Apply Website Example -
[FLINK-3178] - Make Closing Behavior of Window Operators Configurable -
[FLINK-3181] - The vertex-centric SSSP example and library method send unnecessary messages during the first superstep -
[FLINK-3194] - Remove web client -
[FLINK-3198] - Rename Grouping.getDataSet() method and add JavaDocs -
[FLINK-3200] - Use Partitioned State Abstraction in WindowOperator -
[FLINK-3209] - Remove Unused ProcessingTime, EventTime and AbstractTime -
[FLINK-3213] - Union of two streams with different parallelism should adjust parallelism automatically -
[FLINK-3219] - Implement DataSet.count using a single operator -
[FLINK-3232] - Add option to eagerly deploy channels -
[FLINK-3233] - PartitionOperator does not support expression keys on atomic types -
[FLINK-3234] - SortPartition does not support KeySelectorFunctions -
[FLINK-3235] - Drop Flink-on-Tez code -
[FLINK-3244] - Add log messages to savepoint coordinator restore -
[FLINK-3246] - Consolidate maven project names with *-parent suffix -
[FLINK-3249] - Wrong "unknown partition state/input gate" error messages -
[FLINK-3258] - Merge AbstractInvokable's registerInputOutput and invoke -
[FLINK-3259] - Redirect programming guides to new layout -
[FLINK-3262] - Remove fuzzy versioning from Bower dependencies -
[FLINK-3265] - RabbitMQ Source not threadsafe: ConcurrentModificationException -
[FLINK-3270] - Add example for reading and writing to Kafka -
[FLINK-3273] - Remove Scala dependency from flink-streaming-java -
[FLINK-3276] - Move runtime parts of flink-streaming-java to flink-runtime -
[FLINK-3278] - Add Partitioned State Backend Based on RocksDB -
[FLINK-3290] - [py] Generalize OperationInfo transfer -
[FLINK-3299] - Remove ApplicationID from Environment -
[FLINK-3303] - Move all non-batch specific classes in flink-java to flink-core -
[FLINK-3305] - Remove JavaKaffee serializer util -
[FLINK-3306] - Auto type registration at Kryo is buggy -
[FLINK-3308] - [py] Remove debug mode -
[FLINK-3309] - [py] Resolve maven warnings -
[FLINK-3310] - Add back pressure statistics to web frontend -
[FLINK-3334] - Change default log4j.properties Conversion pattern to include year-month-day in the timestamp -
[FLINK-3336] - Add Semi-Rebalance Data Shipping for DataStream -
[FLINK-3347] - TaskManager ActorSystems need to restart themselves in case they notice quarantine -
[FLINK-3348] - taskmanager.memory.off-heap missing bc documentation -
[FLINK-3355] - Allow passing RocksDB Option to RocksDBStateBackend -
[FLINK-3358] - Create constructors for FsStateBackend in RocksDBBackens -
[FLINK-3364] - Don't initialize SavepointStore in JobManager constructor -
[FLINK-3375] - Allow Watermark Generation in the Kafka Source -
[FLINK-3376] - Add an illustration of Event Time and Watermarks to the docs -
[FLINK-3378] - Consolidate TestingCluster and FokableFlinkMiniCluster -
[FLINK-3379] - Refactor TimestampExtractor -
[FLINK-3386] - Kafka consumers are not properly respecting the "auto.offset.reset" behavior -
[FLINK-3389] - Add Pre-defined Options settings for RocksDB State backend -
[FLINK-3395] - Polishing the web UI -
[FLINK-3402] - Refactor Common Parts of Stream/Batch Documentation -
[FLINK-3413] - Remove implicit Seq to DataStream conversion -
[FLINK-3419] - Drop partitionByHash from DataStream -
[FLINK-3421] - Remove all unused ClassTag context bounds in the Streaming Scala API -
[FLINK-3422] - Scramble HashPartitioner hashes -
[FLINK-3430] - Remove "no POJO" warning in TypeAnalyzer -
[FLINK-3435] - Change interplay of Ingestion Time and Event Time -
[FLINK-3436] - Remove ComplexIntegrationITCase -
[FLINK-3455] - Bump Kafka 0.9 connector dependency to Kafka 0.9.0.1 -
[FLINK-3459] - Make build SBT compatible -
[FLINK-3460] - Make flink-streaming-connectors' flink dependencies provided -
[FLINK-3461] - Remove duplicate condition check in ZooKeeperLeaderElectionService -
[FLINK-3469] - Improve documentation for grouping keys -
[FLINK-3509] - Update Hadoop versions in release script and on travis to the latest minor version -
[FLINK-3520] - Periodic watermark operator should emit current watermark in close() -
[FLINK-3521] - Make Iterable part of method signature for WindowFunction -
[FLINK-3522] - Storm examples "PrintSampleStream" throws an error if called without arguments -
[FLINK-3527] - Scala DataStream has no transform method -
[FLINK-3528] - Add Incremental Fold for Non-Keyed Window Operator -
[FLINK-3535] - Decrease logging verbosity of StackTraceSampleCoordinator -
[FLINK-3536] - Make clearer distinction between event time and processing time -
[FLINK-3538] - DataStream join API does not enforce consistent usage -
[FLINK-3548] - Remove unnecessary generic parameter from SingleOutputStreamOperator -
[FLINK-3559] - Don't print pid file check if no active PID -

- -

New Feature -

[FLINK-1723] - Add cross validation for model evaluation -
[FLINK-2157] - Create evaluation framework for ML library -
[FLINK-2390] - Replace iteration timeout with algorithm for detecting termination -
[FLINK-2411] - Add basic graph summarization algorithm -
[FLINK-2435] - Add support for custom CSV field parsers -
[FLINK-2720] - Add Storm-CountMetric in flink-stormcompatibility -
[FLINK-2728] - Add missing features to new web dashboard -
[FLINK-2871] - Add OuterJoin strategy with HashTable on outer side -
[FLINK-2905] - Add intersect method to Graph class -
[FLINK-2951] - Add Union operator to Table API. -
[FLINK-2955] - Add operations introduction in Table API page. -
[FLINK-2978] - Integrate web submission interface into the new dashboard -
[FLINK-2996] - Add config entry to define BlobServer port -
[FLINK-3001] - Add Support for Java 8 Optional type -
[FLINK-3002] - Add an EitherType to the Java API -
[FLINK-3058] - Add Kafka consumer for new 0.9.0.0 Kafka API -
[FLINK-3093] - Introduce annotations for interface stability -
[FLINK-3102] - Allow reading from multiple topics with one FlinkKafkaConsumer -
[FLINK-3170] - Expose task manager metrics via JMX -
[FLINK-3192] - Add explain support to print ast and sql physical execution plan. -

- -

Task

- -

[FLINK-1681] - Remove the old Record API -
[FLINK-2973] - Add flink-benchmark with compliant licenses again -
[FLINK-3112] - Remove unused RecordModelPostPass class -
[FLINK-3113] - Remove unused global order methods from GenericDataSinkBase -
[FLINK-3155] - Update Flink docker version to latest stable Flink version -
[FLINK-3186] - Deprecate DataSink.sortLocalOutput() methods -
[FLINK-3366] - Rename @Experimental annotation to @PublicEvolving -
[FLINK-3367] - Annotate all user-facing API classes with @Public or @PublicEvolving -
[FLINK-3484] - Add setSlotSharingGroup documentation -
[FLINK-3490] - Bump Chill version to 0.7.4 -

- -

Test

- -

[FLINK-2480] - Improving tests coverage for org.apache.flink.streaming.api -
[FLINK-2573] - Add Kerberos test case -
[FLINK-2803] - Add test case for Flink's memory allocation -
[FLINK-3517] - Number of job and task managers not checked in scripts -

- -

Wish

- -

[FLINK-2429] - Remove the "enableCheckpointing()" without interval variant -
[FLINK-2995] - Set default number of retries to larger than 0 -
[FLINK-3377] - Remove final flag from ResultPartitionWriter class -

- -

Known Issues

- -

The mvn clean verify command does not succeed for the source release, because of issues with the Maven shade plugin. This has been resolved in the release-1.0 branch.
FLINK-3578 - Scala DataStream API does not support Rich Window Functions.

- - -