flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From azagre...@apache.org
Subject [flink] branch flip116 updated (3100234 -> 44b10ab)
Date Tue, 21 Apr 2020 08:10:44 GMT
This is an automated email from the ASF dual-hosted git repository.

azagrebin pushed a change to branch flip116
in repository https://gitbox.apache.org/repos/asf/flink.git.


 discard 3100234  change type to cron
 discard c9568d0  [FLINK-17048] Add memory related JVM args to Mesos JM startup scripts
 discard 0c89116  [FLINK-16746][conf] Deprecate jobmanager.heap.size
 discard 34d1295  [FLINK-16746] Add default 'jobmanager.memory.process.size: 1472m' to flink-conf.yaml
 discard 0f1bbee  [hotfix] Remove 'TaskManager' from MemoryBackwardsCompatibilityUtils#getLegacyTaskManagerHeapMemoryIfExplicitlyConfigured
 discard 46e0dc3  [FLINK-16745][coordination] Remove unused container cutoff
 discard 73b0949  [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
 discard 1a831ec  [FLINK-16745][yarn] Start Yarn JM with FLIP-116 JVM memory args
 discard 23f62bd  [FLINK-16745][clients] Parse JobManagerProcessSpec from Configuration into
ClusterSpecification
 discard 0c43e44  [FLINK-16742][runtime] Refactor BashJavaUtils, move to separated package
and extract configuration loading logics.
 discard d3bc2a6  [FLINK-16472][dist] Remove unused memory configuration logics from config.sh.
 discard e099019  [FLINK-16742][dist] Extend and use BashJavaUtils for JM memory configuration
in start-up scripts.
 discard b048e89  [FLINK-16742][runtime] Ignore unknown command line options for BashJavaUtils.
 discard 317b6f5  [FLINK-16742][dist] config.sh does not assume BashJavaUtils always return
two lines of results.
 discard 6ee9c4a  [hotfix][runtime] Code clean-ups in BashJavaUtils and its test cases.
 discard 8d60c0e  [FLINK-16874] Respect the dynamic options when calculating memory options
in taskmanager.sh
 discard c4ffce9  [hotfix] Fix the code style in BashJavaUtilsITCase
     add fabeb9f  Revert "[hotfix][docs] Fix overlapping text"
     add fa816c5  [FLINK-15816][k8s] Limit the value of kubernetes.cluster-id to have no more
than 45 characters
     add 691fed1  [FLINK-16992][table-common] Add all ability interfaces for table sourcs
and sinks
     add ac44e1e  [FLINK-17156][checkpointing][hotfix] simplify cancellation in CheckpointBarrierUnaligner
     add a7d80fb  [FLINK-17156][checkpointing] ignore cancelled checkpoints by CheckpointBarrierUnaligner
     add fe78c9b  [FLINK-17257][yarn][test] Fix AbstractYarnClusterTest not compiling with
hadoop 2.10.
     add 4ed6a0f  [hotfix] Fix the code style in BashJavaUtilsITCase
     add b4618ae  [FLINK-16874] Respect the dynamic options when calculating memory options
in taskmanager.sh
     add 7d65fe2  [hotfix] Add RpcUtils#terminateRpcEndpoints
     add eb56ab4  [hotfix] Add RpcUtils#INF_DURATION
     add 03b5115  [hotfix] Add RpcRuntimeException
     add 067b9c0  [hotfix] Add AkkRpcRuntimeException
     add ef0ae0b  [hotfix] Let AkkaUnknownMessageException extend AkkaRpcRuntimeException
     add db375bb  [FLINK-15347] Bump AkkRpcVersion to 2
     add 4c63f21  [FLINK-15347][tests] Add ActorSystemResource
     add f0529de  [FLINK-15347] Add SupervisorActor which monitors the proper termination
of AkkaRpcActors
     add 2cf7955  [FLINK-15347] Introduce separate dispatcher for SupervisorActor
     add df30edd  [hotfix] Add FutureUtils.forwardAsync
     add 422aa89  [FLINK-15347] Add dedicated termination future executor to SupervisorActor
     add 43876ff  [FLINK-17263][table-planner-blink] Replace RepeatFamilyOperandTypeChecker
with CompositeOperandTypeChecker in planner
     add 34105c7  [FLINK-17135][python][tests] Fix the test testPandasFunctionMixedWithGeneralPythonFunction
to make it more stable
     add 6b80935  [FLINK-17120][python] Add Cython support for operations (#11784)
     add bda9b77  [FLINK-17013][python] Support Python UDTF in old planner under batch mode
(#11668)
     add 67835fe  [FLINK-17014][runtime] TestingSchedulingTopology implements pipelined region
getter interfaces
     add 99cbaa9  [FLINK-17014][runtime] Implement PipelinedRegionSchedulingStrategy
     add 3ae1da0  [FLINK-17264][scripts] Fix broken taskmanager.sh by using -ne to compare
num_lines
     add 036197e  [FLINK-17266] Make WorkerResourceSpec serializable
     add 89cc0b9  [FLINK-17197][runtime] switch ContinuousFileReaderOperator state to FAILED
on error
     add e806e95  [hotfix][runtime] move ContinuousFileReaderOperator unit test to the appropriate
class and module
     add 7269e6c  [hotfix][runtime] rename ContinuousFileReaderOperator.TRANSITIONS to VALID_TRANSITIONS
     add 3df3f60  [hotfix][runtime] Code clean-ups in BashJavaUtils and its test cases.
     add 92e8c69  [FLINK-16742][dist] config.sh does not assume BashJavaUtils always return
two lines of results.
     add 31dd9b6  [FLINK-16742][runtime] Ignore unknown command line options for BashJavaUtils.
     add 795bafc  [FLINK-16742][dist] Extend and use BashJavaUtils for JM memory configuration
in start-up scripts.
     add 1f7dd29  [FLINK-16472][dist] Remove unused memory configuration logics from config.sh.
     add e2ab453  [FLINK-16742][runtime] Refactor BashJavaUtils, move to separated package
and extract configuration loading logics.
     add b1577a2  [FLINK-16745][clients] Parse JobManagerProcessSpec from Configuration into
ClusterSpecification
     add 29a3baf  [FLINK-16745][yarn] Start Yarn JM with FLIP-116 JVM memory args
     add 94feed4  [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
     add 3cbf22f  [FLINK-16745][coordination] Remove unused container cutoff
     add 2f359ca  [hotfix] Remove 'TaskManager' from MemoryBackwardsCompatibilityUtils#getLegacyTaskManagerHeapMemoryIfExplicitlyConfigured
     add b737e73  [FLINK-16746] Add default 'jobmanager.memory.process.size: 1600m' to flink-conf.yaml
     add 3ba8e78  [FLINK-16746][conf] Deprecate jobmanager.heap.size and expose new memory
options
     new 774dba9  fixup! [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
     new 1e4c0b7  [FLINK-17048] Add memory related JVM args to Mesos JM startup scripts
     new 44b10ab  change type to cron

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (3100234)
            \
             N -- N -- N   refs/heads/flip116 (44b10ab)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../generated/kubernetes_config_configuration.html |    2 +-
 docs/fig/program_dataflow.svg                      | 1062 ++++++++++----------
 flink-dist/src/main/flink-bin/bin/config.sh        |   15 +-
 flink-dist/src/main/resources/flink-conf.yaml      |    2 +-
 .../hdfstests/ContinuousFileProcessingTest.java    |   18 -
 flink-jepsen/src/jepsen/flink/db.clj               |    2 +-
 .../kubernetes/KubernetesClusterClientFactory.java |    7 +-
 .../configuration/KubernetesConfigOptions.java     |    4 +-
 .../parameters/AbstractKubernetesParameters.java   |    8 +-
 .../apache/flink/kubernetes/utils/Constants.java   |    2 +
 .../AbstractKubernetesParametersTest.java          |   96 ++
 .../store/MesosWorkerStoreTest.java                |   51 +
 flink-python/pyflink/fn_execution/coders.py        |   20 +-
 .../pyflink/fn_execution/fast_operations.pxd       |   46 +
 .../pyflink/fn_execution/fast_operations.pyx       |  253 +++++
 .../pyflink/fn_execution/sdk_worker_main.py        |    5 +-
 flink-python/pyflink/table/tests/test_udtf.py      |   44 +-
 flink-python/setup.py                              |   41 +-
 ...=> AbstractPythonStatelessFunctionFlatMap.java} |  256 ++---
 .../python/PythonScalarFunctionFlatMap.java        |  274 +----
 .../python/PythonTableFunctionFlatMap.java}        |  128 +--
 .../python/table/PythonTableFunctionOperator.java  |    4 +-
 ...tegy.java => EscalatingSupervisorStrategy.java} |   21 +-
 .../flink/runtime/concurrent/FutureUtils.java      |   22 +-
 .../resourcemanager/WorkerResourceSpec.java        |    5 +-
 .../org/apache/flink/runtime/rpc/RpcUtils.java     |   37 +-
 .../flink/runtime/rpc/akka/AkkaRpcService.java     |  122 ++-
 .../runtime/rpc/akka/AkkaRpcServiceUtils.java      |    4 +-
 .../flink/runtime/rpc/akka/SupervisorActor.java    |  366 +++++++
 ...Exception.java => AkkaRpcRuntimeException.java} |   22 +-
 .../exceptions/AkkaUnknownMessageException.java    |    2 +-
 .../RpcRuntimeException.java}                      |   21 +-
 .../PipelinedRegionSchedulingStrategy.java         |  162 +++
 .../strategy/SchedulingStrategyUtils.java          |   12 +
 .../org/apache/flink/runtime/akka/AkkaUtils.scala  |   11 +-
 .../flink/runtime/concurrent/FutureUtilsTest.java  |   18 +
 .../flink/runtime/rpc/FencedRpcEndpointTest.java   |    4 +-
 .../runtime/rpc/akka/ActorSystemResource.java      |   64 ++
 .../runtime/rpc/akka/AkkaActorSystemTest.java      |   89 ++
 .../flink/runtime/rpc/akka/AkkaRpcActorTest.java   |   38 +
 .../runtime/rpc/akka/SupervisorActorTest.java      |  249 +++++
 .../PipelinedRegionSchedulingStrategyTest.java     |  160 +++
 .../strategy/TestingSchedulingPipelinedRegion.java |   62 ++
 .../strategy/TestingSchedulingResultPartition.java |    4 +
 .../strategy/TestingSchedulingTopology.java        |   47 +
 .../retriever/impl/RpcGatewayRetrieverTest.java    |    3 +-
 .../source/ContinuousFileReaderOperator.java       |   30 +-
 .../runtime/io/CheckpointBarrierUnaligner.java     |   72 +-
 .../source/ContinuousFileReaderOperatorTest.java   |  101 ++
 ...CheckpointBarrierUnalignerCancellationTest.java |  138 +++
 .../PeriodicWatermarkAssignerProvider.java         |   28 +-
 .../PunctuatedWatermarkAssignerProvider.java       |   28 +-
 .../table/connector/sink/DynamicTableSink.java     |    5 +-
 .../sink/abilities/SupportsOverwrite.java          |   31 +-
 .../sink/abilities/SupportsPartitioning.java       |  117 +++
 .../table/connector/source/ScanTableSource.java    |   10 +
 .../abilities/SupportsComputedColumnPushDown.java  |   24 +-
 .../source/abilities/SupportsFilterPushDown.java   |    8 +
 .../source/abilities/SupportsLimitPushDown.java    |   43 +
 .../abilities/SupportsPartitionPushDown.java       |   80 ++
 .../abilities/SupportsProjectionPushDown.java      |   72 ++
 .../abilities/SupportsWatermarkPushDown.java       |   79 ++
 .../functions/python/PythonTableFunction.java      |   10 +-
 .../functions/sql/FlinkSqlOperatorTable.java       |    6 +-
 .../plan/type/RepeatFamilyOperandTypeChecker.java  |  125 ---
 .../plan/rules/logical/PythonCalcSplitRuleTest.xml |    8 +-
 .../rules/logical/PythonCalcSplitRuleTest.scala    |    2 +-
 ...e.java => AbstractPythonCorrelateRuleBase.java} |   67 +-
 .../rules/batch/DataSetPythonCorrelateRule.java    |   73 ++
 .../stream/DataStreamPythonCorrelateRule.java      |   76 ++
 .../table/plan/nodes/CommonPythonCorrelate.scala   |   31 +-
 .../plan/nodes/dataset/DataSetCorrelate.scala      |   48 +-
 ...tCorrelate.scala => DataSetCorrelateBase.scala} |   74 +-
 .../nodes/dataset/DataSetPythonCorrelate.scala     |  107 ++
 .../flink/table/plan/rules/FlinkRuleSets.scala     |    4 +-
 .../plan/rules/dataSet/DataSetCorrelateRule.scala  |   10 +-
 .../flink/table/plan/PythonCalcSplitRuleTest.scala |    6 +-
 .../apache/flink/yarn/AbstractYarnClusterTest.java |   36 +-
 78 files changed, 3834 insertions(+), 1600 deletions(-)
 create mode 100644 flink-kubernetes/src/test/java/org/apache/flink/kubernetes/kubeclient/parameters/AbstractKubernetesParametersTest.java
 create mode 100644 flink-mesos/src/test/java/org/apache/flink/mesos/runtime/clusterframework/store/MesosWorkerStoreTest.java
 create mode 100644 flink-python/pyflink/fn_execution/fast_operations.pxd
 create mode 100644 flink-python/pyflink/fn_execution/fast_operations.pyx
 copy flink-python/src/main/java/org/apache/flink/table/runtime/functions/python/{PythonScalarFunctionFlatMap.java
=> AbstractPythonStatelessFunctionFlatMap.java} (63%)
 copy flink-python/src/main/java/org/apache/flink/table/runtime/{operators/python/table/PythonTableFunctionOperator.java
=> functions/python/PythonTableFunctionFlatMap.java} (53%)
 rename flink-runtime/src/main/java/org/apache/flink/runtime/akka/{StoppingSupervisorWithoutLoggingActorKilledExceptionStrategy.java
=> EscalatingSupervisorStrategy.java} (61%)
 create mode 100644 flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/SupervisorActor.java
 copy flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/exceptions/{AkkaUnknownMessageException.java
=> AkkaRpcRuntimeException.java} (66%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/rpc/{akka/exceptions/AkkaUnknownMessageException.java
=> exceptions/RpcRuntimeException.java} (62%)
 create mode 100644 flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/strategy/PipelinedRegionSchedulingStrategy.java
 create mode 100644 flink-runtime/src/test/java/org/apache/flink/runtime/rpc/akka/ActorSystemResource.java
 create mode 100644 flink-runtime/src/test/java/org/apache/flink/runtime/rpc/akka/AkkaActorSystemTest.java
 create mode 100644 flink-runtime/src/test/java/org/apache/flink/runtime/rpc/akka/SupervisorActorTest.java
 create mode 100644 flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/strategy/PipelinedRegionSchedulingStrategyTest.java
 create mode 100644 flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/strategy/TestingSchedulingPipelinedRegion.java
 create mode 100644 flink-streaming-java/src/test/java/org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperatorTest.java
 create mode 100644 flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/CheckpointBarrierUnalignerCancellationTest.java
 copy flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/exceptions/AkkaUnknownMessageException.java
=> flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/connector/source/abilities/PeriodicWatermarkAssignerProvider.java
(56%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/exceptions/AkkaUnknownMessageException.java
=> flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/connector/source/abilities/PunctuatedWatermarkAssignerProvider.java
(55%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/exceptions/AkkaUnknownMessageException.java
=> flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsOverwrite.java
(58%)
 create mode 100644 flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsPartitioning.java
 create mode 100644 flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/abilities/SupportsLimitPushDown.java
 create mode 100644 flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/abilities/SupportsPartitionPushDown.java
 create mode 100644 flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/abilities/SupportsProjectionPushDown.java
 create mode 100644 flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/abilities/SupportsWatermarkPushDown.java
 delete mode 100644 flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/type/RepeatFamilyOperandTypeChecker.java
 rename flink-table/flink-table-planner/src/main/java/org/apache/flink/table/plan/rules/{datastream/DataStreamPythonCorrelateRule.java
=> AbstractPythonCorrelateRuleBase.java} (62%)
 create mode 100644 flink-table/flink-table-planner/src/main/java/org/apache/flink/table/plan/rules/batch/DataSetPythonCorrelateRule.java
 create mode 100644 flink-table/flink-table-planner/src/main/java/org/apache/flink/table/plan/rules/stream/DataStreamPythonCorrelateRule.java
 copy flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/plan/nodes/dataset/{DataSetCorrelate.scala
=> DataSetCorrelateBase.scala} (57%)
 create mode 100644 flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/plan/nodes/dataset/DataSetPythonCorrelate.scala


Mime
View raw message