beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-301) Add a Beam SQL DSL
Date Wed, 30 Aug 2017 18:13:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147718#comment-16147718
] 

ASF GitHub Bot commented on BEAM-301:
-------------------------------------

GitHub user XuMingmin opened a pull request:

    https://github.com/apache/beam/pull/3794

    [TEST PR] to verify [BEAM-301]

    Follow this checklist to help us incorporate your contribution quickly and easily:
    
     - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/)
filed for the change (usually before you start working on it).  Trivial changes like typos
do not require a JIRA issue.  Your pull request should address just this issue, without pulling
in other changes.
     - [ ] Each commit in the pull request should have a meaningful subject line and body.
     - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`,
where you replace `BEAM-XXX` with the appropriate JIRA issue.
     - [ ] Write a pull request description that is detailed enough to understand what the
pull request does, how, and why.
     - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
     - [ ] If this contribution is large, please file an Apache [Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
    
    ---


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/XuMingmin/beam VERIFY_MERGE

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/3794.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3794
    
----
commit 95e14214cbf49c8358872c9d34edc6713bc02e0d
Author: Kenneth Knowles <klk@google.com>
Date:   2017-05-26T23:07:45Z

    Port DirectRunner WriteFiles override to SDK-agnostic APIs

commit 4a350608100ad3f5dd3bf2798fc9976dcec95467
Author: Kenneth Knowles <klk@google.com>
Date:   2017-06-08T20:39:32Z

    Port DirectRunner TestStream override to SDK-agnostic APIs

commit fd74b5ca83a32afc67a318f1f791726c6326c6b4
Author: Ismaël Mejía <iemejia@apache.org>
Date:   2017-06-05T21:20:27Z

    [BEAM-2412] Update HBaseIO to use HBase client 1.2.6

commit 43b8e5f7b2e33c10d823037fa3af3e04e19c373d
Author: JingsongLi <lzljs3620320@aliyun.com>
Date:   2017-06-07T06:34:25Z

    Use CoderTypeSerializer and remove unuse code in FlinkStateInternals

commit 1d6c466018aa85a7ef41a0535a1db744fd18e87e
Author: JingsongLi <lzljs3620320@aliyun.com>
Date:   2017-06-07T06:40:30Z

    [BEAM-1483] Support SetState in Flink runner and fix MapState to be consistent with InMemoryStateInternals.

commit cfbeee2e5bf2ef2d0d0ca839ed2c915e302b0f14
Author: JingsongLi <lzljs3620320@aliyun.com>
Date:   2017-06-07T17:31:34Z

    [BEAM-2423] Abstract StateInternalsTest for the different state internals

commit 4f406f50fee333e871459bb5b5d54c7b1719a030
Author: chamikara@google.com <chamikara@google.com>
Date:   2017-06-08T21:56:24Z

    Adds ability to dynamically replace PTransforms during runtime.
    
    To this end, adds two interfaces, PTransformMatcher and PTransformOverride.
    
    Currently only supports replacements where input and output types are an exact match (we
have to address complexities due to type hints before supporting replacements with different
types).
    
    This will be used by SplittableDoFn where matching ParDo transforms will be dynamically
replaced by SplittableParDo.

commit 75710a6fd9fa9849680349def917c1c9ddcdd56e
Author: Robert Bradshaw <robertwb@google.com>
Date:   2017-06-09T23:44:55Z

    Actually test the fn_api_runner.
    
    The test suite was not being run due to a typo.
    Fix breakage due to changes in the code in the meantime.

commit cf33f5ee4db646ab5a53f30fb6de084859e3a86a
Author: Maria Garcia Herrero <mariagh@google.com>
Date:   2017-06-10T06:34:59Z

    Make unique test names for value-provider arguments

commit d9887e572902beaf97cf956d8062f61cf9396fb7
Author: Thomas Groh <tgroh@google.com>
Date:   2017-06-12T18:07:47Z

    Cleanup Combine Tests with Context
    
    Split out the "shared" bit of all the accumulators, so they show up as
    an explicit component of the final result string. Update timestamped
    creation logic.

commit 17d2466d5ed6dd8ac9b9fe23f9ae5f9b2583bc23
Author: Eugene Kirpichov <kirpichov@google.com>
Date:   2017-06-12T18:51:39Z

    Improves message when transitively serializing PipelineOptions

commit 7816f9844fa871bfc0e05e87c7876e9a5eb1e0e2
Author: Charles Chen <ccy@google.com>
Date:   2017-06-12T21:17:50Z

    Reverse removal of NativeWrite evaluator in Python DirectRunner

commit 705db38a964525330ae1c13e270d2e793eda59d0
Author: Thomas Groh <tgroh@google.com>
Date:   2017-06-12T23:55:59Z

    Check for Deferral on Non-additional inputs
    
    Because Side Inputs are represented within the expanded inputs, the
    check that the transform is a Combine with Side Inputs would never be
    hit. This ensures that we do not consider additional inputs during the
    check to defer evaluation of the node.

commit 36efd7fa4d4329c9cd1685fcd9174d46c5e72f9b
Author: JingsongLi <lzljs3620320@aliyun.com>
Date:   2017-06-13T02:15:33Z

    Add set and map readable test to StateInternalsTest

commit fb864e3204d2c72f46351378072993fbcf974244
Author: Vikas Kedigehalli <vikasrk@google.com>
Date:   2017-06-13T06:11:22Z

    Fix WindowValueCoder for large timestamps

commit ea5eb9777c5e1ab657d06cddd97a9b378b200cd9
Author: Charles Chen <ccy@google.com>
Date:   2017-06-07T23:09:10Z

    Choose GroupAlsoByWindows implementation based on streaming flag

commit 9071a66e333d967c62bd0d8022c6815d714d90a7
Author: Mairbek Khadikov <mairbek@google.com>
Date:   2017-06-07T23:27:01Z

    SpannerIO: Introduced a MutationGroup.
    
    Allows to group together mutation in a logical bundle that is submitted in the same transaction.

commit 9a97ca3ce7e6f14c22d8a369824be3586d925ce0
Author: Kenneth Knowles <klk@google.com>
Date:   2017-06-12T23:31:32Z

    Fix getAdditionalInputs, etc, for DirectRunner stateful ParDo override

commit dbcf23120278e76a943b50737811b415ac5a4e2a
Author: JingsongLi <lzljs3620320@aliyun.com>
Date:   2017-06-13T03:26:38Z

    [BEAM-2423] Port state internals tests to the new base class StateInternalsTest

commit 9c40bb60170c3997a5dbd64639c630cb17351e52
Author: Thomas Groh <tgroh@google.com>
Date:   2017-06-13T19:50:58Z

    Do not produce Unprocessed Inputs if all inputs were Processed
    
    This stops the WatermarkManager "Pending Bundles" from growing without
    bound.

commit 100cfc2471778e5a8f0aaf0db6826cc7c547c24b
Author: Charles Chen <ccy@google.com>
Date:   2017-06-13T22:03:15Z

    Move Runner API protos to portability/api

commit 0a02e1305d1eb588a68cf3884d3642bfc0ee46b8
Author: Charles Chen <ccy@google.com>
Date:   2017-06-14T23:14:50Z

    Migrate DirectRunner evaluators to use Beam state API

commit d42ce81aad5436618fe65ce6d6872fc7588e2d7d
Author: Sourabh Bajaj <sourabhbajaj@google.com>
Date:   2017-06-14T23:35:45Z

    [BEAM-1585] Add beam plugins as pipeline options

commit 51aed8249c206768ee90e14f33bfb551802dc8b1
Author: Luke Cwik <lcwik@google.com>
Date:   2017-06-15T03:13:01Z

    [BEAM-1348] Mark Runner API like types declared within Fn API as deprecated.
    
    Add additional documentation.

commit 36a3fd40bba92570a0bf2e465ab2ddda8aa64a12
Author: Thomas Groh <tgroh@google.com>
Date:   2017-06-15T20:54:47Z

    Use the appropriate context in CombineTest Coder
    
    The Accumulator was improperly decoding the seed value in the outer
    context, as it is in the nested context.

commit 8c987df122eb215a599ad6a8f35b8ecaae2d627f
Author: Marian Dvorsky <mariand@google.com>
Date:   2017-06-16T13:57:13Z

    Fixed handling of use_public_ips. Added test.

commit 751387da184cbf898e4f36b86df290d68183f2af
Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Date:   2017-05-12T13:21:49Z

    [BEAM-975] Improve default connection options, javadoc and style in MongoDbIO

commit 5b03dfe71edccfec177d3c1a46b60744a1dcadf5
Author: Luke Cwik <lcwik@google.com>
Date:   2017-05-31T15:54:16Z

    [BEAM-1348] Model the Fn State Api as per https://s.apache.org/beam-fn-state-api-and-bundle-processing

commit 2badfe2ddb7ca5aead52e9650f36c39ab34b5740
Author: Ismaël Mejía <iemejia@apache.org>
Date:   2017-06-05T21:48:38Z

    [BEAM-2411] Make the write transform of HBaseIO simpler

commit ece8e001f245c101c9355cb52845d13595587183
Author: Eugene Kirpichov <kirpichov@google.com>
Date:   2017-06-08T23:54:12Z

    Tests for reading windowed side input from resumed SDF call

----


> Add a Beam SQL DSL
> ------------------
>
>                 Key: BEAM-301
>                 URL: https://issues.apache.org/jira/browse/BEAM-301
>             Project: Beam
>          Issue Type: New Feature
>          Components: dsl-sql, sdk-ideas
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Xu Mingmin
>
> The SQL DSL helps developers to build a Beam pipeline from SQL statement in String directly.

> In Phase I, it starts to support INSERT/SELECT queries with FILTERs, one example SQL
as below:
> {code}
> INSERT INTO `SUB_USEREVENT` (`SITEID`, `PAGEID`, `PAGENAME`, `EVENTTIMESTAMP`)
> (SELECT STREAM `USEREVENT`.`SITEID`, `USEREVENT`.`PAGEID`, `USEREVENT`.`PAGENAME`, `USEREVENT`.`EVENTTIMESTAMP`
> FROM `USEREVENT` AS `USEREVENT`
> WHERE `USEREVENT`.`SITEID` > 10)
> {code}
> A design doc is available at https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_0a1Bz5BsCROMzCU/edit?usp=sharing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message