Mailing-List: contact issues-help@carbondata.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@carbondata.apache.org
From: aniketadnaik <git@git.apache.org>
To: issues@carbondata.apache.org
Reply-To: issues@carbondata.apache.org
Message-ID: <git-pr-1064-carbondata@git.apache.org>
Subject: [GitHub] carbondata pull request #1064: [CARBONDATA-<1173>] Stream ingestion - write ...
Content-Type: text/plain
Date: Mon, 19 Jun 2017 23:05:51 +0000 (UTC)
archived-at: Mon, 19 Jun 2017 23:05:53 -0000

GitHub user aniketadnaik opened a pull request:

    https://github.com/apache/carbondata/pull/1064

    [CARBONDATA-<1173>] Stream ingestion - write path framework 

    Description:
    This is a over all streaming write path framework implementation. It is mainly targeted for "**streaming_ingest**" branch.  This framework implementation will help other developers to leverage a common code path for developing other pieces of the streaming ingestion feature. 
    - Whether new unit test cases have been added or why no new tests are required?
        - This is a just framework implementation, unit tests will be added with individual functionality commits        - What manual testing you have done?
       - mvn clean verify 
        [INFO] ------------------------------------------------------------------------
        [INFO] Reactor Summary:
        [INFO] 
        [INFO] Apache CarbonData :: Parent ........................ SUCCESS [  1.533 s]
        [INFO] Apache CarbonData :: Common ........................ SUCCESS [  1.335 s]
        [INFO] Apache CarbonData :: Core .......................... SUCCESS [02:34 min]
        [INFO] Apache CarbonData :: Processing .................... SUCCESS [  5.158 s]
        [INFO] Apache CarbonData :: Hadoop ........................ SUCCESS [  5.208 s]
        [INFO] Apache CarbonData :: Spark Common .................. SUCCESS [ 13.989 s]
        [INFO] Apache CarbonData :: Spark ......................... SUCCESS [02:33 min]
        [INFO] Apache CarbonData :: Spark Common Test ............. SUCCESS [05:15 min]
        [INFO] Apache CarbonData :: Assembly ...................... SUCCESS [  1.683 s]
        [INFO] Apache CarbonData :: Spark Examples ................ SUCCESS [  5.123 s]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 10:58 min
        [INFO] Finished at: 2017-06-16T17:24:23-07:00
        [INFO] Final Memory: 85M/1078M
        [INFO] ------------------------------------------------------------------------
    
       - Made sure write path class invocation happens correctly with with spark structured streaming (2.1)
       - Made sure write path execution work flow with structured streaming(2.1) using both socket and file 
         sources
    
    - Any additional information to help reviewers in testing this change.
       - structured streaming now accepts carbondata as a file format in addition to data source 
       - temporary functionality writes out data in text format instead of carbondata format, will be replaced with new functionality additions.
     

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/aniketadnaik/carbondataStreamIngest streamIngest-1173

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1064.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1064
    
----
commit 5403972898dc0d62471296476d6d5603dccbda10
Author: Aniket Adnaik <aniket.adnaik@gmail.com>
Date:   2017-06-15T18:57:43Z

    [CARBONDATA-1173] Streaming Ingest - write path framework implementation

commit 5270952a43b5e373957682dd75edd15048cb5aa3
Author: Aniket Adnaik <aniket.adnaik@gmail.com>
Date:   2017-06-15T19:11:48Z

    [CARBONDATA-1173] Streaming Ingest - write path framework implementation

commit ac3f1bbbee9c33472726afe10bac73a17a607cce
Author: Aniket Adnaik <aniket.adnaik@gmail.com>
Date:   2017-06-16T18:04:19Z

    [CARBONDATA-1173] Streaming Ingest - write path framework implementation

commit 4fc68b6c7a986ceda2c02713884d9293d29961fc
Author: Aniket Adnaik <aniket.adnaik@gmail.com>
Date:   2017-06-19T22:19:21Z

    [CARBONDATA-1173] Streaming Ingest - write path framework implementation

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---