carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ManoharVanam <...@git.apache.org>
Subject [GitHub] carbondata pull request #972: [CARBONDATA-1065] Added set command in carbon ...
Date Tue, 27 Jun 2017 09:15:33 GMT
GitHub user ManoharVanam reopened a pull request:

    https://github.com/apache/carbondata/pull/972

    [CARBONDATA-1065] Added set command in carbon to update properties dynamically

    **Requirement** : Currently there is no way to set the carbon properties dynamically,
always need to update the carbon.properties file and restart the driver.
    So it will be easy for users to give option to add/update/display/reset the properties
dynamically using set and reset commands.
    
    **Solution :** 
    The below sql command updates the carbon properties in driver and as well as in executor
    
    **Add/update/display session property :** 
    set parameter = value --> add/update carbon session property in the corresponding session
    Example : set enable.unsafe.sort = true;
    
    set parameter --> displays given parameter value in the corresponding session
    Example : set enable.unsafe.sort;
    
    **Reset all session properties :** 
    Reset --> clears all session parameters in the corresponding session 
    Example : reset;
    
    **SET:** 
    SET command displays all session supported parameters (carbon + spark) in the corresponding
session 
    Example : SET;
    
    **SET -V:**
    
    set -v command displays all session parameter list along with default value and usage
doc
    Example : set -v;
    **Supported parameters for dynamic set.**
    carbon.options.bad.records.logger.enable
    carbon.options.bad.records.action
    carbon.options.is.empty.data.bad.record
    carbon.options.dateformat
    carbon.options.sort.scope
    carbon.options.batch.sort.size.inmb
    carbon.options.single.pass
    carbon.options.bad.record.path
    carbon.options.global.sort.partitions
    enable.unsafe.sort
    carbon.custom.block.distribution
    Example:
     To enable bad records logger using set command 
    _set carbon.options.bad.records.logger.enable=true_
    
    **Parameter precedence ** 
    1. Load Options
    2. Set Command options
    3. Global Property
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ManoharVanam/incubator-carbondata dynamic_new

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/972.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #972
    
----
commit 22554e40ff50e5eb14633b1fe1d8319d278cbecd
Author: ravipesala <ravi.pesala@gmail.com>
Date:   2017-05-18T09:34:17Z

    Adding session based properties
    
    Added set command in carbon to update properties dynamically

commit 11f4dff6a67e50eb2778116a24fc191ebe9ce0c0
Author: Manohar <manohar.crazy09@gmail.com>
Date:   2017-06-12T12:36:25Z

    Added set/reset commands in carbon to update/reset properties dynamically

commit 1798a6826a2516676a75e150335ef4659394f2d7
Author: mohammadshahidkhan <mohdshahidkhan1987@gmail.com>
Date:   2017-06-12T13:03:22Z

    1. Refactored the bad record code, by default the bad record path will be empty, if bad
record logger is
       enabled or action is redirect and bad record path is not configured then data-load
will fail.
    2. Support dynamic set command for some of load options
    3. fixed test cases
    4. Added validation for the supported property in the dynamic set command
    5. Change table delete behavior // now the bad record would not be deleted ion table drop
    6. added test case for bad record path in load option
    7. fixed failing test cases
    8. Added "carbon.options." in load options parameters

commit 3736f71c3b9a6d99b60779befe82ef7caee3c5fc
Author: Manohar <manohar.crazy09@gmail.com>
Date:   2017-06-23T10:50:15Z

    Created CarbonSessionInfo wrapper for storing all session details

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message