hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gcaragea <...@git.apache.org>
Subject [GitHub] incubator-hawq pull request: HAWQ-97. Optimizing metadata versioni...
Date Mon, 09 Nov 2015 19:48:44 GMT
GitHub user gcaragea opened a pull request:

    https://github.com/apache/incubator-hawq/pull/88

    HAWQ-97. Optimizing metadata versioning event generation

    A DDL command usually generates more than one Nuke Versioning event during its execution.
All these nuke events are then pushed to Shared Versioning Queue (SVQ) at commit, and then
all of them get propagated to other backends. 
    
    This is extra work and it can be optimized. We should push only one Nuke VE instead of
several.
    
    Also, when moving messages from the Current Command Invalidation Queue (CVQ) to the Prior
Command Invalidation Queue (XVQ), pre-process the event queue to eliminate any events that
will have no effect. If a versioning event is followed by a nuke event, it will have no effect.
     Moreover, when a nuke event is moved to XVQ, everything prior in XVQ will  also have
no effect.
     
    Solution is  to look for Nuke events in CVQ. Find the last one (in the order of creation),
then move all messages from then on to XVQ, overwriting XVQ.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gcaragea/incubator-hawq gcaragea-HAWQ-97

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-hawq/pull/88.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #88
    
----
commit a5baf5b7c9608b6c9e4248b6d158230c1663226d
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-10-28T19:20:10Z

    [HAWQ-97] Optimizing metadata versioning event generation by eliminating a lot of redundant
Nuke messages

commit 8aa660386fe003d1d330acb10b6ee66c2fe3d698
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-10-29T02:34:19Z

    [HAWQ-97] Adding unit tests for mdver_dep_translator changes

commit f156a273f62f201524ed0ed81bd356bf0922e88c
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-10-30T01:09:19Z

    [HAWQ-97] Adding unit tests for MdVer_IsRedundantNukeEvent

commit b448c700e99effd208d5e62d2368b301fef36199
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-10-30T19:11:28Z

    [HAWQ-97] Fixed issue with pre-processing invalidation list for MD Ver

commit 4e1dd59d29ceb337aeca8aab91e4ec625db39e59
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-10-30T19:12:03Z

    [HAWQ-97] Adding unit tests for MdVer_PreProcessInvalidMsgs

commit 09b65f047370ca9e4cd0a905dbd22359e16a05f9
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-10-31T00:31:49Z

    [HAWQ-97] Finished unit tests for MdVer_PreProcessInvalidMsgs

commit a0056992dc324f7ebfcb8791b6643ab07291f77f
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-11-09T18:16:33Z

    [HAWQ-97] Added function commit to test

commit c95f522e309280ed038ea90edfb087c308317731
Author: George Caragea <gcaragea@pivotal.io>
Date:   2015-11-09T18:29:18Z

    [HAWQ-97] Freeing up discarded chunks from versioning event lists when compacting them

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message