hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] satishkotha opened a new pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath
Date Tue, 03 Mar 2020 04:55:25 GMT
satishkotha opened a new pull request #1368: [HUDI-650] Modify handleUpdate path to validate
partitionPath
URL: https://github.com/apache/incubator-hudi/pull/1368
 
 
   ## What is the purpose of the pull request
   
   HoodieTable handleUpdate takes in fileId and list of records. It does not validate all
records belong to same partitionPath. This is error prone - there is already at least one
test that is passing in records that belong to several partitions to this method. Fix to add
partitionPath and also validate all records belong to same partition path. 
    I'm not entirely sure this change is needed though. I think its cleaner to include partition
path everywhere. It is a good safeguard, but maybe its unlikely this can happen in production.
Sending it out since i already spent some time. I can discard if others think this is unnecessary.
   
   ## Brief change log
   - Do not assume first record partition can be used for all remaining records
   - Track partition path in Bucket and in Partitioner
   - Fail updates if they somehow end up in wrong partition
   - Fix broken test to validate requests fail
   - There is one other test commented out (looks like by mistake). Bring it back
   
   ## Verify this pull request
   This pull request is already covered by existing tests, such as *(please describe tests)*.
   TestCopyOnWriteTable (especially testInsertUpsertWithHoodieAvroPayload)
   TestMergeOnReadTable
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella
JIRA.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message