hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] satishkotha opened a new pull request #1341: Add exportToTable option to CLI
Date Tue, 18 Feb 2020 21:48:50 GMT
satishkotha opened a new pull request #1341: Add exportToTable option to CLI
URL: https://github.com/apache/incubator-hudi/pull/1341
 
 
   ## *Tips*
   - *Thank you very much for contributing to Apache Hudi.*
   - *Please review https://hudi.apache.org/contributing.html before opening a pull request.*
   
   ## What is the purpose of the pull request
   CLI shell is very restrictive and its sometimes hard to filter specific rows. Adding ability
to export results of CLI command into temporary table. So CLI users can write HiveQL queries
to look for any specific information. 
   (This idea has been brought up by multiple folks on the team, thanks everyone for great
suggestions)
   
   ## Brief change log
   - Add 'exportToTableName' cli option for commits command.
   - This creates a temp table in spark session. Users can write HiveQL queries against the
table to filter desired row.
   - Note, intentionally not using Hoodie format for temp tables (to avoid versioning and
any potential bugs in hoodie)
   
   Example usage:
   ->commits show archived --startTs "20200121224545" --endTs "20200224004414"   --includeExtraMetadata
true --exportToTableName satishkotha_debug
   Wrote table view: satishkotha_debug
   
   >temp_query --sql "select Instant, NumInserts, NumWrites from satishkotha_debug where
FileId='ed33bd99-466f-4417-bd92-5d914fa58a8f' and Instant > '20200123211217' order by Instant"
   +--------------+----------+---------+
   |Instant       |NumInserts|NumWrites|
   +--------------+----------+---------+
   |20200123221012|0         |2418     |
   |20200123223835|0         |2418     |
   |20200123231230|0         |2418     |
   |20200123233911|0         |2418     |
   |20200124000848|3         |3        |
   |20200124004403|7         |10       |
   |20200124013616|1         |11       |
   |20200124020556|1         |12       |
   |20200124061752|1         |13       |
   
   
   ## Verify this pull request
   
   Only changes CLI. can be verified by running CLI commands.
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella
JIRA.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message