From commits-return-11744-archive-asf-public=cust-asf.ponee.io@hudi.apache.org Tue Feb 18 21:48:51 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id DB4F518060E for ; Tue, 18 Feb 2020 22:48:50 +0100 (CET) Received: (qmail 80975 invoked by uid 500); 18 Feb 2020 21:48:50 -0000 Mailing-List: contact commits-help@hudi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hudi.apache.org Delivered-To: mailing list commits@hudi.apache.org Received: (qmail 80966 invoked by uid 99); 18 Feb 2020 21:48:50 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Feb 2020 21:48:50 +0000 From: GitBox To: commits@hudi.apache.org Subject: [GitHub] [incubator-hudi] satishkotha opened a new pull request #1341: Add exportToTable option to CLI Message-ID: Date: Tue, 18 Feb 2020 21:48:50 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit satishkotha opened a new pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull request CLI shell is very restrictive and its sometimes hard to filter specific rows. Adding ability to export results of CLI command into temporary table. So CLI users can write HiveQL queries to look for any specific information. (This idea has been brought up by multiple folks on the team, thanks everyone for great suggestions) ## Brief change log - Add 'exportToTableName' cli option for commits command. - This creates a temp table in spark session. Users can write HiveQL queries against the table to filter desired row. - Note, intentionally not using Hoodie format for temp tables (to avoid versioning and any potential bugs in hoodie) Example usage: ->commits show archived --startTs "20200121224545" --endTs "20200224004414" --includeExtraMetadata true --exportToTableName satishkotha_debug Wrote table view: satishkotha_debug >temp_query --sql "select Instant, NumInserts, NumWrites from satishkotha_debug where FileId='ed33bd99-466f-4417-bd92-5d914fa58a8f' and Instant > '20200123211217' order by Instant" +--------------+----------+---------+ |Instant |NumInserts|NumWrites| +--------------+----------+---------+ |20200123221012|0 |2418 | |20200123223835|0 |2418 | |20200123231230|0 |2418 | |20200123233911|0 |2418 | |20200124000848|3 |3 | |20200124004403|7 |10 | |20200124013616|1 |11 | |20200124020556|1 |12 | |20200124061752|1 |13 | ## Verify this pull request Only changes CLI. can be verified by running CLI commands. ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services