Return-Path: X-Original-To: apmail-kudu-commits-archive@minotaur.apache.org Delivered-To: apmail-kudu-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8CE76197B9 for ; Mon, 18 Apr 2016 18:12:07 +0000 (UTC) Received: (qmail 71754 invoked by uid 500); 18 Apr 2016 18:12:07 -0000 Delivered-To: apmail-kudu-commits-archive@kudu.apache.org Received: (qmail 71736 invoked by uid 500); 18 Apr 2016 18:12:07 -0000 Mailing-List: contact commits-help@kudu.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kudu.incubator.apache.org Delivered-To: mailing list commits@kudu.incubator.apache.org Received: (qmail 71712 invoked by uid 99); 18 Apr 2016 18:12:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Apr 2016 18:12:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 9892BC0D25 for ; Mon, 18 Apr 2016 18:12:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.221 X-Spam-Level: X-Spam-Status: No, score=-3.221 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id bPl5I4OmI_K8 for ; Mon, 18 Apr 2016 18:12:04 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id F41235FB39 for ; Mon, 18 Apr 2016 18:12:03 +0000 (UTC) Received: (qmail 71620 invoked by uid 99); 18 Apr 2016 18:12:03 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Apr 2016 18:12:03 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 52B53DFE2E; Mon, 18 Apr 2016 18:12:03 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: jdcryans@apache.org To: commits@kudu.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-kudu git commit: Add 4/18 weekly update Date: Mon, 18 Apr 2016 18:12:03 +0000 (UTC) Repository: incubator-kudu Updated Branches: refs/heads/gh-pages 86ae98bc4 -> 65ff8b29a Add 4/18 weekly update Change-Id: I31a5a5274ec30014566cd35405c087de5441e14e Reviewed-on: http://gerrit.cloudera.org:8080/2805 Reviewed-by: Jean-Daniel Cryans Tested-by: Todd Lipcon Project: http://git-wip-us.apache.org/repos/asf/incubator-kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-kudu/commit/65ff8b29 Tree: http://git-wip-us.apache.org/repos/asf/incubator-kudu/tree/65ff8b29 Diff: http://git-wip-us.apache.org/repos/asf/incubator-kudu/diff/65ff8b29 Branch: refs/heads/gh-pages Commit: 65ff8b29ad633ab8a17590eda8413f1e0fa98415 Parents: 86ae98b Author: Todd Lipcon Authored: Mon Apr 18 09:26:47 2016 -0700 Committer: Todd Lipcon Committed: Mon Apr 18 16:44:06 2016 +0000 ---------------------------------------------------------------------- _posts/2016-04-18-weekly-update.md | 80 +++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-kudu/blob/65ff8b29/_posts/2016-04-18-weekly-update.md ---------------------------------------------------------------------- diff --git a/_posts/2016-04-18-weekly-update.md b/_posts/2016-04-18-weekly-update.md new file mode 100644 index 0000000..ac524c8 --- /dev/null +++ b/_posts/2016-04-18-weekly-update.md @@ -0,0 +1,80 @@ +--- +layout: post +title: Apache Kudu (incubating) Weekly Update April 18, 2016 +author: Todd Lipcon +--- +Welcome to the fifth edition of the Kudu Weekly Update. This weekly blog post +covers ongoing development and news in the Apache Kudu (incubating) project. + + + +If you find this post useful, please let us know by emailing the +[kudu-user mailing list](mailto:user@kudu.incubator.apache.org) or +tweeting at [@ApacheKudu](https://twitter.com/ApacheKudu). Similarly, if you're +aware of some Kudu news we missed, let us know so we can cover it in +a future post. + +## Project news + +* Cloudera announced that it has posted [binary packages](http://markmail.org/thread/tghwcux5k4qvcsep) + for the recent 0.8.0 release. These are not official packages from + the Apache Kudu (incubating) project, but users who prefer not to + build from source may find them convenient. + +* Jean-Daniel Cryans has volunteered to continue to act as release manager for + the 0.x release line, and has start a [discussion](http://mail-archives.apache.org/mod_mbox/incubator-kudu-dev/201604.mbox/%3CCAGpTDNcfTOcp%2Beb39h5j%3DoxttZNhOBZ7v%2B%2B6hxRtWCh3t_psbQ%40mail.gmail.com%3E) + detailing what features and improvements he expects will be ready + for an 0.9 release in June. + +## Development discussions and code in progress + +* Chris George posted a [work in progress patch](http://gerrit.cloudera.org:8080/#/c/2754/) + for a native Kudu RDD implementation for Spark. Kudu already ships an RDD + based on the generic HadoopRDD and Kudu's MapReduce integration, but Chris's + new version paves the way for new features like pushing down predicates. + + +* Todd Lipcon has been working on [KUDU-1410](https://issues.apache.org/jira/browse/KUDU-1410), + a small project which makes it easier to diagnose performance issues on a Kudu + cluster. + + The first feature proposed by this JIRA is the idea of collecting + "exemplar" traces: for each type of RPC (e.g. _Write_, _Scan_, etc.) + the RPC system will collect a few _exemplar_ RPCs in different + latency buckets and retain their traces. This makes it easier for + an operator to see what might have caused a slow response from a + server even after the request has been finished for some time. + + The second new feature is the collection of per-RPC-request metrics + such as lock acquisition time, time spent waiting on disk, and other + metrics specific to each type of RPC. In combination with the + exemplar trace feature above, this should make it easy to root-cause + whether a request is slow due to underlying hardware issues, + Kudu-specific issues, or a particular workload characteristic. + + Todd posted a work-in-progress implementation of these features on gerrit + in a five-part patch series: + [(1)](http://gerrit.cloudera.org:8080/#/c/2794/) + [(2)](http://gerrit.cloudera.org:8080/#/c/2795/) + [(3)](http://gerrit.cloudera.org:8080/#/c/2796/) + [(4)](http://gerrit.cloudera.org:8080/#/c/2797/) + [(5)](http://gerrit.cloudera.org:8080/#/c/2798/) + +* Dan Burkert continued working on the [Java implementation of the Scan Token API](http://gerrit.cloudera.org:8080/#/c/2592/) + described in previous weekly updates, with reviews this week from Jean-Daniel + Cryans and Adar Dembo. He also posted a patch for the [C++ implementation](http://gerrit.cloudera.org:8080/#/c/2757/) + which has seen some review action as well. + +* Dan also posted a [design document for non-covering range partitioning](http://gerrit.cloudera.org:8080/#/c/2772/). + This new feature will allow Kudu operators to add or drop tablets to + an existing range-partitioned table. This is very important for time + series use cases where new partitions may need to be added daily, + and old partitions potentially dropped in order to achieve a + "sliding window" table. Read the design document for more details on + use cases and the expected semantics. + +## On the Kudu blog + +* Pat Patterson wrote a post about [Ingesting JSON Data into Apache Kudu with StreamSets + Data Collector](http://getkudu.io/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html). +