From dev-return-7709-archive-asf-public=cust-asf.ponee.io@hawq.incubator.apache.org Mon Jul 9 10:03:21 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 45D78180676 for ; Mon, 9 Jul 2018 10:03:20 +0200 (CEST) Received: (qmail 23401 invoked by uid 500); 9 Jul 2018 08:03:18 -0000 Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list dev@hawq.incubator.apache.org Received: (qmail 22893 invoked by uid 99); 9 Jul 2018 08:03:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Jul 2018 08:03:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 8B6C6C574D for ; Mon, 9 Jul 2018 08:03:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.588 X-Spam-Level: * X-Spam-Status: No, score=1.588 tagged_above=-999 required=6.31 tests=[HTML_FONT_FACE_BAD=0.289, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 9uKIbmGHOJDA for ; Mon, 9 Jul 2018 08:03:16 +0000 (UTC) Received: from mx0b-00296801.pphosted.com (mx0b-00296801.pphosted.com [148.163.153.148]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 1B85A5F20B for ; Mon, 9 Jul 2018 08:03:16 +0000 (UTC) Received: from pps.filterd (m0114584.ppops.net [127.0.0.1]) by mx0b-00296801.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w6980MGF008402 for ; Mon, 9 Jul 2018 08:03:15 GMT Received: from mail-oi0-f70.google.com (mail-oi0-f70.google.com [209.85.218.70]) by mx0b-00296801.pphosted.com with ESMTP id 2k2n9h8v3g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 09 Jul 2018 08:03:15 +0000 Received: by mail-oi0-f70.google.com with SMTP id x18-v6so22337263oie.7 for ; Mon, 09 Jul 2018 01:03:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=ULlY1JxD6/758r9eSrlJiGVyENP40G60YUxyProZ+GA=; b=Og6vefFpA/jKaFkanjSusstPEv8DmFIRBWS0OOZzvTJEJDpIChKmrb6b3cVpBZX9Bn 88Bw2k3ZjL4JVky6ONowaofE3hh+JDPG9ltSGnlOu/VA3EJFUw11GV/YtcDM86KzzFlv RryvkSEzpWJXDl0UipYc9ZyyMrd8tqRpJEX32YUUGNGvMh8chQ+mx/74U3MPImMvLdUi 5P/1ckl1sFWhMnq8Aj0hOxuOx1/xqsQX6SwdH8Far7yJ75+TMbHIB7gLncANVEz4IN2H CwW5GIV5q+1sbPgxeqpBCFAeqZnvs8PK+E8o50gjK8cMiUG5p9lVrUWcRDYg1qZhRD0g dcWQ== X-Gm-Message-State: APt69E0YS5zEl7KvKgmEYD6fQWK9zW31M5Ufi24JSKcJG83IaTOqsY1r TpKpiLZQEE9aloTW4P5ut2EcJ2Isj0tF7UXCQpHO5FR9jyNrLZXMqV3XH83XO2osRkVWZEivdOv 3x7VKdvJIndZAJIEbQs6GRXVCwArp4g5jMHUE390eZIXcEg0mu0yU3NdeNGFJqWLD4J8= X-Received: by 2002:aca:db0a:: with SMTP id s10-v6mr23276846oig.339.1531123394926; Mon, 09 Jul 2018 01:03:14 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdOwF8kMdGrttMpXqaMyOf1Fe9ACUP9AaVtUIgofiP7iO0mNSjusJA3D+qUZcYlmVG8zg0SEV/GOCQ/Ud7tWFA= X-Received: by 2002:aca:db0a:: with SMTP id s10-v6mr23276833oig.339.1531123394727; Mon, 09 Jul 2018 01:03:14 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a9d:3a03:0:0:0:0:0 with HTTP; Mon, 9 Jul 2018 01:02:54 -0700 (PDT) In-Reply-To: <52b280.25e0.1645db8064e.Coremail.huoruilong@163.com> References: <792d5693.b2f9.1645a04e6c6.Coremail.huor@apache.org> <52b280.25e0.1645db8064e.Coremail.huoruilong@163.com> From: Radar Lei Date: Mon, 9 Jul 2018 16:02:54 +0800 Message-ID: Subject: Re: Re: Revisit plan for Apache HAWQ 2.4.0.0 release with Hive access support in pluggable storage framework To: Ruilong Huo , dev Content-Type: multipart/alternative; boundary="0000000000000ceadf05708c7285" X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-07-09_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807090096 --0000000000000ceadf05708c7285 Content-Type: text/plain; charset="UTF-8" Hi Ruilong, As there are no further comments on this topic, I will remove the feature 'Support accessing Hive table data by Pluggable Storage Framework' from current Apache HAWQ 2.4.0.0-incubating release. The updated features including in current release are listed as: - New Feature: Pluggable Vectorized Execution Engine on HAWQ. - New Feature: Support Runtime Filter for HAWQ local hash join. Since the two list features are all completed, I will start the release process soon. Thanks. Regards, Radar On Tue, Jul 3, 2018 at 9:18 AM, HuoRuilong wrote: > Exactly, Ed. The plan is to split the work and make them available in > subsequent releases. For now, there are five sub tasks tracking them > (HAWQ-1628 ~ HAWQ-1632). Though we might separate them even more as we go > deeper with it. > > In fact, the work for the initial task has been started. We might discuss > which releases these features will fit into as we drafting the plan for > future releases. > > > Best regards, > Ruilong > > > At 2018-07-02 22:27:52, "Ed Espino" wrote: > >Thanks for bring this up Ruilong. Given your input, It is reasonable to > >remove the pluggable storage framework features from 2.4.0.0. To keep the > >momentum going, going forward, can we separate the work and incrementally > >commit prerequisite work over several releases? > > > >-=e > > > >On Mon, Jul 2, 2018 at 2:34 AM Radar Lei wrote: > > > >> Hi Ruilong, > >> > >> Thanks for the update, for me, I'm fine with removing this feature from > >> 2.4.0.0 release since the scope is bigger than expected. > >> > >> And we still have Vectorized Execution and Runtime Filter features in > this > >> release, they are good enough to make a new release. > >> > >> Regards, > >> Radar > >> > >> On Mon, Jul 2, 2018 at 4:03 PM, Ruilong Huo wrote: > >> > >> > Hi Radar and All, > >> > > >> > > >> > Had a closer look at the plan for Apache HAWQ 2.4.0.0 release, it > turns > >> > out that the support of Hive data access in pluggable storage > framework > >> has > >> > larger content than expected. In fact, to make Hive protocol > available, > >> the > >> > prerequisite need to be done at first, which including HDFS protocol, > >> ORC, > >> > TEXT/CSV format, etc. The corresponding task have been added in > umbrella > >> > feature HAWQ-786 as below: > >> > > >> > > >> > HAWQ-1628: Support HDFS protocol using pluggable storage framework > >> > HAWQ-1629: Support ORC format using pluggable storage framework > >> > HAWQ-1630: Support TEXT/CSV format using pluggable storage framework > >> > HAWQ-1631: Support Hive protocol using pluggable storage framework > >> > HAWQ-1632: Wrap up pluggable storage framework and HDFS/Hive > >> ORC/TEXT/CSV > >> > support > >> > > >> > > >> > Given that, I think it will be better if we can release 2.4.0.0 > without > >> > this feature for now and accommodate it to subsequent releases. Any > >> comment? > >> > > >> > > >> > Best regards, > >> > Ruilong Huo > >> > > >> > > >> > > >> > > >> > > >> > > >> > --0000000000000ceadf05708c7285--