Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 16316200C46 for ; Wed, 15 Mar 2017 00:34:47 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 14C9A160B7E; Tue, 14 Mar 2017 23:34:47 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 69041160B8C for ; Wed, 15 Mar 2017 00:34:46 +0100 (CET) Received: (qmail 83131 invoked by uid 500); 14 Mar 2017 23:34:45 -0000 Mailing-List: contact dev-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list dev@drill.apache.org Received: (qmail 83004 invoked by uid 99); 14 Mar 2017 23:34:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Mar 2017 23:34:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 6C3D9C11A8 for ; Tue, 14 Mar 2017 23:34:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.651 X-Spam-Level: X-Spam-Status: No, score=0.651 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id zl_lj8BpZ7zF for ; Tue, 14 Mar 2017 23:34:43 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 9FC4B5FE25 for ; Tue, 14 Mar 2017 23:34:43 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6B953E0ADD for ; Tue, 14 Mar 2017 23:34:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id B80D2243BA for ; Tue, 14 Mar 2017 23:34:41 +0000 (UTC) Date: Tue, 14 Mar 2017 23:34:41 +0000 (UTC) From: "Paul Rogers (JIRA)" To: dev@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (DRILL-5356) Refactor Parquet Record Reader MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 14 Mar 2017 23:34:47 -0000 Paul Rogers created DRILL-5356: ---------------------------------- Summary: Refactor Parquet Record Reader Key: DRILL-5356 URL: https://issues.apache.org/jira/browse/DRILL-5356 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.10.0, 1.11.0 Reporter: Paul Rogers Assignee: Paul Rogers Priority: Minor Fix For: 1.11.0 The Parquet record reader class is a key part of Drill that has evolved over time to become somewhat hard to follow. A number of us are working on Parquet-related tasks and find we have to spend an uncomfortable amount of time trying to understand the code. In particular, this writer needs to figure out how to convince the reader to provide higher-density record batches. Rather than continue to decypher the complex code multiple times, this ticket requests to refactor the code to make it functionally identical, but structurally cleaner. The result will be faster time to value when working with this code. This is a lower-priority change and will be coordinated with others working on this code base. -- This message was sent by Atlassian JIRA (v6.3.15#6346)