Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 44D03200CC6 for ; Tue, 4 Jul 2017 05:18:05 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 43B85160C00; Tue, 4 Jul 2017 03:18:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 89EA0160BF9 for ; Tue, 4 Jul 2017 05:18:04 +0200 (CEST) Received: (qmail 52882 invoked by uid 500); 4 Jul 2017 03:18:03 -0000 Mailing-List: contact dev-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list dev@drill.apache.org Received: (qmail 52862 invoked by uid 99); 4 Jul 2017 03:18:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Jul 2017 03:18:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 7122EC023D for ; Tue, 4 Jul 2017 03:18:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 3c3LevF0uCS0 for ; Tue, 4 Jul 2017 03:18:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 089045FDAD for ; Tue, 4 Jul 2017 03:18:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 4F46FE00A0 for ; Tue, 4 Jul 2017 03:18:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0775F245E9 for ; Tue, 4 Jul 2017 03:18:00 +0000 (UTC) Date: Tue, 4 Jul 2017 03:18:00 +0000 (UTC) From: "Paul Rogers (JIRA)" To: dev@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (DRILL-5657) Extend column accessors to be size-aware MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 04 Jul 2017 03:18:05 -0000 Paul Rogers created DRILL-5657: ---------------------------------- Summary: Extend column accessors to be size-aware Key: DRILL-5657 URL: https://issues.apache.org/jira/browse/DRILL-5657 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.11.0 Reporter: Paul Rogers Assignee: Paul Rogers Fix For: 1.11.0 A recent extension to Drill's set of test tools created a "row set" abstraction to allow us to create, and verify, record batches with very few lines of code. Part of this work involved creating a set of "column accessors" in the vector subsystem. Column readers provide a uniform API to obtain data from columns (vectors), while column writers provide a uniform writing interface. DRILL-5211 discusses a set of changes to limit value vectors to 16 MB in size (to avoid memory fragmentation due to Drill's two memory allocators.) The column accessors have proven to be so useful that they will be the basis for the new, size-aware writers used by Drill's record readers. A step in that direction is to retrofit the column writers to use the size-aware {{setScalar()}} and {{setArray()}} methods introduced in DRILL-5517. Since the test framework row set classes are (at present) the only consumer of the accessors, those classes must also be updated with the changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)