From common-issues-return-156586-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Aug 23 03:37:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2D544180662 for ; Thu, 23 Aug 2018 03:37:04 +0200 (CEST) Received: (qmail 56990 invoked by uid 500); 23 Aug 2018 01:37:03 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 56979 invoked by uid 99); 23 Aug 2018 01:37:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Aug 2018 01:37:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 95AB3C00D6 for ; Thu, 23 Aug 2018 01:37:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id rV263iQQKxvj for ; Thu, 23 Aug 2018 01:37:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 3BF005F3DA for ; Thu, 23 Aug 2018 01:37:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 78C33E0101 for ; Thu, 23 Aug 2018 01:37:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2EDB82468F for ; Thu, 23 Aug 2018 01:37:00 +0000 (UTC) Date: Thu, 23 Aug 2018 01:37:00 +0000 (UTC) From: "Sean Mackrory (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-15688) ABFS: InputStream wrapped in FSDataInputStream twice MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589556#comment-16589556 ] Sean Mackrory commented on HADOOP-15688: ---------------------------------------- Thanks, Thomas. +1 on the patch. I didn't think to look in places other than the obvious open() path. I haven't been tracking specific instances of timeout tests, as there have been quite a lot of them when it does happen. I'm also suspicious that my ISP might be partly to blame as I've been having some performance problems not just with Azure networks. > ABFS: InputStream wrapped in FSDataInputStream twice > ---------------------------------------------------- > > Key: HADOOP-15688 > URL: https://issues.apache.org/jira/browse/HADOOP-15688 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Sean Mackrory > Assignee: Sean Mackrory > Priority: Major > Attachments: HADOOP-15688-HADOOP-15407-002.patch, HADOOP-15688.001.patch > > > I can't read Parquet files from ABFS. It has 2 different implementations to read seekable streams, and it'll use the one that uses ByteBuffer reads if it can. It currently decides to use the ByteBuffer read implementation because the FSDataInputStream it gets back wraps another FSDataInputStream, which implements ByteBufferReadable. > That's not the most robust way to check that ByteBufferReads are supported by the ultimately underlying InputStream, but it's unnecessary and probably a mistake to double-wrap the InputStream, so let's not. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org