Return-Path: <issues-return-262410-archive-asf-public=cust-asf.ponee.io@hbase.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id A98D6200BA5
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Tue,  4 Oct 2016 22:33:22 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id A82D0160AC7; Tue,  4 Oct 2016 20:33:22 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id EE456160ADC
	for <archive-asf-public@cust-asf.ponee.io>; Tue,  4 Oct 2016 22:33:21 +0200 (CEST)
Received: (qmail 47889 invoked by uid 500); 4 Oct 2016 20:33:20 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 47589 invoked by uid 99); 4 Oct 2016 20:33:20 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Oct 2016 20:33:20 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 9D2F52C2A68
	for <issues@hbase.apache.org>; Tue,  4 Oct 2016 20:33:20 +0000 (UTC)
Date: Tue, 4 Oct 2016 20:33:20 +0000 (UTC)
From: "Enis Soztutar (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13009678.1475612512000.735182.1475613200641@Atlassian.JIRA>
In-Reply-To: <JIRA.13009678.1475612512000@Atlassian.JIRA>
References: <JIRA.13009678.1475612512000@Atlassian.JIRA> <JIRA.13009678.1475612512493@arcas>
Subject: [jira] [Updated] (HBASE-16766) Do not rely on
 InputStream.available()
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Tue, 04 Oct 2016 20:33:22 -0000


     [ https://issues.apache.org/jira/browse/HBASE-16766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Enis Soztutar updated HBASE-16766:
----------------------------------
    Status: Patch Available  (was: Open)

> Do not rely on InputStream.available() 
> ---------------------------------------
>
>                 Key: HBASE-16766
>                 URL: https://issues.apache.org/jira/browse/HBASE-16766
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: hbase-16766_v1.patch
>
>
> ProtobufLogReader relies on InputStream.available() to figure out whether we have exhausted the file. However InputStream.available() javadoc states: 
> {code}
>      * <p> Note that while some implementations of {@code InputStream} will return
>      * the total number of bytes in the stream, many will not.  It is
>      * never correct to use the return value of this method to allocate
>      * a buffer intended to hold all data in this stream.
> {code}
> HDFS and many other Hadoop FS's, and things like ByteBufferInputStream, etc all return remaining bytes, so the code works on top of HDFS. However, on other file systems, it may or may not be true that IS.available() returns the remaining bytes. In one specific case, the ADLS wrapper FS used implement {{available()}} call with the correct semantics, which ended up causing data loss in the WAL recovery. We have since fixed ADLS to implement the HDFS semantics, but we should fix HBase itself so that we do not rely on available() call. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)