hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: A question of HFile.
Date Mon, 05 Oct 2009 17:19:16 GMT
On Mon, Oct 5, 2009 at 7:49 AM, Schubert Zhang <zsongbo@gmail.com> wrote:

> Hi stack,
>
> In the HFile.java, I found following comments by you.
>
> "The current implementation does not offer true multi-threading for
> reading.
> The implementation uses FSDataInputStream seek()+read(), which is shown to
> be much faster than positioned-read call in single thread mode. However, it
> also means that if multiple threads attempt to access the same HFile (using
> multiple scanners) simultaneously, the actual I/O is carried out
> sequentially even if they access different DFS blocks (Reexamine! pread
> seems to be 10% faster than seek+read in my testing -- stack). "
>
> I want multiple threads or processes to access a same HFile file
> simultaneously, each thread or process create a new HFile.Reader and
> Scanner. I think it is no-problem. Do you think it ok?
>

Its no problem.  Its thread-safe (See down in BoundedRangeFileInputStream
where we synchronize on the FSDIS).  Its a (confusing) comment on my back
and forth on dfs pread vs. seek+read (Interesting.  I thought we'd switched
hfile to do pread always but it doesn't look so).

St.Ack

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message