incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: FYI, some discussion on Lucene HDFS index read in Solr
Date Sun, 18 Aug 2013 14:41:56 GMT
Hey Patrick,

I commented on that issue.  To date I have not tried the short circuit
reads in Blur.  I have in HBase but they do some interesting things to make
the short circuit reads as effective as possible.  Like they disable the
built-in CRC checks in HDFS (during the read) and do the calculation
themselves because they have a put the CRC checks in-line with the data.
 That way it's only a single IO operation, but it does force you to read
more information than necessary in certain circumstances, but that's the
case for the HDFS implementation as well.

Aaron



On Sun, Aug 18, 2013 at 3:11 AM, Patrick Hunt <phunt@apache.org> wrote:

> Thought I'd pass this on, some performance/correctness discussion re
> reading indexes in hdfs.
> https://issues.apache.org/jira/browse/SOLR-5150
>
> Is blur using short circuit reads at all? Has anyone tried this?
>
> Patrick
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message