incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: Presenting "Near Real Time Indexing Kafka Messages into Apache Blur" at Lucene Solr Revolution
Date Mon, 24 Nov 2014 14:38:20 GMT
On Fri, Nov 21, 2014 at 10:50 PM, Dibyendu Bhattacharya <
dibyendu.bhattachary@gmail.com> wrote:

> Hi,
>
> I have recently presented in LuceneSolrRevolution one of our Apache Blur
> use case . This is about how we did Indexing Kafka Stream into Blur via
> Spark Streaming.  Here is the slides :
>
>
> http://www.slideshare.net/lucidworks/near-real-time-indexing-kafka-messages-into-apache-blur-presented-by-dibyendu-bhattacharya-pearson-north-america
>

Thanks!


>
> Aaron, as I told you earlier, using Spark Streaming I can achieve very
> high throughput of indexing, but I can see the IndexImporter is falling
> behind to merge at the same pace indexing is happening. Is the latest fix
> to faster index merger can solve this problem ?
>

We are making progress, there are some interesting read patterns during the
merge that are causing some of the issues in performance.


>
> Regards,
> Dibyendu
>
>
>
>
>
> On Mon, Sep 1, 2014 at 10:45 AM, Dibyendu Bhattacharya <
> dibyendu.bhattachary@gmail.com> wrote:
>
>> Dear All,
>>
>> Here is the session details on "Near Real Time Indexing Kafka Messages
>> into Apache Blur"  which we will be presenting at LuceneSolr Revolution.
>>
>> http://lucenerevolution.org/presentations/
>>
>> See under "What's Hot in OSS" category.
>>
>> I have written distributed Kafka Consumer using "Spark Streaming" and
>> used this consumer to index messages into Apache Blur via Blur queuing API.
>> Here is the github project for Kafka-Spark consumer :
>> https://github.com/dibbhatt/kafka-spark-consumer
>>
>> Regards,
>> Dibyendu
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message