Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 545DF933D for ; Wed, 19 Oct 2011 13:20:58 +0000 (UTC) Received: (qmail 62800 invoked by uid 500); 19 Oct 2011 13:20:56 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 62768 invoked by uid 500); 19 Oct 2011 13:20:56 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 62760 invoked by uid 99); 19 Oct 2011 13:20:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Oct 2011 13:20:56 +0000 X-ASF-Spam-Status: No, hits=1.6 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of boneill42@gmail.com designates 209.85.216.44 as permitted sender) Received: from [209.85.216.44] (HELO mail-qw0-f44.google.com) (209.85.216.44) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Oct 2011 13:20:49 +0000 Received: by qadb12 with SMTP id b12so1592288qad.31 for ; Wed, 19 Oct 2011 06:20:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=user-agent:date:subject:from:to:message-id:thread-topic:in-reply-to :mime-version:content-type; bh=doSEPowsW/olQ1ROwwjPloFaNBXs4MhCVdVLdqytcA0=; b=Qz3tJqaaO9Bf0WXXHz2txnI4g5u1xZzxVZehL84hxSfeTSZrySsuNJc3zKPR5BLv5b 7UaaRiuPU85c2FDOrcR9nOsLmV+i2tCR3UE8zVaLlbszcHXtSMUT6iO6B3ofc7khJvX9 9nq5AvlG+eY62bwv4a4TAxzyxKvDHoEEvU08o= Received: by 10.224.211.194 with SMTP id gp2mr5446398qab.99.1319030428675; Wed, 19 Oct 2011 06:20:28 -0700 (PDT) Received: from [10.60.70.220] ([67.132.206.254]) by mx.google.com with ESMTPS id z17sm6634043qap.19.2011.10.19.06.20.25 (version=SSLv3 cipher=OTHER); Wed, 19 Oct 2011 06:20:27 -0700 (PDT) User-Agent: Microsoft-MacOutlook/14.13.0.110805 Date: Wed, 19 Oct 2011 09:20:21 -0400 Subject: Re: Using elasticsearch on cassandra nodes From: Brian O'Neill To: Message-ID: Thread-Topic: Using elasticsearch on cassandra nodes In-Reply-To: Mime-version: 1.0 Content-type: multipart/alternative; boundary="B_3401860825_148008" X-Virus-Checked: Checked by ClamAV on apache.org > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3401860825_148008 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Anthony, We're in exactly the same boat. We are waiting on DataStax Enterprise to see if it can ease the pain of SOLR schemas. In the meantime, I just submitted a native REST layer for Cassandra. https://issues.apache.org/jira/browse/CASSANDRA-3380 (Hopefully, it will get integrated soon. Vote it up ;) With a simple REST layer, I'm making the case that we can use Cassandra just like CouchDB. (so we don't have to deploy both) Extending that assertion, I think I could enhance the REST layer to provide a stream of changes just like CouchDB does. Elastic Search could tap into that stream as a river. Just like this=8A http://www.elasticsearch.org/guide/reference/river/couchdb.html That combination would be pretty powerful. If we can't get that setup, we may fallback to an AOPish strategy as well. Definitely let me know where you end up. I'll share our findings as well. cheers, -brian ----=20 Brian O'Neill Lead Architect, Software Development Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406 p: 215.588.6024 blog: http://weblogs.java.net/blog/boneill42/ blog: http://brianoneill.blogspot.com/ From: Anthony Ikeda Reply-To: Date: Tue, 18 Oct 2011 14:18:17 -0700 To: Subject: Re: Using elasticsearch on cassandra nodes At the moment we are only prototyping so we haven't bridged the two at all. We had planned on creating a write-through operation that allowed us to filter the calls (AOP perhaps?) to manage the indexing as we stored it in Cassandra. We are still trying to work out if we go the elastic search route or not as DataStax will be releasing DataStax Enterprise 2.0 early next year with Sol= r built in and as you said the index schemas seem to be difficult to deal wit= h - I really don't want to have to configure Solr, the no schema approach sounds much faster to get up and running. Anthony On Tue, Oct 18, 2011 at 6:14 AM, Brian O'Neill wrote: > Anthony, >=20 > We've been looking at elastic search as well. Presently we have SOLR in > place, but it is cumbersome dealing with SOLR schemas when indexing > information out of Cassandra (since you can't anticipate all the columns = ahead > of time). =20 >=20 > What are you using as your bridge between Cassandra and ES? Are you > developing a Cassandra river? >=20 > -brian >=20 >=20 >=20 >=20 > On Mon, Oct 17, 2011 at 5:29 PM, Anthony Ikeda > wrote: >> I've already posted to the elasticsearch groups and thought it prudent t= o >> also ask here. >>=20 >> We are looking at using elastic search to index our data that we current= ly >> store to Cassandra. I was wondering if there are any concerns running el= astic >> search on the same nodes that we use for Cassandra? We have a ring of 6 = nodes >> (2 DCs each with 3 nodes) I was thinking of installing elastic search on= 2 >> nodes in each datacentre - maybe all three. The only reason I'd use the = same >> infrastructure would be because we have the distributed visibility alrea= dy in >> place. >>=20 >> Has anyone else taken this approach? Pros? Cons? >>=20 >> Anthony >>=20 >=20 >=20 >=20 > --=20 > Brian ONeill > Lead Architect, Health Market Science (http://healthmarketscience.com) > mobile:215.588.6024 > blog: http://weblogs.java.net/blog/boneill42/ > blog: http://brianoneill.blogspot.com/ >=20 --B_3401860825_148008 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable
Anthony,

We're in exactly the same boat.  We are waiting on Data= Stax Enterprise to see if it can ease the pain of SOLR schemas.
In the meantime, I just submitted a native REST layer for Cassa= ndra.
(Hopef= ully, it will get integrated soon. Vote it up ;)

Wi= th a  simple REST layer, I'm making the case that we can use Cassandra = just like CouchDB. (so we don't have to deploy both)
Extending tha= t assertion, I think I could enhance the REST layer to provide a stream of c= hanges just like CouchDB does.  Elastic Search could tap into that stre= am as a river.  Just like this…

That com= bination would be pretty powerful.  If we can't get that setup, we may = fallback to an AOPish strategy as well.

Definitely = let me know where you end up.   I'll share our findings as well.
<= div>
cheers,
-brian

---- 
Brian O'Neill
= Lead Architect, Software Development
Health Market S= cience | 2700 Horizon Drive | Kin= g of Prussia, PA 19406
p: 215.588.6024



From: Anthony Ikeda <anthony.ikeda.dev@gmail.com>
= Reply-To: <user@cassan= dra.apache.org>
Date: Tue, = 18 Oct 2011 14:18:17 -0700
To: <= ;user@cassandra.apache.org>= ;
Subject: Re: Using elasticsearch= on cassandra nodes

At the moment we are only protot= yping so we haven't bridged the two at all. We had planned on creating a wri= te-through operation that allowed us to filter the calls (AOP perhaps?) to m= anage the indexing as we stored it in Cassandra.

We are s= till trying to work out if we go the elastic search route or not as DataStax= will be releasing DataStax Enterprise 2.0 early next year with Solr built i= n and as you said the index schemas seem to be difficult to deal with - I re= ally don't want to have to configure Solr, the no schema approach sounds muc= h faster to get up and running.

Anthony
<= br>
On Tue, Oct 18, 2011 at 6:14 AM, Brian O'Nei= ll <bone@alumni.br= own.edu> wrote:
Anthony,

= We've been looking at elastic search as well.  Presently we have SOLR i= n place, but it is cumbersome dealing with SOLR schemas when indexing inform= ation out of Cassandra (since you can't anticipate all the columns ahead of = time). 

What are you using as your bridge between Cassandra and= ES?  Are you developing a Cassandra river?

-brian




On Mon, Oct 17, 2= 011 at 5:29 PM, Anthony Ikeda <anthony.ikeda.dev@gmail.com> wrote:
I've already posted to the elasticsear= ch groups and thought it prudent to also ask here.

We are looking at using elastic search to index our data= that we currently store to Cassandra. I was wondering if there are any=  concerns running elastic search on the same nodes that we use for = ;Cassandra? We have a ring of 6 nodes (2 DCs each with 3 nodes) I was thinki= ng of installing elastic search on 2 nodes in each datacentre - maybe all th= ree. The only reason I'd use the same infrastructure would be because we hav= e the distributed visibility already in place.

Has anyone= else taken this approach? Pros? Cons?

Anthony




--
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketsc= ience.com)
mobile:215.= 588.6024
blog: htt= p://weblogs.java.net/blog/boneill42/
blog: http://brianoneill.blogspot.com/

 --B_3401860825_148008--