lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: integrating Accumulo with solr
Date Thu, 24 Jul 2014 14:57:17 GMT
If you are not a "true hard-core gunslinger" who is willing to dive in and 
integrate the code yourself, instead you should give serious consideration 
to a product such as DataStax Enterprise that fully integrates and packages 
a NoSQL database (Cassandra) and Solr for search. The security aspects are 
still a work in progress, but certainly headed in the right direction. And 
it has Hadoop and Spark integration as well.

See:
http://www.datastax.com/what-we-offer/products-services/datastax-enterprise

-- Jack Krupansky

-----Original Message----- 
From: Ali Nazemian
Sent: Thursday, July 24, 2014 10:30 AM
To: solr-user@lucene.apache.org
Subject: Re: integrating Accumulo with solr

Thank you very much. Nice Idea but how can Solr and Accumulo can be
synchronized in this way?
I know that Solr can be integrated with HDFS and also Accumulo works on the
top of HDFS. So can I use HDFS as integration point? I mean set Solr to use
HDFS as a source of documents as well as the destination of documents.
Regards.


On Thu, Jul 24, 2014 at 4:33 PM, Joe Gresock <jgresock@gmail.com> wrote:

> Ali,
>
> Sounds like a good choice.  It's pretty standard to store the primary
> storage id as a field in Solr so that you can search the full text in Solr
> and then retrieve the full document elsewhere.
>
> I would recommend creating a document structure in Solr with whatever
> fields you want indexed (most likely as text_en, etc.), and then store a
> "string" field named "content_id", which would be the Accumulo row id that
> you look up with a scan.
>
> One caveat -- Accumulo will be protected at the cell level, but if you 
> need
> your Solr search results to be protected by complex authorization strings
> similar to Accumulo, you will need to write your own QParserPlugin and use
> post filtering:
> http://java.dzone.com/articles/custom-security-filtering-solr
>
> The code you see in that article is written for an earlier version of 
> Solr,
> but it's not too difficult to adjust it for the latest (we've done so in
> our project).  Once you've implemented this, you would store an
> "authorizations" string field in each Solr document, and pass in the
> authorizations that the user has access to in the fq parameter of every
> query.  It's also not too bad to write something that parses the Accumulo
> authorizations string (like A&B&(C|D|E|F)) and interpret it accordingly in
> the QParserPlugin.
>
> This will give you true row level security in Solr and Accumulo, and it
> performs quite well in Solr.
>
> Let me know if you have any other questions.
>
> Joe
>
>
> On Thu, Jul 24, 2014 at 4:07 AM, Ali Nazemian <alinazemian@gmail.com>
> wrote:
>
> > Dear Joe,
> > Hi,
> > I am going to store the crawl web pages in accumulo as the main storage
> > part of my project and I need to give these data to solr for indexing 
> > and
> > user searches. I need to do some social and web analysis on my data as
> well
> > as having some security features. Therefore accumulo is my choice for 
> > the
> > database part and for index and search I am going to use Solr. Would you
> > please guide me through that?
> >
> >
> >
> > On Thu, Jul 24, 2014 at 1:28 AM, Joe Gresock <jgresock@gmail.com> wrote:
> >
> > > We store data in both Solr and Accumulo -- do you have more details
> about
> > > what kind of data and indexing you want?  Is there a reason you're
> > thinking
> > > of using both databases in particular?
> > >
> > >
> > > On Wed, Jul 23, 2014 at 5:17 AM, Ali Nazemian <alinazemian@gmail.com>
> > > wrote:
> > >
> > > > Dear All,
> > > > Hi,
> > > > I was wondering is there anybody out there that tried to integrate
> Solr
> > > > with Accumulo? I was thinking about using Accumulo on top of HDFS 
> > > > and
> > > using
> > > > Solr to index data inside Accumulo? Do you have any idea how can I 
> > > > do
> > > such
> > > > integration?
> > > >
> > > > Best regards.
> > > >
> > > > --
> > > > A.Nazemian
> > > >
> > >
> > >
> > >
> > > --
> > > I know what it is to be in need, and I know what it is to have plenty.
>  I
> > > have learned the secret of being content in any and every situation,
> > > whether well fed or hungry, whether living in plenty or in want.  I 
> > > can
> > do
> > > all this through him who gives me strength.    *-Philippians 4:12-13*
> > >
> >
> >
> >
> > --
> > A.Nazemian
> >
>
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.    *-Philippians 4:12-13*
>



-- 
A.Nazemian 


Mime
View raw message