hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peyman Mohajerian <mohaj...@gmail.com>
Subject Re: Suggestion technology/design on this usecase
Date Tue, 28 Jan 2014 18:34:31 GMT
This is what a friend of mine that knows elastic search had to say about
this:

o   Their tagcombinations are no different than say a category or similar
grouping for data

o   A search can then be executed on the index using a mixture of search
functions

§  Search on index for the tags category

§  Then refine the search based on filters such as "India" + "Pen"

§  This would return the IMP002 result





On Tue, Jan 28, 2014 at 9:52 AM, Naresh Yadav <nyadav.ait@gmail.com> wrote:

> i had tried on cassandra, that attempt was not convincing, but not used
> distributed counters....i actually needed tagcombination ids in output, not
> the no of matches, for the given set of tags..
>
> please illustrate a little your thought by taking my tag combination table
> design..
>
>
> On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mohajeri@gmail.com>wrote:
>
>> No-sql solution with real-time counters would work, e.g. Cassandra or
>> hbase. But I think elastic search or Solr would be simpler and can do the
>> counting on access. There are solutions that are the combination of both
>> these approaches.
>>
>>
>> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <nyadav.ait@gmail.com>wrote:
>>
>>> please give suggestions on this...
>>>
>>>
>>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <nyadav.ait@gmail.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am new to big data technologies and design so looking for help from
>>>> java world.
>>>>
>>>> I have concept of tags and tagcombinations.
>>>> For example U.S.A and Pen are two tags AND if they come together in
>>>> some definition then register a tagcombination(U.S.A-Pen) for that..
>>>>
>>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>>> India-Pen-Shampoo)
>>>>
>>>> - millions of tags
>>>> - billions of tagcombinations
>>>> - one tagcombination generally have 2-8 tags....
>>>> - Every day we get lakhs of new tagcombinations to write
>>>> - daily crores of queries to find matching combination by set of tags
>>>>
>>>> Query need to support :
>>>> one tag or set of tags appears in how many tagcombinationids ????
>>>> If i query for Pen,India then it should return two tagcombinaions
>>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>>> realtime.
>>>>
>>>> Please suggest a solution which is distributed with java client and can
>>>> handle scale of data  i am looking for..
>>>>
>>>> Thanks
>>>> Naresh
>>>>
>>>
>>>
>>>
>>>
>>
>
>

Mime
View raw message