lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul നോബിള്‍ नोब्ळ्" <noble.p...@gmail.com>
Subject Re: Blob storage
Date Fri, 26 Dec 2008 14:07:35 GMT
On Fri, Dec 26, 2008 at 2:11 PM, Babak Farhang <farhang@gmail.com> wrote:
> Most of all, I'm trying to communicate an *idea* which itself cannot
> be encumbered by any license, anyway. But if you want to incorporate
> some of this code into an asf project, I'd be happy to also release it
> under the apache license. Hope the license I chose for my project
> doesn't get in the way of this conversation..

It would be more useful if the user could store the data using his own
id (a long).This forces me to have a mapping of a key -> skwish_id
elsewhere . This is a serious limitation. For retrieval it means I may
need to do a lookup and that can be costly if I have 10's of millions
of records

BTW . The license is a problem
>
> On Fri, Dec 26, 2008 at 12:46 AM, Noble Paul നോബിള്‍ नोब्ळ्
> <noble.paul@gmail.com> wrote:
>> The license is GPL . It cannont be used directly in any apache projects
>>
>> On Fri, Dec 26, 2008 at 12:47 PM, Babak Farhang <farhang@gmail.com> wrote:
>>>> I assume one could use Skwish instead of Lucene's normal stored fields to
>>>> store & retrieve document data?
>>>
>>> Exactly: instead of storing the field's value directly in Lucene, you
>>> could store it in skwish and then store its skwish id in the Lucene
>>> field instead.  This works well for serving large streams (e.g.
>>> original document contents).
>>>
>>>> Have you run any threaded performance tests comparing the two?
>>>
>>> No direct comps, yet.
>>>
>>> -b
>>>
>>>
>>> On Thu, Dec 25, 2008 at 5:22 AM, Michael McCandless
>>> <lucene@mikemccandless.com> wrote:
>>>>
>>>> This looks interesting!
>>>> I assume one could use Skwish instead of Lucene's normal stored fields to
>>>> store & retrieve document data?
>>>> Have you run any threaded performance tests comparing the two?
>>>> Mike
>>>>
>>>> Babak Farhang <farhang@gmail.com> wrote:
>>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I've been working on a library called Skwish to complement indexes
>>>>> like Lucene,  for blob storage and retrieval. This is nothing more
>>>>> than a structured implementation of storing all the files in one file
>>>>> and managing their offsets in another.  The idea is to provide a fast,
>>>>> concurrent, lock-free way to serve lots of files to lots of users.
>>>>>
>>>>> Hope you find it useful or interesting.
>>>>>
>>>>> -Babak
>>>>> http://skwish.sourceforge.net/
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>
>>
>>
>>
>> --
>> --Noble Paul
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>



-- 
--Noble Paul
Mime
View raw message