manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammed Olgun <mh.ol...@gmail.com>
Subject Re: ManifoldCF 2.0 plans
Date Thu, 19 Jun 2014 07:10:08 GMT
Hi Karl,

I was thinking about may be we can add shards from the job UI. On second thought it’s out
of our scope. User should do it himself/herself.

I thought that good seeding model increasing the MCF performance. GridFS connector works with
MODEL_ALL. Assume that the user also stores added and changed documents’ metadata(or key)
in a mongodb collection. If the user wants to select other seeding model, may be we can get
from the user a query which returns the added and changed documents then the user can use
MODEL_ADD_CHANGE.

What do you think?

Muhammed

On 19 Jun 2014, at 00:40, Karl Wright <daddywri@gmail.com> wrote:

> Hi Muhammed,
> 
> Can you go into more depth about these:
> 
>>>>>>> 
> 1) Sharding support
> 2) Selectable seeding model.
> <<<<<<
> 
> Thanks,
> Karl
> 
> 
> 
> On Wed, Jun 18, 2014 at 5:38 PM, Karl Wright <daddywri@gmail.com> wrote:
> 
>> bq. What is "non-SQL data store" ? You mean to remove MFC's dependency to
>> PostgreSQL, MySQL, Derby etc?
>> 
>> See CONNECTORS-286.
>> 
>> bq. What do you think about this? Can MCF be dih replacement?  How is our
>> DB crawler compared to DIH?
>> 
>> In theory it could.  I'd hesitate before claiming feature-to-feature
>> compatibility though, and I'm not sure whether Solr people would officially
>> recommend MCF in any case, especially since they have wanted to solve
>> document security in their own way (but have never gotten around to it in
>> the 3+ years this first came to my attention).
>> 
>> Karl
>> 
>> 
>> 
>> On Wed, Jun 18, 2014 at 5:26 PM, Ahmet Arslan <iorixxx@yahoo.com.invalid>
>> wrote:
>> 
>>> Hi,
>>> 
>>> What is "non-SQL data store" ? You mean to remove MFC's dependency to
>>> PostgreSQL, MySQL, Derby etc?
>>> 
>>> 
>>> By the way solr guys are looking for a Data Import Handler (DIH)
>>> replacement.
>>> 
>>> See for the thread : http://search-lucene.com/m/WwzTb2z1w7F
>>> 
>>> DIH is mostly used to sync RDBMS to Solr.
>>> 
>>> What do you think about this? Can MCF be dih replacement?
>>> 
>>> How is our DB crawler compared to DIH?
>>> 
>>> Ahmet
>>> 
>>> 
>>> On Wednesday, June 18, 2014 11:33 PM, Muhammed Olgun <mh.olgun@gmail.com>
>>> wrote:
>>> Hi all,
>>> 
>>> I think that a non-SQL solution would be great. I have also two new ideas
>>> for GridFS connector,
>>> 
>>> 1) Sharding support
>>> 2) Selectable seeding model.
>>> 
>>> Muhammed
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 18 Jun 2014, at 23:22, Karl Wright <daddywri@gmail.com> wrote:
>>> 
>>>> Hi Piergiorgio,
>>>> 
>>>> Just to clarify -- I don't have a workable plan yet for a non-SQL data
>>>> store, so maybe that waits until 3.0.
>>>> 
>>>> Karl
>>>> 
>>>> 
>>>> 
>>>> On Wed, Jun 18, 2014 at 3:13 PM, Piergiorgio Lucidi <
>>> piergiorgio@apache.org>
>>>> wrote:
>>>> 
>>>>> +1 from me for breaking backwords compatibility and focusing on non-SQL
>>>>> data store.
>>>>> 
>>>>> Piergiorgio
>>>>> 
>>>>> 
>>>>> 2014-06-18 18:19 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> By now it is becoming clear that ManifoldCF has accumulated a lot
of
>>>>>> backwards-compatibility dead weight we have to carry around from
>>> release
>>>>> to
>>>>>> release.  However, ManifoldCF 2.0 will present an opportunity to
break
>>>>>> backwards compatibility with the 1.x releases.  Originally, I was
>>>>> thinking
>>>>>> that MCF 2.0 would be the proper release vehicle for an
>>> implementation on
>>>>>> top of a non-SQL data store, but now I am looking at this instead
as a
>>>>>> great way to clean out deprecated tabs, methods, and even whole
>>>>> connectors
>>>>>> from the codebase.
>>>>>> 
>>>>>> I'd like to consider making the MCF 2.0 release be the next one after
>>>>> 1.7.
>>>>>> Since 1.7 is scheduled for end of August, 2.0 would come out some
>>> months
>>>>>> after that.  Please comment on whether you agree with this basic
>>> plan, or
>>>>>> you have other priorities we should know about. ;-)
>>>>>> 
>>>>>> FWIW, if this *is* a good idea to you, please also list one or two
>>> main
>>>>>> areas we should work on for 2.0 that involve breaking backwards
>>>>>> compatibility.
>>>>>> 
>>>>>> Thanks,
>>>>>> Karl
>>>>>> 
>>>>>> --
>>>>>> Piergiorgio Lucidi
>>>>>> Open Source ECM Specialist
>>>>>> http://www.open4dev.com
>>>>>> 
>>>>> 
>>> 
>> 
>> 


Mime
View raw message