Mailing-List: contact dev-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@manifoldcf.apache.org
Received-SPF: pass (nike.apache.org: domain of mh.olgun@gmail.com designates
 74.125.82.41 as permitted sender)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\))
Subject: Re: ManifoldCF 2.0 plans
From: Muhammed Olgun <mh.olgun@gmail.com>
In-Reply-To: 
 <CALUFAGDyJ1sXzw1gHDnhywhMRjYXFmKQdB0wQbstq6YJY+9GFw@mail.gmail.com>
Date: Thu, 19 Jun 2014 10:10:08 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <BC0851F6-669F-4BF9-AB2A-BC40352508E4@gmail.com>
References: 
 <CALUFAGAyVhfP=Yg06K1CgHW4oi87-ziU-3c7CMdb093AOupfJQ@mail.gmail.com>
 <CAEO2op_wFYTwhG8AXjCgThdyU59M4FO3aarCnv0KJcBKVMiRhg@mail.gmail.com>
 <CALUFAGBuZOEUhPh4FUKPyWhT2fZw22pnk1k2=S3oEUqEKbX=1Q@mail.gmail.com>
 <6C9A506B-F3F2-41C4-AC9F-FCCE2788B0A7@gmail.com>
 <1403126791.19027.YahooMailNeo@web124705.mail.ne1.yahoo.com>
 <CALUFAGD0nSbzzaYW_wrSdBQuQdAedoje-N-acw3y=yX3J4qwqQ@mail.gmail.com>
 <CALUFAGDyJ1sXzw1gHDnhywhMRjYXFmKQdB0wQbstq6YJY+9GFw@mail.gmail.com>
To: dev@manifoldcf.apache.org

Hi Karl,

I was thinking about may be we can add shards from the job UI. On second =
thought it=92s out of our scope. User should do it himself/herself.

I thought that good seeding model increasing the MCF performance. GridFS =
connector works with MODEL_ALL. Assume that the user also stores added =
and changed documents=92 metadata(or key) in a mongodb collection. If =
the user wants to select other seeding model, may be we can get from the =
user a query which returns the added and changed documents then the user =
can use MODEL_ADD_CHANGE.

What do you think?

Muhammed

On 19 Jun 2014, at 00:40, Karl Wright <daddywri@gmail.com> wrote:

> Hi Muhammed,
>=20
> Can you go into more depth about these:
>=20
>>>>>>>=20
> 1) Sharding support
> 2) Selectable seeding model.
> <<<<<<
>=20
> Thanks,
> Karl
>=20
>=20
>=20
> On Wed, Jun 18, 2014 at 5:38 PM, Karl Wright <daddywri@gmail.com> =
wrote:
>=20
>> bq. What is "non-SQL data store" ? You mean to remove MFC's =
dependency to
>> PostgreSQL, MySQL, Derby etc?
>>=20
>> See CONNECTORS-286.
>>=20
>> bq. What do you think about this? Can MCF be dih replacement?  How is =
our
>> DB crawler compared to DIH?
>>=20
>> In theory it could.  I'd hesitate before claiming feature-to-feature
>> compatibility though, and I'm not sure whether Solr people would =
officially
>> recommend MCF in any case, especially since they have wanted to solve
>> document security in their own way (but have never gotten around to =
it in
>> the 3+ years this first came to my attention).
>>=20
>> Karl
>>=20
>>=20
>>=20
>> On Wed, Jun 18, 2014 at 5:26 PM, Ahmet Arslan =
<iorixxx@yahoo.com.invalid>
>> wrote:
>>=20
>>> Hi,
>>>=20
>>> What is "non-SQL data store" ? You mean to remove MFC's dependency =
to
>>> PostgreSQL, MySQL, Derby etc?
>>>=20
>>>=20
>>> By the way solr guys are looking for a Data Import Handler (DIH)
>>> replacement.
>>>=20
>>> See for the thread : http://search-lucene.com/m/WwzTb2z1w7F
>>>=20
>>> DIH is mostly used to sync RDBMS to Solr.
>>>=20
>>> What do you think about this? Can MCF be dih replacement?
>>>=20
>>> How is our DB crawler compared to DIH?
>>>=20
>>> Ahmet
>>>=20
>>>=20
>>> On Wednesday, June 18, 2014 11:33 PM, Muhammed Olgun =
<mh.olgun@gmail.com>
>>> wrote:
>>> Hi all,
>>>=20
>>> I think that a non-SQL solution would be great. I have also two new =
ideas
>>> for GridFS connector,
>>>=20
>>> 1) Sharding support
>>> 2) Selectable seeding model.
>>>=20
>>> Muhammed
>>>=20
>>>=20
>>>=20
>>>=20
>>>=20
>>> On 18 Jun 2014, at 23:22, Karl Wright <daddywri@gmail.com> wrote:
>>>=20
>>>> Hi Piergiorgio,
>>>>=20
>>>> Just to clarify -- I don't have a workable plan yet for a non-SQL =
data
>>>> store, so maybe that waits until 3.0.
>>>>=20
>>>> Karl
>>>>=20
>>>>=20
>>>>=20
>>>> On Wed, Jun 18, 2014 at 3:13 PM, Piergiorgio Lucidi <
>>> piergiorgio@apache.org>
>>>> wrote:
>>>>=20
>>>>> +1 from me for breaking backwords compatibility and focusing on =
non-SQL
>>>>> data store.
>>>>>=20
>>>>> Piergiorgio
>>>>>=20
>>>>>=20
>>>>> 2014-06-18 18:19 GMT+02:00 Karl Wright <daddywri@gmail.com>:
>>>>>=20
>>>>>> Hi all,
>>>>>>=20
>>>>>> By now it is becoming clear that ManifoldCF has accumulated a lot =
of
>>>>>> backwards-compatibility dead weight we have to carry around from
>>> release
>>>>> to
>>>>>> release.  However, ManifoldCF 2.0 will present an opportunity to =
break
>>>>>> backwards compatibility with the 1.x releases.  Originally, I was
>>>>> thinking
>>>>>> that MCF 2.0 would be the proper release vehicle for an
>>> implementation on
>>>>>> top of a non-SQL data store, but now I am looking at this instead =
as a
>>>>>> great way to clean out deprecated tabs, methods, and even whole
>>>>> connectors
>>>>>> from the codebase.
>>>>>>=20
>>>>>> I'd like to consider making the MCF 2.0 release be the next one =
after
>>>>> 1.7.
>>>>>> Since 1.7 is scheduled for end of August, 2.0 would come out some
>>> months
>>>>>> after that.  Please comment on whether you agree with this basic
>>> plan, or
>>>>>> you have other priorities we should know about. ;-)
>>>>>>=20
>>>>>> FWIW, if this *is* a good idea to you, please also list one or =
two
>>> main
>>>>>> areas we should work on for 2.0 that involve breaking backwards
>>>>>> compatibility.
>>>>>>=20
>>>>>> Thanks,
>>>>>> Karl
>>>>>>=20
>>>>>> --
>>>>>> Piergiorgio Lucidi
>>>>>> Open Source ECM Specialist
>>>>>> http://www.open4dev.com
>>>>>>=20
>>>>>=20
>>>=20
>>=20
>>=20