manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: ManifoldCF 2.0 plans
Date Thu, 19 Jun 2014 08:06:23 GMT
Hi Muhammed,

I think it is important that each connector describe its seeding model in
the most precise way possible.  If you thing GridFS could specify a tighter
seeding model, then by all means create a ticket and fix it.  But I don't
think it's necessary to wait until MCF 2.0 to do it, since the documents
returned for MODEL_ALL are a superset of those returned for
MODEL_ADD_CHANGE.

Thanks,
Karl




On Thu, Jun 19, 2014 at 3:10 AM, Muhammed Olgun <mh.olgun@gmail.com> wrote:

> Hi Karl,
>
> I was thinking about may be we can add shards from the job UI. On second
> thought it’s out of our scope. User should do it himself/herself.
>
> I thought that good seeding model increasing the MCF performance. GridFS
> connector works with MODEL_ALL. Assume that the user also stores added and
> changed documents’ metadata(or key) in a mongodb collection. If the user
> wants to select other seeding model, may be we can get from the user a
> query which returns the added and changed documents then the user can use
> MODEL_ADD_CHANGE.
>
> What do you think?
>
> Muhammed
>
> On 19 Jun 2014, at 00:40, Karl Wright <daddywri@gmail.com> wrote:
>
> > Hi Muhammed,
> >
> > Can you go into more depth about these:
> >
> >>>>>>>
> > 1) Sharding support
> > 2) Selectable seeding model.
> > <<<<<<
> >
> > Thanks,
> > Karl
> >
> >
> >
> > On Wed, Jun 18, 2014 at 5:38 PM, Karl Wright <daddywri@gmail.com> wrote:
> >
> >> bq. What is "non-SQL data store" ? You mean to remove MFC's dependency
> to
> >> PostgreSQL, MySQL, Derby etc?
> >>
> >> See CONNECTORS-286.
> >>
> >> bq. What do you think about this? Can MCF be dih replacement?  How is
> our
> >> DB crawler compared to DIH?
> >>
> >> In theory it could.  I'd hesitate before claiming feature-to-feature
> >> compatibility though, and I'm not sure whether Solr people would
> officially
> >> recommend MCF in any case, especially since they have wanted to solve
> >> document security in their own way (but have never gotten around to it
> in
> >> the 3+ years this first came to my attention).
> >>
> >> Karl
> >>
> >>
> >>
> >> On Wed, Jun 18, 2014 at 5:26 PM, Ahmet Arslan <iorixxx@yahoo.com.invalid
> >
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> What is "non-SQL data store" ? You mean to remove MFC's dependency to
> >>> PostgreSQL, MySQL, Derby etc?
> >>>
> >>>
> >>> By the way solr guys are looking for a Data Import Handler (DIH)
> >>> replacement.
> >>>
> >>> See for the thread : http://search-lucene.com/m/WwzTb2z1w7F
> >>>
> >>> DIH is mostly used to sync RDBMS to Solr.
> >>>
> >>> What do you think about this? Can MCF be dih replacement?
> >>>
> >>> How is our DB crawler compared to DIH?
> >>>
> >>> Ahmet
> >>>
> >>>
> >>> On Wednesday, June 18, 2014 11:33 PM, Muhammed Olgun <
> mh.olgun@gmail.com>
> >>> wrote:
> >>> Hi all,
> >>>
> >>> I think that a non-SQL solution would be great. I have also two new
> ideas
> >>> for GridFS connector,
> >>>
> >>> 1) Sharding support
> >>> 2) Selectable seeding model.
> >>>
> >>> Muhammed
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On 18 Jun 2014, at 23:22, Karl Wright <daddywri@gmail.com> wrote:
> >>>
> >>>> Hi Piergiorgio,
> >>>>
> >>>> Just to clarify -- I don't have a workable plan yet for a non-SQL data
> >>>> store, so maybe that waits until 3.0.
> >>>>
> >>>> Karl
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Jun 18, 2014 at 3:13 PM, Piergiorgio Lucidi <
> >>> piergiorgio@apache.org>
> >>>> wrote:
> >>>>
> >>>>> +1 from me for breaking backwords compatibility and focusing on
> non-SQL
> >>>>> data store.
> >>>>>
> >>>>> Piergiorgio
> >>>>>
> >>>>>
> >>>>> 2014-06-18 18:19 GMT+02:00 Karl Wright <daddywri@gmail.com>:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> By now it is becoming clear that ManifoldCF has accumulated
a lot of
> >>>>>> backwards-compatibility dead weight we have to carry around
from
> >>> release
> >>>>> to
> >>>>>> release.  However, ManifoldCF 2.0 will present an opportunity
to
> break
> >>>>>> backwards compatibility with the 1.x releases.  Originally,
I was
> >>>>> thinking
> >>>>>> that MCF 2.0 would be the proper release vehicle for an
> >>> implementation on
> >>>>>> top of a non-SQL data store, but now I am looking at this instead
> as a
> >>>>>> great way to clean out deprecated tabs, methods, and even whole
> >>>>> connectors
> >>>>>> from the codebase.
> >>>>>>
> >>>>>> I'd like to consider making the MCF 2.0 release be the next
one
> after
> >>>>> 1.7.
> >>>>>> Since 1.7 is scheduled for end of August, 2.0 would come out
some
> >>> months
> >>>>>> after that.  Please comment on whether you agree with this basic
> >>> plan, or
> >>>>>> you have other priorities we should know about. ;-)
> >>>>>>
> >>>>>> FWIW, if this *is* a good idea to you, please also list one
or two
> >>> main
> >>>>>> areas we should work on for 2.0 that involve breaking backwards
> >>>>>> compatibility.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Karl
> >>>>>>
> >>>>>> --
> >>>>>> Piergiorgio Lucidi
> >>>>>> Open Source ECM Specialist
> >>>>>> http://www.open4dev.com
> >>>>>>
> >>>>>
> >>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message