metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Subject Re: API suggestion for new Data Connectors
Date Wed, 20 May 2015 18:05:08 GMT
Hi Ashish,

I support that idea fully. I don't yet have any great ideas on how to pull
it off - that would probably require more than a few experiments and design
strategies, but what you describe sounds good. Would love to see a simple
example of it, maybe to begin with just to implement it for a single aspect
such as WHERE items or so.

Best regards,
Kasper

2015-05-20 19:12 GMT+02:00 Ashish Mukherjee <ashish.mukherjee@gmail.com>:

> Hello,
>
> I was looking through some of the DataContext classes like ElasticSearch
> etc.  for my needs and was working on the Solr class recently. It seems
> that it is not uncommon not to push-down many querying operations because
> implementing executeQuery() can be tricky, as every kind of SQL is
> supported by executeQuery(). Therefore, some of the connectors may not
> scale very well due to lot of in-memory processing of large data-sets on
> the application side.
>
> I was wondering if the following design would simplify implementation of
> executeQuery() in each DataContext implementation -
>
> Let QueryPostProcessDataContext expose few more granular hooks (like
> executeCountQuery() already exists), such as createFilters(),
> createHaving() etc.
>
> These can have default implementations in QueryPostProcessDataContext class
> which are called in a pipeline pattern for the entire query construction
> and execution.
>
> Those implementing classes which can/want to implement joins etc. natively
> can do so else the functionality can be satisfied by the base class method
> call in the pipeline.
>
> By following such a design, it may be easier to support as many query
> functions natively as possible while leaving the rest to the MM framework.
> Then, it would not be an all or nothing implementation for a data connector
> and keep memory footprint of the application manageable even for large
> data-sets.
>
> These are just high level thoughts as of now, which I wanted to bounce off
> with the group.
>
> Regards,
> Ashish
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message