drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: OK_NEW_SCHEMA: changed schema or maybe just new report?
Date Sat, 03 Oct 2015 04:42:54 GMT
You have to be more careful than that. OK new schema means the schema
changed or the vector instances changed (or both). Depending on whether you
are holding cross batch vector references or not impacts whether checking
for an actual schema difference is sufficient.
On Oct 2, 2015 12:38 PM, "Abdel Hakim Deneche" <adeneche@maprtech.com>
wrote:

> Some operators like external sort or window functions will double check to
> make sure the schema really changed before throwing a schema-change
> exception. But others, like streaming aggregate or hash aggregate, don't
> check and will throw a schema-change exception even if the schema didn't
> really change.
>
> so I guess it's a combination of both
>
> On Fri, Oct 2, 2015 at 11:26 AM, Daniel Barclay <dbarclay@maprtech.com>
> wrote:
>
> > Is OK_NEW_SCHEMA intended to signal that the new schema is actually
> > changed from any previous schema, or is it intended only to signal
> > that there's a new (report of the) schema, but the new schema isn't
> > necessarily different from any previous schema)?
> >
> >
> > Are callers (of XxxBatch next() methods) that receive OK_NEW_SCHEMA
> > responsible for checking whether the new schema is the same as the old
> > schema (or, more generally, can be handled by the caller) and avoiding
> > throwing a schema-change exception in that case?
> >
> > Or are callees (XxxBatch next() methods) responsible for not returning
> > OK_NEW_SCHEMA if the new schema is the same as the previous schema?
> >
> > (Or is it some combination?)
> >
> > Thanks,
> > Daniel
> > --
> > Daniel Barclay
> > MapR Technologies
> >
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message