drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdel Hakim Deneche <adene...@maprtech.com>
Subject Re: The meaning/intent of the following methods - JSON storage format - part 1
Date Tue, 22 Sep 2015 17:05:06 GMT
Question 3:
I would assume getWriterOperatorType() has a similar meaning to
getReaderOperatorType() but it's not used anywhere in Drill. That would
explain why JsonFormatPlugin throws an UnsupportedOperationException

Question 4:
supportsPushDown() is about "projection" push down and not filter push down

Question 5:
I would say, purely random.

On Fri, Sep 18, 2015 at 11:48 AM, Edmon Begoli <ebegoli@gmail.com> wrote:

> Thanks, Abdel.
> I will use your answers to guide my development, but I will also contribute
> them back as a javadoc.
>
> On Fri, Sep 18, 2015 at 2:39 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com>
> wrote:
>
> > Question 2:
> > to my knowledge (what I've found digging through the code). Each Operator
> > has a unique OperatorType that is used when writing the operator's stats
> in
> > the profile. So in this case, Each FormatPlugin implementation should
> have
> > a corresponding unique OperatorType added to CoreOperatorType.
> >
> > I will get back to you after I get more information about the remaining
> > questions.
> >
> > Thanks
> >
> > On Thu, Sep 17, 2015 at 8:14 PM, Edmon Begoli <ebegoli@gmail.com> wrote:
> >
> > > Hello all,
> > >
> > > Could some please with answering my questions 2-5 below?
> > >
> > > Thank you,
> > > Edmon
> > >
> > > On Wednesday, September 16, 2015, Edmon Begoli <ebegoli@gmail.com>
> > wrote:
> > >
> > > > I am studying the JSON Storage plugin.
> > > >
> > > > Can someone please answer what is the meaning (or intent) of the
> > > following
> > > > methods:
> > > >
> > > > # Question 1:
> > > > # This is from inside the getter for RecordWriter. What is
> major/minor
> > > > fragment id?
> > > > String fragmentId = String.format("%d_%d",
> handle.getMajorFragmentId(),
> > > > handle.getMinorFragmentId());
> > > >
> > > >
> > > > # Question 2:
> > > > # Speficially, what is Operator Type, and what is JSON specific about
> > it?
> > > > public int getReaderOperatorType() {
> > > >     return CoreOperatorType.JSON_SUB_SCAN_VALUE;
> > > >   }
> > > >
> > > > # Question 3:
> > > > # same, but for writer
> > > >   @Override
> > > >   public int getWriterOperatorType() {
> > > >     throw new UnsupportedOperationException();
> > > >   }
> > > >
> > > > # Question 4:
> > > > # I generally understand the concept of predicate pushdown, but is
> this
> > > > about predicate push down or something else?
> > > > # How does one implement a push down for storage format?
> > > >   @Override
> > > >   public boolean supportsPushDown() {
> > > >
> > > > # Question 5:
> > > > # Is this int value of 31 purely randomly selected, or is there an
> > index
> > > > somewhere?
> > > >   @Override
> > > >   public int hashCode() {
> > > >       return 31;
> > > >   }
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> >   <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message