Mailing-List: contact dev-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAGj+YsdmSVLtUkvuFPC1PTbyJMJokMP7_cCHisrDa=XRiRLj2g@mail.gmail.com>
References: 
 <CAGj+YscuVy4kAk-9KXSMmJKea74Z_6OtYdice=Ey70=NtOPptw@mail.gmail.com>
	<CAGj+YsdSe8ae5gob2OrwtoXs2f+iW7g+Qc+3Nhe1kzcG6Lzvmg@mail.gmail.com>
	<CAKTYAC9o1AP=BMAZQiN9qhz96Fh5V-Z-56f9rD52+U=wyn-Mqw@mail.gmail.com>
	<CAGj+YsdmSVLtUkvuFPC1PTbyJMJokMP7_cCHisrDa=XRiRLj2g@mail.gmail.com>
Date: Tue, 22 Sep 2015 10:05:06 -0700
Message-ID: 
 <CAKTYAC8Ps763AGkZ4sXEhVBcs7CUhGp-k6WixKXe7c-hzye8Rw@mail.gmail.com>
Subject: Re: The meaning/intent of the following methods - JSON storage format
 - part 1
From: Abdel Hakim Deneche <adeneche@maprtech.com>
To: "dev@drill.apache.org" <dev@drill.apache.org>
Content-Type: multipart/alternative; boundary=047d7bf19850ecca13052058ff6f

--047d7bf19850ecca13052058ff6f
Content-Type: text/plain; charset=UTF-8

Question 3:
I would assume getWriterOperatorType() has a similar meaning to
getReaderOperatorType() but it's not used anywhere in Drill. That would
explain why JsonFormatPlugin throws an UnsupportedOperationException

Question 4:
supportsPushDown() is about "projection" push down and not filter push down

Question 5:
I would say, purely random.

On Fri, Sep 18, 2015 at 11:48 AM, Edmon Begoli <ebegoli@gmail.com> wrote:

> Thanks, Abdel.
> I will use your answers to guide my development, but I will also contribute
> them back as a javadoc.
>
> On Fri, Sep 18, 2015 at 2:39 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com>
> wrote:
>
> > Question 2:
> > to my knowledge (what I've found digging through the code). Each Operator
> > has a unique OperatorType that is used when writing the operator's stats
> in
> > the profile. So in this case, Each FormatPlugin implementation should
> have
> > a corresponding unique OperatorType added to CoreOperatorType.
> >
> > I will get back to you after I get more information about the remaining
> > questions.
> >
> > Thanks
> >
> > On Thu, Sep 17, 2015 at 8:14 PM, Edmon Begoli <ebegoli@gmail.com> wrote:
> >
> > > Hello all,
> > >
> > > Could some please with answering my questions 2-5 below?
> > >
> > > Thank you,
> > > Edmon
> > >
> > > On Wednesday, September 16, 2015, Edmon Begoli <ebegoli@gmail.com>
> > wrote:
> > >
> > > > I am studying the JSON Storage plugin.
> > > >
> > > > Can someone please answer what is the meaning (or intent) of the
> > > following
> > > > methods:
> > > >
> > > > # Question 1:
> > > > # This is from inside the getter for RecordWriter. What is
> major/minor
> > > > fragment id?
> > > > String fragmentId = String.format("%d_%d",
> handle.getMajorFragmentId(),
> > > > handle.getMinorFragmentId());
> > > >
> > > >
> > > > # Question 2:
> > > > # Speficially, what is Operator Type, and what is JSON specific about
> > it?
> > > > public int getReaderOperatorType() {
> > > >     return CoreOperatorType.JSON_SUB_SCAN_VALUE;
> > > >   }
> > > >
> > > > # Question 3:
> > > > # same, but for writer
> > > >   @Override
> > > >   public int getWriterOperatorType() {
> > > >     throw new UnsupportedOperationException();
> > > >   }
> > > >
> > > > # Question 4:
> > > > # I generally understand the concept of predicate pushdown, but is
> this
> > > > about predicate push down or something else?
> > > > # How does one implement a push down for storage format?
> > > >   @Override
> > > >   public boolean supportsPushDown() {
> > > >
> > > > # Question 5:
> > > > # Is this int value of 31 purely randomly selected, or is there an
> > index
> > > > somewhere?
> > > >   @Override
> > > >   public int hashCode() {
> > > >       return 31;
> > > >   }
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> >   <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>


-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

--047d7bf19850ecca13052058ff6f--