tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyunsik Choi <hyun...@apache.org>
Subject Re: [GSoc2013] - Outer Join - 2 very important questions
Date Sun, 15 Sep 2013 09:49:38 GMT
Hi camelia,

I'm sorry for late response. The solution is simple. You can modify
existing source code. I've changed the 121 line in Int4Datum as
follows:

default:
  if (datum instanceof NullDatum) {
    return DatumFactory.createBool(false);
  } else {
    throw new InvalidOperationException();
  }

Then, the all unit tests of TestLeftOuter_NLJoinExec are passed. Also,
other Datum classes need to have the above codes.

Anyway, you work looks very great.

Best regards,
Hyunsik Choi

On Sun, Sep 15, 2013 at 3:55 AM, camelia c <camelie_1985@yahoo.com> wrote:
> Hello,
>
> 1)
> I'm still waiting for Your answer on this question please.
> I re-tell You the problem.
>
> I have a table A with a column fkA of type INT4 which is a foreign key referencing table
B's primary key pkB.
> The data in column fkA contains both numbers and null values (NullDatum).
>
>
> In table B, column pkB is of type INT4 and it is the primary key => all its values
are non-null. So in pkB we have only numbers.
>
> When faced with testing the join condition
>
>
> if (joinQual != null) {
>         joinQual.eval(qualCtx, inSchema, frameTuple);
>         if (joinQual.terminate(qualCtx).asBool()) {
>
> there is a problem:
>
> ERROR worker.Task: org.apache.tajo.datum.exception.InvalidOperationException
>     at org.apache.tajo.datum.Int4Datum.equalsTo(Int4Datum.java:122)
>     at org.apache.tajo.engine.eval.BinaryEval.terminate(BinaryEval.java:158)
>     at org.apache.tajo.engine.planner.physical.LeftOuter_NLJoinExec.next(LeftOuter_NLJoinExec.java:157)
>     at
>  org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:85)
>     at org.apache.tajo.worker.Task.run(Task.java:378)
>     at org.apache.tajo.worker.TaskRunner$2.run(TaskRunner.java:359)
>     at java.lang.Thread.run(Thread.java:662)
>
>
> So, once again my question is: what should be modified in order to allow the comparison
of null values residents in an INT4 column to numbers in a different table's INT4 column.
>
> 2) How can I perform a specific unit test without having to perform all tests. I tried
different choices suggested at http://tajo.incubator.apache.org/build.html,  but no success.
> For example, say You want to perform only test   TestNLJoinExec.java
> What is the complete command that You type in the terminal?
>
>
>
> Please answer me this message as soon as possible, please.
>
>
> Yours sincerely,
>
> Camelia
>
>
>
>
>
>
>
>
>
>
>
>
>
> ________________________________
>  From: camelia c <camelie_1985@yahoo.com>
> To: "dev@tajo.incubator.apache.org" <dev@tajo.incubator.apache.org>
> Sent: Wednesday, September 11, 2013 11:06 PM
> Subject: Re: [GSoc2013] - Outer Join -Possible issue about BinaryEvalCtx (PS)
>
>
> PS:
>
> The problem is either there in BinaryEvalCtx, or in class
>
> public class Int4Datum extends NumericDatum.
>
>
> in the method
> public int compareTo(Datum datum) {
>
> ...
>
>
> @Override
>  127   public int compareTo(Datum datum) {
>  128     switch (datum.type()) {
>  129       case INT2:
>  130         if (val <
>  datum.asInt2()) {
>  131           return -1;
>  132         } else if (datum.asInt2() < val) {
>  133           return 1;
>  134         } else {
>  135           return 0;
>  136         }
>  137       case INT4:
>  138         if (val < datum.asInt4()) {
>  139           return -1;
>  140         } else if (datum.asInt4() < val) {
>  141           return
>  1;
>  142         } else {
>  143           return 0;
>  144         }
>  145       case INT8:
>  146         if (val < datum.asInt8()) {
>  147           return -1;
>  148         } else if (datum.asInt8() < val) {
>  149           return 1;
>  150         } else {
>  151           return 0;
>  152         }
>  153       case
>  FLOAT4:
>  154         if (val < datum.asFloat4()) {
>  155           return -1;
>  156         } else if (datum.asFloat4() < val) {
>  157           return 1;
>  158         } else {
>  159           return 0;
>  160         }
>  161       case FLOAT8:
>  162         if (val < datum.asFloat8()) {
>  163           return -1;
>  164         } else if (datum.asFloat8() < val)
>  {
>  165           return 1;
>  166         } else {
>  167           return 0;
>  168         }
>  169       default:
>  170         throw new InvalidOperationException(datum.type());
>  171     }
>  172   }
>  173
>
>
>
> Just a hint, maybe it helps You find and solve the issue more quickly.
>
> Camelia
>
>
>
>
> ________________________________
> From: camelia c <camelie_1985@yahoo.com>
> To: "dev@tajo.incubator.apache.org" <dev@tajo.incubator.apache.org>
> Sent: Wednesday, September 11, 2013 10:18 PM
> Subject: Re: [GSoc2013] - Outer Join -Possible issue about BinaryEvalCtx
>
>
> Hello,
>
> I would like to kindly ask You to take a moment and explain to me how does this BinaryEvalCtx
handle NULL values in fields. From an error I got recently it seems that when it reads NULL
values and it has to compare them to non-NULL values, it breaks.
>
>
> The portion of code leading to error in  is the same as in the original NLJoinExec and
other join algorithms, where TAJO needs to verify the join condition and then project according
to the output schema:
>
> if (joinQual != null) {
>         joinQual.eval(qualCtx, inSchema,
>  frameTuple);
>         if (joinQual.terminate(qualCtx).asBool()) {
>
> Here the verification of the join condition fails if one operand is a NULL value and
the other is a number, in this case number 10.
>
>
>
> 13/09/11 21:37:01 INFO physical.LeftOuter_NLJoinExec: ********leftChild.next() =(0=>10)
>
> 13/09/11 21:37:01 INFO physical.LeftOuter_NLJoinExec: ********rightChild.next() =(0=>333.0,
1=>10)
>
> 13/09/11 21:37:01 INFO physical.LeftOuter_NLJoinExec: ******** a result matched padded
tuple =(0=>10, 1=>333.0)
>
> 13/09/11 21:37:01 INFO physical.LeftOuter_NLJoinExec: ********rightChild.next() =(0=>555.0,
1=>10)
>
> 13/09/11 21:37:01 INFO physical.LeftOuter_NLJoinExec: ******** a result matched padded
tuple =(0=>10, 1=>555.0)
>
> 13/09/11 21:37:01 INFO physical.LeftOuter_NLJoinExec:
>  ********rightChild.next() =(0=>777.0, 1=>NULL)
>
> 13/09/11 21:37:01 ERROR worker.Task: org.apache.tajo.datum.exception.InvalidOperationException
>     at org.apache.tajo.datum.Int4Datum.equalsTo(Int4Datum.java:122)
>     at org.apache.tajo.engine.eval.BinaryEval.terminate(BinaryEval.java:158)
>     at org.apache.tajo.engine.planner.physical.LeftOuter_NLJoinExec.next(LeftOuter_NLJoinExec.java:157)
>     at org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:85)
>     at org.apache.tajo.worker.Task.run(Task.java:378)
>     at org.apache.tajo.worker.TaskRunner$2.run(TaskRunner.java:359)
>     at java.lang.Thread.run(Thread.java:662)
>
>
> It seems that at this moment, it takes the NULL as being Int4Datum, instead of NullDatum.
>
> These classes exist beyond outer join nodes' scope but maybe they
>  weren't tested for NULL values before. Now, some of the outer join's processing relies
on these classes' correct behavior, that is why I'm trying to signal when I find errors.
>
>
>
> Thank You in advance for Your answer and for the understanding!
>
> Yours sincerely,
> Camelia

Mime
View raw message