spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumedh Wale <sw...@snappydata.io>
Subject Re: benefits of code gen
Date Mon, 13 Feb 2017 05:56:31 GMT
<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">The difference is closure invocation
      instead of a static java.lang.Math call. In many cases JIT may not
      be able to perform inlining and related code optimizations though
      in this specific case it should. This is highly dependent on the
      specific case, but when inlining cannot be done and it leads to a
      method call (especially virtual call) then the difference is quite
      large: few nanoseconds per evaluation vs tens of nanoseconds in my
      experiments.<br>
      Serialization of an additional object as a reference can have a
      measurable effect for low-latency jobs though usually can be
      ignored.<br>
      <br>
      What has been observed is that if an expression uses
      CodegenFallback then it becomes an order of magnitude slower or
      more. Most of it is due to UnsafeRow read/write overhead which is
      avoided here, but still care needs to be taken for (virtual)
      function calls too. In some cases JIT does inline virtual calls
      but may not always happen. In my experience the only reliable case
      where it does inline is when the virtual call is on a local
      variable that does not change for multiple invocations (e.g. a
      final local variable outside the while loop of a doProduce).<br>
      <br>
      I think what should work better is encapsulating such code in
      methods of a scala object rather than a class and those can be
      invoked in generated code like static methods. Such calls should
      be equivalent to inline code generation in most cases since JIT
      will inline the calls where it will determine significant benefit.
      In some cases such method calls will have better CPU instruction
      cache hits (i.e. if same inline code is emitted multiple times vs
      common method calls). All this needs thorough
      micro/macro-benchmarking.<br>
      <br>
      However, I don't recall any large pieces of generated code where
      this can help. Most complex pieces like in
      HashAggregateExec/SortMergeJoinExec/BroadcastHashJoinExec are so
      because they generate schema specific code (to avoid virtual calls
      and boxing/unboxing, and UnsafeRow read/write in some cases) which
      is significantly faster than the equivalent generic code in
      doExecute. Or in your "castToDateCode" example, don't see how you
      can reduce it since bulk of code is already in the static
      stringToDate call.<br>
      <br>
      <br>
      On Saturday 11 February 2017 03:02 AM, Koert Kuipers wrote:<br>
    </div>
    <blockquote
cite="mid:CANx3uAj3M6_3_KgsQs5UcyktL+4AA0NJ64o+KKASiJ+6X4RQGg@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div class="gmail_default" style="font-family:courier
          new,monospace">based on that i take it that math functions
          would be primary beneficiaries since they work on primitives.<br>
          <br>
          so if i take UnaryMathExpression as an example, would i not
          get the same benefit if i change it to this?<br>
          <br>
          abstract class UnaryMathExpression(val f: Double =&gt; Double,
          name: String)<br>
            extends UnaryExpression with Serializable with
          ImplicitCastInputTypes {<br>
          <br>
            override def inputTypes: Seq[AbstractDataType] =
          Seq(DoubleType)<br>
            override def dataType: DataType = DoubleType<br>
            override def nullable: Boolean = true<br>
            override def toString: String = s"$name($child)"<br>
            override def prettyName: String = name<br>
          <br>
            protected override def nullSafeEval(input: Any): Any = {<br>
              f(input.asInstanceOf[Double])<br>
            }<br>
          <br>
            // name of function in java.lang.Math<br>
            def funcName: String = name.toLowerCase<br>
          <br>
            def function(d: Double): Double = f(d)<br>
          <br>
            override def doGenCode(ctx: CodegenContext, ev: ExprCode):
          ExprCode = {<br>
              val self = ctx.addReferenceObj(name, this,
          getClass.getName)<br>
              defineCodeGen(ctx, ev, c =&gt; s"$self.function($c)")<br>
            }<br>
          }<br>
          <br>
        </div>
        <div class="gmail_default" style="font-family:courier
          new,monospace">admittedly in this case the benefit in terms of
          removing complex codegen is not there (the codegen was only
          one line), but if i can remove codegen here i could also
          remove it in lots of other places where it does get very
          unwieldy simply by replacing it with calls to methods.<br>
        </div>
        <div class="gmail_default" style="font-family:courier
          new,monospace"><br>
          Function1 is specialized, so i think (or hope) that my version
          does no extra boxes/unboxing.<br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Fri, Feb 10, 2017 at 2:24 PM,
          Reynold Xin <span dir="ltr">&lt;<a moz-do-not-send="true"
              href="mailto:rxin@databricks.com" target="_blank">rxin@databricks.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">With complex types it doesn't work as well,
              but for primitive types the biggest benefit of whole stage
              codegen is that we don't even need to put the intermediate
              data into rows or columns anymore. They are just variables
              (stored in CPU registers).</div>
            <div class="HOEnZb">
              <div class="h5">
                <div class="gmail_extra"><br>
                  <div class="gmail_quote">On Fri, Feb 10, 2017 at 8:22
                    PM, Koert Kuipers <span dir="ltr">&lt;<a
                        moz-do-not-send="true"
                        href="mailto:koert@tresata.com" target="_blank">koert@tresata.com</a>&gt;</span>
                    wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0
                      .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <div dir="ltr">
                        <div class="gmail_default"
                          style="font-family:courier new,monospace">so i
                          have been looking for a while now at all the
                          catalyst expressions, and all the relative
                          complex codegen going on.<br>
                          <br>
                        </div>
                        <div class="gmail_default"
                          style="font-family:courier new,monospace">so
                          first off i get the benefit of codegen to turn
                          a bunch of chained iterators transformations
                          into a single codegen stage for spark. that
                          makes sense to me, because it avoids a bunch
                          of overhead.<br>
                          <br>
                        </div>
                        <div class="gmail_default"
                          style="font-family:courier new,monospace">but
                          what i am not so sure about is what the
                          benefit is of converting the actual stuff that
                          happens inside the iterator transformations
                          into codegen. <br>
                          <br>
                          say if we have an expression that has 2
                          children and creates a struct for them. why
                          would this be faster in codegen by re-creating
                          the code to do this in a string (which is
                          complex and error prone) compared to simply
                          have the codegen call the normal method for
                          this in my class?<br>
                          <br>
                        </div>
                        <div class="gmail_default"
                          style="font-family:courier new,monospace">i
                          see so much trivial code be re-created in
                          codegen. stuff like this:<br>
                          <br>
                            private[this] def castToDateCode(<br>
                                from: DataType,<br>
                                ctx: CodegenContext): CastFunction =
                          from match {<br>
                              case StringType =&gt;<br>
                                val intOpt = ctx.freshName("intOpt")<br>
                                (c, evPrim, evNull) =&gt; s"""<br>
                                  scala.Option&lt;Integer&gt; $intOpt =<br>
                                    org.apache.spark.sql.catalyst.<wbr>util.DateTimeUtils.stringToDat<wbr>e($c);<br>
                                  if ($intOpt.isDefined()) {<br>
                                    $evPrim = ((Integer)
                          $intOpt.get()).intValue();<br>
                                  } else {<br>
                                    $evNull = true;<br>
                                  }<br>
                                 """<br>
                          <br>
                        </div>
                        <div class="gmail_default"
                          style="font-family:courier new,monospace">is
                          this really faster than simply calling an
                          equivalent functions from the codegen, and
                          keeping the codegen logic restricted to the
                          "unrolling" of chained iterators?<br>
                          <br>
                        </div>
                      </div>
                    </blockquote>
                  </div>
                  <br>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  </body>
</html>

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message