spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant
Date Tue, 07 Jun 2016 02:44:21 GMT

     [ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-15795:
------------------------------------

    Assignee: Apache Spark

> Enable more optimizations in whole stage codegen when isNull is a compile-time constant
> ---------------------------------------------------------------------------------------
>
>                 Key: SPARK-15795
>                 URL: https://issues.apache.org/jira/browse/SPARK-15795
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Hiroshi Inoue
>            Assignee: Apache Spark
>
> Whole stage codegen often creates {{isNull}} variable initialized with constant _false_,
like
> {{boolean mapelements_isNull = false || false;}}
> If there is no further assignment for this {{isNull}} variable, whole stage codegen can
do more optimizations by assuming {{isNull}} as a compile-time constant.
> In the example below, which is generated for a dataset map operation, {{mapelements_isNull}}
defined at line 115 can be assumed by a compile-time constant (false). 
> By assuming this as a constant, the whole stage codegen eliminates {{zeroOutNullBytes}}
at line 119 and an if-statement at line 121.
> In addition to the benefits of improved readability of generated code, eliminating {{zeroOutNullBytes}}
will give performance advantage since it is difficult to remove for Java JIT compiler.
> {code}
> /* 107 */       // CONSUME: Project [id#0L AS l#3L]
> /* 108 */       // CONSUME: DeserializeToObject l#3: bigint, obj#16: bigint
> /* 109 */       // CONSUME: MapElements <function1>, obj#17: bigint
> /* 110 */       // CONSUME: SerializeFromObject [input[0, bigint, true] AS value#18L]
> /* 111 */       // <function1>.apply
> /* 112 */       Object mapelements_obj = ((Expression) references[1]).eval(null);
> /* 113 */       scala.Function1 mapelements_value1 = (scala.Function1) mapelements_obj;
> /* 114 */
> /* 115 */       boolean mapelements_isNull = false || false;
> /* 116 */       final long mapelements_value = mapelements_isNull ? -1L : (Long) mapelements_value1.apply(range_value);
> /* 117 */
> /* 118 */       // CONSUME: WholeStageCodegen
> /* 119 */       serializefromobject_rowWriter.zeroOutNullBytes();
> /* 120 */
> /* 121 */       if (mapelements_isNull) {
> /* 122 */         serializefromobject_rowWriter.setNullAt(0);
> /* 123 */       } else {
> /* 124 */         serializefromobject_rowWriter.write(0, mapelements_value);
> /* 125 */       }
> /* 126 */       append(serializefromobject_result);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message