spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-18952) regex strings not properly escaped in codegen for aggregations
Date Mon, 09 Jan 2017 23:18:58 GMT

     [ https://issues.apache.org/jira/browse/SPARK-18952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Rosen resolved SPARK-18952.
--------------------------------
       Resolution: Fixed
         Assignee: Burak Yavuz
    Fix Version/s: 2.2.0
                   2.1.1

Fixed by Burak's patch for 2.1.1 / 2.2.0.

> regex strings not properly escaped in codegen for aggregations
> --------------------------------------------------------------
>
>                 Key: SPARK-18952
>                 URL: https://issues.apache.org/jira/browse/SPARK-18952
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.0.2
>            Reporter: Burak Yavuz
>            Assignee: Burak Yavuz
>             Fix For: 2.1.1, 2.2.0
>
>
> If I use the function regexp_extract, and then in my regex string, use `\`, i.e. escape
character, this fails codegen, because the `\` character is not properly escaped when codegen'd.
> Example stack trace:
> {code}
> /* 059 */     private int maxSteps = 2;
> /* 060 */     private int numRows = 0;
> /* 061 */     private org.apache.spark.sql.types.StructType keySchema = new org.apache.spark.sql.types.StructType().add("date_format(window#325.start,
yyyy-MM-dd HH:mm)", org.apache.spark.sql.types.DataTypes.StringType)
> /* 062 */     .add("regexp_extract(source#310.description, ([a-zA-Z]+)\[.*, 1)", org.apache.spark.sql.types.DataTypes.StringType);
> /* 063 */     private org.apache.spark.sql.types.StructType valueSchema = new org.apache.spark.sql.types.StructType().add("sum",
org.apache.spark.sql.types.DataTypes.LongType);
> /* 064 */     private Object emptyVBase;
> ...
> org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 62, Column
58: Invalid escape sequence
> 	at org.codehaus.janino.Scanner.scanLiteralCharacter(Scanner.java:918)
> 	at org.codehaus.janino.Scanner.produce(Scanner.java:604)
> 	at org.codehaus.janino.Parser.peekRead(Parser.java:3239)
> 	at org.codehaus.janino.Parser.parseArguments(Parser.java:3055)
> 	at org.codehaus.janino.Parser.parseSelector(Parser.java:2914)
> 	at org.codehaus.janino.Parser.parseUnaryExpression(Parser.java:2617)
> 	at org.codehaus.janino.Parser.parseMultiplicativeExpression(Parser.java:2573)
> 	at org.codehaus.janino.Parser.parseAdditiveExpression(Parser.java:2552)
> {code}
> In the codegend expression, the literal should use `\\` instead of `\`



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message