spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nickpoorman <...@git.apache.org>
Subject [GitHub] spark pull request: [SPARK-5753] [SQL] add JDBCRDD support for pos...
Date Wed, 08 Apr 2015 15:56:34 GMT
Github user nickpoorman commented on the pull request:

    https://github.com/apache/spark/pull/4549#issuecomment-90958807
  
    Has anyone tried compiling in the uuid type? I added `uuid` like in https://github.com/apache/spark/pull/4549/files#diff-2cf03f5b0ccea54efaa82a2cbef4bd7cR76
    
    ``` scala
    def getCatalystType(sqlType: Int, typeName: String, size: Int, md: MetadataBuilder): DataType
= {
        if (sqlType == Types.BIT && typeName.equals("bit") && size != 1) {
          BinaryType
        } else if (sqlType == Types.OTHER && typeName.equals("cidr")) {
          StringType
        } else if (sqlType == Types.OTHER && typeName.equals("inet")) {
          StringType
        } else if (sqlType == Types.OTHER && typeName.equals("uuid")) {
          StringType
        } else null
      }
    ```
    
    As you can see, every `user` has the same value and it is most definitely not the uuid.
    
    ```
    user           rating item  
    uc9aoh6t6au6fg 1      100737
    uc9aoh6t6au6fg 3      4320  
    uc9aoh6t6au6fg 1      100121
    uc9aoh6t6au6fg 2      100001
    uc9aoh6t6au6fg 1      100252
    uc9aoh6t6au6fg 1      1409  
    uc9aoh6t6au6fg 4      3787  
    uc9aoh6t6au6fg 4      100099
    uc9aoh6t6au6fg 3      779   
    uc9aoh6t6au6fg 3      102629
    uc9aoh6t6au6fg 2      101434
    uc9aoh6t6au6fg 2      100000
    uc9aoh6t6au6fg 1      100167
    uc9aoh6t6au6fg 4      104575
    uc9aoh6t6au6fg 4      101362
    uc9aoh6t6au6fg 4      100667
    uc9aoh6t6au6fg 3      101914
    uc9aoh6t6au6fg 4      100310
    uc9aoh6t6au6fg 4      105099
    uc9aoh6t6au6fg 4      100002
    ```
    
    For my own sanity, I checked this straight from the driver and I did get the correct uuid:
    
    ```
    scala> val conn: Connection = DriverManager.getConnection(dbUri)
    conn: java.sql.Connection = org.postgresql.jdbc4.Jdbc4Connection@26d680f
    
    scala> val rs = conn.prepareStatement(s"SELECT * FROM $dbQuery").executeQuery()
    rs: java.sql.ResultSet = org.postgresql.jdbc4.Jdbc4ResultSet@3d3efc92
    
    scala> rs.next
    res1: Boolean = true
    
    scala> rs.getString("user")
    res2: String = 4fec2e8b-2d25-4ef5-a388-39800302da85
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message