madlib-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From njayaram2 <...@git.apache.org>
Subject [GitHub] madlib pull request #274: Handling special characters in MLP and Encode Cate...
Date Wed, 06 Jun 2018 17:26:23 GMT
Github user njayaram2 commented on a diff in the pull request:

    https://github.com/apache/madlib/pull/274#discussion_r193490031
  
    --- Diff: src/ports/postgres/modules/utilities/utilities.py_in ---
    @@ -296,11 +305,24 @@ def py_list_to_sql_string(array, array_type=None, long_format=None):
         if not array:
             return "'{{ }}'::{0}".format(array_type)
         else:
    -        array_str = "ARRAY[ {0} ]" if long_format else "'{{ {0} }}'"
    -        return (array_str + "::{1}").format(','.join(map(str, array)), array_type)
    +        # For non-character array types:
    +        if long_format:
    +            array_str = "ARRAY[ {0} ]";
    +            return (array_str + "::{1}").format(
    +                ','.join(map(str, array)), array_type)
    +        else:
    +        # For character array types, we have to deal with special characters in
    +        # elements an array, i.e, {'ele''one', "ele,two", "ele$three"} :
    +        # We firstly use ",,," to join elements in python list and then call
    +        # postgres string_to_array with a delimiter ",,," to form the final
    +        # psql array, because this sequence of characters will be very
    +        # unlikely to show up in a real world use case and some special
    +        # case (such as "M,M") will be handled.
    +            array_str = "string_to_array($${0}$$, ',,,')"
    +            return (array_str + "::{1}").format(
    +                ',,,'.join(map(str, array)), array_type)
    --- End diff --
    
    Can we have a more obscure delimiter? As mentioned in a comment above, any string that
ends with a `,` will break this code.
    Just a thought, but would something like `,:;^` be an acceptable delimiter?


---

Mime
View raw message