db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DERBY-5236) Client driver silently truncates strings that exceed 32KB
Date Fri, 26 Aug 2011 12:06:29 GMT

     [ https://issues.apache.org/jira/browse/DERBY-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Knut Anders Hatlen updated DERBY-5236:

    Attachment: d5236-1a-client-fetch-complete.diff

The problem with strings that take 65510 bytes is indeed related to the string being split
in three parts on transfer from the server. The client already knows that values can be split,
and handles that by sending a CNTQRY command to the server when it detects that it needs more
data, but it implicitly assumes that each value is only split once. That assumption is valid
currently, since we never send strings long enough to span three blocks. This changed with
the server-side fix to stop truncating the strings, and the client stops sending CNTQRY commands
too early, and there's not enough data in the receive buffer to construct the string when
ResultSet.getString() is called.

Just changing an "if" statement to a "while" statement in NetCursor.skipFdocaBytes() appears
to be enough to get the client to send CNTQRY commands until the entire value has been retreived.
This pattern is repeated in many methods in NetCursor. I think only skipFdocaBytes() will
ever need to send more than one CNTQRY, even when the truncation is fixed on the server, but
for completeness and consistency I factored out the common code into a helper method so that
all of them now call CNTQRY in a loop. See the attached patch d5236-1a-client-fetch-complete.diff.

With the d5236-1a patch in combination with the write-full-string.diff patch, strings up to
65535 (2^16-1) bytes can be fetched. Larger strings will still cause problems because they
overflow the two-byte length field.

> Client driver silently truncates strings that exceed 32KB
> ---------------------------------------------------------
>                 Key: DERBY-5236
>                 URL: https://issues.apache.org/jira/browse/DERBY-5236
>             Project: Derby
>          Issue Type: Bug
>          Components: Network Client
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>         Attachments: d5236-1a-client-fetch-complete.diff, repro.diff, write-full-string.diff
> Can be seen with this JUnit test case that retrieves a VARCHAR value with 20000 characters.
With the client driver, the string is truncated to 10900 characters (32700 bytes when encoded
in UTF-8).
>     public void testLongColumn() throws SQLException {
>         PreparedStatement ps = prepareStatement(
>                 "values cast(? as varchar(20000))");
>         char[] chars = new char[20000];
>         Arrays.fill(chars, '\u4e10');
>         String str = new String(chars);
>         ps.setString(1, str);
>         JDBC.assertSingleValueResultSet(ps.executeQuery(), str);
>     }

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message