hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Mollitor (Jira)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-23096) Review Code Path for getResults
Date Mon, 30 Mar 2020 13:34:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

David Mollitor updated HIVE-23096:
----------------------------------
    Description: 

* Log when a buffer returned to a client is completely full.  This can be helpful for identifying
misconfigured clients that have a fetchSize that is too small
* See below code snippet
* Rows are loaded into an ArrayList called "convey."  This ArrayList is instantiated with
the default size (10) and is expanded on-demand as it is filled.  I changed it so that this
ArrayList's size is set once, on the first call to fetch, so that it doesn't have to "grow"
and waste cycles and memory with GC thrashing for that.
* Some minor cleanup

{code:java|Driver.java}
 try {
        ss = Utilities.readColumn(driverContext.getResStream(), bos);
        if (bos.getLength() > 0) {
          // use JDK StandardCharsets
          row = new String(bos.getData(), 0, bos.getLength(), "UTF-8");
        } else if (ss == Utilities.StreamStatus.TERMINATED) {
          // Do not create a new string for this.  Just use a hard-coded empty string.  JDK:
"Note that use of this constructor is unnecessary since Strings are immutable."
          row = new String();
        }
        ...
}
{code}

  was:See if there's an easy and useful way of detecting when user's are using a small fetch
size inadvertently.  For example, a JDBC fetchSize of 1, for a result set of 1,000,000 rows
across a high-latency link may crush performance.


> Review Code Path for getResults
> -------------------------------
>
>                 Key: HIVE-23096
>                 URL: https://issues.apache.org/jira/browse/HIVE-23096
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: David Mollitor
>            Assignee: David Mollitor
>            Priority: Minor
>         Attachments: HIVE-23096.1.patch
>
>
> * Log when a buffer returned to a client is completely full.  This can be helpful for
identifying misconfigured clients that have a fetchSize that is too small
> * See below code snippet
> * Rows are loaded into an ArrayList called "convey."  This ArrayList is instantiated
with the default size (10) and is expanded on-demand as it is filled.  I changed it so that
this ArrayList's size is set once, on the first call to fetch, so that it doesn't have to
"grow" and waste cycles and memory with GC thrashing for that.
> * Some minor cleanup
> {code:java|Driver.java}
>  try {
>         ss = Utilities.readColumn(driverContext.getResStream(), bos);
>         if (bos.getLength() > 0) {
>           // use JDK StandardCharsets
>           row = new String(bos.getData(), 0, bos.getLength(), "UTF-8");
>         } else if (ss == Utilities.StreamStatus.TERMINATED) {
>           // Do not create a new string for this.  Just use a hard-coded empty string.
 JDK: "Note that use of this constructor is unnecessary since Strings are immutable."
>           row = new String();
>         }
>         ...
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message