impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Robinson (JIRA)" <>
Subject [jira] [Created] (IMPALA-5441) Send larger row batches over the wire
Date Tue, 06 Jun 2017 00:07:12 GMT
Henry Robinson created IMPALA-5441:

             Summary: Send larger row batches over the wire
                 Key: IMPALA-5441
             Project: IMPALA
          Issue Type: Improvement
    Affects Versions: Impala 2.10.0
            Reporter: Henry Robinson

Our on-the-wire row batch size is the same as the in-memory size (1024 rows by default). It
might make sense to increase the wire-size to reduce the RPC-per-row overhead, and decrease
context-switching in the receiver. 

KRPC makes it quite natural to do that: each row batch can be serialized as a sidecar in memory-size
batches. The receiver can then read each batch in turn as though it were sent individually,
without any need to stitch together (or split up) serialized batches. 

This message was sent by Atlassian JIRA

View raw message