From dev-return-59655-archive-asf-public=cust-asf.ponee.io@thrift.apache.org Thu Dec 10 15:28:32 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id DAA9418062B for ; Thu, 10 Dec 2020 16:28:32 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 6DD9448DC8 for ; Thu, 10 Dec 2020 15:28:01 +0000 (UTC) Received: (qmail 4954 invoked by uid 500); 10 Dec 2020 15:28:01 -0000 Mailing-List: contact dev-help@thrift.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@thrift.apache.org Delivered-To: mailing list dev@thrift.apache.org Received: (qmail 4926 invoked by uid 99); 10 Dec 2020 15:28:01 -0000 Received: from ec2-52-204-25-47.compute-1.amazonaws.com (HELO mailrelay1-ec2-va.apache.org) (52.204.25.47) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Dec 2020 15:28:01 +0000 Received: from jira2-he-de.apache.org (jira2-he-de.apache.org [168.119.33.54]) by mailrelay1-ec2-va.apache.org (ASF Mail Server at mailrelay1-ec2-va.apache.org) with ESMTPS id CB8C83EB01 for ; Thu, 10 Dec 2020 15:28:00 +0000 (UTC) Received: from jira2-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira2-he-de.apache.org (ASF Mail Server at jira2-he-de.apache.org) with ESMTP id 26B68C800E7 for ; Thu, 10 Dec 2020 15:28:00 +0000 (UTC) Date: Thu, 10 Dec 2020 15:28:00 +0000 (UTC) From: =?utf-8?Q?Juraci_Paix=C3=A3o_Kr=C3=B6hling_=28Jira=29?= To: dev@thrift.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (THRIFT-5322) Go compact_protocol allocating unreasonable buffer size MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/THRIFT-5322?page=3Dcom.atlassi= an.jira.plugin.system.issuetabpanels:all-tabpanel ] Juraci Paix=C3=A3o Kr=C3=B6hling updated THRIFT-5322: ------------------------------------------- Attachment: main.go > Go compact_protocol allocating unreasonable buffer size > ------------------------------------------------------- > > Key: THRIFT-5322 > URL: https://issues.apache.org/jira/browse/THRIFT-5322 > Project: Thrift > Issue Type: Bug > Components: Go - Library > Affects Versions: 0.13.0 > Reporter: Juraci Paix=C3=A3o Kr=C3=B6hling > Priority: Major > Attachments: main.go > > > I don't yet know all the pieces to this puzzle, and it's quite possible t= hat the problem is on our side, but we use the Thrift Go library in the Jae= ger Agent and we are seeing a case where the memory consumption for a paylo= ad of 4k bytes to result in a buffer allocation in the compact_protocol.go = with unreasonable sizes. I found buffers of 1.4GiB while debugging the issu= e. > =C2=A0 > This is the code that we are seeing this memory usage: > [https://github.com/apache/thrift/blob/b75e88a33d67ae05ef9b5fa001d2a63a2e= ffe377/lib/go/thrift/compact_protocol.go#L556-L577] > =C2=A0 > Here's more information about this, including a reproducer and initial di= agnostics: > [https://github.com/jaegertracing/jaeger/issues/2638#issuecomment-7418482= 01] > =C2=A0 > As mentioned above, I'm still getting all the pieces together, but perhap= s you've seen this before or know what might be going on. What I know for s= ure at the moment is that this happens on malformed payloads, but I would e= xpect the library to have an upper limit on the buffer size. -- This message was sent by Atlassian Jira (v8.3.4#803005)