From issues-return-200843-archive-asf-public=cust-asf.ponee.io@flink.apache.org Fri Nov 9 14:47:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 7B673180627 for ; Fri, 9 Nov 2018 14:47:03 +0100 (CET) Received: (qmail 37465 invoked by uid 500); 9 Nov 2018 13:47:02 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 37456 invoked by uid 99); 9 Nov 2018 13:47:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Nov 2018 13:47:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 25F9819206A for ; Fri, 9 Nov 2018 13:47:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id n9Saq4G-MSSI for ; Fri, 9 Nov 2018 13:47:00 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id A22C162486 for ; Fri, 9 Nov 2018 13:47:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 48C76E015E for ; Fri, 9 Nov 2018 13:47:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 17E6F266D5 for ; Fri, 9 Nov 2018 13:47:00 +0000 (UTC) Date: Fri, 9 Nov 2018 13:47:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (FLINK-10827) Add test for duplicate() to SerializerTestBase MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/FLINK-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681445#comment-16681445 ] ASF GitHub Bot commented on FLINK-10827: ---------------------------------------- StefanRRichter commented on a change in pull request #7061: [FLINK-10827][tests] Add test for duplicate() to SerializerTestBase URL: https://github.com/apache/flink/pull/7061#discussion_r232249327 ########## File path: flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializer.java ########## @@ -181,11 +181,13 @@ public boolean isImmutableType() { } } - if (stateful) { - return new PojoSerializer(clazz, duplicateFieldSerializers, fields, executionConfig); - } else { - return this; + if (!stateful) { + // as a small memory optimization, we can share the same object between instances + duplicateFieldSerializers = fieldSerializers; } + + // we must create a new instance, otherwise the subclassSerializerCache can create concurrency problems Review comment: The point is that the Kryo serializer is not coming at all from the `fieldSerializers` that we duplicate. It comes from `subclassSerializerCache` and there can be serializers as well. But we need a new instance because it would be potentially wrong to share a cache. EDIT: And adding to this, the `fieldSerializers` in this test are actually for int and String, so it will hit the branch the returned `this`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org > Add test for duplicate() to SerializerTestBase > ---------------------------------------------- > > Key: FLINK-10827 > URL: https://issues.apache.org/jira/browse/FLINK-10827 > Project: Flink > Issue Type: Test > Components: Tests > Reporter: Stefan Richter > Assignee: Stefan Richter > Priority: Major > Labels: pull-request-available > Fix For: 1.7.0 > > > In the past, we had many bugs from type serializers that have not properly implemented the {{duplicate()}} method in a proper way. A very common error is to forget about creating a deep copy of some fields that can lead to concurrency problems in the backend. > We should add a test case for that tests duplicated serializer from different threads to expose concurrency problems. -- This message was sent by Atlassian JIRA (v7.6.3#76005)