Return-Path: 
 <user-return-2619-apmail-flink-user-archive=flink.apache.org@flink.apache.org>
X-Original-To: apmail-flink-user-archive@minotaur.apache.org
Delivered-To: apmail-flink-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 8A1F910817
	for <apmail-flink-user-archive@minotaur.apache.org>;
 Thu,  3 Sep 2015 07:57:52 +0000 (UTC)
Received: (qmail 76052 invoked by uid 500); 3 Sep 2015 07:57:52 -0000
Delivered-To: apmail-flink-user-archive@flink.apache.org
Received: (qmail 75972 invoked by uid 500); 3 Sep 2015 07:57:52 -0000
Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@flink.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@flink.apache.org>
List-Post: <mailto:user@flink.apache.org>
List-Id: <user.flink.apache.org>
Reply-To: user@flink.apache.org
Delivered-To: mailing list user@flink.apache.org
Received: (qmail 75962 invoked by uid 99); 3 Sep 2015 07:57:52 -0000
Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Sep 2015 07:57:52 +0000
Received: from localhost (localhost [127.0.0.1])
	by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org)
 with ESMTP id CC027C03F6
	for <user@flink.apache.org>; Thu,  3 Sep 2015 07:57:51 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org
X-Spam-Flag: NO
X-Spam-Score: -0.007
X-Spam-Level: 
X-Spam-Status: No, score=-0.007 tagged_above=-999 required=6.31
	tests=[RP_MATCHES_RCVD=-0.006, SPF_PASS=-0.001] autolearn=disabled
Received: from mx1-us-west.apache.org ([10.40.0.8])
	by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new,
 port 10024)
	with ESMTP id Lfj80cecaoM5 for <user@flink.apache.org>;
	Thu,  3 Sep 2015 07:57:49 +0000 (UTC)
Received: from ad-exchedge02.aau.dk (ad-exchedge04.aau.dk [130.225.194.129])
	by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with
 ESMTPS id 9CFAB20FD3
	for <user@flink.apache.org>; Thu,  3 Sep 2015 07:57:48 +0000 (UTC)
Received: from AD-EXCHHUB1-1.aau.dk (172.25.14.36) by ad-exchedge04.aau.dk
 (130.225.194.129) with Microsoft SMTP Server (TLS) id 14.3.210.2; Thu, 3 Sep
 2015 09:57:43 +0200
Received: from h299.aau1x-2.klient.slv.site.aau.dk (172.25.21.43) by
 smtp.aau.dk (172.25.14.35) with Microsoft SMTP Server (TLS) id 14.3.210.2;
 Thu, 3 Sep 2015 09:57:46 +0200
Message-ID: <55E7FD79.30000@cs.aau.dk>
Date: Thu, 3 Sep 2015 09:57:45 +0200
From: "Andres R. Masegosa " <andres@cs.aau.dk>
Reply-To: <user@flink.apache.org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10;
 rv:31.0) Gecko/20100101 Thunderbird/31.7.0
MIME-Version: 1.0
To: <user@flink.apache.org>
Subject: Re: Bug broadcasting objects (serialization issue)
References: <55E6BEBA.7030401@cs.aau.dk>
In-Reply-To: <55E6BEBA.7030401@cs.aau.dk>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
X-Originating-IP: [172.25.21.43]

Hi,

I get a new similar bug when broadcasting a list of integers if this
list is made unmodifiable,

        elements = Collections.unmodifiableList(elements);


I include this code to reproduce the result,


public class WordCountExample {
    public static void main(String[] args) throws Exception {
        final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();

    DataSet<String> text = env.fromElements(
                "Who's there?",
                "I think I hear them. Stand, ho! Who's there?");

        List<Integer> elements = new ArrayList<Integer>();
        elements.add(0);

        elements = Collections.unmodifiableList(elements);

        DataSet<TestClass> set = env.fromElements(new TestClass(elements));

        DataSet<Tuple2<String, Integer>> wordCounts = text
                .flatMap(new LineSplitter())
                .withBroadcastSet(set, "set")
                .groupBy(0)
                .sum(1);

        wordCounts.print();
    }

    public static class LineSplitter implements FlatMapFunction<String,
Tuple2<String, Integer>> {
        @Override
        public void flatMap(String line, Collector<Tuple2<String,
Integer>> out) {
            for (String word : line.split(" ")) {
                out.collect(new Tuple2<String, Integer>(word, 1));
            }
        }
    }

    public static class TestClass implements Serializable {
        private static final long serialVersionUID = -2932037991574118651L;

        List<Integer> integerList;
        public TestClass(List<Integer> integerList){
            this.integerList=integerList;
        }


    }
}

Thanks for your support,
Andres

On 2/9/15 11:17, Andres R. Masegosa  wrote:
> Hi,
> 
> I get a bug when trying to broadcast a list of integers created with the
> primitive "Arrays.asList(...)".
> 
> For example, if you try to run this "wordcount" example, you can
> reproduce the bug.
> 
> 
> public class WordCountExample {
>     public static void main(String[] args) throws Exception {
>         final ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> 
>     DataSet<String> text = env.fromElements(
>                 "Who's there?",
>                 "I think I hear them. Stand, ho! Who's there?");
> 
>         List<Integer> elements = Arrays.asList(0, 0, 0);
> 
>         DataSet<TestClass> set = env.fromElements(new TestClass(elements));
> 
>         DataSet<Tuple2<String, Integer>> wordCounts = text
>                 .flatMap(new LineSplitter())
>                 .withBroadcastSet(set, "set")
>                 .groupBy(0)
>                 .sum(1);
> 
>         wordCounts.print();
>     }
> 
>     public static class LineSplitter implements FlatMapFunction<String,
> Tuple2<String, Integer>> {
>         @Override
>         public void flatMap(String line, Collector<Tuple2<String,
> Integer>> out) {
>             for (String word : line.split(" ")) {
>                 out.collect(new Tuple2<String, Integer>(word, 1));
>             }
>         }
>     }
> 
>     public static class TestClass implements Serializable {
>         private static final long serialVersionUID = -2932037991574118651L;
> 
>         List<Integer> integerList;
>         public TestClass(List<Integer> integerList){
>             this.integerList=integerList;
>         }
> 
> 
>     }
> }
> 
> 
> However, if instead of using the primitive "Arrays.asList(...)", we use
> instead the ArrayList<> constructor, there is any problem!!!!
> 
> 
> Regards,
> Andres
>