Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 24648200D02 for ; Fri, 8 Sep 2017 21:32:05 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 22F101609A7; Fri, 8 Sep 2017 19:32:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 690111609BF for ; Fri, 8 Sep 2017 21:32:04 +0200 (CEST) Received: (qmail 64064 invoked by uid 500); 8 Sep 2017 19:32:03 -0000 Mailing-List: contact issues-help@systemml.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@systemml.apache.org Delivered-To: mailing list issues@systemml.apache.org Received: (qmail 64054 invoked by uid 99); 8 Sep 2017 19:32:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Sep 2017 19:32:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 01CF5190940 for ; Fri, 8 Sep 2017 19:32:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id A2j-7k_oW_3p for ; Fri, 8 Sep 2017 19:32:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 029275F5B3 for ; Fri, 8 Sep 2017 19:32:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 863C7E08F7 for ; Fri, 8 Sep 2017 19:32:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 45A252414F for ; Fri, 8 Sep 2017 19:32:01 +0000 (UTC) Date: Fri, 8 Sep 2017 19:32:01 +0000 (UTC) From: "Matthias Boehm (JIRA)" To: issues@systemml.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Closed] (SYSTEMML-1884) Unnecessary spark/mr instructions on removeEmpty select MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 08 Sep 2017 19:32:05 -0000 [ https://issues.apache.org/jira/browse/SYSTEMML-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Boehm closed SYSTEMML-1884. ------------------------------------ > Unnecessary spark/mr instructions on removeEmpty select > ------------------------------------------------------- > > Key: SYSTEMML-1884 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1884 > Project: SystemML > Issue Type: Bug > Affects Versions: SystemML 0.14 > Reporter: Matthias Boehm > Assignee: Matthias Boehm > Fix For: SystemML 0.15 > > > The builtin function {{removeEmpty}} takes an optional select parameter. If this parameter is provided, we directly use it instead of explicitly computing it. However, the compilation of removeEmpty handled this incorrectly, setting the forced execution type to the initially compiled execution type of removeEmpty. This is problematic in scenarios with dynamic recompilation. For example, consider the following scenario: > {code} > C = replace(target=A, pattern=2, replacement=0); > D = removeEmpty(B, margin="rows", select=C); > {code} > If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly set the forced execution type of {{replace}} to SPARK. During recompilation, the {{removeEmpty}} might be compiled to CP but because {{replace}} has a forced execution type, it will remain in SPARK. > Workaround: The workaround for this issue is to put a {{while(FALSE){}}} just before the {{removeEmpty}} call (more precisely, C in above example needs to be computed in a separate HOP DAG). -- This message was sent by Atlassian JIRA (v6.4.14#64029)