Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B296B2009DC for ; Tue, 2 May 2017 19:06:08 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B1268160BAB; Tue, 2 May 2017 17:06:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 02B9E160B9D for ; Tue, 2 May 2017 19:06:07 +0200 (CEST) Received: (qmail 69456 invoked by uid 500); 2 May 2017 17:06:07 -0000 Mailing-List: contact issues-help@systemml.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@systemml.incubator.apache.org Delivered-To: mailing list issues@systemml.incubator.apache.org Received: (qmail 69442 invoked by uid 99); 2 May 2017 17:06:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 May 2017 17:06:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C3A16C05B0 for ; Tue, 2 May 2017 17:06:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id ZhFFx6xSziGg for ; Tue, 2 May 2017 17:06:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 5DB525F5F9 for ; Tue, 2 May 2017 17:06:05 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id A4C1FE0045 for ; Tue, 2 May 2017 17:06:04 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 5E76221DE3 for ; Tue, 2 May 2017 17:06:04 +0000 (UTC) Date: Tue, 2 May 2017 17:06:04 +0000 (UTC) From: "Glenn Weidner (JIRA)" To: issues@systemml.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (SYSTEMML-1518) Corrupted input file names in old and new mlcontext apis MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 02 May 2017 17:06:08 -0000 [ https://issues.apache.org/jira/browse/SYSTEMML-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glenn Weidner updated SYSTEMML-1518: ------------------------------------ Fix Version/s: (was: SystemML 1.0) SystemML 0.14 > Corrupted input file names in old and new mlcontext apis > -------------------------------------------------------- > > Key: SYSTEMML-1518 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1518 > Project: SystemML > Issue Type: Bug > Reporter: Matthias Boehm > Assignee: Matthias Boehm > Priority: Blocker > Fix For: SystemML 0.14 > > > Both the new and old mlcontext APIs call {{OptimizerUtils.getUniqueTempFileName()}} to create HDFS filenames for registered input frames or matrices. This call simply forwards the request to {{Dag}} for consistency with hdfs filenames of intermediates and to ensure isolation with regard to concurrently running scripts (from different client processes on a shared cluster). > However, for this code path the internal scratch space configuration is always uninitialized leading to corrupt filenames such as {{/_p1234_1.2.345.678//_t0/temp1_0}}. The missing scratch_space prefix is problematic because the remainder is interpreted as an absolute file path, often leading to permission issues because typical users are not granted write access on HFDS root. > Note that this issue might not be immediately visible in all scenarios because it only affects input variables that are exported to HDFS (e.g., during guarded collect or as specific inputs to remote parfor). -- This message was sent by Atlassian JIRA (v6.3.15#6346)