From issues-return-195715-archive-asf-public=cust-asf.ponee.io@spark.apache.org Sun Jul 8 21:26:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id B875A180630 for ; Sun, 8 Jul 2018 21:26:03 +0200 (CEST) Received: (qmail 32791 invoked by uid 500); 8 Jul 2018 19:26:02 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 32782 invoked by uid 99); 8 Jul 2018 19:26:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 Jul 2018 19:26:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 5B7251A0203 for ; Sun, 8 Jul 2018 19:26:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Zm9xKNowh5fT for ; Sun, 8 Jul 2018 19:26:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 8E0DD5F3AB for ; Sun, 8 Jul 2018 19:26:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id BF79EE0950 for ; Sun, 8 Jul 2018 19:26:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 23B5621EE8 for ; Sun, 8 Jul 2018 19:26:00 +0000 (UTC) Date: Sun, 8 Jul 2018 19:26:00 +0000 (UTC) From: "Maxim Gekk (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (SPARK-24761) Check modifiability of config parameters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Maxim Gekk created SPARK-24761: ---------------------------------- Summary: Check modifiability of config parameters Key: SPARK-24761 URL: https://issues.apache.org/jira/browse/SPARK-24761 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.3.1 Reporter: Maxim Gekk Our customers and support team continuously face to the situation when setting a config parameter via *spark.conf.set()* does not may any effects. It is not clear from parameter's name is it static parameter or one of the parameter that can be set at runtime for current session state. It would be useful to have a method of *RuntimeConfig* which could tell to an user - does the given parameter may effect on the current behavior if he/she change it in the spark-shell or running notebook. The method can have the following signature: {code:scala} def isModifiable(key: String): Boolean {code} Any config parameter can be checked by using the syntax like this: {code:scala} scala> spark.conf.isModifiable("spark.sql.sources.schemaStringLengthThreshold") res0: Boolean = false {code} or for Spark Core parameter: {code:scala} scala> spark.conf.isModifiable("spark.task.cpus") res1: Boolean = false {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org