Return-Path: X-Original-To: apmail-storm-user-archive@minotaur.apache.org Delivered-To: apmail-storm-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 430CB10389 for ; Sat, 1 Feb 2014 04:37:59 +0000 (UTC) Received: (qmail 19388 invoked by uid 500); 1 Feb 2014 04:37:58 -0000 Delivered-To: apmail-storm-user-archive@storm.apache.org Received: (qmail 19067 invoked by uid 500); 1 Feb 2014 04:37:55 -0000 Mailing-List: contact user-help@storm.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.incubator.apache.org Delivered-To: mailing list user@storm.incubator.apache.org Received: (qmail 19059 invoked by uid 99); 1 Feb 2014 04:37:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Feb 2014 04:37:54 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ryanchan404@gmail.com designates 209.85.216.174 as permitted sender) Received: from [209.85.216.174] (HELO mail-qc0-f174.google.com) (209.85.216.174) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Feb 2014 04:37:49 +0000 Received: by mail-qc0-f174.google.com with SMTP id x13so8210286qcv.19 for ; Fri, 31 Jan 2014 20:37:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=/4WaMIW1iuRzn2VvIBB3Mb13OOpG1ADOvWiYDW/Xcd4=; b=ciDAYAm1joyTWpNbF57R28MVIDHDrUuFrkyMX7ifAemZdNrPDXBM2rJgwUGlDGy+U0 jYVc5iBwTl/jXMYxS4WZH9/uh/vcpa8Vv0rvEgO9dGRnxWdKrlgQcbGYihzoflfK3cED Vv7QTC4OIPyqj38tmW0DzsW2MnnPWd876uGjwiQCSJNmUHcUy5/1V3BS7SL9CNJCYhF7 9YOnNSnsGR5vneRy1diCgKqqjqMcnvSNkKDO8YxeJzvowtqJOE3j3Wyx4lDt+q5IOI8s T0npLQql3tajjTNxI6Bq0WSNqApqaUhTLQqSeoLhk28mUuwLMcYqQZogffRmALcYxtrV L9cQ== MIME-Version: 1.0 X-Received: by 10.224.124.133 with SMTP id u5mr37790708qar.79.1391229448920; Fri, 31 Jan 2014 20:37:28 -0800 (PST) Received: by 10.96.121.132 with HTTP; Fri, 31 Jan 2014 20:37:28 -0800 (PST) In-Reply-To: References: Date: Sat, 1 Feb 2014 12:37:28 +0800 Message-ID: Subject: Re: No way to specify config file (storm.yml) location From: Ryan Chan To: user@storm.incubator.apache.org Content-Type: multipart/alternative; boundary=001a11c29f8a1aa16604f150d9aa X-Virus-Checked: Checked by ClamAV on apache.org --001a11c29f8a1aa16604f150d9aa Content-Type: text/plain; charset=ISO-8859-1 Jeff, Thanks. This script is useful, please make it merge into the official release. Just a nitpick, the workers are still using the default logback configuration so they still run into default logs folder. On Mon, Jan 27, 2014 at 10:48 AM, Jeff Potter wrote: > > Hi Ryan -- > > We ended up patching both storm 0.8 and storm 0.9 for this -- here's the > patch I wrote for 0.9. (This has been on my to-do list to get to Nathan, so > thanks for the reminder.) > > This adds a "--conf-dir" option to ./bin/storm that you can then point at > a dir that contains storm.yaml; and also adds a --logback-file option that > you can then point at your log back file. > > In our setup, the storm src code is read-only to the runtime, and the dir > paths that are writeable (for logging, or for generating the storm.yaml > from a template) aren't known at install time. > > best, > Jeff > > > 25a26 > > LOGBACK_FILE = STORM_DIR + "/logback/cluster.xml" > 254c255 > < "-Dlogback.configurationFile=" + STORM_DIR + > "/logback/cluster.xml", > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 274c275 > < "-Dlogback.configurationFile=" + STORM_DIR + > "/logback/cluster.xml", > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 295c296 > < "-Dlogback.configurationFile=" + STORM_DIR + > "/logback/cluster.xml", > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 316c317 > < "-Dlogback.configurationFile=" + STORM_DIR + > "/logback/cluster.xml", > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 336c337 > < "-Dlogback.configurationFile=" + STORM_DIR + > "/logback/cluster.xml" > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 421a423,428 > > elif token == "--logback-file": > > global LOGBACK_FILE > > LOGBACK_FILE = curr.pop() > > elif token == "--conf-dir": > > global USER_CONF_DIR, CLUSTER_CONF_DIR > > USER_CONF_DIR = CLUSTER_CONF_DIR = curr.pop() > > > On Jan 25, 2014, at 11:31 AM, Ryan Chan wrote: > > There is an old post also talking about this and no useful reply (1+ year > ago): > https://groups.google.com/forum/#!msg/storm-user/51wualp1xUk/ywkWaMWiLCcJ > > It mentioned to use the following option if we want to launch the > supervisor, e.g. > > java -client -Dstorm.options= -Dstorm.conf.file=/app/shared/ > storm/storm.yaml supervisor > > However, this option require the storm.yaml must be in the classpath, > however, even I specified the classpath, I still not able to use that > config file, like the OP, so this issue is not doable right? > > > > > > > --001a11c29f8a1aa16604f150d9aa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Jeff,

Thanks. This script is useful, pl= ease make it merge into the official release.
Just a nitpick, the= workers are still using the default logback configuration so they still ru= n into default logs folder.




On Mon, Jan 27, 2014 at 10:48 AM, Jeff Potter <jpotter-storm@codepuppy.com> wrote:
Hi Ryan —

We ended up patching both storm 0= .8 and storm 0.9 for this — here’s the patch I wrote for 0.9. (= This has been on my to-do list to get to Nathan, so thanks for the reminder= .)

This adds a “--conf-dir” option to ./bin/st= orm that you can then point at a dir that contains storm.yaml; and also add= s a --logback-file option that you can then point at your log back file.

In our setup, the storm src code is read-only to the runtime= , and the dir paths that are writeable (for logging, or for generating the = storm.yaml from a template) aren’t known at install time.
<= br>
best,
Jeff


25a26
> LOGBACK_FILE =3D ST= ORM_DIR + "/logback/cluster.xml"
254c255
<     =     "-Dlogback.configurationFile=3D" + STORM_DIR + &quo= t;/logback/cluster.xml",
---
>     &nbs= p;   "-Dlogback.configurationFile=3D" + LOGBACK_FILE
274c275
<= ;         "-Dlogback.configurationFile=3D" + = STORM_DIR + "/logback/cluster.xml",
---
> &n= bsp;       "-Dlogback.configurationFile=3D" + LOGB= ACK_FILE
29= 5c296
<         "-Dlogback.configurationFile=3D"= + STORM_DIR + "/logback/cluster.xml",
---
>         "-Dlogback.configurationFile=3D"= + LOGBACK_FILE
316c317
<         "-Dlogback.configurationFile=3D&quo= t; + STORM_DIR + "/logback/cluster.xml",
---
>     &nbs= p;   "-Dlogback.configurationFile=3D" + LOGBACK_FILE
336c337
<= ;         "-Dlogback.configurationFile=3D" + = STORM_DIR + "/logback/cluster.xml"
---
> &n= bsp;       "-Dlogback.configurationFile=3D" + LOGB= ACK_FILE
42= 1a423,428
>     elif token =3D=3D "--logback-file":
>     &n= bsp; global LOGBACK_FILE
>       LOGBACK_FILE =3D curr.pop()
>   &nbs= p; elif token =3D=3D "--conf-dir":
>       global USER_CON= F_DIR, CLUSTER_CONF_DIR
>   &nbs= p;   USER_CONF_DIR =3D CLUSTER_CONF_DIR =3D curr.pop()


On Jan 25, 2014, at 11:31 AM, Ryan Chan <ryanchan404@gmail.com<= /a>> wrote:

There is= an old post also talking about this and no useful reply (1+ year ago):&nbs= p;https://groups.google.com/forum/#!msg/storm-= user/51wualp1xUk/ywkWaMWiLCcJ

It mentioned to use the following option if we want to launc= h the supervisor, e.g.

java -client -Dstorm.options=3D = -Dstorm.conf.file=3D/app/shared/storm/storm.yaml supervisor

However, this option require the storm.yaml must be in = the classpath, however, even I specified the classpath, I still not able to= use that config file, like the OP, so this issue is not doable right?
=






--001a11c29f8a1aa16604f150d9aa--