hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Pino <juancitomiguel...@gmail.com>
Subject Re: load properties from property file into Configuration object
Date Sat, 03 Aug 2013 13:02:43 GMT
Ok sorry, I think I got it: I would need to run my program like this:

hadoop jar MYJAR MYCLASS -D property1=PROPERTY1 -D property2=PROPERTY2 -conf
MYCONFIGFILE"

Juan


On Sat, Aug 3, 2013 at 1:46 PM, Juan Pino <juancitomiguelito@gmail.com>wrote:

> Thanks Harsh for your reply.
> Yes I think I am using the Tool framework ("public class HadoopJob extends
> Configured implements Tool" + main function is "System.exit(ToolRunner.run(new
> HadoopJob(), args));")
>
> "then you can pass -Dkey=val props directly into a config object at
> runtime"
> I'm a bit confused by this: I can pass -D key1=val1 -D key2=val2 etc. on
> the command line, and with the Tool framework, those properties get
> automatically loaded in the Configuration object, but if I have many of
> these properties, how can I write them into a config file so that these
> properties are loaded into the Configuration object ?
>
> Thanks very much,
>
> Juan
>
>
> On Sat, Aug 3, 2013 at 1:20 PM, Harsh J <harsh@cloudera.com> wrote:
>
>> This is a bit unnecessary to manually do. If you use the Tool
>> framework, then you can pass -Dkey=val props directly into a config
>> object at runtime. See
>> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/util/Tool.html
>> and
>> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/util/GenericOptionsParser.html
>>
>> On Sat, Aug 3, 2013 at 5:30 PM, Juan Pino <juancitomiguelito@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I am trying to simplify code that loads properties from a property file
>> into
>> > a Configuration object:
>> >
>> > Short excerpt of the code:
>> >
>> > String configFile = args[0];
>> > Properties p = new Properties();
>> > p.load(new FileInputStream(configFile));
>> > Configuration conf = getConf();
>> > for (String prop: p.stringPropertyNames()) {
>> >   conf.set(prop, p.getProperty(prop));
>> > }
>> >
>> > The context is that my program can be run with an optional argument
>> which is
>> > a config file as "hadoop jar MYJAR MYCLASS -D property1=PROPERTY1 -D
>> > property2=PROPERTY2 [MYCONFIGFILE]"
>> > The format of MYCONFIGFILE is
>> > "property3=PROPERTY3<NEWLINE>property4=PROPERTY4 etc."
>> >
>> > Looking at the Configuration api, I didn't see any method that directly
>> > loads properties from a property file.
>> > Looking at the Configuration code, Configuration looks like a wrapper
>> around
>> > Properties, so I thought there was a similar method to Properties.load
>> or a
>> > method that sets several properties at the same time.
>> > Is there a way to do this ?
>> >
>> > Thanks very much,
>> >
>> > Juan
>> >
>> > --
>> > In light of the recent NSA scandal, please consider encrypting your
>> reply by
>> > using my public key available at http://mi.eng.cam.ac.uk/~jmp84/pgp.txtIf
>> > you use webmail, you may consider mailvelope (http://mailvelope.com/)
>> which
>> > is a very easy to use plugin available for chrome and probably soon for
>> > firefox.
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> In light of the recent NSA scandal, please consider encrypting your reply
> by using my public key available at http://mi.eng.cam.ac.uk/~jmp84/pgp.txt If
> you use webmail, you may consider mailvelope (http://mailvelope.com/)
> which is a very easy to use plugin available for chrome and probably soon
> for firefox.
>



-- 
In light of the recent NSA scandal, please consider encrypting your reply
by using my public key available at http://mi.eng.cam.ac.uk/~jmp84/pgp.txt If
you use webmail, you may consider mailvelope (http://mailvelope.com/) which
is a very easy to use plugin available for chrome and probably soon for
firefox.

Mime
View raw message