Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: error (nike.apache.org: local policy)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
User-Agent: Workspace Webmail 5.6.40
Message-Id: 
 <20130725121522.7098ff60d8000ac98b0356cd27c73871.a395c3ac19.wbe@email04.secureserver.net>
From: "Chris Folsom" <jcfolsom@pureperfect.com>
To: user@hadoop.apache.org, "user@pig.apache.org" <user@pig.apache.org>
Subject: RE: Is it safe to have static methods in Hadoop Framework
Date: Thu, 25 Jul 2013 12:15:22 -0700
Mime-Version: 1.0

=0A=0AThe keyword "static" in java means that a single instance of it will=
=0Aexist for a given class loader. Two different class loaders will have=0A=
different values for a static variable even within the same JVM running=0Ao=
n the same host. =0A=0ASynchronization in Java works based on locks. In the=
 case of=0Asynchronized keyword applied to static methods, the lock would b=
e the=0Aclass. Same rules apply across multiple class loaders as above.=0A=
=0AThe only time you would need to synchronize something is if it contains=
=0Ashared state and it must be updated in an atomic manner. This isn't=0Ago=
ing to work in any parallel process unless you first have a shared=0Adata s=
tructure. Static only guarantees that it will be shared within the=0Asame c=
lass loader (again see above).=0A=0AA static method is fine if there is no =
shared state (i.e. if it's just a=0Afunction that takes parameters and retu=
rns a value). If you need to=0Ashare state, I would look at writing to HDFS=
 or using an ACID compliant=0Adata store with transaction semantics (e.g. a=
 relational database).=0A=0AYou might want to check out this:=0A=0Ahttps://=
en.wikipedia.org/wiki/Functional_programming=0A=0AI would try to avoid shar=
ed state unless it's absolutely necessary.=0A=0A=0A-------- Original Messag=
e --------=0ASubject: Is it safe to have static methods in Hadoop Framework=
=0AFrom: Huy Pham <phamvh@yahoo-inc.com>=0ADate: Thu, July 25, 2013 2:46 pm=
=0ATo: "user@hadoop.apache.org" <user@hadoop.apache.org>, =0A"user@pig.apac=
he.org" <user@pig.apache.org>=0A=0A Hi All,=0A   I am writing a class (call=
ed Parser) with a couple of static=0Afunctions because I don't want million=
s of instances of this class to be=0Acreated during the run.=0A   However, =
I realized that Hadoop will eventually produce parallel=0Ajobs, and if all =
jobs will call static functions of this Parser class,=0Awould that be safe?=
 =0A   In other words, will all hadoop jobs share the same class Parser or=
=0Awill each of them have their own Parser? In the former case, if all jobs=
=0Ashare the same class, then if I make the methods synchronized, then the=
=0Ajobs would need to wait until the locks to the functions are released,=
=0Athus that would affect the performance. However, in later case, that=0Aw=
ould not cause any problem.=0ACan someone provide some insights?=0AThanks=
=0AHuy