Jens, can i set a smaller value in my = application?

Is this valid?

conf.setInt("mapred= .max.split.size", 50);

This is our mapred-site.xml:=

<?xml version=3D"1.0" encoding=3D"UTF-8"?>


<configuration>
  <property>
    <name>mapre=
d.job.tracker</name>
    <value>ip-1=
0-10-100170.eu-east-1.compute.internal:8021</value>
  </property>
  <property>
    <name>mapre=
d.job.tracker.http.address</<=
span style=3D"color:rgb(95,80,53)">name>
    <value>0.0.0.0:50030</value>
  </property>
  <property>
    <name>mapre=
duce.job.counters.max</name>
    <value>120<=
span style=3D"color:rgb(166,87,0)"></value>
  </property>
  <property>
    <name>mapre=
d.output.compress</name&g=
t;
    <value>fals=
e</value>
  </property>
  <property>
    <name>mapre=
d.output.compression.type</name>
    <value>BLOC=
K</value>
  </property>
  <property>
    <name>mapre=
d.output.compression.codec</<=
span style=3D"color:rgb(95,80,53)">name>
    <value>org.=
apache.hadoop.io.compress.DefaultCodec&=
lt;/value>
  </property>
  <property>
    <name>mapre=
d.map.output.compression.codec</name>
    <value>org.=
apache.hadoop.io.compress.SnappyCodec&l=
t;/value>
  </property>
  <property>
    <name>mapre=
d.compress.map.output</name>
    <value>true=
</value>
  </property>
  <property>
    <name>zlib.=
compress.level</name>=
;
    <value>DEFA=
ULT_COMPRESSION</value&g=
t;
  </property>
  <property>
    <name>io.so=
rt.factor</name>
    <value>64</value>
  </property>
  <property>
    <name>io.so=
rt.record.percent</name&g=
t;
    <value>0.05=
</value>
  </property>
  <property>
    <name>io.so=
rt.spill.percent</name>=
;
    <value>0.8<=
span style=3D"color:rgb(166,87,0)"></value>
  </property>
  <property>
    <name>mapre=
d.reduce.parallel.copies</name>
    <value>10</value>
  </property>
  <property>
    <name>mapre=
d.submit.replication</name>
    <value>2</value>
  </property>
  <property>
    <name>mapre=
d.reduce.tasks</name>=
;
    <value>6</value>
  </property>
  <property>
    <name>mapre=
d.userlog.retain.hours</name>
    <value>24</value>
  </property>
  <property>
    <name>io.so=
rt.mb</name>
    <value>112<=
span style=3D"color:rgb(166,87,0)"></value>
  </property>
  <property>
    <name>mapre=
d.child.java.opts</name&g=
t;
    <value> -Xm=
x471075479</value>
  </property>
  <property>
    <name>mapre=
d.job.reuse.jvm.num.tasks</name>
    <value>1</value>
  </property>
  <property>
    <name>mapre=
d.map.tasks.speculative.execution</<=
/span>name>
    <value>fals=
e</value>
  </property>
  <property>
    <name>mapre=
d.reduce.tasks.speculative.execution<=
;/name>
    <value>fals=
e</value>
  </property>
  <property>
    <name>mapre=
d.reduce.slowstart.completed.maps</<=
/span>name>
    <value>0.8<=
span style=3D"color:rgb(166,87,0)"></value>
  </property>
</configuration>

Suggest ways to overwrite the default value please.<= br>

On Mon,= Aug 26, 2013 at 9:38 AM, anil gupta <anilgupta84@gmail.com> wrote:

Hi Pavan,

=
Standalone cluster? How many RS you are running?What are you trying t= o achieve in MR? Have you tried increasing scanner caching?
Slow is very theoretical unless we know some more details of your stuff.
~Anil

On Sun, Aug 25, 2013 at 5= :52 PM, =C0=EE=BA=E9=D6=D2 <lhztop@hotmail.com> wrote:

You need release your map code here to analy= ze the question. generally, when map/reduce hbase, scanner with filte= r(s) is used. so the mapper count is the hbase region count in your hbase t= able.
As the reason why you reduce so slow, I guess, you have an disaster join on= the three tables, which cause too many rows.

=D3=DA 2013/8/26 4:36, Pavan Sudheendra =D0=B4=B5=C0:

Another Question, why does it indicate number of mappers as 1? Can i change= it so that multiple mappers perform computation?

--
Thanks & Regards,
An= il Gupta