hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Finkelshteyn <iefin...@gmail.com>
Subject Re: Netstat Shows Port 8020 Doesn't Seem to Listen
Date Mon, 09 Jan 2012 23:11:58 GMT
OK, not sure what I did (restarting the firewall, perhaps?), but I now 
have ports 8020 and 8021 listening and no more errors in my logs. Wooo! 
Only problem is I still can't get any hadoop stuff to work from a remote 
client:

hadoop fs -ls /
2012-01-09 17:53:53.559 java[13396:1903] Unable to load realm info from 
SCDynamicStore
12/01/09 17:53:55 INFO ipc.Client: Retrying connect to server: 
*my_server/my_ip*:8020. Already tried 0 time(s).
12/01/09 17:53:56 INFO ipc.Client: Retrying connect to server: 
*my_server/my_ip*:8020. Already tried 1 time(s).
12/01/09 17:53:57 INFO ipc.Client: Retrying connect to server: 
*my_server/my_ip*:8020. Already tried 2 time(s).
...

I feel like I'm almost there. Might this have to do with the fact that 
core-site.xml and mapred-site.xml specify localhost for ports 8020 and 
8021 (thus not listening to any attempted outside connections?)

Thanks for all the help so far, everyone!

Eli

On 1/9/12 5:43 PM, alo.alt wrote:
> Firewall online?
> and be sure that in /etc/hosts ONLY 127.0.0.1 is linked to localhost. Nothing like YOURHOSTNAME.YOURDOMAIN
(Redhat kudzu bug)
>
> - Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Jan 9, 2012, at 2:39 PM, Eli Finkelshteyn wrote:
>
>> Good call! netstat -anl gives me:
>> tcp        0      0 ::ffff:127.0.0.1:8020       :::*                        LISTEN
>>
>> Now it just looks like nothing is running on 8021. And now I'm really confused about
why I get no communication over 8020 from the datanode.
>>
>> Just to reiterate, this definitely is not the firewall, running iptables -nvL gives:
>>
>> ...
>>     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   
       state NEW tcp dpt:50070
>>     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   
       state NEW tcp dpt:50030
>>     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   
       state NEW tcp dpt:8021
>>     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   
       state NEW tcp dpt:8020
>> ...
>>
>> On 1/9/12 5:08 PM, alo.alt wrote:
>>> What happen when you try a "telnet localhost 8020"?
>>> netstat -anl would also useful.
>>>
>>> best,
>>>   Alex
>>>
>>> --
>>> Alexander Lorenz
>>> http://mapredit.blogspot.com
>>>
>>> On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:
>>>
>>>> A bit more info:
>>>>
>>>> When I start up only the namenode by itself, I'm not seeing any errors, but
what I am seeing that's really odd is:
>>>>
>>>>    2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>>>>    Socket Reader #1 for port 8020
>>>>    2012-01-09 16:48:45,531 INFO
>>>>    org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>>>    with hostName=NameNode, port=8020
>>>>    2012-01-09 16:48:45,532 INFO
>>>>    org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>>>>    Metrics with hostName=NameNode, port=8020
>>>>    2012-01-09 16:48:45,541 INFO
>>>>    org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>>>    localhost.localdomain/127.0.0.1:8020
>>>>
>>>> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.
 To me, that makes absolutely no sense. I feel like I should be getting an error telling me
Namenode did not in fact go up on 8020, but I'm not getting that at all.
>>>>
>>>> Eli
>>>>
>>>> On 1/9/12 3:22 PM, Idris Ali wrote:
>>>>> Hi,
>>>>>
>>>>> Looks like problem in starting DFS and MR, can you run 'jps' and see
if NN,
>>>>> DN, SNN, JT and TT are running,
>>>>>
>>>>> also make sure for pseudo-distributed mode, the following entries are
>>>>> present:
>>>>>
>>>>> 1. In core-site.xml
>>>>>   <property>
>>>>>      <name>fs.default.name</name>
>>>>>      <value>hdfs://localhost:8020</value>
>>>>>    </property>
>>>>>
>>>>>    <property>
>>>>>       <name>hadoop.tmp.dir</name>
>>>>>       <value><SOME TMP dir with Read/Write acces not system
temp></value>
>>>>>    </property>
>>>>>    <property>
>>>>>
>>>>> 2.  In hdfs-site.xml
>>>>> <property>
>>>>>      <name>dfs.replication</name>
>>>>>      <value>1</value>
>>>>>    </property>
>>>>>    <property>
>>>>>       <name>dfs.permissions</name>
>>>>>       <value>false</value>
>>>>>    </property>
>>>>>    <property>
>>>>>       <!-- specify this so that running 'hadoop namenode -format'
formats
>>>>> the right dir -->
>>>>>       <name>dfs.name.dir</name>
>>>>>       <value>Local dir with Read/Write access</value>
>>>>>    </property>
>>>>>
>>>>> 3. In mapred-stie.xml
>>>>>    <property>
>>>>>      <name>mapred.job.tracker</name>
>>>>>      <value>localhost:8021</value>
>>>>>    </property>
>>>>>
>>>>> Thanks,
>>>>> -Idris
>>>>>
>>>>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<iefinkel@gmail.com>wrote:
>>>>>
>>>>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>>>>> Even if the firewall was the problem, that should still give me output
that
>>>>>> the port is listening, but I'd just be unable to hit it from an outside
box
>>>>>> (I tested this by blocking port 50070, at which point it still showed
up in
>>>>>> netstat -a, but was inaccessible through http from a remote machine).
This
>>>>>> problem is something else.
>>>>>>
>>>>>>
>>>>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>>>>>
>>>>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<iefinkel@gmail.**com<iefinkel@gmail.com>>
>>>>>>>   wrote:
>>>>>>>
>>>>>>>> More info:
>>>>>>>>
>>>>>>>> In the DataNode log, I'm also seeing:
>>>>>>>>
>>>>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client:
Retrying
>>>>>>>> connect
>>>>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>>>>>
>>>>>>>> Why would things just not load on port 8020? I feel like
all the errors
>>>>>>>> I'm
>>>>>>>> seeing are caused by this, but I can't see any errors about
why this
>>>>>>>> occurred in the first place.
>>>>>>>>
>>>>>>>>   are you sure there isn't a firewall in place blocking port
8020?
>>>>>>> e.g. iptables on the local machines?   if you do
>>>>>>> telnet localhost 8020
>>>>>>> do you make a connection? if you use lsof and/or netstat can
you see
>>>>>>> the port open?
>>>>>>> if you have root access you can try turning off the firewall
with
>>>>>>> iptables -F to see if things work without firewall rules.
>>>>>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message