accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: Getting the IP Address
Date Wed, 28 Aug 2013 21:00:24 GMT
On Wed, Aug 28, 2013 at 4:44 PM, Adam Fuchs <afuchs@apache.org> wrote:

> Seems like a question a common and complex as which IP address to listen
> on would have a fair amount of precedent in open-source projects that we
> could pull from. Are we reinventing the wheel? Does anyone have an example
> of an application like ours with the same set of supported platforms that
> has already solved this problem and whose solution you like? Are there
> elements of what we do that make us better/worse/different that something
> like the scripting and networking code built for HBase or HDFS?
>

Maybe pull this out of the scripts altogether and make it an
accumulo-site.xml setting like dfs.datanode.dns.interface.

I was googling a bit.  Tomcat allows a specific IP addr to be specified.
 This will not work for our case because it does not work well with using
the same config file across a cluster.   Specifiying an interface like
hadoop does in the config and grabbing the IP from that seems like a good
way to go.


>
> Adam
>
>
>
> On Wed, Aug 28, 2013 at 1:35 PM, Keith Turner <keith@deenlo.com> wrote:
>
>>
>>
>>
>> On Wed, Aug 28, 2013 at 4:26 PM, Christopher <ctubbsii@apache.org> wrote:
>>
>>> Ah, you're right, of course.
>>>
>>> In that case, I'm also wondering about NAT situations and other
>>> strange networking situations. For those especially, it seems what we
>>> need to do is treat the bind address differently from the advertised
>>> address.
>>>
>>> Perhaps attempting to use $(hostname -i) and falling back to
>>> $(hostname -I | head -1) would be best?
>>>
>>
>> I just noticed one wrinkle with "hostname -I",  it may return IPV6
>> addresses.   When I first looked at the man page, I thought it would
>> exclude IPV6.  But on closes inspection I noticed it excludes "IPv6
>> link-local addresses".  So hostname -I will probably cause problems if the
>> first thing it returns is a IPV6 addr.
>>
>>
>>>
>>> --
>>> Christopher L Tubbs II
>>> http://gravatar.com/ctubbsii
>>>
>>>
>>> On Wed, Aug 28, 2013 at 3:03 PM, John Vines <vines@apache.org> wrote:
>>> > Christopher,
>>> >
>>> > It's not a matter of determining which port to bind to. It's for
>>> recording
>>> > it's location in zookeeper so other nodes can find it.
>>> >
>>> >
>>> > On Wed, Aug 28, 2013 at 3:00 PM, Christopher <ctubbsii@apache.org>
>>> wrote:
>>> >>
>>> >> I'm not sure this is even very portable. It relies on a specific
>>> >> ifconfig display format intended for human-readability, and I'm not
>>> >> sure that's entirely guaranteed to be static over time. It also won't
>>> >> work if there are multiple public interfaces. It also don't think it
>>> >> works for infiniband or other interface types that have issues in
>>> >> ifconfig.
>>> >>
>>> >> I think we have to make *some* assumptions that things like
>>> >> "networking" is properly configured using standard utilities for
>>> >> name-mapping (like DNS or /etc/hosts). I think it's more confusing for
>>> >> sysadmins if we have these sorts of automatic behaviors that are
>>> >> non-standard and unexpected (like automatically binding to a single,
>>> >> arbitrarily chosen, public IP out of the box).
>>> >>
>>> >> Honestly, though, I'm not sure why we need to be resolving public IP
>>> >> addresses *at all*. It should be configured explicitly, and bind to
>>> >> either 127.0.0.1 or 0.0.0.0 by default (to satisfy the ease for
>>> >> first-time users).
>>> >>
>>> >>
>>> >> --
>>> >> Christopher L Tubbs II
>>> >> http://gravatar.com/ctubbsii
>>> >>
>>> >>
>>> >> On Wed, Aug 28, 2013 at 1:54 PM, John Vines <vines@apache.org>
wrote:
>>> >> > We use this similar logic throughout a lot of our scripts for
>>> >> > determining
>>> >> > the external facing IP address in a portable manner, it's just
that
>>> the
>>> >> > init.d scripts are a bit more strict about it. This is the
>>> equivalent of
>>> >> > using the name defined in the slaves/masters/tracers/etc. files
to
>>> >> > determine
>>> >> > which port to report as.
>>> >> >
>>> >> > Switching to a system that depends on DNS to succeed will fail
for
>>> all
>>> >> > first
>>> >> > time users, which is a penalty that will not be worth it. If
>>> someone can
>>> >> > find a better way to determine outward facing IP address I would
>>> love to
>>> >> > have it, but unfortunately networks are hard.
>>> >> >
>>> >> >
>>> >> > On Wed, Aug 28, 2013 at 1:44 PM, Billie Rinaldi
>>> >> > <billie.rinaldi@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Good point.  I don't care if the init.d scripts work on a Mac.
 I
>>> do
>>> >> >> care
>>> >> >> about the other scripts, though.
>>> >> >>
>>> >> >>
>>> >> >> On Wed, Aug 28, 2013 at 10:32 AM, Christopher <ctubbsii@apache.org
>>> >
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> But... it shouldn't be a supported platform for init scripts...
I
>>> >> >>> imagine.
>>> >> >>>
>>> >> >>> --
>>> >> >>> Christopher L Tubbs II
>>> >> >>> http://gravatar.com/ctubbsii
>>> >> >>>
>>> >> >>>
>>> >> >>> On Wed, Aug 28, 2013 at 1:03 PM, Billie Rinaldi
>>> >> >>> <billie.rinaldi@gmail.com> wrote:
>>> >> >>> > It's a supported development platform.  =)
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > On Wed, Aug 28, 2013 at 9:59 AM, Sean Busbey <
>>> busbey@cloudera.com>
>>> >> >>> > wrote:
>>> >> >>> >>
>>> >> >>> >> hostname -i does not work on a Mac ( 10.8.4 )
>>> >> >>> >>
>>> >> >>> >> Is Mac a supported platform?
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >> On Wed, Aug 28, 2013 at 11:53 AM, Eric Newton
>>> >> >>> >> <eric.newton@gmail.com>
>>> >> >>> >> wrote:
>>> >> >>> >>>
>>> >> >>> >>> Does "hostname -i" work on a mac?  Not being
a mac user, I
>>> can't
>>> >> >>> >>> check.
>>> >> >>> >>>
>>> >> >>> >>> -Eric
>>> >> >>> >>>
>>> >> >>> >>>
>>> >> >>> >>>
>>> >> >>> >>> On Wed, Aug 28, 2013 at 11:38 AM, Ravi Mutyala
>>> >> >>> >>> <ravi@hortonworks.com>
>>> >> >>> >>> wrote:
>>> >> >>> >>>>
>>> >> >>> >>>> Hi,
>>> >> >>> >>>>
>>> >> >>> >>>> I see from the accumulo-tracer init.d
script that IP is
>>> >> >>> >>>> determined
>>> >> >>> >>>> by
>>> >> >>> >>>> this logic.
>>> >> >>> >>>>
>>> >> >>> >>>> ifconfig | grep inet[^6] | awk '{print
$2}' | sed
>>> 's/addr://' |
>>> >> >>> >>>> grep
>>> >> >>> >>>> -v
>>> >> >>> >>>> 0.0.0.0 | grep -v 127.0.0.1 | head -n
1
>>> >> >>> >>>>
>>> >> >>> >>>>
>>> >> >>> >>>> Any reason for using this logic instead
of a hostname -i and
>>> >> >>> >>>> using
>>> >> >>> >>>> reverse dns lookup? I have a cluster where
the order of nics
>>> on
>>> >> >>> >>>> one
>>> >> >>> >>>> of the
>>> >> >>> >>>> nodes is in a different order and ifconfig
returns a IP from
>>> a
>>> >> >>> >>>> different
>>> >> >>> >>>> subnet than for other nodes. But DNS and
reverse DNS are
>>> properly
>>> >> >>> >>>> configured.
>>> >> >>> >>>>
>>> >> >>> >>>> Thanks
>>> >> >>> >>>>
>>> >> >>> >>>> CONFIDENTIALITY NOTICE
>>> >> >>> >>>> NOTICE: This message is intended for the
use of the
>>> individual or
>>> >> >>> >>>> entity
>>> >> >>> >>>> to which it is addressed and may contain
information that is
>>> >> >>> >>>> confidential,
>>> >> >>> >>>> privileged and exempt from disclosure
under applicable law.
>>> If
>>> >> >>> >>>> the
>>> >> >>> >>>> reader of
>>> >> >>> >>>> this message is not the intended recipient,
you are hereby
>>> >> >>> >>>> notified
>>> >> >>> >>>> that any
>>> >> >>> >>>> printing, copying, dissemination, distribution,
disclosure or
>>> >> >>> >>>> forwarding of
>>> >> >>> >>>> this communication is strictly prohibited.
If you have
>>> received
>>> >> >>> >>>> this
>>> >> >>> >>>> communication in error, please contact
the sender
>>> immediately and
>>> >> >>> >>>> delete it
>>> >> >>> >>>> from your system. Thank You.
>>> >> >>> >>>
>>> >> >>> >>>
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >> --
>>> >> >>> >> Sean
>>> >> >>> >
>>> >> >>> >
>>> >> >>
>>> >> >>
>>> >> >
>>> >
>>> >
>>>
>>
>>
>

Mime
View raw message