Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of bob.futrelle@gmail.com
 designates 209.85.215.54 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CANZa=Gv16c0cM3GYPY0_az3iTsQajhn62-zHsdcOCX3-EVNLdg@mail.gmail.com>
References: 
 <CAOsWdXqEVPduFcQZ9kqpMYB5=59De8ev41N=cp2nSbmDnzTi5A@mail.gmail.com>
	<CAOmV22tsOOVe1EYJNYQ47uWzhXfTq419pjpTsF0f65ZCAaWubg@mail.gmail.com>
	<CAOsWdXrChRAtDo2zoUWFLbqj64grn3pRf54qHwJbpzc542=cfA@mail.gmail.com>
	<CANZa=GvDQS8wKXpbP7qajqPzwazhHR4qjsixvtqTYaiA-L_6EQ@mail.gmail.com>
	<CAOsWdXrK67GxyOkn-Wc8xJUBArmQ2c_9vUsA2pnJOEB7aMRD4g@mail.gmail.com>
	<A7862845-FDF9-42D2-B294-8A027C7D1512@gmail.com>
	<CAOsWdXosg8XKVWh14sZzGm7Y2nHJrv+pUFVGbFVo164vPH_iMQ@mail.gmail.com>
	<CANZa=Gv16c0cM3GYPY0_az3iTsQajhn62-zHsdcOCX3-EVNLdg@mail.gmail.com>
Date: Tue, 18 Dec 2012 22:03:28 -0500
Message-ID: 
 <CAOsWdXryDPJcxmKUEy2HVPO3yaYe_twEfFdbPS6bdQNv-T8PDw@mail.gmail.com>
Subject: Re: HBase 0.94 security configurations
From: Bob Futrelle <bob.futrelle@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=f46d04016823d3613804d12bdbc3

--f46d04016823d3613804d12bdbc3
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Nick: Thanks still again for the lengthy comments.
This is a quick reply covering some of the points.

There have been no "Class not found" exceptions in the more recent versions
of my system setup.

jps -l  immediately after starting hbase, shows:

8904
20689 sun.tools.jps.Jps
7899
467
20643 org.apache.hadoop.hbase.master.HMaster


I have been unable to find a version of HBase that doesn't include security=
.
I'd be most happy to use one if I can find it. Suggestions?

-------------------

My goals and needs:  Store/access about one giga words of full text biology
research papers (XML, HTML), about 10GB.  As I analyze them, I'll probably
generate a total of ten times that amount of related data, 100GB, such as
machine learning results, part-of-speech tagged text, lexicons, ontologies,
parse trees, etc.  All will need DBs.

All of this, including a concept retrieval system, will be proof of
concept.  After I get some publications based on the system and results,
I'll distribute the code and data.  I'll leave it to others to use it as
they see fit.

The system is so novel, and hopefully powerful, that I can imagine writing
a monograph describing it.

My focus is, and always has been, basic science, with a career spanning
theoretical physics, experimental biology, and for the last 25 years, a
faculty member of the College of Computer and Information Science at
Northeastern University in Boston.  I retired at the end of June, 2011.

I've gone back to have another look at Cassandra. The CLI is
straightforward, but Java will be the test.

  - Bob  Futrelle


On Tue, Dec 18, 2012 at 6:24 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> Responses inline.
>
> On Tue, Dec 18, 2012 at 2:27 PM, Bob Futrelle <bob.futrelle@gmail.com
> >wrote:
>
> > I'm sure I found all the jars I need.
> >
>
> The suggestion that you didn't have all the jars you need is based on the
> NoClassDefFound exception you mentioned earlier.
>
>  I'm not a Maven user.
> >
>
> I don't think anyone really *wants* to be a maven user. It is a MoreSimpl=
e
> solution to dependency management. If you're an Eclipse and Ant user, you
> are accustomed to pulling in dependencies by hand, and then dependencies'
> dependencies, and so on. Maven, for all the NotSoSimple it introduces,
> makes this part almost pleasant. You don't need to understand maven well =
to
> make basic use of it.
>
> My son understands Maven well.
> > He knows about the system I'm developing.
> > He suggests that there is no need for me to learn and use Maven.
> >
>
> I cannot comment regarding this claim.
>
> My approach is straightforward.
> > I let Eclipse do the builds and have been for,
> > I don't know - the last ten years?
> >
>
> You can still let Eclipse do the builds; use maven to define the
> dependencies and Eclipse can invoke it on your behalf. That's how many of
> us work every day.
>
> I get plenty of "Warn" and "Info" lines in my console.
> > Then it settles down to repeating this block of lines indefinitely:
> >
> > 12/12/18 17:00:22 WARN zookeeper.RecoverableZooKeeper: Possibly transie=
nt
> > ZooKeeper exception:
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode =3D ConnectionLoss for /hbase/master
> >
>
> I wonder if your HMaster process stopped for some reason. You can verify
> that it's running using this command, your output will look something lik=
e
> this:
>
> $ jps -l
> 22783 org.apache.hadoop.hbase.master.HMaster
> 22846 sun.tools.jps.Jps
> 15687
>
> A simple confirmation that the process is up and running. I recommend
> running this a couple minutes after you run start-hbase. It can take a
> while for the JVM to launch, spawn threads, bind to ports, etc. All this
> business is because HBase is a bit more complex than an embedded SQL engi=
ne
> -- different beasts.
>
> java.lang.SecurityException: Unable to locate a login configuration
> > occurred when trying to find JAAS configuration.
> >
>
> This is to annoying. From my understanding, it's related to a bug in the
> JVM that Apple ships. See more details on this ticket:
> https://issues.apache.org/jira/browse/HADOOP-7489. See the comment about
> exporting HADOOP_OPTS for a potential method for silencing this warning.
>
> The serious exception is the java.net.ConnectException.  This develops
> > after the four calls above listed below the exception, starting in
> > ZooKeeper.
> >
>
> Yep, this sounds like try, try, try, fail logic happening. Do be absolute=
ly
> sure the HMaster process is running.
>
> I'm thinking that I have to arrange what ./bin/start-hbase.sh does so it =
is
> > in synch with what my code is trying to do, or vice-versa --- some comm=
on
> > object/protocol that they agree on.  Then a connection should work.
> >
>
> So long as the hbase-client matches the hbase-serve jar versions, you
> should be all good. If something is broken in the start-hbase script, we
> (the HBase project) has much bigger issues.
>
> There's lots of info about enabling HBase security, but I think I should
> > disable it, though all the output above suggest that the SaslClient
> > exception is only a *warn*.  I've found no explicit discussions out the=
re
> > about disabling security or about not enabling it - same thing in the
> end.
> >  All my work is behind my firewall.
> >
>
> Aside from the JAAS error I described above, the best way to not have
> security hassles is to be sure you're running a version of HBase that
> doesn't include security. Hence my earlier questions regarding which
> distribution you're running.
>
> I'm impressed how complicated it is to do something of a Hello World in
> > HBase.
>
> I used Java DB embedded in my apps for some years.  It is simple to use.
> >  It is slow.  It is SQL.
> > I want NOSlow and NOSQL, but I don't want NOSimple  ;-)
> >
>
> Nothing says "complex" like a massively concurrent, distributed
> state-management system like the combined powers of HDFS and HBase. That
> said, I want our client interfaces to be as simple and intuitive as
> possible. HBase is very different from the embedded SQL engines to which
> you're likely accustomed. If you're looking for an embedded NoSQL instead
> of a distributed NoSQL, maybe we can recommend a better alternative to
> HBase? Can you describe further your expectations?
>
> Thanks,
> Nick
>
> On Tue, Dec 18, 2012 at 3:33 PM, Mesika, Asaf <asaf.mesika@gmail.com>
> wrote:
> >
> > > I suggest you use a Maven project.
> > > I can copy-paste a small pom.xml for you to get you started.
> > > It will solve all of those missing JARS you have.
> > >
> > > Just make sure to install M2Eclipse plugin (Eclipse Marketplace)
> > >
> > > On Dec 18, 2012, at 8:15 PM, Bob Futrelle wrote:
> > >
> > > > Nick's suggestions appear to have helped, but we're not home yet.
> > > >
> > > > Start-up, CLI:
> > > >
> > > > $./bin/start-hbase.sh
> > > > starting master, logging to
> > > >
> > >
> >
> /Users/robertfutrelle/Research/HBase/hbase/bin/../logs/hbase-robertfutrel=
le-master-Macintosh.local.out
> > > >
> > > > Then I run my Java code, w.  conf.set(=85)  commented out per Nick'=
s
> > > > suggestion.
> > > > 23 lines of output in Eclipse console.  Last line of the 23 is,
> > > >
> > > > 12/12/18 12:56:51 INFO zookeeper.ClientCnxn: Session establishment
> > > complete
> > > > on server localhost/0:0:0:0:0:0:0:1:2181, sessionid =3D
> > 0x13baf2762e80004,
> > > > negotiated timeout =3D 40000
> > > >
> > > >
> > > > Then this exception brings things to a halt,
> > > >
> > > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > > com/google/protobuf/Message
> > > >
> > > > I'm a bit confused by the mixed approach in which I start HBase fro=
m
> > the
> > > > command line and then execute Java code.  How do the two come
> together?
> > > >
> > > > Thank you, everyone, for your continued help.
> > > >
> > > > As for security, it's not even listed in the index of Lars George's
> > HBase
> > > > book.
> > > >
> > > > I'm confident that *persistence* will get things working (see my PS
> > > below).
> > > >
> > > >   - Bob
> > > >
> > > > PS on persistence:
> > > >
> > > > Nothing in the world can take the place of Persistence. Talent will
> > not;
> > > > nothing is more common than unsuccessful men with talent. Genius wi=
ll
> > > not;
> > > > unrewarded genius is almost a proverb. Education will not; the worl=
d
> is
> > > > full of educated derelicts. Persistence and determination alone are
> > > > omnipotent. The slogan 'Press On' has solved and always will solve
> the
> > > > problems of the human race. <
> > > http://www.quotationspage.com/quote/2771.html>
> > > > [image: [info]] <http://www.quotationspage.com/quote/2771.html
> >[image:
> > > > [add]] <http://www.quotationspage.com/myquotations.php?add=3D2771
> > >[image:
> > > > [mail]] <http://www.quotationspage.com/quote/2771.html#email>[image=
:
> > > > [note]]<http://www.quotationspage.com/quote/2771.html#note>
> > > > *Calvin Coolidge*
> > > >
> > > >
> > > > On Tue, Dec 18, 2012 at 3:19 AM, Nick Dimiduk <ndimiduk@gmail.com>
> > > wrote:
> > > >
> > > >> Are you using secure HBase? Don't -- it'll only get in the way for=
 a
> > > simple
> > > >> example. Is the master running? Be sure to run ./bin/start-hbase.s=
h
> > from
> > > >> the directory where you unpacked the tgz. You can omit the
> > conf.set(...)
> > > >> business from your code. By default, the configuration will point =
to
> > > local
> > > >> host.
> > > >>
> > > >> Does any of that help?
> > > >>
> > > >> -n
> > > >> On Dec 17, 2012 8:47 PM, "Bob Futrelle" <bob.futrelle@gmail.com>
> > wrote:
> > > >>
> > > >>> Thanks for your quick reply.
> > > >>>
> > > >>>
> > > >>> On Mon, Dec 17, 2012 at 11:25 PM, Jimmy Xiang <jxiang@cloudera.co=
m
> >
> > > >> wrote:
> > > >>>
> > > >>>> Have you tried IPv4?
> > > >>>
> > > >>>
> > > >>> I can disable IPv6 in Mountain Lion, but all my "communication" i=
s
> > > >>> *within*my own machine, so I don't understand why I'd be messing
> with
> > > >>> IP, since
> > > >>> nothing I'm doing is going through my ethernet or WiFi.  But that
> may
> > > be
> > > >>> the HBase view of communication, even when entirely within a
> > machine(?)
> > > >>> It's late - will try in the morning.
> > > >>>
> > > >>> As for the cloudera page, it seems like a huge amount of
> information
> > > that
> > > >>> goes way beyond the trivial setup that I'm trying to arrange for.
> > >  Maybe
> > > >>> what I need is buried in the page somewhere  (emphasis on
> "buried").
> > > >>>
> > > >>> As they say:  "My code is so simple, what could possibly go wrong=
?"
> > >  ;-)
> > > >>>
> > > >>> - Bob
> > > >>>
> > > >>>
> > > >>>> As to secure HBase configuration, this
> > > >>>> instruction may be of some help:
> > > >>>>
> > > >>>>
> > https://ccp.cloudera.com/display/CDH4DOC/HBase+Security+Configuration
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Jimmy
> > > >>>>
> > > >>>> On Mon, Dec 17, 2012 at 8:15 PM, Bob Futrelle <
> > bob.futrelle@gmail.com
> > > >
> > > >>>> wrote:
> > > >>>>> I have the same problem that Amit Sela had - an endlessly
> repeating
> > > >>>> message
> > > >>>>> in my Eclipse console:
> > > >>>>>
> > > >>>>> java.net.ConnectException
> > > >>>>> : Connection refused
> > > >>>>>
> > > >>>>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > >>>>>
> > > >>>>> at
> > > >>>
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >>>>>
> > > >>>>> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
> > > >>>>> ClientCnxnSocketNIO.java:286)
> > > >>>>>
> > > >>>>> at
> > > >> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:103=
5)
> > > >>>>>
> > > >>>>> 12/12/17 20:54:51 INFO zookeeper.ClientCnxn: Opening socket
> > > >> connection
> > > >>> to
> > > >>>>> server /fe80:0:0:0:0:0:0:1%1:2181
> > > >>>>>
> > > >>>>> 12/12/17 20:54:51 WARN zookeeper.RecoverableZooKeeper: Possibly
> > > >>> transient
> > > >>>>> ZooKeeper exception:
> > > >>>>> org.apache.zookeeper.KeeperException$ConnectionLossException:
> > > >>>>> KeeperErrorCode =3D ConnectionLoss for /hbase/master
> > > >>>>>
> > > >>>>> 12/12/17 20:54:51 INFO util.RetryCounter: Sleeping 2000ms befor=
e
> > > >> retry
> > > >>>> #1...
> > > >>>>>
> > > >>>>> 12/12/17 20:54:56 WARN client.ZooKeeperSaslClient:
> > SecurityException:
> > > >>>>> java.lang.SecurityException: Unable to locate a login
> configuration
> > > >>>>> occurred when trying to find JAAS configuration.
> > > >>>>>
> > > >>>>> 12/12/17 20:54:56 INFO client.ZooKeeperSaslClient: Client will
> not
> > > >>>>> SASL-authenticate because the default JAAS configuration sectio=
n
> > > >>> 'Client'
> > > >>>>> could not be found. If you are not using SASL, you may ignore
> this.
> > > >> On
> > > >>>> the
> > > >>>>> other hand, if you expected SASL to work, please fix your JAAS
> > > >>>>> configuration.
> > > >>>>>
> > > >>>>> 12/12/17 20:54:56 WARN zookeeper.ClientCnxn: Session 0x0 for
> server
> > > >>> null,
> > > >>>>> unexpected error, closing socket connection and attempting
> > reconnect
> > > >>>>>
> > > >>>>>
> > > >>>>> I'm about as much of a newbie as you can imagine.
> > > >>>>>
> > > >>>>> I found a small piece of code that creates a table.
> > > >>>>> It threw endless exceptions.  I fixed each by finding the corre=
ct
> > > >> jars
> > > >>> (7
> > > >>>>> in all)
> > > >>>>> and occasionally changing a bit of the code, e.g., using, not a
> > > >>>>> constructor, but
> > > >>>>>
> > > >>>>> HBaseConfiguration.create();
> > > >>>>>
> > > >>>>>
> > > >>>>> I'm using hbase-0.94.3 and Eclipse 4.2.1 (Juno).
> > > >>>>> No apache/hbase plugins, just straight Eclipse/Java.
> > > >>>>>
> > > >>>>> The only setting I've touched is hbase-site.xml
> > > >>>>> in which I put the path to the directory that I want my DBs in.
> > > >>>>> It worked from HBase Shell.
> > > >>>>> But of course I want it to work from Java.
> > > >>>>>
> > > >>>>> I suspect that the right info in the hbase-policy.xml would fix
> all
> > > >>> this.
> > > >>>>> But my understanding of HBase so far is too skimpy to know what
> to
> > > >> put
> > > >>>>> there.
> > > >>>>>
> > > >>>>> My goal is to do all this work entirely within my own machine
> > > >> (MacBook
> > > >>>> Pro).
> > > >>>>> Single user, nothing remote, no clients/cluster.
> > > >>>>>
> > > >>>>> For total disclosure, here's the entire source - my little HBas=
e
> > > >> Hello
> > > >>>> World
> > > >>>>> :
> > > >>>>>
> > > >>>>> package us.tsos.nlpng.hbase.test1;
> > > >>>>>
> > > >>>>>
> > > >>>>> import java.io.IOException;
> > > >>>>>
> > > >>>>>
> > > >>>>> import org.apache.hadoop.hbase.HColumnDescriptor;
> > > >>>>>
> > > >>>>> import org.apache.hadoop.hbase.HTableDescriptor;
> > > >>>>>
> > > >>>>> import org.apache.hadoop.hbase.client.HBaseAdmin;
> > > >>>>>
> > > >>>>> import org.apache.hadoop.hbase.HBaseConfiguration;
> > > >>>>>
> > > >>>>>
> > > >>>>> public class JustCreateTable {
> > > >>>>>
> > > >>>>>
> > > >>>>> public static void main(String[] args) throws IOException {
> > > >>>>>
> > > >>>>>
> > > >>>>> org.apache.hadoop.conf.Configuration conf =3D
> > > >>> HBaseConfiguration.create();
> > > >>>>>
> > > >>>>> conf.set("hbase.master","localhost:60000");
> > > >>>>>
> > > >>>>>
> > > >>>>> HBaseAdmin hbase =3D new HBaseAdmin(conf);
> > > >>>>>
> > > >>>>> HTableDescriptor desc =3D new HTableDescriptor("TEST");
> > > >>>>>
> > > >>>>> HColumnDescriptor meta =3D new
> > > >> HColumnDescriptor("personal".getBytes());
> > > >>>>>
> > > >>>>> HColumnDescriptor prefix =3D new
> > > >> HColumnDescriptor("account".getBytes());
> > > >>>>>
> > > >>>>> desc.addFamily(meta);
> > > >>>>>
> > > >>>>> desc.addFamily(prefix);
> > > >>>>>
> > > >>>>> hbase.createTable(desc);
> > > >>>>>
> > > >>>>> }
> > > >>>>>
> > > >>>>> }
> > > >>>>>
> > > >>>>>
> > > >>>>> -- Bob Futrelle
> > > >>>>>
> > > >>>>>
> > > >>>>>    retired CS prof.
> > > >>>>>
> > > >>>>>    working hard on an NLP system
> > > >>>>>
> > > >>>>>    HBase will really fit the ticket
> > > >>>>>
> > > >>>>>
> > > >>>>> "You never *learn* anything, you just get used to it."
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>

--f46d04016823d3613804d12bdbc3--