hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amandeep Khurana <ama...@gmail.com>
Subject Re: Location of HBase's database (database' s files) on the hard disk
Date Tue, 25 Aug 2009 06:12:46 GMT
Data in hbase tables resides in the memory till it gets flushed to
persistent storage. Anything that was in the memory would probably be lost
unless you are using the write ahead logs (which you would be using by
default in 0.20 if you dont disable it in the puts).


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz


On Mon, Aug 24, 2009 at 9:43 PM, Nguyen Thi Ngoc Huong
<huongngtn@gmail.com>wrote:

> How can I restore database when I wasn't stop Hbase and Hadoop when
> restarting my computer?
>
> 2009/8/22 Nguyen Thi Ngoc Huong <huongngtn@gmail.com>
>
> > ok. This is my error. I wasn't stop Hbase and Hadoop when restarting my
> > computer.
> > Thank you very much.
> >
> > 2009/8/21 Jean-Daniel Cryans <jdcryans@apache.org>
> >
> > 2 questions:
> >>
> >> Which version of HBase are you using?
> >>
> >> Are you stopping HBase and Hadoop when restarting your computer?
> >>
> >> Thx,
> >>
> >> J-D
> >>
> >> On Fri, Aug 21, 2009 at 5:46 AM, Nguyen Thi Ngoc
> >> Huong<huongngtn@gmail.com> wrote:
> >> > Thank you very much.
> >> > I deleted everything and configured hadoop.tmp.dir property in
> >> > hadoop-site.xml as follow
> >> > <property>
> >> >  <name>hadoop.tmp.dir</name>
> >> >  <value>/home/huongntn/hadoop-${user.name}</value>
> >> >  <description>A base for other temporary directories.</description>
> >> > </property>
> >> >
> >> > After that, I formated namenode and start-all. When I restarted my
> >> computer
> >> > and typed the command "start-all", hadoop work smoothly. I start hbase
> >> by
> >> > command "./bin/start-hbase.sh" and "./hbase shell"
> >> >
> >> > Now  I can't see my database in hbase shell (by command "list")
> although
> >> I
> >> > can see it in Hadoop site manager,
> >> >
> >> >
> >> > 2009/8/21 Amandeep Khurana <amansk@gmail.com>
> >> >
> >> >> 1. If you have formatted your namenode before starting the first
> time,
> >> >> thats
> >> >> all thats needed.
> >> >>
> >> >> 2. To start from scratch, delete everything thats there in the
> >> directory
> >> >> where you are pointing your hdfs to; format namenode again; start all
> >> >>
> >> >> 3. If it still doesnt work, look at the namenode logs to see whats
> >> >> happening. Post it here if you cant figure it out.
> >> >>
> >> >>
> >> >> Amandeep Khurana
> >> >> Computer Science Graduate Student
> >> >> University of California, Santa Cruz
> >> >>
> >> >>
> >> >> On Fri, Aug 21, 2009 at 1:30 AM, Nguyen Thi Ngoc Huong
> >> >> <huongngtn@gmail.com>wrote:
> >> >>
> >> >> > >You dont need to format the namenode everytime.. Just
> >> bin/start-all.sh
> >> >> >
> >> >> > Really? Just bin/start-all.sh, namnode is not started (when I
type
> >> >> command
> >> >> > "jps", there are only 5 processes
> >> >> > 3421 SecondaryNameNode
> >> >> > 3492 JobTracker
> >> >> > 3582 TaskTracker
> >> >> > 4031 Jps
> >> >> > 3325 DataNode, there isn't Namenode process)
> >> >> > and certainly, the page http://localhost:50070 is died and
> >> connection
> >> >> from
> >> >> > Hbase to hadoop is died, too
> >> >> >
> >> >> >
> >> >> > 2009/8/21 Amandeep Khurana <amansk@gmail.com>
> >> >> >
> >> >> > > On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
> >> >> > > <huongngtn@gmail.com>wrote:
> >> >> > >
> >> >> > > > Thanks you very much. I editted file hbase-site.xml
as follow
> >> >> > > >
> >> >> > > > <property>
> >> >> > > >    <name>hbase.rootdir</name>
> >> >> > > >     <value>hdfs://localhost:54310/hbase</value>
> >> >> > > >     <description>The directory shared by region
servers.
> >> >> > > >    Should be fully-qualified to include the filesystem
to use.
> >> >> > > >    E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> >> >> > > >    </description>
> >> >> > > >  </property>
> >> >> > > >
> >> >> > > > with fs.default.name is hdfs://localhost:54310
> >> >> > > > Now, I can see hbase database in Hadoop site manager
(in
> "hbase"
> >> >> > > directory
> >> >> > > > not "tmp" directory in hdfs ).
> >> >> > > > However, when I restart my computer, I must restart
hadoop (by
> >> >> command
> >> >> > > > ./bin/hadoop format namenode and ./bin/start all) ,
restart
> >> hbase,
> >> >> and
> >> >> > my
> >> >> > > > database is lost. What can I do to save my database?
> >> >> > > >
> >> >> > >
> >> >> > > You dont need to format the namenode everytime.. Just
> >> bin/start-all.sh
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > >
> >> >> > > > 2009/8/21 Amandeep Khurana <amansk@gmail.com>
> >> >> > > >
> >> >> > > > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc
Huong <
> >> >> > > > > huongngtn@gmail.com
> >> >> > > > > > wrote:
> >> >> > > > >
> >> >> > > > > > How can I configure the location of the hbase
directory? I
> >> >> > configured
> >> >> > > > > > hbase-site.xml as follow:
> >> >> > > > > >
> >> >> > > > > > <property>
> >> >> > > > > >    <name>hbase.rootdir</name>
> >> >> > > > > >    <value>*file:///temp/hbase-${user.name}/hbase*</value>
> >> >> > > > > >    <description>The directory shared
by region servers.
> >> >> > > > > >    Should be fully-qualified to include the
filesystem to
> >> use.
> >> >> > > > > >    E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> >> >> > > > > >    </description>
> >> >> > > > > >  </property>
> >> >> > > > > >
> >> >> > > > >
> >> >> > > > > Thats the trouble.. Your data is being stored in
the temp..
> >> instead
> >> >> > > store
> >> >> > > > > it
> >> >> > > > > in your hdfs.
> >> >> > > > > so the value of the above property would be something
like
> >> >> > > > > *hdfs://namenodeserver:port/hbase*
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > >
> >> >> > > > > > and the log file is
> >> >> > > > > > Not starting HMaster because:
> >> >> > > > > > java.io.IOException: Mkdirs failed to create
> >> >> > > > > > file:/temp/hbase-huongntn/hbase
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> >> >> > > > > >    at
> >> >> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> >> >> > > > > >    at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> >> >> > > > > >    at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> >> >> > > > > >    at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> >> >> > > > > >    at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> >> >> > > > > >    at
> >> >> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> >> >> > > > > >    at
> >> >> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> >> >> > > > > > 2009-08-21 13:35:24,163 ERROR
> >> >> > org.apache.hadoop.hbase.master.HMaster:
> >> >> > > > Can
> >> >> > > > > > not start master
> >> >> > > > > > java.io.IOException: Mkdirs failed to create
> >> >> > > > > > file:/temp/hbase-huongntn/hbase
> >> >> > > > > >    at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> >> >> > > > > >    at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> >> >> > > > > >    at
> >> >> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> >> >> > > > > >    at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> >> >> > > > > >    at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> >> >> > > > > >    at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> >> >> > > > > >    at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> >> >> > > > > >    at
> >> >> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> >> >> > > > > >    at
> >> >> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 2009/8/21 Amandeep Khurana <amansk@gmail.com>
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > > You configure the location of the hbase
directory in the
> >> >> > > > hbase-site.xml
> >> >> > > > > > >
> >> >> > > > > > > The data being lost could have multilple
reasons. To rule
> >> out
> >> >> the
> >> >> > > > > > > basic one - where have you pointed the
hdfs to store
> data?
> >> If
> >> >> its
> >> >> > > > > > > going into /tmp, you'll lose data everytime
the tmp
> cleaner
> >> >> comes
> >> >> > > > into
> >> >> > > > > > > action.
> >> >> > > > > > >
> >> >> > > > > > > On 8/20/09, Nguyen Thi Ngoc Huong <huongngtn@gmail.com>
> >> wrote:
> >> >> > > > > > > > Hi all,
> >> >> > > > > > > > I am a beginner to HBase. I have
some question with
> Hbase
> >> >> after
> >> >> > > > setup
> >> >> > > > > > > Hbase
> >> >> > > > > > > > and Hadoop.
> >> >> > > > > > > >
> >> >> > > > > > > > The first, After setup Hbase and
create a new database,
> I
> >> >> don't
> >> >> > > > know
> >> >> > > > > > > where
> >> >> > > > > > > > is location of HBase's database
(database' s files) on
> >> the
> >> >> hard
> >> >> > > > disk.
> >> >> > > > > > At
> >> >> > > > > > > the
> >> >> > > > > > > > first, I think it is on the hbase.rootdir
directory,
> >> however,
> >> >> > > when
> >> >> > > > I
> >> >> > > > > > > delete
> >> >> > > > > > > > directory hbase.rootdir, and type
the command "list",
> all
> >> of
> >> >> > > > database
> >> >> > > > > > > > exist.
> >> >> > > > > > > >
> >> >> > > > > > > > The second, after restart computer
and restart hbase,
> all
> >> >> > > database
> >> >> > > > of
> >> >> > > > > > > HBase
> >> >> > > > > > > > is lost. Is it always true? Or did
I configure wrong?
> How
> >> can
> >> >> i
> >> >> > > > > > configure
> >> >> > > > > > > > Hbase to save  database after restart
computer?
> >> >> > > > > > > >
> >> >> > > > > > > > --
> >> >> > > > > > > > Nguyễn Thị Ngọc Hương
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > > --
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > > Amandeep Khurana
> >> >> > > > > > > Computer Science Graduate Student
> >> >> > > > > > > University of California, Santa Cruz
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > --
> >> >> > > > > > Nguyễn Thị Ngọc Hương
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > --
> >> >> > > > Nguyễn Thị Ngọc Hương
> >> >> > > >
> >> >> > >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Nguyễn Thị Ngọc Hương
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Nguyễn Thị Ngọc Hương
> >> >
> >>
> >
> >
> >
> > --
> > Nguyễn Thị Ngọc Hương
> >
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message