The following /etc/hosts file works correctly for HBase 0.94.x and earlier, on Ubuntu. Use this as a template if you run into trouble.
Return-Path:
Quickstart will get you up and running on a single-node, standalone instance of HBase, followed by a pseudo-distributed single-machine instance, and finally a fully-distributed cluster.
+Quickstart will get you up and running on a single-node, standalone instance of HBase.
This guide describes the setup of a standalone HBase instance running against the local filesystem.
-This is not an appropriate configuration for a production instance of HBase, but will allow you to experiment with HBase.
-This section shows you how to create a table in HBase using the hbase shell
CLI, insert rows into the table, perform put and scan operations against the table, enable or disable the table, and start and stop HBase.
-Apart from downloading HBase, this procedure should take less than 10 minutes.
- - | -
- Local Filesystem and Durability
-The following is fixed in HBase 0.98.3 and beyond. See HBASE-11272 and HBASE-11218.
- |
-
This section describes the setup of a single-node standalone HBase.
+A standalone instance has all HBase daemons — the Master, RegionServers,
+and ZooKeeper — running in a single JVM persisting to the local filesystem.
+It is our most basic deploy profile. We will show you how
+to create a table in HBase using the hbase shell
CLI,
+insert rows into the table, perform put and scan operations against the
+table, enable or disable the table, and start and stop HBase.
Using HBase with a local filesystem does not guarantee durability. -The HDFS local filesystem implementation will lose edits if files are not properly closed. -This is very likely to happen when you are experimenting with new software, starting and stopping the daemons often and not always cleanly. -You need to run HBase on HDFS to ensure all writes are preserved. -Running against the local filesystem is intended as a shortcut to get you familiar with how the general system works, as the very first phase of evaluation. -See HBASE-3696 and its associated issues for more details about the issues of running on the local filesystem.
+Apart from downloading HBase, this procedure should take less than 10 minutes.
- Loopback IP - HBase 0.94.x and earlier
-The below advice is for hbase-0.94.x and older versions only. This is fixed in hbase-0.96.0 and beyond.
- |
-
-
Prior to HBase 0.94.x, HBase expected the loopback IP address to be 127.0.0.1. Ubuntu and some other distributions default to 127.0.1.1 and this will cause problems for you. See Why does HBase care about /etc/hosts? for detail
+Prior to HBase 0.94.x, HBase expected the loopback IP address to be 127.0.0.1. +Ubuntu and some other distributions default to 127.0.1.1 and this will cause +problems for you. See Why does HBase care about /etc/hosts? for detail
The following /etc/hosts file works correctly for HBase 0.94.x and earlier, on Ubuntu. Use this as a template if you run into trouble.
This issue has been fixed in hbase-0.96.0 and beyond.
Choose a download site from this list of Apache Download Mirrors. Click on the suggested top link. -This will take you to a mirror of HBase -Releases. +This will take you to a mirror of HBase Releases. Click on the folder named stable and then download the binary file that ends in .tar.gz to your local filesystem. -Prior to 1.x version, be sure to choose the version that corresponds with the version of Hadoop you are -likely to use later (in most cases, you should choose the file for Hadoop 2, which will be called -something like hbase-0.98.13-hadoop2-bin.tar.gz). Do not download the file ending in src.tar.gz for now.
For HBase 0.98.5 and later, you are required to set the JAVA_HOME
environment variable before starting HBase.
-Prior to 0.98.5, HBase attempted to detect the location of Java if the variables was not set.
-You can set the variable via your operating system’s usual mechanism, but HBase provides a central mechanism, conf/hbase-env.sh.
-Edit this file, uncomment the line starting with JAVA_HOME
, and set it to the appropriate location for your operating system.
+
You are required to set the JAVA_HOME
environment variable before starting HBase.
+You can set the variable via your operating system’s usual mechanism, but HBase
+provides a central mechanism, conf/hbase-env.sh.
+Edit this file, uncomment the line starting with JAVA_HOME
, and set it to the
+appropriate location for your operating system.
The JAVA_HOME
variable should be set to a directory which contains the executable file bin/java.
Most modern Linux operating systems provide a mechanism, such as /usr/bin/alternatives on RHEL or CentOS, for transparently switching between versions of executables such as Java.
In this case, you can set JAVA_HOME
to the directory containing the symbolic link to bin/java, which is usually /usr.
JAVA_HOME
to the directory containing the
JAVA_HOME=/usr
- - | -
-These instructions assume that each node of your cluster uses the same configuration.
-If this is not the case, you may need to set JAVA_HOME separately for each node.
- |
-
Edit conf/hbase-site.xml, which is the main HBase configuration file.
@@ -560,7 +527,7 @@ Many servers are configured to delete the contents of /tmp upon reboot,
The following configuration will store HBase’s data in the hbase directory, in the home directory of the user called testuser
.
Paste the <property>
tags beneath the <configuration>
tags, which should be empty in a new HBase install.
<property>
tags beneath the <configuration
You do not need to create the HBase data directory. -HBase will do this for you. -If you create the directory, HBase will attempt to do a migration, which is not what you want.
+HBase will do this for you. If you create the directory, +HBase will attempt to do a migration, which is not what you want. ++ + | ++The hbase.rootdir in the above example points to a directory +in the local filesystem. The 'file:/' prefix is how we denote local filesystem. +To home HBase on an existing instance of HDFS, set the hbase.rootdir to point at a +directory up on your instance: e.g. hdfs://namenode.example.org:8020/hbase. +For more on this variant, see the section below on Standalone HBase over HDFS. + | +
jps
command to verify that you have one running process called HMaster
.
In standalone mode HBase runs all daemons within this single JVM, i.e.
-the HMaster, a single HRegionServer, and the ZooKeeper daemon.
+the HMaster, a single HRegionServer, and the ZooKeeper daemon.
+Go to http://localhost:16010 to view the HBase Web UI.
Java needs to be installed and available.
-If you get an error indicating that Java is not installed, but it is on your system, perhaps in a non-standard location, edit the conf/hbase-env.sh file and modify the JAVA_HOME setting to point to the directory that contains bin/java your system.
+If you get an error indicating that Java is not installed,
+but it is on your system, perhaps in a non-standard location,
+edit the conf/hbase-env.sh file and modify the JAVA_HOME
+setting to point to the directory that contains bin/java your system.
|
jps
to be sure that the HMaster and HRegionServer processes
The above has shown you how to start and stop a standalone instance of HBase. +In the next sections we give a quick overview of other modes of hbase deploy.
+After working your way through quickstart, you can re-configure HBase to run in pseudo-distributed mode.
-Pseudo-distributed mode means that HBase still runs completely on a single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate process.
-By default, unless you configure the hbase.rootdir
property as described in quickstart, your data is still stored in /tmp/.
+
After working your way through quickstart standalone mode,
+you can re-configure HBase to run in pseudo-distributed mode.
+Pseudo-distributed mode means that HBase still runs completely on a single host,
+but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate process:
+in standalone mode all daemons ran in one jvm process/instance.
+By default, unless you configure the hbase.rootdir
property as described in
+quickstart, your data is still stored in /tmp/.
In this walk-through, we store your data in HDFS instead, assuming you have HDFS available.
You can skip the HDFS configuration to continue storing your data in the local filesystem.
This quickstart assumes that each node is a virtual machine and that they are all on the same network.
-It builds upon the previous quickstart, Intermediate - Pseudo-Distributed Local Install, assuming that the system you configured in that procedure is now node-a
.
+It builds upon the previous quickstart, Pseudo-Distributed Local Install, assuming that the system you configured in that procedure is now node-a
.
Stop HBase on node-a
before continuing.
node-a
jps
Outputnode-a
jps
Outputnode-b
jps
Outputnode-b
jps
Outputnode-a
jps
Outputnode-a
jps
OutputConfiguring the maximum number of file descriptors and processes for the user who is running the HBase process is an operating system configuration, rather than an HBase configuration. It is also important to be sure that the settings are changed for the user that actually runs HBase. To see which user started HBase, and that user’s ulimit configuration, look at the first line of the HBase log for that instance. A useful read setting config on your hadoop cluster is Aaron Kimball’s Configuration Parameters: What can you just ignore?
ulimit
Settings on Ubuntuulimit
Settings on UbuntuTo configure ulimit settings on Ubuntu, edit /etc/security/limits.conf, which is a space-delimited file with four columns. Refer to the man page for limits.conf for details about the format of this file. In the following example, the first line sets both soft and hard limits for the number of open files (nofile) to 32768 for the operating system user with the username hadoop. The second line sets the number of processes to 32000 for the same user.
@@ -1976,6 +1971,40 @@ Standalone mode is what is described in the quickstart In standalone mode, HBase does not use HDFS — it uses the local filesystem instead — and it runs all HBase daemons and a local ZooKeeper all up in the same JVM. ZooKeeper binds to a well known port so clients may talk to HBase.A sometimes useful variation on standalone hbase has all daemons running inside the +one JVM but rather than persist to the local filesystem, instead +they persist to an HDFS instance.
+You might consider this profile when you are intent on +a simple deploy profile, the loading is light, but the +data must persist across node comings and goings. Writing to +HDFS where data is replicated ensures the latter.
+To configure this standalone variant, edit your hbase-site.xml +setting the hbase.rootdir to point at a directory in your +HDFS instance but then set hbase.cluster.distributed +to false. For example:
+<configuration>
+ <property>
+ <name>hbase.rootdir</name>
+ <value>hdfs://namenode.example.org:8020/hbase</value>
+ </property>
+ <property>
+ <name>hbase.cluster.distributed</name>
+ <value>false</value>
+ </property>
+</configuration>
+See the ZooKeeper section for ZooKeeper setup instructions for HBase.
This is a bare-bones conf/hbase-site.xml for a distributed HBase cluster. @@ -6585,7 +6614,7 @@ Spawning HBase Shell commands in this way is slow, so keep that in mind when you
You can pass commands to the HBase Shell in non-interactive mode (see hbase.shell.noninteractive) using the echo
command and the |
(pipe) operator.
@@ -6621,7 +6650,7 @@ DESCRIPTION ENABLED
Since scripts are not designed to be run interactively, you need a way to check whether your command failed or succeeded. @@ -6669,7 +6698,7 @@ For instance, if your script creates a table, but returns a non-zero exit value,
You can enter HBase Shell commands into a text file, one command per line, and pass that file to the HBase Shell.
Pass the path to the command file as the only argument to the hbase shell
command.
@@ -7319,7 +7348,7 @@ Namespace membership is determined during table creation by specifying a fully-q
3
, but in 0.96 and newer has been changed to 1
.
This example uses HBase Shell to keep a maximum of 5 versions of all columns in column family You can also specify the minimum number of versions to store per column family.
@@ -8008,7 +8037,7 @@ Salting can be helpful if you have a few "hot" row key patterns which come up ov
Consider the following example, which shows that salting can spread write load across multiple RegionServers, and illustrates some of the negative implications for reads. Suppose you have the following list of row keys, and your table is split such that there is one region for each letter of the alphabet.
@@ -8063,7 +8092,7 @@ In this way, salting attempts to increase throughput on writes, but has a cost d
Using a deterministic hash allows the client to reconstruct the complete rowkey and use a Get operation to retrieve that row as normal. The following example shows how to grant access at the table level. Showing a User’s Effective Permissions Define the List of Visibility Labels Associate Labels with Users Clear Labels From Users Adjust these settings to suit your environment. Download the code from http://code.google.com/p/hbase-jdo/. This example uses JDO to create a table and an index, insert a row into a table, get
@@ -18257,7 +18286,7 @@ $ bin/hbase org.python.util.jython The following Jython code example creates a table, populates it with data, fetches
@@ -18314,7 +18343,7 @@ admin.deleteTable(desc.getName())
This example scans a table and returns the results that match a given family qualifier. You can combine multiple operators to create a hierarchy of filters, such as the following example: This example shows how HBaseContext can be used to do a Below is an example of bulkPut with DStreams. It is very close in feel to the RDD
@@ -19063,7 +19092,7 @@ out directly from the reduce phase. First lets look at an example of using the basic bulk load functionality The following example shows bulk loading with Spark. Now lets look at how you would call the thin record bulk load implementation In HBaseSparkConf, four parameters related to timestamp can be set. They are TIMESTAMP,
@@ -19443,7 +19472,7 @@ sqlContext.sql("select count(col1) from table").show
HBase-Spark Connector support different data formats like Avro, Jason, etc. The use case below
@@ -20953,7 +20982,7 @@ It is useful for tuning the IO impact of prefetching versus the time before all
To enable prefetching on a given column family, you can use HBase Shell or use the API. HBase expects the loopback IP Address to be 127.0.0.1.
-See the Getting Started section on Loopback IP - HBase 0.94.x and earlier. This example shows each of the properties with valid values. A commonly used macro in Jira is {code}. Everything inside the tags is preformatted, as in this example. Publishing to maven requires you sign the artifacts you want to upload.
@@ -29846,7 +29875,7 @@ This is the indication that the patch was not created with One thing you will notice with these examples is that there are a lot of git pull commands.
@@ -33217,7 +33246,7 @@ You do not need to re-create the table or copy data.
If you are changing codecs, be sure the old codec is still available until all the old StoreFiles have been compacted.f1
.
@@ -7551,7 +7580,7 @@ You could also use
-foo0003
to always, and predictably, receive the a
prefix.
Then, to retrieve that row, you would already know the key.
@@ -8424,7 +8453,7 @@ This allows for point-in-time queries even in the presence of deletes.
A new "raw" scan options returns all deleted rows and the delete markers.
KEEP_DELETED_CELLS
Using HBase ShellKEEP_DELETED_CELLS
Using HBase ShellKEEP_DELETED_CELLS
Using the APIKEEP_DELETED_CELLS
Using the API
@@ -11758,7 +11787,7 @@ For some examples of scanner specifications, issue the following HBase Shell com
Connection
Connection
IS_MOB
is required. If yo
specify the MOB_THRESHOLD
, the default threshold value of 100 KB is used.
hbase-site.xml
suit your environment, and restart or rolling restart the RegionServer.
75. Examples
81.2. Jython Code Examples
foreachPartition
on a RDD
@@ -18988,7 +19017,7 @@ access to HBasebulkPut
Example with DStreamsbulkPut
Example with DStreams86.6. Others
115.2. Loopback IP
--no-prefix
-write
or -update-read
as your first parameter, and if you do not specify another parameter, usage advice is printed for each option.
public static final String
-date
+"Wed Dec 7 14:13:58 UTC 2016"
"Wed Dec 7 20:37:37 UTC 2016"
public static final String
-revision
+"61220e4d7c8d7e5fb8ed3bbe2469bc86632c48de"
"6f25f838c0af80d2deaec91ed13248ce5024bd29"
http://git-wip-us.apache.org/repos/asf/hbase-site/blob/4ac23c25/devapidocs/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.ImplData.html
----------------------------------------------------------------------
diff --git a/devapidocs/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.ImplData.html b/devapidocs/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.ImplData.html
index dae4f1c..d1bcacf 100644
--- a/devapidocs/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.ImplData.html
+++ b/devapidocs/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.ImplData.html
@@ -393,166 +393,166 @@ extends org.jamon.AbstractTemplateProxy.ImplData
private HMaster m_master
-
+
-
+
-
+
m_format__IsNotDefault
-private boolean m_format__IsNotDefault
+m_frags__IsNotDefault
+private boolean m_frags__IsNotDefault
-
+
m_catalogJanitorEnabled
-private boolean m_catalogJanitorEnabled
+m_serverManager
+private ServerManager m_serverManager
-
+
m_catalogJanitorEnabled__IsNotDefault
-private boolean m_catalogJanitorEnabled__IsNotDefault
+m_serverManager__IsNotDefault
+private boolean m_serverManager__IsNotDefault
-
+
m_assignmentManager
-private AssignmentManager m_assignmentManager
+m_deadServers
+private Set<ServerName> m_deadServers
-
+
m_assignmentManager__IsNotDefault
-private boolean m_assignmentManager__IsNotDefault
+m_deadServers__IsNotDefault
+private boolean m_deadServers__IsNotDefault
-
+
m_servers
-private List<ServerName> m_servers
+m_filter
+private String m_filter
-
+
m_servers__IsNotDefault
-private boolean m_servers__IsNotDefault
+m_filter__IsNotDefault
+private boolean m_filter__IsNotDefault
-
+
m_frags
-private Map<String,Integer> m_frags
+m_assignmentManager
+private AssignmentManager m_assignmentManager
-
+
m_frags__IsNotDefault
-private boolean m_frags__IsNotDefault
+m_assignmentManager__IsNotDefault
+private boolean m_assignmentManager__IsNotDefault
-
+
m_serverManager
-private ServerManager m_serverManager
+m_catalogJanitorEnabled
+private boolean m_catalogJanitorEnabled
-
+
m_serverManager__IsNotDefault
-private boolean m_serverManager__IsNotDefault
+m_catalogJanitorEnabled__IsNotDefault
+private boolean m_catalogJanitorEnabled__IsNotDefault
-
+
m_filter
-private String m_filter
+m_servers
+private List<ServerName> m_servers
-
+
m_filter__IsNotDefault
-private boolean m_filter__IsNotDefault
+m_servers__IsNotDefault
+private boolean m_servers__IsNotDefault
-
+
m_deadServers
-private Set<ServerName> m_deadServers
+m_metaLocation
+private ServerName m_metaLocation
-
+
m_deadServers__IsNotDefault
-private boolean m_deadServers__IsNotDefault
+m_metaLocation__IsNotDefault
+private boolean m_metaLocation__IsNotDefault
-
+
m_metaLocation
-private ServerName m_metaLocation
+m_format
+private String m_format
@@ -598,247 +598,247 @@ extends org.jamon.AbstractTemplateProxy.ImplData
m_metaLocation__IsNotDefault
-private boolean m_metaLocation__IsNotDefault
+m_format__IsNotDefault
+private boolean m_format__IsNotDefault
public HMaster getMaster()
-
+
-
+
-
+
setFormat
-public void setFormat(String format)
+setFrags
+public void setFrags(Map<String,Integer> frags)
-
+
getFormat__IsNotDefault
-public boolean getFormat__IsNotDefault()
+getFrags__IsNotDefault
+public boolean getFrags__IsNotDefault()
-
+
setCatalogJanitorEnabled
-public void setCatalogJanitorEnabled(boolean catalogJanitorEnabled)
+setServerManager
+public void setServerManager(ServerManager serverManager)
-
+
getCatalogJanitorEnabled
-public boolean getCatalogJanitorEnabled()
+getServerManager
+public ServerManager getServerManager()
-
+
getCatalogJanitorEnabled__IsNotDefault
-public boolean getCatalogJanitorEnabled__IsNotDefault()
+getServerManager__IsNotDefault
+public boolean getServerManager__IsNotDefault()
-
+
setAssignmentManager
-public void setAssignmentManager(AssignmentManager assignmentManager)
+setDeadServers
+public void setDeadServers(Set<ServerName> deadServers)
-
+
getAssignmentManager
-public AssignmentManager getAssignmentManager()
+getDeadServers
+public Set<ServerName> getDeadServers()
-
+
getAssignmentManager__IsNotDefault
-public boolean getAssignmentManager__IsNotDefault()
+getDeadServers__IsNotDefault
+public boolean getDeadServers__IsNotDefault()
-
+
setServers
-public void setServers(List<ServerName> servers)
+setFilter
+public void setFilter(String filter)
-
+
getServers
-public List<ServerName> getServers()
+getFilter
+public String getFilter()
-
+
getServers__IsNotDefault
-public boolean getServers__IsNotDefault()
+getFilter__IsNotDefault
+public boolean getFilter__IsNotDefault()
-
+
setFrags
-public void setFrags(Map<String,Integer> frags)
+setAssignmentManager
+public void setAssignmentManager(AssignmentManager assignmentManager)
-
+
getFrags
-public Map<String,Integer> getFrags()
+getAssignmentManager
+public AssignmentManager getAssignmentManager()
-
+
getFrags__IsNotDefault
-public boolean getFrags__IsNotDefault()
+getAssignmentManager__IsNotDefault
+public boolean getAssignmentManager__IsNotDefault()
-
+
setServerManager
-public void setServerManager(ServerManager serverManager)
+setCatalogJanitorEnabled
+public void setCatalogJanitorEnabled(boolean catalogJanitorEnabled)
-
+
getServerManager
-public ServerManager getServerManager()
+getCatalogJanitorEnabled
+public boolean getCatalogJanitorEnabled()
-
+
getServerManager__IsNotDefault
-public boolean getServerManager__IsNotDefault()
+getCatalogJanitorEnabled__IsNotDefault
+public boolean getCatalogJanitorEnabled__IsNotDefault()
-
+
setFilter
-public void setFilter(String filter)
+setServers
+public void setServers(List<ServerName> servers)
-
+
getFilter
-public String getFilter()
+getServers
+public List<ServerName> getServers()
-
+
getFilter__IsNotDefault
-public boolean getFilter__IsNotDefault()
+getServers__IsNotDefault
+public boolean getServers__IsNotDefault()
-
+
setDeadServers
-public void setDeadServers(Set<ServerName> deadServers)
+setMetaLocation
+public void setMetaLocation(ServerName metaLocation)
-
+
getDeadServers
-public Set<ServerName> getDeadServers()
+getMetaLocation
+public ServerName getMetaLocation()
-
+
getDeadServers__IsNotDefault
-public boolean getDeadServers__IsNotDefault()
+getMetaLocation__IsNotDefault
+public boolean getMetaLocation__IsNotDefault()
-
+
setMetaLocation
-public void setMetaLocation(ServerName metaLocation)
+setFormat
+public void setFormat(String format)
-
+
getMetaLocation
-public ServerName getMetaLocation()
+getFormat
+public String getFormat()
getMetaLocation__IsNotDefault
-public boolean getMetaLocation__IsNotDefault()
+getFormat__IsNotDefault
+public boolean getFormat__IsNotDefault()