hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Omkar Joshi <Omkar.Jo...@lntinfotech.com>
Subject Loading text files from local file system
Date Wed, 17 Apr 2013 04:31:09 GMT
The background thread is here :


Following are the commands that I'm using to load files onto HBase :

HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-
-Dimporttsv.bulk.output=hdfs://cldx-1139-1033:9000/hbase/storefileoutput_6 PRODUCTS hdfs://cldx-1139-1033:9000/hbase/copiedFromLocal/product_6.txt

HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-
completebulkload hdfs://cldx-1139-1033:9000/hbase/storefileoutput_6 PRODUCTS

As seen, the text files to be loaded in HBase first need to be loaded on HDFS. Given our infrastructure
constraints/limitations, I'm getting space issues. The data in the text files is around 20GB
+ replication is consuming a lot of DFS.

Is there a way wherein a text file can be loaded directly from the local file system onto

Omkar Joshi

The contents of this e-mail and any attachment(s) may contain confidential or privileged information
for the intended recipient(s). Unintended recipients are prohibited from taking action on
the basis of information in this e-mail and using or disseminating the information, and must
notify the sender and delete it from their system. L&T Infotech will not accept responsibility
or liability for the accuracy or completeness of, or the presence of any virus or disabling
code in this e-mail"

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message