hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ados1984@gmail.com" <ados1...@gmail.com>
Subject Re: Architecture question on Injesting Data into Hadoop
Date Tue, 25 Mar 2014 03:27:24 GMT
Thank you Shahab but I am using Impala and so I can directly create tables
using Impala on hdfs without using hbase or any other noSQL technology.

Other question that I have here is let's say that I have 3 nodes...now if
am installing impala on one node then on that node am able to create tables
but how can i create tables on other nodes and how is data divided between
different nodes and where do we need to put in information about what range
of data we want in node 1 and what range of data we want in node 2 and node
3?




On Mon, Mar 24, 2014 at 4:22 PM, Shahab Yunus <shahab.yunus@gmail.com>wrote:

> @ados1984, HDFS is a file system and HBase is a data store on top of that.
> You cannot create tables (in the conventional meaning of the word table in
> database/store) directly on HDFS without HBase.
>
> Regards,
> Shahab
>
>
> On Mon, Mar 24, 2014 at 4:11 PM, Geoffry Roberts <threadedblue@gmail.com>wrote:
>
>> Based on what you have said, it sounds as if you want to append records
>> to a file(s) in hdfs.  I was able to do this with WebHDFS and with the
>> hadoop client.  But you asked about architecture.  Would a POST to a url
>> satisfy you as to architecture?  If so setup WebHDFS as POST to it.
>>
>>
>> On Mon, Mar 24, 2014 at 1:00 PM, ados1984@gmail.com <ados1984@gmail.com>wrote:
>>
>>> Hello Team,
>>>
>>> I am doing POC in Hadoop and want to understand what is recommended
>>> architecture to injest data from different data stream like web log,
>>> portal, mobile, pos system into Hadoop system? Also what are the use cases
>>> where we need to have hbase on top of HDFS? Can't we only have hdfs and no
>>> hbase and if we have only hdfs can we create tables directly on hdfs which
>>> impala can query on?
>>>
>>> Kindly advise !!!
>>> Regards, Apurva
>>>
>>
>>
>>
>> --
>> There are ways and there are ways,
>>
>> Geoffry Roberts
>>
>
>

Mime
View raw message