atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guan....@zte.com.cn
Subject 答复: Re: atlas support HDFS and HBase
Date Sun, 30 Oct 2016 15:17:03 GMT
Hi Vimal Sharma ,

thanks for your responds to the question,

Now, I'll start with a few typical HBase scenarios: 
To cite two examples of the banking industry: 
Historical transaction data query 
Because the characteristics of HBase BigTable, HBase is widely used in 
historical transaction data throughout the life cycle of the user's query, 
data from the first trading system in daily increments, in the form of a 
file, through the transmission system into HDFS ,map-reduce program 
written by the programmer, processing data, and through the HBase import 
interface into the HBase table.
 At the same time, HBase will go through the subsequent processing, the 
production of other new data table, HBase table front end access through 
the WAS service using the HBase API interface, providing the user queries 
on the data, 
the older historical data from the tape library is gradually phased into 
the system, using the same processing method into HBase table.
 in general, for universal programming , the programmer will describe some 
structure directly in the database, and some simply stored in the program 
In the system, or it will be processed  by  the dependence of the program 
business. 
There is another important usage scenario: the content management system, 
which is usually a combination of HBase and solr. 
The front-end system records data through the put interface hbaseput data 
to the HBase table, in the put record at the same time, to build the 
record of the index, and inserted into the Solr, in order to achieve the 
two HBase index. 
Because of the existence of Solr, the front-end system is also very easy 
to query according to a variety of conditions in the HBase data, for the 
user to query the use of. 
Before the operation of the system, no concept of data governance, so 
there is no development of metadata interface provided to the outside 
system, when the organization is required to improve the management level, 
data governance requirements are naturally appeared, the emergence of 
atlas is consistent with such a demand, hope atlas can be a good at this 
or that “ old ”system, we also hope we be able to dive into this  to 
improve the value of atlas.



Best regards
guanyun
 




发件人:         Vimal Sharma <visharma@hortonworks.com>
收件人:         "guan.yun@zte.com.cn" <guan.yun@zte.com.cn>, 
抄送:   "dev@atlas.incubator.apache.org" <dev@atlas.incubator.apache.org>
日期:   2016/10/21 18:06
主题:   Re: atlas support HDFS and HBase



Hi Guanyun,
Atlas doesn’t have support for HBase currently. Can you elaborate your 
use case so that we can have a better understanding.

Thanks
Vimal



On 10/21/16, 3:03 PM, "guan.yun@zte.com.cn" <guan.yun@zte.com.cn> wrote:

>Hi,
>
>imagine the following scenario:
>Some of the old data processing system, only  deploy Hadoop and HBase, 
USE 
>MR processing data from the HDFS into a HBase, 
>the metadata in the HBase is R-K table , HDFS is presented in the form of 

>multi data directory, also do not introduce the deployment of Falcon ,
>under the condition ,how to  improve atlas in order to  support metadata 

>management function. if atlas so can be achieve independent support HDFS 
>and HBase that will be a better thing.
>
>Best regards
>guanyun
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message