incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BLUR-397) Improve data loading from M/R
Date Tue, 16 Dec 2014 13:36:13 GMT

    [ https://issues.apache.org/jira/browse/BLUR-397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14248237#comment-14248237
] 

Aaron McCurry commented on BLUR-397:
------------------------------------

I agree with your points.  I like the idea of making the controller have to delegate FS calls
to the shard servers.  The only real issue with this is that some of the actions made during
certain metadata calls like create table can no longer be performed without the shard cluster
actually running.  Although I think this is a good tradeoff to make.

Aaorn

> Improve data loading from M/R
> -----------------------------
>
>                 Key: BLUR-397
>                 URL: https://issues.apache.org/jira/browse/BLUR-397
>             Project: Apache Blur
>          Issue Type: Improvement
>          Components: Blur, Blur MapReduce
>            Reporter: Tim Williams
>
> There's an awkward permissions dilemma when writing data into Blur from Map/Reduce. 

> A job would typically create a table, then load the data.  The challenge is that the
table itself is created through the controller, which means it's written to DFS as the user
actually running the controller daemon - typically 'blur'.  The Map/Reduce job may be run
as some other user totally, but it may be a user that you don't want to have write access
inside blur's directory paths. In other words, you'd like arbitrary user(s) to be able to
create/populate table data without necessarily having write access to blur's internal stuffs.
> One approach is to have the user's job write to any location they have access to, the
"tell" Blur to 'import' it - at which time, Blur would literally move the data into it's control.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message