hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "piyush mukati (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6893) MultipleOutputs to have configuration overrides for a named output
Date Mon, 29 May 2017 06:39:05 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

piyush mukati updated MAPREDUCE-6893:
-------------------------------------
    Description: 
To support the use-cases where we have to give different config for different named output.
one use case may be that 
we need different schemas for different named output and schema is passed as config in Record
Writer.  

or example: 
we have two named output  "schema1" and "schema2" 
for schema1 we want the value of key "schema" as "int"
while for  schema2 it should be "string".

so we can provide config as
"multioutput.overrideKeys= schema"
"multioutput. schema1.schema=int"
"multioutput. schema2.schema=string"

and while creating context in getContext 
config for schema1 will resolved to  "schema=int"
and for  schema2 it will  be "schema=string"


  was:
To support the use-cases where we have to give different config for different named output.
for example we may need different schemas for different named output and schema is passed
as config in Record Writer.  


> MultipleOutputs to have configuration overrides for a named output
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6893
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6893
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: piyush mukati
>            Priority: Minor
>
> To support the use-cases where we have to give different config for different named output.
> one use case may be that 
> we need different schemas for different named output and schema is passed as config in
Record Writer.  
> or example: 
> we have two named output  "schema1" and "schema2" 
> for schema1 we want the value of key "schema" as "int"
> while for  schema2 it should be "string".
> so we can provide config as
> "multioutput.overrideKeys= schema"
> "multioutput. schema1.schema=int"
> "multioutput. schema2.schema=string"
> and while creating context in getContext 
> config for schema1 will resolved to  "schema=int"
> and for  schema2 it will  be "schema=string"



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message