hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhagaban Khatai <email.bhaga...@gmail.com>
Subject Request - How to copy the file from MR after each run -
Date Mon, 20 Oct 2014 08:18:55 GMT
Hi Users,

Need a immediate help on this. Each time my MR runs it will create below
directory structure but necessary it will create all file1,file2 and file2,
even sometime it will create only file1/file2 etc. And all the below folder
structure will be dynamically created in MR not by manually, only I need to
give Temp (high level folder in Oozie) and each time my job run it will
delete Temp and re-create again for other file.

Temp  (each file Temp will generate and it will automatic re-created after
each run in MR and we need to move the below files into diff folder called
Actaul)

     staging
            file1 <-- Dymanically created by MR job
     reject
           rejectedA
                file1a <-- Dymanically created by MR job
           rejectedB
                 file1b<-- Dymanically created by MR job

final strucure of my staging folder.
Actual
      staging
            file1a <-- Dymanically created by MR job
            file1b <-- Dymanically created by MR job

     reject
           rejectedA
                file2a <-- Dymanically created by MR job
                file2a <-- Dymanically created by MR job
           rejectedB
                 file3b <-- Dymanically created by MR job
                 file2b <-- Dymanically created by MR job

Please advice how to set-up in Oozie to perform this task.

Mime
View raw message