hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vidhya Parvathy R (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4094) Specific file type bulk Transfer into HDFS to a specified HDFS location with a track of the transfer number
Date Sun, 21 Oct 2012 06:42:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vidhya Parvathy R  updated HDFS-4094:
-------------------------------------

    Attachment: hadoopBulkTransfer.txt

Attaching a txt file. Save it as an sh file  in the directory where your input files resides.
Run it from the console as  ./<filename>.sh <file type> <hdfs destination>
                
> Specific file type bulk Transfer into HDFS to a specified HDFS location with a track
of the transfer number 
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4094
>                 URL: https://issues.apache.org/jira/browse/HDFS-4094
>             Project: Hadoop HDFS
>          Issue Type: Wish
>          Components: scripts
>    Affects Versions: 3.0.0
>         Environment: Unix
>            Reporter: Anurag G Vyas
>             Fix For: 3.0.0
>
>         Attachments: hadoopBulkTransfer.txt
>
>
> Need a script for a bulk transfer process into HDFS in such a way that the script must
be able to identify a specific file type and move only  that file type into the specified
HDFS location.
> For example : Say I have a local directory called user/dir . In it I have txt files such
as anurag123 , anurag234, vyas678 , ganesh345 , anurag277 , vyas345 ganesh789 etc. The script
must take 2 inputs. One is the file type , say anurag. And the other an HDFS location. Then
it must move all the files which has the name anurag in the directory to the specified HDFS
location. 
> Also it would be good if the script could keep track of the number of files it moved
to HDFS by saving the details in a new file either in HDFS or in the local directory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message