hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Gonzalez <zlgonza...@yahoo.com.INVALID>
Subject Re: Reading video files from HDFS using OpenCV
Date Sun, 24 Jul 2016 21:19:06 GMT
<p dir="ltr">In a manner of speaking. I would imagine that you would like to take advantage
of resource management that comes with yarn. If you're planning to make this a product that
your customers can submit jobs to do some image processing, then you're better off creating
a yarn application master that would handle taking files from Hdfs, localize then process
as a local process.</p>
<p dir="ltr"> If not and more of a one-time effort, then it's as simple as:</p>
<p dir="ltr">hadoop fs -copyToLocal hdfspath localpath</p>
<p dir="ltr">You can then do whatever you want to do with this local file.</p>
<p dir="ltr">Thanks,<br>
Ron</p>
<div class="gmail_extra"><br><div class="gmail_quote">On Jul 24, 2016 12:28
PM, Madhav Sharan &lt;msharan@usc.edu&gt; wrote:<br type="attribution"><blockquote
class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div
dir="ltr"><div>Hi Ron, Thanks for replying.</div><div><br /></div>Unfortunately
I could not find a VideoCapture method accepting stream input.<div><br /></div><div>I
will look into second option. Will it be similar to copying file from hdfs to a tmp directory
and then using tmp file?</div><div><br clear="all" /><div><div><div
dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div
dir="ltr">--</div><div dir="ltr"><div><span style="font-size:12.8px">Madhav
Sharan</span><br /></div><div><div><br /></div></div></div></div></div></div></div></div></div></div>
<br /><div class="elided-text">On Sun, Jul 24, 2016 at 12:08 PM, Ron&#39;s
Yahoo! <span dir="ltr">&lt;<a href="mailto:zlgonzalez&#64;yahoo.com">zlgonzalez&#64;yahoo.com</a>&gt;</span>
wrote:<br /><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div
style="word-wrap:break-word">You can create a stream for an HDFS file and have OpenCV read
from the stream.<div>Other option is to create a yarn job and you can resource localize
the HDFS file so you can run OpenCV just like any other application.</div><div><br
/></div><div>Thanks,</div><div>Ron</div><div><div><div><br
/><div><blockquote><div>On Jul 24, 2016, at 12:03 PM, Madhav Sharan &lt;<a
href="mailto:msharan&#64;usc.edu">msharan&#64;usc.edu</a>&gt; wrote:</div><br
/><div><div dir="ltr">Hi hadoop users,<div><br /></div><div>I
need some help in reading video files from HDFS. I am processing videos using OpenCV and opencv
is not able to read video file from hdfs. I tried prefixing path with &#34;hdfs://&#34;
but it still does not works. Is there a way we can directly read from HDFS using opencv? </div><div><br
/></div><div><br /></div><div>[0] Video capture code -  <a
href="https://urldefense.proofpoint.com/v2/url?u&#61;https-3A__github.com_smadha_pooled-5Ftime-5Fseries_blob_master_src_main_java_gov_nasa_jpl_memex_pooledtimeseries_PoT.java-23L441&amp;d&#61;DQMFAg&amp;c&#61;clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&amp;r&#61;DhBa2eLkbd4gAFB01lkNgg&amp;m&#61;9cu9u3AmZ3FP8PejFB3CInJW4mTPFq1H-B3USMRuP8E&amp;s&#61;Mr2L2gdpcCZNngViWbS0XOGfcb8oP98lDKJLzJoyMXE&amp;e&#61;">https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/PoT.java#L441</a><br
clear="all" /><div><div><div dir="ltr"><div><div dir="ltr"><div
dir="ltr"><div dir="ltr"><div dir="ltr"><br /></div><div dir="ltr"><div>[1]
TMP file solution - <a href="https://urldefense.proofpoint.com/v2/url?u&#61;https-3A__github.com_smadha_pooled-5Ftime-5Fseries_blob_master_src_main_java_gov_nasa_jpl_memex_pooledtimeseries_OpticalTimeSeries.java-23L66&amp;d&#61;DQMFAg&amp;c&#61;clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&amp;r&#61;DhBa2eLkbd4gAFB01lkNgg&amp;m&#61;9cu9u3AmZ3FP8PejFB3CInJW4mTPFq1H-B3USMRuP8E&amp;s&#61;Xas8nIDCouC0_Qx2RiQDDV9okicT_KWd2ZPV6gTEWzg&amp;e&#61;">https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/OpticalTimeSeries.java#L66</a><br
/></div><div><br /></div></div><div dir="ltr">--</div><div
dir="ltr"><div><span style="font-size:12.8px">Madhav Sharan</span><br
/></div><div><div><br /></div></div></div></div></div></div></div></div></div></div>
</div></div>
</div></blockquote></div><br /></div></div></div></div></blockquote></div><br
/></div></div>
</blockquote></div><br></div>
Mime
View raw message