flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Khurana <skhurana...@gmail.com>
Subject Re: Use Flume to Stream Real Time Video into HDFS
Date Tue, 02 Sep 2014 08:52:56 GMT
You may try  writing a custom source which can use a gstreamer pipeline
(which you will build) and send data to channel etc .


On Tue, Sep 2, 2014 at 1:51 PM, Arthur.hk.chan@gmail.com <
arthur.hk.chan@gmail.com> wrote:

> Hi,
>
> Is there any sample or tutorial about streaming Real Time Videos into
> HDFS?
>
> I am trying to use net cat but it might not be a good choice (I am new to
> flume).
>
> Regards
> Arthur
>
> (below is my test configuration)
> ### A single-node Flume configuration
> ### Name the components on this agent
> a3.sources = r1
> a3.channels = c1
> a3.sinks = k1
> ### Describe/configure the source
> a3.sources.r1.type = netcat
> a3.sources.r1.bind = localhost
> a3.sources.r1.port = 44444
> a3.sources.r1.channels = c1
> ### Describe the sink
> ### Use a channel which buffers events in memory
> a3.channels.c1.type                                     = memory
> a3.channels.c1.capacity                                 = 10000
> a3.channels.c1.transactionCapacity                      = 10000
> ### Bind the source and sink to the channel
> a3.sinks.k1.type = hdfs
> a3.sinks.k1.channel = c1
> a3.sinks.k1.hdfs.path = hdfs://hdfs/flume_test
> a3.sinks.k1.hdfs.filePrefix = events-
> a3.sinks.k1.hdfs.round = true
> a3.sinks.k1.hdfs.roundValue = 10
> a3.sinks.k1.hdfs.roundUnit = minute
>
>
>


-- 
Thanks and regards
Sandeep Khurana

Mime
View raw message