apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyanka Gugale <priya...@datatorrent.com>
Subject Re: Mentoring Apache Apex projects in Google Summer of Code
Date Tue, 23 Feb 2016 09:45:00 GMT
I am adding one more project to the list. *Integration of deeplearning4j
with Apex.*

*What is deeplearning4j?*
Deeplearning4j is an open source deep learning library written for Java and
and a computing framework with wide support for deep learning algorithms.
Deeplearning4j includes implementations of the restricted Boltzmann
machine, deep belief net, deep autoencoder, stacked denoising autoencoder
and recursive neural tensor network, as well as word2vec, doc2vec and
GloVe. These algorithms all include distributed parallel versions that
integrate with Hadoop. *Deep learning excels at identifying patterns in
unstructured data*, which most people know as media such as images, sound,
video and text.

On Mon, Feb 22, 2016 at 7:04 PM, Priyanka Gugale <priyanka@datatorrent.com>
wrote:

> Hi Justin,
>
> This is quiet interesting that we as Apache community can mentor and each
> project doesn't have to be registered individually. Also using jira to
> track everything is more open and interactive. However before receiving
> your mail we had already registered. So may be can't do it as apache
> community this time.
> We can still following similar procedure as apache is already doing so
> that we can be consistent. Thanks for sharing this.
>
> -Priyanka
>
> On Fri, Feb 19, 2016 at 10:09 AM, Pradeep A. Dalvi <prad@apache.org>
> wrote:
>
>> I've following idea in my mind for sometime in Networking & Network
>> Security Domain, which I shall be able to mentor.
>>
>> *Working with Network Capture files in Real Time Streaming*
>>
>> Network packet data is captured and stored in Network Capture Files. Some
>> of most common and widely used Network Capture file formats are PCAP &
>> PCAPng. Network packeted data can be used for various purposes from
>> Analytics, Deep Packet Inspection to Network Security.
>> Here we want to Read from PCAP & PCAPng files and Write Packets into PCAP
>> &
>> PCAPng format. These Network Capture files are stored in binary format
>> e.g.
>> PCAP file format:
>> [ PCAP File Header ] [ Packet Header ] [ Packet ] [ Packet Header ] [
>> Packet ] … [ Packet Header ] [ Packet ]
>> Each Packet Header contains Actual Packet Length as well as Captured
>> Packet
>> Length (=< Actual Packet Length). Captured Packet Length is used to
>> determine the offset of the next Packet.
>>
>> On Thu, Feb 18, 2016 at 6:51 PM, HARSH PATHAK <harshnpathak@gmail.com>
>> wrote:
>>
>> > Ok ,Thank You.
>> >
>> > On Thu, Feb 18, 2016 at 6:44 PM, Priyanka Gugale <
>> priyanka@datatorrent.com
>> > >
>> > wrote:
>> >
>> > > Hi Hash,
>> > >
>> > > You still have to work on details. We would help you doing that. I
>> would
>> > > recommend you first start thinking of some specific usecase so that it
>> > > would be clear.
>> > > For students proposals there still still some time. So you don't have
>> to
>> > > hurry so much for this.
>> > >
>> > > -Priyanka
>> > >
>> > > On Thu, Feb 18, 2016 at 6:35 PM, HARSH PATHAK <harshnpathak@gmail.com
>> >
>> > > wrote:
>> > >
>> > > > *By Apex-IOT App- Aim to deliver at the end -->*Real time data
that
>> > Apex
>> > > > receives from the sensors can easily be moniterized. Also some Data
>> > > > Analytics on the captured data can be done as Big Data can be
>> Handled
>> > by
>> > > > Apex Architecture .
>> > > > Timelines I am not sure about. But it is an achivable task within
>> > > 3-months
>> > > > of time
>> > > >
>> > > > On Thu, Feb 18, 2016 at 6:20 PM, Atri Sharma <atri.jiit@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Thats fine.
>> > > > >
>> > > > > It would be helpful if you could elaborate on what you aim to
>> deliver
>> > > at
>> > > > > the end, timelines etc.
>> > > > >
>> > > > > On Thu, Feb 18, 2016 at 6:19 PM, HARSH PATHAK <
>> > harshnpathak@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > I have worked few IOT Projects before like Smart Dustbin
and
>> Smart
>> > > Door
>> > > > > > lock.
>> > > > > > These Projects Included Hardware and Software Interface
, which
>> > > > involved
>> > > > > > technologies like IBM BlueMix,Arduino,ESP,BeagleBoard.
>> > > > > > On other hand i have developed a Android app and a website
on
>> jsp.
>> > > > > > Most of all the Projects were done as a Team projects and
I was
>> one
>> > > of
>> > > > > them
>> > > > > > But , I am totally new to Apache Apex and Apache Hadoop.
>> > > > > >
>> > > > > > On Thu, Feb 18, 2016 at 6:10 PM, Priyanka Gugale <
>> > > > > priyanka@datatorrent.com
>> > > > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Atri, can you tell me your previous organization/community
>> name
>> > > with
>> > > > > > which
>> > > > > > > you participated in GCoS?
>> > > > > > >
>> > > > > > > -Priyanka
>> > > > > > >
>> > > > > > > On Thu, Feb 18, 2016 at 6:05 PM, Atri Sharma <
>> > atri.jiit@gmail.com>
>> > > > > > wrote:
>> > > > > > >
>> > > > > > > > Harsh,
>> > > > > > > >
>> > > > > > > > Please elaborate on the points. Specifically,
deliverables,
>> > > > timeline,
>> > > > > > > your
>> > > > > > > > coding experience and a high level design.
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > >
>> > > > > > > > Atri
>> > > > > > > >
>> > > > > > > > On Thu, Feb 18, 2016 at 6:03 PM, HARSH PATHAK
<
>> > > > > harshnpathak@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Abstract
>> > > > > > > > >
>> > > > > > > > > The *Internet of Things* is the network of
physical
>> > > > > objects—devices,
>> > > > > > > > > vehicles,
>> > > > > > > > >  buildings and other items which are embedded
with
>> > electronics,
>> > > > > > > software,
>> > > > > > > > > sensors,
>> > > > > > > > >  and network connectivity, which enables
these objects to
>> > > collect
>> > > > > and
>> > > > > > > > > exchange data.
>> > > > > > > > > *Apex* is a YARN-native platform that unifies
stream and
>> > batch
>> > > > > > > > processing.
>> > > > > > > > > It
>> > > > > > > > > processes big data in-motion in a way that
is highly
>> > scalable,
>> > > > > highly
>> > > > > > > > > performant, fault tolerant, stateful, secure,
distributed,
>> > and
>> > > > > easily
>> > > > > > > > > operable.
>> > > > > > > > > *Idea *is to build an Integrated Apex-Iot
App that can
>> truly
>> > > > define
>> > > > > > the
>> > > > > > > > the
>> > > > > > > > > need of IOT and Real time Apex processing.This
will lead
>> to
>> > > most
>> > > > > > > accurate
>> > > > > > > > > results in a secured way. So, no data would
be stored in
>> IOT
>> > > > > devices
>> > > > > > > > rather
>> > > > > > > > > it  will  just redirect it to Apex Distributed
>> > System.Eg.Smart
>> > > > > > Dustbin
>> > > > > > > > for
>> > > > > > > > > a State or Country, Traffic co-ordination
in city and its
>> > > futher
>> > > > > > > analysis
>> > > > > > > > > for predictions,Weather Stations,Factories
and Power
>> Plants
>> > for
>> > > > > > saftey
>> > > > > > > > etc.
>> > > > > > > > > *Future Scope* it can be further integrated
with some RTOS
>> > like
>> > > > > > > Google's
>> > > > > > > > > Brillo (especially designed for IOT Apps).
>> > > > > > > > >
>> > > > > > > > > -Harsh Pathak
>> > > > > > > > > Thank You.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Thu, Feb 18, 2016 at 5:39 PM, Priyanka
Gugale <
>> > > > > > > > priyanka@datatorrent.com
>> > > > > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > You can put it on this mail thread.
>> > > > > > > > > >
>> > > > > > > > > > -Priyanka
>> > > > > > > > > >
>> > > > > > > > > > On Thu, Feb 18, 2016 at 5:36 PM, HARSH
PATHAK <
>> > > > > > > harshnpathak@gmail.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Yes I am a (Third year B.E.)Student,
I recently
>> attended
>> > > the
>> > > > > > Apache
>> > > > > > > > > Apex
>> > > > > > > > > > > workshop at PICT,Pune.
>> > > > > > > > > > >
>> > > > > > > > > > > So do I need to mail you the abstract.
>> > > > > > > > > > >
>> > > > > > > > > > > On Thu, Feb 18, 2016 at 5:22 PM,
Priyanka Gugale <
>> > > > > > > > > > priyanka@datatorrent.com
>> > > > > > > > > > > >
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > We have to submit ideas by
tomorrow, tomorrow is
>> last
>> > > day.
>> > > > > Here
>> > > > > > > is
>> > > > > > > > > one
>> > > > > > > > > > > > example which I have written,
>> > > > > > > > > > > >
>> > > > > > > > > > > > *Apache Apex: Using rules
engine for real time
>> > analytics
>> > > on
>> > > > > > > Hadoop*
>> > > > > > > > > > > > *Abstract*
>> > > > > > > > > > > > Apex is a YARN-native platform
that unifies stream
>> and
>> > > > batch
>> > > > > > > > > > processing.
>> > > > > > > > > > > It
>> > > > > > > > > > > > processes big data in-motion
in a way that is highly
>> > > > > scalable,
>> > > > > > > > highly
>> > > > > > > > > > > > performant, fault tolerant,
stateful, secure,
>> > > distributed,
>> > > > > and
>> > > > > > > > easily
>> > > > > > > > > > > > operable. Most of real time
analytics requires data
>> > > > > evaluation
>> > > > > > > > > against
>> > > > > > > > > > > set
>> > > > > > > > > > > > of rules. There are plenty
of rules engine
>> frameworks
>> > in
>> > > > > market
>> > > > > > > > which
>> > > > > > > > > > > helps
>> > > > > > > > > > > > to implement rules evaluation
logic. Drools is one
>> of
>> > the
>> > > > > > > Business
>> > > > > > > > > > Rules
>> > > > > > > > > > > > Management System (BRMS) solution.
Idea is to
>> integrate
>> > > > > drools
>> > > > > > > > engine
>> > > > > > > > > > > with
>> > > > > > > > > > > > Apache Apex to do real time
rules evaluation against
>> > > > incoming
>> > > > > > > > events
>> > > > > > > > > to
>> > > > > > > > > > > > achieve real time analysis.
>> > > > > > > > > > > > If you are a student, I would
also encourage you to
>> > > > > participate
>> > > > > > > in
>> > > > > > > > > this
>> > > > > > > > > > > > program as a student and work
on one of the
>> projects, I
>> > > > would
>> > > > > > > > > > recommend a
>> > > > > > > > > > > > Apex project of course :)
>> > > > > > > > > > > >
>> > > > > > > > > > > > -Priyanka
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Thu, Feb 18, 2016 at 5:11
PM, HARSH PATHAK <
>> > > > > > > > > harshnpathak@gmail.com>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Sure, just tell me where
and how to begin with.
>> And
>> > > > > timeline
>> > > > > > is
>> > > > > > > > for
>> > > > > > > > > > > > Project
>> > > > > > > > > > > > > completion or Project
idea completion?
>> > > > > > > > > > > > > -Harsh
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Thu, Feb 18, 2016
at 4:53 PM, Priyanka Gugale <
>> > > > > > > > > > > > priyanka@datatorrent.com
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi Harsh,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > That's a great idea.
Would you like to come up
>> > with a
>> > > > > > project
>> > > > > > > > > idea
>> > > > > > > > > > > for
>> > > > > > > > > > > > > > same. The project
idea should be clear which has
>> > > little
>> > > > > > > > technical
>> > > > > > > > > > > depth
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > describe what should
be the end result. The
>> project
>> > > > > > timeline
>> > > > > > > > is 3
>> > > > > > > > > > > > months.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > -Priyanka
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Thu, Feb 18,
2016 at 4:48 PM, HARSH PATHAK <
>> > > > > > > > > > > harshnpathak@gmail.com>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Hi,
>> > > > > > > > > > > > > > > Internet of
Things can be the target area.
>> Since
>> > > > stream
>> > > > > > > flow
>> > > > > > > > of
>> > > > > > > > > > > data
>> > > > > > > > > > > > > > > through sensors
is growing immensely surely
>> > taking
>> > > > the
>> > > > > > > > picture
>> > > > > > > > > if
>> > > > > > > > > > > > > > Big-data
>> > > > > > > > > > > > > > > ,which also
requires quick responsiveness.
>> Apache
>> > > > Apex
>> > > > > > > could
>> > > > > > > > be
>> > > > > > > > > > the
>> > > > > > > > > > > > > > perfect
>> > > > > > > > > > > > > > > fit.For eg
.Smart Dustbin in which sensors
>> > > > continuously
>> > > > > > > read
>> > > > > > > > > and
>> > > > > > > > > > > send
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > data and also
requires a quick action ,but
>> when
>> > > Smart
>> > > > > > > > dustbins
>> > > > > > > > > > are
>> > > > > > > > > > > > all
>> > > > > > > > > > > > > > over
>> > > > > > > > > > > > > > > the city or
state it will surely generate
>> > Big-data
>> > > > and
>> > > > > > this
>> > > > > > > > > could
>> > > > > > > > > > > be
>> > > > > > > > > > > > > > > handled by
Apache Apex.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Wed, Feb
17, 2016 at 11:55 AM, Priyanka
>> > Gugale <
>> > > > > > > > > > > > > > > priyanka@datatorrent.com>
>> > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Hi,
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Google
sponsors students projects for open
>> > source
>> > > > > > project
>> > > > > > > > > > > > > development.
>> > > > > > > > > > > > > > > Any
>> > > > > > > > > > > > > > > > company
who works on open source code base
>> can
>> > > > > > > participate
>> > > > > > > > as
>> > > > > > > > > > > > Mentor
>> > > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > > can provide
project ideas which will be
>> worked
>> > on
>> > > > by
>> > > > > > > > > students.
>> > > > > > > > > > As
>> > > > > > > > > > > > > > Apache
>> > > > > > > > > > > > > > > > Apex project
we should submit some ideas
>> under
>> > > this
>> > > > > > > > program.
>> > > > > > > > > > > > > > > > Please
share your ideas on this mail
>> thread, so
>> > > > that
>> > > > > we
>> > > > > > > can
>> > > > > > > > > > > > register
>> > > > > > > > > > > > > > with
>> > > > > > > > > > > > > > > > google
as a mentors.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > *Note:
Please share your ideas by 18th
>> > February,
>> > > as
>> > > > > > 19th
>> > > > > > > is
>> > > > > > > > > the
>> > > > > > > > > > > > last
>> > > > > > > > > > > > > > day
>> > > > > > > > > > > > > > > > for registration.*
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > -Priyanka
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > Regards,
>> > > > > > > >
>> > > > > > > > Atri
>> > > > > > > > *l'apprenant*
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Regards,
>> > > > >
>> > > > > Atri
>> > > > > *l'apprenant*
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message