Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@streams.incubator.apache.org
Date: Mon, 21 Nov 2016 11:48:26 -0600
From: sblackmon <sblackmon@apache.org>
To: dev@community.apache.org
Cc: "=?utf-8?Q?dev=40streams.incubator.apache.org?="
 <dev@streams.incubator.apache.org>
Message-ID: <etPan.5833336a.61ca6d68.252b@apache.org>
Subject: Social Media Metrics using Apache stack
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="5833336a_25905af2_252b"
archived-at: Mon, 21 Nov 2016 17:48:35 -0000

--5833336a_25905af2_252b
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hello ComDev,

The Streams podling has been brainstorming ways to increase awareness of =
the project and it=E2=80=99s capabilities. =C2=A0We=E2=80=99ve also been =
working to make it easier to get started as a user, without starting the =
journey by downloading JDK Maven and friends. =C2=A0Using the software to=
 provide benefit to the =46oundation seems like a good thing to try.

One use case for Streams is to build personal or organizational datasets =
of social media profiles and content for internal development and analysi=
s, using the technologies and tools you and your organization prefer,=C2=A0=
rather than those provided by the upstream system.

I took the liberty of creating a few Zeppelin notebooks which collect Apa=
che project profiles and posts, normalize them to activity streams format=
, and interact with them using spark data frames.

The notebooks are currently hosted in my zeppelinhub account, which anyon=
e with the link below can access. =C2=A0

https://www.zeppelinhub.com/viewer/notebooks/bm90ZTovL3N0ZXZlYmxhY2ttb24v=
YXBhY2hlLXplcHBlbGluLWRhc2hib2=46yZC84YjQ5YmY3MWIxYTU0ZTE2YjlkMDQyMTliMzN=
lMjQzYS9ub3RlLmpzb24

https://www.zeppelinhub.com/viewer/notebooks/bm90ZTovL3N0ZXZlYmxhY2ttb24v=
YXBhY2hlLXplcHBlbGluLWRhc2hib2=46yZC9lNzQzZjRkZGVkMGY0YjA3YTkzZTQ2NW=46kY=
jU2ZTQxOS9ub3RlLmpzb24

https://www.zeppelinhub.com/viewer/notebooks/bm90ZTovL3N0ZXZlYmxhY2ttb24v=
YXBhY2hlLXplcHBlbGluLWRhc2hib2=46yZC8zZmQ3M2Y1OWEzOGE0YmM2Yj=46kMGM4MzBkN=
TczZDU0Mi9ub3RlLmpzb24

If this group sees potential benefit, I=E2=80=99d be happy to work to set=
 them up for use by anyone at Apache in a dedicated Zeppelin deployment a=
nd take the lead on maintaining them going forward.

In any case we=E2=80=99d appreciate any feedback on what could would make=
 this prototype more valuable..

Background on Streams:

Apache Streams (incubating) unifies a diverse world of digital profiles a=
nd online activities into common formats and vocabularies, and makes thes=
e datasets accessible across a variety of databases, devices, and platfor=
ms for streaming, browsing, search, sharing, and analytics use-cases.

Streams contains libraries and patterns for specifying, publishing, and i=
nter-linking schemas, and assists with conversion of activities (posts, s=
hares, likes, follows, etc.) and objects (profiles, pages, photos, videos=
, etc.) between the representation, format, and encoding preferred by sup=
ported data providers (Twitter, Instagram, etc.), and storage services (C=
assandra, Elasticsearch, HBase, HD=46S, Neo4J, etc.)

In theory pretty much any JSON or XML API which uses a =22look-up by ID a=
nd type=E2=80=9D model can be co-erced into collections of activity-strea=
ms normalized profiles and posts - systems such as GitHub, JIRA, MeetUp c=
ould be added to the roadmap and have notebooks created once those provid=
ers are built.
--5833336a_25905af2_252b--