streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan B <pub...@ryanb.org>
Subject Re: hello from granary
Date Tue, 27 Sep 2016 21:20:27 GMT
On Tue, Sep 27, 2016 at 10:37 AM, sblackmon <sblackmon@apache.org> wrote:

>
> It would be really cool if we could write and apply (and share) JSON
> document conversions using a cross-language DSL.
>
> We’ve done a bit of that using HOCON (see links below) but there’s only so
> much that can do, and anything beyond that we have to write code - so
> nearly all converters wind up written with code.
>

interesting! agreed, declarative schema mapping tools like that have been
around forever, especially in academia, and they can definitely work and be
more maintainable than code! you're right though, they can't usually do the
whole thing on their own, so we often need to fall back to code for some
things.

keep us posted if you end up using HOCON or something similar more broadly!



> https://github.com/apache/incubator-streams/blob/master/
> streams-components/streams-converters/src/test/java/org/
> apache/streams/converter/test/HoconConverterTest.java
>
> https://github.com/apache/incubator-streams/tree/master/
> streams-components/streams-converters/src/test/resources
>
> Best,
> Steve
>
> On June 15, 2016 at 12:16:07 AM, Ryan B (public@ryanb.org) wrote:
>
> hi steve, all! i just discovered apache streams. it looks great! can't
> wait
> to dig in more.
>
> i'm the main developer on a similar project called granary:
>
> https://github.com/snarfed/granary
> https://granary-demo.appspot.com
>
> it has a lot of the same functionality as streams. it's focused on social
> networking data, so it doesn't do databases or data processing pipelines
> like streams, but it supports writing to silo APIs as well as reading
> them,
> and it supports a few formats streams doesn't: Flickr, Atom, microformats
> HTML and JSON.
>
> we've spent a fair amount of effort adapting to silo changes over time, eg
> Facebook's 2.x API, reconstructing Twitter @-reply chains, scraping
> Instagram's HTML now that they've disabled their API for many apps. I'd be
> happy to compare notes if you're interested.
>
> keep up the good work!
>
> --
> https://snarfed.org/
>
>


-- 
https://snarfed.org/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message