streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sblackmon <sblack...@apache.org>
Subject Re: hello from granary
Date Tue, 27 Sep 2016 17:37:27 GMT
Ryan,

Thanks for the note.  Good to know that another effort like this is happening in the python
world.

It would be really cool if we could write and apply (and share) JSON document conversions
using a cross-language DSL.

We’ve done a bit of that using HOCON (see links below) but there’s only so much that can
do, and anything beyond that we have to write code - so nearly all converters wind up written
with code.

https://github.com/apache/incubator-streams/blob/master/streams-components/streams-converters/src/test/java/org/apache/streams/converter/test/HoconConverterTest.java

https://github.com/apache/incubator-streams/tree/master/streams-components/streams-converters/src/test/resources

Best,
Steve

On June 15, 2016 at 12:16:07 AM, Ryan B (public@ryanb.org) wrote:
hi steve, all! i just discovered apache streams. it looks great! can't wait  
to dig in more.  

i'm the main developer on a similar project called granary:  

https://github.com/snarfed/granary  
https://granary-demo.appspot.com  

it has a lot of the same functionality as streams. it's focused on social  
networking data, so it doesn't do databases or data processing pipelines  
like streams, but it supports writing to silo APIs as well as reading them,  
and it supports a few formats streams doesn't: Flickr, Atom, microformats  
HTML and JSON.  

we've spent a fair amount of effort adapting to silo changes over time, eg  
Facebook's 2.x API, reconstructing Twitter @-reply chains, scraping  
Instagram's HTML now that they've disabled their API for many apps. I'd be  
happy to compare notes if you're interested.  

keep up the good work!  

--  
https://snarfed.org/  

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message