nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Lim <Keith....@ds-iq.com>
Subject Apache Nifi and user group.
Date Sat, 09 Apr 2016 00:06:51 GMT
I just found out about Apache Nifi and am extremely existed of the possibility of using it
for our end to end processing from acquiring external data feeds, ingesting (ETL) and finally
landing in Hadoop (HBASE and HDFS) for used to data scientists.

I want to find out how this can be used to support complicated processes for those processes
that are not easily configurable using the built-in processors, e.g.

This case is extracting google places,

Breaking down the continental US into small squares, querying each square using radar search
of latitude and longitude, if the call returns the max locations meaning possibly there are
more locations in this query, the logic will need to drill down further by dividing into 4
smaller radar searches.   This is iterated until all locations in continental US is covered.

Now, I can build a workflow by stitching various processors and possibly writing some custom
code along the way.  Is this how I should proceed?   Or I would need to build a single processor
to encapsulate the above.

Any suggestion and pointing me to a user group would be appreciated.

Thanks,
Keith


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message