metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yerex, Tom" <tom.ye...@ubc.ca>
Subject RE: Development Activity has dropped to effectively 0, what should we do?
Date Wed, 08 Apr 2020 23:38:16 GMT
To me Metron is big and broad in the scope of technology required to get it running. If things
were more modular that would go a long way to reducing the learning curve or at least putting
it into smaller bites (and it might encourage more people to get involved).

If the UI were an add-on module in another project, it would have made it easier for me and
it could also encourage my hypothetical buddy who is a web developer expert to get involved
since he could focus on the web-ui module instead of trying to tackle all the other pieces
that are probably not part of his bailiwick.

Stellar is very intriguing, maybe that is not unique to Metron? The architecture of Metron
with respect to parsing, enriching, etc., makes a lot of sense to anyone I talk with. These
two aspects of Metron seem like standout examples that make for a powerful platform to develop
on.

Thanks for continuing this discussion,

Tom.


On 2020-04-08 15:32:46-07:00 Casey Stella wrote:

As far as I know there is no minimum bar of development activity to keep a project open. 
I think we would all be grateful for any investment that you or your organization would want
to make.
It also occurs to me that your observation is absolutely spot on: we have a LOT of moving
parts.
I see some deficiencies here:

  *   We depend on a lot of the various hadoop ecosystem projects and they have to work together
very precisely:
     *   This makes for a system that is hard to install.
     *   This also makes for a system which is hard to tune/manage
  *   We have a large surface area of coverage
     *   We have an installer, backend system and front-end UI, which stretches our developers
a bit thin, especially since there isn't even interest in those systems

Perhaps a reconsideration of the scope and technologies that we use would be merited?  If
we were to decide to, for instance:

  *   Consolidate scope: focus on a viable backend/API rather than a UI
  *   Consolidate technology: reposition ourselves on top of Spark as a consolidated streaming/batch
system
  *   Make SQL our external interface: write out to parquet + the Hive metastore and let users
pin up presto tables or hive tables as they see fit

This might reduce some of our surface area and make it more viable to get started?
Anyway, just some thoughts.
Casey

On Wed, Apr 8, 2020 at 6:20 PM Yerex, Tom <tom.yerex@ubc.ca<mailto:tom.yerex@ubc.ca>>
wrote:
Hi Casey,

I'm new here and new to contributing to an open source project. Thus far my contribution has
been questions, however the steep learning curve has had me working to understand all the
moving parts for the last 18 months and I see that as a big investment by my organization.

What is a level that would be viable?

If my organization were to contribute I don't know that it would be soon enough or at the
volume that is recognized as viable, which is why I ask the question.


On 2020-04-08 15:05:51-07:00 Casey Stella wrote:

Hi all,

When composing the board report today, I realized that we have effectively
had no development in the last quarter on this project.  Please be aware
that I say this without a shred of blame or judgement (especially so
considering I have not contributed in a long time).  That being said, I
would like to pose the question to the community:

Do we feel that this project is viable?  If so, how are we going to spur
new contributions?  If not, then should we begin the process to fold the
project?


Best,

Casey


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message