fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Walch <mwa...@apache.org>
Subject Re: Spark + Fluo
Date Tue, 13 Mar 2018 14:51:13 GMT
Hi Alan!

No worries about emailing the list. Your email is actually helpful. It's
made it clear that we need to improve our troubleshooting docs. There is
some documentation at the link below but we could use more:

https://fluo.apache.org/docs/fluo/1.2/administration/manage-applications

What version of Fluo are you running?
Have you run the 'fluo wait' command?  Do you have outstanding
notifications?  Is the number decreasing or staying the same?
Have you checked the logs? Is your observer code throwing any exceptions?
Another thing that may help debugging is to set up metrics:
https://fluo.apache.org/docs/fluo/1.2/administration/metrics

-Mike

On Tue, Mar 13, 2018 at 7:11 AM, Alan Camillo <alan@blueshift.com.br> wrote:

> Hey fellas!
> Sorry to demand so much from you. But we are really trying to put Fluo to
> work here and we are facing some issues.
>
> Recently we decided to use Apache Spark to star the process to ingest 300
> millions of lines with 62 columns each.
>
> We study this:
> https://fluo.apache.org/blog/2016/12/22/spark-load/ carefully and decided
> to implement the first strategy described. Executing load transactions in
> Spark
>
> In that way we could reuse the code we build for the application
> transactions. But...
> But we are not going well. Fluo stop to insert after a while and we are
> not able to know why.
> We tried to adjust the loader queue and size to see what happens but
> nothing really helps.
> I need a help to debug Fluo and understanding what’s going on. Can someone
> point me a direction?
>
> Thanks!
> Alan Camillo
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message