fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Camillo <a...@blueshift.com.br>
Subject Spark + Fluo
Date Tue, 13 Mar 2018 11:11:20 GMT
Hey fellas!
Sorry to demand so much from you. But we are really trying to put Fluo to work here and we
are facing some issues.

Recently we decided to use Apache Spark to star the process to ingest 300 millions of lines
with 62 columns each.

We study this:
https://fluo.apache.org/blog/2016/12/22/spark-load/ carefully and decided to implement the
first strategy described. Executing load transactions in Spark

In that way we could reuse the code we build for the application transactions. But...
But we are not going well. Fluo stop to insert after a while and we are not able to know why.
We tried to adjust the loader queue and size to see what happens but nothing really helps.
I need a help to debug Fluo and understanding what’s going on. Can someone point me a direction?

Thanks!
Alan Camillo

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message