apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyanka Gugale <pri...@apache.org>
Subject Re: Information Needed
Date Fri, 12 Aug 2016 05:13:59 GMT
Hi,

We have parsers examples at:
https://github.com/DataTorrent/examples/tree/master/tutorials/parser (The
csv parser example will be available in couple of days)
Check if this can help you.

-Priyanka

On Thu, Aug 11, 2016 at 7:05 PM, Mukkamula, Suryavamshivardhan (CWM-NR) <
suryavamshivardhan.mukkamula@rbc.com> wrote:

> Thanks Priyanka, I would try with caching in my upcoming project.
>
>
>
> Do you have any sample that can be helpful to me?
>
>
>
> Regards,
>
> Surya Vamshi
>
>
>
> *From:* Priyanka Gugale [mailto:priyag@apache.org]
> *Sent:* 2016, August, 11 12:53 AM
> *To:* users@apex.apache.org
> *Subject:* Re: Information Needed
>
>
>
> IMO, it's okay to keep the files in HDFS, at startup you can load them in
> memory which will be one time activity in application lifetime. Considering
> they are format/config files, it won't be difficult to cache it in memory
> of your current operator (no need to caching operator separately).
>
>
>
> -Priyanka
>
>
>
> On Wed, Aug 10, 2016 at 7:06 PM, Mukkamula, Suryavamshivardhan (CWM-NR) <
> suryavamshivardhan.mukkamula@rbc.com> wrote:
>
> Hello Team,
>
>
>
> I have a use case where I am trying to read number of input files with
> different formats(may be 100 source systems) and I have to parse each
> record.
>
>
>
> To achieve this we need to have metadata(configuration files) of the input
> file structures. Can you please let me know that, having this metadata on
> HDFS is good or having the metadata in some sql/oracle database is good?
>
>
>
> Assume that I don’t have caching operator for now, I need to get the
> configuration file for each source and I have to parse them at run time. To
> do this going to HDFS is good or going to database is good?
>
>
>
> Regards,
>
> Surya Vamshi
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>

Mime
View raw message