apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yogi Devendra <devendra.vyavah...@gmail.com>
Subject Re: Information Needed
Date Thu, 11 Aug 2016 05:59:30 GMT

On a side note, I have seen multiple emails with subject line "Information
I urge you to use more informative subject line for your subsequent posts.

This would help future users to look through the mailing list archive.

~ Yogi

On 11 August 2016 at 10:23, Priyanka Gugale <priyag@apache.org> wrote:

> IMO, it's okay to keep the files in HDFS, at startup you can load them in
> memory which will be one time activity in application lifetime. Considering
> they are format/config files, it won't be difficult to cache it in memory
> of your current operator (no need to caching operator separately).
> -Priyanka
> On Wed, Aug 10, 2016 at 7:06 PM, Mukkamula, Suryavamshivardhan (CWM-NR) <
> suryavamshivardhan.mukkamula@rbc.com> wrote:
>> Hello Team,
>> I have a use case where I am trying to read number of input files with
>> different formats(may be 100 source systems) and I have to parse each
>> record.
>> To achieve this we need to have metadata(configuration files) of the
>> input file structures. Can you please let me know that, having this
>> metadata on HDFS is good or having the metadata in some sql/oracle database
>> is good?
>> Assume that I don’t have caching operator for now, I need to get the
>> configuration file for each source and I have to parse them at run time. To
>> do this going to HDFS is good or going to database is good?
>> Regards,
>> Surya Vamshi
>> _______________________________________________________________________
>> If you received this email in error, please advise the sender (by return
>> email or otherwise) immediately. You have consented to receive the attached
>> electronically at the above-noted email address; please retain a copy of
>> this confirmation for future reference.
>> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
>> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
>> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
>> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
>> cette confirmation pour les fins de reference future.

View raw message