hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From venito camelas <robotirlan...@gmail.com>
Subject Help designing application architecture
Date Thu, 07 Jul 2016 23:40:40 GMT
I'm pretty new to this and I have a use case I'm not sure how to implement,
I'll try to explain it and I'd appreciate if anyone could point me in the
right direction.

The case has these requirements:
 1 - Any user shoud be able to define the format of the information they
want to store (channel). For example, user X defines a channel named
coordinate = {
"X" : "Float",
"Y" : "Float",
"instant" : "Timestamp"
  Every channel has some time value, it can be an instant (like above) or a
period of time ("start" : "Timestamp", "end" : "Timestamp")

 2 - Given the previous example, the user should be able to ask the
following questions:
2.1 When was the last time I went near {X : x, Y : y}?  --> Process the
information in order to get the "near" places and return the newest one.
2.2 Where was I on march 6th between 1pm and 2pm?       --> Query by time

For 1) I was thinking of using some Document oriented storage because of
the channels lack of structure, not sure that's the only thing to consider

For 2.1) I'd use some MR job

For 2.2) I think it would be better to have the information in the document
storage and make the queries there.

Is it a good approach to have the information stored both in the hdfs and
the document oriented storage (for processing and querying respectively)?

As I mentioned in the beginning, I'm really new to this and I'm just trying
to learn..so sorry if my doubts are silly.

Any suggestion or any good reference related to this will be much

View raw message