hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qing Yan <qing...@gmail.com>
Subject Hive vs. DryadLINQ
Date Thu, 15 Oct 2009 07:31:29 GMT

   Has anyone looked into the Microsoft Dryad project?

   Their basic idea is using DAG(connect computational "vertices" with
communication "edges") to model distributed computing flows. And they have
something called DryadLINQ which seems to be the Hive equivilent.

     Since the DAG model doesn't distingish inter-job(workflow) and
intra-job(map/reduce..etc) layer, their approach of doing Query
translation,Workflow/Job Scheduling,Execution in one box may score better
optimization and fine tuning opportunties compared to the Hadoop/Hive

   Also giving majority of the hard work will be encapsulated and performed
by the translation/optimizing layer, the simplicity
beauty of Map/Reduce becomes irrelevant or even hindrance because it doesn't
permit more generic and flexible
operations like Dryad does.

  Seems M$ got it right this time, at least on paper :-P ...thought?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message