hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinoth Chandar <vin...@apache.org>
Subject Re: Long test run times
Date Fri, 30 Jul 2021 15:54:50 GMT
There is probably some good amount of low hanging fruit, we can make some
headway and see how it goes from there?

On Thu, Jul 29, 2021 at 7:18 PM Danny Chan <danny0405@apache.org> wrote:

> What should we do for these long running tests ? Simplify them to more
> simple UTs ?
>
> Vinoth Chandar <vinoth@apache.org> 于2021年7月30日周五 上午6:53写道:
>
> > I am looking into
> >
> > 614.322 org.apache.hudi.client.TestHoodieClientOnCopyOnWriteStorage
> > 556.392 org.apache.hudi.metadata.TestHoodieBackedMetadata
> >
> > On Thu, Jul 29, 2021 at 1:54 PM Sivabalan <n.siva.b@gmail.com> wrote:
> >
> > > I will take up some.
> > >
> > > org.apache.hudi.utilities.functional.TestHoodieMultiTableDeltaStreamer
> > > org.apache.hudi.spark3.internal.TestHoodieDataSourceInternalBatchWrite
> > > org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer
> > > org.apache.hudi.spark3.internal.TestHoodieBulkInsertDataInternalWriter
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Jul 29, 2021 at 4:04 PM Vinoth Chandar <vinoth@apache.org>
> > wrote:
> > >
> > > > Folks,
> > > >
> > > > Our tests are now exceeding 60 minutes and suffering timeouts on
> azure
> > > > (travis is slow to actually schedule, but seems to finish on time).
> > > >
> > > > Following are the list of top slow tests. I am working from the top
> of
> > > the
> > > > list. Any one interested in chipping in?  Please respond with the
> test
> > > you
> > > > are interested in fixing.
> > > >
> > > > 927.371
> > > >
> org.apache.hudi.utilities.functional.TestHoodieMultiTableDeltaStreamer
> > > > 894.438
> > > >
> org.apache.hudi.spark3.internal.TestHoodieDataSourceInternalBatchWrite
> > > > 866.088 org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer
> > > > 614.322 org.apache.hudi.client.TestHoodieClientOnCopyOnWriteStorage
> > > > 556.392 org.apache.hudi.metadata.TestHoodieBackedMetadata
> > > > 291.229 org.apache.hudi.table.TestHoodieMergeOnReadTable
> > > > 245.922 org.apache.hudi.functional.TestStructuredStreaming
> > > > 234.737 org.apache.hudi.index.TestHoodieIndex
> > > > 224.893 org.apache.hudi.functional.TestMORDataSource
> > > > 217.354
> > > >
> org.apache.hudi.spark3.internal.TestHoodieBulkInsertDataInternalWriter
> > > > 210.717 org.apache.hudi.common.functional.TestHoodieLogFormat
> > > > 204.989 org.apache.hudi.table.TestCleaner
> > > > 198.623 org.apache.hudi.client.TestBootstrap
> > > > 170.029 org.apache.hudi.hive.TestHiveSyncTool
> > > > 135.295 org.apache.hudi.table.action.compact.TestInlineCompaction
> > > > 116.381
> > org.apache.hudi.table.action.commit.TestCopyOnWriteActionExecutor
> > > > 113.559 org.apache.hudi.functional.TestCOWDataSource
> > > > 111.297 org.apache.hudi.table.upgrade.TestUpgradeDowngrade
> > > >
> > > > Thanks
> > > > Vinoth
> > > >
> > >
> > >
> > > --
> > > Regards,
> > > -Sivabalan
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message