hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ambarish Giri <ambarishgir...@gmail.com>
Subject Re: Reg. org.apache.hudi.exception.HoodieException: Invalid query type :read_optimized
Date Tue, 03 Aug 2021 16:09:16 GMT
Hi Siva,

Sure will update the issue and follow there.
Just for FYI I have gone through the  FAQs
<https://cwiki.apache.org/confluence/display/HUDI/FAQ> as well.

In addition to the above issue,  I was also trying to run async compaction
on the MOR Hudi table but was not able to do that in the current setup.
Can you please guide me through that as well as how to
achieve scheduled compaction or explicit compaction on the Hudi table.

Regards,
Ambarish Giri

On Tue, Aug 3, 2021 at 8:40 PM Sivabalan <n.siva.b@gmail.com> wrote:

> Hey Ambarish,
>      I have created a github issue.
> <https://github.com/apache/hudi/issues/3395> Let's follow up there.
>
> On Mon, Aug 2, 2021 at 11:58 PM Ambarish Giri <ambarishgiri04@gmail.com>
> wrote:
>
> > Hi Team,
> >
> > I am an Architect in a reputable Product based IT firm.
> > I am in the evaluation process to use Hudi to incorporate a refreshable
> > data lake.
> > I am currently running the setup in my local machine and using a spark
> > datasource to write and read from the Hudi temp table.
> > I have evaluated the Cow and MoR write mechanisms but while trying to
> read
> > the Hudi table using Read_Optimized type I am getting the below
> exception:
> >
> > Exception in thread "main" org.apache.hudi.exception.HoodieException:
> > Invalid query type :read_optimized
> > at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:81)
> > at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:46)
> > at
> >
> >
> org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:332)
> > at
> >
> >
> org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:242)
> >
> > Below is the how I am trying to read from the Hudi location:
> >
> > spark.read
> >   .format("hudi")
> >   .option(DataSourceReadOptions.QUERY_TYPE_OPT_KEY,
> > DataSourceReadOptions.QUERY_TYPE_READ_OPTIMIZED_OPT_VAL)
> >   .load(s"$basePath/$tableName")
> >   .show(50,false)
> >
> > Kindly suggest if I am doing anything wrong?
> >
> > Below are the versions I am currently using :
> > apache hudi 0.7.0
> > spark 2.4.7
> > scala 2.12
> >
> > Kindly let me know in case anymore details are required.
> >
> > Regards,
> > Ambarish Giri
> > 9951742695
> >
>
>
> --
> Regards,
> -Sivabalan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message