Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Fri, 21 Jul 2017 04:55:00 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.13085018.1499302289000.299298.1500612900087@Atlassian.JIRA>
In-Reply-To: <JIRA.13085018.1499302289000@Atlassian.JIRA>
References: <JIRA.13085018.1499302289000@Atlassian.JIRA> <JIRA.13085018.1499302289715@jira-lw-us.apache.org>
Subject: [jira] [Commented] (DRILL-5660) Drill 1.10 queries fail due to
 Parquet Metadata "corruption" from DRILL-3867
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Fri, 21 Jul 2017 04:55:14 -0000


    [ https://issues.apache.org/jira/browse/DRILL-5660?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1609=
5779#comment-16095779 ]=20

ASF GitHub Bot commented on DRILL-5660:
---------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/877#discussion_r128683724
 =20
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parq=
uet/Metadata.java ---
    @@ -132,23 +135,57 @@ public static ParquetTableMetadata_v3 getParquetT=
ableMetadata(FileSystem fs,
       }
    =20
       /**
    -   * Get the parquet metadata for a directory by reading the metadata =
file
    +   * Get the parquet metadata for the table by reading the metadata fi=
le
        *
    -   * @param fs
    +   * @param fs current file system
        * @param path The path to the metadata file, located in the directo=
ry that contains the parquet files
    -   * @return
    -   * @throws IOException
    +   * @param metaContext metadata context
    +   * @param formatConfig parquet format plugin configs
    +   * @return parquet table metadata
    +   * @throws IOException if metadata file can't be read or updated
        */
    -  public static ParquetTableMetadataBase readBlockMeta(FileSystem fs, =
Path path, MetadataContext metaContext, ParquetFormatConfig formatConfig) t=
hrows IOException {
    -    Metadata metadata =3D new Metadata(fs, formatConfig);
    -    metadata.readBlockMeta(path, false, metaContext);
    -    return metadata.parquetTableMetadata;
    +  public static @Nullable ParquetTableMetadataBase readBlockMeta(FileS=
ystem fs, Path path, MetadataContext metaContext,
    +      ParquetFormatConfig formatConfig) {
    +    if (metaContext.isMetaCacheFileCorrect) {
    +      Metadata metadata =3D new Metadata(fs, formatConfig);
    +      try {
    +        metadata.readBlockMeta(path, false, metaContext);
    +        return metadata.parquetTableMetadata;
    +      } catch (IOException e) {
    +        logger.error(e.toString());
    +        metaContext.isMetaCacheFileCorrect =3D false;
    +      }
    +    }
    +    logger.warn("Ignoring unsupported or corrupted metadata file versi=
on. Query performance may be slow. Make sure " +
    +        "the cache file is up-to-date by running the REFRESH TABLE MET=
ADATA command");
    +    return null;
       }
    =20
    -  public static ParquetTableMetadataDirs readMetadataDirs(FileSystem f=
s, Path path, MetadataContext metaContext, ParquetFormatConfig formatConfig=
) throws IOException {
    -    Metadata metadata =3D new Metadata(fs, formatConfig);
    -    metadata.readBlockMeta(path, true, metaContext);
    -    return metadata.parquetTableMetadataDirs;
    +  /**
    +   * Get the parquet metadata for all subdirectories by reading the me=
tadata file
    +   *
    +   * @param fs current file system
    +   * @param path The path to the metadata file, located in the directo=
ry that contains the parquet files
    +   * @param metaContext metadata context
    +   * @param formatConfig parquet format plugin configs
    +   * @return parquet metadata for a directory
    +   * @throws IOException if metadata file can't be read or updated
    +   */
    +  public static @Nullable ParquetTableMetadataDirs readMetadataDirs(Fi=
leSystem fs, Path path,
    +      MetadataContext metaContext, ParquetFormatConfig formatConfig) {
    +    if (metaContext.isMetaDirsCacheFileCorrect) {
    +      Metadata metadata =3D new Metadata(fs, formatConfig);
    +      try {
    +        metadata.readBlockMeta(path, true, metaContext);
    +        return metadata.parquetTableMetadataDirs;
    +      } catch (IOException e) {
    +        logger.error(e.toString());
    +        metaContext.isMetaDirsCacheFileCorrect =3D false;
    +      }
    +    }
    +    logger.warn("Ignoring corrupted metadata file. Query performance m=
ay be slow. Make sure the cache file" +
    --- End diff --
   =20
    Maybe move the message to constant so it does not have to be maintained=
 in two places? And, maybe add the file or directory that is corrupt?


> Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3=
867
> -------------------------------------------------------------------------=
---
>
>                 Key: DRILL-5660
>                 URL: https://issues.apache.org/jira/browse/DRILL-5660
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.11.0
>            Reporter: Paul Rogers
>            Assignee: Vitalii Diravka
>
> Drill recently accepted a PR for the following bug:
> DRILL-3867: Store relative paths in metadata file
> This PR turned out to have a flaw which affects version compatibility.
> The DRILL-3867 PR changed the format of Parquet metadata files to store r=
elative paths. All Drill servers after that PR create files with relative p=
aths. But, the version number of the file is unchanged, so that older Drill=
bits don't know that they can't understand the file.
> Instead, if an older Drill tries to read the file, queries fail left and =
right. Drill will resolve the paths, but does so relative to the user's HDF=
S home directory, which is wrong.
> What should have happened is that we should have bumped the parquet metad=
ata file version number so that older Drillbits can=E2=80=99t read the file=
. This ticket requests that we do that.
> Now, one could argue that the lack of version number change is fine. Once=
 a user upgrades Drill, they won't use an old Drillbit. But, things are not=
 that simple:
> * A developer tests a branch based on a pre-DRILL-3867 build on a cluster=
 in which metadata files have been created by a post-DRILL-3867 build. (Thi=
s has already occurred multiple times in our shop.)
> * A user tries to upgrade to Drill 1.11, finds an issue, and needs to rol=
l back to Drill 1.10. Doing so will cause queries to fail due to seemingly-=
corrupt metadata files.
> * A user tries to do a rolling upgrade: running 1.11 on some servers, 1.1=
0 on others. Once a 1.11 server is installed, the metadata is updated ("cor=
rupted" from the perspective of 1.10) and queries fail.
> Standard practice in this scenario is to:
> * Bump the file version number when the file format changes, and
> * Software refuses to read files with a version newer than the software w=
as designed for.
> Of course, it is highly desirable that newer servers read old files, but =
that is not the issue here.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)