Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 15 Aug 2017 12:21:01 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.13085018.1499302289000.33086.1502799661754@Atlassian.JIRA>
In-Reply-To: <JIRA.13085018.1499302289000@Atlassian.JIRA>
References: <JIRA.13085018.1499302289000@Atlassian.JIRA> <JIRA.13085018.1499302289715@jira-lw-us.apache.org>
Subject: [jira] [Commented] (DRILL-5660) Drill 1.10 queries fail due to
 Parquet Metadata "corruption" from DRILL-3867
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 15 Aug 2017 12:21:10 -0000


    [ https://issues.apache.org/jira/browse/DRILL-5660?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1612=
7137#comment-16127137 ]=20

ASF GitHub Bot commented on DRILL-5660:
---------------------------------------

Github user vdiravka commented on a diff in the pull request:

    https://github.com/apache/drill/pull/877#discussion_r133140000
 =20
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parq=
uet/ParquetGroupScan.java ---
    @@ -974,8 +1005,17 @@ private int updateRowGroupInfo(long maxRecords) {
       public ParquetGroupScan clone(FileSelection selection) throws IOExce=
ption {
         ParquetGroupScan newScan =3D new ParquetGroupScan(this);
         newScan.modifyFileSelection(selection);
    -    newScan.setCacheFileRoot(selection.cacheFileRoot);
    -    newScan.init(selection.getMetaContext());
    +    MetadataContext metaContext =3D selection.getMetaContext();
    +    if (metaContext =3D=3D null) {
    +      metaContext =3D new MetadataContext();
    +    }
    +    if (newScan.usedMetadataCache) {
    +      newScan.setCacheFileRoot(selection.cacheFileRoot);
    --- End diff --
   =20
    Done


> Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3=
867
> -------------------------------------------------------------------------=
---
>
>                 Key: DRILL-5660
>                 URL: https://issues.apache.org/jira/browse/DRILL-5660
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.11.0
>            Reporter: Paul Rogers
>            Assignee: Vitalii Diravka
>              Labels: doc-impacting
>             Fix For: 1.12.0
>
>
> Drill recently accepted a PR for the following bug:
> DRILL-3867: Store relative paths in metadata file
> This PR turned out to have a flaw which affects version compatibility.
> The DRILL-3867 PR changed the format of Parquet metadata files to store r=
elative paths. All Drill servers after that PR create files with relative p=
aths. But, the version number of the file is unchanged, so that older Drill=
bits don't know that they can't understand the file.
> Instead, if an older Drill tries to read the file, queries fail left and =
right. Drill will resolve the paths, but does so relative to the user's HDF=
S home directory, which is wrong.
> What should have happened is that we should have bumped the parquet metad=
ata file version number so that older Drillbits can=E2=80=99t read the file=
. This ticket requests that we do that.
> Now, one could argue that the lack of version number change is fine. Once=
 a user upgrades Drill, they won't use an old Drillbit. But, things are not=
 that simple:
> * A developer tests a branch based on a pre-DRILL-3867 build on a cluster=
 in which metadata files have been created by a post-DRILL-3867 build. (Thi=
s has already occurred multiple times in our shop.)
> * A user tries to upgrade to Drill 1.11, finds an issue, and needs to rol=
l back to Drill 1.10. Doing so will cause queries to fail due to seemingly-=
corrupt metadata files.
> * A user tries to do a rolling upgrade: running 1.11 on some servers, 1.1=
0 on others. Once a 1.11 server is installed, the metadata is updated ("cor=
rupted" from the perspective of 1.10) and queries fail.
> Standard practice in this scenario is to:
> * Bump the file version number when the file format changes, and
> * Software refuses to read files with a version newer than the software w=
as designed for.
> Of course, it is highly desirable that newer servers read old files, but =
that is not the issue here.
> *Main technical points of working of parquet metadata caching for now.*
> Only process of reading the parquet metadata is changed (the process of w=
riting isn't changed):
> +1. Metadata files are valid:+
> Metadata objects are created by deserialization of parquet metadata files=
 in the process of creating ParquetGroupScan physical operator.=20
> All supported versions are stored in the "MetadataVersion.Constants" clas=
s and in the Jackson annotations for Metadata.ParquetTableMetadataBase clas=
s.
> +2. Metadata files version isn't supported (created by newer Drill versio=
n). Drill table has at least one metadata file of unsupported version:+
> JsonMappingException is obtained and swallowed without creating metadata =
object. Error message is logged. The state is stored in MetadataContext, th=
erefore further there will be no attempt to deserialize metadata file again=
 in context of performing current query. The physical plan will be created =
without using parquet metadata caching. Warning message is logged for every=
 further check "is metadata corrupted".
> +3. Drill table has at least one corrupted metadata file, which can't be =
deserialized:+
> JsonParseException is obtained. Then the same behaviour as for the unsupp=
orted version files.
> +4. The metadata file was removed by other process:+
> FileNotFound is obtained. Then the same behaviour as for the unsupported =
version files.
> The new versions of metadata should be added in such manner:
> 1. Increasing of the metadata major version if metadata structure is chan=
ged.
> 2. Increasing of the metadata minor version if only metadata content is c=
hanged, but metadata structure is the same.
> For the first case a new metadata structure (class) should be created (po=
ssible an improvement of deserializing metadata files of any version into o=
ne strucure by using special converting)
> For the second case only annotation for the last metadata structure can b=
e updated.
> *Summary*
> 1. Drill will read and use metadata files if files are valid, all present=
 and supported. Under supported we mean that files were created before and =
under current Drill version.
> 2. Drill will ignore reading metadata files if at least one file is missi=
ng, empty, corrupted or unsupported. Under unsupported we mean files create=
d after current Drill version. When metadata files are not used, warning me=
ssage will be written to the logs.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)