orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From omalley <...@git.apache.org>
Subject [GitHub] orc pull request #40: ORC-54: Evolve schemas based on field name rather than...
Date Wed, 06 Jul 2016 18:22:23 GMT
Github user omalley commented on a diff in the pull request:

    https://github.com/apache/orc/pull/40#discussion_r69784134
  
    --- Diff: java/core/src/java/org/apache/orc/impl/SchemaEvolution.java ---
    @@ -45,16 +53,65 @@ public SchemaEvolution(TypeDescription readerSchema, boolean[] included)
{
       }
     
       public SchemaEvolution(TypeDescription fileSchema,
    -                         TypeDescription readerSchema,
    -                         boolean[] included) throws IOException {
    +      TypeDescription readerSchema,
    +      boolean allowMissingMetadata,
    +      boolean[] included) {
         readerToFile = new HashMap<>(readerSchema.getMaximumId() + 1);
         this.included = included;
         if (checkAcidSchema(fileSchema)) {
           this.readerSchema = createEventSchema(readerSchema);
         } else {
           this.readerSchema = readerSchema;
         }
    -    buildMapping(fileSchema, this.readerSchema);
    +
    +    boolean useFieldNames;
    +    if (!hasColumnNames(fileSchema)){
    +      if (!allowMissingMetadata && !fileSchema.equals(readerSchema)){
    +        throw new RuntimeException("Found that schema metadata is missing from file.
This is likely caused by a writer earlier than HIVE-4243. Will not try to reconcile schemas");
    +      } else {
    +        LOG.warn("Found that schema metadata is missing from file. This is likely caused
by a writer earlier than HIVE-4243. Will try to reconcile schemas based on index");
    --- End diff --
    
    How about "Found that column names are missing from file, which is caused by ORC writers
before HIVE-4243."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message