impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Behm (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] Improve logging of table loading.
Date Sat, 14 Jan 2017 18:49:39 GMT
Alex Behm has posted comments on this change.

Change subject: Improve logging of table loading.
......................................................................


Patch Set 1:

(10 comments)

Thanks, Bharath. Good suggestions.

http://gerrit.cloudera.org:8080/#/c/5709/1//COMMIT_MSG
Commit Message:

Line 7: Improve logging of table loading.
> JIRA id? I think it helps in backporting to other branches.
Don't think we're going to backport this anywhere, but done anyway.


http://gerrit.cloudera.org:8080/#/c/5709/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:

Line 528:       LOG.trace("Loading native functions for database: " + db.getName());
> Should we enable logging related to fns (here and below) ? I think it helps
I agree. The basic loading steps should be included in the logs.

To make the intention clearer, I changed the new log messages to LOG.info() because LOG.debug()
could mislead a reader.


Line 898:       LOG.debug(String.format("Refreshing metadata: %s", tbl.getFullName()));
> Given we support both table & partition granularity, Is it better to explic
Agree. Done.


Line 1036:           tableName.getDb_name(), tableName.getTable_name()));
> this and invalidateDb()? Can potentially help the frequency with which inva
Agree. invalidateDb() is a private function only called from reset(), so I added the logging
there (indicates an "invalidate metadata").


Line 1262:   public Table reloadPartition(Table tbl, List<TPartitionKeyValue> partitionSpec)
> Should we add this too?
Yea I think so.


http://gerrit.cloudera.org:8080/#/c/5709/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:

Line 681:       LOG.debug("Creating empty partition objects: " + getFullName());
> Log statment would probably be confusing for non-partitioned tables?  Shoul
I'm not even sure this message is useful. Removed.


PS1, Line 1057: db_.getName() + "." + name_
> getFullName()?
Done


Line 1081:           LOG.debug("Incrementally loaded metadata for: " + tableName);
> I have a feeling these could be noisy and can do a lot of logging.
We only get into this code path for INSERT, ALTER TABLE, TRUNCATE TABLE and DROP STATS. Do
you still think it will be too noisy?


http://gerrit.cloudera.org:8080/#/c/5709/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:

Line 570:       LOG.debug("Loading table metadata: " + tbl.getFullName());
> This will be logged twice for non-hdfs tables, once here and once inside Ta
Assuming you mean HDFS tables. The HDFS loading message is a little more detailed, it says
whether an incremental or complete load is being done.

I think it also makes sense to track the caller of load in this case. What do you think?


http://gerrit.cloudera.org:8080/#/c/5709/1/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:

Line 915:               LOG.trace(String.format("Missing tables were not received in %dms.
Load " +
> Isn't this a warn/error?
Done


-- 
To view, visit http://gerrit.cloudera.org:8080/5709
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8de96d0cb6d09b2272b1925d42cb059367fe7196
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message