db-ddlutils-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Oshinsky <doshin...@commvault.com>
Subject strange build problem with generated sources
Date Fri, 05 Feb 2016 16:37:15 GMT
Hi Drill-ers,

I am looking into fixing JIRA https://issues.apache.org/jira/browse/DRILL-4184.  I've encountered
a number of strange build problems along the way with my drill 1.4 snapshot, including inability
to rebuild after running "mvn clean", no matter what I try.  So, I'm building from scratch
for the second time, at least.  The latest problem really has me stumped at the moment.  I
added a "forceBits(int,int)" method that I see in generated source file NullableDecimal28SparseVector.java
(and NullableDecimal38SparseVector.java), but somehow this doesn't get compiled properly into
the *.class and my build keeps failing as if the new forceBits method isn't there:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
(default-compile) on project drill-java-exec: Compilation failure: Compilation failure:

[ERROR] C:\apache\apache-drill-1.4.0\rebuild2\drill-1.4.0\exec\java-exec\src\main\java\org\apache\drill\exec\store\parquet\columnreaders\VarLengthColumnReaders.java:[108,29]
error: cannot find symbol

[ERROR] symbol:   method forceBits(int,int)

[ERROR] location: variable nullableDecimal28Vector of type NullableDecimal28SparseVector [ERROR]
error: cannot find symbol

Can anyone suggest how to fix this without starting over from scratch in a new build node
(again)?  Any advice would be greatly appreciated.

I will send a separate email eventually regarding the design of my fix, which I know is only
a short-term solution to the problem of handling variable width decimal fields in Parquet
files.  To make a long story short, all the decimal vectors are fixed width vectors, which
don't have the ability to "remember" varying sizes from one decimal field to the next.  I've
hacked up something to "remember" the varying field sizes (BigDecimal array sizes) in NullableVarLengthValuesColumn
and VarLengthValuesColumn, not in the decimal vectors.  This seems to work, though it's admittedly
ugly.  However, I ran into a problem with nullable varying width decimal columns where the
"isSet" always returns 0, as if the column is null, when it is not, and the sparse decimal
data is present in the vector (but Drill won't send the decimal value, because it thinks it's
null).  Hence the "forceBits" hack to try to work around this.  It seemed like I was close
to running a successful Drill query on the varying width decimal Parquet data, but alas, I
ran into (another) build problem.  I do have a LOT of questions as to why the decimal stuff
was designed the way it is, but that's for another email....


Dave Oshinsky

***************************Legal Disclaimer***************************
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message