drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefán Baxter <ste...@activitystream.com>
Subject Re: Building master
Date Fri, 17 Jul 2015 23:38:09 GMT
Hi again,

We are using the REST interface for now and we would like to look into the
following things:

   - Support protobuf / messagepack as an alternative to json (query.json)
   - Stop formatting JSON output (unnecessary white-spaces are being sent)
   - Support gziped results
   - Stop sending numeric values as strings in JSON. (otpional)

Hopefully we will be able to contribute something meaningful but these, a
long with a few issues we have created, are of specific interest to us.

It looks to me like running Drillbit.java might be the way to go (I find no
REST Test files) but running it produces the following error:

Exception in thread "main"
org.apache.drill.exec.exception.DrillbitStartupException: Failure during
initial startup of Drillbit.
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:87)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:66)
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.lang.RuntimeException:
com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type
id 'hbase' into a subtype of [simple type, class
org.apache.drill.common.logical.StoragePluginConfig]
 at [Source: [B@51e7589f; line: 2, column: 3]
at
org.apache.drill.exec.store.sys.zk.ZkAbstractStore$Iter$DeferredEntry.getValue(ZkAbstractStore.java:197)
at
org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:168)
at
org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:132)
at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:244)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:84)
... 7 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not
resolve type id 'hbase' into a subtype of [simple type, class
org.apache.drill.common.logical.StoragePluginConfig]
 at [Source: [B@51e7589f; line: 2, column: 3]
at
com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at
com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:849)
at
com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
at
com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
at
com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41)
at
com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1269)
at
com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:912)
at
org.apache.drill.exec.store.sys.serialize.JacksonSerializer.deserialize(JacksonSerializer.java:44)
at
org.apache.drill.exec.store.sys.zk.ZkAbstractStore$Iter$DeferredEntry.getValue(ZkAbstractStore.java:195)
... 11 more

Could you please help us with the java options/config need to run the
drillbit in local mode (fds only).

Regards,
 -Stefan


On Fri, Jul 17, 2015 at 11:19 PM, Stefán Baxter <stefan@activitystream.com>
wrote:

> thank you
>
> On Fri, Jul 17, 2015 at 11:16 PM, Jacques Nadeau <jacques@apache.org>
> wrote:
>
>> We typically extend BaseTestQuery and use unit tests for trying out new
>> functionality.  BaseTestQuery provides a large number of ways to build
>> tests.  You can look here for some examples using the great test builder
>> api that Jason put together:
>>
>>
>> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java#L36
>>
>> On Fri, Jul 17, 2015 at 3:33 PM, Stefán Baxter <stefan@activitystream.com
>> >
>> wrote:
>>
>> > Hi,
>> >
>> > Yes, I have done that.
>> >
>> > Can you please walk me through your development setup / processes.
>> >
>> > I'm reluctant to believe that your development cycle includes a full
>> build
>> > and as we are hoping to contribute (or at least try to be helpful) then
>> I
>> > would very much like to understand how you work on top of this without
>> > requiring full build+packaging.
>> >
>> > Regards,
>> >  -Stefan
>> >
>> >
>> > On Fri, Jul 17, 2015 at 10:29 PM, Jacques Nadeau <jacques@apache.org>
>> > wrote:
>> >
>> > > There is no maven target for this.  However, a full build creates a
>> > > directory: distribution/target/apache-drill-1.2.0-SNAPSHOT that you
>> can
>> > run
>> > > those things from.
>> > >
>> > > On Fri, Jul 17, 2015 at 3:11 PM, Stefán Baxter <
>> > stefan@activitystream.com>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > This worked, than you.
>> > > >
>> > > > What maven target can I run that is the equivalent of
>> > > bin/drill-localhost
>> > > > or bin/drillbit.sh start ?
>> > > >
>> > > > Regards,
>> > > >  -Stefán
>> > > >
>> > > > On Fri, Jul 17, 2015 at 3:48 PM, Jacques Nadeau <jacques@apache.org
>> >
>> > > > wrote:
>> > > >
>> > > > > Try to exit Idea and then do a mvn clean; mvn install -DskipTests.
>> > > > >
>> > > > > On Fri, Jul 17, 2015 at 8:40 AM, Stefán Baxter <
>> > > > stefan@activitystream.com>
>>
>> > > > > wrote:
>> > > > >
>> > > > > > Yeah, you are right. (Idea)
>> > > > > >
>> > > > > > I get this error when I try to run that target:
>> > > > > > [ERROR] Failed to execute goal
>> > > > > > org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
>> > > > > > (default-compile) on project drill-common: Compilation failure:
>> > > > > Compilation
>> > > > > > failure:
>> > > > > > [ERROR] Picked up JAVA_TOOL_OPTIONS:
>> > > > > > -javaagent:/usr/share/java/jayatanaag.jar
>> > > > > > [ERROR]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /var/www/as/drill/common/src/main/java/org/apache/drill/common/expression/LogicalExpression.java:[73,8]
>> > > > > > error: cannot find symbol
>> > > > > > [ERROR] symbol:   class ExprLexer
>> > > > > > [ERROR] location: class De
>> > > > > > [ERROR]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /var/www/as/drill/common/src/main/java/org/apache/drill/common/expression/LogicalExpression.java:[73,30]
>> > > > > > error: cannot find symbol
>> > > > > > [ERROR] symbol:   class ExprLexer
>> > > > > > [ERROR] location: class De
>> > > > > > [ERROR]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /var/www/as/drill/common/src/main/java/org/apache/drill/common/expression/LogicalExpression.java:[75,8]
>> > > > > > error: cannot find symbol
>> > > > > > [ERROR] symbol:   class ExprParser
>> > > > > > [ERROR] location: class De
>> > > > > > [ERROR]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /var/www/as/drill/common/src/main/java/org/apache/drill/common/expression/LogicalExpression.java:[75,32]
>> > > > > > error: cannot find symbol
>> > > > > > [ERROR] symbol:   class ExprParser
>> > > > > > [ERROR] location: class De
>> > > > > > [ERROR]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /var/www/as/drill/common/src/main/java/org/apache/drill/common/expression/LogicalExpression.java:[79,8]
>> > > > > > error: cannot find symbol
>> > > > > > [ERROR] -> [Help 1]
>> > > > > > [ERROR]
>> > > > > > [ERROR] To see the full stack trace of the errors, re-run
Maven
>> > with
>> > > > the
>> > > > > -e
>> > > > > > switch.
>> > > > > > [ERROR] Re-run Maven using the -X switch to enable full
debug
>> > > logging.
>> > > > > > [ERROR]
>> > > > > > [ERROR] For more information about the errors and possible
>> > solutions,
>> > > > > > please read the following articles:
>> > > > > > [ERROR] [Help 1]
>> > > > > >
>> > > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>> > > > > > [ERROR]
>> > > > > > [ERROR] After correcting the problems, you can resume the
build
>> > with
>> > > > the
>> > > > > > command
>> > > > > > [ERROR]   mvn <goals> -rf :drill-common
>> > > > > >
>> > > > > > Looks to be another version of the same error.
>> > > > > >
>> > > > > > Any ideas?
>> > > > > >
>> > > > > > Regards,
>> > > > > >  -Stefan
>> > > > > >
>> > > > > > On Fri, Jul 17, 2015 at 3:35 PM, Jacques Nadeau <
>> > jacques@apache.org>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > I'm guessing that you're trying to use a IDE to build.
 You
>> > should
>> > > do
>> > > > > one
>> > > > > > > run of mvn install -DskipTests before trying to use
an IDE.
>> > > > > > >
>> > > > > > > On Fri, Jul 17, 2015 at 8:29 AM, Stefán Baxter <
>> > > > > > stefan@activitystream.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi,
>> > > > > > > >
>> > > > > > > > I now see that this is a part of the ExprLexer.g
(I will
>> sort
>> > it
>> > > > out)
>> > > > > > > >
>> > > > > > > > Thank you,
>> > > > > > > >  -Stefan
>> > > > > > > >
>> > > > > > > > On Fri, Jul 17, 2015 at 3:20 PM, Stefán Baxter
<
>> > > > > > > stefan@activitystream.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi,
>> > > > > > > > >
>> > > > > > > > > I'm trying to build Drill from master branch.
>> > > > > > > > >
>> > > > > > > > > Is it possible that someone has not pushed:
>> > > > > > > > >
>> > > > > > > > > import
>> org.apache.drill.common.expression.parser.ExprLexer;
>> > > > > > > > > import
>> org.apache.drill.common.expression.parser.ExprParser;
>> > > > > > > > > import
>> > > > > > >
>> > org.apache.drill.common.expression.parser.ExprParser.parse_return;
>> > > > > > > > >
>> > > > > > > > > Regards,
>> > > > > > > > >   -Stefán
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > ps. Sent this again as I was not verified
my dev
>> subscription
>> > > > when
>> > > > > a
>> > > > > > > > previous copy was sent (in case this is a duplicate)
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message