tajo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schwabe <Christian.Schw...@gmx.com>
Subject Re: starting difficulties
Date Fri, 15 Aug 2014 15:00:04 GMT
Hello together,

yesterday i pulled the new status from tajo github.
Recompiled the sources with the known build-statement: mvn clean package -DskipTests -Pdist
-Dtar
First, however, I still ensures that the variable Platform has the value of x64
Since I made this step i have the same error for the first day i tried Apache Tajo.
I have only pulled Tajo, recompiled.

I don’t believe that this message () is a new issue 

I don't think this is a new error (http://mail-archives.apache.org/mod_mbox/tajo-user/201408.mbox/%3cB99E4FEE-997C-4B4C-BEDD-C4618244BB90@gmx.com%3e),
but rather resulted from me at that time not yet noticed the COUNT (*) did not work. Otherwise
I would have surely noted earlier that something else is wrong here.

I mean so far I've been able to gain some experience. Therefore, I can not understand what
could have gone wrong now with these few steps currently. It is a reason for never change
a running system ... :(

I have attached the logs from worker and master. 

P.S.: I use Hadoop in Version 3.0.0-Snapshot. Previously, there were no problems with this
version of Hadoop. 

default> SELECT * FROM table1;
id,  name,  score,  type
-------------------------------
1,  abc,  1.1,  a
2,  def,  2.3,  b
3,  ghi,  3.4,  c
4,  jkl,  4.5,  d
5,  mno,  5.6,  e
(5 rows, 0.229 sec, 59 B selected)

default> SELECT COUNT(*) FROM table1;
Progress: 0%, response time: 1.224 sec
Progress: 50%, response time: 2.225 sec
Progress: 50%, response time: 3.227 sec
Progress: 50%, response time: 4.229 sec
ERROR: 1
java.lang.ArrayIndexOutOfBoundsException: 1
	at org.apache.tajo.util.BitArray.fromByteBuffer(BitArray.java:84)
	at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:243)
	at org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:254)
	at org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:51)
	at org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:87)
	at org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:77)
	at org.apache.tajo.worker.Task.run(Task.java:425)
	at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:425)
	at java.lang.Thread.run(Thread.java:744)


Best regards,
a sad Chris

Am 28.07.2014 um 04:55 schrieb Jose Alvarez Muguerza <jose.muguerza@globant.com>:

> Hi Christian, 
> An alternative, please try recompiling your ASF hadoop distribution with "+ export Platform=x64"
 parameter since your platform is 64-Bit 
> 
> Good luck 
> 
> 
> 
> On Sunday, July 27, 2014, Christian Schwabe <Christian.Schwabe@gmx.com> wrote:
> Hello Hyunsik,
> 
> Please excuse my incomplete answer. I forgot to mention that also the recompile and leads
to the already mentioned java.lang.ArrayIndexOutOfBoundsException.
> 
> Kind regards,
> Christian
> 
> Am 27.07.2014 um 21:46 schrieb Christian Schwabe <Christian.Schwabe@gmx.com>:
> 
>> Hello Hyunsik, today I have started the trial to recompile with following 'mvn clean
package -DskipTests -Pdist -Dtar -Dhadoop.version=2.4.0' statement. I regret to inform you
that the recompile not brought the desired success.
>> 
>> I use Hadoop in version 2.4.0. 
>> 
>> This is my current JVM version:
>> java version "1.8.0_05"
>> Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
>> 
>> My Tajo environment (tajo-env.sh) has following content:
>> 
>> # Licensed to the Apache Software Foundation (ASF) under one
>> # or more contributor license agreements.  See the NOTICE file
>> # distributed with this work for additional information
>> # regarding copyright ownership.  The ASF licenses this file
>> # to you under the Apache License, Version 2.0 (the
>> # "License"); you may not use this file except in compliance
>> # with the License.  You may obtain a copy of the License at
>> #
>> #     http://www.apache.org/licenses/LICENSE-2.0
>> #
>> # Unless required by applicable law or agreed to in writing, software
>> # distributed under the License is distributed on an "AS IS" BASIS,
>> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> # See the License for the specific language governing permissions and
>> # limitations under the License.
>> 
>> # Set Tajo-specific environment variables here.
>> 
>> # The only required environment variable is JAVA_HOME.  All others are
>> # optional.  When running a distributed configuration it is best to
>> # set JAVA_HOME in this file, so that it is correctly defined on
>> # remote nodes.
>> 
>> # —————————————————————————————
>> # Hadoop home. Required
>> #export HADOOP_HOME=/home/christian/6_Semester/Thesis/02_Hadoop/hadoop-2.4.0
>> 
>> #Mac OS X Konfiguration
>> export HADOOP_HOME=/Users/chris/6_Semester/Thesis/02_Hadoop/hadoop-2.4.0
>> # —————————————————————————————
>> 
>> # The java implementation to use.  Required.
>> # export JAVA_HOME=/usr/java/default
>> #export JAVA_HOME=/usr
>> #Mac OS X Konfiguration
>> export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home
>> 
>> # Extra Java CLASSPATH elements.  Optional.
>> # export TAJO_CLASSPATH=/xxx/extlib/*:/xxx/xxx.jar
>> 
>> # The maximum amount of heap to use, in MB. Default is 1000.
>> # export TAJO_MASTER_HEAPSIZE=1000
>> 
>> # The maximum amount of heap to use, in MB. Default is 1000.
>> # export TAJO_WORKER_HEAPSIZE=1000
>> 
>> # The maximum amount of heap to use, in MB. Default is 1000.
>> # export TAJO_QUERYMASTER_HEAPSIZE=1000
>> 
>> # Extra Java runtime options.  Empty by default.
>> # export TAJO_OPTS=-server
>> 
>> # Extra TajoMaster's java runtime options for TajoMaster. Empty by default
>> # export TAJO_MASTER_OPTS=
>> 
>> # Extra TajoWorker's java runtime options for TajoMaster. Empty by default
>> # export TAJO_WORKER_OPTS=
>> 
>> # Extra  QueryMaster mode TajoWorker's java runtime options for TajoMaster. Empty
by default
>> # export TAJO_QUERYMASTER_OPTS=
>> 
>> # Where log files are stored.  $TAJO_HOME/logs by default.
>> # export TAJO_LOG_DIR=${TAJO_HOME}/logs
>> 
>> # The directory where pid files are stored. /tmp by default.
>> # export TAJO_PID_DIR=/var/tajo/pids
>> 
>> # A string representing this instance of tajo. $USER by default.
>> # export TAJO_IDENT_STRING=$USER
>> 
>> # The scheduling priority for daemon processes.  See 'man nice'.
>> # export TAJO_NICENESS=10
>> 
>> # Tajo cluster mode. the default mode is standby mode.
>> export TAJO_WORKER_STANDBY_MODE=true
>> 
>> # It must be required to use HCatalogStore
>> # export HIVE_HOME=
>> # export HIVE_JDBC_DRIVER_DIR=
>> 
>> Findest du da einen Fehler in der Konfiguration oder hast du noch einen Tip was genau
falsch sein könnte? Ich kann in der nächsten Woche leider erst einmal nicht antworten, werde
mich sobald ich wieder da bin wieder melden.
>> 
>> Am 27.07.2014 um 06:06 schrieb Hyunsik Choi <hyunsik@apache.org>:
>> 
>>> Hi Christian,
>>> 
>>> In order to clarify your problem, could you share your environments,
>>> such as Hadoop version and JVM version? For some hadoop version, you
>>> need to give '-Dhadoop.version=2.4.0' to maven build command.
>>> Otherwise, incorrect class sources may be built for your binary dist.
>>> 
>>> Best regards,
>>> Hyunsik
>>> 
>>> On Sat, Jul 26, 2014 at 6:11 AM, Christian Schwabe
>>> <Christian.Schwabe@gmx.com> wrote:
>>>> Hello Hyunsik,
>>>> 
>>>> thanks for your quick response.
>>>> I clone the actual github repository and compile the Version by my own.
>>>> So actual I use the version 0.9.0-Snapshot.
>>>> After your answer it seems als the bugs exist in the current version? Can
>>>> you confirm this?
>>>> 
>>>> Kind regards,
>>>> Chris
>>>> 
>>>> On 25. Juli 2014 18:39:37 MESZ, Hyunsik Choi <hyunsik@apache.org> wrote:
>>>>> 
>>>>> Hi Christian,
>>>>> 
>>>>> Above all, thank you for your interest in Tajo.
>>>>> 
>>>>> The problem seems to be an old bug.
>>>>> Are you using Tajo 0.9.0? Did you get Tajo via git?
>>>>> 
>>>>> Best regards,
>>>>> Hyunsik
>>>>> 
>>>>> On Fri, Jul 25, 2014 at 9:23 PM, Christian Schwabe
>>>>> <Christian.Schwabe@gmx.com> wrote:
>>>>>> 
>>>>>> Hello guys,
>>>>>> 
>>>>>> this is the first time for me to write to an email list. Sorry for
any
>>>>>> wrong
>>>>>> formatting.
>>>>>> I am a student of computer science and use the framework Apache Tajo
as
>>>>>> part
>>>>>> of my thesis. However, I have initial teething problems and
>>>>>> unfortunately do
>>>>>> not know how to remedy them.
>>>>>> 
>>>>>> First, I want to describe what steps do I do to start Tajo:
>>>>>> 1) Start Tajo with tajo-start.sh
>>>>>> 2) Create a new Table
>>>>>> 3) Start bash tsql
>>>>>> 
>>>>>> Here details to my table contents which is composed from a csv file:
>>>>>> 
>>>>>> default> SELECT * from table1;
>>>>>> id,  name,  score,  type
>>>>>> ________________________________
>>>>>> 
>>>>>> 1,  abc,  1.1,  a
>>>>>> 2,  def,  2.3,  b
>>>>>> 3,  ghi,  3.4,  c
>>>>>> 4,  jkl,  4.5,  d
>>>>>> 5,  mno,  5.6,  e
>>>>>> (5 rows, 0.275 sec, 59 B selected)
>>>>>> 
>>>>>> Everything seems to work up here. Also substr () or concat () function.
>>>>>> Do I
>>>>>> use now but the count (*) or sum () I get the following exception:
>>>>>> 
>>>>>> default> SELECT count(*) FROM table1;
>>>>>> ERROR: 1
>>>>>> java.lang.ArrayIndexOutOfBoundsException: 1
>>>>>> at org.apache.tajo.util.BitArray.fromByteBuffer(BitArray.java:84)
>>>>>> at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:243)
>>>>>> at
>>>>>> 
>>>>>> org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:254)
>>>>>> at
>>>>>> 
>>>>>> org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:51)
>>>>>> at
>>>>>> 
>>>>>> org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:87)
>>>>>> at
>>>>>> 
>>>>>> org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:77)
>>>>>> at org.apache.tajo.worker.Task.run(Task.java:425)
>>>>>> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:406)
>>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>>> 
>>>>>> 
>>>>>> Can you tell me what I'm doing wrong at this point that I have neither
>>>>>> the
>>>>>> count (*) still can use sum ()?
>>>>>> Is this a known problem? Is it fixed? When will it be fixed? How
can I
>>>>>> fix
>>>>>> it myself? I suppose that I have a wrong configuration, although
I have
>>>>>> not
>>>>>> had much konfikuriert. Because I can hardly imagine that such trivial
>>>>>> functionalities not in my used version 0.9.0 are available.
>>>>>> 
>>>>>> Kind regards,
>>>>>> Christian Schwabe
>> 
> 
> 
> 
> -- 
> 
> Jose Alvarez Muguerza | Architect 
> GLOBANT | AR: +54 11 4109 1700 ext. 15895 | US: +1 877 215 5230 ext. 15895 |
> 					
>  
> 
> 
> 
> 
> 
> 
> 
>  Please consider the environment before printing this email
> 


Mime
View raw message