impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aleksei Maželis <olm...@gmail.com>
Subject Re: Impala-Kudu - minimal set of dependencies
Date Thu, 27 Apr 2017 11:20:14 GMT
It seems that Impala indeed starts without HDFS when the flag is reset as
suggested. I'm also observing some errors in the logs, due to the inability
to find/connect to HDFS:

Could not read the root directory at hdfs://0.0.0.0:8020.
Error was: Call From localhost/ZZ.ZZZ.ZZ.ZZZ to 0.0.0.0:8020 failed on
connection exception: java.net.ConnectException: Connection refused; For
more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

This is likely due to the fact that HDFS is still mentioned in the configs
(specifically, core-site.xml still refers to fs.default.name as to hdfs://
0.0.0.0:8020). Should this be modified somehow to refer to the local file
system or removed altogether?

Regards,
Alex

On Thu, Apr 27, 2017 at 8:53 AM, Aleksei Maželis <olmazh@gmail.com> wrote:

> Ok, great, thanks for elaborating! I will give it a try with the abort_on_config_error
> flag reset!
>
> As to the Hive dependency, I will stay tuned for the future updates then.
>
> Regards,
> Alex
>
> On Wed, Apr 26, 2017 at 6:31 PM, Alexander Behm <alex.behm@cloudera.com>
> wrote:
>
>> You should be able to bring Impala up without HDFS by passing the
>> "--abort_on_config_error=false" startup flag.
>>
>> Just to clarify:
>> You still need the Hive Metastore with Kudu because table and column
>> statistics are stored in the Metastore; our plan is to eventually remove
>> this dependency
>>
>> On Wed, Apr 26, 2017 at 1:28 AM, Aleksei Maželis <olmazh@gmail.com>
>> wrote:
>>
>>> Ok, I see. I remember trying to run Impala-Kudu without HDFS, and
>>> remember that Impala failed to start. But perhaps there is a way to make
>>> Impala work without HDFS after re-configuring it properly; any hints on
>>> which configurations to change would be very helpful!
>>>
>>> BR,
>>> Alex
>>>
>>> On Tue, Apr 25, 2017 at 7:36 PM, Jim Apple <jbapple@cloudera.com> wrote:
>>>
>>>> I believe the Hive Metastore is needed even when Kudu is the storage
>>>> engine. I don't know if the HDFS namenode and datanodes are needed.
>>>>
>>>> On Mon, Apr 24, 2017 at 10:28 PM, Aleksei Maželis <olmazh@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am looking for the minimal set of dependencies that an Impala-Kudu
>>>>> setup will have. When reading how-tos and checking available dockerfiles,
>>>>> the list of items that Impala depends on seems to include at least:
>>>>> - hadoop-hdfs-namenode
>>>>> - hadoop-hdfs-datanode
>>>>> - hive-metastore.
>>>>>
>>>>> However, if I understand correctly, when Kudu is used along with
>>>>> Impala, these aren't necessarily used. So, the question is if the above
>>>>> dependencies of HDFS and Hive can be avoided, and if so, whether there
are
>>>>> side effects this would bring. And of course, if I am missing some other
>>>>> mandatory dependencies, hints on those would be more then welcome!
>>>>>
>>>>> With best regards,
>>>>> Alex
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message