hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tim robertson <timrobertson...@gmail.com>
Subject Re: Annotating all rows
Date Fri, 24 Jul 2009 16:24:24 GMT
Thanks St.Ack,

The memory isn't ECC memory.
I'll run some disk utilities on the failed region and see what is up.

Would a native HDFS put throw this error?  I have loaded 100s of GB
into the cluster using the HDFS put without issue.  It is only the
HBase loading throwing it.  Perhaps that does not use checksums.

Cheers

Tim


On Fri, Jul 24, 2009 at 6:20 PM, stack<stack@duboce.net> wrote:
> I tried to reply up on hbase-user to your checksum issue but its bouncing
> for me for some reason.
>
>
> Do these macs have ECC memory in them? Are the disks having issue?
>
> Checksum errors are usually indicative of hardware issues.
>
> St.Ack
>
>
> Also, thanks for repeating your system details in your original message.
> Its hard for old fellas like me to keep it all straight so its good having
> the details repeated for them.
>
>
>
>
> On Thu, Jul 23, 2009 at 2:23 PM, stack <stack@duboce.net> wrote:
>>
>> On Thu, Jul 23, 2009 at 11:35 AM, tim robertson
>> <timrobertson100@gmail.com> wrote:
>>>
>>> I thought you had said there might be issues updating a row while
>>> scanning because of a RegionSplit caused when updating might break the
>>> running scan.  I must have misunderstood - happy to hear it should
>>> work.
>>
>>
>> Hmm. I probably said that, yeah.  Should work in TRUNK.  If it don't its a
>> bug.  Lets fix.
>>
>>
>>> Yeah the mac minis do fail on loading.  I haven't got more than 20
>>> million rows in yet.  One time I mysteriously lost a Region due to a
>>> timeout and quite a few times both in the mapreduce and the single
>>> threaded load it just all hangs and does nothing.
>>
>> Send me logs next time - put them somewhere I can pull them.
>>
>>
>>>
>>> When I come to shut
>>> it down, the master is hanging shortly after a log message saying
>>> something like "stopping master thread" - sorry I'm at home and don't
>>> know the exact error.
>>> The thing is Stack, I've have been changing a lot of config and trying
>>> various things so I have not really been too concerned.  Now I am
>>> pretty happy I know how to do correct config I will post any errors I
>>> spot to the list and try and debug it - my feeling is at this stage it
>>> is probably been my fault with dodgy config.
>>
>> Well, it shouldn't be so hard.
>>
>> Lets figure a way where I can get logs easy or get on the machines even so
>> can poke around.
>>
>>>
>>> One recurring pattern I do see is if anything goes wrong with
>>> ZooKeeper (errors like "my IP does not appear in the ZK list"), the
>>> master will not shutdown at all and I have to kill the Master process.
>>
>>
>>
>> Yeah, I've seen that one.  We're hosed w/o ZK.
>>
>> St.Ack
>>
>>
>>>
>>> Cheers,
>>>
>>> Tim
>>>
>>> On Thu, Jul 23, 2009 at 8:19 PM, stack<stack@duboce.net> wrote:
>>> > On Thu, Jul 23, 2009 at 11:12 AM, tim robertson
>>> > <timrobertson100@gmail.com>
>>> > wrote:
>>> >>
>>> >> Not tried yet - I meant I must have misunderstood something from
>>> >> Amsterdam.
>>> >
>>> > I'm not clear on what was misunderstood.  I want to know in case I gave
>>> > you
>>> > wrong impression.
>>> >
>>> > If you want to scan and update as you go, that works.
>>> >
>>> >
>>> >>
>>> >> I'm have never succeeded in getting all the data in yet - I'm learning
>>> >> a lot at the moment though at least
>>> >
>>> > This is on the mac minis?
>>> >
>>> > The job fails? I thought it just ran slow?
>>> >
>>> > St.Ack
>>> >
>>> >>
>>> >> I'll keep you posted
>>> >>
>>> >> Cheers
>>> >>
>>> >> Tim
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Jul 23, 2009 at 7:40 PM, stack<stack@duboce.net> wrote:
>>> >> > On Thu, Jul 23, 2009 at 12:29 AM, tim robertson
>>> >> > <timrobertson100@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Thanks Stack
>>> >> >>
>>> >> >> I must have misunderstood about the scanning
>>> >> >>
>>> >> >
>>> >> > Its not working for you?
>>> >> > St.Ack
>>> >> >
>>> >> >
>>> >
>>> >
>>
>
>

Mime
View raw message