accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject Re: Table splitting
Date Wed, 12 Dec 2012 18:36:12 GMT
No.  If a compaction takes place, it will tend to make a block local again.

We do keep track of the last location in which a file was written so we can
attempt to put a tablet back there, but that's about all the server does to
preserve locality.

-Eric


On Wed, Dec 12, 2012 at 1:32 PM, ameet kini <ameetkini@gmail.com> wrote:

> If its moved for balance or recovery purposes, are there any mechanisms to
> copy the blocks over to the new location? Would compaction be this
> mechanism? Or is it automatically done as part of tablet migration?
>
>
> On Wed, Dec 12, 2012 at 1:29 PM, Eric Newton <eric.newton@gmail.com>wrote:
>
>> Well, you have to assume the tablet does not get moved for balancing or
>> recovery.
>>
>> -Eric
>>
>>
>> On Wed, Dec 12, 2012 at 1:27 PM, ameet kini <ameetkini@gmail.com> wrote:
>>
>>>
>>> Ok, so in short, assuming that there's sufficient local disk space, a
>>> given tablet having all its blocks local relies on HDFS's guarantee that
>>> the first replica of a block will be local as long as the tablet server is
>>> also a data node. Yes?
>>>
>>>
>>>
>>> On Wed, Dec 12, 2012 at 1:18 PM, Eric Newton <eric.newton@gmail.com>wrote:
>>>
>>>> Check out o.a.a.server.util.LocalityCheck
>>>>
>>>> -Eric
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 1:17 PM, John Vines <vines@apache.org> wrote:
>>>>
>>>>> When a file gets written to hdfs, there is a guarantee the file is
>>>>> local as long as that systems disks are not full. Accumulo does not have
a
>>>>> locality guarantee as tablets will migrate on occasion. However, as data
is
>>>>> added, major compactions will occur which will restore locality.
>>>>>  On Dec 12, 2012 1:09 PM, "ameet kini" <ameetkini@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> Along these lines....
>>>>>>
>>>>>> Can someone help me understand how tablets map to files on disk in
>>>>>> HDFS? From what I understand, after a compaction, there may be one
(or
>>>>>> more?) files on HDFS for a given tablet. Each file can consist of
multiple
>>>>>> HDFS blocks. Does Accumulo guarantee that the tablet serving a given
data
>>>>>> range finds all its blocks locally? If so, how does it keep this
guarantee?
>>>>>> Wouldn't HDFS distribute these blocks around based on HDFS balancing
>>>>>> strategy?
>>>>>>
>>>>>> Thanks,
>>>>>> Ameet
>>>>>>
>>>>>> On Tue, Dec 11, 2012 at 9:37 AM, William Slacum <
>>>>>> wilhelm.von.cloud@accumulo.net> wrote:
>>>>>>
>>>>>>> Tablets will split automatically, down to the granularity of
a row.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 11, 2012 at 9:32 AM, Mathias Herberts <
>>>>>>> mathias.herberts@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've read the user manual for v1.4.2 and I have not seen
any
>>>>>>>> mention of automatic tablet splitting. Is there such a thing
in Accumulo or
>>>>>>>> is pre-splitting the only way to split a table?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Mathias.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>

Mime
View raw message