hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: the performance of HDFS
Date Wed, 26 Jan 2011 08:12:47 GMT
You may also see it in /proc/scsi/scsi, perhaps.

On Wed, Jan 26, 2011 at 9:27 AM, Da Zheng <zhengda1936@gmail.com> wrote:
> unfortunately, the command isn't available in the system and I don't have
> privilege to install software:-(
>
>
> On 1/25/11 9:37 PM, Ted Dunning wrote:
>> Perhaps lshw would help you.
>>
>> ubuntu:~$ sudo lshw
>>    ...
>>         *-storage
>>              description: RAID bus controller
>>              product: SB700/SB800 SATA Controller [Non-RAID5 mode]
>>              vendor: ATI Technologies Inc
>>              physical id: 11
>>              bus info: pci@0000:00:11.0
>>              logical name: scsi0
>>              logical name: scsi1
>>              version: 00
>>              width: 32 bits
>>              clock: 66MHz
>>              capabilities: storage pm bus_master cap_list emulated
>>              configuration: driver=ahci latency=64
>>              resources: irq:22 ioport:b000(size=8) ioport:a000(size=4)
>> ioport:9000(size=8) ioport:8000(size=4) ioport:7000(size=16)
>> memory:fe7ffc00-fe7fffff
>>            *-disk
>>                 description: ATA Disk
>>                 product: ST3750528AS
>>                 vendor: Seagate
>>                 physical id: 0
>>                 bus info: scsi@0:0.0.0
>>    ...
>>
>> On Tue, Jan 25, 2011 at 6:29 PM, Da Zheng <zhengda1936@gmail.com> wrote:
>>
>>> No, each node in the cluster is powerful server. I was told the nodes are
>>> Dell
>>> Poweredge SC1435, but I cannot figure out the configuration of hard drives.
>>> Dell
>>> provides several possible hard drives for this model.
>>>
>>> On 1/25/11 7:59 PM, Ted Dunning wrote:
>>>> This is a really slow drive or controller.
>>>>
>>>> Consumer grade 3.5 inch 2TB drives typically can handle 100MB/s.  I would
>>>> suspect in the absence of real information that your controller is more
>>>> likely to be deficient than your drive.  If this is on a laptop or
>>>> something, then I withdraw my thought.
>>>>
>>>> On Tue, Jan 25, 2011 at 4:50 PM, Da Zheng <zhengda1936@gmail.com> wrote:
>>>>
>>>>> The aggregate write-rate can get much higher if you use more drives,
but
>>> a
>>>>>> single stream throughput is limited to the speed of one disk spindle.
>>>>>>
>>>>>>  You are right. I measure the performance of the hard drive. It
seems
>>> the
>>>>> bottleneck is the hard drive, but the hard drive is a little too slow.
>>> The
>>>>> average writing rate is 50MB/s.
>>>>
>>>
>>>
>>
>
>



-- 
Harsh J
www.harshj.com

Mime
View raw message