Mailing-List: contact users-help@trafficserver.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@trafficserver.apache.org
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Subject: Re: FATAL: ats_memalign: couldn't allocate -548249600 bytes
From: James Peach <jpeach@apache.org>
In-Reply-To: <50784612.3020400@nottheoilrig.com>
Date: Sat, 13 Oct 2012 11:58:10 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <9ED91AE2-2F52-4BDB-9088-E14D40642C34@apache.org>
References: <50784612.3020400@nottheoilrig.com>
To: users@trafficserver.apache.org

On 12/10/2012, at 9:32 AM, Jack Bates <duhapa@nottheoilrig.com> wrote:

> Hi, I'm consistently getting the following error whenever I try to =
start Traffic Server (release 3.2.0). Yesterday I built Traffic Server =
from Git HEAD (34a2ba) to check if it behaves any differently, but I =
consistently reproduce the same error whenever I try to start it, too
>=20
> Can anyone please tell me anything about this error? Do you think it's =
a bug, and should I open an issue about it?
>=20
> "couldn't allocate -548249600 bytes" - is this an overflow of some =
kind?
>=20
> Here's my configuration (which is pretty minimal): =
http://nottheoilrig.com/trafficserver/201210120/
>=20
> Please let me know if any more details would be helpful
>=20
> administrator@debian$ TS_ROOT=3D/home/administrator/trafficserver =
trafficserver/traffic_server
> [TrafficServer] using root directory =
'/home/administrator/trafficserver'
> FATAL: ats_memalign: couldn't allocate -548249600 bytes at alignment =
4096 - insufficient memory
> trafficserver/traffic_server - STACK TRACE:
> trafficserver/libtsutil.so.3(+0x1075b)[0xb76d075b]
> trafficserver/libtsutil.so.3(ats_memalign+0xa1)[0xb76d34c1]

That memalign is from Vol::init()

  1077   raw_dir =3D (char *)ats_memalign(sysconf(_SC_PAGESIZE), =
vol_dirlen(this));

where vol_dirlen is:

   331  TS_INLINE int
   332  vol_headerlen(Vol *d) {
   333    return ROUND_TO_STORE_BLOCK(sizeof(VolHeaderFooter) + =
sizeof(uint16_t) * (d->segments-1));
   334  }
   335 =20
   336  TS_INLINE size_t
   337  vol_dirlen(Vol *d)
   338  {
   339    return vol_headerlen(d) +=20
   340      ROUND_TO_STORE_BLOCK(((size_t)d->buckets) * DIR_DEPTH * =
d->segments * SIZEOF_DIR) +
   341      ROUND_TO_STORE_BLOCK(sizeof(VolHeaderFooter));
   342  }
   343 =20


So I'd guess that either d->buckets or d->segments is bogus. Since it's =
persistent, that might mean that the on-disk cache structure are corrupt =
or invalid. I think that this deserves a jira ticket; if you can attach =
some kind of dump of the volume header that would be helpful. I'd =
suggest asking in the #traffic-server IRC channel about what is needed =
to debug this.

To recover your installation, I expect you can zero the block device =
with dd.

> trafficserver/traffic_server(_ZN3Vol4initEPcxxb+0x282)[0x827bd52]
> trafficserver/traffic_server(_ZN5Cache4openEbb+0x5d8)[0x827dc48]
> =
trafficserver/traffic_server(_ZN14CacheProcessor15diskInitializedEv+0x323)=
[0x827e0d3]
> =
trafficserver/traffic_server(_ZN9CacheDisk9openStartEiPv+0x483)[0x828c9c3]=

> =
trafficserver/traffic_server(_ZN19AIOCallbackInternal11io_completeEiPv+0x2=
5)[0x8280a75]
> =
trafficserver/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8b)[0x8=
30343b]
> trafficserver/traffic_server(_ZN7EThread7executeEv+0x723)[0x8304003]
> trafficserver/traffic_server(main+0x178d)[0x80c572d]
> =
/lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7039e46=
]
> trafficserver/traffic_server[0x80cabdd]
> administrator@debian:~$