Mailing-List: contact opennlp-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: opennlp-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of jasonbaldridge@gmail.com
 designates 209.85.220.175 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:reply-to:date:message-id:subject:from:to:content-type;
        b=dF9LvuSGEb4UTvK22HGP6fFFzyQDHq4Z1wkltV3bRZMwWRrdXhZMVxR4fFU2Al57p/
         hH/OrvCjW4p8IWklbjgbsplu0Tbw3xg65RhHUSAFU8ZLHHOnoj35j9cWnKTJQiZEWjoq
         C1eKSAVljJex2Lk/sSyYxEg7dIq73dSK68/Vg=
MIME-Version: 1.0
Reply-To: jbaldrid@mail.utexas.edu
Date: Wed, 13 Apr 2011 22:15:19 -0500
Message-ID: <BANLkTi=VO8_Rt426vsy4RXGj4vyr5iqZ6A@mail.gmail.com>
Subject: universal tagset
From: Jason Baldridge <jasonbaldridge@gmail.com>
To: opennlp-dev@incubator.apache.org
Content-Type: multipart/alternative; boundary=bcaec50162a1d1950204a0d8558d

--bcaec50162a1d1950204a0d8558d
Content-Type: text/plain; charset=ISO-8859-1

For many applications, it would be useful to have a universal tagset for any
language you are working with. See below for details on a project that
provides mappings from many standard treebanks to a course-grained tagset
(12 tags). We might want to support these mappings to simple tags in our
models (e.g. have a model that uses corpus-native tags and another that uses
universal tags).

Jason

-- 
Jason Baldridge
Assistant Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge


Hi everyone,

some of you have already heard about our universal part-of-speech
tagset (and are even using it), to others this might be new.

We sat down and read through the annotation guidelines of 25 treebanks
and created a mapping to an universal set of 12 coarse-grained
part-of-speech categories. We have described the tagset and
illustrated some use cases in a short write-up (see attached pdf).
Additionally, we have uploaded the mappings to a code repository with
version control so that new languages can be added or modification can
be made if necessary:
http://code.google.com/p/universal-pos-tags/

The paper is for now on arXiv:
http://arxiv.org/abs/1104.2086

We hope that you will find this resource useful for your own work. Let
us know if you have any comments,
Cheers,
Dipanjan, Ryan & Slav

--bcaec50162a1d1950204a0d8558d--