Author: sebor
Date: Thu Sep 21 17:42:16 2006
New Revision: 448754

URL: http://svn.apache.org/viewvc?view=rev&rev=448754
Log:
2006-09-21  Martin Sebor  <sebor@roguewave.com>

	* aliases.cpp (get_installed_locales): Redirected locale's stderr
	to /dev/null to prevent it from cluttering our own. Eliminated
	the handling of invalid category values.
	* charmap.h (<list>): Included.
	(portable_charset): Declared.
	(get_n_cmap, get_rn_cmap, get_w_cmap, get_rw_cmap): Documented.
	(get_n_cmap2, get_rn_cmap2, get_strval_map, increment_val): Renamed...
	(get_mb_cmap, get_rmb_cmap, get_symnames_list, increment_wchar): ...to
	this.
	(get_strval_map): Replaced...
	(get_symnames_list): ...with this.
	(process_ellipsis): Changed to return the number of characters denoted
	by the ellipsis instead of void.
	(increment_strval): Renamed...
	(increment_encoding): ...to this for clarity and changed it to return
	true on success and false otherwise.
	(verify_portable_charset): Made const.
	(parse_ext_strval): Renamed...
	(encoding_to_mbchar): ...to this for clarity.
	(add_to_cmaps): Added a bool argument.
	(n_cmap2_, rn_cmap2_): Renamed...
	(mb_cmap_, rmb_cmap_): ...to this for clarity.
	(valid_mb_set_, valid_wchar_set_, strval_map_): Removed unused members.
	(symnames_list_): Added new member.
	* def.h (process_abs_ellipsis): New.
	(process_sym_ellipsis): Returned the number of characters denoted by
	the ellipsis instead of void.
	(hex_increment, dec_increment): Removed (defined as static non-members
	in collate.cpp).
	(process_mask): Added an argument denoting the human readable name
	of the mask for progress diagnostics.
	(process_transliteration, process_transliteration_statement,
	generate_xliteration_data): Renamed...
	(process_xlit, process_xlit_statement, gen_xlit_data): ...to this for
	brevity.
	(extract_string_array): New.
	(ctype_offset_tab_t): Renamed...
	(codecvt_offset_tab_t): ...to this.
	(wchar_off_map_, mb_char_offs_, wchar_offs_, wchar_utf8_to_ext_,
	utf8_offs_, valid_codecvt_utf8_set_, utf8_map_, next_codecvt_tab_num_,
	next_wchar_codecvt_tab_num_, next_utf8_codecvt_tab_num_, codecvt_out_):
	Removed data members.
	(mb_char_offs_iter, wchar_offs_iter, utf8_offs_iter, utf8_map_iter,
	(gen_valid_codecvt_wchar_set, gen_valid_codecvt_utf8_set, gen_utf8_map,
	generate_wchar_codecvt_table, generate_utf8_codecvt_table):
	Replaced member functions...
	(gen_mbchar_tables, gen_wchar_tables, gen_utf8_tables): ...with these.
	* diagnostic.h (E_NOTSUP, W_NOTSUP, W_CHARMAP, I_SKIP): Added.
	(W_COLVAL): Removed.
	* scanner.h (token_id): Arranged enumerators by LC_XXX section and
	documented. Added enumerators for LC_ADDRESS and other sections defined
	by ISO/IEC TR 14652.
	(tok_ellipsis, tok_dellipsis, tok_qellipsis, tok_doub_inc_ellipsis):
	Renamed...
	(tok_abs_ellipsis, tok_hex_ellipsis, tok_dec_ellipsis,
	tok_dbl_ellipsis): ...to this for better correspondence to ISO/IEC
	TR 14652 names.
	(tok_eof, tok_cont_line): Removed unused names.
	(tok_octal_value, tok_decimal_value, tok_hex_value): Replaced...
	(tok_char_value): ...with this for simplicity.
	* charmap.cpp (convert_escape): Removed dead code.
	(portable_charset): Defined.
	(convert_to_wc): Simplified, added comments.
	(increment_val): Renamed...
	(increment_wchar): ...to this, renamed locals for clarity, and replaced
	hardcoded 0xff with UCHAR_MAX (is this really correct when char is more
	that 8 bits wide?).
	(increment_strval): Renamed...
	(increment_encoding): ...to this, removed local statics, and optimized.
	The function is unused since it is not possible to increment a multibyte
	character so that its last byte overflows (i.e., wraps around to 0). It
	should/will be removed in a future commit.
	(parse_ext_strval): Renamed...
	(encoding_to_mbchar): ...to this and renamed locals.
	(convert_sym_to_ucs): Tightened up the checking of symbolic character
	names for validity (expect a hex digit after the leading "<U" prefix).
	Used strtoul instead strtol for simplicity.
	(convert_to_ucs): Adjusted formatting.
	(add_to_cmaps): Added an argument indicating whether the encoding
	is multibyte or whether it should be converted to it. Optimized and
	added inline comments.
	(process_ellipsis): Renamed locals, added inline comments and adjusted
	formatting for clarity. Corrected sprintf format. Avoided calling
	increment_strval and instead incremented the last byte of a multibyte
	character, detecting and diagnosing overflow, for efficiency.
	(process_chars): Added a progress message at the beginning of the
	function. Counted the number of tokens, ellipses, and characters
	processed. Added inline comments. Simplified the handling of numeric
	character values (tok_char_value). Used tok_abs_ellipsis,
	tok_hex_ellipsis, and tok_dec_ellipsis.  Added a progress message
	at the end of the function indicating the number of of tokens, ellipses,
	and characters processed.
	* codecvt.cpp (gen_valid_codecvt_mb_set, gen_valid_codecvt_wchar_set,
	gen_valid_codecvt_utf8_set, generate_codecvt_table, gen_utf8_map,
	generate_wchar_codecvt_table, -generate_utf8_codecvt_table): Replaced...
	(gen_mbchar_tables, gen_wchar_tables, gen_utf8_tables): ...with these
	functions and simplified by replacing member variables with function
	arguments wherever possible.
	(create_wchar_utf8_table): Moved to collate.cpp where the function
	is actually used.	
	(generate_xliteration_data): Renamed...
	(gen_xlit_data): ...to this.
	(write_codecvt): Called gen_mbchar_tables, gen_wchar_tables, and
	gen_utf8_tables for better locality of reference and documented.
	* collate.cpp (lc_name): New.
	(create_wchar_utf8_table): Moved from codecvt.cpp.
	(dec_increment, hex_increment): Made static non-members.
	(process_weights, add_missing_values, process_collate, write_collate):
	Used preincrement instead of postincrement with iterators for better
	efficiency.
	* ctype.cpp (lc_name): New.
	(convert_literal_to_ucs4): Handled tok_char_value instead of
	tok_octal_value, tok_decimal_value, and tok_hex_value.
	(process_abs_ellipsis): New function.
	(process_sym_ellipsis): Simplified.
	(process_mask): Added an argument, displayed progress message, and
	simplified.
	(process_upper_lower): Displayed progress message, and simplified.
	(process_transliteration, process_transliteration_statement): Renamed...
	(process_xlit, process_xlit_statement): ...to this.
	(process_ctype): Added a progress diagnostic.
	* def.cpp (convert_to_ext): Used rmb_cmap_iter.
	(process_input): Removed progress diagnostics.
	(verify_portable_charset): Used portable_charset.
	* diagnostic.cpp (issue_diag): Used fputs instead of fprintf where
	appropriate and simplified.
	* messages.cpp (lc_name): New.
	(process_messages): Used lc_name, issued an informative diagnostic,
	and simplified ever so slightly.
	* monetary.cpp (lc_name): New.
	(process_monetary): Used lc_name. Issued an informative diagnostic.
	* numeric.cpp (lc_name): New.
	(process_numeric): Used lc_name, issued an informative diagnostic,
	and simplified ever so slightly.
	* time.cpp (lc_name): New.
	(parse_era): Handled empty era strings and constified locals.
	(extract_string_array): Defined new member function of class Def.
	(process_time): Used lc_name, issued an informative diagnostic,
	called extract_string_array.
	(write_time): Issued an informative diagnostic and simplified ever
	so slightly.
	* scanner.cpp (ScannerContext): Declared copy ctor and assignment
	operator private and removed definitions.
	(process_token): Simplified the handling of numeric character values,
	removed unnecessary qualification from the definition of tok_map array
	elements.
	(next_token): Used the new names tok_abs_ellipsis, tok_hex_ellipsis,
	tok_dec_ellipsis, and tok_dbl_ellipsis, eliminated a C-style const
	cast to avoid a gcc warning.
	* localedef.cpp (--notes): Documented option.
	* locale.cpp: Declared all definitions static, renamed many locals for
	clarity and/or brevity.
	(collate_charmap, ctype_charmap, time_charmap, num_charmap, mon_charmap,
	messages_charmap, current_locales, is_env_set): Removed globals (moved
	into get_charmap()).
	(sect_charmap, sect_lc_all, sect_lc_collate, sect_lc_ctype, ect_lc_time,
	sect_lc_messages, sect_lc_monetary, sect_lc_numeric): Defined new global
	constants.
	(UChar): New convenience typedef.
	(validate): Tweaked text of error message.
	(initialize_struct): Renamed...
	(init_struct): ...to this for brevity, removed redundant arguments
	and added their static local equivalents.
	(initialize): Renamed...
	(init_section): ...to this and removed redundant arguments.
	(initialize_structs): Renamed...
	(init_sections): ...to this. Called init_section with fewer arguments.
	(get_charmap): Added static locals for better encapsulation and locality
	of reference.
	(escape_value): New function to handling the formatting of characters
	and weights using escape sequences.
	(create_str): Simolified.
	(print_section, print_keyword): New helper functions.
	(print_weight): Simplified.
	(write_coll_info): Renamed locals.
	(print_ce_info): Added comments, adjusted code formatting.
	(print_lc_collate, print_lc_ctype, print_lc_messages, print_lc_numeric,
	print_lc_time, print_lc_all): Printed comments on stdout with additional
	detail about each section. Simplified.
	(print_int): New helper to handle the printing of numeric values.
	(print_time_keyword): New helper to handle the printing of LC_TIME
	keywords.
	(print_mask, print_toupper, print_tolower): Simplified, added inline
	comments, adjusted formatting.
	(print_frac_digits, ...): Called print_int or print_time_keyword
	as appropriate for simplicity and to avoid code duplication.
	(print_help_msg): Renamed...
	(print_help): ...to this.
	(initialize_env_vars): Removed (done in get_charmap).
	(print_locale_name): Removed.
	(print_charmap): Collected character statistics.
	(print_charmap): Added a default argument, computed additional stats.
	(print_lc_var): New function to handle the formatting of localization
	environment variables according to POSIX rules.
	(handlers): New array of command line argument handlers.
	(main): Used handlers. Added -p to force locale to use symbolic
	character names from the Portable Character Set whenever possible.

Modified:
    incubator/stdcxx/trunk/util/aliases.cpp
    incubator/stdcxx/trunk/util/charmap.cpp
    incubator/stdcxx/trunk/util/charmap.h
    incubator/stdcxx/trunk/util/codecvt.cpp
    incubator/stdcxx/trunk/util/collate.cpp
    incubator/stdcxx/trunk/util/ctype.cpp
    incubator/stdcxx/trunk/util/def.cpp
    incubator/stdcxx/trunk/util/def.h
    incubator/stdcxx/trunk/util/diagnostic.cpp
    incubator/stdcxx/trunk/util/diagnostic.h
    incubator/stdcxx/trunk/util/locale.cpp
    incubator/stdcxx/trunk/util/localedef.cpp
    incubator/stdcxx/trunk/util/messages.cpp
    incubator/stdcxx/trunk/util/monetary.cpp
    incubator/stdcxx/trunk/util/numeric.cpp
    incubator/stdcxx/trunk/util/scanner.cpp
    incubator/stdcxx/trunk/util/scanner.h
    incubator/stdcxx/trunk/util/time.cpp

Modified: incubator/stdcxx/trunk/util/aliases.cpp
URL: http://svn.apache.org/viewvc/incubator/stdcxx/trunk/util/aliases.cpp?view=diff&rev=448754&r1=448753&r2=448754
==============================================================================
--- incubator/stdcxx/trunk/util/aliases.cpp (original)
+++ incubator/stdcxx/trunk/util/aliases.cpp Thu Sep 21 17:42:16 2006
@@ -38,7 +38,7 @@
 #include <cstdlib>
 #include <cstdio>
 #include <cstring>   // for memcpy(), strlen()
-#include <clocale>
+#include <clocale>   // for setlocale()
 #include <locale>
 #include <iostream>
 #include <string>
@@ -541,10 +541,10 @@
     }
 
     char* locname = slocname;
-    char* save_localename = 0;
 
-    if (loc_cat != int (LC_INVALID_CAT))
-        save_localename = std::setlocale (loc_cat, 0);
+    // save the current locale setting and set the locale to "C"
+    const char* const save_localename = std::setlocale (LC_ALL, 0);
+    std::setlocale (LC_ALL, "C");
 
 #if __GNUG__ == 2 && __GNUC_MINOR__ == 96
 
@@ -582,11 +582,11 @@
     //   sizeof ("locale -a | grep \"\" > ")   // 22
     // + strlen (fname)                        // must be <= L_tmpnam
 
-    char cmd [22 + L_tmpnam];
+    char cmd [80 + L_tmpnam];
 
-    std::sprintf (cmd, "/usr/bin/locale -a > %s 2>/dev/null", fname);
+    std::sprintf (cmd, "LC_ALL=C /usr/bin/locale -a >%s 2>/dev/null", fname);
 
-    int ret = std::system(cmd);
+    const int ret = std::system (cmd);
 
     if (ret && ret != 256) {
         std::strcpy (slocname, "call to system ");
@@ -653,8 +653,10 @@
         *locname = '\0';
 
     }
-    if (loc_cat != int (LC_INVALID_CAT))
-        std::setlocale (loc_cat, save_localename);
+
+    // restore the original locale
+    if (save_localename)
+        std::setlocale (LC_ALL, save_localename);
 
     std::fclose (f);
     std::remove (fname);

Modified: incubator/stdcxx/trunk/util/charmap.cpp
URL: http://svn.apache.org/viewvc/incubator/stdcxx/trunk/util/charmap.cpp?view=diff&rev=448754&r1=448753&r2=448754
==============================================================================
--- incubator/stdcxx/trunk/util/charmap.cpp (original)
+++ incubator/stdcxx/trunk/util/charmap.cpp Thu Sep 21 17:42:16 2006
@@ -45,7 +45,7 @@
 #include <clocale>    // for LC_CTYPE, setlocale()
 #include <cstdio>
 #include <cstdlib>
-#include <cstring>    // for strerror()
+#include <cstring>    // for strrchr(), strerror()
 
 #include <map>
 #include <string>
@@ -234,66 +234,164 @@
 }
 
 
-#if 0
-
-// convert the first character of a string to an unsigned char 
-unsigned char Charmap::
-convert_escape (const char *str, const char **pend /* = 0 */) const
+// count the number of bytes in a multibyte sequence denoted
+// by the argument by counting the number of escape characters
+std::size_t Charmap::mbcharlen (const std::string &str) const
 {
-    assert (str != 0);
-
-    if (!*str || *str != scanner_.escape_char ())
-        issue_diag (E_CVT, true, &next,
-                    "unable to convert character %s\n", str);
+    std::size_t count = 1;
 
-    long ch = 0;
+    const char escape = scanner_.escape_char ();
 
-    char *end;
+    for (std::size_t idx = 0; ; ++idx, ++count) {
+        idx = str.find (escape, idx);
 
-    switch (str [1]) {
-    case '%': ch = std::strtol (str + 2, &end, 8); break;
-    case 'd': ch = std::strtol (str + 2, &end, 10); break;
-    case 'x': ch = std::strtol (str + 2, &end, 16); break;
-    default: ch = -1; end = _RWSTD_CONST_CAST (char*, str) + 1;
+        if (std::string::npos == idx)
+            break;
     }
 
-    if (end == str + 2 || (ch < 0 || ch > UCHAR_MAX))
-        issue_diag (E_IFMT, true, &next,
-                    "unable to convert character %s\n", str);
-
-    if (pend)
-        *pend = end;
-
-    typedef unsigned char UChar;
-
-    return UChar (ch);
+    return count;
 }
 
-#endif
 
+/**************************************************************************/
 
-// find the number of bytes in the multibyte string by counting the
-// number of escape chars in the string
-size_t Charmap::mbcharlen (const std::string &str) const
-{
-    std::size_t count = 0;
-
-    std::size_t idx = str.find (scanner_.escape_char ());
-
-    for (; std::string::npos != idx; ++count) {
-        idx = str.find (scanner_.escape_char (), idx + 1);
-    }
-
-    return count;
-}
-
+const char* const Charmap::
+portable_charset[] = {
+    /* 0x00       */ "<NUL>",
+    /* 0x01   SOH */ 0,
+    /* 0x02   STX */ 0,
+    /* 0x03   ETX */ 0,
+    /* 0x04   EOT */ 0,
+    /* 0x05   ENQ */ 0,
+    /* 0x06   ACK */ 0,
+    /* 0x07   BEL */ "<alert>",
+    /* 0x08       */ "<backspace>",
+    /* 0x09   TAB */ "<tab>",
+    /* 0x0a       */ "<newline>", 
+    /* 0x0b       */ "<vertical-tab>",
+    /* 0x0c       */ "<form-feed>",
+    /* 0x0d       */ "<carriage-return>",
+    /* 0x0e   SO  */ 0,
+    /* 0x0f   SI  */ 0,
+    /* 0x10   DLE */ 0,
+    /* 0x11   DC1 */ 0,
+    /* 0x12   DC2 */ 0,
+    /* 0x13   DC3 */ 0,
+    /* 0x14   DC4 */ 0,
+    /* 0x15   NAK */ 0,
+    /* 0x16   SYN */ 0,
+    /* 0x17   ETB */ 0,
+    /* 0x18   CAN */ 0,
+    /* 0x19   EM  */ 0,
+    /* 0x1a   SUB */ 0,
+    /* 0x1b   ESC */ 0,
+    /* 0x1c   IS4 */ 0,
+    /* 0x1d   IS3 */ 0,
+    /* 0x1e   IS2 */ 0,
+    /* 0x1f   IS1 */ 0,
+    /* 0x20   SPC */ "<space>",
+    /* 0x21    !  */ "<exclamation-mark>",
+    /* 0x22    '  */ "<quotation-mark>",
+    /* 0x23    #  */ "<number-sign>", 
+    /* 0x24    $  */ "<dollar-sign>",
+    /* 0x25    %  */ "<percent-sign>",
+    /* 0x26    &  */ "<ampersand>",
+    /* 0x27    '  */ "<apostrophe>",
+    /* 0x28    (  */ "<left-parenthesis>",
+    /* 0x29    )  */ "<right-parenthesis>",
+    /* 0x2a    *  */ "<asterisk>", 
+    /* 0x2b    +  */ "<plus-sign>",
+    /* 0x2c    ,  */ "<comma>",
+    /* 0x2d    -  */ "<hyphen>",   // "<hyphen-minus>",
+    /* 0x2e    .  */ "<period>",   // "<full-stop>",
+    /* 0x2f    /  */ "<slash>",    // "<solidus>",
+    /* 0x30    0  */ "<zero>",
+    /* 0x31    1  */ "<one>",
+    /* 0x32    2  */ "<two>",
+    /* 0x33    3  */ "<three>",
+    /* 0x34    4  */ "<four>",
+    /* 0x35    5  */ "<five>",
+    /* 0x36    6  */ "<six>",
+    /* 0x37    7  */ "<seven>",
+    /* 0x38    8  */ "<eight>",
+    /* 0x39    9  */ "<nine>",
+    /* 0x3a    :  */ "<colon>",
+    /* 0x3b    ;  */ "<semicolon>",
+    /* 0x3c    <  */ "<less-than-sign>", 
+    /* 0x3d    =  */ "<equals-sign>",
+    /* 0x3e    >  */ "<greater-than-sign>",
+    /* 0x3f    ?  */ "<question-mark>",
+    /* 0x40    @  */ "<commercial-at>",
+    /* 0x41    A  */ "<A>",
+    /* 0x42    B  */ "<B>",
+    /* 0x43    C  */ "<C>",
+    /* 0x44    D  */ "<D>",
+    /* 0x45    E  */ "<E>",
+    /* 0x46    F  */ "<F>",
+    /* 0x47    G  */ "<G>",
+    /* 0x48    H  */ "<H>",
+    /* 0x49    I  */ "<I>",
+    /* 0x4a    J  */ "<J>",
+    /* 0x4b    K  */ "<K>",
+    /* 0x4c    L  */ "<L>",
+    /* 0x4d    M  */ "<M>",
+    /* 0x4e    N  */ "<N>",
+    /* 0x4f    O  */ "<O>",
+    /* 0x50    P  */ "<P>",
+    /* 0x51    Q  */ "<Q>",
+    /* 0x52    R  */ "<R>",
+    /* 0x53    S  */ "<S>",
+    /* 0x54    T  */ "<T>",
+    /* 0x55    U  */ "<U>",
+    /* 0x56    V  */ "<V>",
+    /* 0x57    W  */ "<W>",
+    /* 0x58    X  */ "<X>",
+    /* 0x59    Y  */ "<Y>",
+    /* 0x5a    Z  */ "<Z>",
+    /* 0x5b    [  */ "<left-square-bracket>",
+    /* 0x5c    \  */ "<backslash>",    // "<reverse-solidus>",
+    /* 0x5d    ]  */ "<right-square-bracket>",
+    /* 0x5e    ^  */ "<circumflex>",   // "<circumflex-accent>",
+    /* 0x5f    _  */ "<underscore>",   // "<low-line>",
+    /* 0x60    `  */ "<grave-accent>",
+    /* 0x61    a  */ "<a>",
+    /* 0x62    b  */ "<b>",
+    /* 0x63    c  */ "<c>",
+    /* 0x64    d  */ "<d>",
+    /* 0x65    e  */ "<e>",
+    /* 0x66    f  */ "<f>",
+    /* 0x67    g  */ "<g>",
+    /* 0x68    h  */ "<h>",
+    /* 0x69    i  */ "<i>",
+    /* 0x6a    j  */ "<j>",
+    /* 0x6b    k  */ "<k>",
+    /* 0x6c    l  */ "<l>",
+    /* 0x6d    m  */ "<m>",
+    /* 0x6e    n  */ "<n>",
+    /* 0x6f    o  */ "<o>",
+    /* 0x70    p  */ "<p>",
+    /* 0x71    q  */ "<q>",
+    /* 0x72    r  */ "<r>",
+    /* 0x73    s  */ "<s>",
+    /* 0x74    t  */ "<t>",
+    /* 0x75    u  */ "<u>",
+    /* 0x76    v  */ "<v>",
+    /* 0x77    w  */ "<w>",
+    /* 0x78    x  */ "<x>",
+    /* 0x79    y  */ "<y>",
+    /* 0x7a    z  */ "<z>",
+    /* 0x7b    {  */ "<left-brace>",    // "<left-curly-bracket>",
+    /* 0x7c    |  */ "<vertical-line>", 
+    /* 0x7d    }  */ "<right-brace>",   // "<right-curly-bracket>",
+    /* 0x7e    ~  */ "<tilde>",
+    /* 0x7f       */ 0
+};
 
 
 // convert a string of narrow character into a wchar_t
 bool Charmap::convert_to_wc (const std::string& sym_name,
                              const std::string& ext_enc, wchar_t& wc)
 {
-
 #ifndef _RWSTD_NO_ISO_10646_WCHAR_T
 
     // the internal wchar_t representation for all characters
@@ -302,17 +400,14 @@
 
 #else   // if defined _RWSTD_NO_ISO_10646_WCHAR_T
 
-    if (UCS4_internal_) {
+    if (UCS4_internal_ || Clocale_.empty ()) {
 
-        // translate the character to ISO-10646 (UCS)
+        // when using UCS as the internal encoding or for a locale
+        // that has no corresponding C library locale convert the
+        // character to ISO-10646 (UCS)
         return convert_to_ucs (sym_name, ext_enc, wc);
     }
 
-    // for a locale that has no corresponding C library locale
-    // convert the character to ISO-10646 (UCS)
-    if (Clocale_.empty ()) 
-        return convert_to_ucs (sym_name, ext_enc, wc);
-
     // otherwise use libc to convert the multi-byte character
     // to its wchar_t value
     if (-1 == std::mbtowc (&wc, ext_enc.c_str (), ext_enc.size ())) {
@@ -332,6 +427,7 @@
     return true;
 
 #endif   // _RWSTD_NO_ISO_10646_WCHAR_T
+
 }
 
 
@@ -377,153 +473,198 @@
 }
 
 
-wchar_t Charmap::increment_val (const wchar_t val) const
+wchar_t Charmap::increment_wchar (wchar_t val) const
 {
 #ifndef _RWSTD_NO_ISO_10646_WCHAR_T
+
     // to increment a wchar_t value and keep the encoding all we have
-    // to do is increment the val because the internal encoding is utf8
+    // to do is increment the val because the internal encoding is UCS
     return val + 1;
 
 #else
     // to increment a wchar_t value and keep the encoding we have to
     // convert the wchar_t to the external encoding, increment that
     // string value, and convert back to the internal representation
-    rn_cmap2_iter it = rn_cmap2_.find (val);
+    const rmb_cmap_iter it = rmb_cmap_.find (val);
+
+    if (it != rmb_cmap_.end ()) {
 
-    if (it != rn_cmap2_.end ()) {
+        mb_cmap_iter ret;
 
-        n_cmap2_iter ret;
+        // multibyte character corresponding to the wchar_t value
+        std::string encoding = it->second;
 
-        std::string mb_str = it->second;
         // continue incrementing the multi-byte value until we get a valid 
         // character.  NOTE: this must be done for encodings such as SJIS where
         // \x7f in the last byte of a multibyte string is not a valid character
         // NOTE: this will not detect errors in the sequence, since the program
         // will continue until it finds a valid character
         do {
-            int last_elm = mb_str.size() - 1;
+            int last_elm = encoding.size () - 1;
 
             while (last_elm >= 0) {
 
                 typedef unsigned char UChar;
 
-                const unsigned ic = UChar (mb_str [last_elm]) + 1;
+                const unsigned ic = UChar (encoding [last_elm]) + 1;
 
                 // if incrementing the last element caused it to exceed
-                // 0xff increment the next higher byte if there is one
-                if (ic > 0xff)
-                    mb_str [last_elm--] = 0;
+                // UCHAR_MAX increment the next higher byte if there is
+                // one
+                if (UCHAR_MAX < ic)
+                    encoding [last_elm--] = '\0';
                 else {
-                    mb_str [last_elm] = char (ic);
+                    encoding [last_elm] = char (ic);
                     break;
                 }
             }
 
             if (last_elm < 0)
-                std::cerr << "cannot convert character\n";
-        } while ((ret = n_cmap2_.find (mb_str)) == n_cmap2_.end ());
+                return -1;   // error
+
+        } while ((ret = mb_cmap_.find (encoding)) == mb_cmap_.end ());
 
         return ret->second;
     }
 
-    return -1;
+    return -1;   // error
 
 #endif   // _RWSTD_NO_ISO_10646_WCHAR_T    
+
 }
 
-const char* Charmap::increment_strval (const char* str)
+
+bool Charmap::
+increment_encoding (std::string &encoding)
 {
-    static char s  [64];
-    static char sd [64];
-    int         i = 0;
+    // find the last escape character in the human readable representation
+    // of the encoding (i.e., in the multibyte character such as "/xf0/x80")
+    const std::string::size_type pos =
+        encoding.rfind (scanner_.escape_char ());
+
+    // the escape character must be there (guaranteed by the scanner)
+    assert (pos < encoding.size ());
+
+    const char* end = 0;
+
+    // convert the last character in the multibyte character to a numeric
+    // value representing the last byte of the sequence
+    unsigned long last_byte =
+        scanner_.convert_escape (encoding.c_str () + pos, &end);
+
+    // POSIX requires that the incremented value be non-NUL
+    if (UCHAR_MAX <= last_byte || *end)
+        return false;
 
-    char* ps = s;
+    // increment the last byte
+    ++last_byte;
 
-    // zero the first element of string s
-    *ps = 0;
+    // format the last byte in the same notation (octal, decimal,
+    // or hexadecimal escape sequence)
+    static const char xdigits[] = "0123456789ABCDEF";
 
-    if (str == 0 || *str == 0)
-        return s;
+    char byte_str [5];
+    char *pdig = byte_str;
 
-    for (i = 0; *str && i < 64; i++) {
+    switch (encoding [pos + 1]) {
+    case 'd': {   // decimal escape
+        const unsigned hundreds = last_byte / 100;
+        const unsigned tens     = (last_byte - hundreds) / 10;
+        const unsigned units    = last_byte % 10;
 
-        *ps++ = scanner_.convert_escape (str, (const char**)&str);
+        *pdig++ = 'd';
+
+        if (hundreds)
+            *pdig++ = xdigits [hundreds];
+
+        *pdig++ = xdigits [tens];
+        *pdig++ = xdigits [units];
+        *pdig   = '\0';
+        break;
     }
 
-    *ps = 0;
+    case 'x': {   // hex escape
+        const unsigned hi = last_byte >> 4;
+        const unsigned lo = last_byte & 0xfU;
 
-    //  now  attempt to  increment the  last character  in the  string  if the
-    //  character gets incremented above  /xff then we increment the next char
-    if (ps == s) {
-        return s;
+        *pdig++ = 'x';
+        *pdig++ = xdigits [hi];
+        *pdig++ = xdigits [lo];
+        *pdig   = '\0';
+        break;
     }
+    default: {   // octal escape
+        const unsigned hi  = last_byte >> 6;
+        const unsigned mid = (last_byte >> 3) & 07U;
+        const unsigned lo  = last_byte & 07U;
 
-    for (ps--; ps >= s; ps--) 
-        if ((unsigned char)(*ps) == 0xff) {
-            *ps = 0;
-            continue;
-        } else {
-            (*ps)++;
-            break;
-        }
+        if (hi)
+            *pdig++ = xdigits [hi];
 
-    const char fmt [] = {scanner_.escape_char (), 
-                         'x', '%', '0', '2', 'x', '\0'};
-    ps = s;
-    for (i = 0; *ps; i += 4, ps++)
-        std::sprintf (&sd [i], fmt, *(unsigned char*)ps);
+        *pdig++ = xdigits [mid];
+        *pdig++ = xdigits [lo];
+        *pdig   = '\0';
+    }
+    }   // switch
+
+    // replace the last escape sequence with the new one
+    encoding.replace (pos + 1, std::string::npos, byte_str);
 
-    return sd;
+    return true;
 }
 
 
-std::string Charmap::parse_ext_strval (const std::string &strval) const
+std::string Charmap::
+encoding_to_mbchar (const std::string &encoding) const
 {
-    std::string ext_enc;
+    std::string mbchar;
 
-    for (const char *str = strval.c_str (); str && *str; )
-        ext_enc += char (scanner_.convert_escape (str, &str));
+    for (const char *pbyte = encoding.c_str (); pbyte && *pbyte; )
+        mbchar += char (scanner_.convert_escape (pbyte, &pbyte));
 
-    return ext_enc;
+    return mbchar;
 }
 
 
 // convert the locale's encoded character to UCS4 wchar_t
-wchar_t Charmap::convert_sym_to_ucs (const std::string& s) const
+wchar_t Charmap::
+convert_sym_to_ucs (const std::string &sym) const
 {
-    std::string::const_iterator it (s.begin ());
+    std::string::const_iterator it (sym.begin ());
 
-    if (s.size () < 4 || *it != '<' || *++it != 'U') {
-        issue_diag (E_UCS, true, 0, 
-                    "Attempt to convert symbolic name to UCS value failed. "
-                    "Name %s not in <Uxxxxxxxx> form.\n", s.c_str ());
+    if (   sym.size () < 4 || *it != '<' || *++it != 'U'
+        || !isxdigit (*++it)) {
+        issue_diag (E_UCS, true, 0,
+                    "Unable to convert symbolic name %s to UCS.\n",
+                    sym.c_str ());
     }
 
-    long w = std::strtol (&*++it, (char**)0, 16);
-    if (w == _RWSTD_LONG_MIN || w == _RWSTD_LONG_MAX ||
-        w >  _RWSTD_WCHAR_T_MAX )
+    const unsigned long val = std::strtoul (&*++it, (char**)0, 16);
+
+    if (_RWSTD_WCHAR_T_MAX <= val)
         issue_diag (E_UCS, true, 0, 
-                    "Attempt to convert symbolic name to UCS value failed. "
-                    "Value of %s out of range.\n", s.c_str ());
+                    "UCS value %lu of symbolic character %s out of range.\n",
+                    val, sym.c_str ());
 
-    return wchar_t (w);
+    return wchar_t (val);
 }
 
+
 // convert the locale's encoded character to UCS4/UCS2 wchar_t
 bool Charmap::convert_to_ucs (const std::string &sym_name, 
-                              const std::string &ext_enc, wchar_t& wc)
+                              const std::string &encoding, wchar_t& wc)
 {
 #ifndef _MSC_VER
 
     if (in_utf8_) {
-        wc = utf8_decode (ext_enc.c_str (), &*(ext_enc.end () - 1));
+        wc = utf8_decode (encoding.c_str (), &*(encoding.end () - 1));
         return true;
     }
 
     char utf8_enc [_RWSTD_MB_LEN_MAX + 1];
 
     const char* const ch_end =
-        convert_to_utf8  (ext_enc.c_str (), ext_enc.size (),
+        convert_to_utf8  (encoding.c_str (), encoding.size (),
                           utf8_enc, sizeof utf8_enc);
     if (ch_end)
         // only if conversion to utf8 succeeded
@@ -538,7 +679,7 @@
 
     if (0 != codepage_) {
         wchar_t ret[2] = {0};
-        MultiByteToWideChar (codepage_, 0, ext_enc.c_str(), -1, ret, 2);
+        MultiByteToWideChar (codepage_, 0, encoding.c_str(), -1, ret, 2);
         if (ret[1] != 0)
             return false;
 
@@ -556,16 +697,27 @@
 
 
 void Charmap::add_to_cmaps (const std::string &sym_name, 
-                            const std::string &strval)
+                            const std::string &encoding,
+                            bool               is_mbchar /* = false */)
 {
-    strval_map_.insert (std::make_pair (strval, sym_name));
+    // compute the external (multibyte) encoding of the character
+    // if necessary (i.e., unless already done by the caller)
+    const std::string mbchar =
+        is_mbchar ? encoding : encoding_to_mbchar (encoding);
+
+    symnames_list_.push_back (sym_name);
 
-    if (mbcharlen (strval) == 1) {
+    if (1 == mbchar.size ()) {
+        // strval is a single-byte character
 
-        const unsigned char ch = scanner_.convert_escape (strval.c_str ());
+        const unsigned char ch = mbchar [0];
 
-        if (forward_maps)
+        // add the wide character and its symbolic name to the narrow
+        // character maps
+        if (forward_maps) {
+            // the locale utility doesn't need reverse maps
             n_cmap_.insert (std::make_pair (sym_name, ch));
+        }
 
         if (reverse_maps)
             rn_cmap_.insert (std::make_pair (ch, sym_name));
@@ -574,176 +726,204 @@
             largest_nchar_ = ch;
     }
 
-    const std::string ext_enc = parse_ext_strval (strval);
-
-    wchar_t ch;
+    // (try to) compute the wide character value of the character
+    wchar_t wch;
 
-    if (convert_to_wc (sym_name, ext_enc, ch)) {
+    if (convert_to_wc (sym_name, mbchar, wch)) {
 
-        if (forward_maps)
-            w_cmap_.insert (std::make_pair (sym_name, ch));
+        // add the wide character and its symbolic name to the wide
+        // character maps
+        if (forward_maps) {
+            // the locale utility doesn't need forward maps
+            w_cmap_.insert (std::make_pair (sym_name, wch));
+        }
 
         if (reverse_maps)
-            rw_cmap_.insert (std::make_pair (ch, sym_name));
-
-        std::string n_strval = parse_ext_strval (strval);
-
-        n_cmap2_.insert (std::make_pair (n_strval, ch));
-        rn_cmap2_.insert (std::make_pair (ch, n_strval));
-
-        assert (n_strval.size () != 0);
+            rw_cmap_.insert (std::make_pair (wch, sym_name));
 
-        for (std::string::size_type i = n_strval.size (); --i; )
-            valid_mb_set_.insert (n_strval.substr (0, i));
+        // add the corresponding multibyte character to the multibyte
+        // character maps
+        mb_cmap_.insert (std::make_pair (mbchar, wch));
+        rmb_cmap_.insert (std::make_pair (wch, mbchar));
     }
 
-    wchar_t ucs_val;
+    // compute the UCS value of the character
+    wchar_t uch;
 
-    if (convert_to_ucs (sym_name, ext_enc, ucs_val)) {
-        ucs4_cmap_.insert (std::make_pair (sym_name, ucs_val));
-        rucs4_cmap_.insert (std::make_pair (ucs_val, sym_name));
+    if (convert_to_ucs (sym_name, mbchar, uch)) {
+
+        // add UCS character and its symbolic name to the UCS
+        // character maps
+        ucs4_cmap_.insert (std::make_pair (sym_name, uch));
+        rucs4_cmap_.insert (std::make_pair (uch, sym_name));
     }
 }
 
 
 // process the characters implicitly defined by using ellipsis between
 // two explicitly defined characters
-void Charmap::
+std::size_t Charmap::
 process_ellipsis (const Scanner::token_t &beg_tok, int num_ellipsis)
 {
+    // get the upper end of the range denoted by the ellipsis
     const Scanner::token_t end_tok = scanner_.next_token ();
-    std::string strval = scanner_.next_token ().name;
 
-    // first add the beg_tok symbol name to the maps
-    add_to_cmaps (beg_tok.name, strval);
+    // get the human readabale encoding of the character
+    // denoted by the lower end of the ellipsis
+    const std::string encoding = scanner_.next_token ().name;
+
+    // convert the encoding to a multibyte character
+    std::string mbchar = encoding_to_mbchar (encoding);
+
+    // add the beg_tok symbol name to the maps
+    add_to_cmaps (beg_tok.name, mbchar, true);
     
-    // seperate the numeric portion of the symbolic name from the
-    // character portion in order to dynamically create symbolic
-    // names with increasing numeric values
-    std::string begin;
-    size_t idx = 0;
+    // extract the numeric portion of the symbolic character name
+    // denoted by the lower end of the ellipsis
+    std::size_t idx = 0;
 
     int base;           // numeric base
     const char *fmat;   // sprintf() format specifier
 
+    const std::size_t beg_len = beg_tok.name.size ();
+
     // determine the value of the beginning of the range
     // denoted by the ellipsis
-    if (num_ellipsis == 2) {
+    if (2 == num_ellipsis) {
         base = 16;
-        fmat = "%s%0*lX>";
+        fmat = "%.*s%0*lX>";
 
         // advance to the first hex digit
-        while (   idx < beg_tok.name.size ()
-               && !(std::isxdigit)(beg_tok.name [idx]))
-            begin += beg_tok.name [idx++];
+        while (idx < beg_len && !(std::isxdigit)(beg_tok.name [idx]))
+            ++idx;
     }
     else {
         base = 10;
-        fmat = "%s%0*ld>";
+        fmat = "%.*s%0*ld>";
 
         // advance to the first decimal digit
-        while (   idx < beg_tok.name.size ()
-               && !(std::isdigit)(beg_tok.name [idx]))
-            begin += beg_tok.name [idx++];
+        while (idx < beg_len && !(std::isdigit)(beg_tok.name [idx]))
+            ++idx;
     }
 
-    std::string num_str;  // the numeric portion of the sym name
-
-    // get the numeric portion of the sym_name, this is the portion
-    // that will be different for each sym_name within the ellipsis
-    while (idx < beg_tok.name.size () && beg_tok.name [idx] != '>')
-        num_str += beg_tok.name [idx++];
-
-    const int num_size = int (num_str.size ());
-    
-    // convert the num_str to a long
-    unsigned long start_num = std::strtoul (num_str.c_str(), (char**)0, base);
+    // length of non-numeric prefix of the symbolic character name
+    const std::size_t pfx_len = idx;
 
-    // increment the start_num once because we already added the 
-    // beg_tok symbol name to the cmaps
-    ++start_num;
+    // get the character value plus one (since the first value
+    // has already been added to the map earlier)
+    char *num_end;
+    const unsigned long beg_val =
+        1 + std::strtoul (beg_tok.name.c_str () + pfx_len, &num_end, base);
+
+    // the length of the numeric portion
+    const std::size_t num_size =
+        num_end - (beg_tok.name.c_str () + pfx_len);
 
     // find the end of the range denoted by the ellipsis
     idx = 0;
 
-    if (num_ellipsis == 2) {
+    const std::size_t end_len = end_tok.name.size ();
+
+    if (2 == num_ellipsis) {
         // advance to the next hex digit
-        while (   idx < end_tok.name.size ()
-               && !(std::isxdigit)(end_tok.name [idx]))
+        while (idx < end_len && !(std::isxdigit)(end_tok.name [idx]))
             ++idx;
     }
     else {
         // advance to the next dec digit
-        while (   idx < end_tok.name.size ()
-               && (std::isdigit)(end_tok.name [idx]))
+        while (idx < end_len && (std::isdigit)(end_tok.name [idx]))
             ++idx;
     }
 
-    num_str.clear ();
-
-    // advance to the closing bracket ('>')
-    while (idx < end_tok.name.size() && end_tok.name[idx] != '>')
-        num_str += end_tok.name [idx++];
-    
-    unsigned long end_num = std::strtoul (num_str.c_str(), (char**)0, base);
+    const unsigned long end_val =
+        std::strtoul (end_tok.name.c_str () + idx, (char**)0, base);
 
-    // the ending numeric value should be greater then the start numeric value
-    if (end_num < start_num)
+    // the ending numeric value must be greater than or equal
+    // to the beginning numeric value
+    if (end_val < beg_val)
         issue_diag (E_RANGE, true, &end_tok, 
                     "invalid range found in character map file\n");
     
     char next_name [MAX_SYM_NAME_LEN];
 
-    for (; start_num <= end_num; ++start_num) {
+    std::size_t nchars = 0;
 
-        std::sprintf (next_name, fmat, begin.c_str (), num_size, start_num);
-        
-        // increment the string value to the next encoded character value
-        strval = increment_strval (strval.c_str ());
+    const char* const pfx = beg_tok.name.c_str ();
+
+    for (unsigned long val = beg_val; val <= end_val; ++val, ++nchars) {
 
-        // and finally add the generated name and string value to the maps
-        add_to_cmaps (next_name, strval);
+        std::sprintf (next_name, fmat, pfx_len, pfx, num_size, val);
+        
+        // increment the last byte of the multibyte character
+        // and if the result is valid (i.e., doesn't contain
+        // an embedded NUL) add the generated name and the
+        // multibyte character to the maps
+        const unsigned char last_byte = mbchar [mbchar.size () - 1];
+        if (last_byte < UCHAR_MAX) {
+            mbchar [mbchar.size () - 1] = last_byte + 1;
+            add_to_cmaps (next_name, mbchar, true);
+        }
+        else {
+            // an ellipsis must not specify a range that includes
+            // an encoding with an embedded NUL
+            issue_diag (E_RANGE, true, &beg_tok, 
+                        "encoding of an element in range contains NUL\n");
+        }
     }
+
+    // return the number of characters denoted by the ellipsis
+    return nchars;
 }
 
 
 // process all the characters in the character map file.
 void Charmap::process_chars()
 {
+    issue_diag (I_STAGE, false, 0, "processing CHARMAP section\n");
+
+    std::size_t ntokens = 0;
+    std::size_t nellips = 0;
+    std::size_t nchars  = 0;
+
     next = scanner_.next_token();
     Scanner::token_t nextnext;
 
     // loop until we find the closing charmap token
-    while (next.token != Scanner::tok_charmap) {
+    for ( ; next.token != Scanner::tok_charmap; ++ntokens) {
 
-        switch (next.token){
+        switch (next.token) {
 
         case Scanner::tok_nl:
         case Scanner::tok_end:
             break;
 
         case Scanner::tok_sym_name:
-            // the next token may be either ellipsis if this line of the
-            // charmap is in the form: 
+            // the next token may be either ellipsis if this line
+            // of the charmap is in the form:
             // "%s...%s %s\n", <sym_name>, <sym_name>, <encoding>
             // or an encoding if this line is in the format:
             // "%s %s\n", <sym_name>, <encoding>
-            nextnext = scanner_.next_token();
+            nextnext = scanner_.next_token ();
+            ntokens += 3;
 
             switch (nextnext.token) {
 
-            case Scanner::tok_ellipsis:
-                process_ellipsis (next, 3);
+            case Scanner::tok_abs_ellipsis:
+                // absolute ellipsis (see ISO/IEC TR 14652)
+                nchars += process_ellipsis (next, 3);
+                ++nellips;
                 break;
 
-            case Scanner::tok_dellipsis:
-                process_ellipsis (next, 2);
+            case Scanner::tok_hex_ellipsis:
+                // hexadecimal symbolic ellipsis (see ISO/IEC TR 14652)
+                nchars += process_ellipsis (next, 2);
+                ++nellips;
                 break;
 
-            case Scanner::tok_decimal_value:
-            case Scanner::tok_hex_value:
-            case Scanner::tok_octal_value:
+            case Scanner::tok_char_value:
+                // character represented as a numeric constant
                 add_to_cmaps (next.name, nextnext.name);
+                ++nchars;
                 break;
 
             default:
@@ -752,7 +932,7 @@
                             "name in character map file\n");
             }
 
-            scanner_.ignore_line();
+            scanner_.ignore_line ();
             break;
 
         default:
@@ -762,9 +942,13 @@
         }
 
         next = scanner_.next_token();
-
     }
-    
+
+    issue_diag (I_STAGE, false, 0,
+                "done processing CHARMAP section (%lu tokens, "
+                "%lu ellipses, %lu characters)\n",
+                ntokens, nellips, nchars);
+
     // make sure that all characters in the portable character set
     // are in the charmap
     if (forward_maps)
@@ -772,37 +956,20 @@
 }
 
 
-void Charmap::verify_portable_charset()
+void Charmap::verify_portable_charset () const
 {
-    static const char* const charset[] = {
-        "<NUL>","<alert>", "<backspace>", "<tab>", "<newline>", 
-        "<vertical-tab>", "<form-feed>", "<carriage-return>", "<space>",
-        "<exclamation-mark>", "<quotation-mark>", "<number-sign>", 
-        "<dollar-sign>", "<percent-sign>", "<ampersand>", "<apostrophe>",
-        "<left-parenthesis>", "<right-parenthesis>", "<asterisk>", 
-        "<plus-sign>", "<comma>", "<hyphen>", "<hyphen-minus>", "<period>",
-        "<full-stop>", "<slash>", "<solidus>", "<zero>", "<one>", "<two>",
-        "<three>", "<four>", "<five>", "<six>", "<seven>", "<eight>",
-        "<nine>", "<colon>", "<semicolon>", "<less-than-sign>", 
-        "<equals-sign>", "<greater-than-sign>", "<question-mark>",
-        "<commercial-at>", "<A>", "<B>", "<C>", "<D>", "<E>", "<F>", "<G>",
-        "<H>", "<I>", "<J>", "<K>", "<L>","<M>", "<N>", "<O>", "<P>","<Q>",
-        "<R>", "<S>", "<T>", "<U>", "<V>", "<W>", "<X>", "<Y>", "<Z>",
-        "<left-square-bracket>", "<backslash>", "<reverse-solidus>",
-        "<right-square-bracket>", "<circumflex>", "<circumflex-accent>",
-        "<underscore>", "<low-line>", "<grave-accent>",
-        "<a>", "<b>", "<c>", "<d>", "<e>", "<f>", "<g>",
-        "<h>", "<i>", "<j>", "<k>", "<l>","<m>", "<n>", "<o>", "<p>","<q>",
-        "<r>", "<s>", "<t>", "<u>", "<v>", "<w>", "<x>", "<y>", "<z>",
-        "<left-brace>", "<left-curly-bracket>", "<vertical-line>", 
-        "<right-brace>", "<right-curly-bracket>", "<tilde>"
-    };
+    const std::size_t nchars =
+        sizeof portable_charset / sizeof *portable_charset;
+
+    for (std::size_t i = 0; i < nchars; ++i) {
+        if (0 == portable_charset [i])
+            continue;
 
-    for (std::size_t i = 0; i < sizeof charset / sizeof (char*); ++i)
-        if (n_cmap_.find (charset [i]) == n_cmap_.end ())
+        if (n_cmap_.find (portable_charset [i]) == n_cmap_.end ())
             issue_diag (W_NOPCS, false, 0, 
                         "member of portable character set %s not found "
-                        "in the character map\n", charset[i]);
+                        "in the character map\n", portable_charset [i]);
+    }
 }
 
 

Modified: incubator/stdcxx/trunk/util/charmap.h
URL: http://svn.apache.org/viewvc/incubator/stdcxx/trunk/util/charmap.h?view=diff&rev=448754&r1=448753&r2=448754
==============================================================================
--- incubator/stdcxx/trunk/util/charmap.h (original)
+++ incubator/stdcxx/trunk/util/charmap.h Thu Sep 21 17:42:16 2006
@@ -2,20 +2,27 @@
  *
  * charmap.h
  *
- * $Id: //stdlib/dev/source/stdlib/util/charmap.h#41 $
+ * $Id$
  *
  ***************************************************************************
  *
- * Copyright (c) 1994-2005 Quovadx,  Inc., acting through its  Rogue Wave
- * Software division. Licensed under the Apache License, Version 2.0 (the
- * "License");  you may  not use this file except  in compliance with the
- * License.    You    may   obtain   a   copy   of    the   License    at
- * http://www.apache.org/licenses/LICENSE-2.0.    Unless   required    by
- * applicable law  or agreed to  in writing,  software  distributed under
- * the License is distributed on an "AS IS" BASIS,  WITHOUT WARRANTIES OR
- * CONDITIONS OF  ANY KIND, either  express or implied.  See  the License
- * for the specific language governing permissions  and limitations under
- * the License.
+ * Licensed to the Apache Software  Foundation (ASF) under one or more
+ * contributor  license agreements.  See  the NOTICE  file distributed
+ * with  this  work  for  additional information  regarding  copyright
+ * ownership.   The ASF  licenses this  file to  you under  the Apache
+ * License, Version  2.0 (the  "License"); you may  not use  this file
+ * except in  compliance with the License.   You may obtain  a copy of
+ * the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the  License is distributed on an  "AS IS" BASIS,
+ * WITHOUT  WARRANTIES OR CONDITIONS  OF ANY  KIND, either  express or
+ * implied.   See  the License  for  the  specific language  governing
+ * permissions and limitations under the License.
+ *
+ * Copyright 2001-2006 Rogue Wave Software.
  * 
  **************************************************************************/
 
@@ -24,6 +31,7 @@
 #define _RWSTD_LOC_CHARMAP_H_INCLUDED
 
 #include <string>
+#include <list>
 #include <map>
 #include <set>
 
@@ -36,6 +44,8 @@
 class Charmap
 {
 public:
+    static const char* const portable_charset [];
+
     Charmap(const char* /*corresponding C library locale*/,
             const char* /*filename*/, 
             bool /*is utf8 encoding?*/, 
@@ -43,37 +53,45 @@
             bool /*create_reverse_charmaps*/, 
             bool /*use UCS4 internally*/);    
   
-    // get the narrow character map
-    const std::map<std::string, unsigned char >& get_n_cmap() const {
+    // returns the narrow character map which maps a symbolic character
+    // name to its narrow character value
+    const std::map<std::string, unsigned char>& get_n_cmap() const {
         return n_cmap_;
     }
     
-    // get the reverse narrow character map
-    const std::map<unsigned char, std::string >& get_rn_cmap() const {
+    // returns the reverse narrow character map which maps a narrow
+    // character value to its symbolic name
+    const std::map<unsigned char, std::string>& get_rn_cmap() const {
         return rn_cmap_;
     }
 
-    // get the wide character map
-    const std::map<std::string, wchar_t >& get_w_cmap() const {
+    // returns the wide character map which maps a symbolic character
+    // name to its wide character value
+    const std::map<std::string, wchar_t>& get_w_cmap() const {
         return w_cmap_;
     }
 
-    // get the reverse wide character map
-    const std::map<wchar_t, std::string >& get_rw_cmap() const {
+    // returns the reverse wide character map which maps a wide
+    // character value to its symbolic name
+    const std::map<wchar_t, std::string>& get_rw_cmap() const {
         return rw_cmap_;
     }
 
-    const std::map<std::string, wchar_t>& get_n_cmap2() const {
-        return n_cmap2_;
+    // returns the multibyte character map which maps a multibyte
+    // character to its corresponding wide character value
+    const std::map<std::string, wchar_t>& get_mb_cmap() const {
+        return mb_cmap_;
     }
 
-    const std::map<wchar_t, std::string>& get_rn_cmap2() const {
-        return rn_cmap2_;
+    // returns the reverse multibyte character map which maps a wide
+    // character value to its corresponding multibyte character
+    const std::map<wchar_t, std::string>& get_rmb_cmap() const {
+        return rmb_cmap_;
     }
 
     // get the string value map
-    const std::map<std::string, std::string>& get_strval_map() const {
-        return strval_map_;
+    const std::list<std::string>& get_symnames_list() const {
+        return symnames_list_;
     }
 
     const std::map <std::string, wchar_t>& get_ucs4_cmap () const {
@@ -84,14 +102,6 @@
         return rucs4_cmap_;
     }
 
-    const std::set<std::string>& get_valid_mb_set() const {
-        return valid_mb_set_;
-    }
-
-    const std::set<std::string>& get_valid_wchar_set() const {
-        return valid_wchar_set_;
-    }
-
     // return the value of mb_cur_max
     int get_mb_cur_max() const {
         return mb_cur_max_;
@@ -128,32 +138,36 @@
 
     unsigned char get_largest_nchar () const;
     
-    // increment the wide character value to the next encoded character in
-    // this codeset
-    wchar_t increment_val (const wchar_t) const;
+    // increments the wide character value to the next encoded character
+    // in the current codeset; returns the incremented value or -1 on
+    // error
+    wchar_t increment_wchar (wchar_t) const;
 
 private:
     
-    // process the characters implicitly defined by using ellipsis between
-    // two explicitly defined characters
-    void process_ellipsis (const Scanner::token_t&, int);
+    // processes characters implicitly defined by an ellipsis denoted
+    // by two explicitly defined characters; returns the number of
+    // characters in the range, -1 on error
+    std::size_t process_ellipsis (const Scanner::token_t&, int);
 
     // process the charmap file making the necessary mappings in the cmaps
     void process_chars();
 
-    // increment the multi-byte string by 1.
-    const char* increment_strval (const char*);
+    // increment the encoded multi byte character argument
+    bool increment_encoding (std::string&);
 
-    // make sure that all the characters in the portable character set are
-    // defined in the character map
-    void verify_portable_charset ();
+    // verify that all the characters in the portable character set
+    // are defined in the character map
+    void verify_portable_charset () const;
 
 #ifndef _MSC_VER
     // open the iconv descriptor to convert to utf8
     iconv_t open_iconv_to_utf8 () const;
 #endif  // _MSC_VER
 
-    std::string parse_ext_strval (const std::string&) const;
+    // convert a human-readable encoding of a character
+    // to its raw multibyte character representation
+    std::string encoding_to_mbchar (const std::string&) const;
 
     // convert a multi-byte string to a utf8 multi-byte string
     char* convert_to_utf8 (const char *inbuf, std::size_t inbuf_s, 
@@ -167,8 +181,11 @@
 #  endif   // _RWSTD_NO_ISO_10646_WCHAR_T
 #endif  // _MSC_VER
     
-    // add the sym_name and multi-byte character to the character maps
-    void add_to_cmaps (const std::string&, const std::string&);
+    // add the symbolic name of a character and the raw multibyte
+    // character corresponding to it to the character maps
+    void add_to_cmaps (const std::string&,
+                       const std::string&,
+                       bool = false);
         
     // the scanner used to process the charmap file
     Scanner scanner_;
@@ -181,16 +198,18 @@
 #endif // _MSC_VER
 
     // n_cmap maps the symbolic name to a narrow character value
-    // rn_cmap does exactly the opposite
+    // rn_cmap does the opposite
     std::map <std::string, unsigned char> n_cmap_;
     std::map <unsigned char, std::string> rn_cmap_;
-    std::map <std::string, wchar_t> n_cmap2_;
-    std::map <wchar_t, std::string> rn_cmap2_;
-    typedef std::map <wchar_t, std::string>::const_iterator rn_cmap2_iter;
-    typedef std::map <std::string, wchar_t>::const_iterator n_cmap2_iter;
 
-    std::set<std::string> valid_mb_set_;
-    std::set<std::string> valid_wchar_set_;
+    // mb_cmap maps a multibyte character representation to its
+    // corresponding wide character value
+    // rmb_cmap does the opposite
+    std::map <std::string, wchar_t> mb_cmap_;
+    std::map <wchar_t, std::string> rmb_cmap_;
+
+    typedef std::map <wchar_t, std::string>::const_iterator rmb_cmap_iter;
+    typedef std::map <std::string, wchar_t>::const_iterator mb_cmap_iter;
 
     // w_cmap maps the symbolic name to a wide character value
     // rw_cmap does exactly the opposite 
@@ -232,11 +251,8 @@
     // should we use UCS4 as the internal representation
     bool UCS4_internal_;
 
-    // maps the string value to the symbolic name
-    // this map is required for the UNDEFINED keyword
-    // in localedef where the elements must be added in
-    // increasing encoded order.
-    std::map<std::string, std::string> strval_map_;
+    // list of all known symbolic character names
+    std::list<std::string> symnames_list_;
 
     Scanner::token_t next;
 };