locale - Add two new API functions
authorMatthew Dillon <dillon@apollo.backplane.com>
Sun, 16 Aug 2015 22:03:03 +0000 (15:03 -0700)
committerMatthew Dillon <dillon@apollo.backplane.com>
Sun, 16 Aug 2015 22:28:10 +0000 (15:28 -0700)
commit8a84c799639d2e842952ce56fad530a085b16712
treebdfaf8740e64a22ab4185b03d23d2880d94f1fe1
parente9e78086c16862f7a18f42143846d8d62986691f
locale - Add two new API functions

* Add wcrtombin() and mbintowcr(), powerful buffer-to-buffer functions
  to translate UTF8, which can also perform escaping and de-escaping,
  allow legacy parsing, and do checks with varying levels of strictness.

  NOTE: If no locale or "C" locale, these functions are 1:1 (do not do
UTF8 processing).  A UTF8 locale must be used to access these
functions.

* Add wcrtoutf8() and utf8towcr().  These work exactly the same as
  the above functions but are locale-agnostic.  They always operate in
  UTF8 and locale initialization is not necessary.

* Escaping and de-escaping for 8-bit-clean round-trips with two of the
  modes.

* Can construct UTF8B from unescaped 8-bit-clean data, can deconstruct
  UTF8B (restore the original clean data), and can also be flagged
  generically to take already-escaped UTF8B or wchars as input.

* Can be used to perform strict checks.

* Very powerful incremental buffer validation and processing capabilities.
  Far more visibility to the caller rather than hiding state in a tracking
  state structure.  No tracking state structure.

* Does not do idiotic nul buffer termination.  Caller passes explicitly
  sized data, can opt to include or not include a terminator (depending
  on what is desired), and the byte buffers can contain any 8-bit code
  including 0x00.
17 files changed:
include/locale.h
include/wchar.h
include/xlocale/_wchar.h
lib/libc/locale/Makefile.inc
lib/libc/locale/Symbol.map
lib/libc/locale/locale_isstrict.c [new file with mode: 0644]
lib/libc/locale/mbintowcr.3 [new file with mode: 0644]
lib/libc/locale/mbintowcr.c [new file with mode: 0644]
lib/libc/locale/mblocal.h
lib/libc/locale/mbtowc.3
lib/libc/locale/multibyte.3
lib/libc/locale/none.c
lib/libc/locale/utf8.c
lib/libc/locale/wcrtomb.3
lib/libc/locale/wcrtombin.c [new file with mode: 0644]
lib/libc/locale/xlocale_private.h
test/debug/utf8bin.c [new file with mode: 0644]