aboutsummaryrefslogtreecommitdiff
path: root/lib/libc/locale/collate.c
Commit message (Collapse)AuthorAgeFilesLines
* collate: Add support for equivalence classes and collating symbolsBojan Novković2025-04-101-0/+259
| | | | | | | | | | | This change adds support for matching single and multi-character equivalence classes and collating symbols, as specified by POSIX1, section 9.3.5. Sponsored by: Klara, Inc. Obtained from: https://github.com/apple-oss-distributions/Libc Differential Revision: https://reviews.freebsd.org/D49659 Reviewed by: markj
* libc: Purge unneeded cdefs.hWarner Losh2023-11-011-1/+0
| | | | | | | | | These sys/cdefs.h are not needed. Purge them. They are mostly left-over from the $FreeBSD$ removal. A few in libc are still required for macros that cdefs.h defines. Keep those. Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D42385
* Remove $FreeBSD$: one-line .c patternWarner Losh2023-08-161-2/+0
| | | | Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
* libc: locale: fix collation file size validationKyle Evans2023-05-281-1/+1
| | | | | | | | At a minimum, we need enough for the colllation format version string + locale definition version string and a full collate_info definition, rather than just the first two and a pointer. Sponsored by: Klara, Inc.
* spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSDWarner Losh2023-05-121-1/+1
| | | | | | | | | The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix
* libc: use separate collate objects for C, POSIX, and C.UTF-8Yuri Pankov2023-03-281-3/+14
| | | | | | | | | | Fix newlocale() overwriting the locale name in collate object when same instance was used for those locales, and querylocale() reporting unexpected value for LC_COLLATE_MASK. PR: 255646, 269375 Reviewed by: markj, bapt (previous version) Differential Revision: https://reviews.freebsd.org/D30146
* libc: drop "All rights reserved" from Foundation copyrightsEd Maste2022-08-041-1/+1
| | | | | | | | | This has already been done for most files that have the Foundation as the only listed copyright holder. Do it now for files that list multiple copyright holders, but have the Foundation copyright in its own section. Sponsored by: The FreeBSD Foundation
* __collate_load(): check for calloc failureKonstantin Belousov2022-02-031-0/+2
| | | | | | | Noted and reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D34140
* libc/locale/collate.c: minor styleKonstantin Belousov2022-02-031-6/+10
| | | | | | | Reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D34140
* libc/locale: Use O_CLOEXEC when opening locale tablesMark Johnston2021-07-291-1/+1
| | | | | | Reviewed by: kib MFC after: 1 week Sponsored by: The FreeBSD Foundation
* Add collation version support to querylocale(3).Thomas Munro2020-11-081-3/+7
| | | | | | | | | | | | | | | | | | | | | | | Provide a way to ask for an opaque version string for a locale_t, so that potential changes in sort order can be detected. Similar to ICU's ucol_getVersion() and Windows' GetNLSVersionEx(), this API is intended to allow databases to detect when text order-based indexes might need to be rebuilt. The CLDR version is extracted from CLDR source data by the Makefile under tools/tools/locale, written into the machine-generated Makefile under shared/colldef, passed to localedef -V, and then written into LC_COLLATE file headers. The initial version is 34.0. tools/tools/locale was recently updated to pull down 35.0, but the output hasn't been committed under share/colldef yet, so that will provide the first observable change when it happens. Other versioning schemes are possible in future, because the format is unspecified. Reviewed by: bapt, 0mp, kib, yuripv (albeit a long time ago) Differential Revision: https://reviews.freebsd.org/D17166 Notes: svn path=/head/; revision=367476
* libc: Fix a few bugs in the xlocale collation code.Mark Johnston2020-01-091-2/+7
| | | | | | | | | | | | | | | | | | | | - Fix checks for mmap() failures. [1] - Set the "map" and "maplen" fields of struct xlocale_collate so that the table destructor actually does something. - Free an already-mapped collation file before loading a new one into the global table. - Harmonize the prototype and definition of __collate_load_tables_l() by adding the "static" qualifier to the latter. PR: 243195 Reported by: cem [1] Reviewed by: cem, yuripv MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D23109 Notes: svn path=/head/; revision=356569
* Add hybrid C.UTF-8 locale being identical to default C locale exceptYuri Pankov2018-11-041-2/+4
| | | | | | | | | | | that it uses the same ctype maps and functions as other UTF-8 locales. Reviewed by: bapt, cem, eadler Approved by: kib (mentor, implicit) Differential Revision: https://reviews.freebsd.org/D17833 Notes: svn path=/head/; revision=340144
* Add -b/-l options to localedef(1) to specify output endianness and useYuri Pankov2018-10-201-22/+21
| | | | | | | | | | | | | | | | it appropriately when building share/ctypedef and share/colldef. This makes the resulting locale data in EL->EB (amd64->powerpc64) cross build and in the native EB build match. Revert the changes done to libc in r308170 as they are no longer needed. PR: 231965 Reviewed by: bapt, emaste, sbruno, 0mp Approved by: kib (mentor) Differential Revision: https://reviews.freebsd.org/D17603 Notes: svn path=/head/; revision=339489
* libc: further adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-251-0/+2
| | | | | | | | | | | | | | | Mainly focus on files that use BSD 2-Clause license, however the tool I was using mis-identified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Notes: svn path=/head/; revision=326193
* __setrunelocale: Fix asprintf(3) failure not returning an error.Bryan Drewery2017-09-291-2/+1
| | | | | | | | | | | | | Also fix the style of the asprintf(3) call in __collate_load_tables_l(). Both of these lines were modified away from snprintf(3) during the import from DragonFly/Illumos. Reviewed by: jilles (briefly over shoulder) MFC after: 2 weeks Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=324103
* Locale fix for endian big (EB) machines.Ruslan Bukin2016-11-011-21/+23
| | | | | | | | | | | | | | | | We have locale files generated on EL machines (e.g. during cross-build on amd64 host), but then we are using them on EB machines (e.g. MIPS64EB), so proceed byte-swap if necessary. All the libc tests passed successfully, including Russian collation. Tested by: br@, Hongyan Xia <hx242@cam.ac.uk> Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D8281 Notes: svn path=/head/; revision=308170
* Revert 302324 and properly fix the crash with ISO-8859-5 localesBaptiste Daroussin2016-07-151-1/+5
| | | | | | | | | | PR: 211135 Reported by: jkim Tested by: jkim MFC after: 2 days Notes: svn path=/head/; revision=302916
* Fix a bad test resulting in a segfault with ISO-8859-5 localesBaptiste Daroussin2016-07-031-1/+1
| | | | | | | | Reported by: Lauri Tirkkonen from Illumos Approved by: re@ (gjb) Notes: svn path=/head/; revision=302324
* libc: spelling fixes.Pedro F. Giffuni2016-04-301-1/+1
| | | | | | | Mostly on comments. Notes: svn path=/head/; revision=298830
* collate: Fix expansion substitions (broken upstream too)Baptiste Daroussin2015-10-231-17/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Through testing, the user noted that some Cyrillic characters were not sorting correctly, and this was confirmed. After extensive testing and review, the localedef tool was eliminated as the culprit. The sustitutions were encoded correctly in LC_COLLATE. The error was mainly in wcscoll where character expansions were mishandled. The main directive pass routines had to be written to go back for a new collation value when the "state" variable was set. Before pointers were being advanced, the second lookup was gettting applied to the wrong character, etc. The "eat expansion codes" section on collate.c also had a bug. Later own, the "state" variable logic was changed to only set if next code was greater than zero (rather than >= 0). Some additional cleanups got captured from previous work: 1) The previous commit moved the binary search comment from the correct location to a wrong location because it's wrong upstream in Illumos. The comment has little value so I just removed it. 2) Don't check if pointers are null before freeing, this is redundant as free() handles null pointers. 3) The two binary search trees were standardized wrt initialization 4) On the binary search trees, a negative "high" exits rather than checking the table count again. Submitted by: marino Obtained from: DragonflyBSD Notes: svn path=/projects/collation/; revision=289860
* libc/collate: minor tweaks / fixBaptiste Daroussin2015-10-221-27/+30
| | | | | | | | | | | | | The main "fix" here is properly setting a collate loading error for each early return. Tweaks include removing unnecessary null checks, adding assertions (from Illumos) and a couple of variables to reduces code differences and improve readability. For normal use, there are no functional changes here. Obtained from: DragonflyBSD, Illumos Notes: svn path=/projects/collation/; revision=289754
* Include sys/*.h earlierBaptiste Daroussin2015-10-141-3/+5
| | | | | | | Reported by: kib Notes: svn path=/projects/collation/; revision=289314
* Fix typoBaptiste Daroussin2015-08-091-1/+1
| | | | Notes: svn path=/projects/collation/; revision=286523
* Use more asprintfBaptiste Daroussin2015-08-091-1/+3
| | | | | | | Plug memory leak introduced in previous asprintf addition Notes: svn path=/projects/collation/; revision=286522
* Use asprintf/free instead of snprintfBaptiste Daroussin2015-08-091-3/+5
| | | | Notes: svn path=/projects/collation/; revision=286521
* Remove useless variableBaptiste Daroussin2015-08-091-2/+2
| | | | Notes: svn path=/projects/collation/; revision=286520
* Mark __collate_load_tables_l as staticBaptiste Daroussin2015-08-091-1/+1
| | | | | | | Remove useless addition to Symbols.map Notes: svn path=/projects/collation/; revision=286517
* Fix typoBaptiste Daroussin2015-08-091-2/+1
| | | | | | | | | Fix bad location for include Reported by: jilles Notes: svn path=/projects/collation/; revision=286494
* Revamp CTYPE support (from Illumos & Dragonfly)Baptiste Daroussin2015-08-081-0/+1
| | | | | | | Obtained from: Dragonfly Notes: svn path=/projects/collation/; revision=286459
* The collate functions within libc have been using version 1 and 1.2 of theBaptiste Daroussin2015-08-071-202/+558
| | | | | | | | | | | | | | | | | | | | | | | | | packed LC_COLLATE binary formats. These were generated with the colldef tool, but the new LC_COLLATE files are going to be generated by the new localedef tool using CLDR POSIX files as input. The BSD-flavored version of localedef identifies the format as "BSD 1.0". Any LC_COLLATE file with a different version will simply not be loaded, and all LC* categories will get set to "C" (aka "POSIX") locale. This work is based off of Nexenta's contribution to Illumos. The integration with xlocale is John Marino's work for Dragonfly. The following commits will enable localedef tool, disable the colldef tool, add generated colldef directory, and finally remove colldef from base. The only difference with Dragonfly are: - a few fixes to build with clang - And identification of the flavor as "BSD 1.0" instead of "Dragonfly 4.4" Obtained from: Dragonfly Notes: svn path=/projects/collation/; revision=286430
* libc: Make various internal file descriptors close-on-exec.Jilles Tjoelker2012-12-111-1/+1
| | | | | | | These are obtained via fopen(). Notes: svn path=/head/; revision=244126
* Restore the __collate_load_error global that was accidentally removed in theDavid Chisnall2012-07-061-4/+6
| | | | | | | | | xlocale refactoring. MFC after: 1 week Notes: svn path=/head/; revision=238182
* Remove some duplicated copyright notices.David Chisnall2012-03-061-5/+0
| | | | | | | Approved by: dim (mentor) Notes: svn path=/head/; revision=232601
* Implement xlocale APIs from Darwin, mainly for use by libc++. This adds aDavid Chisnall2011-11-201-21/+71
| | | | | | | | | | | | | | | load of _l suffixed versions of various standard library functions that use the global locale, making them take an explicit locale parameter. Also adds support for per-thread locales. This work was funded by the FreeBSD Foundation. Please test any code you have that uses the C standard locale functions! Reviewed by: das (gdtoa changes) Approved by: dim (mentor) Notes: svn path=/head/; revision=227753
* Make the format of LC_COLLATE files architecture independent.Ruslan Ermilov2005-02-271-2/+13
| | | | Notes: svn path=/head/; revision=142686
* ANSI'fy prototypesAlexey Zelkin2005-02-271-7/+3
| | | | Notes: svn path=/head/; revision=142653
* Prefer C99's __func__ over GCC's __FUNCTION__.Stefan Farfeleder2004-09-221-3/+3
| | | | Notes: svn path=/head/; revision=135576
* Remove unnecessary inclusion of <rune.h> to make it obvious that this fileTim J. Robbins2002-10-291-1/+0
| | | | | | | does not use the deprecated rune system. Notes: svn path=/head/; revision=106119
* Use ntohl() to read cnains number in new formatAndrey A. Chernov2002-08-311-2/+4
| | | | Notes: svn path=/head/; revision=102658
* Style fixAndrey A. Chernov2002-08-301-3/+3
| | | | Notes: svn path=/head/; revision=102641
* Prepare for switching to unlimited chains format.Andrey A. Chernov2002-08-301-20/+43
| | | | | | | Optimize chains lookup a bit. Notes: svn path=/head/; revision=102638
* Reduce BSS size for programs which not load collate by eliminatingAndrey A. Chernov2002-08-131-10/+10
| | | | | | | static buffer. Notes: svn path=/head/; revision=101820
* Now malloc() is fixed, remove errno hardcoding to ENOMEMAndrey A. Chernov2002-08-121-3/+6
| | | | Notes: svn path=/head/; revision=101747
* Rewrite locale loading procedures, so any load failure will not affectAndrey A. Chernov2002-08-081-39/+87
| | | | | | | | | | | currently cached data. It allows a number of nice things, like: removing fallback code from single locale loading, remove memory leak when LC_CTYPE data loaded again and again, efficient cache use, not only for setlocale(locale1); setlocale(locale1), but for setlocale(locale1); setlocale("C"); setlocale(locale1) too (i.e. data file loaded only once). Notes: svn path=/head/; revision=101498
* Style fixes in preparation for rewrittingAndrey A. Chernov2002-08-071-15/+15
| | | | Notes: svn path=/head/; revision=101478
* Do not use __progname directly (except in [gs]etprogname(3)).Mark Murray2002-03-291-2/+3
| | | | | | | | | Also, make an internal _getprogname() that is used only inside libc. For libc, getprogname(3) is a weak symbol in case a function of the same name is defined in userland. Notes: svn path=/head/; revision=93399
* Fix the style of the SCM ID's.David E. O'Brien2002-03-221-2/+3
| | | | | | | I believe have made all of libc .c's as consistent as possible. Notes: svn path=/head/; revision=92986
* Remove _THREAD_SAFE and make libc thread-safe by default byDaniel Eischen2001-01-241-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | adding (weak definitions to) stubs for some of the pthread functions. If the threads library is linked in, the real pthread functions will pulled in. Use the following convention for system calls wrapped by the threads library: __sys_foo - actual system call _foo - weak definition to __sys_foo foo - weak definition to __sys_foo Change all libc uses of system calls wrapped by the threads library from foo to _foo. In order to define the prototypes for _foo(), we introduce namespace.h and un-namespace.h (suggested by bde). All files that need to reference these system calls, should include namespace.h before any standard includes, then include un-namespace.h after the standard includes and before any local includes. <db.h> is an exception and shouldn't be included in between namespace.h and un-namespace.h namespace.h will define foo to _foo, and un-namespace.h will undefine foo. Try to eliminate some of the recursive calls to MT-safe functions in libc/stdio in preparation for adding a mutex to FILE. We have recursive mutexes, but would like to avoid using them if possible. Remove uneeded includes of <errno.h> from a few files. Add $FreeBSD$ to a few files in order to pass commitprep. Approved by: -arch Notes: svn path=/head/; revision=71579
* Simplify sytem call renaming. Instead of _foo() <-- _libc_foo <-- foo(),Jason Evans2000-01-271-6/+6
| | | | | | | | | | | | | | | | | | | | just use _foo() <-- foo(). In the case of a libpthread that doesn't do call conversion (such as linuxthreads and our upcoming libpthread), this is adequate. In the case of libc_r, we still need three names, which are now _thread_sys_foo() <-- _foo() <-- foo(). Convert all internal libc usage of: aio_suspend(), close(), fsync(), msync(), nanosleep(), open(), fcntl(), read(), and write() to _foo() instead of foo(). Remove all internal libc usage of: creat(), pause(), sleep(), system(), tcdrain(), wait(), and waitpid(). Make thread cancellation fully POSIX-compliant. Suggested by: deischen Notes: svn path=/head/; revision=56698
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-03 16:55:05 +0000