summaryrefslogtreecommitdiff
path: root/lib/libc/regex
Commit message (Collapse)AuthorAgeFilesLines
* libc: regex: retire internal EMPTBR ("Empty branch present")Kyle Evans2020-12-052-7/+0
| | | | | | | | | It was realized just a little too late that this was a hack that belonged in individual regex(3)-using applications. It was surrounded in NOTYET and not implemented in the engine, so remove it. Notes: svn path=/head/; revision=368359
* libregex: implement \b and \B (word boundary, not word boundary)Kyle Evans2020-12-053-0/+55
| | | | | | | | | | This is the last of the needed GNU expressions before we can unleash bsdgrep by default. \b is effectively an agnostic equivalent of \< and \>, while \B will match every space that isn't making a transition from nonchar -> char or char -> nonchar. Notes: svn path=/head/; revision=368358
* libregex: implement \` and \' (begin-of-subj, end-of-subj)Kyle Evans2020-12-053-8/+65
| | | | | | | | | | These are GNU extensions, generally equivalent to ^ and $ except that the new syntax will not match beginning of line after the first in a multi-line expression or the end of line before absolute last in a multi-line expression. Notes: svn path=/head/; revision=368357
* libc: regex: factor out ISBOW/ISEOW macrosKyle Evans2020-12-051-8/+13
| | | | | | | | | These will be reused for \b (word boundary, which matches both sides). No functional change. Notes: svn path=/head/; revision=368356
* libregex: Implement a subset of the GNU extensionsKyle Evans2020-08-042-84/+229
| | | | | | | | | | | | | | | | | | | | The entire patch-set is not yet mature enough for commit, but this usable subset is generally enough for googletest to be happy with and mostly map to some existing concepts, so they're not as invasive. The specific changes included here are: - Branching in BREs with \| - \w and \W for [[:alnum:]] and [^[:alnum:]] respectively - \s and \S for [[:space:]] and [^[:space:]] respectively - Additional quantifiers in BREs, \? and \+ (self-explanatory) There's some #ifdef'd out work for allowing empty branches as a match-all. This is a feature that's under assessment... future work will determine how standard this behavior is and act accordingly. Notes: svn path=/head/; revision=363818
* regex(3): belatedly document REG_POSIX from r363734Kyle Evans2020-08-041-1/+12
| | | | | | | | My original patch included this documented, but it appears that I failed to include the manpage update. Do so now. Notes: svn path=/head/; revision=363817
* regex(3): Interpret many escaped ordinary characters as EESCAPEKyle Evans2020-07-292-18/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In IEEE 1003.1-2008 [1] and earlier revisions, BRE/ERE grammar allows for any character to be escaped, but "ORD_CHAR preceded by an unescaped <backslash> character [gives undefined results]". Historically, we've interpreted an escaped ordinary character as the ordinary character itself. This becomes problematic when some extensions give special meanings to an otherwise ordinary character (e.g. GNU's \b, \s, \w), meaning we may have two different valid interpretations of the same sequence. To make this easier to deal with and given that the standard calls this undefined, we should throw an error (EESCAPE) if we run into this scenario to ease transition into a state where some escaped ordinaries are blessed with a special meaning -- it will either error out or have extended behavior, rather than have two entirely different versions of undefined behavior that leave the consumer of regex(3) guessing as to what behavior will be used or leaving them with false impressions. This change bumps the symbol version of regcomp to FBSD_1.6 and provides the old escape semantics for legacy applications, just in case one has an older application that would immediately turn into a pumpkin because of an extraneous escape that's embedded or otherwise critical to its operation. This is the final piece needed before enhancing libregex with GNU extensions and flipping the switch on bsdgrep. [1] http://pubs.opengroup.org/onlinepubs/9699919799.2016edition/ PR: 229925 (exp-run, courtesy of antoine) Differential Revision: https://reviews.freebsd.org/D10510 Notes: svn path=/head/; revision=363679
* lib/libc/regex: fix build with REDEBUG definedYuri Pankov2019-09-242-7/+1
| | | | | | | | Reviewed by: kevans Differential Revision: https://reviews.freebsd.org/D21760 Notes: svn path=/head/; revision=352651
* regcomp: revert part of r341838 which turned out to be unrelatedYuri Pankov2018-12-191-11/+3
| | | | | | | | | | | and caused issues with search in less. PR: 234066 Reviewed by: pfg Differential revision: https://reviews.freebsd.org/D18611 Notes: svn path=/head/; revision=342265
* regcomp: reduce size of bitmap for multibyte localesYuri Pankov2018-12-123-7/+22
| | | | | | | | | | | | | | | | | | | | | | | | | This fixes the obscure endless loop seen with case-insensitive patterns containing characters in 128-255 range; originally found running GNU grep test suite. Our regex implementation being kludgy translates the characters in case-insensitive pattern to bracket expression containing both cases for the character and doesn't correctly handle the case when original character is in bitmap and the other case is not, falling into the endless loop going through in p_bracket(), ordinary(), and bothcases(). Reducing the bitmap to 0-127 range for multibyte locales solves this as none of these characters have other case mapping outside of bitmap. We are also safe in the case when the original character outside of bitmap has other case mapping in the bitmap (there are several of those in our current ctype maps having unidirectional mapping into bitmap). Reviewed by: bapt, kevans, pfg Differential revision: https://reviews.freebsd.org/D18302 Notes: svn path=/head/; revision=341838
* regexec: fix processing multibyte strings.Yuri Pankov2018-11-231-2/+44
| | | | | | | | | | | | | | | Matcher function incorrectly assumed that moffset that we get from findmust is in bytes. Fix this by introducing a stepback function, taking short path if MB_CUR_MAX is 1, and going back byte-by-byte, checking if we have a legal character sequence otherwise. PR: 153502 Reviewed by: pfg, kevans Approved by: kib (mentor, implicit) Differential revision: https://reviews.freebsd.org/D18297 Notes: svn path=/head/; revision=340835
* regex/engine.c: error: variable 'dp' set but not usedToomas Soome2018-07-141-1/+1
| | | | | | | | | | The issue found with gcc6 build (originally on illumos, confirmed on FreeBSD). Mark it __unused. Differential Revision: https://reviews.freebsd.org/D13109 Notes: svn path=/head/; revision=336274
* Add missing patch from r328240Kyle Evans2018-01-221-5/+29
| | | | | | | | | | | regcomp uses some libc internal collation bits that are not available in the libregex context. It's easy enough to bring in the needed parts that can work in a libregex world, so do so. Pointy hat to: me Notes: svn path=/head/; revision=328241
* Add libregex, connect it to the buildKyle Evans2018-01-221-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | libregex is a regex(3) implementation intended to feature GNU extensions and any other non-POSIX compliant extensions that are deemed worthy. These extensions are separated out into a separate library for the sake of not cluttering up libc further with them as well as not deteriorating the speed (or lack thereof) of the libc implementation. libregex is implemented as a build of the libc implementation with LIBREGEX defined to distinguish this from a libc build. The reasons for implementation like this are two-fold: 1.) Maintenance- This reduces the overhead induced by adding yet another regex implementation to base. 2.) Ease of use- Flipping on GNU extensions will be as simple as linking against libregex, and POSIX-compliant compilations can be guaranteed with a REG_POSIX cflag that should be ignored by libc/regex and disables extensions in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when implemented in this fashion. Tests are added for future functionality, but left disconnected for the time being while other testing is done. Reviewed by: cem (previous version) Differential Revision: https://reviews.freebsd.org/D12934 Notes: svn path=/head/; revision=328240
* regex(3): Resolve issues with higher WARNS levelsKyle Evans2018-01-212-25/+28
| | | | | | | | | | | | | libc is set for WARNS=2, but the incoming libregex will use WARNS=6. Sprinkle some casts and (void)bc's to alleviate the warnings that come along with the higher WARNS level. These 'bc' parameters could be outright removed, but as of right now they will be used in some parts of libregex land. Silence the warnings instead rather than flip-flopping. Notes: svn path=/head/; revision=328211
* General further adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-208-0/+16
| | | | | | | | | | | | | | | | | Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point. Notes: svn path=/head/; revision=326025
* Fix out-of-bounds read in libc/regex.Pedro F. Giffuni2017-10-281-1/+1
| | | | | | | | | | | | | | | | | | | The bug is an out-of-bounds read detected with address sanitizer that happens when 'sp' in p_b_coll_elems() includes NUL byte[s], e.g. if it's equal to "GS\x00". In that case len will be equal to 4, and the strncmp(cp->name, sp, len) call will succeed when cp->name is "GS" but the cp->name[len] == '\0' comparison will cause the read to go out-of-bounds. Checking the length using strlen() instead eliminates the issue. The bug was found in LLVM with oss-fuzz: https://reviews.llvm.org/D39380 MFC after: 1 week Obtained from: Vlad Tsyrklevich through posting on openbsd-tech Notes: svn path=/head/; revision=325066
* regex(3): Refactor fast/slow stepping bits in the matching engineKyle Evans2017-08-091-142/+50
| | | | | | | | | | | | | | Adding features for matching is fairly straightforward, but this requires some duplication because of this fast/slow setup. They can be fairly trivially combined into a single walk(), so do it to make future additions less error prone. Reviewed by: cem (earlier version), emaste, pfg Approved by: emaste (mentor) Differential Revision: https://reviews.freebsd.org/D11233 Notes: svn path=/head/; revision=322288
* regex(3): Handle invalid {} constructs consistently and adjust testsKyle Evans2017-08-081-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | Currently, regex(3) exhibits the following wrong behavior as demonstrated with sed: - echo "a{1,2,3}b" | sed -r "s/{/_/" (1) - echo "a{1,2,3}b" | sed "s/\}/_/" (2) - echo "a{1,2,3}b" | sed -r "s/{}/_/" (3) Cases (1) and (3) should throw errors but they actually succeed, and (2) throws an error when it should match the literal '}'. The correct behavior was decided by comparing to the behavior with the equivalent BRE (1)(3) or ERE (2) and consulting POSIX, along with some reasonable evaluation. Tests were also adjusted/added accordingly. PR: 166861 Reviewed by: emaste, ngie, pfg Approved by: emaste (mentor) MFC after: never Differential Revision: https://reviews.freebsd.org/D10315 Notes: svn path=/head/; revision=322211
* Correctly ignore branch operators in the top-level parser when applicable.Kyle Evans2017-07-071-1/+1
| | | | | | | | | | | | | | | | An oversight in r320742 caused BREs to become sensitive to the branching operator prematurely, which caused breakage in some limited situations -- namely, those that tried to use branching in a BRE. Most of these scenarios had already been corrected beforehand to properly use gsed or grep for GNU extensions, so the damage is slightly mitigated. Reported by: antoine Reported by: antoine Approved by: emaste (mentor) Differential Revision: https://reviews.freebsd.org/D11522 Notes: svn path=/head/; revision=320796
* Fix sparc64 libc build after r320742.Kyle Evans2017-07-061-4/+12
| | | | | | | | | | | | | | p_branch_empty was declared but never used due to an oversight. Use it as designed, further comment on its return value. Reported by: Jenkins (head-sparc64) Reviewed by: emaste Approved by: emaste (mentor) MFC with: r320742 Differential Revision: https://reviews.freebsd.org/D11506 Notes: svn path=/head/; revision=320750
* The impending libregex will implement GNU extensions to bring BREs andKyle Evans2017-07-062-92/+224
| | | | | | | | | | | | | | | | | | | | | | | EREs closer together. Prepare for this and reduce the diff of libregex changes by refactoring and combining the top-level parsers for EREs/BREs ahead of time. Branching functionality has been split out to make it easier to follow the combined version of the top-level parser. It may also be enabled in the parsing context to make it easier when libregex enables branching for BREs. A branching context was also added for the various branching functions and so that BREs, for instance, can determine if they're the first expression in a chain of expressions within the current branch and treat '*' as ordinary if so. This should have no functional impact and negligible performance impact. Reviewed by: cem, emaste, pfg Approved by: emaste (mentor) MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D10920 Notes: svn path=/head/; revision=320742
* Correct an out-of-bounds read in regcomp when the RE is bad.Brooks Davis2017-05-021-0/+4
| | | | | | | | | | | | | | | | | | | | | When passed the invalid regular expression "a**", the error is eventually detected and seterr() is called. It sets p->error appropriatly and p->next and p->end to nuls which is a never used char nuls[10] which is zeros due to .bss initialization. Unfortunatly, p_ere_exp() and p_simp_re() both have fall through cases where they set the error, decrement p->next and access it which means a read from what ever .bss variable comes before nuls. Found with regex_test:repet_multi and CHERI bounds checking. Reviewed by: ngie, pfg, emaste Obtained from: CheriBSD Sponsored by: DARPA, AFRL MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D10541 Notes: svn path=/head/; revision=317707
* regex: unsign and constify some variables.Pedro F. Giffuni2017-04-236-28/+27
| | | | | | | | | | | Taking some hints from the regex variant in nvi(1) and higher-level compiler warnings, update some types in our regex(3) implementation. Joint work with: Kyle Evans MFC after: 2 weeks Notes: svn path=/head/; revision=317346
* libc: provide some bounds-checking through reallocarray(3).Pedro F. Giffuni2017-03-121-6/+6
| | | | | | | | | | | | reallocarray(3) is a non portable extension that originated in OpenBSD. Given that it is already in FreeBSD's libc it is useful for the cases where reallocation involves a multiplication. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D9955 Notes: svn path=/head/; revision=315162
* Renumber copyright clause 4Warner Losh2017-02-289-9/+9
| | | | | | | | | | | | Renumber cluase 4 to 3, per what everybody else did when BSD granted them permission to remove clause 3. My insistance on keeping the same numbering for legal reasons is too pedantic, so give up on that point. Submitted by: Jan Schaumann <jschauma@stevens.edu> Pull Request: https://github.com/freebsd/freebsd/pull/96 Notes: svn path=/head/; revision=314436
* Remove SVR4 (System V Release 4) binary compatibility support.Gleb Smirnoff2017-02-281-1/+1
| | | | | | | | UNIX System V Release 4 is operating system released in 1988. It ceased to exist in early 2000-s. Notes: svn path=/head/; revision=314373
* Replace dot-dot relative pathing with SRCTOP-relative paths where possibleEnji Cooper2017-01-201-1/+1
| | | | | | | | | | | | | | | | | | | | This reduces build output, need for recalculating paths, and makes it clearer which paths are relative to what areas in the source tree. The change in performance over a locally mounted UFS filesystem was negligible in my testing, but this may more positively impact other filesystems like NFS. LIBC_SRCTOP was left alone so Juniper (and other users) can continue to manipulate lib/libc/Makefile (and other Makefile.inc's under lib/libc) as include Makefiles with custom options. Discussed with: marcel, sjg MFC after: 1 week Reviewed by: emaste Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D9207 Notes: svn path=/head/; revision=312451
* Use SRCTOP where possible and use :H to manipulate .CURDIR to get rid ofEnji Cooper2017-01-171-1/+1
| | | | | | | | | | unnecessarily long relative path .PATH values with make MFC after: 1 days Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=312332
* 1) Eliminate possibility to call __*collate_range_cmp() with inclompleteAndrey A. Chernov2016-07-141-6/+5
| | | | | | | | | | | | | | | locale (which cause core dump) by removing whole 'table' argument by which it passed. 2) Restore __collate_range_cmp() in __sccl(). 3) Collating [a-z] range in regcomp() only for single bytes locales (we can't do it now for other ones). In previous state only first 256 wchars are considered and all others are just silently dropped from the range. Notes: svn path=/head/; revision=302824
* Back out non-collating [a-z] ranges.Andrey A. Chernov2016-07-141-2/+18
| | | | | | | | | | Instead of changing whole course to another POSIX-permitted way for consistency and uniformity I decide to completely ignore missing regex fucntionality and concentrace on fixing bugs in what we have now, too many small obstacles instead, counting ports. Notes: svn path=/head/; revision=302820
* Remove broken support for collation in [a-z] type ranges.Andrey A. Chernov2016-07-101-18/+2
| | | | | | | | | | | | | | | | | | | | | Only first 256 wide chars are considered currently, all other are just dropped from the range. Proper implementation require reverse tables database lookup, since objects are really big as max UTF-8 (1114112 code points), so just the same scanning as it was for 256 chars will slow things down. POSIX does not require collation for [a-z] type ranges and does not prohibit it for non-POSIX locales. POSIX require collation for ranges only for POSIX (or C) locale which is equal to ASCII and binary for other chars, so we already have it. No other *BSD implements collation for [a-z] type ranges. Restore ABI compatibility with unused now __collate_range_cmp() which is visible from outside (will be removed later). Notes: svn path=/head/; revision=302512
* libc/locale: Fix type breakage in __collate_range_cmp().Pedro F. Giffuni2016-06-051-3/+3
| | | | | | | | | | | | | | | | | | | When collation support was brought in, the second and third arguments in __collate_range_cmp() were changed from int to wchar_t, breaking the ABI. Change them to a "char" type which makes more sense and keeps the ABI compatible. Also introduce __wcollate_range_cmp() which does work with wide characters. This function is used only internally in libc so we don't export it. Use the new function in glob(3), fnmatch(3), and regexec(3). PR: 179721 Suggested by: ache. jilles MFC after: 3 weeks (perhaps partial only) Notes: svn path=/head/; revision=301461
* libc: regexec(3) adjustment.Pedro F. Giffuni2016-05-252-24/+50
| | | | | | | | | | | | | | | | | | | | | | | | | Change the behavior of when REG_STARTEND is combined with REG_NOTBOL. From the original posting[1]: "Enable the assumption that pmatch[0].rm_so is a continuation offset to a string and allows us to do a proper assessment of the character in regards to it's word position ('^' or '\<'), without risking going into unallocated memory." This change makes us similar to how glibc handles REG_STARTEND | REG_NOTBOL, and is closely related to a soon-to-land fix to sed. Special thanks to Martijn van Duren and Ingo Schwarze for working out some consistent behaviour. Differential Revision: https://reviews.freebsd.org/D6257 Taken from: openbsd-tech 2016-05-24 [1] (Martijn van Duren) Relnotes: yes MFC after: 1 month Notes: svn path=/head/; revision=300683
* libc/regex: fix two buffer underruns.Pedro F. Giffuni2016-05-211-9/+6
| | | | | | | | | | | | | | | Fix some rather complex regex issues found on OpenBSD as part of some ongoing work to fix a sed(1) bug. Curiously the OpenBSD tests don't trigger segfaults on FreeBSD but the bugs were confirmed by running a port of FreeBSD's regex under OpenBSD's malloc. Huge thanks to Ingo for confirming the behavior. Taken from: Ingo Schwarze (through openbsd-tech 2016-05-15) MFC after: 1 week Notes: svn path=/head/; revision=300378
* libc: spelling fixes.Pedro F. Giffuni2016-04-301-1/+1
| | | | | | | Mostly on comments. Notes: svn path=/head/; revision=298830
* regex: prevent two improbable signed integer overflows.Pedro F. Giffuni2016-04-231-2/+2
| | | | | | | | | | | | | | In matcher() we used an integer to index nsub of type size_t. In print() we used an integer to index nstates of type sopno, typedef'd long. In both cases the indexes never take negative values. Match the types to avoid any error. MFC after: 5 days Notes: svn path=/head/; revision=298521
* Add -static to CFLAGS to unbreak the tests by using a libc.a withEnji Cooper2015-12-131-1/+1
| | | | | | | | | | | | the xlocale private symbols exposed which aren't exposed publicly via the DSO PR: 191354 MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=292153
* Fix -Wformat issues and minor whitespace issues in surrounding areasEnji Cooper2015-12-051-29/+25
| | | | | | | | MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291838
* split.ih:Enji Cooper2015-12-053-4/+6
| | | | | | | | | | | | - Create automatically generated include header for split.c main.c: - Use function definitions from debug.ih and split.ih instead of externs Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291837
* Use `==` instead of `=` in the function comment above split(..) so mkh -pEnji Cooper2015-12-051-1/+1
| | | | | | | | | | exposes split(..). MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291836
* Use ANSI C function prototypes/definitions instead of K&R style onesEnji Cooper2015-12-051-16/+11
| | | | | | | | MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291835
* Add missing headers and sort #includes per style(9)Enji Cooper2015-12-051-3/+5
| | | | | | | | MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291834
* - Use ANSI C function prototypes/definitions instead of K&R style onesEnji Cooper2015-12-051-28/+12
| | | | | | | | | | - Add a missing return type for main(..) MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291833
* Fix -Wformat warnings by using the correct format qualifiersEnji Cooper2015-12-051-1/+1
| | | | | | | | MFC after: 1 week Sponsored by: EMC / Isilon Storage Division Notes: svn path=/head/; revision=291832
* mdoc: rendering fixesBaptiste Daroussin2015-04-262-4/+8
| | | | Notes: svn path=/head/; revision=282007
* computematchjumps(): fix allocator sizeof operand mismatch.Pedro F. Giffuni2015-04-221-2/+2
| | | | | | | | | Mostly cosmetical warning. Found by: Clang static analyzer Notes: svn path=/head/; revision=281858
* Prevent NULL pointer de-reference.Pedro F. Giffuni2015-02-211-1/+1
| | | | | | | | As a follow up to r279090, if dp hasn't been defined, we shouldn't attempt to do an optimization here. Notes: svn path=/head/; revision=279104
* regex(3): Fix uninitialized pointer values.Pedro F. Giffuni2015-02-202-3/+3
| | | | | | | | CID: 405582 (also clang static checker) CID: 1018724 Notes: svn path=/head/; revision=279090
* Disallow pattern spaces which would cause intermediate calculations toXin LI2015-02-141-0/+17
| | | | | | | | | | | overflow size_t. Obtained from: DragonFly (2841837793bd095a82f477e9c370cfe6cfb3862c dillon) Security: CERT VU#695940 MFC after: 3 days Notes: svn path=/head/; revision=278739
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-05 20:53:10 +0000