summaryrefslogtreecommitdiff
path: root/lib/libc/regex/engine.c
Commit message (Collapse)AuthorAgeFilesLines
* libregex: implement \b and \B (word boundary, not word boundary)Kyle Evans2020-12-051-0/+37
| | | | | | | | | | This is the last of the needed GNU expressions before we can unleash bsdgrep by default. \b is effectively an agnostic equivalent of \< and \>, while \B will match every space that isn't making a transition from nonchar -> char or char -> nonchar. Notes: svn path=/head/; revision=368358
* libregex: implement \` and \' (begin-of-subj, end-of-subj)Kyle Evans2020-12-051-8/+49
| | | | | | | | | | These are GNU extensions, generally equivalent to ^ and $ except that the new syntax will not match beginning of line after the first in a multi-line expression or the end of line before absolute last in a multi-line expression. Notes: svn path=/head/; revision=368357
* libc: regex: factor out ISBOW/ISEOW macrosKyle Evans2020-12-051-8/+13
| | | | | | | | | These will be reused for \b (word boundary, which matches both sides). No functional change. Notes: svn path=/head/; revision=368356
* lib/libc/regex: fix build with REDEBUG definedYuri Pankov2019-09-241-1/+1
| | | | | | | | Reviewed by: kevans Differential Revision: https://reviews.freebsd.org/D21760 Notes: svn path=/head/; revision=352651
* regexec: fix processing multibyte strings.Yuri Pankov2018-11-231-2/+44
| | | | | | | | | | | | | | | Matcher function incorrectly assumed that moffset that we get from findmust is in bytes. Fix this by introducing a stepback function, taking short path if MB_CUR_MAX is 1, and going back byte-by-byte, checking if we have a legal character sequence otherwise. PR: 153502 Reviewed by: pfg, kevans Approved by: kib (mentor, implicit) Differential revision: https://reviews.freebsd.org/D18297 Notes: svn path=/head/; revision=340835
* regex/engine.c: error: variable 'dp' set but not usedToomas Soome2018-07-141-1/+1
| | | | | | | | | | The issue found with gcc6 build (originally on illumos, confirmed on FreeBSD). Mark it __unused. Differential Revision: https://reviews.freebsd.org/D13109 Notes: svn path=/head/; revision=336274
* regex(3): Resolve issues with higher WARNS levelsKyle Evans2018-01-211-14/+14
| | | | | | | | | | | | | libc is set for WARNS=2, but the incoming libregex will use WARNS=6. Sprinkle some casts and (void)bc's to alleviate the warnings that come along with the higher WARNS level. These 'bc' parameters could be outright removed, but as of right now they will be used in some parts of libregex land. Silence the warnings instead rather than flip-flopping. Notes: svn path=/head/; revision=328211
* General further adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-201-0/+2
| | | | | | | | | | | | | | | | | Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point. Notes: svn path=/head/; revision=326025
* regex(3): Refactor fast/slow stepping bits in the matching engineKyle Evans2017-08-091-142/+50
| | | | | | | | | | | | | | Adding features for matching is fairly straightforward, but this requires some duplication because of this fast/slow setup. They can be fairly trivially combined into a single walk(), so do it to make future additions less error prone. Reviewed by: cem (earlier version), emaste, pfg Approved by: emaste (mentor) Differential Revision: https://reviews.freebsd.org/D11233 Notes: svn path=/head/; revision=322288
* Renumber copyright clause 4Warner Losh2017-02-281-1/+1
| | | | | | | | | | | | Renumber cluase 4 to 3, per what everybody else did when BSD granted them permission to remove clause 3. My insistance on keeping the same numbering for legal reasons is too pedantic, so give up on that point. Submitted by: Jan Schaumann <jschauma@stevens.edu> Pull Request: https://github.com/freebsd/freebsd/pull/96 Notes: svn path=/head/; revision=314436
* libc: regexec(3) adjustment.Pedro F. Giffuni2016-05-251-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Change the behavior of when REG_STARTEND is combined with REG_NOTBOL. From the original posting[1]: "Enable the assumption that pmatch[0].rm_so is a continuation offset to a string and allows us to do a proper assessment of the character in regards to it's word position ('^' or '\<'), without risking going into unallocated memory." This change makes us similar to how glibc handles REG_STARTEND | REG_NOTBOL, and is closely related to a soon-to-land fix to sed. Special thanks to Martijn van Duren and Ingo Schwarze for working out some consistent behaviour. Differential Revision: https://reviews.freebsd.org/D6257 Taken from: openbsd-tech 2016-05-24 [1] (Martijn van Duren) Relnotes: yes MFC after: 1 month Notes: svn path=/head/; revision=300683
* libc/regex: fix two buffer underruns.Pedro F. Giffuni2016-05-211-9/+6
| | | | | | | | | | | | | | | Fix some rather complex regex issues found on OpenBSD as part of some ongoing work to fix a sed(1) bug. Curiously the OpenBSD tests don't trigger segfaults on FreeBSD but the bugs were confirmed by running a port of FreeBSD's regex under OpenBSD's malloc. Huge thanks to Ingo for confirming the behavior. Taken from: Ingo Schwarze (through openbsd-tech 2016-05-15) MFC after: 1 week Notes: svn path=/head/; revision=300378
* regex: prevent two improbable signed integer overflows.Pedro F. Giffuni2016-04-231-2/+2
| | | | | | | | | | | | | | In matcher() we used an integer to index nsub of type size_t. In print() we used an integer to index nstates of type sopno, typedef'd long. In both cases the indexes never take negative values. Match the types to avoid any error. MFC after: 5 days Notes: svn path=/head/; revision=298521
* Prevent NULL pointer de-reference.Pedro F. Giffuni2015-02-211-1/+1
| | | | | | | | As a follow up to r279090, if dp hasn't been defined, we shouldn't attempt to do an optimization here. Notes: svn path=/head/; revision=279104
* regex(3): Fix uninitialized pointer values.Pedro F. Giffuni2015-02-201-1/+1
| | | | | | | | CID: 405582 (also clang static checker) CID: 1018724 Notes: svn path=/head/; revision=279090
* regex: Remove some unreachable breaks.Pedro F. Giffuni2014-05-011-6/+1
| | | | | | | | | | | | This is based on a much bigger cleanup done in Illumos. Reference: https://www.illumos.org/issues/2077 MFC after: 1 week Notes: svn path=/head/; revision=265202
* Fix an off-by-one error in the marking of the O_CH operatorDiomidis Spinellis2009-09-161-1/+1
| | | | | | | | | | following an OOR2 operator. PR: 130504 MFC after: 2 weeks Notes: svn path=/head/; revision=197246
* Add a couple of debugging statements.Diomidis Spinellis2009-09-161-0/+3
| | | | Notes: svn path=/head/; revision=197245
* Const'ify and ANSIfy the internal interfaces of regex(3).Xin LI2007-05-251-109/+100
| | | | | | | | This is the final change that makes libc to compile with WERROR on my amd64 crashbox. Notes: svn path=/head/; revision=169982
* Only stop evaluation of a back reference if the match length isXin LI2007-03-051-15/+17
| | | | | | | | | zero and the recursion level is too deep. Obtained from: OpenBSD Notes: svn path=/head/; revision=167222
* Avoid infinite recursion on:Xin LI2007-03-051-0/+2
| | | | | | | | | echo "foo foo bar bar bar baz" | sed 's/\([^ ]*\)\( *\1\)*/\1/g' Obtained from: OpenBSD via NetBSD (rev. 1.18) Notes: svn path=/head/; revision=167216
* Per Regents of the University of Calfornia letter, remove advertisingWarner Losh2007-01-091-4/+0
| | | | | | | | | clause. # If I've done so improperly on a file, please let me know. Notes: svn path=/head/; revision=165903
* Fix a boundary condition error in slow() and fast() in multibyte locales:Tim J. Robbins2005-08-171-6/+7
| | | | | | | | we must allow the character beginning at "p" to be converted to a wide character for the purposes of EOL processing and word-boundary matching. Notes: svn path=/head/; revision=149180
* Plug memory leak.Diomidis Spinellis2004-12-301-0/+4
| | | | | | | | PR: bin/75656 MFC after: 2 weeks Notes: svn path=/head/; revision=139437
* Fix computation of the 'n' argument to mbrtowc (through XMBRTOWC) to avoidTim J. Robbins2004-11-211-4/+4
| | | | | | | | | | | | | reading past 'stop' in various places when converting multibyte characters. Reading too far caused truncation to not be detected when it should have been, eventually causing regexec() to loop infinitely in with certain combinations of patterns and strings in multibyte locales. PR: 74020 MFC after: 4 weeks Notes: svn path=/head/; revision=137959
* Make regular expression matching aware of multibyte characters. The generalTim J. Robbins2004-07-121-40/+92
| | | | | | | | | | | | | | | | | | | | idea is that we perform multibyte->wide character conversion while parsing and compiling, then convert byte sequences to wide characters when they're needed for comparison and stepping through the string during execution. As with tr(1), the main complication is to efficiently represent sets of characters in bracket expressions. The old bitmap representation is replaced by a bitmap for the first 256 characters combined with a vector of individual wide characters, a vector of character ranges (for [A-Z] etc.), and a vector of character classes (for [[:alpha:]] etc.). One other point of interest is that although the Boyer-Moore algorithm had to be disabled in the general multibyte case, it is still enabled for UTF-8 because of its self-synchronizing nature. This greatly speeds up matching by reducing the number of multibyte conversions that need to be done. Notes: svn path=/head/; revision=132019
* Eliminate 61 warnings emitted at WARNS=2 (leaving 53 to go).Jacques Vidrine2003-02-161-3/+3
| | | | | | | | | | Only warnings that could be fixed without changing the generated object code and without restructuring the source code have been handled. Reviewed by: /sbin/md5 Notes: svn path=/head/; revision=111010
* Fix the style of the SCM ID's.David E. O'Brien2002-03-221-2/+3
| | | | | | | I believe have made all of libc .c's as consistent as possible. Notes: svn path=/head/; revision=92986
* Remove __P() usage.David E. O'Brien2002-03-211-9/+9
| | | | Notes: svn path=/head/; revision=92905
* Remove 'register' keyword.David E. O'Brien2002-03-211-78/+78
| | | | Notes: svn path=/head/; revision=92889
* Since g->moffset points to the _maximum_ offset at which the mustDaniel C. Sobral2000-07-091-1/+1
| | | | | | | | | | | | | | | | string may be found (from the beginning of the pattern), the point at which must is found minus that offset may actually point to some place before the start of the text. In that case, make start = start. Alternatively, this could be tested for in the preceding if, but it did not occur to me. :-) Caught by: regex(3) test code Notes: svn path=/head/; revision=62854
* Deal with the signed/unsigned chars issue in a more proper manner. WeDaniel C. Sobral2000-07-071-18/+12
| | | | | | | | | | | | use a CHAR_MIN-based array, like elsewhere in the code. Remove a number of unused variables (some due to the above change, one that was left after a number of optimizing steps through the source). Brucified by: bde Notes: svn path=/head/; revision=62754
* Enhance the optimization provided by pre-matching. Fix style bugs withDaniel C. Sobral2000-07-021-7/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | previous commits. At the time we search the pattern for the "must" string, we now compute the longest offset from the beginning of the pattern at which the must string might be found. If that offset is found to be infinite (through use of "+" or "*"), we set it to -1 to disable the heuristics applied later. After we are done with pre-matching, we use that offset and the point in the text at which the must string was found to compute the earliest point at which the pattern might be found. Special care should be taken here. The variable "start" is passed to the automata-processing functions fast() and slow() to indicate the point in the text at which they should start working from. The real beginning of the text is passed in a struct match variable m, which is used to check for anchors. That variable, though, is initialized with "start", so we must not adjust "start" before "m" is properly initialized. Simple tests showed a speed increase from 100% to 400%, but they were biased in that regexec() was called for the whole file instead of line by line, and parenthized subexpressions were not searched for. This change adds a single integer to the size of the "guts" structure, and does not change the ABI. Further improvements possible: Since the speed increase observed here is so huge, one intuitive optimization would be to introduce a bias in the function that computes the "must" string so as to prefer a smaller string with a finite offset over a larger one with an infinite offset. Tests have shown this to be a bad idea, though, as the cost of false pre-matches far outweights the benefits of a must offset, even in biased situations. A number of other improvements suggest themselves, though: * identify the cases where the pattern is identical to the must string, and avoid entering fast() and slow() in these cases. * compute the maximum offset from the must string to the end of the pattern, and use that to set the point at which fast() and slow() should give up trying to find a match, and return then return to pre-matching. * return all the way to pre-matching if a "match" was found and later invalidated by back reference processing. Since back references are evil and should be avoided anyway, this is of little use. Notes: svn path=/head/; revision=62391
* Add Boyler-Moore algorithm to pre-matching test.Daniel C. Sobral2000-06-291-6/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The BM algorithm works by scanning the pattern from right to left, and jumping as many characters as viable based on the text's mismatched character and the pattern's already matched suffix. This typically enable us to test only a fraction of the text's characters, but has a worse performance than the straight-forward method for small patterns. Because of this, the BM algorithm will only be used if the pattern size is at least 4 characters. Notice that this pre-matching is done on the largest substring of the regular expression that _must_ be present on the text for a succesful match to be possible at all. For instance, "(xyzzy|grues)" will yield a null "must" substring, and, therefore, not benefit from the BM algorithm at all. Because of the lack of intelligence of the algorithm that finds the "must" string, things like "charjump|matchjump" will also yield a null string. To optimize that, "(char|match)jump" should be used. The setup time (at regcomp()) for the BM algorithm will most likely outweight any benefits for one-time matches. Given the slow regex(3) we have, this is unlikely to be even perceptible, though. The size of a regex_t structure is increased by 2*sizeof(char*) + 256*sizeof(int) + strlen(must)*sizeof(int). This is all inside the regex_t's "guts", which is allocated dynamically by regcomp(). If allocation of either of the two tables fail, the other one is freed. In this case, the straight-forward algorithm is used for pre-matching. Tests exercising the code path affected have shown a speed increase of 50% for "must" strings of length four or five. API and ABI remain unchanged by this commit. The patch submitted on the PR was not used, as it was non-functional. PR: 14342 Notes: svn path=/head/; revision=62232
* Use locale for character classes instead of hardcoded valuesAndrey A. Chernov1996-08-111-1/+1
| | | | | | | Misc 8bit cleanup Notes: svn path=/head/; revision=17508
* General -Wall warning cleanup, part I.Jordan K. Hubbard1996-07-121-2/+2
| | | | | | | Submitted-By: Kent Vander Velden <graphix@iastate.edu> Notes: svn path=/head/; revision=17141
* More cleanup.Poul-Henning Kamp1995-10-221-0/+1
| | | | | | | Uhm, I also forgot: I took "EXTRA_SANITY" out of malloc.c Notes: svn path=/head/; revision=11664
* Remove trailing whitespace.Rodney W. Grimes1995-05-301-1/+1
| | | | Notes: svn path=/head/; revision=8870
* BSD 4.4 Lite Lib SourcesRodney W. Grimes1994-05-271-0/+1091
Notes: svn path=/cvs2svn/branches/unlabeled-1.1.1/; revision=1573
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-04 14:29:39 +0000