src/lib/libedit/tokenizer.c, branch releng/12.4

MFC r352136, r352204, r352275, r352341

2020-02-19T14:49:32Z

r352136: Import libedit 2019-09-10 Compared to current version in base: - great improvements on the Unicode support - full support for filename completion including quoting which means we do not need anymore our custom addition) - Improved readline compatiblity Upgrading libedit has been a pain in the past, because somehow we never managed to properly cleanup the tree in lib/libedit and each merge has always been very painful. After years of fighting give up and refresh a merge from scrarch properly in contrib. Note that the switch to this version will be done in another commit. r352204: Readd _el_fn_sh_complete for backward compatibility This function is not needed anymore, it allows old sh binary to continue to run and avoid breaking backward compatibility. Note that is now just calls the regular _el_fn_complete which does a proper job at quoting. Discussed with: jilles r352275: Update libedit to a snapshot from 2019-09-10 This version bring many fixes regarding unicode support It also adds proper support for filename completion (we do not need our custom patches anymore) Improves the libreadline compatibility Note that the same work was done by Yuichiro Naito in https://reviews.freebsd.org/D21196 the main difference is in this case we have reimported libedit in contrib to fix a long standing mess in the previous merges which prevented a proper update workflow. (discussed long ago with pfg@) The only difference with upstream libedit is we have added a compatibility shim for the _elf_fn_sh_complete function which we previously added to support quoting in filename completion and is not needed anymore. This was added to continue supported old /bin/sh binaries and not break backward compatibility (as discussed with jilles@) Reviewed by: Yuichiro Naito Differential Revision: https://reviews.freebsd.org/D21584 r352341: Fix arm and aarch64 builds of libedit after r352275 On arm and arm64, where chars are unsigned by default, buildworld dies with: --- terminal.o --- /usr/src/contrib/libedit/terminal.c:569:41: error: comparison of integers of different signs: 'wint_t' (aka 'int') and 'wchar_t' (aka 'unsigned int') [-Werror,-Wsign-compare] el->el_cursor.v][where & 0370] != ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ /usr/src/contrib/libedit/terminal.c:659:28: error: comparison of integers of different signs: 'wint_t' (aka 'int') and 'wchar_t' (aka 'unsigned int') [-Werror,-Wsign-compare] [el->el_cursor.h] == MB_FILL_CHAR) ~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~ Fix this by making MB_FILL_CHAR a wint_t, so no casting is needed. Note that in https://reviews.freebsd.org/D21584 this was also proposed by Yuichiro Naito . Reviewed by: bapt Subscribers: naito.yuichiro_gmail.com, ml_vishwin.info X-MFC-With: r352275 Differential Revision: https://reviews.freebsd.org/D21657

MFV r312996:

2017-01-30T22:11:53Z

Re-import libedit 2016-02-27 This reverts r296435: the issues related to lldb and this update appear to have been identified (in lldb). Obtained from: NetBSD Reported by: emaste MFC after: 3 weeks

Revert r296175

2016-03-06T21:32:54Z

Undo update of libedit 2016-02-27 Something in libedit appears to be causing breakage in lldb38. The changes are not generally huge but they are suficient to to justify reverting for now. Reported by: novel, bapt

MFV r296159

2016-02-29T00:15:25Z

Sync our libedit with NetBSD's libedit 2016-02-27. Obtained from: NetBSD

MFV r295360

2016-02-07T15:26:21Z

Sync our libedit with NetBSD's libedit 2016-01-16 Obtained from: NetBSD

Synchronize libedit with NetBSD and activate UTF-8 support [1]

2015-01-09T07:40:56Z

Differences with NetBSD Reapply our local patches on top of it Fix Unicode environement detection Fix reading a line in unicode environment. It allows /bin/sh to works in UTF-8 envs Differential Revision: https://reviews.freebsd.org/D1455 Reviewed by: jilles, pfg Obtained from: NetBSD [1] MFC after: 1 month Relnotes: yes

Merge more cosmetical changes from NetBSD's libedit.

2012-07-19T19:15:47Z

makelist: change tr '[a-z]' '[A-Z]' to tr '[:lower:]' '[:upper:]' so that POSIX systems work proerly regardsless of environment variable settings. tokenizer.c: pass lint on _LP64 Obtained from: NetBSD MFC after: 2 weeks

Sync libedit with recent NetBSD developments. Including improvements to the

2005-08-07T20:55:59Z

vi-mode, removal of clause 3, cleanups and the export of the tokenization functions. Not included: config.h, filecomplete.{c,h}

Revert the replacement of realloc() with reallocf() (el.h:1.2, map.c:1.5 and

2005-08-07T08:35:39Z

tokenizer.c:1.3). Contrary to the commit log there were no memory leaks, but the change introduced a bug because the free'd pointer was not zeroed and calling the appropriate _end() function would call free() a second time.

Fix ID's after a total boch CVS merge due to multiple people making

2001-10-01T23:00:29Z

commits to these files. As I sing to CVS: Have I told you lately that I hate your guts? Have I told you all SCM's are above you? You fill my heart with pain, take away all my merging joy, grow my troubles that's what you do."