aboutsummaryrefslogtreecommitdiff
path: root/lib/libfetch
Commit message (Collapse)AuthorAgeFilesLines
* libfetch: Gracefully skip unsupported protocolsDag-Erling Smørgrav2026-02-261-1/+5
| | | | | | | | | | | If socket() fails because the address family or protocol is unsupported, just continue with the next address. MFC after: 1 week Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D55407 (cherry picked from commit b5d570e711da1dad303312bebaf1bd2fb720f0dc)
* libfetch: Fail hard if interrupted while connectingDag-Erling Smørgrav2026-02-261-1/+7
| | | | | | | | | | | | | | | | | | | This fixes an issue where the first address that DNS returns is blocked by a packet filter, so we hang for a while, then the user hits Ctrl-C, interrupting connect(2), whereupon we move on to the next address, get a connection, request the file, and return to fetch(1), which sees that SIGINT was caught and bails. Note that we make no attempt to enforce fetchTimeout in the connection phase, and never have. It's feasible, but non-trivial, so we'll leave it as an exercise for future us. PR: 293312 MFC after: 1 week Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D55406 (cherry picked from commit afbdcd402bb439bd3d487baaad63b68e95929265)
* libfetch: Clean up fetch_info usageDag-Erling Smørgrav2026-02-261-54/+34
| | | | | | | | | | | | | | * Provide a wrapper for the common if (verbose) fetch_info(...) idiom. * Replace remaining instances of fprintf(stderr, ...) with fetch_info(). * Fix a few style nits. MFC after: 1 week Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D55405 (cherry picked from commit 1b7c4d29fdfc975facdc43f6da50947cf3bb8e4a)
* libfetch: Restore timeout functionalityDag-Erling Smørgrav2026-02-181-11/+8
| | | | | | | | | | | | PR: 293124 MFC after: 1 week Fixes: 792ef1ae7b94 ("Refactor fetch_connect() and fetch_bind() to improve readability and avoid repeating the same DNS lookups.") Reverts: 8f8a7f6fffd7 ("libfetch: apply timeout to SSL_read()") Reviewed by: eugen, imp Differential Revision: https://reviews.freebsd.org/D55293 (cherry picked from commit 73b82d1b0a2f09224e6d0f7a13dd73c66d740207) (insta-mfc requested by re@)
* libfetch: Check for failure to create SSL contextDag-Erling Smørgrav2026-02-112-4/+7
| | | | | | | | | | | | | | | * Drop the ssl_meth member, there is no reason to hang on to it. * Replace deprecated SSLv23_client_method() with TLS_client_method(). * Check the return value from SSL_CTX_new(). MFC after: 1 week PR: 292903 Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D55098 (cherry picked from commit 4e160c6197f75fda3d5d5997ce893087058cf718)
* libfetch: remove all old OpenSSL supportEnji Cooper2026-02-111-27/+1
| | | | | | | | | | | This change removes pre-OpenSSL 1.1 supporting code and removes/adjusted preprocessor conditionals which were tautilogically true as FreeBSD main has shipped with OpenSSL 1.1+ for some time. Reviewed by: emaste Differential Revision: https://reviews.freebsd.org/D40711 (cherry picked from commit bc1027a7785166fde9c2a3b48e6e70d198377d4b)
* libfetch: do not call deprecated OpenSSL functionsEd Maste2026-02-111-0/+2
| | | | | | | | | | | | | | | | | | As of OpenSSL 1.1 SSL_library_init() and SSL_load_error_strings() are deprecated. There are replacement initialization functions but they do not need to be called: "As of version 1.1.0 OpenSSL will automatically allocate all resources that it needs so no explicit initialisation is required." Wrap both calls in an OPENSSL_VERSION_NUMBER block. PR: 271615 Reviewed by: Pierre Pronchery <pierre@freebsdfoundation.org> Event: Kitchener-Waterloo Hackathon 202305 Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40265 (cherry picked from commit 01aee8c92d936470c44821736e0d9e11ed7ce812)
* MFC: libfetch: allow disabling TLS v1.3 when negotiating the connectionEugene Grosbein2026-01-252-4/+10
| | | | (cherry picked from commit 129aec72250266e60c07ff4643623188f7c27a9d)
* MFC: libfetch: apply timeout to SSL_read()Eugene Grosbein2026-01-251-0/+8
| | | | | | | Currently, fetchTimeout works for non-SSL connections only, so does fetch -T. Fix it applying specified timeout to SSL_read(). (cherry picked from commit 8f8a7f6fffd7dca09013f7c4bfa075bc3825fb8e)
* libfetch: parse scheme://domain:/ correctlyKa Ho Ng2024-04-031-1/+1
| | | | | | | | | | | | This improves URL-parsing compability with cURL, and unbreaks parsing of similar kinds of URLs after commit 8d9de5b10a24. Sponsored by: Juniper Networks, Inc. Reviewed by: des MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D44493 (cherry picked from commit fb860ed0c52c2c1e7792ef86718620a439663c7f)
* libfetch, fetch: Stop recommending the use of ca_root_nss.Dag-Erling Smørgrav2023-12-131-14/+1
| | | | | | | | MFC after: 3 days Reviewed by: kevans, emaste Differential Revision: https://reviews.freebsd.org/D42119 (cherry picked from commit 2821a7498f65d357c68166e1978b491abef1ca4a)
* libfetch: don't rely on ca_root_nss for certificate validationMichael Osipov2023-10-051-8/+0
| | | | | | | | | | | | | | | | | | | | | Before certctl(8), there was no system trust store, and libfetch relied on the CA certificate bundle from the ca_root_nss port to verify peers. We now have a system trust store and a reliable mechanism for manipulating it (to explicitly add, remove, or revoke certificates), but if ca_root_nss is installed, libfetch will still prefer that to the system trust store. With this change, unless explicitly overridden, libfetch will rely on OpenSSL to pick up the default system trust store. PR: 256902 MFC after: 3 days Reviewed by: kevans Differential Revision: https://reviews.freebsd.org/D42059 (cherry picked from commit 09f5c1e118bb4eca77b83a0d08f559b20f60aa59)
* Remove $FreeBSD$: two-line nroff patternWarner Losh2023-08-231-2/+0
| | | | | | | Remove /^\.\\"\n\.\\"\s*\$FreeBSD\$$\n/ Similar commit in main: (cherry picked from commit fa9896e082a1)
* Remove $FreeBSD$: one-line sh patternWarner Losh2023-08-235-5/+0
| | | | | | | Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/ Similar commit in main: (cherry picked from commit d0b2dbfa0ecf)
* Remove $FreeBSD$: one-line .c patternWarner Losh2023-08-235-10/+0
| | | | | | | Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/ Similar commit in main: (cherry picked from commit 1d386b48a555)
* Remove $FreeBSD$: two-line .h patternWarner Losh2023-08-232-4/+0
| | | | | | | Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/ Similar commit in main: (cherry picked from commit b3e7694832e8)
* libfetch: Pass a zeroed digest to DigestCalcResponse.John Baldwin2023-01-241-2/+3
| | | | | | | | | | | | | GCC 12 warns that passing "" (a constant of char[1]) to a parameter of type char[33] could potentially overread. It is not clear from the context that c->qops can never be "auth-int" (and if it can't, then the "auth-int" handling in DigestCalcResponse is dead code that should be removed since this is the only place the function is called). Reviewed by: emaste Differential Revision: https://reviews.freebsd.org/D36825 (cherry picked from commit 57fbafb8deac75b924faf1fa6c2222a7719fdfec)
* fetch: support EAI_ADDRFAMILY error, correct two error messagesMike Karels2022-12-031-2/+5
| | | | | | | | | | | | | | | With the change to return EAI_ADDRFAMILY from getaddrinfo(), fetch would print "Unknown resolver error" for that error. Add that error and its string to libfetch's table, using an #ifdef just in case. Correct error strings for EAI_NODATA (although it is currently unused) and EAI_NONAME. Should maybe rework the code to use gai_strerror(3), but that doesn't map directly, and the current strings are shortened. Reviewed in https://reviews.freebsd.org/D37139 with related changes. Reviewed by: bz (cherry picked from commit 631b82aca0fd41c8e0d48eebdb9c4e38b7306472)
* libfetch: Use memcpy in place of an odd strncpy.John Baldwin2022-11-111-1/+1
| | | | | | | | | | | | | The length passed to strncpy is the length of the source string, not the destination buffer. This triggers a non-fatal warning in GCC 12. Hoewver, the code is also odd. It is really just a memcpy of the string without its nul terminator. For that use case, memcpy is clearer. Reviewed by: imp, emaste Differential Revision: https://reviews.freebsd.org/D36824 (cherry picked from commit 611cf392672cf7aa52a593412fb2537546a7d6a4)
* libfetch: remove a set-but-not-used variableStefan Eßer2022-04-301-5/+2
| | | | (cherry picked from commit ce700f78f7fb28a252978382a1d0a66d08b6469a)
* pkgbase: Create a FreeBSD-fetch packageEmmanuel Vadot2022-01-051-0/+2
| | | | | | | | | | | It's useful for small image to fetch some data but we don't want to install utilities nor bloat runtime. MFC after: 2 weeks Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D33463 (cherry picked from commit 13ef8134efd1c0a39c2dab0197b5c4558101253e)
* libfetch: use more portable getline() interfaceDaniel Kolesa2021-09-221-3/+6
| | | | | | | | this is for better portability in order to avoid using a function which is BSD-only or available via libbsd (cherry picked from commit ee3ca711a898cf41330c320826ea1e0e6e451f1d) (cherry picked from commit 635eb7ac7990a2bb29e1992b739617a9db012bf2)
* Fix libfetch out of bounds read.Gordon Tetlow2021-08-241-1/+4
| | | | | | | | Approved by: so Security: SA-21:15.libfetch Security: CVE-2021-36159 (cherry picked from commit 3be62d49ae2b6f9050f39fe74210c88f35901fa5)
* libfetch: Retry with proxy auth when server returns 407Renato Botelho2021-04-281-6/+49
| | | | | | | | | | | | PR: 220468 Submitted by: Egil Hasting <egil.hasting@higen.org> (based on) Reviewed by: kevans, kp Approved by: kp MFC after: 2 weeks Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D29533 (cherry picked from commit 345c30a94f6425954163f1e0b075a75f603d27cd)
* Remove support for SSLv3 from fetch(3).Jung-uk Kim2020-11-242-9/+3
| | | | | | | | | Support for SSLv3 was already removed from OpenSSL (r361392). Differential Revision: https://reviews.freebsd.org/D24947 Notes: svn path=/head/; revision=368000
* Replace literal uses of /usr/local in C sources with _PATH_LOCALBASEStefan Eßer2020-10-271-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Literal references to /usr/local exist in a large number of files in the FreeBSD base system. Many are in contributed software, in configuration files, or in the documentation, but 19 uses have been identified in C source files or headers outside the contrib and sys/contrib directories. This commit makes it possible to set _PATH_LOCALBASE in paths.h to use a different prefix for locally installed software. In order to avoid changes to openssh source files, LOCALBASE is passed to the build via Makefiles under src/secure. While _PATH_LOCALBASE could have been used here, there is precedent in the construction of the path used to a xauth program which depends on the LOCALBASE value passed on the compiler command line to select a non-default directory. This could be changed in a later commit to make the openssh build consistently use _PATH_LOCALBASE. It is considered out-of-scope for this commit. Reviewed by: imp MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D26942 Notes: svn path=/head/; revision=367075
* Don't explicitly specify c99 or gnu99 as the default is now gnu99.Xin LI2020-08-171-2/+0
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=364292
* fetch(3): plug some leaksKyle Evans2020-02-211-1/+5
| | | | | | | | | | | | | | In the successful case, sockshost is not freed prior to return. The failure case can now be hit after fetch_reopen(), which was not true before. Thus, we need to make sure to clean up all of the conn resources which will also close sd. For all of the points prior to fetch_reopen(), we continue to just close sd. CID: 1419598, 1419616 Notes: svn path=/head/; revision=358227
* fetch(3): don't leak sockshost on failureKyle Evans2020-02-151-10/+13
| | | | | | | | | | | fetch_socks5_getenv will allocate memory for the host (or set it to NULL) in all cases through the function; the caller is responsible for freeing it if we end up allocating. While I'm here, I've eliminated a label that just jumps to the next line... Notes: svn path=/head/; revision=357979
* fetch(3): fix regression in IPv6:port spec from r357977Kyle Evans2020-02-151-5/+5
| | | | | | | | | | In case the port was specified, we never actually populated *host. Do so now. Pointy hat: kevans Notes: svn path=/head/; revision=357978
* fetch(3): move bits of fetch_socks5_getenv aroundKyle Evans2020-02-151-36/+32
| | | | | | | | | This commit separates out port parsing and validation from grabbing the host from the env var. The only related bit really is that we need to be more specific with the delimiter in the IPv6 case. Notes: svn path=/head/; revision=357977
* fetch(3): Add SOCKS5 supportKyle Evans2020-02-153-11/+351
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds SOCKS5 support to the library fetch(3) and updates the man page. Details: Within the fetch_connect() function, fetch(3) checks if the SOCKS5_PROXY environment variable is set. If so, it connects to this host rather than the end-host. It then initializes the SOCKS5 connection in accordance with RFC 1928 and returns the resulting conn_t (file descriptor) for usage by the regular FTP/HTTP handlers. Design Decision: This change defaults all DNS resolutions through the proxy by sending all IPs as hostnames. Going forward, another feature might be to create another environmental variable to toggle resolutions through the proxy or not.. One may set the SOCKS5_PROXY environment variable in any of the formats: SOCKS5_PROXY=proxy.example.com SOCKS5_PROXY=proxy.example.com:1080 SOCKS5_PROXY=192.0.2.0 SOCKS5_PROXY=198.51.100.0:1080 SOCKS5_PROXY=[2001:db8::1] SOCKS5_PROXY=[2001:db8::2]:1080 Then perform a request with fetch(1). (note by kevans) I've since been informed that Void Linux/xbps has a fork of libfetch that also implements SOCKS5. I may compare/contrast the two in the mid-to-near future. Submitted by: Farhan Khan <farhan farhan codes> Differential Revision: https://reviews.freebsd.org/D18908 Notes: svn path=/head/; revision=357968
* libfetch: disallow invalid escape sequencesEd Maste2020-02-051-0/+3
| | | | | | | | | | | Per RFC1738 escape is "% hex hex"; other sequences do not form a valid URL. Suggested by: Matthew Dillon Reviewed by: Matthew Dillon MFC after: 1 week Notes: svn path=/head/; revision=357579
* Fix urldecode buffer overrun.Gordon Tetlow2020-01-281-2/+8
| | | | | | | | Reported by: Duncan Overbruck Security: CVE-2020-7450 Notes: svn path=/head/; revision=357212
* Update Makefile.depend filesSimon J. Gerraty2019-12-111-3/+0
| | | | | | | | | | | | | Update a bunch of Makefile.depend files as a result of adding Makefile.depend.options files Reviewed by: bdrewery MFC after: 1 week Sponsored by: Juniper Networks Differential Revision: https://reviews.freebsd.org/D22494 Notes: svn path=/head/; revision=355617
* Add Makefile.depend.optionsSimon J. Gerraty2019-12-111-0/+9
| | | | | | | | | | | | | | | | | | | | Leaf directories that have dependencies impacted by options need a Makefile.depend.options file to avoid churn in Makefile.depend DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc can be set in local.dirdeps-options.mk which can add to those set in Makefile.depend.options See share/mk/dirdeps-options.mk Reviewed by: bdrewery MFC after: 1 week Sponsored by: Juniper Networks Differential Revision: https://reviews.freebsd.org/D22469 Notes: svn path=/head/; revision=355616
* pkgbase: Create a FreeBSD-utilities package and make it the default oneEmmanuel Vadot2019-09-051-1/+0
| | | | | | | | | | | | | The default package use to be FreeBSD-runtime but it should only contain binaries and libs enough to boot to single user and repair the system, it is also very handy to have a package that can be tranform to a small mfsroot. So create a new package named FreeBSD-utilities and make it the default one. Also move a few binaries and lib into this package when it make sense. Reviewed by: bapt, gjb Differential Revision: https://reviews.freebsd.org/D21506 Notes: svn path=/head/; revision=351858
* Document fetchReqHTTP().Mark Johnston2019-08-283-3/+23
| | | | | | | | | | Submitted by: Farhan Khan <khanzf@gmail.com> Reviewed by: 0mp MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D18788 Notes: svn path=/head/; revision=351573
* [libfetch] Fix compilation with WITHOUT_CRYPT.Adrian Chadd2019-05-031-0/+1
| | | | Notes: svn path=/head/; revision=347050
* When deciding whether to send the complete URL or just the document part,Dag-Erling Smørgrav2018-11-271-1/+1
| | | | | | | | | | | | we were looking at the original URL rather than the one we were currently processing. This meant that if we were trying to retrieve an HTTP URL but were redirected to an HTTPS URL, and HTTPS proxying was enabled, we would send an invalid request and most likely get garbage back. MFC after: 3 days Notes: svn path=/head/; revision=341072
* A few more cases where strcasecmp() is no longer required.Dag-Erling Smørgrav2018-11-271-4/+4
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=341014
* Improve URL parsing. In particular, convert scheme and host to lowercase.Dag-Erling Smørgrav2018-11-273-43/+56
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=341013
* Support proxying FTP over HTTPS, not just HTTP.Dag-Erling Smørgrav2018-11-271-1/+2
| | | | | | | | | | | There is probably a PR for this, but I can't find this, or remember who submitted it. The patch got lost in the noise of another that wasn't ready to commit. MFC after: 3 days Notes: svn path=/head/; revision=341011
* Make libfetch buildable.Jung-uk Kim2018-09-191-0/+4
| | | | Notes: svn path=/projects/openssl111/; revision=338779
* Fix an inverted conditional in the netrc code, which would ignore theDag-Erling Smørgrav2018-05-293-14/+34
| | | | | | | | | | | | value of $HOME and always use the home directory from the passwd database, unless $HOME was unset, in which case it would use (null). While there, clean up handling of netrcfd and add debugging aids. MFC after: 3 weeks Notes: svn path=/head/; revision=334326
* Fix a few (but far from all) style issues.Dag-Erling Smørgrav2018-05-291-24/+29
| | | | | | | MFC after: 3 weeks Notes: svn path=/head/; revision=334319
* Use __VA_ARGS__ to simplify the DEBUG macro.Dag-Erling Smørgrav2018-05-295-51/+52
| | | | | | | MFC after: 3 weeks Notes: svn path=/head/; revision=334317
* Preserve if-modified-since timestamps across redirects.Dag-Erling Smørgrav2018-05-121-0/+1
| | | | | | | | PR: 224426 MFC after: 1 week Notes: svn path=/head/; revision=333571
* SPDX: use the Beerware identifier.Pedro F. Giffuni2017-11-301-1/+1
| | | | Notes: svn path=/head/; revision=326408
* lib: further adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-267-0/+14
| | | | | | | | | | | | | | | Mainly focus on files that use BSD 2-Clause license, however the tool I was using mis-identified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Notes: svn path=/head/; revision=326219
generated by cgit v1.3 (git 2.53.0) at 2026-08-01 14:51:37 +0000