aboutsummaryrefslogtreecommitdiff
path: root/lib/libfetch/common.h
Commit message (Collapse)AuthorAgeFilesLines
* MFH (r314778): use reallocarray(3) for extra bounds checksDag-Erling Smørgrav2018-10-091-2/+9
| | | | | | | | | | | | | | MFH (r333306): fix typo in man page MFH (r333571, r333572): preserve if-modified-since across redirects MFH (r334317): simplify the DEBUG macro MFH (r334319): style bug roundup MFH (r334326): fix netrc file location logic, improve netrcfd handling MFH (r338572): fix end-of-transfer statistics, improve no-tty display PR: 202424, 224426, 228017 Notes: svn path=/stable/11/; revision=339250
* Revert r330897:Eitan Adler2018-03-291-2/+0
| | | | | | | | | | | | | | | | | | | | This was intended to be a non-functional change. It wasn't. The commit message was thus wrong. In addition it broke arm, and merged crypto related code. Revert with prejudice. This revert skips files touched in r316370 since that commit was since MFCed. This revert also skips files that require $FreeBSD$ property changes. Thank you to those who helped me get out of this mess including but not limited to gonzo, kevans, rgrimes. Requested by: gjb (re) Notes: svn path=/stable/11/; revision=331722
* Partial merge of the SPDX changesEitan Adler2018-03-141-0/+2
| | | | | | | | | | These changes are incomplete but are making it difficult to determine what other changes can/should be merged. No objections from: pfg Notes: svn path=/stable/11/; revision=330897
* MFH (r313974,r314596): open .netrc early in case we want to drop privsDag-Erling Smørgrav2017-03-241-1/+1
| | | | | | | | | | | MFH (r314396,r315143): fix a crash caused by an incorrect format string MFH (r314701): fix handling of 416 errors when requesting a range MFH (r315455): fix parsing of IP literals (square brackets) PR: 212065, 217723 Notes: svn path=/stable/11/; revision=315902
* MFH (r308996, r309051, r309738): refactor, avoid repeating DNS requestsDag-Erling Smørgrav2016-12-141-0/+1
| | | | Notes: svn path=/stable/11/; revision=310059
* Add support for arbitrary http requestsBaptiste Daroussin2014-06-051-0/+3
| | | | | | | | | | Submitted by: Alex Hornung <alex@alexhornung.com> Reviewed by: des Obtained from: Dragonfly MFC after: 3 week Notes: svn path=/head/; revision=267133
* Bump copyright datesDag-Erling Smørgrav2014-01-301-1/+1
| | | | Notes: svn path=/head/; revision=261284
* Solve http buffering issues and hangs once and for all (hopefully!) byDag-Erling Smørgrav2014-01-281-7/+0
| | | | | | | | | | | | | | | | | | | | simply not trying to return exactly what the caller asked for - just return whatever we got and let the caller be the judge of whether it was enough. If an error occurs or the connection times out after we already received some data, return a short read, under the assumption that the next call will fail or time out before we read anything. As it turns out, none of the code that calls fetch_read() assumes an all-or-nothing result anyway, except for a couple of lines where we read the CR LF at the end of a hunk in HTTP hunked encoding, so the changes outside of fetch_read() and http_readfn() are minimal. While there, replace select(2) with poll(2). MFC after: 3 days Notes: svn path=/head/; revision=261230
* Implement certificate verification, and many other SSL-relatedDag-Erling Smørgrav2013-07-261-1/+4
| | | | | | | | | | | imrovements; complete details in the PR. PR: kern/175514 Submitted by: Michael Gmelin <freebsd@grem.de> MFC after: 1 week Notes: svn path=/head/; revision=253680
* Merge OpenSSL 1.0.1c.Jung-uk Kim2012-07-121-1/+1
| | | | | | | Approved by: benl (maintainer) Notes: svn path=/head/; revision=238405
* Fix two issues related to the use of SIGINFO in fetch(1) to displayDag-Erling Smørgrav2012-01-181-0/+7
| | | | | | | | | | | | | | | | progress information. The first is that fetch_read() (used in the HTTP code but not the FTP code) can enter an infinite loop if it has previously been interrupted by a signal. The second is that when it is interrupted, fetch_read() will discard any data it may have read up to that point. Luckily, both bugs are extremely timing-sensitive and therefore difficult to trigger. PR: bin/153240 Submitted by: Mark <markjdb@gmail.com> MFC after: 3 weeks Notes: svn path=/head/; revision=230307
* latin1 -> utf8Dag-Erling Smørgrav2011-10-191-1/+1
| | | | Notes: svn path=/head/; revision=226537
* Update copyright dates and strip my middle name.Dag-Erling Smørgrav2011-09-271-1/+1
| | | | Notes: svn path=/head/; revision=225814
* Add support for the NO_PROXY / no_proxy environment variable as used byDag-Erling Smørgrav2007-12-181-0/+1
| | | | | | | | | | | | lynx, curl etc. Note that this patch differs significantly from that in the PR, as the submitter refined it after submitting the PR. PR: 110388 Submitted by: Alexander Pohoyda <alexander.pohoyda@gmx.net> MFC after: 3 weeks Notes: svn path=/head/; revision=174752
* Clean up namespace violations.Dag-Erling Smørgrav2007-12-141-26/+26
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=174588
* Update copyright years.Dag-Erling Smørgrav2004-09-211-1/+1
| | | | Notes: svn path=/head/; revision=135546
* Add and document support for a FETCH_BIND_ADDRESS environment variableDag-Erling Smørgrav2003-03-031-0/+1
| | | | | | | | | specifying a local address to bind sockets to. Caveat: lightly tested. PR: bin/37572 Notes: svn path=/head/; revision=111816
* Experimental support for .netrc.Dag-Erling Smørgrav2003-01-221-0/+1
| | | | Notes: svn path=/head/; revision=109695
* Introduce _fetch_writev(), which is the conn_t version of writev(2). InDag-Erling Smørgrav2002-10-271-0/+4
| | | | | | | | | | | | | | | | | the SSL case, it is no different from the old _fetch_write(), but in the non-SSL case it uses writev(2) to send the entire vector as a single packet (provided it can fit in one packet). Implement _fetch_write() and _fetch_putln() in terms of _fetch_writev(). This should improve performance in the non-SSL case (by reducing protocol overhead) and solve the problem where too-smart-for-their-own-good firewalls reject FTP packets that do not end in CRLF. PR: bin/44123 Submitted by: fenner Notes: svn path=/head/; revision=106046
* Add a reference count to struct fetchconn so we don't prematurely close andDag-Erling Smørgrav2002-06-111-0/+2
| | | | | | | free a cached FTP connection. Notes: svn path=/head/; revision=98117
* Make SSL support conditional on NOCRYPT.Dag-Erling Smørgrav2002-06-051-0/+4
| | | | Notes: svn path=/head/; revision=97891
* Add SSL support + slight cleanup.Dag-Erling Smørgrav2002-06-051-2/+3
| | | | | | | Submitted by: Henry Whincup <henry@techiebod.com> (in principle) Notes: svn path=/head/; revision=97868
* Wrap everything in struct connection, and enforce timeouts everywhereDag-Erling Smørgrav2002-06-051-4/+5
| | | | | | | | (except for DNS operations). Always use funopen() for HTTP, to support both timeouts and SSL. Notes: svn path=/head/; revision=97866
* First step towards SSL support: wrap connections in a 'struct connection'Dag-Erling Smørgrav2002-06-051-3/+26
| | | | | | | | | | | | | which contains the socket descriptor, the input buffer and (yet unused) SSL state variables. This has the neat side effect of greatly improving reentrance (though we're not *quite* there yet) and opening the door to HTTP connection caching. This commit is inspired by email conversations with and patches from Henry Whincup <henry@techiebod.com> last fall. Notes: svn path=/head/; revision=97856
* Reindent, and add parentheses to return statements. Some functions inDag-Erling Smørgrav2002-02-051-7/+6
| | | | | | | | ftp.c and http.c now have exceedingly long lines due to deep nesting; this will be corrected by reorganizing the code in a later revision. Notes: svn path=/head/; revision=90267
* Introduce a fetchDebug global. Change the DEBUG macro so it only runs theDag-Erling Smørgrav2001-12-091-1/+1
| | | | | | | | | | debugging code if fetchDebug is set. PR: bin/32615 MFC after: 1 week Notes: svn path=/head/; revision=87560
* Add prototype for _ftp_request().Dag-Erling Smørgrav2001-12-041-4/+8
| | | | Notes: svn path=/head/; revision=87316
* Tons of type, style and warning fixes that have been rotting in my tree forDag-Erling Smørgrav2001-10-181-10/+10
| | | | | | | | ages - some of which wouldn't be necessary if gcc wasn't broken or TPTB were willing to do something (-fno-builtin) about it. Notes: svn path=/head/; revision=85093
* Apply 'const' liberally.Archie Cobbs2001-04-241-8/+9
| | | | | | | Fix some other minor glitches. Notes: svn path=/head/; revision=75891
* Use the documented (and historical) defaults. Centralize the decision logicDag-Erling Smørgrav2000-11-101-0/+4
| | | | | | | | | in order to avoid this bug in the future. Submitted by: se Notes: svn path=/head/; revision=68551
* Add CHECK_FLAGS, a macro for (safely) checking if a particular flag is set.Dag-Erling Smørgrav2000-10-291-0/+5
| | | | Notes: svn path=/head/; revision=67887
* ftp.c needs _http_request()Dag-Erling Smørgrav2000-10-121-0/+11
| | | | Notes: svn path=/head/; revision=67044
* Centralize the default port finding code.Dag-Erling Smørgrav2000-07-251-0/+3
| | | | | | | | Work around YA Apache bug: don't send port in Host: header if it's the default port. Notes: svn path=/head/; revision=63842
* Add _fetch_putln()Dag-Erling Smørgrav2000-07-111-1/+1
| | | | Notes: svn path=/head/; revision=62981
* Dump com_err, it's a stinking crock of shit.Dag-Erling Smørgrav2000-05-251-1/+1
| | | | Notes: svn path=/head/; revision=60924
* IPv6 support.Hajimu UMEMOTO2000-05-201-1/+1
| | | | Notes: svn path=/head/; revision=60737
* More old uncommitted patches: implement timeouts at the protocol level.Dag-Erling Smørgrav2000-01-071-2/+3
| | | | | | | Currently only supported for ftp connections. Notes: svn path=/head/; revision=55557
* $Id$ -> $FreeBSD$Peter Wemm1999-08-281-1/+1
| | | | Notes: svn path=/head/; revision=50476
* Implement and document file list retrieval.Dag-Erling Smørgrav1998-12-211-2/+4
| | | | Notes: svn path=/head/; revision=41989
* Don't specify a language to compile_et.Dag-Erling Smørgrav1998-12-181-2/+2
| | | | | | | Rename fetchConnect to _fetch_connect since it's internal. Notes: svn path=/head/; revision=41923
* Add verbose flag, and support functions.Dag-Erling Smørgrav1998-12-161-2/+3
| | | | | | | | | | | | Brucify the Makefile. Differentiate atime and mtime in fetch*Stat(). Fix a few pointer bugs. Tweak some error messages. Don't #include sys/param.h and stdio.h in fetch.h. Document that sys/param.h and stdio.h must be #included before fetch.h. Notes: svn path=/head/; revision=41862
* Second of a series of cleanups to libfetch.Dag-Erling Smørgrav1998-11-061-4/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit introduces the following features: a) the fetchStat*() functions, which return meta-information for a document, such as size, modification time, etc. b) the use of the com_err(3) facilities to report errors. It also fixes a bunch of style bugs and a few logic bugs and somewhat improves the man page. Changed files, in alphabetical order: Makefile: Don't generate macros in {ftp,http}err.c. Generate category fields for the error message lists. Compile the error table. Install fetch_err.h along with fetch.h. common.c: Remove the _netdb_errstring() macro, and add FETCH_ERR_NETDB to the error code in the _netdb_seterr() macro. Add categories to the _netdb_errlist table. Report errors through the Common Error library. common.h: Add the DEBUG macros. Add prototype for fetchConnect(). Remove the prototype for _fetch_errstring(), which is local to common.c Add a categroy field to struct fetcherr, and define constants for error categories. Define macros for _{url,netdb,ftp,http}_seterr(). errors.et: (new file) List error categories. fetch.3: Document the fetchStat*() functions. Move the "unimplemented functionality" comments from NOTES to BUGS. Document that applications which use libfetch must also use libcom_err, and list existing error codes. Undocument fetchLastErr{Code,String}. Remove the (empty) DIAGNOSTICS section. Mention Eugene Skepner in the AUTHORS section. fetch.c: Move the DEBUG macros to common.c Add fetchStat() and fetchStatURL(). Generate error messages for URL parser errors, and fix a minor bug in the parser. Use 'struct url' instead of 'url_t'. Remove fetchLastErr{Code,String}. fetch.h: Use 'struct url' instead of 'url_t', and remove the typedef. Define struct url_stat (used by fetchStat()). Add prototypes for fetchStat*(). Remove the declarations for fetchLastErr{Code,String}. Include fetch_err.h. fetch_err.et: (new file) Error table for libfetch. file.c: Add fetchStatFile(). Use 'struct url' instead of 'url_t'. ftp.c: Add fetchStatFTP(). Use 'struct url' instead of 'url_t'. Don't use fetchLastErrCode. ftp.errors: Add categories to all error messages. http.c: Add fetchStatHTTP(). Use 'struct url' instead of 'url_t'. Don't use fetchLastErr{Code,Text}. http.errors: Add categories to all error messages. Prompted by: jkh and Eugene Skepner Numerous sugestions from: Garett Wollman and Eugene Skepner Notes: svn path=/head/; revision=40975
* First of a series of cleanups to libfetch. Changed files, inDag-Erling Smørgrav1998-11-051-0/+44
alphabetical order: Makefile: Add common.c to SRCS. Make debugging easier by making 'CFLAGS += -DNDEBUG' conditional on DEBUG Don't declare struct {ftp,http}err in {ftp,http}err.c; use struct fetcherr instead. README: Remove the todo list, which is out of date anyway. common.c: (new file) Gather utility functions in this file. Merge the error reporting functions intp _fetch_errstring(), _fetch_seterr() and _fetch_syserr(). Set fetchLastErrCode and fetchLastErrText appropriately when fetchConnect fails. common.h: (new file) Gather internal prototypes and structures in this files. fetch.3: Undocument fetchFreeURL(). Document a few more known bugs. Document fetchLastErrCode and fetchLastErrText. fetch.c: Add descriptive comments to all functions that lacked them. Move fetchConnect() to common.c. Obviate the need for fetchFreeURL(), and remove it. fetch.h: Modify struct url_t so the document part is at the end. ftp.c: Remove code that is duplicated elsewhere. http.c: Remove code that is duplicated elsewhere. Prompted by: jkh Notes: svn path=/head/; revision=40939
generated by cgit v1.3 (git 2.53.0) at 2026-08-03 23:33:57 +0000