Patch 1 of 2 to integrate the icap-2_5 branch into the FreeBSD squid port. Created by Thomas-Martin Seck . This patch only contains the parts of the original patchset that actually implement the ICAP client functionality. The updates to the build infrastructure are omitted to avoid the need to run an autotools bootstrap. Instead, we simulate said bootstrapping with a second patch, icap-2.5-bootstrap.patch. The patchset was pulled from the project's CVS repository at cvs.devel.squid-cache.org using cvs diff -u -b -N -kk -rs2_5 -ricap-2_5 See also for the "official" auto-generated patchset. See http://devel.squid-cache.org/icap/ for further information about the ICAP client project. Patch last updated: 2005-12-17 Index: errors/Bulgarian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Bulgarian/ERR_ICAP_FAILURE diff -N errors/Bulgarian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Bulgarian/ERR_ICAP_FAILURE 8 Dec 2003 12:30:56 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Catalan/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Catalan/ERR_ICAP_FAILURE diff -N errors/Catalan/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Catalan/ERR_ICAP_FAILURE 8 Dec 2003 12:30:57 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Czech/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Czech/ERR_ICAP_FAILURE diff -N errors/Czech/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Czech/ERR_ICAP_FAILURE 8 Dec 2003 12:30:57 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Danish/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Danish/ERR_ICAP_FAILURE diff -N errors/Danish/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Danish/ERR_ICAP_FAILURE 8 Dec 2003 12:30:57 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Dutch/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Dutch/ERR_ICAP_FAILURE diff -N errors/Dutch/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Dutch/ERR_ICAP_FAILURE 8 Dec 2003 12:30:57 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/English/ERR_ICAP_FAILURE =================================================================== RCS file: errors/English/ERR_ICAP_FAILURE diff -N errors/English/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/English/ERR_ICAP_FAILURE 8 Dec 2003 12:30:57 -0000 1.1.2.2 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Estonian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Estonian/ERR_ICAP_FAILURE diff -N errors/Estonian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Estonian/ERR_ICAP_FAILURE 8 Dec 2003 12:30:58 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Finnish/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Finnish/ERR_ICAP_FAILURE diff -N errors/Finnish/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Finnish/ERR_ICAP_FAILURE 8 Dec 2003 12:30:58 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/French/ERR_ICAP_FAILURE =================================================================== RCS file: errors/French/ERR_ICAP_FAILURE diff -N errors/French/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/French/ERR_ICAP_FAILURE 8 Dec 2003 12:30:58 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/German/ERR_ICAP_FAILURE =================================================================== RCS file: errors/German/ERR_ICAP_FAILURE diff -N errors/German/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/German/ERR_ICAP_FAILURE 23 Mar 2004 08:20:05 -0000 1.1.2.2 @@ -0,0 +1,33 @@ + + +FEHLER: Der angeforderte URL konnte nicht geholt werden + + +

FEHLER

Der angeforderte URL konnte nicht geholt werden

+Während des Versuches, den URL
+%U + +
+zu laden, trat der folgende Fehler auf: +

+ +ICAP-Protokollfehler + +

+ +

+Es trat ein Problem bei der ICAP-Kommunikation auf. Mögliche Gründe: +

Nicht erreichbarer ICAP-Server +
Ungültige Antwort vom ICAP-Server + +

+ +

Ihr Cache Administrator ist %w. + Index: errors/Greek/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Greek/ERR_ICAP_FAILURE diff -N errors/Greek/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Greek/ERR_ICAP_FAILURE 24 Sep 2005 10:31:00 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Hebrew/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Hebrew/ERR_ICAP_FAILURE diff -N errors/Hebrew/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Hebrew/ERR_ICAP_FAILURE 8 Dec 2003 12:30:59 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Hungarian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Hungarian/ERR_ICAP_FAILURE diff -N errors/Hungarian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Hungarian/ERR_ICAP_FAILURE 8 Dec 2003 12:30:59 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Italian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Italian/ERR_ICAP_FAILURE diff -N errors/Italian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Italian/ERR_ICAP_FAILURE 8 Dec 2003 12:31:00 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Japanese/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Japanese/ERR_ICAP_FAILURE diff -N errors/Japanese/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Japanese/ERR_ICAP_FAILURE 8 Dec 2003 12:31:00 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Korean/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Korean/ERR_ICAP_FAILURE diff -N errors/Korean/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Korean/ERR_ICAP_FAILURE 8 Dec 2003 12:31:00 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Lithuanian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Lithuanian/ERR_ICAP_FAILURE diff -N errors/Lithuanian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Lithuanian/ERR_ICAP_FAILURE 8 Dec 2003 12:31:00 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Polish/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Polish/ERR_ICAP_FAILURE diff -N errors/Polish/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Polish/ERR_ICAP_FAILURE 8 Dec 2003 12:31:00 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Portuguese/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Portuguese/ERR_ICAP_FAILURE diff -N errors/Portuguese/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Portuguese/ERR_ICAP_FAILURE 8 Dec 2003 12:31:01 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Romanian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Romanian/ERR_ICAP_FAILURE diff -N errors/Romanian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Romanian/ERR_ICAP_FAILURE 8 Dec 2003 12:31:01 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Russian-1251/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Russian-1251/ERR_ICAP_FAILURE diff -N errors/Russian-1251/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Russian-1251/ERR_ICAP_FAILURE 8 Dec 2003 12:31:02 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Russian-koi8-r/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Russian-koi8-r/ERR_ICAP_FAILURE diff -N errors/Russian-koi8-r/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Russian-koi8-r/ERR_ICAP_FAILURE 8 Dec 2003 12:31:02 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Serbian/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Serbian/ERR_ICAP_FAILURE diff -N errors/Serbian/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Serbian/ERR_ICAP_FAILURE 8 Dec 2003 12:31:02 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Simplify_Chinese/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Simplify_Chinese/ERR_ICAP_FAILURE diff -N errors/Simplify_Chinese/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Simplify_Chinese/ERR_ICAP_FAILURE 8 Dec 2003 12:31:02 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Slovak/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Slovak/ERR_ICAP_FAILURE diff -N errors/Slovak/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Slovak/ERR_ICAP_FAILURE 8 Dec 2003 12:31:03 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Spanish/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Spanish/ERR_ICAP_FAILURE diff -N errors/Spanish/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Spanish/ERR_ICAP_FAILURE 8 Dec 2003 12:31:03 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Swedish/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Swedish/ERR_ICAP_FAILURE diff -N errors/Swedish/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Swedish/ERR_ICAP_FAILURE 8 Dec 2003 12:31:03 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Traditional_Chinese/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Traditional_Chinese/ERR_ICAP_FAILURE diff -N errors/Traditional_Chinese/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Traditional_Chinese/ERR_ICAP_FAILURE 8 Dec 2003 12:31:03 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: errors/Turkish/ERR_ICAP_FAILURE =================================================================== RCS file: errors/Turkish/ERR_ICAP_FAILURE diff -N errors/Turkish/ERR_ICAP_FAILURE --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ errors/Turkish/ERR_ICAP_FAILURE 8 Dec 2003 12:31:04 -0000 1.1.2.1 @@ -0,0 +1,31 @@ + + +ERROR: The requested URL could not be retrieved + + +

ERROR

The requested URL could not be retrieved

+While attempting to retrieve the URL: +%U +

+the following error was encountered: +

+ +ICAP protocol error. + +

+ +

+Some aspect of the ICAP communication failed. Possible problems: +

ICAP server is not reachable. +
Illegal response from ICAP server. +

+ +

Your cache administrator is %w. + Index: include/util.h =================================================================== RCS file: /cvsroot/squid/squid/include/util.h,v retrieving revision 1.10 retrieving revision 1.10.30.2 diff -p -u -b -r1.10 -r1.10.30.2 --- include/util.h 17 Oct 2001 12:30:51 -0000 1.10 +++ include/util.h 6 Apr 2004 13:04:37 -0000 1.10.30.2 @@ -132,4 +132,12 @@ double drand48(void); */ int statMemoryAccounted(void); +#ifndef HAVE_STRNSTR +extern char *strnstr(const char *haystack, const char *needle, size_t haystacklen); +#endif + +#ifndef HAVE_STRCASESTR +extern char *strcasestr(const char *haystack, const char *needle); +#endif + #endif /* SQUID_UTIL_H */ Index: lib/strcasestr.c =================================================================== RCS file: lib/strcasestr.c diff -N lib/strcasestr.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ lib/strcasestr.c 6 Apr 2004 13:04:38 -0000 1.1.2.1 @@ -0,0 +1,126 @@ +/* Return the offset of one string within another. + Copyright (C) 1994,1996,1997,1998,1999,2000 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, write to the Free + Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA + 02111-1307 USA. */ + +/* + * My personal strstr() implementation that beats most other algorithms. + * Until someone tells me otherwise, I assume that this is the + * fastest implementation of strstr() in C. + * I deliberately chose not to comment it. You should have at least + * as much fun trying to understand it, as I had to write it :-). + * + * Stephen R. van den Berg, berg@pool.informatik.rwth-aachen.de */ + +/* + * modified to work outside of glibc (rhorstmann, 06/04/2004) + */ + +#include "config.h" +#ifndef HAVE_STRCASESTR +#include + +typedef unsigned chartype; + +char * +strcasestr (phaystack, pneedle) + const char *phaystack; + const char *pneedle; +{ + register const unsigned char *haystack, *needle; + register chartype b, c; + + haystack = (const unsigned char *) phaystack; + needle = (const unsigned char *) pneedle; + + b = tolower (*needle); + if (b != '\0') + { + haystack--; /* possible ANSI violation */ + do + { + c = *++haystack; + if (c == '\0') + goto ret0; + } + while (tolower (c) != (int) b); + + c = tolower (*++needle); + if (c == '\0') + goto foundneedle; + ++needle; + goto jin; + + for (;;) + { + register chartype a; + register const unsigned char *rhaystack, *rneedle; + + do + { + a = *++haystack; + if (a == '\0') + goto ret0; + if (tolower (a) == (int) b) + break; + a = *++haystack; + if (a == '\0') + goto ret0; +shloop: + ; + } + while (tolower (a) != (int) b); + +jin: a = *++haystack; + if (a == '\0') + goto ret0; + + if (tolower (a) != (int) c) + goto shloop; + + rhaystack = haystack-- + 1; + rneedle = needle; + a = tolower (*rneedle); + + if (tolower (*rhaystack) == (int) a) + do + { + if (a == '\0') + goto foundneedle; + ++rhaystack; + a = tolower (*++needle); + if (tolower (*rhaystack) != (int) a) + break; + if (a == '\0') + goto foundneedle; + ++rhaystack; + a = tolower (*++needle); + } + while (tolower (*rhaystack) == (int) a); + + needle = rneedle; /* took the register-poor approach */ + + if (a == '\0') + break; + } + } +foundneedle: + return (char*) haystack; +ret0: + return 0; +} +#endif Index: lib/strnstr.c =================================================================== RCS file: lib/strnstr.c diff -N lib/strnstr.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ lib/strnstr.c 16 May 2005 20:52:40 -0000 1.1.2.2 @@ -0,0 +1,52 @@ +/* + * Copyright (C) 2003 Nikos Mavroyanopoulos + * + * This file is part of GNUTLS. + * + * The GNUTLS library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + */ + + /* + * DW 2003/10/17: + * Changed 'ssize_t' types to 'size_t' + */ + +#include "config.h" +#ifndef HAVE_STRNSTR +#include +#include + +char *strnstr(const char *haystack, const char *needle, size_t haystacklen) +{ + char *p; + size_t plen; + size_t len = strlen(needle); + + if (*needle == '\0') /* everything matches empty string */ + return (char*) haystack; + + plen = haystacklen; + for (p = (char*) haystack; p != NULL; p = memchr(p + 1, *needle, plen-1)) { + plen = haystacklen - (p - haystack); + + if (plen < len) return NULL; + + if (strncmp(p, needle, len) == 0) + return (p); + } + return NULL; +} +#endif Index: src/MemBuf.c =================================================================== RCS file: /cvsroot/squid/squid/src/MemBuf.c,v retrieving revision 1.5.30.3 retrieving revision 1.5.44.8 diff -p -u -b -r1.5.30.3 -r1.5.44.8 --- src/MemBuf.c 26 Mar 2005 03:15:54 -0000 1.5.30.3 +++ src/MemBuf.c 28 Mar 2005 18:02:04 -0000 1.5.44.8 @@ -386,3 +386,15 @@ memBufReport(MemBuf * mb) assert(mb); memBufPrintf(mb, "memBufReport is not yet implemented @?@\n"); } + +int +memBufRead(int fd, MemBuf * mb) +{ + int len; + if (mb->capacity == mb->size) + memBufGrow(mb, SQUID_TCP_SO_RCVBUF); + len = FD_READ_METHOD(fd, mb->buf + mb->size, mb->capacity - mb->size); + if (len) + mb->size += len; + return len; +} Index: src/cache_cf.c =================================================================== RCS file: /cvsroot/squid/squid/src/cache_cf.c,v retrieving revision 1.38.6.29 retrieving revision 1.38.6.11.2.22 diff -p -u -b -r1.38.6.29 -r1.38.6.11.2.22 --- src/cache_cf.c 27 Oct 2005 02:13:24 -0000 1.38.6.29 +++ src/cache_cf.c 23 Nov 2005 20:38:56 -0000 1.38.6.11.2.22 @@ -2198,6 +2198,587 @@ check_null_body_size_t(dlink_list bodyli return bodylist.head == NULL; } +#ifdef HS_FEAT_ICAP + +/*************************************************** + * prototypes + */ +static int icap_service_process(icap_service * s); +static void icap_service_init(icap_service * s); +static void icap_service_destroy(icap_service * s); +icap_service *icap_service_lookup(char *name); +static int icap_class_process(icap_class * c); +static void icap_class_destroy(icap_class * c); +static void icap_access_destroy(icap_access * a); +static void dump_wordlist(StoreEntry * entry, const char *name, wordlist * list); +static void icap_class_add(icap_class * c); + +/*************************************************** + * icap_service + */ + +/* + * example: + * icap_service reqmode_precache 0 icap://192.168.0.1:1344/respmod + */ + +static void +parse_icap_service_type(IcapConfig * cfg) +{ + char *token; + icap_service *A = NULL; + icap_service *B = NULL; + icap_service **T = NULL; + + A = cbdataAlloc(icap_service); + icap_service_init(A); + parse_string(&A->name); + parse_string(&A->type_name); + parse_ushort(&A->bypass); + parse_string(&A->uri); + while ((token = strtok(NULL, w_space))) { + if (strcasecmp(token, "no-keep-alive") == 0) { + A->keep_alive = 0; + } else { + debug(3, 0) ("parse_peer: token='%s'\n", token); + self_destruct(); + } + } + debug(3, 5) ("parse_icap_service_type (line %d): %s %s %d %s\n", config_lineno, A->name, A->type_name, A->bypass, A->name); + if (icap_service_process(A)) { + /* put into linked list */ + for (B = cfg->service_head, T = &cfg->service_head; B; T = &B->next, B = B->next); + *T = A; + } else { + /* clean up structure */ + debug(3, 0) ("parse_icap_service_type (line %d): skipping %s\n", config_lineno, A->name); + icap_service_destroy(A); + cbdataFree(A); + } + +} + +static void +dump_icap_service_type(StoreEntry * e, const char *name, IcapConfig cfg) +{ + icap_service *current_node = NULL; + + if (!cfg.service_head) { + storeAppendPrintf(e, "%s 0\n", name); + return; + } + current_node = cfg.service_head; + + while (current_node) { + storeAppendPrintf(e, "%s %s %s %d %s", name, current_node->name, current_node->type_name, current_node->bypass, current_node->uri); + if (current_node->keep_alive == 0) { + storeAppendPrintf(e, " no-keep-alive"); + } + storeAppendPrintf(e, "\n"); + current_node = current_node->next; + } + +} + +static void +free_icap_service_type(IcapConfig * cfg) +{ + while (cfg->service_head) { + icap_service *current_node = cfg->service_head; + cfg->service_head = current_node->next; + icap_service_destroy(current_node); + cbdataFree(current_node); + } +} + +/* + * parse the raw string and cache some parts that are needed later + * returns 1 if everything was ok + */ +static int +icap_service_process(icap_service * s) +{ + char *start, *end, *tempEnd; + char *tailp; + unsigned int len; + int port_in_uri, resource_in_uri = 0; + s->type = icapServiceToType(s->type_name); + if (s->type >= ICAP_SERVICE_MAX) { + debug(3, 0) ("icap_service_process (line %d): wrong service type %s\n", config_lineno, s->type_name); + return 0; + } + if (s->type == ICAP_SERVICE_REQMOD_PRECACHE) + s->method = ICAP_METHOD_REQMOD; + else if (s->type == ICAP_SERVICE_REQMOD_PRECACHE) + s->method = ICAP_METHOD_REQMOD; + else if (s->type == ICAP_SERVICE_REQMOD_POSTCACHE) + s->method = ICAP_METHOD_REQMOD; + else if (s->type == ICAP_SERVICE_RESPMOD_PRECACHE) + s->method = ICAP_METHOD_RESPMOD; + else if (s->type == ICAP_SERVICE_RESPMOD_POSTCACHE) + s->method = ICAP_METHOD_RESPMOD; + debug(3, 5) ("icap_service_process (line %d): type=%s\n", config_lineno, icapServiceToStr(s->type)); + if (strncmp(s->uri, "icap://", 7) != 0) { + debug(3, 0) ("icap_service_process (line %d): wrong uri: %s\n", config_lineno, s->uri); + return 0; + } + start = s->uri + 7; + if ((end = strchr(start, ':')) != NULL) { + /* ok */ + port_in_uri = 1; + debug(3, 5) ("icap_service_process (line %d): port given\n", config_lineno); + } else { + /* ok */ + port_in_uri = 0; + debug(3, 5) ("icap_service_process (line %d): no port given\n", config_lineno); + } + + if ((tempEnd = strchr(start, '/')) != NULL) { + /* ok */ + resource_in_uri = 1; + debug(3, 5) ("icap_service_process (line %d): resource given\n", config_lineno); + if (end == '\0') { + end = tempEnd; + } + } else { + /* ok */ + resource_in_uri = 0; + debug(3, 5) ("icap_service_process (line %d): no resource given\n", config_lineno); + } + + tempEnd = strchr(start, '\0'); + if (end == '\0') { + end = tempEnd; + } + len = end - start; + s->hostname = xstrndup(start, len + 1); + s->hostname[len] = 0; + debug(3, 5) ("icap_service_process (line %d): hostname=%s\n", config_lineno, s->hostname); + start = end; + + if (port_in_uri) { + start++; /* skip ':' */ + if (resource_in_uri) + end = strchr(start, '/'); + else + end = strchr(start, '\0'); + s->port = strtoul(start, &tailp, 0) % 65536; + if (tailp != end) { + debug(3, 0) ("icap_service_process (line %d): wrong service uri (port could not be parsed): %s\n", config_lineno, s->uri); + return 0; + } + debug(3, 5) ("icap_service_process (line %d): port=%d\n", config_lineno, s->port); + start = end; + } else { + /* no explicit ICAP port; first ask by getservbyname or default to + * hardwired port 1344 per ICAP specification section 4.2 */ + struct servent *serv = getservbyname("icap", "tcp"); + if (serv) { + s->port = htons(serv->s_port); + debug(3, 5) ("icap_service_process (line %d): default port=%d getservbyname(icap,tcp)\n", config_lineno, s->port); + } else { + s->port = 1344; + debug(3, 5) ("icap_service_process (line %d): default hardwired port=%d\n", config_lineno, s->port); + } + } + + if (resource_in_uri) { + start++; /* skip '/' */ + /* the rest is resource name */ + end = strchr(start, '\0'); + len = end - start; + if (len > 1024) { + debug(3, 0) ("icap_service_process (line %d): long resource name (>1024), probably wrong\n", config_lineno); + } + s->resource = xstrndup(start, len + 1); + s->resource[len] = 0; + debug(3, 5) ("icap_service_process (line %d): service=%s\n", config_lineno, s->resource); + } + /* check bypass */ + if ((s->bypass != 0) && (s->bypass != 1)) { + debug(3, 0) ("icap_service_process (line %d): invalid bypass value\n", config_lineno); + return 0; + } + return 1; +} + +/* + * constructor + */ +static void +icap_service_init(icap_service * s) +{ + s->type = ICAP_SERVICE_MAX; /* means undefined */ + s->preview = Config.icapcfg.preview_size; + s->opt = 0; + s->keep_alive = 1; + s->istag = StringNull; + s->transfer_preview = StringNull; + s->transfer_ignore = StringNull; + s->transfer_complete = StringNull; +} + +/* + * destructor + * frees only strings, but don't touch the linked list + */ +static void +icap_service_destroy(icap_service * s) +{ + xfree(s->name); + xfree(s->uri); + xfree(s->type_name); + xfree(s->hostname); + xfree(s->resource); + assert(s->opt == 0); /* there should be no opt request running now */ + stringClean(&s->istag); + stringClean(&s->transfer_preview); + stringClean(&s->transfer_ignore); + stringClean(&s->transfer_complete); +} + +icap_service * +icap_service_lookup(char *name) +{ + icap_service *iter; + for (iter = Config.icapcfg.service_head; iter; iter = iter->next) { + if (!strcmp(name, iter->name)) { + return iter; + } + } + return NULL; +} + +/*************************************************** + * icap_service_list + */ + +static void +icap_service_list_add(icap_service_list ** isl, char *service_name) +{ + icap_service_list **iter; + icap_service_list *new; + icap_service *gbl_service; + int i; + int max_services; + + new = memAllocate(MEM_ICAP_SERVICE_LIST); + /* Found all services with that name, and add to the array */ + max_services = sizeof(new->services) / sizeof(icap_service *); + gbl_service = Config.icapcfg.service_head; + i = 0; + while (gbl_service && i < max_services) { + if (!strcmp(service_name, gbl_service->name)) + new->services[i++] = gbl_service; + gbl_service = gbl_service->next; + } + new->nservices = i; + + if (*isl) { + iter = isl; + while ((*iter)->next) + iter = &((*iter)->next); + (*iter)->next = new; + } else { + *isl = new; + } +} + +/* + * free the linked list without touching references icap_service + */ +static void +icap_service_list_destroy(icap_service_list * isl) +{ + icap_service_list *current; + icap_service_list *next; + + current = isl; + while (current) { + next = current->next; + memFree(current, MEM_ICAP_SERVICE_LIST); + current = next; + } +} + +/*************************************************** + * icap_class + */ +static void +parse_icap_class_type(IcapConfig * cfg) +{ + icap_class *s = NULL; + + s = memAllocate(MEM_ICAP_CLASS); + parse_string(&s->name); + parse_wordlist(&s->services); + + if (icap_class_process(s)) { + /* if ok, put into linked list */ + icap_class_add(s); + } else { + /* clean up structure */ + debug(3, 0) ("parse_icap_class_type (line %d): skipping %s\n", config_lineno, s->name); + icap_class_destroy(s); + memFree(s, MEM_ICAP_CLASS); + } +} + +static void +dump_icap_class_type(StoreEntry * e, const char *name, IcapConfig cfg) +{ + icap_class *current_node = NULL; + LOCAL_ARRAY(char, nom, 64); + + if (!cfg.class_head) { + storeAppendPrintf(e, "%s 0\n", name); + return; + } + current_node = cfg.class_head; + + while (current_node) { + snprintf(nom, 64, "%s %s", name, current_node->name); + dump_wordlist(e, nom, current_node->services); + current_node = current_node->next; + } +} + +static void +free_icap_class_type(IcapConfig * cfg) +{ + while (cfg->class_head) { + icap_class *current_node = cfg->class_head; + cfg->class_head = current_node->next; + icap_class_destroy(current_node); + memFree(current_node, MEM_ICAP_CLASS); + } +} + +/* + * process services list, return 1, if at least one service was found + */ +static int +icap_class_process(icap_class * c) +{ + icap_service_list *isl = NULL; + wordlist *iter; + icap_service *service; + /* take services list and build icap_service_list from it */ + for (iter = c->services; iter; iter = iter->next) { + service = icap_service_lookup(iter->key); + if (service) { + icap_service_list_add(&isl, iter->key); + } else { + debug(3, 0) ("icap_class_process (line %d): skipping service %s in class %s\n", config_lineno, iter->key, c->name); + } + } + + if (isl) { + c->isl = isl; + return 1; + } + return 0; +} + +/* + * search for an icap_class in the global IcapConfig + * classes with hidden-flag are skipped + */ +static icap_class * +icap_class_lookup(char *name) +{ + icap_class *iter; + for (iter = Config.icapcfg.class_head; iter; iter = iter->next) { + if ((!strcmp(name, iter->name)) && (!iter->hidden)) { + return iter; + } + } + return NULL; +} + +/* + * adds an icap_class to the global IcapConfig + */ +static void +icap_class_add(icap_class * c) +{ + icap_class *cp = NULL; + icap_class **t = NULL; + IcapConfig *cfg = &Config.icapcfg; + if (c) { + for (cp = cfg->class_head, t = &cfg->class_head; cp; t = &cp->next, cp = cp->next); + *t = c; + } +} + +/* + * free allocated memory inside icap_class + */ +static void +icap_class_destroy(icap_class * c) +{ + xfree(c->name); + wordlistDestroy(&c->services); + icap_service_list_destroy(c->isl); +} + +/*************************************************** + * icap_access + */ + +/* format: icap_access {allow|deny} acl, ... */ +static void +parse_icap_access_type(IcapConfig * cfg) +{ + icap_access *A = NULL; + icap_access *B = NULL; + icap_access **T = NULL; + icap_service *s = NULL; + icap_class *c = NULL; + ushort no_class = 0; + + A = memAllocate(MEM_ICAP_ACCESS); + parse_string(&A->service_name); + + /* + * try to find a class with the given name first. if not found, search + * the services. if a service is found, create a new hidden class with + * only this service. this is for backward compatibility. + * + * the special classname All is allowed only in deny rules, because + * the class is not used there. + */ + if (!strcmp(A->service_name, "None")) { + no_class = 1; + } else { + A->class = icap_class_lookup(A->service_name); + if (!A->class) { + s = icap_service_lookup(A->service_name); + if (s) { + c = memAllocate(MEM_ICAP_CLASS); + c->name = xstrdup("(hidden)"); + c->hidden = 1; + wordlistAdd(&c->services, A->service_name); + c->isl = memAllocate(MEM_ICAP_SERVICE_LIST); + /* FIXME:luc: check what access do */ + c->isl->services[0] = s; + c->isl->nservices = 1; + icap_class_add(c); + A->class = c; + } else { + debug(3, 0) ("parse_icap_access_type (line %d): servicename %s not found. skipping.\n", config_lineno, A->service_name); + memFree(A, MEM_ICAP_ACCESS); + return; + } + } + } + + aclParseAccessLine(&(A->access)); + debug(3, 5) ("parse_icap_access_type (line %d): %s\n", config_lineno, A->service_name); + + /* check that All class is only used in deny rule */ + if (no_class && A->access->allow) { + memFree(A, MEM_ICAP_ACCESS); + debug(3, 0) ("parse_icap_access (line %d): special class 'None' only allowed in deny rule. skipping.\n", config_lineno); + return; + } + if (A->access) { + for (B = cfg->access_head, T = &cfg->access_head; B; T = &B->next, B = B->next); + *T = A; + } else { + debug(3, 0) ("parse_icap_access_type (line %d): invalid line skipped\n", config_lineno); + memFree(A, MEM_ICAP_ACCESS); + } +} + +static void +dump_icap_access_type(StoreEntry * e, const char *name, IcapConfig cfg) +{ + icap_access *current_node = NULL; + LOCAL_ARRAY(char, nom, 64); + + if (!cfg.access_head) { + storeAppendPrintf(e, "%s 0\n", name); + return; + } + current_node = cfg.access_head; + + while (current_node) { + snprintf(nom, 64, "%s %s", name, current_node->service_name); + dump_acl_access(e, nom, current_node->access); + current_node = current_node->next; + } +} + +static void +free_icap_access_type(IcapConfig * cfg) +{ + while (cfg->access_head) { + icap_access *current_node = cfg->access_head; + cfg->access_head = current_node->next; + icap_access_destroy(current_node); + memFree(current_node, MEM_ICAP_ACCESS); + } +} + +/* + * destructor + * frees everything but the linked list + */ +static void +icap_access_destroy(icap_access * a) +{ + xfree(a->service_name); + aclDestroyAccessList(&a->access); +} + +/*************************************************** + * for debugging purposes only + */ +void +dump_icap_config(IcapConfig * cfg) +{ + icap_service *s_iter; + icap_class *c_iter; + icap_access *a_iter; + icap_service_list *isl_iter; + acl_list *l; + debug(3, 0) ("IcapConfig: onoff = %d\n", cfg->onoff); + debug(3, 0) ("IcapConfig: service_head = %d\n", (int) cfg->service_head); + debug(3, 0) ("IcapConfig: class_head = %d\n", (int) cfg->class_head); + debug(3, 0) ("IcapConfig: access_head = %d\n", (int) cfg->access_head); + + debug(3, 0) ("IcapConfig: services =\n"); + for (s_iter = cfg->service_head; s_iter; s_iter = s_iter->next) { + printf(" %s: \n", s_iter->name); + printf(" bypass = %d\n", s_iter->bypass); + printf(" hostname = %s\n", s_iter->hostname); + printf(" port = %d\n", s_iter->port); + printf(" resource = %s\n", s_iter->resource); + } + debug(3, 0) ("IcapConfig: classes =\n"); + for (c_iter = cfg->class_head; c_iter; c_iter = c_iter->next) { + printf(" %s: \n", c_iter->name); + printf(" services = \n"); + for (isl_iter = c_iter->isl; isl_iter; isl_iter = isl_iter->next) { + int i; + for (i = 0; i < isl_iter->nservices; i++) + printf(" %s\n", isl_iter->services[i]->name); + } + } + debug(3, 0) ("IcapConfig: access =\n"); + for (a_iter = cfg->access_head; a_iter; a_iter = a_iter->next) { + printf(" service_name = %s\n", a_iter->service_name); + printf(" access = %s", a_iter->access->allow ? "allow" : "deny"); + for (l = a_iter->access->acl_list; l != NULL; l = l->next) { + printf(" %s%s", + l->op ? null_string : "!", + l->acl->name); + } + printf("\n"); + } +} +#endif /* HS_FEAT_ICAP */ static void parse_kb_size_t(squid_off_t * var) Index: src/cbdata.c =================================================================== RCS file: /cvsroot/squid/squid/src/cbdata.c,v retrieving revision 1.14.6.1 retrieving revision 1.14.32.2 diff -p -u -b -r1.14.6.1 -r1.14.32.2 --- src/cbdata.c 17 Jul 2003 02:13:28 -0000 1.14.6.1 +++ src/cbdata.c 14 Sep 2003 01:36:26 -0000 1.14.32.2 @@ -144,6 +144,10 @@ cbdataInit(void) CREATE_CBDATA(statefulhelper); CREATE_CBDATA(helper_stateful_server); CREATE_CBDATA(HttpStateData); +#ifdef HS_FEAT_ICAP + CREATE_CBDATA(IcapStateData); + CREATE_CBDATA(icap_service); +#endif CREATE_CBDATA_FREE(peer, peerDestroy); CREATE_CBDATA(ps_state); CREATE_CBDATA(RemovalPolicy); Index: src/cf.data.pre =================================================================== RCS file: /cvsroot/squid/squid/src/cf.data.pre,v retrieving revision 1.49.2.84 retrieving revision 1.49.2.33.2.32 diff -p -u -b -r1.49.2.84 -r1.49.2.33.2.32 --- src/cf.data.pre 21 Oct 2005 02:13:47 -0000 1.49.2.84 +++ src/cf.data.pre 24 Oct 2005 17:07:42 -0000 1.49.2.33.2.32 @@ -2397,7 +2397,6 @@ DOC_START ensure correct results it is best to set server_persisten_connections to off when using this directive in such configurations. DOC_END - NAME: reply_header_max_size COMMENT: (KB) TYPE: b_size_t @@ -2716,6 +2715,177 @@ DOC_START DOC_END COMMENT_START + ICAP OPTIONS + ----------------------------------------------------------------------------- +COMMENT_END + +NAME: icap_enable +TYPE: onoff +IFDEF: HS_FEAT_ICAP +COMMENT: on|off +LOC: Config.icapcfg.onoff +DEFAULT: off +DOC_START + If you want to enable the ICAP client module, set this to on. +DOC_END + +NAME: icap_preview_enable +TYPE: onoff +IFDEF: HS_FEAT_ICAP +COMMENT: on|off +LOC: Config.icapcfg.preview_enable +DEFAULT: off +DOC_START + Set this to 'on' if you want to enable the ICAP preview + feature in Squid. +DOC_END + +NAME: icap_preview_size +TYPE: int +IFDEF: HS_FEAT_ICAP +LOC: Config.icapcfg.preview_size +DEFAULT: -1 +DOC_START + The default size of preview data to be sent to the ICAP server. + -1 means no preview. This value might be overwritten on a per server + basis by OPTIONS requests. +DOC_END + +NAME: icap_check_interval +TYPE: int +IFDEF: HS_FEAT_ICAP +LOC: Config.icapcfg.check_interval +DEFAULT: 300 +DOC_START + If an ICAP server does not respond, it gets marked as unreachable. Squid + will try again to reach it after this time. +DOC_END + +NAME: icap_send_client_ip +TYPE: onoff +IFDEF: HS_FEAT_ICAP +COMMENT: on|off +LOC: Config.icapcfg.send_client_ip +DEFAULT: off +DOC_START + This adds the header "X-Client-IP" to ICAP requests. Can also be + set from the server's response to OPTIONS. +DOC_END + +NAME: icap_send_auth_user +TYPE: onoff +IFDEF: HS_FEAT_ICAP +COMMENT: on|off +LOC: Config.icapcfg.send_auth_user +DEFAULT: off +DOC_START + This adds the header "X-Authenticated-User" to ICAP requests + if proxy access is authentified. Can also be set from the server's + response to OPTIONS. +DOC_END + +NAME: icap_auth_scheme +TYPE: string +IFDEF: HS_FEAT_ICAP +LOC: Config.icapcfg.auth_scheme +DEFAULT: Local://%u +DOC_START + Authentification scheme to pass to ICAP requests if + icap_send_auth_user is enabled. The first occurence of "%u" + is replaced by the authentified user name. If no "%u" is found, + the username is added at the end of the scheme. + + See http://www.ietf.org/internet-drafts/draft-stecher-icap-subid-00.txt, + section 3.4 for details on this. + + Examples: + + icap_auth_scheme Local://%u + icap_auth_scheme LDAP://ldap-server/cn=%u,dc=company,dc=com + icap_auth_scheme WinNT://nt-domain/%u + icap_auth_scheme Radius://radius-server/%u +DOC_END + +NAME: icap_service +TYPE: icap_service_type +IFDEF: HS_FEAT_ICAP +LOC: Config.icapcfg +DEFAULT: none +DOC_START + Defines a single ICAP service + + icap_service servicename vectoring_point bypass service_url [options ...] + + vectoring_point = reqmod_precache|reqmod_postcache|respmod_precache|respmod_postcache + This specifies at which point of request processing the ICAP + service should be plugged in. + bypass = 1|0 + If set to 1 and the ICAP server cannot be reached, the request will go + through without being processed by an ICAP server + service_url = icap://servername:port/service + + Options: + + no-keep-alive To always close the connection to icap server + after the transaction completes + + + Note: reqmod_precache and respmod_postcache is not yet implemented + + Load-balancing and high availability: + You can obtain load-balancing and high availability by defining a + named service with different definitions. Then, the client + loops through the different entities of the service providing + load-balancing. If an entity is marked as unreachable, the client goes + one step further to the next entity: you have the high-availability. + See the service_1 definition below + +Example: +icap_service service_1 reqmod_precache 0 icap://icap1.mydomain.net:1344/reqmod +icap_service service_1 reqmod_precache 0 icap://icap2.mydomain.net:1344/reqmod no-keep-alive +icap_service service_2 respmod_precache 0 icap://icap3.mydomain.net:1344/respmod +DOC_END + +NAME: icap_class +TYPE: icap_class_type +IFDEF: HS_FEAT_ICAP +LOC: Config.icapcfg +DEFAULT: none +DOC_START + Defines an ICAP service chain. If there are multiple services per + vectoring point, they are processed in the specified order. + + icap_class classname servicename... + +Example: +icap_class class_1 service_1 service_2 +icap class class_2 service_1 service_3 +DOC_END + +NAME: icap_access +TYPE: icap_access_type +IFDEF: HS_FEAT_ICAP +LOC: Config.icapcfg +DEFAULT: none +DOC_START + Redirects a request through an ICAP service class, depending + on given acls + + icap_access classname allow|deny [!]aclname... + + The icap_access statements are processed in the order they appear in + this configuration file. If an access list matches, the processing stops. + For an "allow" rule, the specified class is used for the request. A "deny" + rule simply stops processing without using the class. You can also use the + special classname "None". + + For backward compatibility, it is also possible to use services + directly here. +Example: +icap_access class_1 allow all +DOC_END + +COMMENT_START MISCELLANEOUS ----------------------------------------------------------------------------- COMMENT_END Index: src/cf_gen_defines =================================================================== RCS file: /cvsroot/squid/squid/src/cf_gen_defines,v retrieving revision 1.5 retrieving revision 1.5.48.3 diff -p -u -b -r1.5 -r1.5.48.3 --- src/cf_gen_defines 3 Dec 2001 08:03:21 -0000 1.5 +++ src/cf_gen_defines 13 Mar 2005 17:58:44 -0000 1.5.48.3 @@ -18,12 +18,13 @@ BEGIN { define["USE_UNLINKD"]="--enable-unlinkd" define["USE_USERAGENT_LOG"]="--enable-useragent-log" define["USE_WCCP"]="--enable-wccp" + define["HS_FEAT_ICAP"]="--enable-icap-support" } /^IFDEF:/ { if (define[$2] != "") - DEFINE=define[$2] + DEFINE = define[$2] else - DEFINE="-D" $2 + DEFINE = "-D" $2 print "{\"" $2 "\", \"" DEFINE "\", " print "#if " $2 print "1" Index: src/client_side.c =================================================================== RCS file: /cvsroot/squid/squid/src/client_side.c,v retrieving revision 1.47.2.71 retrieving revision 1.47.2.28.2.40 diff -p -u -b -r1.47.2.71 -r1.47.2.28.2.40 --- src/client_side.c 19 Oct 2005 02:13:20 -0000 1.47.2.71 +++ src/client_side.c 6 Dec 2005 21:53:44 -0000 1.47.2.28.2.40 @@ -109,7 +109,7 @@ static const char *const crlf = "\r\n"; static CWCB clientWriteComplete; static CWCB clientWriteBodyComplete; static PF clientReadRequest; -static PF connStateFree; +PF connStateFree; static PF requestTimeout; static PF clientLifetimeTimeout; static int clientCheckTransferDone(clientHttpRequest *); @@ -136,20 +136,23 @@ static void clientSetKeepaliveFlag(clien static void clientPackRangeHdr(const HttpReply * rep, const HttpHdrRangeSpec * spec, String boundary, MemBuf * mb); static void clientPackTermBound(String boundary, MemBuf * mb); static void clientInterpretRequestHeaders(clientHttpRequest *); -static void clientProcessRequest(clientHttpRequest *); +void clientProcessRequest(clientHttpRequest *); static void clientProcessExpired(void *data); static void clientProcessOnlyIfCachedMiss(clientHttpRequest * http); -static int clientCachable(clientHttpRequest * http); -static int clientHierarchical(clientHttpRequest * http); -static int clientCheckContentLength(request_t * r); +int clientCachable(clientHttpRequest * http); +int clientHierarchical(clientHttpRequest * http); +int clientCheckContentLength(request_t * r); static DEFER httpAcceptDefer; static log_type clientProcessRequest2(clientHttpRequest * http); static int clientReplyBodyTooLarge(clientHttpRequest *, squid_off_t clen); static int clientRequestBodyTooLarge(squid_off_t clen); static void clientProcessBody(ConnStateData * conn); static void clientEatRequestBody(clientHttpRequest *); -static BODY_HANDLER clientReadBody; +BODY_HANDLER clientReadBody; static void clientAbortBody(request_t * req); +#if HS_FEAT_ICAP +static int clientIcapReqMod(clientHttpRequest * http); +#endif static int checkAccelOnly(clientHttpRequest * http) @@ -392,6 +395,10 @@ clientRedirectDone(void *data, char *res http->request = requestLink(new_request); } clientInterpretRequestHeaders(http); +#if HS_FEAT_ICAP + if (Config.icapcfg.onoff) + icapCheckAcl(http); +#endif #if HEADERS_LOG headersLog(0, 1, request->method, request); #endif @@ -931,11 +938,22 @@ httpRequestFree(void *data) *H = http->next; http->next = NULL; dlinkDelete(&http->active, &ClientActiveRequests); +#if HS_FEAT_ICAP + /*In the case that the upload of data breaks, we need this code here .... */ + if (NULL != http->icap_reqmod) { + if (cbdataValid(http->icap_reqmod)) + if (http->icap_reqmod->icap_fd > -1) { + comm_close(http->icap_reqmod->icap_fd); + } + cbdataUnlock(http->icap_reqmod); + http->icap_reqmod = NULL; + } +#endif cbdataFree(http); } /* This is a handler normally called by comm_close() */ -static void +void connStateFree(int fd, void *data) { ConnStateData *connState = data; @@ -958,7 +976,6 @@ connStateFree(int fd, void *data) } else safe_free(connState->in.buf); /* XXX account connState->in.buf */ - pconnHistCount(0, connState->nrequests); cbdataFree(connState); #ifdef _SQUID_LINUX_ /* prevent those nasty RST packets */ @@ -1103,7 +1120,7 @@ clientSetKeepaliveFlag(clientHttpRequest } } -static int +int clientCheckContentLength(request_t * r) { switch (r->method) { @@ -1122,7 +1139,7 @@ clientCheckContentLength(request_t * r) /* NOT REACHED */ } -static int +int clientCachable(clientHttpRequest * http) { request_t *req = http->request; @@ -1148,7 +1165,7 @@ clientCachable(clientHttpRequest * http) } /* Return true if we can query our neighbors for this object */ -static int +int clientHierarchical(clientHttpRequest * http) { const char *url = http->uri; @@ -2439,7 +2456,7 @@ clientProcessRequest2(clientHttpRequest return LOG_TCP_HIT; } -static void +void clientProcessRequest(clientHttpRequest * http) { char *url = http->uri; @@ -2449,6 +2466,11 @@ clientProcessRequest(clientHttpRequest * debug(33, 4) ("clientProcessRequest: %s '%s'\n", RequestMethodStr[r->method], url); +#if HS_FEAT_ICAP + if (clientIcapReqMod(http)) { + return; + } +#endif if (r->method == METHOD_CONNECT && !http->redirect.status) { http->log_type = LOG_TCP_MISS; sslStart(http, &http->out.size, &http->al.http.code); @@ -2993,6 +3015,20 @@ clientReadRequest(int fd, void *data) (long) conn->in.offset, (long) conn->in.size); len = conn->in.size - conn->in.offset - 1; } +#if HS_FEAT_ICAP + /* + * This check exists because ICAP doesn't always work well + * with persistent (reused) connections. One version of the + * REQMOD code creates a fake ConnStateData, which doesn't have + * an in.buf. We want to make sure that the fake ConnStateData + * doesn't get used here. + */ + if (NULL == conn->in.buf) { + debug(33, 1) ("clientReadRequest: FD %d aborted; conn->in.buf is NULL\n", fd); + comm_close(fd); + return; + } +#endif statCounter.syscalls.sock.reads++; size = FD_READ_METHOD(fd, conn->in.buf + conn->in.offset, len); if (size > 0) { @@ -3096,7 +3132,8 @@ clientReadRequest(int fd, void *data) /* add to the client request queue */ for (H = &conn->chr; *H; H = &(*H)->next); *H = http; - conn->nrequests++; + F->pconn.uses++; + F->pconn.type = 0; /* * I wanted to lock 'http' here since its callback data for * clientLifetimeTimeout(), but there's no logical place to @@ -3266,7 +3303,7 @@ clientReadRequest(int fd, void *data) } /* file_read like function, for reading body content */ -static void +void clientReadBody(request_t * request, char *buf, size_t size, CBCB * callback, void *cbdata) { ConnStateData *conn = request->body_reader_data; @@ -3390,7 +3427,7 @@ clientProcessBody(ConnStateData * conn) } /* Abort a body request */ -static void +void clientAbortBody(request_t * request) { ConnStateData *conn = request->body_reader_data; @@ -3432,7 +3469,7 @@ requestTimeout(int fd, void *data) * Some data has been sent to the client, just close the FD */ comm_close(fd); - } else if (conn->nrequests) { + } else if (fd_table[fd].pconn.uses) { /* * assume its a persistent connection; just close it */ @@ -3948,3 +3985,49 @@ varyEvaluateMatch(StoreEntry * entry, re } } } + +#if HS_FEAT_ICAP +static int +clientIcapReqMod(clientHttpRequest * http) +{ + ErrorState *err; + icap_service *service; + if (http->flags.did_icap_reqmod) + return 0; + if (NULL == (service = icapService(ICAP_SERVICE_REQMOD_PRECACHE, http->request))) + return 0; + debug(33, 3) ("clientIcapReqMod: calling icapReqModStart for %p\n", http); + /* + * Note, we pass 'start' and 'log_addr' to ICAP so the access.log + * entry comes out right. The 'clientHttpRequest' created by + * the ICAP side is the one that gets logged. The first + * 'clientHttpRequest' does not get logged because its out.size + * is zero and log_type is unset. + */ + http->icap_reqmod = icapReqModStart(service, + http->uri, + http->request, + http->conn->fd, + http->start, + http->conn->log_addr, + (void *) http->conn); + if (NULL == http->icap_reqmod) { + return 0; + } else if (-1 == (int) http->icap_reqmod) { + /* produce error */ + http->icap_reqmod = NULL; + debug(33, 2) ("clientIcapReqMod: icap told us to send an error\n"); + http->log_type = LOG_TCP_DENIED; + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->xerrno = ETIMEDOUT; + err->request = requestLink(http->request); + err->src_addr = http->conn->peer.sin_addr; + http->entry = clientCreateStoreEntry(http, http->request->method, null_request_flags); + errorAppendEntry(http->entry, err); + return 1; + } + cbdataLock(http->icap_reqmod); + http->flags.did_icap_reqmod = 1; + return 1; +} +#endif Index: src/comm.c =================================================================== RCS file: /cvsroot/squid/squid/src/comm.c,v retrieving revision 1.18.6.6 retrieving revision 1.18.6.2.12.9 diff -p -u -b -r1.18.6.6 -r1.18.6.2.12.9 --- src/comm.c 11 Sep 2005 02:13:22 -0000 1.18.6.6 +++ src/comm.c 23 Nov 2005 20:33:06 -0000 1.18.6.2.12.9 @@ -653,8 +653,8 @@ comm_close(int fd) #endif CommWriteStateCallbackAndFree(fd, COMM_ERR_CLOSING); commCallCloseHandlers(fd); - if (F->uses) /* assume persistent connect count */ - pconnHistCount(1, F->uses); + if (F->pconn.uses) + pconnHistCount(F->pconn.type, F->pconn.uses); #if USE_SSL if (F->ssl) { SSL_free(F->ssl); Index: src/enums.h =================================================================== RCS file: /cvsroot/squid/squid/src/enums.h,v retrieving revision 1.29.2.18 retrieving revision 1.29.2.8.2.17 diff -p -u -b -r1.29.2.18 -r1.29.2.8.2.17 --- src/enums.h 12 Nov 2005 03:13:48 -0000 1.29.2.18 +++ src/enums.h 23 Nov 2005 20:38:56 -0000 1.29.2.8.2.17 @@ -93,6 +93,7 @@ typedef enum { ERR_ONLY_IF_CACHED_MISS, /* failure to satisfy only-if-cached request */ ERR_TOO_BIG, TCP_RESET, + ERR_ICAP_FAILURE, ERR_INVALID_RESP, ERR_MAX } err_type; @@ -438,6 +439,9 @@ typedef enum { PROTO_WHOIS, PROTO_INTERNAL, PROTO_HTTPS, +#if HS_FEAT_ICAP + PROTO_ICAP, +#endif PROTO_MAX } protocol_t; @@ -610,6 +614,12 @@ typedef enum { MEM_TLV, MEM_SWAP_LOG_DATA, MEM_CLIENT_REQ_BUF, +#if HS_FEAT_ICAP + MEM_ICAP_OPT_DATA, + MEM_ICAP_SERVICE_LIST, + MEM_ICAP_CLASS, + MEM_ICAP_ACCESS, +#endif MEM_MAX } mem_type; @@ -709,9 +719,14 @@ typedef enum { CBDATA_RemovalPolicyWalker, CBDATA_RemovalPurgeWalker, CBDATA_store_client, +#ifdef HS_FEAT_ICAP + CBDATA_IcapStateData, + CBDATA_icap_service, +#endif CBDATA_FIRST_CUSTOM_TYPE = 1000 } cbdata_type; + /* * Return codes from checkVary(request) */ @@ -742,4 +757,68 @@ enum { #endif +#if HS_FEAT_ICAP +typedef enum { + ICAP_STATUS_NONE = 0, + ICAP_STATUS_CONTINUE = 100, + ICAP_STATUS_SWITCHING_PROTOCOLS = 101, + ICAP_STATUS_STATUS_OK = 200, + ICAP_CREATED = 201, + ICAP_STATUS_ACCEPTED = 202, + ICAP_STATUS_NON_AUTHORITATIVE_INFORMATION = 203, + ICAP_STATUS_NO_MODIFICATION_NEEDED = 204, + ICAP_STATUS_RESET_CONTENT = 205, + ICAP_STATUS_PARTIAL_CONTENT = 206, + ICAP_STATUS_MULTIPLE_CHOICES = 300, + ICAP_STATUS_MOVED_PERMANENTLY = 301, + ICAP_STATUS_MOVED_TEMPORARILY = 302, + ICAP_STATUS_SEE_OTHER = 303, + ICAP_STATUS_NOT_MODIFIED = 304, + ICAP_STATUS_USE_PROXY = 305, + ICAP_STATUS_BAD_REQUEST = 400, + ICAP_STATUS_UNAUTHORIZED = 401, + ICAP_STATUS_PAYMENT_REQUIRED = 402, + ICAP_STATUS_FORBIDDEN = 403, + ICAP_STATUS_SERVICE_NOT_FOUND = 404, + ICAP_STATUS_METHOD_NOT_ALLOWED = 405, + ICAP_STATUS_NOT_ACCEPTABLE = 406, + ICAP_STATUS_PROXY_AUTHENTICATION_REQUIRED = 407, + ICAP_STATUS_REQUEST_TIMEOUT = 408, + ICAP_STATUS_CONFLICT = 409, + ICAP_STATUS_GONE = 410, + ICAP_STATUS_LENGTH_REQUIRED = 411, + ICAP_STATUS_PRECONDITION_FAILED = 412, + ICAP_STATUS_REQUEST_ENTITY_TOO_LARGE = 413, + ICAP_STATUS_REQUEST_URI_TOO_LARGE = 414, + ICAP_STATUS_UNSUPPORTED_MEDIA_TYPE = 415, + ICAP_STATUS_INTERNAL_SERVER_ERROR = 500, + ICAP_STATUS_NOT_IMPLEMENTED = 501, + ICAP_STATUS_BAD_GATEWAY = 502, + ICAP_STATUS_SERVICE_OVERLOADED = 503, + ICAP_STATUS_GATEWAY_TIMEOUT = 504, + ICAP_STATUS_ICAP_VERSION_NOT_SUPPORTED = 505, + ICAP_STATUS_INVALID_HEADER = 600 +} icap_status; + +/* + * these values are used as index in an array, so it seems to be better to + * assign some numbers + */ +typedef enum { + ICAP_SERVICE_REQMOD_PRECACHE = 0, + ICAP_SERVICE_REQMOD_POSTCACHE = 1, + ICAP_SERVICE_RESPMOD_PRECACHE = 2, + ICAP_SERVICE_RESPMOD_POSTCACHE = 3, + ICAP_SERVICE_MAX = 4 +} icap_service_t; + +typedef enum { + ICAP_METHOD_NONE, + ICAP_METHOD_OPTION, + ICAP_METHOD_REQMOD, + ICAP_METHOD_RESPMOD +} icap_method_t; + +#endif /* HS_FEAT_ICAP */ + #endif /* SQUID_ENUMS_H */ Index: src/forward.c =================================================================== RCS file: /cvsroot/squid/squid/src/forward.c,v retrieving revision 1.13.6.15 retrieving revision 1.13.6.3.2.15 diff -p -u -b -r1.13.6.15 -r1.13.6.3.2.15 --- src/forward.c 2 Sep 2005 02:13:43 -0000 1.13.6.15 +++ src/forward.c 30 Nov 2005 21:52:15 -0000 1.13.6.3.2.15 @@ -262,7 +262,8 @@ fwdConnectDone(int server_fd, int status else hierarchyNote(&fwdState->request->hier, fs->code, request->host); fd_note(server_fd, storeUrl(fwdState->entry)); - fd_table[server_fd].uses++; + fd_table[server_fd].pconn.uses++; + fd_table[server_fd].pconn.type = 1; if (fs->peer) peerConnectSucceded(fs->peer); fwdDispatch(fwdState); @@ -704,6 +705,8 @@ fwdCheckDeferRead(int fd, void *data) void fwdFail(FwdState * fwdState, ErrorState * errorState) { + if (NULL == fwdState) + return; debug(17, 3) ("fwdFail: %s \"%s\"\n\t%s\n", err_type_str[errorState->type], httpStatusString(errorState->http_status), @@ -742,6 +745,8 @@ fwdPeerClosed(int fd, void *data) void fwdUnregister(int fd, FwdState * fwdState) { + if (NULL == fwdState) + return; debug(17, 3) ("fwdUnregister: %s\n", storeUrl(fwdState->entry)); assert(fd == fwdState->server_fd); assert(fd > -1); @@ -758,7 +763,10 @@ fwdUnregister(int fd, FwdState * fwdStat void fwdComplete(FwdState * fwdState) { - StoreEntry *e = fwdState->entry; + StoreEntry *e; + if (NULL == fwdState) + return; + e = fwdState->entry; assert(e->store_status == STORE_PENDING); debug(17, 3) ("fwdComplete: %s\n\tstatus %d\n", storeUrl(e), e->mem_obj->reply->sline.status); Index: src/globals.h =================================================================== RCS file: /cvsroot/squid/squid/src/globals.h,v retrieving revision 1.14.6.7 retrieving revision 1.14.6.3.2.5 diff -p -u -b -r1.14.6.7 -r1.14.6.3.2.5 --- src/globals.h 14 Jun 2005 02:15:00 -0000 1.14.6.7 +++ src/globals.h 12 Sep 2005 18:34:41 -0000 1.14.6.3.2.5 @@ -165,6 +165,9 @@ extern char *WIN32_OS_string; /* NULL */ #if HAVE_SBRK extern void *sbrk_start; /* 0 */ #endif +#if HS_FEAT_ICAP +extern char *icap_service_type_str[]; +#endif extern int opt_send_signal; /* -1 */ extern int opt_no_daemon; /* 0 */ Index: src/http.c =================================================================== RCS file: /cvsroot/squid/squid/src/http.c,v retrieving revision 1.17.6.32 retrieving revision 1.17.6.3.6.39 diff -p -u -b -r1.17.6.32 -r1.17.6.3.6.39 --- src/http.c 19 Oct 2005 02:13:21 -0000 1.17.6.32 +++ src/http.c 23 Nov 2005 20:33:07 -0000 1.17.6.3.6.39 @@ -47,7 +47,7 @@ static CWCB httpSendRequestEntry; static PF httpReadReply; static void httpSendRequest(HttpStateData *); -static PF httpStateFree; +PF httpStateFree; static PF httpTimeout; static void httpCacheNegatively(StoreEntry *); static void httpMakePrivate(StoreEntry *); @@ -55,11 +55,12 @@ static void httpMakePublic(StoreEntry *) static int httpCachableReply(HttpStateData *); static void httpMaybeRemovePublic(StoreEntry *, http_status); -static void +void httpStateFree(int fd, void *data) { HttpStateData *httpState = data; #if DELAY_POOLS + if (fd >= 0) delayClearNoDelay(fd); #endif if (httpState == NULL) @@ -79,6 +80,9 @@ httpStateFree(int fd, void *data) requestUnlink(httpState->orig_request); httpState->request = NULL; httpState->orig_request = NULL; +#if HS_FEAT_ICAP + cbdataUnlock(httpState->icap_writer); +#endif cbdataFree(httpState); } @@ -392,7 +396,7 @@ httpMakeVaryMark(request_t * request, Ht } /* rewrite this later using new interfaces @?@ */ -static void +void httpProcessReplyHeader(HttpStateData * httpState, const char *buf, int size) { StoreEntry *entry = httpState->entry; @@ -527,24 +531,35 @@ httpPconnTransferDone(HttpStateData * ht MemObject *mem = httpState->entry->mem_obj; HttpReply *reply = mem->reply; squid_off_t clen; + squid_off_t content_bytes_read; debug(11, 3) ("httpPconnTransferDone: FD %d\n", httpState->fd); debug(11, 5) ("httpPconnTransferDone: content_length=%" PRINTF_OFF_T "\n", reply->content_length); /* If we haven't seen the end of reply headers, we are not done */ - if (httpState->reply_hdr_state < 2) + if (httpState->reply_hdr_state < 2) { + debug(11, 3) ("httpPconnTransferDone: reply_hdr_state=%d, returning 0\n", + httpState->reply_hdr_state); return 0; + } clen = httpReplyBodySize(httpState->request->method, reply); +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) { + content_bytes_read = httpState->icap_writer->fake_content_length; + debug(11, 3) ("using fake conten length %" PRINTF_OFF_T "\n", content_bytes_read); + } else +#endif + content_bytes_read = mem->inmem_hi; /* If the body size is unknown we must wait for EOF */ if (clen < 0) return 0; /* Barf if we got more than we asked for */ - if (mem->inmem_hi > clen + reply->hdr_sz) + if (content_bytes_read > clen + reply->hdr_sz) return -1; /* If there is no message body, we can be persistent */ if (0 == clen) return 1; /* If the body size is known, we must wait until we've gotten all of it. */ - if (mem->inmem_hi < clen + reply->hdr_sz) + if (content_bytes_read < clen + reply->hdr_sz) return 0; /* We got it all */ return 1; @@ -568,6 +583,17 @@ httpReadReply(int fd, void *data) delay_id delay_id; #endif +#if HS_FEAT_ICAP + if (httpState->icap_writer) { + if (!httpState->icap_writer->respmod.entry) { + debug(11, 3) ("httpReadReply: FD: %d: icap respmod aborded!\n", fd); + comm_close(fd); + return; + } + /*The folowing entry can not be marked as aborted. + * The StoreEntry icap_writer->respmod.entry used when the icap_write used...... */ + } else +#endif if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) { comm_close(fd); return; @@ -579,6 +605,33 @@ httpReadReply(int fd, void *data) else delay_id = delayMostBytesAllowed(entry->mem_obj, &read_sz); #endif + +#if HS_FEAT_ICAP + if (httpState->icap_writer) { + IcapStateData *icap = httpState->icap_writer; + /* + * Ok we have a received a response from the web server, so try to + * connect the icap server if it's the first attemps. If we try + * to connect to the icap server, defer this request (do not read + * the buffer), and defer until icapConnectOver() is not called. + */ + if (icap->flags.connect_requested == 0) { + debug(81, 2) ("icapSendRespMod: Create a new connection to icap server\n"); + if (!icapConnect(icap, icapConnectOver)) { + debug(81, 2) ("icapSendRespMod: Something strange while creating a socket to icap server\n"); + commSetSelect(fd, COMM_SELECT_READ, httpReadReply, httpState, 0); + return; + } + debug(81, 2) ("icapSendRespMod: new connection to icap server (using FD=%d)\n", icap->icap_fd); + icap->flags.connect_requested = 1; + /* Wait for more data or EOF condition */ + commSetTimeout(fd, httpState->flags.keepalive_broken ? 10 : Config.Timeout.read, NULL, NULL); + commSetSelect(fd, COMM_SELECT_READ, httpReadReply, httpState, 0); + return; + } + } +#endif + errno = 0; statCounter.syscalls.sock.reads++; len = FD_READ_METHOD(fd, buf, read_sz); @@ -595,7 +648,13 @@ httpReadReply(int fd, void *data) clen >>= 1; IOStats.Http.read_hist[bin]++; } - if (!httpState->reply_hdr.size && len > 0 && fd_table[fd].uses > 1) { +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) + (void) 0; + else +#endif + + if (!httpState->reply_hdr.size && len > 0 && fd_table[fd].pconn.uses > 1) { /* Skip whitespace */ while (len > 0 && xisspace(*buf)) xmemmove(buf, buf + 1, len--); @@ -625,6 +684,12 @@ httpReadReply(int fd, void *data) } else if (len == 0) { /* Connection closed; retrieval done. */ httpState->eof = 1; +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer && cbdataValid(httpState->icap_writer)) { + debug(81, 3) ("httpReadReply: EOF for ICAP writer\n"); + icapSendRespMod(httpState->icap_writer, buf, len, 1); + } +#endif if (httpState->reply_hdr_state < 2) /* * Yes Henrik, there is a point to doing this. When we @@ -677,7 +742,28 @@ httpReadReply(int fd, void *data) EBIT_CLR(entry->flags, ENTRY_FWD_HDR_WAIT); } } +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) { + debug(81, 5) ("calling icapSendRespMod from %s:%d\n", __FILE__, __LINE__); + if (cbdataValid(httpState->icap_writer)) { + icapSendRespMod(httpState->icap_writer, buf, len, 0); + httpState->icap_writer->fake_content_length += len; + } + } else +#endif storeAppend(entry, buf, len); + + + debug(11, 5) ("httpReadReply: after storeAppend FD %d read %d\n", fd, len); +#if HS_FEAT_ICAP + if (httpState->icap_writer) { + if (!httpState->icap_writer->respmod.entry) { + debug(11, 3) ("httpReadReply: FD: %d: icap respmod aborded!\n", fd); + comm_close(fd); + return; + } + } else +#endif if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) { /* * the above storeAppend() call could ABORT this entry, @@ -724,10 +810,21 @@ httpReadReply(int fd, void *data) ("httpReadReply: Excess data from \"%s %s\"\n", RequestMethodStr[httpState->orig_request->method], storeUrl(entry)); +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) { + debug(81, 5) ("calling icapSendRespMod from %s:%d\n", __FILE__, __LINE__); + icapSendRespMod(httpState->icap_writer, buf, len, 0); + httpState->icap_writer->fake_content_length += len; + } else +#endif storeAppend(entry, buf, len); keep_alive = 0; } } +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) + icapSendRespMod(httpState->icap_writer, NULL, 0, 1); +#endif if (keep_alive) { /* yes we have to clear all these! */ commSetDefer(fd, NULL, NULL); @@ -766,6 +863,10 @@ httpReadReply(int fd, void *data) ("httpReadReply: Excess data from \"%s %s\"\n", RequestMethodStr[httpState->orig_request->method], storeUrl(entry)); +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) + icapSendRespMod(httpState->icap_writer, NULL, 0, 1); +#endif fwdComplete(httpState->fwd); comm_close(fd); return; @@ -776,6 +877,34 @@ httpReadReply(int fd, void *data) } } +#ifdef HS_FEAT_ICAP +static int +httpReadReplyWaitForIcap(int fd, void *data) +{ + HttpStateData *httpState = data; + if (NULL == httpState->icap_writer) + return 0; + /* + * Do not defer when we are not connected to the icap server. + * Defer when the icap server connection is not established but pending + * Defer when the icap server is busy (writing on the socket) + */ + debug(11, 5) ("httpReadReplyWaitForIcap: FD %d, connect_requested=%d\n", + fd, httpState->icap_writer->flags.connect_requested); + if (!httpState->icap_writer->flags.connect_requested) + return 0; + debug(11, 5) ("httpReadReplyWaitForIcap: FD %d, connect_pending=%d\n", + fd, httpState->icap_writer->flags.connect_pending); + if (httpState->icap_writer->flags.connect_pending) + return 1; + debug(11, 5) ("httpReadReplyWaitForIcap: FD %d, write_pending=%d\n", + fd, httpState->icap_writer->flags.write_pending); + if (httpState->icap_writer->flags.write_pending) + return 1; + return 0; +} +#endif + /* This will be called when request write is complete. Schedule read of * reply. */ static void @@ -803,6 +932,63 @@ httpSendComplete(int fd, char *bufnotuse comm_close(fd); return; } else { + /* Schedule read reply. */ +#ifdef HS_FEAT_ICAP + if (icapService(ICAP_SERVICE_RESPMOD_PRECACHE, httpState->orig_request)) { + httpState->icap_writer = icapRespModStart( + ICAP_SERVICE_RESPMOD_PRECACHE, + httpState->orig_request, httpState->entry, httpState->flags); + if (-1 == (int) httpState->icap_writer) { + /* TODO: send error here and exit */ + ErrorState *err; + httpState->icap_writer = 0; + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->xerrno = errno; + err->request = requestLink(httpState->orig_request); + errorAppendEntry(entry, err); + comm_close(fd); + return; + } else if (httpState->icap_writer) { + request_flags fake_flags = httpState->orig_request->flags; + method_t fake_method = entry->mem_obj->method; + const char *fake_msg = "this is a fake entry for " + " response sent to an ICAP RESPMOD server"; + cbdataLock(httpState->icap_writer); + /* + * this httpState will give the data it reads to + * the icap server, rather than put it into + * a StoreEntry + */ + storeUnlockObject(httpState->entry); + storeUnregisterAbort(httpState->entry); + /* + * create a bogus entry because the code assumes one is + * always there. + */ + fake_flags.cachable = 0; + fake_flags.hierarchical = 0; /* force private key */ + httpState->entry = storeCreateEntry("fake", "fake", fake_flags, fake_method); + storeAppend(httpState->entry, fake_msg, strlen(fake_msg)); + /* + * pull a switcheroo on fwdState->entry. + */ + storeUnlockObject(httpState->fwd->entry); + httpState->fwd->entry = httpState->entry; + storeLockObject(httpState->fwd->entry); + /* + * Note that we leave fwdState connected to httpState, + * but we changed the entry. So when fwdComplete + * or whatever is called it does no harm -- its + * just the fake entry. + */ + } else { + /* + * failed to open connection to ICAP server. + * But bypass request, so just continue here. + */ + } + } +#endif /* * Set the read timeout here because it hasn't been set yet. * We only set the read timeout after the request has been @@ -811,8 +997,18 @@ httpSendComplete(int fd, char *bufnotuse * the timeout for POST/PUT requests that have very large * request bodies. */ + + /* removed in stable5: + * commSetSelect(fd, COMM_SELECT_READ, httpReadReply, httpState, 0); + */ commSetTimeout(fd, Config.Timeout.read, httpTimeout, httpState); - commSetDefer(fd, fwdCheckDeferRead, entry); +#ifdef HS_FEAT_ICAP + if (httpState->icap_writer) { + debug(11, 5) ("FD %d, setting httpReadReplyWaitForIcap\n", httpState->fd); + commSetDefer(httpState->fd, httpReadReplyWaitForIcap, httpState); + } else +#endif + commSetDefer(httpState->fd, fwdCheckDeferRead, entry); } httpState->flags.request_sent = 1; } @@ -1010,8 +1206,11 @@ httpBuildRequestHeader(request_t * reque if (!EBIT_TEST(cc->mask, CC_MAX_AGE)) { const char *url = entry ? storeUrl(entry) : urlCanonical(orig_request); httpHdrCcSetMaxAge(cc, getMaxAge(url)); +#ifndef HS_FEAT_ICAP + /* Don;t bother - if the url you want to cache is redirected? */ if (strLen(request->urlpath)) assert(strstr(url, strBuf(request->urlpath))); +#endif } /* Set no-cache if determined needed but not found */ if (orig_request->flags.nocache && !httpHeaderHas(hdr_in, HDR_PRAGMA)) @@ -1119,6 +1318,7 @@ httpStart(FwdState * fwd) int fd = fwd->server_fd; HttpStateData *httpState; request_t *proxy_req; + /* ErrorState *err; */ request_t *orig_req = fwd->request; debug(11, 3) ("httpStart: \"%s %s\"\n", RequestMethodStr[orig_req->method], @@ -1156,12 +1356,22 @@ httpStart(FwdState * fwd) httpState->request = requestLink(orig_req); httpState->orig_request = requestLink(orig_req); } +#ifdef HS_FEAT_ICAP + if (icapService(ICAP_SERVICE_REQMOD_POSTCACHE, httpState->orig_request)) { + httpState->icap_writer = icapRespModStart(ICAP_SERVICE_REQMOD_POSTCACHE, + httpState->orig_request, httpState->entry, httpState->flags); + if (httpState->icap_writer) { + return; + } + } +#endif /* * register the handler to free HTTP state data when the FD closes */ comm_add_close_handler(fd, httpStateFree, httpState); statCounter.server.all.requests++; statCounter.server.http.requests++; + httpSendRequest(httpState); /* * We used to set the read timeout here, but not any more. Index: src/icap_common.c =================================================================== RCS file: src/icap_common.c diff -N src/icap_common.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/icap_common.c 22 Nov 2005 22:41:48 -0000 1.1.2.39 @@ -0,0 +1,785 @@ +/* + * $Id$ + * + * DEBUG: section 81 Internet Content Adaptation Protocol (ICAP) Client + * AUTHOR: Geetha Manjunath, Hewlett Packard Company + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + */ + +/* _GNU_SOURCE is required for strcasestr */ +#define _GNU_SOURCE 1 + +#include "squid.h" +#include "util.h" + +extern PF httpStateFree; + +#define EXPECTED_ICAP_HEADER_LEN 256 +#define ICAP_OPTIONS_REQUEST + + +void +icapInit() +{ +#ifdef ICAP_OPTIONS_REQUEST + if (Config.icapcfg.onoff) { + icapOptInit(); + } +#endif +} + +void +icapClose() +{ + icapOptShutdown(); +} + +/* + * search for a HTTP-like header in the buffer. + * Note, buf must be 0-terminated + * + * This function is not very good. It should probably look for + * header tokens only at the start of a line, not just anywhere in + * the buffer. + */ +int +icapFindHeader(const char *buf, const char *hdr, const char **Start, + const char **End) +{ + const char *start = NULL; + const char *end = NULL; + start = strcasestr(buf, hdr); + if (NULL == start) + return 0; + end = start + strcspn(start, "\r\n"); + if (start == end) + return 0; + *Start = start; + *End = end; + return 1; +} + +/* + * parse the contents of the encapsulated header (buffer between enc_start + * and enc_end) and put the result into IcapStateData + */ +void +icapParseEncapsulated(IcapStateData * icap, const char *enc_start, + const char *enc_end) +{ + char *current, *end; + + assert(icap); + assert(enc_start); + assert(enc_end); + + current = strchr(enc_start, ':'); + current++; + while (current < enc_end) { + while (isspace(*current)) + current++; + if (!strncmp(current, "res-hdr=", 8)) { + current += 8; + icap->enc.res_hdr = strtol(current, &end, 10); + } else if (!strncmp(current, "req-hdr=", 8)) { + current += 8; + icap->enc.req_hdr = strtol(current, &end, 10); + } else if (!strncmp(current, "null-body=", 10)) { + current += 10; + icap->enc.null_body = strtol(current, &end, 10); + } else if (!strncmp(current, "res-body=", 9)) { + current += 9; + icap->enc.res_body = strtol(current, &end, 10); + } else if (!strncmp(current, "req-body=", 9)) { + current += 9; + icap->enc.req_body = strtol(current, &end, 10); + } else if (!strncmp(current, "opt-body=", 9)) { + current += 9; + icap->enc.opt_body = strtol(current, &end, 10); + } else { + /* invalid header */ + debug(81, 5) ("icapParseEncapsulated: error in: %s\n", current); + return; + } + current = end; + current = strchr(current, ','); + if (current == NULL) + break; + else + current++; /* skip ',' */ + } + debug(81, + 3) ("icapParseEncapsulated: res-hdr=%d, req-hdr=%d, null-body=%d, " + "res-body=%d, req-body=%d, opt-body=%d\n", icap->enc.res_hdr, + icap->enc.req_hdr, icap->enc.null_body, icap->enc.res_body, + icap->enc.req_body, icap->enc.opt_body); + +} + +icap_service * +icapService(icap_service_t type, request_t * r) +{ + icap_service_list *isl_iter; + int is_iter; + int nb_unreachable = 0; + icap_service *unreachable_one = NULL; + + debug(81, 8) ("icapService: type=%s\n", icapServiceToStr(type)); + if (NULL == r) { + debug(81, 8) ("icapService: no request_t\n"); + return NULL; + } + if (NULL == r->class) { + debug(81, 8) ("icapService: no class\n"); + return NULL; + } + for (isl_iter = r->class->isl; isl_iter; isl_iter = isl_iter->next) { + /* TODO:luc: Do a round-robin, choose a random value ? + * For now, we use a simple round robin with checking is the + * icap server is available */ + is_iter = isl_iter->last_service_used; + do { + is_iter = (is_iter + 1) % isl_iter->nservices; + debug(81, 8) ("icapService: checking service %s/id=%d\n", + isl_iter->services[is_iter]->name, is_iter); + if (type == isl_iter->services[is_iter]->type) { + if (!isl_iter->services[is_iter]->unreachable) { + debug(81, 8) ("icapService: found service %s/id=%d\n", + isl_iter->services[is_iter]->name, is_iter); + isl_iter->last_service_used = is_iter; + return isl_iter->services[is_iter]; + } + debug(81, + 8) + ("icapService: found service %s/id=%d, but it's unreachable. I don't want to use it\n", + isl_iter->services[is_iter]->name, is_iter); + unreachable_one = isl_iter->services[is_iter]; + nb_unreachable++; + /* FIXME:luc: in response mod, if we return an NULL pointer, user can bypass + * the filter, is it normal ? */ + } + } while (is_iter != isl_iter->last_service_used); + } + debug(81, 8) ("icapService: no service found\n"); + isl_iter = r->class->isl; + + if (nb_unreachable > 0) { + debug(81, + 8) + ("All the services are unreachable, returning an unreachable one\n"); + return unreachable_one; + } else { + return NULL; + } +} + +int +icapConnect(IcapStateData * icap, CNCB * theCallback) +{ + int rc; + icap->icap_fd = pconnPop(icap->current_service->hostname, + icap->current_service->port); + if (icap->icap_fd >= 0) { + debug(81, 3) ("icapConnect: reused pconn FD %d\n", icap->icap_fd); + fd_note(icap->icap_fd, icap->current_service->uri); + comm_add_close_handler(icap->icap_fd, icapStateFree, icap); + theCallback(icap->icap_fd, 0, icap); + return 1; + } + icap->icap_fd = comm_open(SOCK_STREAM, 0, getOutgoingAddr(NULL), 0, + COMM_NONBLOCKING, icap->current_service->uri); + debug(81, 5) ("icapConnect: new socket, FD %d, local address %s\n", + icap->icap_fd, inet_ntoa(getOutgoingAddr(NULL))); + if (icap->icap_fd < 0) { + icapStateFree(-1, icap); /* XXX test */ + return 0; + } + icap->flags.connect_pending = 1; + /* + * Configure timeout and close handler before calling + * connect because commConnectStart() might get an error + * immediately and close the descriptor before it returns. + */ + commSetTimeout(icap->icap_fd, Config.Timeout.connect, + icapConnectTimeout, icap); + comm_add_close_handler(icap->icap_fd, icapStateFree, icap); + /* + * This sucks. commConnectStart() may fail before returning, + * so lets lock the data and check its validity afterwards. + */ + cbdataLock(icap); + commConnectStart(icap->icap_fd, + icap->current_service->hostname, + icap->current_service->port, theCallback, icap); + rc = cbdataValid(icap); + cbdataUnlock(icap); + debug(81, 3) ("icapConnect: returning %d\n", rc); + return rc; +} + +IcapStateData * +icapAllocate(void) +{ + IcapStateData *icap; + + if (!Config.icapcfg.onoff) + return 0; + + icap = cbdataAlloc(IcapStateData); + icap->icap_fd = -1; + icap->enc.res_hdr = -1; + icap->enc.res_body = -1; + icap->enc.req_hdr = -1; + icap->enc.req_body = -1; + icap->enc.opt_body = -1; + icap->enc.null_body = -1; + icap->chunk_size = -1; + memBufDefInit(&icap->icap_hdr); + + debug(81, 3) ("New ICAP state\n"); + return icap; +} + +void +icapStateFree(int fd, void *data) +{ + IcapStateData *icap = data; + debug(81, 3) ("icapStateFree: FD %d, icap %p\n", fd, icap); + assert(icap); + assert(-1 == fd || fd == icap->icap_fd); + if (icap->respmod.entry) { + /* + * If we got some error on this side (like ECONNRESET) + * we must signal the other side(s) with a storeAbort() + * call. + */ + if (icap->respmod.entry->store_status != STORE_OK) + storeAbort(icap->respmod.entry); + storeUnlockObject(icap->respmod.entry); + icap->respmod.entry = NULL; + } + requestUnlink(icap->request); + icap->request = NULL; + if (!memBufIsNull(&icap->icap_hdr)) + memBufClean(&icap->icap_hdr); + if (!memBufIsNull(&icap->respmod.buffer)) + memBufClean(&icap->respmod.buffer); + if (!memBufIsNull(&icap->respmod.req_hdr_copy)) + memBufClean(&icap->respmod.req_hdr_copy); + if (!memBufIsNull(&icap->respmod.resp_copy)) + memBufClean(&icap->respmod.resp_copy); + if (!memBufIsNull(&icap->reqmod.hdr_buf)) + memBufClean(&icap->reqmod.hdr_buf); + if (!memBufIsNull(&icap->reqmod.http_entity.buf)) + memBufClean(&icap->reqmod.http_entity.buf); + if (!memBufIsNull(&icap->chunk_buf)) + memBufClean(&icap->chunk_buf); + if (icap->httpState) + httpStateFree(-1, icap->httpState); + cbdataUnlock(icap->reqmod.client_cookie); + cbdataFree(icap); +} + +void +icapConnectTimeout(int fd, void *data) +{ + IcapStateData *icap = data; + debug(81, 3) ("icapConnectTimeout: FD %d, unreachable=1\n", fd); + assert(fd == icap->icap_fd); + icapOptSetUnreachable(icap->current_service); + comm_close(fd); +} + +void +icapReadTimeout(int fd, void *data) +{ + IcapStateData *icap = data; + assert(fd == icap->icap_fd); + if (icap->flags.wait_for_preview_reply || icap->flags.http_server_eof) { + debug(81, 3) ("icapReadTimeout: FD %d, unreachable=1\n", fd); + icapOptSetUnreachable(icap->current_service); + } else + debug(81, 3) ("icapReadTimeout: FD %d, still reachable\n", fd); + comm_close(fd); +} + +icap_service_t +icapServiceToType(const char *s) +{ + if (!strcmp(s, "reqmod_precache")) + return ICAP_SERVICE_REQMOD_PRECACHE; + if (!strcmp(s, "reqmod_postcache")) + return ICAP_SERVICE_REQMOD_POSTCACHE; + if (!strcmp(s, "respmod_precache")) + return ICAP_SERVICE_RESPMOD_PRECACHE; + if (!strcmp(s, "respmod_postcache")) + return ICAP_SERVICE_RESPMOD_POSTCACHE; + return ICAP_SERVICE_MAX; +} + +const char * +icapServiceToStr(const icap_service_t type) +{ + if (type >= 0 && type < ICAP_SERVICE_MAX) + return icap_service_type_str[type]; + else + return "error"; +} + + +/* copied from clientAclChecklistCreate */ +static aclCheck_t * +icapAclChecklistCreate(const acl_access * acl, const clientHttpRequest * http) +{ + aclCheck_t *ch; + ConnStateData *conn = http->conn; + ch = aclChecklistCreate(acl, http->request, 0); + ch->conn = conn; + cbdataLock(ch->conn); + return ch; +} + +/* + * check wether we do icap for a request + */ +int +icapCheckAcl(clientHttpRequest * http) +{ + icap_access *iter; + aclCheck_t *icapChecklist; + + for (iter = Config.icapcfg.access_head; iter; iter = iter->next) { + acl_access *A = iter->access; + icapChecklist = icapAclChecklistCreate(A, http); + if (aclMatchAclList(A->acl_list, icapChecklist)) { + debug(81, 5) ("icapCheckAcl: match for class=%s\n", + iter->class->name); + if (A->allow) { + /* allow rule, do icap and use associated class */ + http->request->class = iter->class; + aclChecklistFree(icapChecklist); + return 1; + } else { + /* deny rule, stop processing */ + aclChecklistFree(icapChecklist); + return 0; + } + } + aclChecklistFree(icapChecklist); + } + return 0; +} + +/* icapLineLength + * + * returns the amount of data until lineending ( \r\n ) + * This function is NOT tolerant of variations of \r\n. + */ +size_t +icapLineLength(const char *start, int len) +{ + size_t lineLen = 0; + char *end = (char *) memchr(start, '\r', len); + if (NULL == end) + return 0; + end++; /* advance to where '\n' should be */ + lineLen = end - start + 1; + if (lineLen > len) { + debug(0, 0) ("icapLineLength: warning lineLen (%d) > len (%d)\n", + lineLen, len); + return 0; + } + if (*end != '\n') { + debug(0, 0) ("icapLineLength: warning *end (%x) != '\\n'\n", *end); + return 0; + } + debug(81, 7) ("icapLineLength: returning %d\n", lineLen); + return lineLen; +} + +/* + * return: + * -1 if EOF before getting end of ICAP header + * 0 if we don't have the entire ICAP header yet + * 1 if we got the whole header + */ +int +icapReadHeader(int fd, IcapStateData * icap, int *isIcap) +{ + int headlen = 0; + int len = 0; + int peek_sz = EXPECTED_ICAP_HEADER_LEN; + int read_sz = 0; + LOCAL_ARRAY(char, tmpbuf, SQUID_TCP_SO_RCVBUF); + for (;;) { + len = recv(fd, tmpbuf, peek_sz, MSG_PEEK); + debug(81, 5) ("recv(FD %d, ..., MSG_PEEK) ret %d\n", fd, len); + if (len < 0) { + debug(81, 1) ("icapReadHeader: FD %d recv error: %s\n", fd, + xstrerror()); + return -1; + } + if (len == 0) { + debug(81, 2) ("icapReadHeader: FD %d recv EOF\n", fd); + return -1; + } + headlen = headersEnd(tmpbuf, len); + debug(81, 3) ("headlen=%d\n", headlen); + /* + * break if we now know where the ICAP headers end + */ + if (headlen) + break; + /* + * break if we know there is no more data to read + */ + if (len < peek_sz) + break; + /* + * The ICAP header is larger than (or equal to) our read + * buffer, so double it and try to peek again. + */ + peek_sz *= 2; + if (peek_sz >= SQUID_TCP_SO_RCVBUF) { + debug(81, + 1) ("icapReadHeader: Failed to find end of ICAP header\n"); + debug(81, 1) ("\twithin first %d bytes of response\n", + SQUID_TCP_SO_RCVBUF); + debug(81, 1) ("\tpossible persistent connection bug/confusion\n"); + return -1; + } + } + /* + * Now actually read the data from the kernel + */ + if (headlen) + read_sz = headlen; + else + read_sz = len; + len = FD_READ_METHOD(fd, tmpbuf, read_sz); + assert(len == read_sz); + fd_bytes(fd, len, FD_READ); + memBufAppend(&icap->icap_hdr, tmpbuf, len); + if (headlen) { + /* End of ICAP header found */ + if (icap->icap_hdr.size < 4) + *isIcap = 0; + else if (0 == strncmp(icap->icap_hdr.buf, "ICAP", 4)) + *isIcap = 1; + else + *isIcap = 0; + return 1; + } + /* + * We don't have all the headers yet + */ + return 0; +} + +static int +icapParseConnectionClose(const IcapStateData * icap, const char *s, + const char *e) +{ + char *t; + char *q; + /* + * s points to the start of the line "Connection: ... " + * e points to *after* the last character on the line + */ + s += 11; /* skip past Connection: */ + while (s < e && isspace(*s)) + s++; + if (e - s < 5) + return 0; + /* + * create a buffer that we can use strtok on + */ + t = xmalloc(e - s + 1); + strncpy(t, s, e - s); + *(t + (e - s)) = '\0'; + for (q = strtok(t, ","); q; q = strtok(NULL, ",")) { + if (0 == strcasecmp(q, "close")) { + xfree(t); + return 1; + } + } + xfree(t); + return 0; +} + +/* returns icap status, version and subversion extracted from status line or -1 on parsing failure + * The str_status pointr points to the text returned from the icap server. + * sline probably is NOT terminated with '\0' + */ +int +icapParseStatusLine(const char *sline, int slinesize, int *version_major, + int *version_minor, const char **str_status) +{ + char *sp, *stmp, *ep = (char *) sline + slinesize; + int status; + if (slinesize < 14) /*The format of this line is: "ICAP/x.x xxx[ msg....]\r\n" */ + return -1; + + if (strncmp(sline, "ICAP/", 5) != 0) + return -1; + if (sscanf(sline + 5, "%d.%d", version_major, version_minor) != 2) + return -1; + + if (!(sp = memchr(sline, ' ', slinesize))) + return -1; + + while (sp < ep && xisspace(*++sp)); + + if (!xisdigit(*sp) || sp >= ep) + return -1; + + if ((status = strtol(sp, &stmp, 10)) <= 0) + return -1; + sp = stmp; + + while (sp < ep && xisspace(*++sp)); + *str_status = sp; + /*Must add a test for "\r\n" end headers .... */ + return status; +} + + +void +icapSetKeepAlive(IcapStateData * icap, const char *hdrs) +{ + const char *start; + const char *end; + if (0 == icap->flags.keep_alive) + return; + if (0 == icapFindHeader(hdrs, "Connection:", &start, &end)) { + icap->flags.keep_alive = 1; + return; + } + if (icapParseConnectionClose(icap, start, end)) + icap->flags.keep_alive = 0; + else + icap->flags.keep_alive = 1; +} + +/* + * icapParseChunkSize + * + * Returns the offset where the next chunk starts + * return parameter chunk_size; + */ +static int +icapParseChunkSize(const char *buf, int len, int *chunk_size) +{ + int chunkSize = 0; + char c; + size_t start; + size_t end; + size_t nextStart = 0; + debug(81, 3) ("icapParseChunkSize: buf=%p, len=%d\n", buf, len); + do { + start = nextStart; + debug(81, 3) ("icapParseChunkSize: start=%d\n", start); + if (len <= start) { + /* + * end of buffer, so far no lines or only empty lines, + * wait for more data. read chunk size with next buffer. + */ + *chunk_size = 0; + return 0; + } + end = start + icapLineLength(buf + start, len - start); + nextStart = end; + if (end <= start) { + /* + * no line found, need more code here, now we are in + * deep trouble, buffer stops with half a chunk size + * line. For now stop here. + */ + debug(81, 1) ("icapParseChunkSize: WARNING in mid-line, ret 0\n"); + *chunk_size = 0; + return 0; + } + while (start < end) { + if (NULL == strchr(w_space, buf[start])) + break; + start++; + } + while (start < end) { + if (NULL == strchr(w_space, buf[end - 1])) + break; + end--; + } + /* + * if now end <= start we got an empty line. The previous + * chunk data should stop with a CRLF. In case that the + * other end does not follow the specs and sends no CRLF + * or too many empty lines, just continue till we have a + * non-empty line. + */ + } while (end <= start); + debug(81, 3) ("icapParseChunkSize: start=%d, end=%d\n", start, end); + + /* Non-empty line: Parse the chunk size */ + while (start < end) { + c = buf[start++]; + if (c >= 'a' && c <= 'f') { + chunkSize = chunkSize * 16 + c - 'a' + 10; + } else if (c >= 'A' && c <= 'F') { + chunkSize = chunkSize * 16 + c - 'A' + 10; + } else if (c >= '0' && c <= '9') { + chunkSize = chunkSize * 16 + c - '0'; + } else { + if (!(c == ';' || c == ' ' || c == '\t')) { + /*Syntax error: Chunksize expected. */ + *chunk_size = -2; /* we are done */ + return nextStart; + } + /* Next comes a chunk extension */ + break; + } + } + /* + * if we read a zero chunk, we reached the end. Mark this for + * icapPconnTransferDone + */ + *chunk_size = (chunkSize > 0) ? chunkSize : -2; + debug(81, 3) ("icapParseChunkSize: return nextStart=%d\n", nextStart); + return nextStart; +} + +/* + * icapParseChunkedBody + * + * De-chunk an HTTP entity received from the ICAP server. + * The 'store' function pointer is storeAppend() or memBufAppend(). + */ +size_t +icapParseChunkedBody(IcapStateData * icap, STRCB * store, void *store_data) +{ + int bufOffset = 0; + size_t bw = 0; + MemBuf *cb = &icap->chunk_buf; + const char *buf = cb->buf; + int len = cb->size; + + if (icap->chunk_size == -2) { + debug(81, 3) ("zero end chunk reached\n"); + return 0; + } + debug(81, 3) ("%s:%d: chunk_size=%d\n", __FILE__, __LINE__, + icap->chunk_size); + if (icap->chunk_size < 0) { + store(store_data, buf, len); + cb->size = 0; + return (size_t) len; + } + debug(81, 3) ("%s:%d: bufOffset=%d, len=%d\n", __FILE__, __LINE__, + bufOffset, len); + while (bufOffset < len) { + debug(81, 3) ("%s:%d: bufOffset=%d, len=%d\n", __FILE__, __LINE__, + bufOffset, len); + if (icap->chunk_size == 0) { + int x; + x = icapParseChunkSize(buf + bufOffset, + len - bufOffset, &icap->chunk_size); + if (x < 1) { + /* didn't find a valid chunk spec */ + break; + } + bufOffset += x; + debug(81, 3) ("got chunksize %d, new offset %d\n", + icap->chunk_size, bufOffset); + if (icap->chunk_size == -2) { + debug(81, 3) ("zero end chunk reached\n"); + break; + } + } + debug(81, 3) ("%s:%d: X\n", __FILE__, __LINE__); + if (icap->chunk_size > 0) { + if (icap->chunk_size >= len - bufOffset) { + store(store_data, buf + bufOffset, len - bufOffset); + bw += (len - bufOffset); + icap->chunk_size -= (len - bufOffset); + bufOffset = len; + } else { + store(store_data, buf + bufOffset, icap->chunk_size); + bufOffset += icap->chunk_size; + bw += icap->chunk_size; + icap->chunk_size = 0; + } + } + } + if (0 == bufOffset) { + (void) 0; + } else if (bufOffset == cb->size) { + cb->size = 0; + } else { + assert(bufOffset <= cb->size); + xmemmove(cb->buf, cb->buf + bufOffset, cb->size - bufOffset); + cb->size -= bufOffset; + } + return bw; +} + +/* + * icapAddAuthUserHeader + * + * Builds and adds the X-Authenticated-User header to an ICAP request headers. + */ +void +icapAddAuthUserHeader(MemBuf * mb, auth_user_request_t * auth_user_request) +{ + char *user = authenticateUserRequestUsername(auth_user_request); + char *authuser; + size_t len, userlen, schemelen, userofslen; + char *userofs; + + if (user == NULL) { + debug(81, 5) ("icapAddAuthUserHeader: NULL username\n"); + return; + } + userlen = strlen(user); + schemelen = strlen(Config.icapcfg.auth_scheme); + len = userlen + schemelen + 1; + authuser = xcalloc(len, 1); + + if ((userofs = strstr(Config.icapcfg.auth_scheme, "%u")) == NULL) { + /* simply add user at end of string */ + snprintf(authuser, len, "%s%s", Config.icapcfg.auth_scheme, user); + } else { + userofslen = userofs - Config.icapcfg.auth_scheme; + xmemcpy(authuser, Config.icapcfg.auth_scheme, userofslen); + xmemcpy(authuser + userofslen, user, userlen); + xmemcpy(authuser + userofslen + userlen, + userofs + 2, schemelen - (userofslen + 2) + 1); + } + + memBufPrintf(mb, "X-Authenticated-User: %s\r\n", base64_encode(authuser)); + xfree(authuser); +} Index: src/icap_opt.c =================================================================== RCS file: src/icap_opt.c diff -N src/icap_opt.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/icap_opt.c 22 Nov 2005 22:41:48 -0000 1.1.2.17 @@ -0,0 +1,519 @@ + +/* + * $Id$ + * + * DEBUG: section 81 Internet Content Adaptation Protocol (ICAP) Client OPTIONS + * AUTHOR: Ralf Horstmann + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + */ + +#include "squid.h" + +/*************************************************************/ + +/* + * network related functions for OPTIONS request + */ +static void icapOptStart(void *data); +static void icapOptTimeout(int fd, void *data); +static void icapOptConnectDone(int server_fd, int status, void *data); +static void icapOptWriteComplete(int fd, char *bufnotused, size_t size, int errflag, void *data); +static void icapOptReadReply(int fd, void *data); + +/* + * reply parsing functions + */ +static int icapOptParseReply(icap_service * s, IcapOptData * i); +static void icapOptParseEntry(icap_service * s, const char *blk_start, const char *blk_end); +static int icapIsolateLine(const char **parse_start, const char **blk_start, const char **blk_end); + +/* + * helper functions + */ +static void icapOptDataInit(IcapOptData * i); +static void icapOptDataFree(IcapOptData * i); + +/*************************************************************/ + +#define TIMEOUT 10 + +void +icapOptInit() +{ + icap_service *s; + + /* iterate over configured services */ + s = Config.icapcfg.service_head; + while (s) { + eventAdd("icapOptStart", icapOptStart, s, 5.0, 1); + s = s->next; + } +} + +void +icapOptShutdown() +{ + icap_service *s; + + s = Config.icapcfg.service_head; + while (s) { + if (eventFind(icapOptStart, s)) { + eventDelete(icapOptStart, s); + } + s = s->next; + } +} + +/* + * mark a service as unreachable + */ +void +icapOptSetUnreachable(icap_service * s) +{ + s->unreachable = 1; + debug(81, 5) ("icapOptSetUnreachable: got called for %s\n", s->uri); + /* + * if there is an options request scheduled, delete it and add + * it again to reset the time to the default check_interval. + */ + if (eventFind(icapOptStart, s)) { + eventDelete(icapOptStart, s); + eventAdd("icapOptStart", icapOptStart, s, Config.icapcfg.check_interval, 1); + } +} + +static void +icapOptStart(void *data) +{ + icap_service *s = data; + int fd; + int ctimeout = TIMEOUT; + const char *host = s->hostname; + unsigned short port = s->port; + debug(81, 3) ("icapOptStart: starting OPTIONS request for %s (%s)\n", s->name, s->uri); + fd = comm_open(SOCK_STREAM, + 0, + getOutgoingAddr(NULL), + 0, + COMM_NONBLOCKING, + "ICAP OPTIONS connection"); + if (fd < 0) { + debug(81, 4) ("icapConnectStart: %s\n", xstrerror()); + eventAdd("icapOptStart", icapOptStart, s, Config.icapcfg.check_interval, 1); + return; + } + assert(s->opt == NULL); /* if not null, another options request might be running, which should not happen */ + s->opt = memAllocate(MEM_ICAP_OPT_DATA); + icapOptDataInit(s->opt); + cbdataLock(s); + commSetTimeout(fd, ctimeout, icapOptTimeout, s); + commConnectStart(fd, host, port, icapOptConnectDone, s); +} + +static void +icapOptTimeout(int fd, void *data) +{ + icap_service *s = data; + IcapOptData *i = s->opt; + int valid; + + debug(81, 4) ("icapOptConnectTimeout: fd=%d, service=%s\n", fd, s->uri); + + comm_close(fd); + valid = cbdataValid(s); + cbdataUnlock(s); + if (!valid) { + icapOptDataFree(i); + s->opt = NULL; + return; + } + /* try again later */ + icapOptDataFree(i); + s->opt = NULL; + s->unreachable = 1; + debug(81, 3) ("icapOptConnectTimeout: unreachable=1, service=%s\n", s->uri); + eventAdd("icapOptStart", icapOptStart, s, Config.icapcfg.check_interval, 1); + +} + +static void +icapOptConnectDone(int server_fd, int status, void *data) +{ + icap_service *s = data; + IcapOptData *i = s->opt; + MemBuf request; + int valid; + + valid = cbdataValid(s); + cbdataUnlock(s); + if (!valid) { + comm_close(server_fd); + icapOptDataFree(i); + s->opt = NULL; + return; + } + if (status != COMM_OK) { + debug(81, 3) ("icapOptConnectDone: unreachable=1, service=%s\n", s->uri); + comm_close(server_fd); + icapOptDataFree(i); + s->opt = NULL; + s->unreachable = 1; + eventAdd("icapOptStart", icapOptStart, s, Config.icapcfg.check_interval, 1); + return; + } + debug(81, 3) ("icapOptConnectDone: Connection ok. Sending Options request for %s\n", s->name); + memBufDefInit(&request); + memBufPrintf(&request, "OPTIONS %s ICAP/1.0\r\n", s->uri); + memBufPrintf(&request, "Host: %s\r\n", s->hostname); + memBufPrintf(&request, "Connection: close\r\n"); + memBufPrintf(&request, "User-Agent: ICAP-Client-Squid/1.2\r\n"); + memBufPrintf(&request, "\r\n"); + cbdataLock(s); + commSetTimeout(server_fd, TIMEOUT, icapOptTimeout, s); + comm_write_mbuf(server_fd, request, icapOptWriteComplete, s); +} + +static void +icapOptWriteComplete(int fd, char *bufnotused, size_t size, int errflag, void *data) +{ + icap_service *s = data; + IcapOptData *i = s->opt; + int valid; + + valid = cbdataValid(s); + cbdataUnlock(s); + if (!valid) { + comm_close(fd); + icapOptDataFree(i); + s->opt = NULL; + return; + } + debug(81, 5) ("icapOptWriteComplete: FD %d: size %d: errflag %d.\n", + fd, size, errflag); + if (size > 0) { + fd_bytes(fd, size, FD_WRITE); + kb_incr(&statCounter.icap.all.kbytes_out, size); + } + if (errflag) { + /* cancel this for now */ + debug(81, 3) ("icapOptWriteComplete: unreachable=1, service=%s\n", s->uri); + icapOptDataFree(i); + s->opt = NULL; + s->unreachable = 1; + eventAdd("icapOptStart", icapOptStart, s, Config.icapcfg.check_interval, 1); + comm_close(fd); + return; + } + cbdataLock(s); + commSetSelect(fd, COMM_SELECT_READ, icapOptReadReply, s, 0); +} + +static void +icapOptReadReply(int fd, void *data) +{ + icap_service *s = data; + IcapOptData *i = s->opt; + int size; + int len = i->size - i->offset - 1; + int valid; + + valid = cbdataValid(s); + cbdataUnlock(s); + if (!valid) { + comm_close(fd); + icapOptDataFree(i); + s->opt = NULL; + return; + } + if (len == 0) { + /* Grow the request memory area to accomodate for a large request */ + printf("PANIC: not enough memory\n"); +#if 0 + i->buf = memReallocBuf(i->buf, i->size * 2, &i->size); + debug(81, 2) ("icapoptReadReply: growing reply buffer: offset=%ld size=%ld\n", + (long) i->offset, (long) i->size); + len = i->size - i->offset - 1; +#endif + } + size = FD_READ_METHOD(fd, i->buf + i->offset, len); + i->offset += size; + debug(81, 3) ("icapOptReadReply: Got %d bytes of data\n", size); + if (size > 0) { + /* do some statistics */ + fd_bytes(fd, size, FD_READ); + kb_incr(&statCounter.icap.all.kbytes_in, size); + + /* + * some icap servers seem to ignore the "Connection: close" header. so + * after getting the complete option reply we close the connection + * ourself. + */ + if ((i->headlen = headersEnd(i->buf, i->offset))) { + debug(81, 3) ("icapOptReadReply: EndOfResponse\n"); + size = 0; + } + } + if (size < 0) { + debug(81, 3) ("icapOptReadReply: FD %d: read failure: %s.\n", fd, xstrerror()); + debug(81, 3) ("icapOptReadReply: unreachable=1, service=%s.\n", s->uri); + s->unreachable = 1; + icapOptDataFree(i); + s->opt = NULL; + eventAdd("icapOptStart", icapOptStart, s, Config.icapcfg.check_interval, 1); + comm_close(fd); + } else if (size == 0) { + /* no more data, now we can parse the reply */ + debug(81, 3) ("icapOptReadReply: FD %d: connection closed\n", fd); + i->buf[i->offset] = '\0'; /* for string functions */ + debug(81, 3) ("icapOptReadReply: unreachable=0, service=%s\n", s->uri); + + if (!icapOptParseReply(s, i)) { + debug(81, 3) ("icapOptReadReply: OPTIONS request not successful. scheduling again in %d seconds\n", Config.icapcfg.check_interval); + s->unreachable = 1; + } else + s->unreachable = 0; + + if (s->options_ttl <= 0) + s->options_ttl = Config.icapcfg.check_interval; + eventAdd("icapOptStart", icapOptStart, s, s->options_ttl, 1); + + icapOptDataFree(i); + s->opt = NULL; + comm_close(fd); + } else { + /* data received */ + /* commSetSelect(fd, Type, handler, client_data, timeout) */ + cbdataLock(s); + commSetSelect(fd, COMM_SELECT_READ, icapOptReadReply, data, 0); + } +} + +static int +icapIsolateLine(const char **parse_start, const char **blk_start, const char **blk_end) +{ + int slen = strcspn(*parse_start, "\r\n"); + + if (!(*parse_start)[slen]) /* no crlf */ + return 0; + + if (slen == 0) /* empty line */ + return 0; + + *blk_start = *parse_start; + *blk_end = *blk_start + slen; + + /* set it to the beginning of next line */ + *parse_start = *blk_end; + while (**parse_start == '\r') /* CR */ + (*parse_start)++; + if (**parse_start == '\n') /* LF */ + (*parse_start)++; + return 1; +} + +/* process a single header entry between blk_start and blk_end */ +static void +icapOptParseEntry(icap_service * s, const char *blk_start, const char *blk_end) +{ + const char *name_end = strchr(blk_start, ':'); + const int name_len = name_end ? name_end - blk_start : 0; + const char *value_start = blk_start + name_len + 1; /* skip ':' */ + int value_len; + int new; + + if (!name_len || name_end > blk_end) { + debug(81, 5) ("icapOptParseEntry: strange header. skipping\n"); + return; + } + if (name_len > 65536) { + debug(81, 5) ("icapOptParseEntry: unusual long header item. skipping.\n"); + return; + } + while (xisspace(*value_start) && value_start < blk_end) { + value_start++; + } + if (value_start >= blk_end) { + debug(81, 5) ("icapOptParseEntry: no value found\n"); + return; + } + value_len = blk_end - value_start; + + + /* extract information */ + if (!strncasecmp("Allow", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Allow\n"); + if (!strncmp("204", value_start, 3)) { + s->flags.allow_204 = 1; + } else { + debug(81, 3) ("icapOptParseEntry: Allow value unknown"); + } + } else if (!strncasecmp("Connection", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Connection\n"); + } else if (!strncasecmp("Encapsulated", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Encapsulated\n"); + } else if (!strncasecmp("ISTAG", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found ISTAG\n"); + stringClean(&s->istag); + stringLimitInit(&s->istag, value_start, value_len); + } else if (!strncasecmp("Max-Connections", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Max-Connections\n"); + errno = 0; + new = strtol(value_start, NULL, 10); + if (errno) { + debug(81, 5) ("icapOptParseEntry: Max-Connections: could not parse value\n"); + } else { + debug(81, 5) ("icapOptParseEntry: Max-Connections: new value=%d\n", new); + s->max_connections = new; + } + } else if (!strncasecmp("Methods", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Methods\n"); + } else if (!strncasecmp("Options-TTL", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Options-TTL\n"); + errno = 0; + new = strtol(value_start, NULL, 10); + if (errno) { + debug(81, 5) ("icapOptParseEntry: Options-TTL: could not parse value\n"); + } else { + debug(81, 5) ("icapOptParseEntry: Options-TTL: new value=%d\n", new); + s->options_ttl = new; + } + } else if (!strncasecmp("Preview", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Preview\n"); + errno = 0; + new = strtol(value_start, NULL, 10); + if (errno) { + debug(81, 5) ("icapOptParseEntry: Preview: could not parse value\n"); + } else { + debug(81, 5) ("icapOptParseEntry: Preview: new value=%d\n", new); + s->preview = new; + } + } else if (!strncasecmp("Service", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Service\n"); + } else if (!strncasecmp("Service-ID", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Service-ID\n"); + } else if (!strncasecmp("Transfer-Preview", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Transfer-Preview\n"); + stringClean(&s->transfer_preview); + stringLimitInit(&s->transfer_preview, value_start, value_len); + } else if (!strncasecmp("Transfer-Ignore", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Transfer-Ignore\n"); + stringClean(&s->transfer_ignore); + stringLimitInit(&s->transfer_ignore, value_start, value_len); + } else if (!strncasecmp("Transfer-Complete", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found Transfer-Complete\n"); + stringClean(&s->transfer_complete); + stringLimitInit(&s->transfer_complete, value_start, value_len); + } else if (!strncasecmp("X-Include", blk_start, name_len)) { + debug(81, 5) ("icapOptParseEntry: found X-Include\n"); + if (strstr(value_start, "X-Client-IP")) { + debug(81, 5) ("icapOptParseEntry: X-Include: found X-Client-IP\n"); + s->flags.need_x_client_ip = 1; + } + if (strstr(value_start, "X-Authenticated-User")) { + debug(81, 5) ("icapOptParseEntry: X-Include: found X-Authenticated-User\n"); + s->flags.need_x_authenticated_user = 1; + } + } else { + debug(81, 5) ("icapOptParseEntry: unknown options header\n"); + } +} + +/* parse OPTIONS reply */ +static int +icapOptParseReply(icap_service * s, IcapOptData * i) +{ + int version_major, version_minor; + const char *str_status; + int status; + const char *buf = i->buf; + const char *parse_start; + const char *head_end; + const char *blk_start; + const char *blk_end; + + if ((status = + icapParseStatusLine(i->buf, i->offset, + &version_major, &version_minor, &str_status)) < 0) { + debug(81, 2) ("icapOptParseReply: bad status line <%s>\n", i->buf); + return 0; + } + debug(81, 3) ("icapOptParseReply: got reply: \n", version_major, version_minor, status, str_status); + + if (status != 200) { + debug(81, 3) ("icapOptParseReply: status = %d != 200\n", status); + return 0; + } + parse_start = buf; + if (i->headlen == 0) + i->headlen = headersEnd(parse_start, s->opt->offset); + + if (!i->headlen) { + debug(81, 2) ("icapOptParseReply: end of headers could not be found\n"); + return 0; + } + head_end = parse_start + i->headlen - 1; + while (*(head_end - 1) == '\r') + head_end--; + assert(*(head_end - 1) == '\n'); + if (*head_end != '\r' && *head_end != '\n') + return 0; /* failure */ + + /* skip status line */ + if (!icapIsolateLine(&parse_start, &blk_start, &blk_end)) { + debug(81, 3) ("icapOptParseReply: failure in isolating status line\n"); + return 0; + + } + /* now we might start real parsing */ + while (icapIsolateLine(&parse_start, &blk_start, &blk_end)) { + if (blk_end > head_end || blk_start > head_end || blk_start >= blk_end) { + debug(81, 3) ("icapOptParseReply: header limit exceeded. finished.\n"); + break; + } + icapOptParseEntry(s, blk_start, blk_end); + } + return 1; +} + +static void +icapOptDataInit(IcapOptData * i) +{ + i->buf = memAllocBuf(HTTP_REPLY_BUF_SZ, &i->size); + i->offset = 0; + i->headlen = 0; +} + +static void +icapOptDataFree(IcapOptData * i) +{ + if (i) { + memFreeBuf(i->size, i->buf); + memFree(i, MEM_ICAP_OPT_DATA); + } +} Index: src/icap_reqmod.c =================================================================== RCS file: src/icap_reqmod.c diff -N src/icap_reqmod.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/icap_reqmod.c 6 Dec 2005 21:53:44 -0000 1.1.2.58 @@ -0,0 +1,976 @@ + +/* + * $Id$ + * + * DEBUG: section 81 Internet Content Adaptation Protocol (ICAP) Client + * AUTHOR: Geetha Manjunath, Hewlett Packard Company + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + */ + +#include "squid.h" + +#define ICAP_PROXY_KEEP_ALIVE 0 + +/* + * These once-static functions are required to be global for ICAP + */ + +PF clientReadRequest; +PF connStateFree; +int clientReadDefer(int fd, void *data); +int clientCheckContentLength(request_t * r); +void clientProcessRequest(clientHttpRequest *); +int clientCachable(clientHttpRequest *); +int clientHierarchical(clientHttpRequest *); +void clientReadBody(request_t * request, char *buf, size_t size, + CBCB * callback, void *cbdata); +static void icapReqModPassHttpBody(IcapStateData * icap, char *buf, size_t size, + CBCB * callback, void *cbdata); + +static PF icapReqModReadHttpHdrs; +static PF icapReqModReadHttpBody; +static CWCB icapReqModSendBodyChunk; +static CBCB icapReqModBodyHandler; +static BODY_HANDLER icapReqModBodyReader; +static STRCB icapReqModMemBufAppend; + +#define EXPECTED_ICAP_HEADER_LEN 256 +static const char *crlf = "\r\n"; + +/* + * icapExpectedHttpReqHdrSize + * + * calculate the size of the HTTP headers that we expect + * to read from the ICAP server. + */ +static int +icapExpectedHttpReqHdrSize(IcapStateData * icap) +{ + if (icap->enc.req_body > -1 && icap->enc.req_hdr > -1) + return (icap->enc.req_body - icap->enc.req_hdr); + if (icap->enc.null_body > -1) + return icap->enc.null_body; + fatal("icapExpectedHttpReqHdrSize: unexpected case"); + return 0; +} + +/* + * icapReqModCreateClientState + * + * Creates fake client_side data structures so we can use + * that module to read/parse the HTTP request that we read + * from the ICAP server. + */ +static clientHttpRequest * +icapReqModCreateClientState(IcapStateData * icap, request_t * request) +{ + clientHttpRequest *http; + if (!cbdataValid(icap->reqmod.client_cookie)) { + debug(81, 3) ("Whups, client cookie invalid\n"); + icap->reqmod.client_fd = -1; + return NULL; + } + http = cbdataAlloc(clientHttpRequest); + /* + * use our own urlCanonicalClean here, because urlCanonicalClean + * may strip everything after a question-mark. As http->uri + * is used when doing a request to a parent proxy, we need the full + * url here. + */ + http->uri = xstrdup(urlCanonical(icap->request)); + http->log_uri = xstrndup(http->uri, MAX_URL); + http->range_iter.boundary = StringNull; + http->request = requestLink(request ? request : icap->request); + http->flags.did_icap_reqmod = 1; + http->start = icap->reqmod.start; +#if ICAP_PROXY_KEEP_ALIVE + /* + * Here it is possible becouse we are using as client_cookie the original http->conn + * if we will keep this code we must declare an icap->conn field........ + * Will work if pipeline_prefetch is not enabled + * We are using a dummy ConnStateData structure, just to free + * old clientHttpRequest :-( + * OK,all this code is a hack and possibly must not exists in cvs ...... + */ + + http->conn = icap->reqmod.client_cookie; + assert(http->conn->chr->next == NULL); + { + ConnStateData *dummyconn; + dummyconn = cbdataAlloc(ConnStateData); + dummyconn->fd = icap->reqmod.client_fd; + dummyconn->chr = http->conn->chr; + dummyconn->chr->conn = dummyconn; + comm_add_close_handler(dummyconn->fd, connStateFree, dummyconn); + } + + http->conn->chr = http; + +#else + http->conn = cbdataAlloc(ConnStateData); + http->conn->fd = icap->reqmod.client_fd; + http->conn->in.size = 0; + http->conn->in.buf = NULL; + http->conn->log_addr = icap->reqmod.log_addr; + http->conn->chr = http; + comm_add_close_handler(http->conn->fd, connStateFree, http->conn); +#endif + http->icap_reqmod = NULL; + return http; +} + +/* + * icapReqModInterpretHttpRequest + * + * Interpret an HTTP request that we read from the ICAP server. + * Create some "fake" clientHttpRequest and ConnStateData structures + * so we can pass this new request off to the routines in + * client_side.c. + */ +static void +icapReqModInterpretHttpRequest(IcapStateData * icap, request_t * request) +{ + clientHttpRequest *http = icapReqModCreateClientState(icap, request); + if (NULL == http) + return; + /* + * bits from clientReadRequest + */ + request->content_length = httpHeaderGetSize(&request->header, + HDR_CONTENT_LENGTH); + if (!urlCheckRequest(request) || + httpHeaderHas(&request->header, HDR_TRANSFER_ENCODING)) { + ErrorState *err; + err = errorCon(ERR_UNSUP_REQ, HTTP_NOT_IMPLEMENTED); + err->request = requestLink(request); + request->flags.proxy_keepalive = 0; + http->entry = + clientCreateStoreEntry(http, request->method, null_request_flags); + errorAppendEntry(http->entry, err); + return; + } + if (!clientCheckContentLength(request)) { + ErrorState *err; + err = errorCon(ERR_INVALID_REQ, HTTP_LENGTH_REQUIRED); + err->request = requestLink(request); + http->entry = + clientCreateStoreEntry(http, request->method, null_request_flags); + errorAppendEntry(http->entry, err); + return; + } + /* Do we expect a request-body? */ + if (request->content_length > 0) { + debug(81, 5) ("handing request bodies in ICAP REQMOD\n"); + if (request->body_reader_data) + cbdataUnlock(request->body_reader_data); + request->body_reader = icapReqModBodyReader; + request->body_reader_data = icap; /* XXX cbdataLock? */ + cbdataLock(icap); /*Yes sure ..... */ + memBufDefInit(&icap->reqmod.http_entity.buf); + } + if (clientCachable(http)) + request->flags.cachable = 1; + if (clientHierarchical(http)) + request->flags.hierarchical = 1; + clientProcessRequest(http); +} + +/* + * icapReqModParseHttpError + * + * Handle an error when parsing the new HTTP request we read + * from the ICAP server. + */ +static void +icapReqModParseHttpError(IcapStateData * icap, const char *reason) +{ + debug(81, 1) ("icapReqModParseHttpError: %s\n", reason); +} + +/* + * icapEntryError + * + * A wrapper for errorCon() and errorAppendEntry(). + */ +static void +icapEntryError(IcapStateData * icap, err_type et, http_status hs, int xerrno) +{ + ErrorState *err; + clientHttpRequest *http = icapReqModCreateClientState(icap, NULL); + if (NULL == http) + return; + http->entry = clientCreateStoreEntry(http, + icap->request->method, null_request_flags); + err = errorCon(et, hs); + err->xerrno = xerrno; + err->request = requestLink(icap->request); + errorAppendEntry(http->entry, err); +} + +/* + * icapReqModParseHttpRequest + * + * Parse the HTTP request that we read from the ICAP server. + * Creates and fills in the request_t structure. + */ +static void +icapReqModParseHttpRequest(IcapStateData * icap) +{ + char *mstr; + char *uri; + char *inbuf; + char *t; + char *token; + char *headers; + method_t method; + request_t *request; + http_version_t http_ver; + int reqlen = icap->reqmod.hdr_buf.size; + int hdrlen; + + /* + * Lazy, make a copy of the buf so I can chop it up with strtok() + */ + inbuf = xcalloc(reqlen + 1, 1); + memcpy(inbuf, icap->reqmod.hdr_buf.buf, reqlen); + + if ((mstr = strtok(inbuf, "\t ")) == NULL) { + debug(81, 1) ("icapReqModParseHttpRequest: Can't get request method\n"); + icapReqModParseHttpError(icap, "error:invalid-request-method"); + xfree(inbuf); + return; + } + method = urlParseMethod(mstr); + if (method == METHOD_NONE) { + debug(81, 1) ("icapReqModParseHttpRequest: Unsupported method '%s'\n", + mstr); + icapReqModParseHttpError(icap, "error:unsupported-request-method"); + xfree(inbuf); + return; + } + /* look for URL+HTTP/x.x */ + if ((uri = strtok(NULL, "\n")) == NULL) { + debug(81, 1) ("icapReqModParseHttpRequest: Missing URI\n"); + icapReqModParseHttpError(icap, "error:missing-url"); + xfree(inbuf); + return; + } + while (xisspace(*uri)) + uri++; + t = uri + strlen(uri); + assert(*t == '\0'); + token = NULL; + while (t > uri) { + t--; + if (xisspace(*t) && !strncmp(t + 1, "HTTP/", 5)) { + token = t + 1; + break; + } + } + while (t > uri && xisspace(*t)) + *(t--) = '\0'; + debug(81, 5) ("icapReqModParseHttpRequest: URI is '%s'\n", uri); + if (token == NULL) { + debug(81, 3) ("icapReqModParseHttpRequest: Missing HTTP identifier\n"); + icapReqModParseHttpError(icap, "error:missing-http-ident"); + xfree(inbuf); + return; + } + if (sscanf(token + 5, "%d.%d", &http_ver.major, &http_ver.minor) != 2) { + debug(81, 3) ("icapReqModParseHttpRequest: Invalid HTTP identifier.\n"); + icapReqModParseHttpError(icap, "error:invalid-http-ident"); + xfree(inbuf); + return; + } + debug(81, 6) ("icapReqModParseHttpRequest: Client HTTP version %d.%d.\n", + http_ver.major, http_ver.minor); + + headers = strtok(NULL, null_string); + hdrlen = inbuf + reqlen - headers; + + if ((request = urlParse(method, uri)) == NULL) { + debug(81, 3) ("Invalid URL: %s at %s:%d\n", uri, __FILE__, __LINE__); + icapEntryError(icap, ERR_INVALID_URL, HTTP_BAD_REQUEST, 0); + xfree(inbuf); + return; + } + /* compile headers */ + if (!httpHeaderParse(&request->header, headers, headers + hdrlen)) { + debug(81, 3) ("Failed to parse HTTP headers for: %s at %s:%d", + uri, __FILE__, __LINE__); + icapEntryError(icap, ERR_INVALID_REQ, HTTP_BAD_REQUEST, 0); + xfree(inbuf); + return; + } + debug(81, + 3) + ("icapReqModParseHttpRequest: successfully parsed the HTTP request\n"); + request->http_ver = http_ver; + request->client_addr = icap->request->client_addr; + request->my_addr = icap->request->my_addr; + request->my_port = icap->request->my_port; + request->class = icap->request->class; + if (icap->request->auth_user_request != NULL) { + /* Copy authentification info in new request */ + request->auth_user_request = icap->request->auth_user_request; + authenticateAuthUserRequestLock(request->auth_user_request); + } +#if ICAP_PROXY_KEEP_ALIVE + /* + * Copy the proxy_keepalive flag from the original request + */ + request->flags.proxy_keepalive = icap->request->flags.proxy_keepalive; + /* + * If proxy_keepalive was set for the original request, make + * sure that the adapated request also has the necessary headers + * for keepalive + */ + if (request->flags.proxy_keepalive) { + if (!httpMsgIsPersistent(http_ver, &request->header)) + request->flags.proxy_keepalive = 0; + } +#endif + icapReqModInterpretHttpRequest(icap, request); + xfree(inbuf); +} + +/* + * icapReqModHandoffRespMod + * + * Handles the case where a REQMOD request results in an HTTP REPLY + * (instead of an ICAP REPLY that contains a new HTTP REQUEST). We + * prepare the IcapStateData for passing off to the icap_reqmod + * code, where we have functions for reading HTTP replies in ICAP + * messages. + */ +static void +icapReqModHandoffRespMod(IcapStateData * icap) +{ + extern PF icapReadReply; + clientHttpRequest *http = icapReqModCreateClientState(icap, NULL); + if (NULL == http) + return; + assert(icap->request); + + http->entry = clientCreateStoreEntry(http, + icap->request->method, icap->request->flags); + icap->respmod.entry = http->entry; + storeLockObject(icap->respmod.entry); + + /* icap->http_flags = ? */ + memBufDefInit(&icap->respmod.buffer); + memBufDefInit(&icap->chunk_buf); + assert(icap->current_service); + icapReadReply(icap->icap_fd, icap); +} + +/* + * icapReqModKeepAliveOrClose + * + * Called when we are done reading from the ICAP server. + * Either close the connection or keep it open for a future + * transaction. + */ +static void +icapReqModKeepAliveOrClose(IcapStateData * icap) +{ + int fd = icap->icap_fd; + debug(81, 3) ("%s:%d FD %d\n", __FILE__, __LINE__, fd); + if (fd < 0) + return; + if (!icap->flags.keep_alive) { + debug(81, 3) ("%s:%d keep_alive not set, closing\n", __FILE__, + __LINE__); + comm_close(fd); + return; + } + if (icap->request->content_length < 0) { + /* no message body */ + debug(81, 3) ("%s:%d no message body\n", __FILE__, __LINE__); + if (1 != icap->reqmod.hdr_state) { + /* didn't get to end of HTTP headers */ + debug(81, 3) ("%s:%d didnt find end of headers, closing\n", + __FILE__, __LINE__); + comm_close(fd); + return; + } + } else if (icap->reqmod.http_entity.bytes_read != + icap->request->content_length) { + debug(81, 3) ("%s:%d bytes_read (%" PRINTF_OFF_T ") != content_length (%" PRINTF_OFF_T ")\n", + __FILE__, __LINE__, icap->reqmod.http_entity.bytes_read, + icap->request->content_length); + /* an error */ + comm_close(fd); + return; + } + debug(81, 3) ("%s:%d looks good, keeping alive\n", __FILE__, __LINE__); + commSetDefer(fd, NULL, NULL); + commSetTimeout(fd, -1, NULL, NULL); + commSetSelect(fd, COMM_SELECT_READ, NULL, NULL, 0); + comm_remove_close_handler(fd, icapStateFree, icap); + pconnPush(fd, icap->current_service->hostname, icap->current_service->port); + icap->icap_fd = -1; + icapStateFree(-1, icap); +} + +/* + * icapReqModReadHttpHdrs + * + * Read the HTTP reply from the ICAP server. Uses the values + * from the ICAP Encapsulation header to know how many bytes + * to read. + */ +static void +icapReqModReadHttpHdrs(int fd, void *data) +{ + IcapStateData *icap = data; + LOCAL_ARRAY(char, tmpbuf, SQUID_TCP_SO_RCVBUF); + int rl; + debug(81, 3) ("icapReqModReadHttpHdrs:\n"); + assert(fd == icap->icap_fd); + assert(icap->enc.req_hdr == 0); + if (0 == icap->reqmod.hdr_state) { + int expect = icapExpectedHttpReqHdrSize(icap); + int so_far = icap->http_header_bytes_read_so_far; + int needed = expect - so_far; + debug(81, 3) ("expect=%d\n", expect); + debug(81, 3) ("so_far=%d\n", so_far); + debug(81, 3) ("needed=%d\n", needed); + assert(needed >= 0); + if (0 == expect) { + fatalf("unexpected condition in %s:%d", __FILE__, __LINE__); + } + rl = FD_READ_METHOD(fd, tmpbuf, needed); + debug(81, 3) ("icapReqModReadHttpHdrs: read %d bytes\n", rl); + if (rl < 0) { + fatalf("need to handle read error at %s:%d", __FILE__, __LINE__); + } + fd_bytes(fd, rl, FD_READ); + kb_incr(&statCounter.icap.all.kbytes_in, rl); + memBufAppend(&icap->reqmod.hdr_buf, tmpbuf, rl); + icap->http_header_bytes_read_so_far += rl; + if (rl != needed) { + /* still more header data to read */ + commSetSelect(fd, COMM_SELECT_READ, icapReqModReadHttpHdrs, icap, + 0); + return; + } + icap->reqmod.hdr_state = 1; + } + assert(1 == icap->reqmod.hdr_state); + debug(81, 3) ("icapReqModReadHttpHdrs: read the entire request headers\n"); + icapReqModParseHttpRequest(icap); + if (-1 == icap->reqmod.client_fd) { + /* we detected that the original client_side went away */ + icapReqModKeepAliveOrClose(icap); + } else if (icap->enc.req_body > -1) { + icap->chunk_size = 0; + memBufDefInit(&icap->chunk_buf); + commSetSelect(fd, COMM_SELECT_READ, icapReqModReadHttpBody, icap, 0); + } else { + icapReqModKeepAliveOrClose(icap); + } +} + + +/* + * icapReqModReadIcapPart + * + * Read the ICAP reply header. + */ +static void +icapReqModReadIcapPart(int fd, void *data) +{ + IcapStateData *icap = data; + int version_major, version_minor; + const char *str_status; + int x; + const char *start; + const char *end; + int status; + int isIcap = 0; + int directResponse = 0; + + debug(81, 5) ("icapReqModReadIcapPart: FD %d httpState = %p\n", fd, data); + statCounter.syscalls.sock.reads++; + + x = icapReadHeader(fd, icap, &isIcap); + if (x < 0) { + /* Did not find a proper ICAP response */ + debug(81, 3) ("ICAP : Error path!\n"); + icapEntryError(icap, ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, + errno); + comm_close(fd); + return; + } + if (x == 0) { + /* + * Waiting for more headers. Schedule new read hander, but + * don't reset timeout. + */ + commSetSelect(fd, COMM_SELECT_READ, icapReqModReadIcapPart, icap, 0); + return; + } + /* + * Parse the ICAP header + */ + assert(icap->icap_hdr.size); + debug(81, 3) ("Read icap header : <%s>\n", icap->icap_hdr.buf); + if ((status = + icapParseStatusLine(icap->icap_hdr.buf, icap->icap_hdr.size, + &version_major, &version_minor, &str_status)) < 0) { + debug(81, 1) ("BAD ICAP status line <%s>\n", icap->icap_hdr.buf); + /* is this correct in case of ICAP protocol error? */ + icapEntryError(icap, ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, + errno); + comm_close(fd); + return; + }; + if (200 != status && 201 != status) { + debug(81, 1) ("Unsupported status '%d' from ICAP server\n", status); + icapEntryError(icap, ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, + errno); + comm_close(fd); + return; + } + icapSetKeepAlive(icap, icap->icap_hdr.buf); + if (icapFindHeader(icap->icap_hdr.buf, "Encapsulated:", &start, &end)) { + icapParseEncapsulated(icap, start, end); + } else { + debug(81, + 1) + ("WARNING: icapReqModReadIcapPart() did not find 'Encapsulated' header\n"); + } + if (icap->enc.res_hdr > -1) + directResponse = 1; + else if (icap->enc.res_body > -1) + directResponse = 1; + else + directResponse = 0; + debug(81, 3) ("icapReqModReadIcapPart: directResponse=%d\n", + directResponse); + + /* Check whether it is a direct reply - if so over to http part */ + if (directResponse) { + debug(81, + 3) + ("icapReqModReadIcapPart: FD %d, processing HTTP response for REQMOD!\n", + fd); + /* got the reply, no need to come here again */ + icap->flags.wait_for_reply = 0; + icap->flags.got_reply = 1; + icapReqModHandoffRespMod(icap); + return; + } + memBufDefInit(&icap->reqmod.hdr_buf); + commSetSelect(fd, COMM_SELECT_READ, icapReqModReadHttpHdrs, icap, 0); + return; +} + +/* + * icapSendReqModDone + * + * Called after we've sent the ICAP request. Checks for errors + * and installs the handler functions for the next step. + */ +static void +icapSendReqModDone(int fd, char *bufnotused, size_t size, int errflag, + void *data) +{ + IcapStateData *icap = data; + + debug(81, 5) ("icapSendReqModDone: FD %d: size %d: errflag %d.\n", + fd, size, errflag); + if (size > 0) { + fd_bytes(fd, size, FD_WRITE); + kb_incr(&statCounter.icap.all.kbytes_out, size); + } + if (errflag == COMM_ERR_CLOSING) + return; + if (errflag) { + debug(81, 3) ("icapSendReqModDone: unreachable=1, service=%s\n", + icap->current_service->uri); + icapOptSetUnreachable(icap->current_service); + icapEntryError(icap, ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, + errno); + comm_close(fd); + return; + } + /* Schedule read reply. */ + commSetSelect(fd, COMM_SELECT_READ, icapReqModReadIcapPart, icap, 0); + /* + * Set the read timeout here because it hasn't been set yet. + * We only set the read timeout after the request has been + * fully written to the server-side. If we start the timeout + * after connection establishment, then we are likely to hit + * the timeout for POST/PUT requests that have very large + * request bodies. + */ + commSetTimeout(fd, Config.Timeout.read, icapConnectTimeout, icap); +} + + +/* + * icapSendReqMod + * + * Send the ICAP request, including HTTP request, to the ICAP server + * after connection has been established. + */ +static void +icapSendReqMod(int fd, int status, void *data) +{ + MemBuf mb; + MemBuf mb_hdr; + Packer p; + IcapStateData *icap = data; + char *client_addr; + int icap_fd = icap->icap_fd; + icap_service *service; + CWCB *theCallback; + + debug(81, 5) ("icapSendReqMod FD %d, status %d\n", fd, status); + icap->flags.connect_pending = 0; + + if (COMM_OK != status) { + debug(81, 1) ("Could not connect to ICAP server %s:%d: %s\n", + icap->current_service->hostname, + icap->current_service->port, xstrerror()); + debug(81, 3) ("icapSendReqMod: unreachable=1, service=%s\n", + icap->current_service->uri); + icapOptSetUnreachable(icap->current_service); + icapEntryError(icap, ERR_ICAP_FAILURE, HTTP_SERVICE_UNAVAILABLE, errno); + comm_close(fd); + return; + } + fd_table[fd].pconn.uses++; + fd_table[fd].pconn.type = 2; + if (icap->request->content_length > 0) + theCallback = icapReqModSendBodyChunk; + else + theCallback = icapSendReqModDone; + + memBufDefInit(&mb); + memBufDefInit(&mb_hdr); + memBufPrintf(&mb_hdr, "%s %s HTTP/%d.%d\r\n", + RequestMethodStr[icap->request->method], + icap->reqmod.uri, + icap->request->http_ver.major, icap->request->http_ver.minor); + packerToMemInit(&p, &mb_hdr); + httpHeaderPackInto(&icap->request->header, &p); + packerClean(&p); + memBufAppend(&mb_hdr, crlf, 2); + service = icap->current_service; + assert(service); + client_addr = inet_ntoa(icap->request->client_addr); + + memBufPrintf(&mb, "REQMOD %s ICAP/1.0\r\n", service->uri); + memBufPrintf(&mb, "Encapsulated: req-hdr=0"); + /* TODO: Change the offset using 'request' if needed */ + if (icap->request->content_length > 0) + memBufPrintf(&mb, ", req-body=%d", mb_hdr.size); + else + memBufPrintf(&mb, ", null-body=%d", mb_hdr.size); + memBufAppend(&mb, crlf, 2); + if (Config.icapcfg.send_client_ip || service->flags.need_x_client_ip) + memBufPrintf(&mb, "X-Client-IP: %s\r\n", client_addr); + if ((Config.icapcfg.send_auth_user + || service->flags.need_x_authenticated_user) + && (icap->request->auth_user_request != NULL)) + icapAddAuthUserHeader(&mb, icap->request->auth_user_request); + if (service->keep_alive) { + icap->flags.keep_alive = 1; + } else { + icap->flags.keep_alive = 0; + memBufAppend(&mb, "Connection: close\r\n", 19); + } + memBufAppend(&mb, crlf, 2); + memBufAppend(&mb, mb_hdr.buf, mb_hdr.size); + memBufClean(&mb_hdr); + + debug(81, 5) ("icapSendReqMod: FD %d writing {%s}\n", icap->icap_fd, + mb.buf); + comm_write_mbuf(icap_fd, mb, theCallback, icap); +} + +/* + * icapReqModStart + * + * Initiate an ICAP REQMOD transaction. Create and fill in IcapStateData + * structure and request a TCP connection to the server. + */ +IcapStateData * +icapReqModStart(icap_service *service, const char *uri, request_t * request, + int fd, struct timeval start, struct in_addr log_addr, void *cookie) +{ + IcapStateData *icap = NULL; + + debug(81, 3) ("icapReqModStart: type=%d\n", (int) service->type); + + switch (service->type) { + case ICAP_SERVICE_REQMOD_PRECACHE: + break; + default: + fatalf("icapReqModStart: unsupported service type '%s'\n", + icap_service_type_str[service->type]); + break; + } + + if (service->unreachable) { + if (service->bypass) { + debug(81, + 5) ("icapReqModStart: BYPASS because service unreachable: %s\n", + service->uri); + return NULL; + } else { + debug(81, + 5) ("icapReqModStart: ERROR because service unreachable: %s\n", + service->uri); + return (IcapStateData *) - 1; + } + } + icap = icapAllocate(); + if (!icap) { + debug(81, 3) ("icapReqModStart: icapAllocate() failed\n"); + return NULL; + } + icap->current_service = service; + icap->preview_size = service->preview; + icap->reqmod.uri = uri; /* XXX should be xstrdup? */ + icap->reqmod.start = start; + icap->reqmod.log_addr = log_addr; + icap->request = requestLink(request); + icap->reqmod.hdr_state = 0; + icap->reqmod.client_fd = fd; + icap->reqmod.client_cookie = cookie; + cbdataLock(icap->reqmod.client_cookie); + + if (!icapConnect(icap, icapSendReqMod)) + return NULL; + + statCounter.icap.all.requests++; + debug(81, 3) ("icapReqModStart: returning %p\n", icap); + return icap; +} + +/* + * icapReqModSendBodyChunk + * + * A "comm_write" callback. This is called after comm_write() does + * its job to let us know how things went. If there are no errors, + * get another chunk of the body from client_side. + */ +static void +icapReqModSendBodyChunk(int fd, char *bufnotused, size_t size, int errflag, + void *data) +{ + IcapStateData *icap = data; + debug(81, 3) ("icapReqModSendBodyChunk: FD %d wrote %d errflag %d.\n", + fd, (int) size, errflag); + if (errflag == COMM_ERR_CLOSING) + return; + if (errflag) { + icapEntryError(icap, ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, + errno); + comm_close(fd); + return; + } + clientReadBody(icap->request, + memAllocate(MEM_8K_BUF), 8192, icapReqModBodyHandler, icap); +} + +/* + * icapReqModBodyHandler + * + * Called after Squid gets a chunk of the request entity from the + * client side. The body is chunkified and passed to comm_write. + * The comm_write callback depends on whether or not this is the + * last chunk. + */ +static void +icapReqModBodyHandler(char *buf, ssize_t size, void *data) +{ + IcapStateData *icap = data; + MemBuf mb; + CWCB *theCallback = icapReqModSendBodyChunk; + if (size < 0) { + debug(81, 1) ("icapReqModBodyHandler: %s\n", xstrerror()); + memFree8K(buf); + return; + } + memBufDefInit(&mb); + debug(81, 3) ("icapReqModBodyHandler: writing chunk size %d\n", size); + memBufPrintf(&mb, "%x\r\n", size); + if (size) + memBufAppend(&mb, buf, size); + else + theCallback = icapSendReqModDone; + memBufAppend(&mb, crlf, 2); + memFree8K(buf); + comm_write_mbuf(icap->icap_fd, mb, theCallback, icap); +} + +/* + * icapReqModReadHttpBody + * + * The read handler for the client's HTTP connection when reading + * message bodies. Called by comm_select(). + */ +static void +icapReqModReadHttpBody(int fd, void *data) +{ + IcapStateData *icap = data; + int len; + debug(81, 3) ("icapReqModReadHttpBody: FD %d called\n", fd); + len = memBufRead(fd, &icap->chunk_buf); + debug(81, 3) ("icapReqModReadHttpBody: read returns %d\n", len); + if (len < 0) { + debug(81, 3) ("icapReqModReadHttpBody: FD %d %s\n", fd, xstrerror()); + if (!ignoreErrno(errno)) + icap->flags.reqmod_http_entity_eof = 1; + } else if (0 == len) { + debug(81, 3) ("icapReqModReadHttpBody: FD %d EOF\n", fd); + icap->flags.reqmod_http_entity_eof = 1; + } else { + fd_bytes(fd, len, FD_READ); + kb_incr(&statCounter.icap.all.kbytes_in, len); + icap->reqmod.http_entity.bytes_read += + icapParseChunkedBody(icap, + icapReqModMemBufAppend, &icap->reqmod.http_entity.buf); + } + if (icap->reqmod.http_entity.bytes_read >= icap->request->content_length) + icap->flags.reqmod_http_entity_eof = 1; + + if (!icap->flags.reqmod_http_entity_eof) + commSetSelect(fd, COMM_SELECT_READ, icapReqModReadHttpBody, icap, 0); + /* + * Notify the other side if it is waiting for data from us + */ + debug(81, 3) ("%s:%d http_entity.callback=%p\n", __FILE__, __LINE__, + icap->reqmod.http_entity.callback); + debug(81, 3) ("%s:%d http_entity.buf.size=%d\n", __FILE__, __LINE__, + icap->reqmod.http_entity.buf.size); + if (icap->reqmod.http_entity.callback && icap->reqmod.http_entity.buf.size) { + icapReqModPassHttpBody(icap, + icap->reqmod.http_entity.callback_buf, + icap->reqmod.http_entity.callback_bufsize, + icap->reqmod.http_entity.callback, + icap->reqmod.http_entity.callback_data); + icap->reqmod.http_entity.callback = NULL; + cbdataUnlock(icap->reqmod.http_entity.callback_data); + + } +} + +/* + * icapReqModPassHttpBody + * + * Called from http.c after request headers have been sent. + * This function feeds the http.c module chunks of the request + * body that were stored in the http_entity.buf MemBuf. + */ +static void +icapReqModPassHttpBody(IcapStateData * icap, char *buf, size_t size, + CBCB * callback, void *cbdata) +{ + debug(81, 3) ("icapReqModPassHttpBody: called\n"); + if (!buf) { + debug(81, 1) ("icapReqModPassHttpBody: FD %d called with %p, %d, %p (request aborted)\n", + icap->icap_fd, buf, (int) size, cbdata); + comm_close(icap->icap_fd); + return; + } + if (!cbdataValid(cbdata)) { + debug(81, + 1) + ("icapReqModPassHttpBody: FD %d callback data invalid, closing\n", + icap->icap_fd); + comm_close(icap->icap_fd); /*It is better to be sure that the connection will be closed..... */ + /*icapReqModKeepAliveOrClose(icap); */ + return; + } + debug(81, 3) ("icapReqModPassHttpBody: entity buf size = %d\n", + icap->reqmod.http_entity.buf.size); + if (icap->reqmod.http_entity.buf.size) { + int copy_sz = icap->reqmod.http_entity.buf.size; + if (copy_sz > size) + copy_sz = size; + xmemcpy(buf, icap->reqmod.http_entity.buf.buf, copy_sz); + /* XXX don't let Alex see this ugliness */ + xmemmove(icap->reqmod.http_entity.buf.buf, + icap->reqmod.http_entity.buf.buf + copy_sz, + icap->reqmod.http_entity.buf.size - copy_sz); + icap->reqmod.http_entity.buf.size -= copy_sz; + debug(81, 3) ("icapReqModPassHttpBody: giving %d bytes to other side\n", + copy_sz); + callback(buf, copy_sz, cbdata); + debug(81, 3) ("icapReqModPassHttpBody: entity buf size now = %d\n", + icap->reqmod.http_entity.buf.size); + return; + } + if (icap->flags.reqmod_http_entity_eof) { + debug(81, 3) ("icapReqModPassHttpBody: signalling EOF\n"); + callback(buf, 0, cbdata); + icapReqModKeepAliveOrClose(icap); + return; + } + /* + * We have no data for the other side at this point. Save all + * these values and use them when we do have data. + */ + assert(NULL == icap->reqmod.http_entity.callback); + icap->reqmod.http_entity.callback = callback; + icap->reqmod.http_entity.callback_data = cbdata; + icap->reqmod.http_entity.callback_buf = buf; + icap->reqmod.http_entity.callback_bufsize = size; + cbdataLock(icap->reqmod.http_entity.callback_data); +} + +/* + * Body reader handler for use with request->body_reader function + * Simple a wrapper for icapReqModPassHttpBody function + */ + +static void +icapReqModBodyReader(request_t * request, char *buf, size_t size, + CBCB * callback, void *cbdata) +{ + IcapStateData *icap = request->body_reader_data; + icapReqModPassHttpBody(icap, buf, size, callback, cbdata); +} + +/* + * icapReqModMemBufAppend + * + * stupid wrapper to eliminate compiler warnings + */ +static void +icapReqModMemBufAppend(void *data, const char *buf, ssize_t size) +{ + memBufAppend(data, buf, size); +} Index: src/icap_respmod.c =================================================================== RCS file: src/icap_respmod.c diff -N src/icap_respmod.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/icap_respmod.c 23 Nov 2005 20:34:34 -0000 1.1.2.60 @@ -0,0 +1,1039 @@ + +/* + * $Id$ + * + * DEBUG: section 81 Internet Content Adaptation Protocol (ICAP) Client + * AUTHOR: Geetha Manjunath, Hewlett Packard Company + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + */ + +#include "squid.h" + +static CWCB icapSendRespModDone; +static PF icapRespModGobble; +extern PF icapReadReply; +static PF icapRespModReadReply; +static int icapReadReply2(IcapStateData * icap); +static void icapReadReply3(IcapStateData * icap); + +#define EXPECTED_ICAP_HEADER_LEN 256 +const char *crlf = "\r\n"; + +static void +getICAPRespModString(MemBuf * mb, int o1, int o2, int o3, + const char *client_addr, IcapStateData * icap, const icap_service * service) +{ + memBufPrintf(mb, "RESPMOD %s ICAP/1.0\r\nEncapsulated:", service->uri); + if (o1 >= 0) + memBufPrintf(mb, " req-hdr=%1d", o1); + if (o2 >= 0) + memBufPrintf(mb, ", res-hdr=%1d", o2); + if (o3 >= 0) + memBufPrintf(mb, ", res-body=%1d", o3); + else + memBufPrintf(mb, ", null-body=%1d", -o3); + + memBufPrintf(mb, crlf); + if (Config.icapcfg.send_client_ip || service->flags.need_x_client_ip) { + memBufPrintf(mb, "X-Client-IP: %s\r\n", client_addr); + } + if ((Config.icapcfg.send_auth_user + || service->flags.need_x_authenticated_user) + && (icap->request->auth_user_request != NULL)) { + icapAddAuthUserHeader(mb, icap->request->auth_user_request); + } +#if NOT_YET_FINISHED + if (Config.icapcfg.trailers) { + memBufPrintf(mb, "X-TE: trailers\r\n"); + } +#endif + if (service->flags.allow_204) + memBufPrintf(mb, "Allow: 204\r\n"); +} + +static int +buildRespModHeader(MemBuf * mb, IcapStateData * icap, char *buf, + ssize_t len, int theEnd) +{ + MemBuf mb_hdr; + char *client_addr; + int o2 = 0; + int o3 = 0; + int hlen; + int consumed; + icap_service *service; + HttpReply *r; + + if (memBufIsNull(&icap->respmod.req_hdr_copy)) + memBufDefInit(&icap->respmod.req_hdr_copy); + + memBufAppend(&icap->respmod.req_hdr_copy, buf, len); + + if (icap->respmod.req_hdr_copy.size > 4 && strncmp(icap->respmod.req_hdr_copy.buf, "HTTP/", 5)) { + debug(81, 3) ("buildRespModHeader: Non-HTTP-compliant header: '%s'\n", buf); + /* + *Possible we can consider that we did not have http responce headers + *(maybe HTTP 0.9 protocol), lets returning -1... + */ + consumed = -1; + o2 = -1; + memBufDefInit(&mb_hdr); + } else { + + hlen = headersEnd(icap->respmod.req_hdr_copy.buf, + icap->respmod.req_hdr_copy.size); + debug(81, 3) ("buildRespModHeader: headersEnd = %d(%s)\n", hlen, buf); + if (0 == hlen) + return 0; + + /* + * calc how many bytes from this 'buf' went towards the + * reply header. + */ + consumed = hlen - (icap->respmod.req_hdr_copy.size - len); + debug(81, 3) ("buildRespModHeader: consumed = %d\n", consumed); + + + /* + * now, truncate our req_hdr_copy at the header end. + * this 'if' statement might be unncessary? + */ + if (hlen < icap->respmod.req_hdr_copy.size) + icap->respmod.req_hdr_copy.size = hlen; + + /* Copy request header */ + memBufDefInit(&mb_hdr); + httpBuildRequestPrefix(icap->request, icap->request, + icap->respmod.entry, &mb_hdr, icap->http_flags); + o2 = mb_hdr.size; + } + + /* Copy response header - Append to request header mbuffer */ + memBufAppend(&mb_hdr, + icap->respmod.req_hdr_copy.buf, icap->respmod.req_hdr_copy.size); + o3 = mb_hdr.size; + + service = icap->current_service; + assert(service); + client_addr = inet_ntoa(icap->request->client_addr); + + r = httpReplyCreate(); + httpReplyParse(r, icap->respmod.req_hdr_copy.buf, + icap->respmod.req_hdr_copy.size); + icap->respmod.res_body_sz = httpReplyBodySize(icap->request->method, r); + httpReplyDestroy(r); + if (icap->respmod.res_body_sz) + getICAPRespModString(mb, 0, o2, o3, client_addr, icap, service); + else + getICAPRespModString(mb, 0, o2, -o3, client_addr, icap, service); + if (Config.icapcfg.preview_enable) + if (icap->preview_size >= 0) { + memBufPrintf(mb, "Preview: %d\r\n", icap->preview_size); + icap->flags.preview_done = 0; + } + if (service->keep_alive) { + icap->flags.keep_alive = 1; + memBufAppend(mb, "Connection: keep-alive\r\n", 24); + } else { + icap->flags.keep_alive = 0; + memBufAppend(mb, "Connection: close\r\n", 19); + } + memBufAppend(mb, crlf, 2); + memBufAppend(mb, mb_hdr.buf, mb_hdr.size); + memBufClean(&mb_hdr); + + + return consumed; +} + + +void +icapSendRespMod(IcapStateData * icap, char *buf, int len, int theEnd) +{ + MemBuf mb; +#if ICAP_PREVIEW + int size; + const int preview_size = icap->preview_size; +#endif + debug(81, 5) ("icapSendRespMod: FD %d, len %d, theEnd %d\n", + icap->icap_fd, len, theEnd); + + if (icap->flags.no_content) { + /* + * ICAP server said there are no modifications to make, so + * just append this data to the StoreEntry + */ + if (icap->respmod.resp_copy.size) { + /* + * first copy the data that we already sent to the ICAP server + */ + memBufAppend(&icap->chunk_buf, + icap->respmod.resp_copy.buf, icap->respmod.resp_copy.size); + icap->respmod.resp_copy.size = 0; + } + debug(81, 5) ("icapSendRepMod: len=%d theEnd=%d write_pending=%d\n", + len, theEnd, icap->flags.write_pending); + if (len) { + /* + * also copy any new data from the HTTP side + */ + memBufAppend(&icap->chunk_buf, buf, len); + } + (void) icapReadReply2(icap); + return; + } + if (theEnd) { + if (icap->respmod.res_body_sz) + icap->flags.send_zero_chunk = 1; + icap->flags.http_server_eof = 1; + } + /* + * httpReadReply is going to call us with a chunk and then + * right away again with an EOF if httpPconnTransferDone() is true. + * Since the first write is already dispatched, we'll have to + * hack this in somehow. + */ + if (icap->flags.write_pending) { + debug(81, 3) ("icapSendRespMod: oops, write_pending=1\n"); + assert(theEnd); + assert(len == 0); + return; + } + if (!cbdataValid(icap)) { + debug(81, 3) ("icapSendRespMod: failed to establish connection?\n"); + return; + } + memBufDefInit(&mb); + +#if SUPPORT_ICAP_204 || ICAP_PREVIEW + /* + * make a copy of the response in case ICAP server gives us a 204 + */ + /* + * This piece of code is problematic for 204 responces outside preview. + * The icap->respmod.resp_copy continues to filled until we had responce + * If the icap server waits to gets all data before sends its responce + * then we are puting all downloading object to the main system memory. + * My opinion is that 204 responces outside preview must be disabled ..... + * /chtsanti + */ + + if (len && icap->flags.copy_response) { + if (memBufIsNull(&icap->respmod.resp_copy)) + memBufDefInit(&icap->respmod.resp_copy); + memBufAppend(&icap->respmod.resp_copy, buf, len); + } +#endif + + if (icap->sc == 0) { + /* No data sent yet. Start with headers */ + if ((icap->sc = buildRespModHeader(&mb, icap, buf, len, theEnd)) > 0) { + buf += icap->sc; + len -= icap->sc; + } + /* + * Then we do not have http responce headers. All data (previous and those in buf) + * now are exist to icap->respmod.req_hdr_copy. Lets get them back....... + */ + if (icap->sc < 0) { + memBufAppend(&icap->respmod.buffer, + icap->respmod.req_hdr_copy.buf, + icap->respmod.req_hdr_copy.size); + icap->sc = icap->respmod.req_hdr_copy.size; + icap->respmod.req_hdr_copy.size = 0; + buf = NULL; + len = 0; + } + } + if (0 == icap->sc) { + /* check again; bail if we're not ready to send ICAP/HTTP hdrs */ + debug(81, 5) ("icapSendRespMod: dont have full HTTP response hdrs\n"); + memBufClean(&mb); + return; + } +#if ICAP_PREVIEW + if (preview_size < 0 || !Config.icapcfg.preview_enable) /* preview feature off */ + icap->flags.preview_done = 1; + + if (!icap->flags.preview_done) { + /* preview not yet sent */ + if (icap->sc > 0 && icap->respmod.buffer.size <= preview_size + && len > 0) { + /* Try to collect at least preview_size+1 bytes */ + /* By collecting one more byte than needed for preview we know best */ + /* whether we have to send the ieof chunk extension */ + size = icap->respmod.buffer.size + len; + if (size > preview_size + 1) + size = preview_size + 1; + size -= icap->respmod.buffer.size; + debug(81, + 3) + ("icapSendRespMod: FD %d: copy %d more bytes to preview buffer.\n", + icap->icap_fd, size); + memBufAppend(&icap->respmod.buffer, buf, size); + buf = ((char *) buf) + size; + len -= size; + } + if (icap->respmod.buffer.size > preview_size || theEnd) { + /* we got enough bytes for preview or this is the last call */ + /* add preview preview now */ + if (icap->respmod.buffer.size > 0) { + size = icap->respmod.buffer.size; + if (size > preview_size) + size = preview_size; + memBufPrintf(&mb, "%x\r\n", size); + memBufAppend(&mb, icap->respmod.buffer.buf, size); + memBufAppend(&mb, crlf, 2); + icap->sc += size; + } + if (icap->respmod.buffer.size <= preview_size) { + /* content length is less than preview size+1 */ + if (icap->respmod.res_body_sz) + memBufAppend(&mb, "0; ieof\r\n\r\n", 11); + memBufReset(&icap->respmod.buffer); /* will now be used for other data */ + } else { + char ch; + memBufAppend(&mb, "0\r\n\r\n", 5); + /* end of preview, wait for continue or 204 signal */ + /* copy the extra byte and all other data to the icap buffer */ + /* so that it can be handled next time */ + ch = icap->respmod.buffer.buf[preview_size]; + memBufReset(&icap->respmod.buffer); /* will now be used for other data */ + memBufAppend(&icap->respmod.buffer, &ch, 1); + debug(81, + 3) + ("icapSendRespMod: FD %d: sending preview and keeping %d bytes in internal buf.\n", + icap->icap_fd, len + 1); + if (len > 0) + memBufAppend(&icap->respmod.buffer, buf, len); + } + icap->flags.preview_done = 1; + icap->flags.wait_for_preview_reply = 1; + } + } else if (icap->flags.wait_for_preview_reply) { + /* received new data while waiting for preview response */ + /* add data to internal buffer and send later */ + debug(81, + 3) + ("icapSendRespMod: FD %d: add %d more bytes to internal buf while waiting for preview-response.\n", + icap->icap_fd, len); + if (len > 0) + memBufAppend(&icap->respmod.buffer, buf, len); + /* do not send any data now while waiting for preview response */ + /* but prepare for read more data on the HTTP connection */ + memBufClean(&mb); + return; + } else +#endif + { + /* after preview completed and ICAP preview response received */ + /* there may still be some data in the buffer */ + if (icap->respmod.buffer.size > 0) { + memBufPrintf(&mb, "%x\r\n", icap->respmod.buffer.size); + memBufAppend(&mb, icap->respmod.buffer.buf, + icap->respmod.buffer.size); + memBufAppend(&mb, crlf, 2); + icap->sc += icap->respmod.buffer.size; + memBufReset(&icap->respmod.buffer); + } + if (len > 0) { + memBufPrintf(&mb, "%x\r\n", len); + memBufAppend(&mb, buf, len); + memBufAppend(&mb, crlf, 2); + icap->sc += len; + } + if (icap->flags.send_zero_chunk) { + /* send zero end chunk */ + icap->flags.send_zero_chunk = 0; + icap->flags.http_server_eof = 1; + memBufAppend(&mb, "0\r\n\r\n", 5); + } + /* wait for data coming from ICAP server as soon as we sent something */ + /* but of course only until we got the response header */ + if (!icap->flags.got_reply) + icap->flags.wait_for_reply = 1; + } + commSetTimeout(icap->icap_fd, -1, NULL, NULL); + + if (!mb.size) { + memBufClean(&mb); + return; + } + debug(81, 5) ("icapSendRespMod: FD %d writing {%s}\n", icap->icap_fd, + mb.buf); + icap->flags.write_pending = 1; + comm_write_mbuf(icap->icap_fd, mb, icapSendRespModDone, icap); +} + +static void +icapRespModReadReply(int fd, void *data) +{ + IcapStateData *icap = data; + int version_major, version_minor; + const char *str_status; + int x; + int status = 0; + int isIcap = 0; + int directResponse = 0; + ErrorState *err; + const char *start; + const char *end; + + debug(81, 5) ("icapRespModReadReply: FD %d data = %p\n", fd, data); + statCounter.syscalls.sock.reads++; + + x = icapReadHeader(fd, icap, &isIcap); + if (x < 0) { + /* Did not find a proper ICAP response */ + debug(81, 3) ("ICAP : Error path!\n"); + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->request = requestLink(icap->request); + err->xerrno = errno; + errorAppendEntry(icap->respmod.entry, err); + comm_close(fd); + return; + } + if (x == 0) { + /* + * Waiting for more headers. Schedule new read hander, but + * don't reset timeout. + */ + commSetSelect(fd, COMM_SELECT_READ, icapRespModReadReply, icap, 0); + return; + } + /* + * Parse the ICAP header + */ + assert(icap->icap_hdr.size); + debug(81, 3) ("Parse icap header : <%s>\n", icap->icap_hdr.buf); + if ((status = + icapParseStatusLine(icap->icap_hdr.buf, icap->icap_hdr.size, + &version_major, &version_minor, &str_status)) < 0) { + debug(81, 1) ("BAD ICAP status line <%s>\n", icap->icap_hdr.buf); + /* is this correct in case of ICAP protocol error? */ + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->request = requestLink(icap->request); + err->xerrno = errno; + errorAppendEntry(icap->respmod.entry, err); + comm_close(fd); + return; + }; + /* OK here we have responce. Lets stop filling the + * icap->respmod.resp_copy buffer .... + */ + icap->flags.copy_response = 0; + + icapSetKeepAlive(icap, icap->icap_hdr.buf); +#if ICAP_PREVIEW + if (icap->flags.wait_for_preview_reply) { + if (100 == status) { + debug(81, 5) ("icapRespModReadReply: 100 Continue received\n"); + icap->flags.wait_for_preview_reply = 0; + /* if http_server_eof + * call again icapSendRespMod to handle data that + * was received while waiting for this ICAP response + * else let http to call icapSendRespMod when new data arrived + */ + if (icap->flags.http_server_eof) + icapSendRespMod(icap, NULL, 0, 0); + /* + * reset the header to send the rest of the preview + */ + if (!memBufIsNull(&icap->icap_hdr)) + memBufReset(&icap->icap_hdr); + + /*We do n't need it any more ....... */ + if (!memBufIsNull(&icap->respmod.resp_copy)) + memBufClean(&icap->respmod.resp_copy); + + return; + } + if (204 == status) { + debug(81, + 5) ("icapRespModReadReply: 204 No modification received\n"); + icap->flags.wait_for_preview_reply = 0; + } + } +#endif /*ICAP_PREVIEW */ + +#if SUPPORT_ICAP_204 || ICAP_PREVIEW + if (204 == status) { + debug(81, 3) ("got 204 status from ICAP server\n"); + debug(81, 3) ("setting icap->flags.no_content\n"); + icap->flags.no_content = 1; + /* + * copy the response already written to the ICAP server + */ + debug(81, 3) ("copying %d bytes from resp_copy to chunk_buf\n", + icap->respmod.resp_copy.size); + memBufAppend(&icap->chunk_buf, + icap->respmod.resp_copy.buf, icap->respmod.resp_copy.size); + icap->respmod.resp_copy.size = 0; + if (icapReadReply2(icap) < 0) + comm_close(fd); + /* + * XXX ideally want to clean icap->respmod.resp_copy here + * XXX ideally want to "close" ICAP server connection here + * OK do it.... + */ + if (!memBufIsNull(&icap->respmod.resp_copy)) + memBufClean(&icap->respmod.resp_copy); + return; + } +#endif + if (200 != status && 201 != status) { + debug(81, 1) ("Unsupported status '%d' from ICAP server\n", status); + /* Did not find a proper ICAP response */ + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->request = requestLink(icap->request); + err->xerrno = errno; + errorAppendEntry(icap->respmod.entry, err); + comm_close(fd); + return; + } + if (icapFindHeader(icap->icap_hdr.buf, "Encapsulated:", &start, &end)) { + icapParseEncapsulated(icap, start, end); + } else { + debug(81, + 1) + ("WARNING: icapRespModReadReply() did not find 'Encapsulated' header\n"); + } + if (icap->enc.res_hdr > -1) + directResponse = 1; + else if (icap->enc.res_body > -1) + directResponse = 1; + else + directResponse = 0; + + /* + * "directResponse" is the normal case here. If we don't have + * a response header or body, it is an error. + */ + if (!directResponse) { + /* Did not find a proper ICAP response */ + debug(81, 3) ("ICAP : Error path!\n"); + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->request = requestLink(icap->request); + err->xerrno = errno; + errorAppendEntry(icap->respmod.entry, err); + comm_close(fd); + return; + } + /* got the reply, no need to come here again */ + icap->flags.wait_for_reply = 0; + icap->flags.got_reply = 1; + /* Next, gobble any data before the HTTP response starts */ + if (icap->enc.res_hdr > -1) + icap->bytes_to_gobble = icap->enc.res_hdr; + commSetSelect(fd, COMM_SELECT_READ, icapRespModGobble, icap, 0); +} + + +/* + * Gobble up (read) some bytes until we get to the start of the body + */ +static void +icapRespModGobble(int fd, void *data) +{ + IcapStateData *icap = data; + int len; + LOCAL_ARRAY(char, junk, SQUID_TCP_SO_RCVBUF); + debug(81, 3) ("icapRespModGobble: FD %d gobbling %d bytes\n", fd, + icap->bytes_to_gobble); + len = FD_READ_METHOD(fd, junk, icap->bytes_to_gobble); + debug(81, 3) ("icapRespModGobble: gobbled %d bytes\n", len); + if (len < 0) { + /* XXX error */ + abort(); + } + icap->bytes_to_gobble -= len; + if (icap->bytes_to_gobble) + commSetSelect(fd, COMM_SELECT_READ, icapRespModGobble, icap, 0); + else + icapReadReply(fd, icap); +} + + +static void +icapSendRespModDone(int fd, char *bufnotused, size_t size, int errflag, + void *data) +{ + IcapStateData *icap = data; + ErrorState *err; + + icap->flags.write_pending = 0; + debug(81, 5) ("icapSendRespModDone: FD %d: size %d: errflag %d.\n", + fd, size, errflag); + if (size > 0) { + fd_bytes(fd, size, FD_WRITE); + kb_incr(&statCounter.icap.all.kbytes_out, size); + } + if (errflag == COMM_ERR_CLOSING) + return; + if (errflag) { + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->xerrno = errno; + if (cbdataValid(icap)) + err->request = requestLink(icap->request); + storeEntryReset(icap->respmod.entry); + errorAppendEntry(icap->respmod.entry, err); + comm_close(fd); + return; + } + if (EBIT_TEST(icap->respmod.entry->flags, ENTRY_ABORTED)) { + debug(81, 3) ("icapSendRespModDone: Entry Aborded\n"); + comm_close(fd); + return; + } + if (icap->flags.send_zero_chunk) { + debug(81, + 3) ("icapSendRespModDone: I'm supposed to send zero chunk now\n"); + icap->flags.send_zero_chunk = 0; + icapSendRespMod(icap, NULL, 0, 1); + return; + } + if (icap->flags.wait_for_preview_reply || icap->flags.wait_for_reply) { + /* Schedule reading the ICAP response */ + debug(81, + 3) + ("icapSendRespModDone: FD %d: commSetSelect on read icapRespModReadReply.\n", + fd); + commSetSelect(fd, COMM_SELECT_READ, icapRespModReadReply, icap, 0); +#if 1 + commSetTimeout(fd, Config.Timeout.read, icapReadTimeout, icap); +#else + if (icap->flags.wait_for_preview_reply || icap->flags.http_server_eof) { + /* + * Set the read timeout only after all data has been sent + * or we are waiting for a preview response + * If the ICAP server does not return any data till all data + * has been sent, we are likely to hit the timeout for large + * HTTP bodies + */ + commSetTimeout(fd, Config.Timeout.read, icapReadTimeout, icap); + } +#endif + } +} + +void +icapConnectOver(int fd, int status, void *data) +{ + ErrorState *err; + IcapStateData *icap = data; + debug(81, 3) ("icapConnectOver: FD %d, status=%d\n", fd, status); + icap->flags.connect_pending = 0; + if (status < 0) { + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->xerrno = errno; + err->request = requestLink(icap->request); + errorAppendEntry(icap->respmod.entry, err); + comm_close(fd); + debug(81, 3) ("icapConnectOver: status < 0, unreachable=1\n"); + icapOptSetUnreachable(icap->current_service); + return; + } + fd_table[fd].pconn.uses++; + fd_table[fd].pconn.type = 2; + commSetSelect(fd, COMM_SELECT_READ, icapRespModReadReply, icap, 0); +} + + + +IcapStateData * +icapRespModStart(icap_service_t type, request_t * request, StoreEntry * entry, + http_state_flags http_flags) +{ + IcapStateData *icap = NULL; + CNCB *theCallback = NULL; + icap_service *service = NULL; + + debug(81, 3) ("icapRespModStart: type=%d\n", (int) type); + assert(type >= 0 && type < ICAP_SERVICE_MAX); + + service = icapService(type, request); + if (!service) { + debug(81, 3) ("icapRespModStart: no service found\n"); + return NULL; /* no service found */ + } + if (service->unreachable) { + if (service->bypass) { + debug(81, + 5) + ("icapRespModStart: BYPASS because service unreachable: %s\n", + service->uri); + return NULL; + } else { + debug(81, + 5) + ("icapRespModStart: ERROR because service unreachable: %s\n", + service->uri); + return (IcapStateData *) - 1; + } + } + switch (type) { + /* TODO: When we support more than ICAP_SERVICE_RESPMOD_PRECACHE, we needs to change + * this switch, because callbacks isn't keep */ + case ICAP_SERVICE_RESPMOD_PRECACHE: + theCallback = icapConnectOver; + break; + default: + fatalf("icapRespModStart: unsupported service type '%s'\n", + icap_service_type_str[type]); + break; + } + + icap = icapAllocate(); + if (!icap) { + debug(81, 3) ("icapRespModStart: icapAllocate() failed\n"); + return NULL; + } + icap->request = requestLink(request); + icap->respmod.entry = entry; + if (entry) + storeLockObject(entry); + icap->http_flags = http_flags; + memBufDefInit(&icap->respmod.buffer); + memBufDefInit(&icap->chunk_buf); + + icap->current_service = service; + icap->preview_size = service->preview; + + /* + * Don't create socket to the icap server now, but only for the first + * packet receive from the http server. This will resolve all timeout + * between the web server and icap server. + */ + debug(81, 3) ("icapRespModStart: setting connect_requested to 0\n"); + icap->flags.connect_requested = 0; + + /* + * make a copy the HTTP response that we send to the ICAP server in + * case it turns out to be a 204 + */ +#ifdef SUPPORT_ICAP_204 + icap->flags.copy_response = 1; +#elif ICAP_PREVIEW + if (preview_size < 0 || !Config.icapcfg.preview_enable) + icap->flags.copy_response = 0; + else + icap->flags.copy_response = 1; +#else + icap->flags.copy_response = 0; +#endif + + statCounter.icap.all.requests++; + debug(81, 3) ("icapRespModStart: returning %p\n", icap); + return icap; +} + +static int +icapHttpReplyHdrState(IcapStateData * icap) +{ + assert(icap); + if (NULL == icap->httpState) + return 0; + return icap->httpState->reply_hdr_state; +} + +static void +icapProcessHttpReplyHeader(IcapStateData * icap, const char *buf, int size) +{ + if (NULL == icap->httpState) { + icap->httpState = cbdataAlloc(HttpStateData); + icap->httpState->request = requestLink(icap->request); + icap->httpState->orig_request = requestLink(icap->request); + icap->httpState->entry = icap->respmod.entry; + storeLockObject(icap->httpState->entry); /* lock it */ + } + httpProcessReplyHeader(icap->httpState, buf, size); + if (2 == icap->httpState->reply_hdr_state) + EBIT_CLR(icap->httpState->entry->flags, ENTRY_FWD_HDR_WAIT); +} + +/* + * icapRespModKeepAliveOrClose + * + * Called when we are done reading from the ICAP server. + * Either close the connection or keep it open for a future + * transaction. + */ +static void +icapRespModKeepAliveOrClose(IcapStateData * icap) +{ + int fd = icap->icap_fd; + if (fd < 0) + return; + if (!icap->flags.keep_alive) { + debug(81, 3) ("%s:%d keep_alive not set, closing\n", __FILE__, + __LINE__); + comm_close(fd); + return; + } + debug(81, 3) ("%s:%d FD %d looks good, keeping alive\n", __FILE__, __LINE__, + fd); + commSetDefer(fd, NULL, NULL); + commSetTimeout(fd, -1, NULL, NULL); + commSetSelect(fd, COMM_SELECT_READ, NULL, NULL, 0); + comm_remove_close_handler(fd, icapStateFree, icap); + pconnPush(fd, icap->current_service->hostname, icap->current_service->port); + icap->icap_fd = -1; + icapStateFree(-1, icap); +} + + + +/* + * copied from httpPconnTransferDone + * + */ +static int +icapPconnTransferDone(int fd, IcapStateData * icap) +{ + debug(81, 3) ("icapPconnTransferDone: FD %d\n", fd); + /* + * Be careful with 204 responses. Normally we are done when we + * see the zero-end chunk, but that won't happen for 204s, so we + * use an EOF indicator on the HTTP side instead. + */ + if (icap->flags.no_content && icap->flags.http_server_eof) { + debug(81, 5) ("icapPconnTransferDone: no content, ret 1\n"); + return 1; + } + if (icapHttpReplyHdrState(icap) != 2) { + debug(81, + 5) ("icapPconnTransferDone: didn't see end of HTTP hdrs, ret 0\n"); + return 0; + } + if (icap->enc.null_body > -1) { + debug(81, 5) ("icapPconnTransferDone: no message body, ret 1\n"); + return 1; + } + if (icap->chunk_size == -2) { //AI: was != -2 ; and change content with bottom + /* zero end chunk reached */ + debug(81, 5) ("icapPconnTransferDone: got zero end chunk\n"); + return 1; + } + debug(81, 5) ("icapPconnTransferDone: didnt get zero end chunk yet\n"); //AI: change with second top condition + + return 0; +} + +static int +icapExpectedHttpReplyHdrSize(IcapStateData * icap) +{ + if (icap->enc.res_body > -1 && icap->enc.res_hdr > -1) + return (icap->enc.res_body - icap->enc.res_hdr); + if (icap->enc.null_body > -1 && icap->enc.res_hdr > -1) + return icap->enc.null_body - icap->enc.res_hdr; + /*The case we did not get res_hdr ..... */ + if (icap->enc.res_body > -1) + return icap->enc.res_body; + if (icap->enc.null_body > -1) + return icap->enc.null_body; + return -1; +} + +/* + * copied from httpReadReply() + * + * by the time this is called, the ICAP headers have already + * been read. + */ +void +icapReadReply(int fd, void *data) +{ + IcapStateData *icap = data; + StoreEntry *entry = icap->respmod.entry; + const request_t *request = icap->request; + int len; + debug(81, 5) ("icapReadReply: FD %d: icap %p.\n", fd, data); + if (icap->flags.no_content && !icap->flags.http_server_eof) { //AI + + return; + } + if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) { + comm_close(fd); + return; + } + errno = 0; + statCounter.syscalls.sock.reads++; + len = memBufRead(fd, &icap->chunk_buf); + debug(81, 5) ("icapReadReply: FD %d: len %d.\n", fd, len); + if (len > 0) { + fd_bytes(fd, len, FD_READ); + kb_incr(&statCounter.icap.all.kbytes_in, len); + commSetTimeout(fd, Config.Timeout.read, icapReadTimeout, icap); + if (icap->chunk_buf.size < icap->chunk_buf.capacity) { + *(icap->chunk_buf.buf + icap->chunk_buf.size) = '\0'; + debug(81, 9) ("{%s}\n", icap->chunk_buf.buf); + } + } + if (len <= 0) { + debug(81, 2) ("icapReadReply: FD %d: read failure: %s.\n", + fd, xstrerror()); + if (ignoreErrno(errno)) { + debug(81, 2) ("icapReadReply: FD %d: ignored errno\n", fd); + commSetSelect(fd, COMM_SELECT_READ, icapReadReply, icap, 0); + } else if (entry->mem_obj->inmem_hi == 0) { + ErrorState *err; + debug(81, 2) ("icapReadReply: FD %d: generating error page\n", fd); + err = errorCon(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR); + err->request = requestLink((request_t *) request); + err->xerrno = errno; + errorAppendEntry(entry, err); + comm_close(fd); + } else { + debug(81, 2) ("icapReadReply: FD %d: just calling comm_close()\n", + fd); + comm_close(fd); + } + return; + } + if (icapReadReply2(icap) < 0) + comm_close(fd); +} + +static int +icapReadReply2(IcapStateData * icap) +{ + StoreEntry *entry = icap->respmod.entry; + const request_t *request = icap->request; + debug(81, 3) ("icapReadReply2\n"); + if (icap->chunk_buf.size == 0 && entry->mem_obj->inmem_hi == 0) { + ErrorState *err; + err = errorCon(ERR_ZERO_SIZE_OBJECT, HTTP_SERVICE_UNAVAILABLE); + err->xerrno = errno; + err->request = requestLink((request_t *) request); + errorAppendEntry(entry, err); + icap->flags.http_server_eof = 1; + return -1; + } + if (icap->chunk_buf.size == 0) { + /* Retrieval done. */ + if (icapHttpReplyHdrState(icap) < 2) + icapProcessHttpReplyHeader(icap, icap->chunk_buf.buf, + icap->chunk_buf.size); + icap->flags.http_server_eof = 1; + icapReadReply3(icap); + return 0; + } + if (icapHttpReplyHdrState(icap) == 0) { + int expect = icapExpectedHttpReplyHdrSize(icap); + int so_far = icap->http_header_bytes_read_so_far; + int needed = expect - so_far; + debug(81, 3) ("expect=%d\n", expect); + debug(81, 3) ("so_far=%d\n", so_far); + debug(81, 3) ("needed=%d\n", needed); + assert(needed < 0 || needed >= 0); + if (0 > expect) { + icapProcessHttpReplyHeader(icap, + icap->chunk_buf.buf, icap->chunk_buf.size); + } else if (0 == expect) { + /* + * this icap reply doesn't give us new HTTP headers + * so we must copy them from our copy + */ + debug(81, 1) ("WARNING: untested code at %s:%d\n", __FILE__, + __LINE__); + if (icap->respmod.req_hdr_copy.size) { /*For HTTP 0.9 we do not have headers */ + storeAppend(entry, + icap->respmod.req_hdr_copy.buf, + icap->respmod.req_hdr_copy.size); + } + icapProcessHttpReplyHeader(icap, icap->chunk_buf.buf, + icap->chunk_buf.size); + assert(icapHttpReplyHdrState(icap) == 2); + icap->chunk_size = 0; /*we are ready to read chunks of data now.... */ + } else if (needed) { + icapProcessHttpReplyHeader(icap, + icap->chunk_buf.buf, icap->chunk_buf.size); + if (icap->chunk_buf.size >= needed) { + storeAppend(entry, icap->chunk_buf.buf, needed); + so_far += needed; + xmemmove(icap->chunk_buf.buf, + icap->chunk_buf.buf + needed, + icap->chunk_buf.size - needed); + icap->chunk_buf.size -= needed; + assert(icapHttpReplyHdrState(icap) == 2); + icap->chunk_size = 0; + } else { + /* + * We don't have the full HTTP reply headers yet, so keep + * the partial reply buffered in 'chunk_buf' and wait + * for more. + */ + debug(81, 3) ("We don't have full Http headers.Schedule a new read\n"); + commSetSelect(icap->icap_fd, COMM_SELECT_READ, icapReadReply, icap, 0); + } + } + icap->http_header_bytes_read_so_far = so_far; + } + debug(81, 3) ("%s:%d: icap->chunk_buf.size=%d\n", __FILE__, __LINE__, + (int) icap->chunk_buf.size); + debug(81, 3) ("%s:%d: flags.no_content=%d\n", __FILE__, __LINE__, + icap->flags.no_content); + if (icap->flags.no_content) { + /* data from http.c is not chunked */ + if (!EBIT_TEST(entry->flags, ENTRY_ABORTED)) { + debug(81, 3) ("copying %d bytes from chunk_buf to entry\n", + icap->chunk_buf.size); + storeAppend(entry, icap->chunk_buf.buf, icap->chunk_buf.size); + icap->chunk_buf.size = 0; + } + } else if (2 == icapHttpReplyHdrState(icap)) { + if (icap->chunk_buf.size) + icapParseChunkedBody(icap, (STRCB *) storeAppend, entry); + } + icapReadReply3(icap); + return 0; +} + +static void +icapReadReply3(IcapStateData * icap) +{ + StoreEntry *entry = icap->respmod.entry; + int fd = icap->icap_fd; + debug(81, 3) ("icapReadReply3\n"); + if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) { + debug(81, 3) ("icapReadReply3: Entry Aborded\n"); + comm_close(fd); + } else if (icapPconnTransferDone(fd, icap)) { + storeComplete(entry); + icapRespModKeepAliveOrClose(icap); + } else if (!icap->flags.no_content) { + /* Wait for EOF condition */ + commSetSelect(fd, COMM_SELECT_READ, icapReadReply, icap, 0); + debug(81, + 3) + ("icapReadReply3: Going to read mode data throught icapReadReply\n"); + } else { + debug(81, 3) ("icapReadReply3: Nothing\n"); + } +} Index: src/main.c =================================================================== RCS file: /cvsroot/squid/squid/src/main.c,v retrieving revision 1.28.6.25 retrieving revision 1.28.6.8.2.11 diff -p -u -b -r1.28.6.25 -r1.28.6.8.2.11 --- src/main.c 28 Jun 2005 02:16:51 -0000 1.28.6.25 +++ src/main.c 12 Sep 2005 18:34:41 -0000 1.28.6.8.2.11 @@ -350,6 +350,9 @@ mainReconfigure(void) #else idnsShutdown(); #endif +#ifdef HS_FEAT_ICAP + icapClose(); +#endif redirectShutdown(); authenticateShutdown(); externalAclShutdown(); @@ -378,6 +381,9 @@ mainReconfigure(void) idnsInit(); #endif redirectInit(); +#ifdef HS_FEAT_ICAP + icapInit(); +#endif authenticateInit(&Config.authConfig); externalAclInit(); #if USE_WCCP @@ -507,6 +513,9 @@ mainInitialize(void) idnsInit(); #endif redirectInit(); +#ifdef HS_FEAT_ICAP + icapInit(); +#endif authenticateInit(&Config.authConfig); externalAclInit(); useragentOpenLog(); Index: src/mem.c =================================================================== RCS file: /cvsroot/squid/squid/src/mem.c,v retrieving revision 1.13 retrieving revision 1.13.28.2 diff -p -u -b -r1.13 -r1.13.28.2 --- src/mem.c 7 Sep 2001 23:55:49 -0000 1.13 +++ src/mem.c 27 Jun 2003 01:15:18 -0000 1.13.28.2 @@ -243,6 +243,13 @@ memInit(void) memDataInit(MEM_CLIENT_REQ_BUF, "clientRequestBuffer", CLIENT_REQ_BUF_SZ, 0); memDataInit(MEM_SWAP_LOG_DATA, "storeSwapLogData", sizeof(storeSwapLogData), 0); +#ifdef HS_FEAT_ICAP + memDataInit(MEM_ICAP_OPT_DATA, "IcapOptData", sizeof(IcapOptData), 0); + memDataInit(MEM_ICAP_SERVICE_LIST, "icap_service_list", sizeof(icap_service_list), 0); + memDataInit(MEM_ICAP_CLASS, "icap_class", sizeof(icap_class), 0); + memDataInit(MEM_ICAP_ACCESS, "icap_access", sizeof(icap_access), 0); +#endif + /* init string pools */ for (i = 0; i < mem_str_pool_count; i++) { StrPools[i].pool = memPoolCreate(StrPoolsAttrs[i].name, StrPoolsAttrs[i].obj_size); Index: src/mk-string-arrays.pl =================================================================== RCS file: /cvsroot/squid/squid/src/mk-string-arrays.pl,v retrieving revision 1.2 retrieving revision 1.2.140.1 diff -p -u -b -r1.2 -r1.2.140.1 --- src/mk-string-arrays.pl 23 Oct 2000 15:04:21 -0000 1.2 +++ src/mk-string-arrays.pl 4 Apr 2003 16:55:44 -0000 1.2.140.1 @@ -16,6 +16,7 @@ $pat{'err_type'} = "err_type_str"; $pat{'icp_opcode'} = "icp_opcode_str"; $pat{'swap_log_op'} = "swap_log_op_str"; $pat{'lookup_t'} = "lookup_t_str"; +$pat{'icap_service_t'} = "icap_service_type_str"; $state = 0; # start state while (<>) { Index: src/pconn.c =================================================================== RCS file: /cvsroot/squid/squid/src/pconn.c,v retrieving revision 1.6.38.2 retrieving revision 1.6.60.2 diff -p -u -b -r1.6.38.2 -r1.6.60.2 --- src/pconn.c 16 Dec 2003 03:13:59 -0000 1.6.38.2 +++ src/pconn.c 23 Nov 2005 20:33:07 -0000 1.6.60.2 @@ -46,6 +46,9 @@ struct _pconn { #define PCONN_HIST_SZ (1<<16) int client_pconn_hist[PCONN_HIST_SZ]; int server_pconn_hist[PCONN_HIST_SZ]; +#ifdef HS_FEAT_ICAP +int icap_server_pconn_hist[PCONN_HIST_SZ]; +#endif static PF pconnRead; static PF pconnTimeout; @@ -159,6 +162,20 @@ pconnHistDump(StoreEntry * e) continue; storeAppendPrintf(e, "\t%4d %9d\n", i, server_pconn_hist[i]); } +#ifdef HS_FEAT_ICAP + storeAppendPrintf(e, + "\n" + "ICAP-server persistent connection counts:\n" + "\n" + "\treq/\n" + "\tconn count\n" + "\t---- ---------\n"); + for (i = 0; i < PCONN_HIST_SZ; i++) { + if (icap_server_pconn_hist[i] == 0) + continue; + storeAppendPrintf(e, "\t%4d %9d\n", i, icap_server_pconn_hist[i]); + } +#endif } /* ========== PUBLIC FUNCTIONS ============================================ */ @@ -173,6 +190,9 @@ pconnInit(void) for (i = 0; i < PCONN_HIST_SZ; i++) { client_pconn_hist[i] = 0; server_pconn_hist[i] = 0; +#ifdef HS_FEAT_ICAP + icap_server_pconn_hist[i] = 0; +#endif } pconn_data_pool = memPoolCreate("pconn_data", sizeof(struct _pconn)); pconn_fds_pool = memPoolCreate("pconn_fds", PCONN_FDS_SZ * sizeof(int)); @@ -248,11 +268,15 @@ pconnHistCount(int what, int i) { if (i >= PCONN_HIST_SZ) i = PCONN_HIST_SZ - 1; - /* what == 0 for client, 1 for server */ + /* what == 0 for client, 1 for server, 2 for ICAP server */ if (what == 0) client_pconn_hist[i]++; else if (what == 1) server_pconn_hist[i]++; +#ifdef HS_FEAT_ICAP + else if (what == 2) + icap_server_pconn_hist[i]++; +#endif else assert(0); } Index: src/protos.h =================================================================== RCS file: /cvsroot/squid/squid/src/protos.h,v retrieving revision 1.41.6.33 retrieving revision 1.41.6.13.2.37 diff -p -u -b -r1.41.6.33 -r1.41.6.13.2.37 --- src/protos.h 16 Sep 2005 02:13:25 -0000 1.41.6.33 +++ src/protos.h 6 Dec 2005 21:53:44 -0000 1.41.6.13.2.37 @@ -292,6 +292,8 @@ extern void whoisStart(FwdState *); /* http.c */ extern int httpCachable(method_t); extern void httpStart(FwdState *); +extern void httpParseReplyHeaders(const char *, http_reply *); +extern void httpProcessReplyHeader(HttpStateData *, const char *, int); extern int httpBuildRequestPrefix(request_t * request, request_t * orig_request, StoreEntry * entry, @@ -614,6 +616,7 @@ extern void memBufVPrintf(MemBuf * mb, c extern FREE *memBufFreeFunc(MemBuf * mb); /* puts report on MemBuf _module_ usage into mb */ extern void memBufReport(MemBuf * mb); +extern int memBufRead(int fd, MemBuf * mb); extern char *mime_get_header(const char *mime, const char *header); extern char *mime_get_header_field(const char *mime, const char *name, const char *prefix); @@ -1341,4 +1344,49 @@ extern void externalAclShutdown(void); extern int externalAclRequiresAuth(void *acl_data); extern char *strtokFile(void); +#ifdef HS_FEAT_ICAP +/* + * icap_common.c + */ +void icapInit(void); +void icapClose(void); +void icapParseEncapsulated(IcapStateData *, const char *, const char *); +icap_service *icapService(icap_service_t, request_t *); +int icapConnect(IcapStateData *, CNCB *); +IcapStateData *icapAllocate(void); +PF icapStateFree; +PF icapConnectTimeout; +PF icapReadTimeout; +icap_service_t icapServiceToType(const char *); +const char *icapServiceToStr(const icap_service_t); +int icapCheckAcl(clientHttpRequest *); +size_t icapLineLength(const char *, int); +int icapReadHeader(int, IcapStateData *, int *); +int icapFindHeader(const char *, const char *, const char **, const char **); +int icapParseKeepAlive(const IcapStateData *, const char *, const char *); +void icapSetKeepAlive(IcapStateData * icap, const char *hdrs); +size_t icapParseChunkedBody(IcapStateData *, STRCB *, void *); +void icapAddAuthUserHeader(MemBuf *, auth_user_request_t *); +int icapParseStatusLine(const char *, int, int *, int *, const char **); + +/* + * icap_respmod.c + */ +IcapStateData *icapRespModStart(icap_service_t, request_t *, StoreEntry *, http_state_flags); +void icapSendRespMod(IcapStateData *, char *, int, int); +CNCB icapConnectOver; + +/* + * icap_reqmod.c + */ +IcapStateData *icapReqModStart(icap_service*, const char *, request_t *, int, struct timeval, struct in_addr, void *); + +/* icap_opt.c */ +void icapOptInit(void); +void icapOptShutdown(void); +void icapOptSetUnreachable(icap_service * s); +/* for debugging purposes only */ +void dump_icap_config(IcapConfig * cfg); +#endif + #endif /* SQUID_PROTOS_H */ Index: src/squid.h =================================================================== RCS file: /cvsroot/squid/squid/src/squid.h,v retrieving revision 1.13.6.8 retrieving revision 1.13.6.6.2.11 diff -p -u -b -r1.13.6.8 -r1.13.6.6.2.11 --- src/squid.h 26 Mar 2005 03:15:58 -0000 1.13.6.8 +++ src/squid.h 15 May 2005 20:10:33 -0000 1.13.6.6.2.11 @@ -38,6 +38,14 @@ #include "config.h" /* + * experimental defines for ICAP + */ +#ifdef HS_FEAT_ICAP +#define ICAP_PREVIEW 1 +#define SUPPORT_ICAP_204 0 +#endif + +/* * On some systems, FD_SETSIZE is set to something lower than the * actual number of files which can be opened. IRIX is one case, * NetBSD is another. So here we increase FD_SETSIZE to our Index: src/stat.c =================================================================== RCS file: /cvsroot/squid/squid/src/stat.c,v retrieving revision 1.13.6.14 retrieving revision 1.13.6.7.2.7 diff -p -u -b -r1.13.6.14 -r1.13.6.7.2.7 --- src/stat.c 30 Mar 2005 02:17:46 -0000 1.13.6.14 +++ src/stat.c 23 Nov 2005 20:33:07 -0000 1.13.6.7.2.7 @@ -775,6 +775,17 @@ statAvgDump(StoreEntry * sentry, int min storeAppendPrintf(sentry, "server.other.kbytes_out = %f/sec\n", XAVG(server.other.kbytes_out.kb)); +#ifdef HS_FEAT_ICAP + storeAppendPrintf(sentry, "icap.all.requests = %f/sec\n", + XAVG(icap.all.requests)); + storeAppendPrintf(sentry, "icap.all.errors = %f/sec\n", + XAVG(icap.all.errors)); + storeAppendPrintf(sentry, "icap.all.kbytes_in = %f/sec\n", + XAVG(icap.all.kbytes_in.kb)); + storeAppendPrintf(sentry, "icap.all.kbytes_out = %f/sec\n", + XAVG(icap.all.kbytes_out.kb)); +#endif + storeAppendPrintf(sentry, "icp.pkts_sent = %f/sec\n", XAVG(icp.pkts_sent)); storeAppendPrintf(sentry, "icp.pkts_recv = %f/sec\n", @@ -1160,6 +1171,17 @@ statCountersDump(StoreEntry * sentry) storeAppendPrintf(sentry, "server.other.kbytes_out = %d\n", (int) f->server.other.kbytes_out.kb); +#if HS_FEAT_ICAP + storeAppendPrintf(sentry, "icap.all.requests = %d\n", + (int) f->icap.all.requests); + storeAppendPrintf(sentry, "icap.all.errors = %d\n", + (int) f->icap.all.errors); + storeAppendPrintf(sentry, "icap.all.kbytes_in = %d\n", + (int) f->icap.all.kbytes_in.kb); + storeAppendPrintf(sentry, "icap.all.kbytes_out = %d\n", + (int) f->icap.all.kbytes_out.kb); +#endif + storeAppendPrintf(sentry, "icp.pkts_sent = %d\n", f->icp.pkts_sent); storeAppendPrintf(sentry, "icp.pkts_recv = %d\n", @@ -1459,8 +1481,6 @@ statClientRequests(StoreEntry * s) storeAppendPrintf(s, "\tme: %s:%d\n", inet_ntoa(conn->me.sin_addr), ntohs(conn->me.sin_port)); - storeAppendPrintf(s, "\tnrequests: %d\n", - conn->nrequests); storeAppendPrintf(s, "\tdefer: n %d, until %ld\n", conn->defer.n, (long int) conn->defer.until); } Index: src/store.c =================================================================== RCS file: /cvsroot/squid/squid/src/store.c,v retrieving revision 1.16.6.9 retrieving revision 1.16.6.2.2.8 diff -p -u -b -r1.16.6.9 -r1.16.6.2.2.8 --- src/store.c 2 Sep 2005 02:13:43 -0000 1.16.6.9 +++ src/store.c 12 Sep 2005 18:34:41 -0000 1.16.6.2.2.8 @@ -520,7 +520,16 @@ storeAppend(StoreEntry * e, const char * MemObject *mem = e->mem_obj; assert(mem != NULL); assert(len >= 0); - assert(e->store_status == STORE_PENDING); + debug(20, 3) ("storeAppend: '%s'\n", storeKeyText(e->hash.key)); + if (e->store_status != STORE_PENDING) { + /* + * if we're not STORE_PENDING, then probably we got aborted + * and there should be NO clients on this entry + */ + assert(EBIT_TEST(e->flags, ENTRY_ABORTED)); + assert(e->mem_obj->nclients == 0); + return; + } if (len) { debug(20, 5) ("storeAppend: appending %d bytes for '%s'\n", len, Index: src/structs.h =================================================================== RCS file: /cvsroot/squid/squid/src/structs.h,v retrieving revision 1.48.2.43 retrieving revision 1.48.2.9.2.48 diff -p -u -b -r1.48.2.43 -r1.48.2.9.2.48 --- src/structs.h 4 Sep 2005 02:13:28 -0000 1.48.2.43 +++ src/structs.h 30 Nov 2005 21:52:15 -0000 1.48.2.9.2.48 @@ -384,6 +384,22 @@ struct _RemovalPolicySettings { wordlist *args; }; +#if HS_FEAT_ICAP +struct _IcapConfig { + int onoff; + int preview_enable; + icap_service *service_head; + icap_class *class_head; + icap_access *access_head; + int preview_size; + int check_interval; + int send_client_ip; + int send_auth_user; + char *auth_scheme; +}; + +#endif /* HS_FEAT_ICAP */ + struct _SquidConfig { struct { squid_off_t maxSize; @@ -714,6 +730,9 @@ struct _SquidConfig { char *store_dir_select_algorithm; int sleep_after_fork; /* microseconds */ external_acl *externalAclHelperList; +#ifdef HS_FEAT_ICAP + IcapConfig icapcfg; +#endif }; struct _SquidConfig2 { @@ -787,7 +806,10 @@ struct _fde { } flags; squid_off_t bytes_read; squid_off_t bytes_written; - int uses; /* ie # req's over persistent conn */ + struct { + int uses; + int type; + } pconn; struct _fde_disk { DWCB *wrt_handle; void *wrt_handle_data; @@ -982,6 +1004,130 @@ struct _http_state_flags { unsigned int request_sent:1; }; +#ifdef HS_FEAT_ICAP +struct _IcapStateData { + request_t *request; + http_state_flags http_flags; + HttpStateData *httpState; /* needed to parse HTTP headers only */ + int icap_fd; + int sc; + icap_service *current_service; + MemBuf icap_hdr; + struct { + int res_hdr; + int res_body; + int req_hdr; + int req_body; + int opt_body; + int null_body; + } enc; + int bytes_to_gobble; + int chunk_size; + MemBuf chunk_buf; + int preview_size; + squid_off_t fake_content_length; + int http_header_bytes_read_so_far; + struct { + const char *uri; /* URI for REQMODs */ + int client_fd; + struct timeval start; /* for logging */ + struct in_addr log_addr; /* for logging */ + int hdr_state; + MemBuf hdr_buf; + void *client_cookie; + struct { + MemBuf buf; + CBCB *callback; + void *callback_data; + char *callback_buf; + size_t callback_bufsize; + squid_off_t bytes_read; + } http_entity; + } reqmod; + struct { + StoreEntry *entry; + MemBuf buffer; + MemBuf req_hdr_copy; /* XXX barf */ + MemBuf resp_copy; /* XXX barf^max */ + squid_off_t res_body_sz; + } respmod; + struct { + unsigned int connect_requested:1; + unsigned int connect_pending:1; + unsigned int write_pending:1; + unsigned int keep_alive:1; + unsigned int http_server_eof:1; + unsigned int send_zero_chunk:1; + unsigned int got_reply:1; + unsigned int wait_for_reply:1; + unsigned int wait_for_preview_reply:1; + unsigned int preview_done:1; + unsigned int copy_response:1; + unsigned int no_content:1; + unsigned int reqmod_http_entity_eof:1; + } flags; +}; + +struct _icap_service { + icap_service *next; + char *name; /* name to be used when referencing ths service */ + char *uri; /* uri of server/service to use */ + char *type_name; /* {req|resp}mod_{pre|post}cache */ + + char *hostname; + unsigned short int port; + char *resource; + icap_service_t type; /* parsed type */ + icap_method_t method; + ushort bypass; /* flag: bypass allowed */ + ushort unreachable; /* flag: set to 1 if options request fails */ + IcapOptData *opt; /* temp data needed during opt request */ + struct { + unsigned int allow_204:1; + unsigned int need_x_client_ip:1; + unsigned int need_x_authenticated_user:1; + } flags; + int preview; + String istag; + String transfer_preview; + String transfer_ignore; + String transfer_complete; + int max_connections; + int options_ttl; + int keep_alive; +}; + +struct _icap_service_list { + icap_service_list *next; + icap_service *services[16]; + int nservices; /* Number of services already used */ + int last_service_used; /* Last services used, use to do a round robin */ +}; + +struct _icap_class { + icap_class *next; + char *name; + wordlist *services; + icap_service_list *isl; + ushort hidden; /* for unnamed classes */ +}; + +struct _icap_access { + icap_access *next; + char *service_name; + icap_class *class; + acl_access *access; +}; + +struct _IcapOptData { + char *buf; + off_t offset; + size_t size; + off_t headlen; +}; + +#endif + struct _HttpStateData { StoreEntry *entry; request_t *request; @@ -993,10 +1139,14 @@ struct _HttpStateData { int fd; http_state_flags flags; FwdState *fwd; +#ifdef HS_FEAT_ICAP + struct _IcapStateData *icap_writer; +#endif char *body_buf; int body_buf_sz; }; + struct _icpUdpData { struct sockaddr_in address; void *msg; @@ -1092,6 +1242,7 @@ struct _clientHttpRequest { unsigned int internal:1; unsigned int done_copying:1; unsigned int purging:1; + unsigned int did_icap_reqmod:1; unsigned int hit:1; } flags; struct { @@ -1100,6 +1251,9 @@ struct _clientHttpRequest { } redirect; dlink_node active; squid_off_t maxBodySize; +#if HS_FEAT_ICAP + IcapStateData *icap_reqmod; +#endif }; struct _ConnStateData { @@ -1127,7 +1281,6 @@ struct _ConnStateData { struct sockaddr_in me; struct in_addr log_addr; char rfc931[USER_IDENT_SZ]; - int nrequests; struct { int n; time_t until; @@ -1678,6 +1831,9 @@ struct _request_t { char *peer_login; /* Configured peer login:password */ time_t lastmod; /* Used on refreshes */ const char *vary_headers; /* Used when varying entities are detected. Changes how the store key is calculated */ +#if HS_FEAT_ICAP + icap_class *class; +#endif BODY_HANDLER *body_reader; void *body_reader_data; }; @@ -1784,7 +1940,11 @@ struct _StatCounters { kb_t kbytes_in; kb_t kbytes_out; } all , http, ftp, other; - } server; + } +#if HS_FEAT_ICAP + icap, +#endif + server; struct { int pkts_sent; int queries_sent; Index: src/typedefs.h =================================================================== RCS file: /cvsroot/squid/squid/src/typedefs.h,v retrieving revision 1.25.6.8 retrieving revision 1.25.6.1.6.13 diff -p -u -b -r1.25.6.8 -r1.25.6.1.6.13 --- src/typedefs.h 27 Mar 2005 02:16:17 -0000 1.25.6.8 +++ src/typedefs.h 28 Mar 2005 18:05:08 -0000 1.25.6.1.6.13 @@ -131,6 +131,15 @@ typedef struct _HttpHeaderStat HttpHeade typedef struct _HttpBody HttpBody; typedef struct _HttpReply HttpReply; typedef struct _HttpStateData HttpStateData; +#ifdef HS_FEAT_ICAP +typedef struct _IcapStateData IcapStateData; +typedef struct _IcapConfig IcapConfig; +typedef struct _icap_service icap_service; +typedef struct _icap_service_list icap_service_list; +typedef struct _icap_class icap_class; +typedef struct _icap_access icap_access; +typedef struct _IcapOptData IcapOptData; +#endif typedef struct _icpUdpData icpUdpData; typedef struct _clientHttpRequest clientHttpRequest; typedef struct _ConnStateData ConnStateData; Index: src/url.c =================================================================== RCS file: /cvsroot/squid/squid/src/url.c,v retrieving revision 1.7.6.6 retrieving revision 1.7.6.5.2.2 diff -p -u -b -r1.7.6.6 -r1.7.6.5.2.2 --- src/url.c 12 Nov 2005 03:13:48 -0000 1.7.6.6 +++ src/url.c 23 Nov 2005 20:38:56 -0000 1.7.6.5.2.2 @@ -103,6 +103,9 @@ const char *ProtocolStr[] = "whois", "internal", "https", +#ifdef HS_FEAT_ICAP + "icap", +#endif "TOTAL" }; @@ -221,6 +224,10 @@ urlParseProtocol(const char *s) return PROTO_WHOIS; if (strcasecmp(s, "internal") == 0) return PROTO_INTERNAL; +#ifdef HS_FEAT_ICAP + if (strcasecmp(s, "icap") == 0) + return PROTO_ICAP; +#endif return PROTO_NONE; } @@ -244,6 +251,10 @@ urlDefaultPort(protocol_t p) return CACHE_HTTP_PORT; case PROTO_WHOIS: return 43; +#ifdef HS_FEAT_ICAP + case PROTO_ICAP: + return 1344; +#endif default: return 0; }