FreeBSD source tree https://cgit-dev.freebsd.org/src/atom?h=releng%2F9.3 2014-05-16T15:53:27Z 2014-05-16T15:53:27Z Christian Brueffer brueffer@FreeBSD.org 2014-05-16T15:53:27Z urn:sha1:9024499b25d2b85fdac7b153c58fbfbcca7e22eb Properly free resources in case of error. CID: 1007032 Found with: Coverity Prevent(tm) 2014-02-07T05:23:04Z Alexander Motin mav@FreeBSD.org 2014-02-07T05:23:04Z urn:sha1:d6112afe6ab57c59849be84ac45bad791f4c4dc9 Fix lock acquisition in case no request space available, missed in r260097. 2014-01-23T17:27:16Z Alexander Motin mav@FreeBSD.org 2014-01-23T17:27:16Z urn:sha1:d5e74ed89d55fcad0f96c7053269087b3816cc09 I am sorry. :( 2014-01-23T00:46:29Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:46:29Z urn:sha1:f16f013c51280c2ac373f4d46b927556cd449966 Rework NFS Duplicate Request Cache cleanup logic. - Introduce additional hash to group requests by hash of sockref. This allows to process TCP acknowledgements without looping though all the cache, and as result allows to do it every time. - Indroduce additional callbacks to notify application layer about sockets disconnection. Without this last few requests processed just before socket disconnection never processed their ACKs and stuck in cache for many hours. - Implement transport-specific method for tracking reply acknowledgements. New implementation does not cross multiple stack layers to get the data and does not have race conditions that previously made some requests stuck in cache. This could be done more efficiently at sockbuf layer, but that would broke some KBIs, while I don't know other consumers for it aside NFS. - Instead of traversing all DRC twice per request, run cleaning only once per request, and except in some conditions traverse only single hash slot at a time. Together this limits NFS DRC growth only to situations of real connectivity problems. If network is working well, and so all replies are acknowledged, cache remains almost empty even after hours of heavy load. Without this change on the same test cache was growing to many thousand requests even with perfectly working local network. As another result this reduces CPU time spent on the DRC handling during SPEC NFS benchmark from about 10% to 0.5%. Sponsored by: iXsystems, Inc. 2014-01-23T00:45:20Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:45:20Z urn:sha1:18c4a9798c36105fd96ae1d8ba48eec2d3c72cac Move most of NFS file handle affinity code out of the heavily congested global RPC thread pool lock and protect it with own set of locks. On synthetic benchmarks this improves peak NFS request rate by 40%. 2014-01-23T00:44:45Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:44:45Z urn:sha1:99feeb7e6922507481c66a112e70caa362a1ee03 Introduce xprt_inactive_self() -- variant for use when sure that port is assigned to thread. For example, withing receive handlers. In that case the function reduces to single assignment and can avoid locking. 2014-01-23T00:44:14Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:44:14Z urn:sha1:4029dec2a33d462ee899e114a024165ca5ae49eb In addition to r259632 completely block receive upcalls if we have more data than we need. This reduces lock pressure from xprt_active() side. 2014-01-23T00:42:55Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:42:55Z urn:sha1:7690a974609b937f99d644a0ff14a131c553d934 Fix a bug introduced at r259632, triggering infinite loop in some cases. 2014-01-23T00:41:23Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:41:23Z urn:sha1:4ce8d27998c3d0e09d53fd899d80b849a10b1e0c Remove several linear list traversals per request from RPC server code. Do not insert active ports into pool->sp_active list if they are success- fully assigned to some thread. This makes that list include only ports that really require attention, and so traversal can be reduced to simple taking the first one. Remove idle thread from pool->sp_idlethreads list when assigning some work (port of requests) to it. That again makes possible to replace list traversals with simple taking the first element. 2014-01-23T00:40:28Z Alexander Motin mav@FreeBSD.org 2014-01-23T00:40:28Z urn:sha1:d2845e7e0395b5138d7dc7ff462e57d0dc91524e Rework flow control for connection-oriented (TCP) RPC server. When processing receive buffer, write the amount of data, expected in present request record, into socket's so_rcv.sb_lowat to make stack aware about our needs. When processing following upcalls, ignore them until socket collect enough data to be read and processed in one turn. This change reduces number of context switches and other operations in RPC stack during large NFS writes (especially via non-Jumbo networks) by order of magnitude. After precessing current packet, take another look into the pending buffer to find out whether the next packet had been already received. If not, deactivate this port right there without making RPC code to push this port to another thread just to find that there is nothing. If the next packet is received partially, also deactivate the port, but also update socket's so_rcv.sb_lowat to not be woken up prematurely. This change additionally reduces number of context switches per NFS request about in half.