FreeBSD source tree https://cgit-dev.freebsd.org/src-test2/atom?h=master 2020-12-20T02:59:44Z 2020-12-20T02:59:44Z Li-Wen Hsu lwhsu@FreeBSD.org 2020-12-20T02:59:44Z urn:sha1:3cc0c0d66a065554459bd2f9b4f80cc07426464a This is the last Subversion commit to src. Sponsored by: The FreeBSD Foundation 2020-12-19T22:04:46Z Andrew Gallatin gallatin@FreeBSD.org 2020-12-19T22:04:46Z urn:sha1:a034518ac8793059220af22e6ab25f84c5a6ddb8 In order to efficiently serve web traffic on a NUMA machine, one must avoid as many NUMA domain crossings as possible. With SO_REUSEPORT_LB, a number of workers can share a listen socket. However, even if a worker sets affinity to a core or set of cores on a NUMA domain, it will receive connections associated with all NUMA domains in the system. This will lead to cross-domain traffic when the server writes to the socket or calls sendfile(), and memory is allocated on the server's local NUMA node, but transmitted on the NUMA node associated with the TCP connection. Similarly, when the server reads from the socket, he will likely be reading memory allocated on the NUMA domain associated with the TCP connection. This change provides a new socket ioctl, TCP_REUSPORT_LB_NUMA. A server can now tell the kernel to filter traffic so that only incoming connections associated with the desired NUMA domain are given to the server. (Of course, in the case where there are no servers sharing the listen socket on some domain, then as a fallback, traffic will be hashed as normal to all servers sharing the listen socket regardless of domain). This allows a server to deal only with traffic that is local to its NUMA domain, and avoids cross-domain traffic in most cases. This patch, and a corresponding small patch to nginx to use TCP_REUSPORT_LB_NUMA allows us to serve 190Gb/s of kTLS encrypted https media content from dual-socket Xeons with only 13% (as measured by pcm.x) cross domain traffic on the memory controller. Reviewed by: jhb, bz (earlier version), bcr (man page) Tested by: gonzo Sponsored by: Netfix Differential Revision: https://reviews.freebsd.org/D21636 2020-12-19T21:46:09Z Andrew Gallatin gallatin@FreeBSD.org 2020-12-19T21:46:09Z urn:sha1:02bc3865aadf1bfd907bbf555e26719744ebf3c9 When ktls_bind_thread is 2, we pick a ktls worker thread that is bound to the same domain as the TCP connection associated with the socket. We use roughly the same code as netinet/tcp_hpts.c to do this. This allows crypto to run on the same domain as the TCP connection is associated with. Assuming TCP_REUSPORT_LB_NUMA (D21636) is in place & in use, this ensures that the crypto source and destination buffers are local to the same NUMA domain as we're running crypto on. This change (when TCP_REUSPORT_LB_NUMA, D21636, is used) reduces cross-domain traffic from over 37% down to about 13% as measured by pcm.x on a dual-socket Xeon using nginx and a Netflix workload. Reviewed by: jhb Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D21648 2020-12-19T14:54:28Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T14:54:28Z urn:sha1:f6d234d8706fa4ad513bb5db2976b0e354a0f0fe - varios "new sentence, new line" warnings - varios "sections out of conventional order" warnings - varios "unusual Xr order" warnings - varios "missing section argument" warnings - varios "no blank before trailing delimiter" warnings - varios "normalizing date format" warnings MFC after: 1 month 2020-12-19T13:56:19Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T13:56:19Z urn:sha1:a652079615ac3cb503e516e50246284bb4ad60eb - new sentence, new line - unusual Xr order: ioctl(2) after da(4) - unusual Xr order: sysexits(3) after nda(4) MFC after: 1 week 2020-12-19T13:51:46Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T13:51:46Z urn:sha1:8ef235787bcf90735e76bd476084f1ff016b7619 - Add missing quotation mark for a comment above the .Dd - inserting missing end of block: Sh breaks Bd - skipping paragraph macro: Pp before Bl - skipping paragraph macro: Pp before Bd - empty block: Bd MFC after: 1 week 2020-12-19T13:45:39Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T13:45:39Z urn:sha1:749ddf491b4370300a2edbd47aed9ee1b5d71178 - new sentence, new line - function name without markup: rtalloc() - function name without markup: VOP_RECLAIM() MFC after: 1 week 2020-12-19T13:36:59Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T13:36:59Z urn:sha1:60ee05db4b19763137c399a0d79509af14e7440a - new sentence, new line - sections out of conventional order: Sh FILES - unusual Xr order: bthost(1) after bthidd(8) - no blank before trailing delimiter - whitespace at end of input line - sections out of conventional order: Sh EXIT STATUS MFC after: 1 week 2020-12-19T13:23:26Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T13:23:26Z urn:sha1:84572415b4b55354811282cbc184f4a46f6952a1 MFC after: 1 week 2020-12-19T13:21:40Z Gordon Bergling gbe@FreeBSD.org 2020-12-19T13:21:40Z urn:sha1:1c77a783677b48893b777a727134eabd707d3e89 MFC after: 1 week