summaryrefslogtreecommitdiff
path: root/sys/netinet/tcp_output.c
Commit message (Collapse)AuthorAgeFilesLines
* Stop sending tiny new data segments during SACK recoveryRichard Scheffenegger2020-10-091-2/+1
| | | | | | | | | | | | | | | | | Consider the currently in-use TCP options when calculating the amount of new data to be injected during SACK loss recovery. That addresses the effect that very small (new) segments could be injected on partial ACKs while still performing a SACK loss recovery. Reported by: Liang Tian Reviewed by: tuexen, chengc_netapp.com MFC after: 2 weeks Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D26446 Notes: svn path=/head/; revision=366570
* TCP: send full initial window when timestamps are in useRichard Scheffenegger2020-09-251-0/+14
| | | | | | | | | | | | | | | | | | | | The fastpath in tcp_output tries to send out full segments, and avoid sending partial segments by comparing against the static t_maxseg variable. That value does not consider tcp options like timestamps, while the initial window calculation is using the correct dynamic tcp_maxseg() function. Due to this interaction, the last, full size segment is considered too short and not sent out immediately. Reviewed by: tuexen MFC after: 2 weeks Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D26478 Notes: svn path=/head/; revision=366150
* net: clean up empty lines in .c and .h filesMateusz Guzik2020-09-011-1/+0
| | | | Notes: svn path=/head/; revision=365071
* TCP: remove special treatment for hardware (ifnet) TLSAndrew Gallatin2020-08-191-11/+0
| | | | | | | | | | | | | | | | | | | | | | | | | Remove most special treatment for ifnet TLS in the TCP stack, except for code to avoid mixing handshakes and bulk data. This code made heroic efforts to send down entire TLS records to NICs. It was added to improve the PCIe bus efficiency of older TLS offload NICs which did not keep state per-session, and so would need to re-DMA the first part(s) of a TLS record if a TLS record was sent in multiple TCP packets or TSOs. Newer TLS offload NICs do not need this feature. At Netflix, we've run extensive QoE tests which show that this feature reduces client quality metrics, presumably because the effort to send TLS records atomically causes the server to both wait too long to send data (leading to buffers running dry), and to send too much data at once (leading to packet loss). Reviewed by: hselasky, jhb, rrs Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D26103 Notes: svn path=/head/; revision=364405
* TCP: make after-idle work for transactional sessions.Richard Scheffenegger2020-06-241-1/+3
| | | | | | | | | | | | | | | | | | | The use of t_rcvtime as proxy for the last transmission fails for transactional IO, where the client requests data before the server can respond with a bulk transfer. Set aside a dedicated variable to actually track the last locally sent segment going forward. Reported by: rrs Reviewed by: rrs, tuexen (mentor) Approved by: tuexen (mentor), rgrimes (mentor) MFC after: 2 weeks Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D25016 Notes: svn path=/head/; revision=362577
* So it turns out with the right window scaling you can get the code in all ↵Randall Stewart2020-06-121-1/+4
| | | | | | | | | | | | | | | | | | | | | stacks to always want to do a window update, even when no data can be sent. Now in cases where you are not pacing thats probably ok, you just send an extra window update or two. However with bbr (and rack if its paced) every time the pacer goes off its going to send a "window update". Also in testing bbr I have found that if we are not responding to data right away we end up staying in startup but incorrectly holding a pacing gain of 192 (a loss). This is because the idle window code does not restict itself to only work with PROBE_BW. In all other states you dont want it doing a PROBE_BW state change. Sponsored by: Netflix Inc. Differential Revision: https://reviews.freebsd.org/D25247 Notes: svn path=/head/; revision=362113
* With RFC3168 ECN, CWR SHOULD only be sent with new dataRichard Scheffenegger2020-05-211-9/+10
| | | | | | | | | | | | | | | | Overly conservative data receivers may ignore the CWR flag on other packets, and keep ECE latched. This can result in continous reduction of the congestion window, and very poor performance when ECN is enabled. Reviewed by: rgrimes (mentor), rrs Approved by: rgrimes (mentor), tuexen (mentor) MFC after: 3 days Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D23364 Notes: svn path=/head/; revision=361347
* Handle ECN handshake in simultaneous openRichard Scheffenegger2020-05-211-0/+6
| | | | | | | | | | | | | | While testing simultaneous open TCP with ECN, found that negotiation fails to arrive at the expected final state. Reviewed by: tuexen (mentor) Approved by: tuexen (mentor), rgrimes (mentor) MFC after: 2 weeks Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D23373 Notes: svn path=/head/; revision=361345
* Step 4.2: start divorce of M_EXT and M_EXTPGGleb Smirnoff2020-05-031-1/+1
| | | | | | | | | | | | | They have more differencies than similarities. For now there is lots of code that would check for M_EXT only and work correctly on M_EXTPG buffers, so still carry M_EXT bit together with M_EXTPG. However, prepare some code for explicit check for M_EXTPG. Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598 Notes: svn path=/head/; revision=360583
* Step 4.1: mechanically rename M_NOMAP to M_EXTPGGleb Smirnoff2020-05-031-4/+4
| | | | | | | | Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598 Notes: svn path=/head/; revision=360581
* Step 3: anonymize struct mbuf_ext_pgs and move all its fields into mbufGleb Smirnoff2020-05-031-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | within m_epg namespace. All edits except the 'struct mbuf' declaration and mb_dupcl() were done mechanically with sed: s/->m_ext_pgs.nrdy/->m_epg_nrdy/g s/->m_ext_pgs.hdr_len/->m_epg_hdrlen/g s/->m_ext_pgs.trail_len/->m_epg_trllen/g s/->m_ext_pgs.first_pg_off/->m_epg_1st_off/g s/->m_ext_pgs.last_pg_len/->m_epg_last_len/g s/->m_ext_pgs.flags/->m_epg_flags/g s/->m_ext_pgs.record_type/->m_epg_record_type/g s/->m_ext_pgs.enc_cnt/->m_epg_enc_cnt/g s/->m_ext_pgs.tls/->m_epg_tls/g s/->m_ext_pgs.so/->m_epg_so/g s/->m_ext_pgs.seqno/->m_epg_seqno/g s/->m_ext_pgs.stailq/->m_epg_stailq/g Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598 Notes: svn path=/head/; revision=360579
* Prevent premature shrinking of the scaled receive windowRichard Scheffenegger2020-04-291-1/+4
| | | | | | | | | | | | | | | | | which can cause a TCP client to use invalid or stale TCP sequence numbers for ACK packets. Packets with old sequence numbers are ignored and not used to update the send window size. This might cause the TCP session to hang indefinitely under some circumstances. Reported by: Cui Cheng Reviewed by: tuexen (mentor), rgrimes (mentor) Approved by: tuexen (mentor), rgrimes (mentor) MFC after: 3 weeks Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D24515 Notes: svn path=/head/; revision=360479
* Convert route caching to nexthop caching.Alexander V. Chernikov2020-04-251-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This change is build on top of nexthop objects introduced in r359823. Nexthops are separate datastructures, containing all necessary information to perform packet forwarding such as gateway interface and mtu. Nexthops are shared among the routes, providing more pre-computed cache-efficient data while requiring less memory. Splitting the LPM code and the attached data solves multiple long-standing problems in the routing layer, drastically reduces the coupling with outher parts of the stack and allows to transparently introduce faster lookup algorithms. Route caching was (re)introduced to minimise (slow) routing lookups, allowing for notably better performance for large TCP senders. Caching works by acquiring rtentry reference, which is protected by per-rtentry mutex. If the routing table is changed (checked by comparing the rtable generation id) or link goes down, cache record gets withdrawn. Nexthops have the same reference counting interface, backed by refcount(9). This change merely replaces rtentry with the actual forwarding nextop as a cached object, which is mostly mechanical. Other moving parts like cache cleanup on rtable change remains the same. Differential Revision: https://reviews.freebsd.org/D24340 Notes: svn path=/head/; revision=360292
* KTLS: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself.Andrew Gallatin2020-04-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While the original implementation of unmapped mbufs was a large step forward in terms of reducing cache misses by enabling mbufs to carry more than a single page for sendfile, they are rather cache unfriendly when accessing the ext_pgs metadata and data. This is because the ext_pgs part of the mbuf is allocated separately, and almost guaranteed to be cold in cache. This change takes advantage of the fact that unmapped mbufs are never used at the same time as pkthdr mbufs. Given this fact, we can overlap the ext_pgs metadata with the mbuf pkthdr, and carry the ext_pgs meta directly in the mbuf itself. Similarly, we can carry the ext_pgs data (TLS hdr/trailer/array of pages) directly after the existing m_ext. In order to be able to carry 5 pages (which is the minimum required for a 16K TLS record which is not perfectly aligned) on LP64, I've had to steal ext_arg2. The only user of this in the xmit path is sendfile, and I've adjusted it to use arg1 when using unmapped mbufs. This change is almost entirely mechanical, except that we change mb_alloc_ext_pgs() to no longer allow allocating pkthdrs, the change to avoid ext_arg2 as mentioned above, and the removal of the ext_pgs zone, This change saves roughly 2% "raw" CPU (~59% -> 57%), or over 3% "scaled" CPU on a Netflix 100% software kTLS workload at 90+ Gb/s on Broadwell Xeons. In a follow-on commit, I plan to remove some hacks to avoid access ext_pgs fields of mbufs, since they will now be in cache. Many thanks to glebius for helping to make this better in the Netflix tree. Reviewed by: hselasky, jhb, rrs, glebius (early version) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D24213 Notes: svn path=/head/; revision=359919
* Avoid a cache miss accessing an mbuf ext_pgs pointer when doing SW kTLS.Andrew Gallatin2020-03-161-1/+1
| | | | | | | | | | | | For a Netflix 90Gb/s 100% TLS software kTLS workload, this reduces the CPI of tcp_m_copym() from ~3.5 to ~2.5 as reported by vtune. Reviewed by: jtl, rrs Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D23998 Notes: svn path=/head/; revision=359016
* sack_newdata and snd_recover hold the same value. Therefore, use onlyMichael Tuexen2020-02-131-1/+1
| | | | | | | | | | a single instance: use snd_recover also where sack_newdata was used. Submitted by: Richard Scheffenegger Differential Revision: https://reviews.freebsd.org/D18811 Notes: svn path=/head/; revision=357858
* White space cleanup -- remove trailing tab's or spacesRandall Stewart2020-02-121-12/+12
| | | | | | | | | from any line. Sponsored by: Netflix Inc. Notes: svn path=/head/; revision=357818
* Don't set the ECT codepoint on retransmitted packets during SACK lossMichael Tuexen2020-01-251-0/+1
| | | | | | | | | | | | recovery. This is required by RFC 3168. Submitted by: Richard Scheffenegger Reviewed by: rgrimes@, tuexen@, Cheng Cui MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D23118 Notes: svn path=/head/; revision=357115
* Make tcp_output() require network epoch.Gleb Smirnoff2020-01-221-3/+1
| | | | | | | | | | Enter the epoch before calling into tcp_output() from those functions, that didn't do that before. This eliminates a bunch of epoch recursions in TCP. Notes: svn path=/head/; revision=356975
* Make ip6_output() and ip_output() require network epoch.Gleb Smirnoff2020-01-221-0/+3
| | | | | | | | All callers that before may called into these functions without network epoch now must enter it. Notes: svn path=/head/; revision=356974
* Make use of the stats(3) framework in the TCP stack.Edward Tomasz Napierala2019-12-021-2/+30
| | | | | | | | | | | | | | | | | | | | | | | | | This makes it possible to retrieve per-connection statistical information such as the receive window size, RTT, or goodput, using a newly added TCP_STATS getsockopt(3) option, and extract them using the stats_voistat_fetch(3) API. See the net/tcprtt port for an example consumer of this API. Compared to the existing TCP_INFO system, the main differences are that this mechanism is easy to extend without breaking ABI, and provides statistical information instead of raw "snapshots" of values at a given point in time. stats(3) is more generic and can be used in both userland and the kernel. Reviewed by: thj Tested by: thj Obtained from: Netflix Relnotes: yes Sponsored by: Klara Inc, Netflix Differential Revision: https://reviews.freebsd.org/D20655 Notes: svn path=/head/; revision=355304
* Move all ECN related flags from the flags to the flags2 field.Michael Tuexen2019-12-011-4/+4
| | | | | | | | | | | | This allows adding more ECN related flags in the future. No functional change intended. Submitted by: Richard Scheffenegger Reviewed by: rrs@, tuexen@ Differential Revision: https://reviews.freebsd.org/D22497 Notes: svn path=/head/; revision=355273
* RFC 7112 requires a host to put the complete IP header chainMichael Tuexen2019-09-291-0/+14
| | | | | | | | | | | | | | | | including the TCP header in the first IP packet. Enforce this in tcp_output(). In addition make sure that at least one byte payload fits in the TCP segement to allow making progress. Without this check, a kernel with INVARIANTS will panic. This issue was found by running an instance of syzkaller. Reviewed by: jtl@ MFC after: 3 days Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D21665 Notes: svn path=/head/; revision=352868
* Initialize if_hw_tsomaxsegsize to 0 to appease gcc's flow analysis as aWarner Losh2019-09-061-1/+1
| | | | | | | fail-safe. Notes: svn path=/head/; revision=351950
* Add kernel-side support for in-kernel TLS.John Baldwin2019-08-271-3/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KTLS adds support for in-kernel framing and encryption of Transport Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports offload of TLS for transmitted data. Key negotation must still be performed in userland. Once completed, transmit session keys for a connection are provided to the kernel via a new TCP_TXTLS_ENABLE socket option. All subsequent data transmitted on the socket is placed into TLS frames and encrypted using the supplied keys. Any data written to a KTLS-enabled socket via write(2), aio_write(2), or sendfile(2) is assumed to be application data and is encoded in TLS frames with an application data type. Individual records can be sent with a custom type (e.g. handshake messages) via sendmsg(2) with a new control message (TLS_SET_RECORD_TYPE) specifying the record type. At present, rekeying is not supported though the in-kernel framework should support rekeying. KTLS makes use of the recently added unmapped mbufs to store TLS frames in the socket buffer. Each TLS frame is described by a single ext_pgs mbuf. The ext_pgs structure contains the header of the TLS record (and trailer for encrypted records) as well as references to the associated TLS session. KTLS supports two primary methods of encrypting TLS frames: software TLS and ifnet TLS. Software TLS marks mbufs holding socket data as not ready via M_NOTREADY similar to sendfile(2) when TLS framing information is added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then called to schedule TLS frames for encryption. In the case of sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving the mbufs marked M_NOTREADY until encryption is completed. For other writes (vn_sendfile when pages are available, write(2), etc.), the PRUS_NOTREADY is set when invoking pru_send() along with invoking ktls_enqueue(). A pool of worker threads (the "KTLS" kernel process) encrypts TLS frames queued via ktls_enqueue(). Each TLS frame is temporarily mapped using the direct map and passed to a software encryption backend to perform the actual encryption. (Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if someone wished to make this work on architectures without a direct map.) KTLS supports pluggable software encryption backends. Internally, Netflix uses proprietary pure-software backends. This commit includes a simple backend in a new ktls_ocf.ko module that uses the kernel's OpenCrypto framework to provide AES-GCM encryption of TLS frames. As a result, software TLS is now a bit of a misnomer as it can make use of hardware crypto accelerators. Once software encryption has finished, the TLS frame mbufs are marked ready via pru_ready(). At this point, the encrypted data appears as regular payload to the TCP stack stored in unmapped mbufs. ifnet TLS permits a NIC to offload the TLS encryption and TCP segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS) is allocated on the interface a socket is routed over and associated with a TLS session. TLS records for a TLS session using ifnet TLS are not marked M_NOTREADY but are passed down the stack unencrypted. The ip_output_send() and ip6_output_send() helper functions that apply send tags to outbound IP packets verify that the send tag of the TLS record matches the outbound interface. If so, the packet is tagged with the TLS send tag and sent to the interface. The NIC device driver must recognize packets with the TLS send tag and schedule them for TLS encryption and TCP segmentation. If the the outbound interface does not match the interface in the TLS send tag, the packet is dropped. In addition, a task is scheduled to refresh the TLS send tag for the TLS session. If a new TLS send tag cannot be allocated, the connection is dropped. If a new TLS send tag is allocated, however, subsequent packets will be tagged with the correct TLS send tag. (This latter case has been tested by configuring both ports of a Chelsio T6 in a lagg and failing over from one port to another. As the connections migrated to the new port, new TLS send tags were allocated for the new port and connections resumed without being dropped.) ifnet TLS can be enabled and disabled on supported network interfaces via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported across both vlan devices and lagg interfaces using failover, lacp with flowid enabled, or lacp with flowid enabled. Applications may request the current KTLS mode of a connection via a new TCP_TXTLS_MODE socket option. They can also use this socket option to toggle between software and ifnet TLS modes. In addition, a testing tool is available in tools/tools/switch_tls. This is modeled on tcpdrop and uses similar syntax. However, instead of dropping connections, -s is used to force KTLS connections to switch to software TLS and -i is used to switch to ifnet TLS. Various sysctls and counters are available under the kern.ipc.tls sysctl node. The kern.ipc.tls.enable node must be set to true to enable KTLS (it is off by default). The use of unmapped mbufs must also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS. KTLS is enabled via the KERN_TLS kernel option. This patch is the culmination of years of work by several folks including Scott Long and Randall Stewart for the original design and implementation; Drew Gallatin for several optimizations including the use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records awaiting software encryption, and pluggable software crypto backends; and John Baldwin for modifications to support hardware TLS offload. Reviewed by: gallatin, hselasky, rrs Obtained from: Netflix Sponsored by: Netflix, Chelsio Communications Differential Revision: https://reviews.freebsd.org/D21277 Notes: svn path=/head/; revision=351522
* This is the second in a number of patches needed toRandall Stewart2019-07-141-1/+7
| | | | | | | | | | | | | | | | get BBRv1 into the tree. This fixes the DSACK bug but is also needed by BBR. We have yet to go two more one will be for the pacing code (tcp_ratelimit.c) and the second will be for the new updated LRO code that allows a transport to know the arrival times of packets and (tcp_lro.c). After that we should finally be able to get BBRv1 into head. Sponsored by: Netflix Inc Differential Revision: https://reviews.freebsd.org/D20908 Notes: svn path=/head/; revision=349987
* Undo my previous erroneous commit changing the tcp_output kassert.Randall Stewart2019-04-031-2/+1
| | | | | | | Hmm now the question is where did the tcp_log_id change go :o Notes: svn path=/head/; revision=345851
* Fix a small bug in the tcp_log_id where the bucketRandall Stewart2019-03-261-1/+2
| | | | | | | | | | | | | was unlocked and yet the bucket-unlock flag was not changed to false. This can cause a panic if INVARIANTS is on and we go through the right path (though rare). Reported by: syzbot+179a1ad49f3c4c215fa2@syzkaller.appspotmail.com Reviewed by: tuexen@ MFC after: 1 week Notes: svn path=/head/; revision=345527
* Fix a KASSERT() in tcp_output().Michael Tuexen2019-03-231-9/+3
| | | | | | | | | | | | | | | | When checking the length of the headers at this point, the IP level options have not been added to the mbuf chain. So don't take them into account. Reported by: syzbot+16025fff7ee5f7c5957b@syzkaller.appspotmail.com Reported by: syzbot+adb5836b8a9ff621b2aa@syzkaller.appspotmail.com Reported by: syzbot+d25a5352bcdf40acdbb8@syzkaller.appspotmail.com Reviewed by: rrs@ MFC after: 3 days Sponsored by: Netflix, Inc. Notes: svn path=/head/; revision=345441
* Fix window update issue when scaling disabledStephen Hurd2019-01-151-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | When the TCP window scale option is not used, and the window opens up enough in one soreceive, a window update will not be sent. For example, if recwin == 65535, so->so_rcv.sb_hiwat >= 262144, and so->so_rcv.sb_hiwat <= 524272, the window update will never be sent. This is because recwin and adv are clamped to TCP_MAXWIN << tp->rcv_scale, and so will never be >= so->so_rcv.sb_hiwat / 4 or <= so->so_rcv.sb_hiwat / 8. This patch ensures a window update is sent if the window opens by TCP_MAXWIN << tp->rcv_scale, which should only happen when the window size goes from zero to the max expressible. This issue looks like it was introduced in r306769 when recwin was clamped to TCP_MAXWIN << tp->rcv_scale. MFC after: 1 week Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D18821 Notes: svn path=/head/; revision=343047
* Ensure that TCP RST-segments announce consistently a receiver window ofMichael Tuexen2018-11-221-6/+10
| | | | | | | | | | | | zero. This was already done when sending them via tcp_respond(). Reviewed by: rrs@ MFC after: 1 week Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D17949 Notes: svn path=/head/; revision=340774
* A couple of style fixes in recent TCP changes.John Baldwin2018-10-221-0/+1
| | | | | | | | | | | | | - Add a blank line before a block comment to match other block comments in the same function. - Sort the prototype for sbsndptr_adv and fix whitespace between return type and function name. Reviewed by: gallatin, bz Differential Revision: https://reviews.freebsd.org/D17474 Notes: svn path=/head/; revision=339619
* Fix some TCP fast open issues.Michael Tuexen2018-07-301-5/+7
| | | | | | | | | | | | | | | | | | | | The following issues are fixed: * Whenever a TCP server with TCP fast open enabled, calls accept(), recv(), send(), and close() before the TCP-ACK segment has been received, the TCP connection is just dropped and the reception of the TCP-ACK segment triggers the sending of a TCP-RST segment. * Whenever a TCP server with TCP fast open enabled, calls accept(), recv(), send(), send(), and close() before the TCP-ACK segment has been received, the first byte provided in the second send call is not transferred. * Whenever a TCP client with TCP fast open enabled calls sendto() followed by close() the TCP connection is just dropped. Reviewed by: jtl@, kbowling@, rrs@ Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D16485 Notes: svn path=/head/; revision=336934
* Revert r334843, and partially revert r335180.Gleb Smirnoff2018-06-231-20/+1
| | | | | | | | tcp_outflags[] were defined since 4BSD and are defined nowadays in all its descendants. Removing them breaks third party application. Notes: svn path=/head/; revision=335577
* This adds in an optimization so that we only walk oneRandall Stewart2018-06-211-66/+22
| | | | | | | | | | time through the mbuf chain during copy and TSO limiting. It is used by both Rack and now the FreeBSD stack. Sponsored by: Netflix Inc Differential Revision: https://reviews.freebsd.org/D15937 Notes: svn path=/head/; revision=335502
* TCPOUTFLAGS no longer exists since r334843.Gleb Smirnoff2018-06-141-1/+0
| | | | Notes: svn path=/head/; revision=335180
* avoid 'tcp_outflags defined but not used'Matt Macy2018-06-081-0/+20
| | | | Notes: svn path=/head/; revision=334843
* This commit brings in a new refactored TCP stack called Rack.Randall Stewart2018-06-071-6/+139
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rack includes the following features: - A different SACK processing scheme (the old sack structures are not used). - RACK (Recent acknowledgment) where counting dup-acks is no longer done instead time is used to knwo when to retransmit. (see the I-D) - TLP (Tail Loss Probe) where we will probe for tail-losses to attempt to try not to take a retransmit time-out. (see the I-D) - Burst mitigation using TCPHTPS - PRR (partial rate reduction) see the RFC. Once built into your kernel, you can select this stack by either socket option with the name of the stack is "rack" or by setting the global sysctl so the default is rack. Note that any connection that does not support SACK will be kicked back to the "default" base FreeBSD stack (currently known as "default"). To build this into your kernel you will need to enable in your kernel: makeoptions WITH_EXTRA_TCP_STACKS=1 options TCPHPTS Sponsored by: Netflix Inc. Differential Revision: https://reviews.freebsd.org/D15525 Notes: svn path=/head/; revision=334804
* Fix spurious retransmit recovery on low latency networksMatt Macy2018-05-081-2/+5
| | | | | | | | | | | | | | | | | | TCP's smoothed RTT (SRTT) can be much larger than an actual observed RTT. This can be either because of hz restricting the calculable RTT to 10ms in VMs or 1ms using the default 1000hz or simply because SRTT recently incorporated a larger value. If an ACK arrives before the calculated badrxtwin (now + SRTT): tp->t_badrxtwin = ticks + (tp->t_srtt >> (TCP_RTT_SHIFT + 1)); We'll erroneously reset snd_una to snd_max. If multiple segments were dropped and this happens repeatedly the transmit rate will be limited to 1MSS per RTO until we've retransmitted all drops. Reported by: rstone Reviewed by: hiren, transport Approved by: sbruno MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D8556 Notes: svn path=/head/; revision=333346
* Immediately propagate EACCES error code to application from tcp_output.Andrey V. Elsukov2018-05-041-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | In r309610 and r315514 the behavior of handling EACCES was changed, and tcp_output() now returns zero when EACCES happens. The reason of this change was a hesitation that applications that use TCP-MD5 will be affected by changes in project/ipsec. TCP-MD5 code returns EACCES when security assocition for given connection is not configured. But the same error code can return pfil(9), and this change has affected connections blocked by pfil(9). E.g. application doesn't return immediately when SYN segment is blocked, instead it waits when several tries will be failed. Actually, for TCP-MD5 application it doesn't matter will it get EACCES after first SYN, or after several tries. Security associtions must be configured before initiating TCP connection. I left the EACCES in the switch() to show that it has special handling. Reported by: Andreas Longwitz <longwitz at incore dot de> MFC after: 10 days Notes: svn path=/head/; revision=333244
* Move the TCP Blackbox Recorder probe in tcp_output.c to be with theJonathan T. Looney2018-04-101-4/+4
| | | | | | | | | other tracing/debugging code. Sponsored by: Netflix, Inc. Notes: svn path=/head/; revision=332379
* Add the "TCP Blackbox Recorder" which we discussed at the developerJonathan T. Looney2018-03-221-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | summits at BSDCan and BSDCam in 2017. The TCP Blackbox Recorder allows you to capture events on a TCP connection in a ring buffer. It stores metadata with the event. It optionally stores the TCP header associated with an event (if the event is associated with a packet) and also optionally stores information on the sockets. It supports setting a log ID on a TCP connection and using this to correlate multiple connections that share a common log ID. You can log connections in different modes. If you are doing a coordinated test with a particular connection, you may tell the system to put it in mode 4 (continuous dump). Or, if you just want to monitor for errors, you can put it in mode 1 (ring buffer) and dump all the ring buffers associated with the connection ID when we receive an error signal for that connection ID. You can set a default mode that will be applied to a particular ratio of incoming connections. You can also manually set a mode using a socket option. This commit includes only basic probes. rrs@ has added quite an abundance of probes in his TCP development work. He plans to commit those soon. There are user-space programs which we plan to commit as ports. These read the data from the log device and output pcapng files, and then let you analyze the data (and metadata) in the pcapng files. Reviewed by: gnn (previous version) Obtained from: Netflix, Inc. Relnotes: yes Differential Revision: https://reviews.freebsd.org/D11085 Notes: svn path=/head/; revision=331347
* Greatly reduce the number of #ifdefs supporting the TCP_RFC7413 kernel option.Patrick Kelsey2018-02-261-18/+2
| | | | | | | | | | | | | | | | | | | | | The conditional compilation support is now centralized in tcp_fastopen.h and tcp_var.h. This doesn't provide the minimum theoretical code/data footprint when TCP_RFC7413 is disabled, but nearly all the TFO code should wind up being removed by the optimizer, the additional footprint in the syncache entries is a single pointer, and the additional overhead in the tcpcb is at the end of the structure. This enables the TCP_RFC7413 kernel option by default in amd64 and arm64 GENERIC. Reviewed by: hiren MFC after: 1 month Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D14048 Notes: svn path=/head/; revision=330002
* This is an implementation of the client side of TCP Fast Open (TFO)Patrick Kelsey2018-02-261-19/+65
| | | | | | | | | | | | | | | | | [RFC7413]. It also includes a pre-shared key mode of operation in which the server requires the client to be in possession of a shared secret in order to successfully open TFO connections with that server. The names of some existing fastopen sysctls have changed (e.g., net.inet.tcp.fastopen.enabled -> net.inet.tcp.fastopen.server_enable). Reviewed by: tuexen MFC after: 1 month Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D14047 Notes: svn path=/head/; revision=330001
* Do pass removing some write-only variables from the kernel.Alexander Kabaev2017-12-251-0/+4
| | | | | | | | | | | | This reduces noise when kernel is compiled by newer GCC versions, such as one used by external toolchain ports. Reviewed by: kib, andrew(sys/arm and sys/arm64), emaste(partial), erj(partial) Reviewed by: jhb (sys/dev/pci/* sys/kern/vfs_aio.c and sys/kern/kern_synch.c) Differential Revision: https://reviews.freebsd.org/D10385 Notes: svn path=/head/; revision=327173
* Fix mbuf leak when TCPMD5_OUTPUT() method returns error.Andrey V. Elsukov2017-12-141-2/+3
| | | | | | | | PR: 223817 MFC after: 1 week Notes: svn path=/head/; revision=326847
* Separate out send buffer autoscaling code into function, so thatGleb Smirnoff2017-12-071-53/+60
| | | | | | | | | alternative TCP stacks may reuse it instead of pasting. Obtained from: Netflix Notes: svn path=/head/; revision=326673
* sys: further adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-201-0/+2
| | | | | | | | | | | | | | | | | Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point. Notes: svn path=/head/; revision=326023
* Style r320614: don't initialize at declaration, new line after declarations,Gleb Smirnoff2017-11-131-5/+5
| | | | | | | | shorten variable name to avoid extra long lines. No functional changes. Notes: svn path=/head/; revision=325782
* Declare more TCP globals in tcp_var.h, so that alternative TCP stacksGleb Smirnoff2017-10-111-4/+0
| | | | | | | | | | can use them. Gather all TCP tunables in tcp_var.h in one place and alphabetically sort them, to ease maintainance of the list. Don't copy and paste declarations in tcp_stacks/fastpath.c. Notes: svn path=/head/; revision=324539
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-05 11:44:29 +0000