FreeBSD source tree https://cgit-dev.freebsd.org/src/atom?h=releng%2F12.2 2020-01-07T00:54:23Z 2020-01-07T00:54:23Z Alexander Motin mav@FreeBSD.org 2020-01-07T00:54:23Z urn:sha1:2617fcf388234340d8ae3e37b67e72e31fe57660 If somebody else holds that lock, it will likely do the work for us. If it won't, then we return here later and retry. Under heavy load it allows to avoid lock congestion between interrupt and polling threads. 2019-11-29T01:00:06Z Alexander Motin mav@FreeBSD.org 2019-11-29T01:00:06Z urn:sha1:8c2de3473aae6d122fa4785012d2c86334a7a16c - Deduce allowed address range for bus_dma(9) from the hardware version. Different versions (CPU generations) have different documented limits. - Remove difference between address ranges for src/dst and crc. At least docs for few recent generations of CPUs do not mention anything like that, while older are already limited with above limits. - Remove address assertions from arguments. While I do not think the addresses out of allowed ranges should realistically happen there due to the platforms physical address limitations, there is now bus_dma(9) to make sure of that, preferably via IOMMU. - Since crc now has the same address range as src/dst, remove crc_dmamap, reusing dst2_dmamap instead. 2019-11-29T00:58:18Z Alexander Motin mav@FreeBSD.org 2019-11-29T00:58:18Z urn:sha1:2c74e433ec59b983b3821feea6000686f7345fb4 ioat(4) should use bus_dma(9) for the operation source and destination addresses 2019-11-29T00:38:16Z Alexander Motin mav@FreeBSD.org 2019-11-29T00:38:16Z urn:sha1:8bfe05b80511d9b95a1bf6b347632f34c230bf2b ioatcontrol(8) could exercise 8k-aligned copy with page-break, crc and crc-copy modes. 2019-11-29T00:25:39Z Alexander Motin mav@FreeBSD.org 2019-11-29T00:25:39Z urn:sha1:1fa8f59c4f7d6d74cbef8ee2d2c1641ade031240 I've noticed that sometimes with enabled DMAR initial write from device to this address is somehow getting delayed, triggering assertion due to zero default being invalid. 2019-11-28T00:42:45Z Alexander Motin mav@FreeBSD.org 2019-11-28T00:42:45Z urn:sha1:3844374396fbe1abb58f8eccaadd9977e4779800 2019-11-26T15:23:24Z Alexander Motin mav@FreeBSD.org 2019-11-26T15:23:24Z urn:sha1:d83ad00e6b383067c48a1116c55395969a54fd1d This allows NUMA-aware consumers to reduce inter-domain traffic. 2019-10-15T18:39:32Z Alexander Motin mav@FreeBSD.org 2019-10-15T18:39:32Z urn:sha1:5e17786d6a55ab4b3025efc32a97ed1fb3c6b22b Those functions are used by kernel, and we can't check all possible argument errors in production kernel. Plus according to docs many of those errors are checked by hardware. Assertions should just help with code debugging. 2019-09-28T00:59:39Z Alexander Motin mav@FreeBSD.org 2019-09-28T00:59:39Z urn:sha1:cd6b397159844962e00bf126638cddc4e2ba7558 Allocate ioat->ring memory from the device domain. Schedule ioat->poll_timer to the first CPU of the device domain. According to pcm-numa tool from intel-pcm port, this reduces number of remote DRAM accesses while copying data by 75%. And unless it is a noise, I've noticed some speed improvement when copying data to other domain. 2019-02-28T00:28:44Z Alexander Motin mav@FreeBSD.org 2019-02-28T00:28:44Z urn:sha1:91aa7b1775354c67cc3e4ef2ccdcffc358e386f9 - Do not explicitly count active descriptors. It allows hardware reset to happen while device is still referenced, plus simplifies locking. - Do not stop/start callout each time the queue becomes empty. Let it run to completion and rearm if needed, that is much cheaper then to touch it every time, plus also simplifies locking. - Decouple submit and cleanup locks, making driver reentrant. - Avoid memory mapped status register read on every interrupt. - Improve locking during device attach/detach. - Remove some no longer used variables. Sponsored by: iXsystems, Inc.