FreeBSD source tree https://cgit-dev.freebsd.org/src/atom?h=releng%2F12.4 2022-06-10T12:37:23Z 2022-06-10T12:37:23Z Gordon Bergling gbe@FreeBSD.org 2022-06-04T09:46:03Z urn:sha1:ef3d4e2b1c541e902afc7dfbfc03128b150cebbe - s/inaccessable/inaccessible/ (cherry picked from commit 6e8ab6715d3589ad5577ad5fa408f81b4a7077e9) 2022-04-14T06:33:30Z Gordon Bergling gbe@FreeBSD.org 2022-04-08T19:02:19Z urn:sha1:95c83fc4ccdfe7cf938c10bf931777b1a041b5cf - s/is is/is/ (cherry picked from commit dfa01f4f98a1343bf375c54a5cd44718c4211bec) 2021-12-03T15:57:29Z Gordon Bergling gbe@FreeBSD.org 2021-11-30T09:26:25Z urn:sha1:d891ee6ac7e6af693959c6d11c5af7c440919ea9 - s/printting/printing/ (cherry picked from commit 5f8ccf6515288fd98896eaf0511eedd654cd3530) 2021-09-07T01:25:34Z Alexander Motin mav@FreeBSD.org 2021-08-31T17:34:48Z urn:sha1:900b7ac652b3f5c8bcf42621aa07b12ae0d883ea If we can't allocate more MSI-X vectors, accept using single shared. If we can't allocate any MSI-X, try to allocate 2 MSI vectors, but accept single shared. If still no luck, fall back to shared INTx. This provides maximal flexibility in some limited scenarios. For example, vmd(4) does not support INTx and can handle only limited number of MSI/MSI-X vectors without sharing. MFC after: 1 week (cherry picked from commit e3bdf3da769a55f0944d9c337bb4d91b6435f02c) 2021-09-07T01:25:32Z Alexander Motin mav@FreeBSD.org 2021-08-31T00:36:22Z urn:sha1:4ccde1f0f101412e4f9072e302aadeaa19a63f18 MFC after: 1 week (cherry picked from commit 31111372e6bad7212dbee36dd312e3b53fdfd3f6) 2021-07-31T00:02:53Z Warner Losh imp@FreeBSD.org 2021-07-02T22:00:42Z urn:sha1:f76c34659f9d94683e1161b8913b6466a2fdbaba Coherently read the phase bit of the status completion record. We loop over the completion record array, looking for all the transactions in the same phase that have been completed. In doing that, we have to be careful to read the status field first, and if it indicates a complete record, we need to read and process that record. Otherwise, the host might be overtaken by device when reading this completion record, leading to a mistaken belief that the record is in phase. This leads to the code using old values and looking at an already completed entry, which has no current tracker. To work around this problem, we read the status and make sure it is in phase, we then re-read the entire completion record guaranteeing it's complete, valid, and consistent . In addition we resync the dmatag to reflect changes since the prior loop for the bouncing dma case. Reviewed by: jrtc27@, chuck@ Found by: jrtc27 (this fix is based in part on her D30995 fix) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D31002 (cherry picked from commit aa0ab681ae755e01cd69435fab50f6852f248c42) 2021-07-31T00:02:53Z Warner Losh imp@FreeBSD.org 2021-07-02T21:58:19Z urn:sha1:d85d7244d8bb20d5d5040045856c8038333736ce Remove __packed from nvme_command, nvme_completion and nvme_dsm_trim. Add super-alignment to nvme_completion since it's always at least that aligned in hardware (and in our existing uses of it embedded in structures). It generates better code in nvme_qpair_process_completions on riscv64 because otherwise the ABI assumes a 4-byte alignment, and the same on all other platforms. Reviewed by: jrtc27@, mav@, chuck@ Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D31001 (cherry picked from commit fea3cf1d6da0acf40bc1d3dadeeea7eeccbc10dd) 2021-07-31T00:02:53Z Warner Losh imp@FreeBSD.org 2021-07-02T21:58:14Z urn:sha1:4fad4eeb75fe440a9f307af4804cc211a451e603 Put the { on the same line as the struct nvme_foo when we define these structures. It's FreeBSD standard and these were inconsistent. Sponsored by: Netflix (cherry picked from commit 80a75155e1601bddc2c595c06ab6ea916c603071) 2021-07-31T00:02:53Z Warner Losh imp@FreeBSD.org 2021-05-29T05:01:52Z urn:sha1:225c0eda7efb73c9af75fc54c434b0ec694f506a Part of the nvme recovery process for errors is to reset the card. Sometimes, this results in failing the entire controller. When nda is in use, we free the sim, which will sleep until all the I/O has completed. However, with only one thread, the request fail task never runs once the reset thread sleeps here. Create two threads to allow I/O to fail until it's all processed and the reset task can proceed. This is a temporary kludge until I can work out questions that arose during the review, not least is what was the race that queueing to a failure task solved. The original commit is vague and other error paths in the same context do a direct failure. I'll investigate that more completely before committing changing that to a direct failure. mav@ raised this issue during the review, but didn't otherwise object. Multiple threads, though, solve the problem in the mean time until other such means can be perfected. Reviewed by: jhb@ Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D30366 (cherry picked from commit f0f47121653e88197d8537572294b90f5aef7f17) 2021-07-31T00:02:53Z Warner Losh imp@FreeBSD.org 2021-03-11T15:42:44Z urn:sha1:5b59c5186902f8ac402b89796567fa630abb10c2 nvme drives are configured early in boot. However, a number of the configuration steps takes which take a while, so we defer those to a config intrhook that runs before the root filesystem is mounted. At the same time, the PCI hot plug wakes up and tests the status of the card. It may decide that the card has gone away and deletes the child. As part of that process nvme_detach is called. If this call happens after the config_intrhook starts to run, but before it is finished, there's a race where we can tear down the device's soft state while the config_intrhook is still using it. Use the new config_intrhook_drain to disestablish the hook. Either it will be removed w/o running, or the routine will wait for it to finish. This closes the race and allows safe hotplug at any time, even very early in boot. Sponsored by: Netflix, Inc Reviewed by: jhb, mav Differential Revision: https://reviews.freebsd.org/D29006 (cherry picked from commit 8423f5d4c127f18e7500bc455bc7b6b1691385ef)