summaryrefslogtreecommitdiff
path: root/sys/kern/subr_taskqueue.c
Commit message (Collapse)AuthorAgeFilesLines
* Copy releng/7.3 to release/7.3.0 to mark 7.3-RELEASE.release/7.3.0_cvsKen Smith2010-03-211-1/+1
| | | | | | Approved by: re (implicit) This commit was manufactured to restore the state of the 7.3-RELEASE image.
* MFC 198411:John Baldwin2010-01-211-2/+2
| | | | | | | | | | | | | | - Fix several off-by-one errors when using MAXCOMLEN. The p_comm[] and td_name[] arrays are actually MAXCOMLEN + 1 in size and a few places that created shadow copies of these arrays were just using MAXCOMLEN. - Prefer using sizeof() of an array type to explicit constants for the array length in a few places. - Ensure that all of p_comm[] is always zero'd during execve() to guard against any possible information leaks. Previously trailing garbage in p_comm[] could be leaked to userland in ktrace record headers. Notes: svn path=/stable/7/; revision=202765
* MFC several ZFS related commits:Alexander Leidinger2010-01-061-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - taskq changes - fixes for race conditions - locking fixes - bug fixes - ... r185310: ---snip--- Remove unused variable. Found with: Coverity Prevent(tm) CID: 3669,3671 ---snip--- r185319: ---snip--- Fix locking (file descriptor table and Giant around VFS). Most submitted by: kib Reviewed by: kib ---snip--- r192689: ---snip--- Fix comment. ---snip--- r193110: ---snip--- work around snapshot shutdown race reported by Henri Hennebert ---snip--- r193440: ---snip--- Support shared vnode locks for write operations when the offset is provided on filesystems that support it. This really improves mysql + innodb performance on ZFS. Reviewed by: jhb, kmacy, jeffr ---snip--- ATTENTION: this commit to releng7 does not allow shared vnode locks (there are some VFS changes needed before it can be enabled), it only provides the infrastructure and serves mostly as a diff reduction in the ZFS code. A comment has been added to the locking part to explain why no shared locks are used. r195627: ---snip--- In nvpair_native_embedded_array(), meaningless pointers are zeroed. The programmer was aware that alignment was not guaranteed in the packed structure and used bzero() to NULL out the pointers. However, on ia64, the compiler is quite agressive in finding ILP and calls to bzero() are often replaced by simple assignments (i.e. stores). Especially when the width or size in question corresponds with a store instruction (i.e. st1, st2, st4 or st8). The problem here is not a compiler bug. The address of the memory to zero-out was given by '&packed->nvl_priv' and given the type of the 'packed' pointer the compiler could assume proper alignment for the replacement of bzero() with an 8-byte wide store to be valid. The problem is with the programmer. The programmer knew that the address did not have the alignment guarantees needed for a regular assignment, but failed to inform the compiler of that fact. In fact, the programmer told the compiler the opposite: alignment is guaranteed. The fix is to avoid using a pointer of type "nvlist_t *" and instead use a "char *" pointer as the basis for calculating the address. This tells the compiler that only 1-byte alignment can be assumed and the compiler will either keep the bzero() call or instead replace it with a sequence of byte-wise stores. Both are valid. ---snip--- r195822: ---snip--- Fix extattr_list_file(2) on ZFS in case the attribute directory doesn't exist and user doesn't have write access to the file. Without this fix, it returns bogus value instead of 0. For some reason this didn't manifest on my kernel compiled with -O0. PR: kern/136601 Submitted by: Jaakko Heinonen <jh at saunalahti dot fi> ---snip--- r195909 ---snip--- We don't support ephemeral IDs in FreeBSD and without this fix ZFS can panic when in zfs_fuid_create_cred() when userid is negative. It is converted to unsigned value which makes IS_EPHEMERAL() macro to incorrectly report that this is ephemeral ID. The most reasonable solution for now is to always report that the given ID is not ephemeral. PR: kern/132337 Submitted by: Matthew West <freebsd@r.zeeb.org> Tested by: Thomas Backman <serenity@exscape.org>, Michael Reifenberger <mike@reifenberger.com> ---snip--- r196291: ---snip--- - Fix a race where /dev/zfs control device is created before ZFS is fully initialized. Also destroy /dev/zfs before doing other deinitializations. - Initialization through taskq is no longer needed and there is a race where one of the zpool/zfs command loads zfs.ko and tries to do some work immediately, but /dev/zfs is not there yet. Reported by: pav ---snip--- r196269: ---snip--- Fix misalignment in nvpair_native_embedded() caused by the compiler replacing the bzero(). See also revision 195627, which fixed the misalignment in nvpair_native_embedded_array(). ---snip--- r196295: ---snip--- Remove OpenSolaris taskq port (it performs very poorly in our kernel) and replace it with wrappers around our taskqueue(9). To make it possible implement taskqueue_member() function which returns 1 if the given thread was created by the given taskqueue. ---snip--- The taskqueue_member() function is different due to kproc/kthread changes in releng8 and head, the function was... Revieved by: jhb r196297: ---snip--- Fix panic in zfs recv code. The last vnode (mountpoint's vnode) can have 0 usecount. Reported by: Thomas Backman <serenity@exscape.org> ---snip--- r196299: ---snip--- - We need to recycle vnode instead of freeing znode. Submitted by: avg - Add missing vnode interlock unlock. - Remove redundant znode locking. ---snip--- r196301: ---snip--- If z_buf is NULL, we should free znode immediately. Noticed by: avg ---snip--- r196307: ---snip--- Manage asynchronous vnode release just like Solaris. Discussed with: kmacy ---snip--- Notes: svn path=/stable/7/; revision=201633
* MFC r188548Andrew Thompson2009-03-111-2/+2
| | | | | | | | | Check the exit flag at the start of the taskqueue loop rather than the end. It is possible to tear down the taskqueue before the thread has run and the taskqueue loop would sleep forever. Notes: svn path=/stable/7/; revision=189658
* Merge rev 1.41, 1.18: Implement taskqueue_block, taskqueue_unblock.Scott Long2008-08-031-1/+28
| | | | Notes: svn path=/stable/7/; revision=181241
* Commit 14/14 of sched_lock decomposition.Jeff Roberson2007-06-051-2/+2
| | | | | | | | | | | | | | - Use thread_lock() rather than sched_lock for per-thread scheduling sychronization. - Use the per-process spinlock rather than the sched_lock for per-process scheduling synchronization. Tested by: kris, current@ Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc. Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each) Notes: svn path=/head/; revision=170307
* - Remove setrunqueue and replace it with direct calls to sched_add().Jeff Roberson2007-01-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | setrunqueue() was mostly empty. The few asserts and thread state setting were moved to the individual schedulers. sched_add() was chosen to displace it for naming consistency reasons. - Remove adjustrunqueue, it was 4 lines of code that was ifdef'd to be different on all three schedulers where it was only called in one place each. - Remove the long ifdef'd out remrunqueue code. - Remove the now redundant ts_state. Inspect the thread state directly. - Don't set TSF_* flags from kern_switch.c, we were only doing this to support a feature in one scheduler. - Change sched_choose() to return a thread rather than a td_sched. Also, rely on the schedulers to return the idlethread. This simplifies the logic in choosethread(). Aside from the run queue links kern_switch.c mostly does not care about the contents of td_sched. Discussed with: julian - Move the idle thread loop into the per scheduler area. ULE wants to do something different from the other schedulers. Suggested by: jhb Tested on: x86/amd64 sched_{4BSD, ULE, CORE}. Notes: svn path=/head/; revision=166188
* When starting up threads in taskqueue_start_threads create themSam Leffler2006-05-241-9/+16
| | | | | | | | | | | | | | | stopped before adjusting their priority and setting them on the run q so they cannot race for resources (pointed out by njl). While here add a console printf on thread create fails; otherwise noone may notice (e.g. return value is always 0 and caller has no way to verify). Reviewed by: jhb, scottl MFC after: 2 weeks Notes: svn path=/head/; revision=158904
* Change msleep() and tsleep() to not alter the calling thread's priorityJohn Baldwin2006-04-171-1/+1
| | | | | | | | | | | | | | | if the specified priority is zero. This avoids a race where the calling thread could read a snapshot of it's current priority, then a different thread could change the first thread's priority, then the original thread would call sched_prio() inside msleep() undoing the change made by the second thread. I used a priority of zero as no thread that calls msleep() or tsleep() should be specifying a priority of zero anyway. The various places that passed 'curthread->td_priority' or some variant as the priority now pass 0. Notes: svn path=/head/; revision=157815
* fixup error handling in taskqueue_start_threads: check for kthread_createSam Leffler2006-03-301-11/+20
| | | | | | | | | | | failing, print a message when we fail for some reason as most callers do not check the return value (e.g. 'cuz they're called from SYSINIT) Reviewed by: scottl MFC after: 1 week Notes: svn path=/head/; revision=157314
* Add the following to the taskqueue api:Scott Long2006-01-141-16/+56
| | | | | | | | | | | | | | | | | | | | taskqueue_start_threads(struct taskqueue **, int count, int pri, const char *name, ...); This allows the creation of 1 or more threads that will service a single taskqueue. Also rework the taskqueue_create() API to remove the API change that was introduced a while back. Creating a taskqueue doesn't rely on the presence of a process structure, and the proc mechanics are much better encapsulated in taskqueue_start_threads(). Also clean up the taskqueue_terminate() and taskqueue_free() functions to safely drain pending tasks and remove all associated threads. The TASKQUEUE_DEFINE and TASKQUEUE_DEFINE_THREAD macros have been changed to use the new API, but drivers compiled against the old definitions will still work. Thus, recompiling drivers is not a strict requirement. Notes: svn path=/head/; revision=154333
* The interlock in taskqueue_terminate() is completely wrong for taskqueuesScott Long2006-01-111-5/+1
| | | | | | | that use spinlocks. Remove it for now. Notes: svn path=/head/; revision=154205
* Add functions and macros and refactor code to make it easier to manageScott Long2006-01-101-116/+86
| | | | | | | | | | | | | | | | | | | | | | | | fast taskqueues. The following have been added: TASKQUEUE_FAST_DEFINE() - create a global task queue. an arbitrary execution context. TASKQUEUE_FAST_DEFINE_THREAD() - create a global taskqueue that uses a dedicated kthread. taskqueue_create_fast() - create a local/private taskqueue. These are all complimentary of the standard taskqueue functions. They are primarily useful for fast interrupt handlers that can only use spinlock for synchronization. I personally think that the taskqueue API is starting to get too narrow and hairy, but fixing it will require a major redesign on the API. Such a redesign would be good but would break compatibility with FreeBSD 6.x, so it really isn't desirable at this time. Submitted by: sam Notes: svn path=/head/; revision=154167
* Create the taskqueue_fast handler with INTR_MPSAFE so that it doesn't runScott Long2005-12-231-1/+1
| | | | | | | | | with Giant. MFC After: 3 days Notes: svn path=/head/; revision=153676
* Use shorter names for the Giant and fast taskqueues so that their namesJohn Baldwin2005-10-251-2/+2
| | | | | | | actually fit. Notes: svn path=/head/; revision=151656
* Revert previous change to this file. I accidentally committed whileJohn Baldwin2005-10-241-19/+0
| | | | | | | fixing spelling in a comment. Notes: svn path=/head/; revision=151624
* Spell hierarchy correctly in comments.John Baldwin2005-10-241-0/+19
| | | | | | | Submitted by: Wojciech A. Koszek dunstan at freebsd dot czest dot pl Notes: svn path=/head/; revision=151623
* o enable shutdown of taskqueue threads; the thread servicing the queue checksSam Leffler2005-05-011-10/+38
| | | | | | | | | | | | | | | a new entry in the taskqueue struct each time it wakes up to see if it should terminate o adjust TASKQUEUE_DEFINE_THREAD & co. to record the thread/proc identity for the shutdown rendezvous o replace wakeup after adding a task to a queue with wakeup_one; this helps queues where multiple threads are used to service tasks (e.g. acpi) o remove NULL check of tq_enqueue method; it should never be NULL Reviewed by: dfr, njl Notes: svn path=/head/; revision=145729
* o eliminate modification of task structures after their run to avoidSam Leffler2005-04-241-3/+4
| | | | | | | | | | | | modify-after-free races when the task structure is malloc'd o shrink task structure by removing ta_flags (no longer needed with avoid fix) and combining ta_pending and ta_priority Reviewed by: dwhite, dfr MFC after: 4 days Notes: svn path=/head/; revision=145473
* Add taskqueue_drain. This waits for the specified task to finish, ifWarner Losh2004-10-051-0/+14
| | | | | | | | | | running, or returns. The calling program is responsible for making sure that nothing new is enqueued. # man page coming soon. Notes: svn path=/head/; revision=136131
* rearange some code that handles the thread taskqueue so that it is moreJohn-Mark Gurney2004-08-081-13/+16
| | | | | | | | | | generic. Introduce a new define TASKQUEUE_DEFINE_THREAD that takes a single arg, which is the name of the queue. Document these changes. Notes: svn path=/head/; revision=133305
* - Execute all of the tasks on the taskqueue during taskqueue_free() afterJohn Baldwin2004-06-281-39/+24
| | | | | | | | | | | | | | | | | the queue has been removed from the global taskqueue_queues list. This removes the need for the draining queue hack. - Allow taskqueue_run() to be called with the taskqueue mutex held. It can still be called without the lock for API compatiblity. In that case it will acquire the lock internally. - Don't lock the individual queue mutex in taskqueue_find() until after the strcmp as the global queues mutex is sufficient for the strcmp. - Simplify taskqueue_thread_loop() now that it can hold the lock across taskqueue_run(). Submitted by: bde (mostly) Notes: svn path=/head/; revision=131246
* Tidy up the thread taskqueue implementation and close a lost wakeup race.John Baldwin2004-02-191-14/+9
| | | | | | | | | | Instead of creating a mutex that we msleep on but don't actually lock when doing the corresponding wakeup(), in the kthread, lock the mutex associated with our taskqueue and msleep while the queue is empty. Assert that the queue is locked when the callback function is called to wake the kthread. Notes: svn path=/head/; revision=126027
* Various style fixes.John Baldwin2003-12-171-7/+5
| | | | | | | Submitted by: bde (mostly, if not all) Notes: svn path=/head/; revision=123614
* Fix a bug where the taskqueue kproc was being parented by initAlfred Perlstein2003-11-101-1/+1
| | | | | | | | | | | | | | | | because RFNOWAIT was being passed to kproc_create. The result was that shutdown took quite a bit longer because this errant "child" would not respond to termination signals from init at system shutdown. RFNOWAIT dissassociates itself from the caller by attaching to init as a parent proc. We could have had the taskqueue proc listen for SIGKILL, but being able to SIGKILL a potentially critical system process doesn't seem like a good idea. Notes: svn path=/head/; revision=122436
* correct fast swi taskqueue spinlock name to be different from the sleep lockSam Leffler2003-09-061-1/+1
| | | | | | | Submitted by: Tor Egge <Tor.Egge@cvsup.no.freebsd.org> Notes: svn path=/head/; revision=119812
* "fast swi" taskqueue support. This is a taskqueue that uses spinlocksSam Leffler2003-09-051-0/+118
| | | | | | | | | making it useful for dispatching swi tasks from fast interrupt handlers. Sponsered by: FreeBSD Foundation Notes: svn path=/head/; revision=119789
* Move dynamic sysctl(8) variable creation for the cd(4) and da(4) driversKenneth D. Merry2003-09-031-0/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | out of cdregister() and daregister(), which are run from interrupt context. The sysctl code does blocking mallocs (M_WAITOK), which causes problems if malloc(9) actually needs to sleep. The eventual fix for this issue will involve moving the CAM probe process inside a kernel thread. For now, though, I have fixed the issue by moving dynamic sysctl variable creation for these two drivers to a task queue running in a kernel thread. The existing task queues (taskqueue_swi and taskqueue_swi_giant) run in software interrupt handlers, which wouldn't fix the problem at hand. So I have created a new task queue, taskqueue_thread, that runs inside a kernel thread. (It also runs outside of Giant -- clients must explicitly acquire and release Giant in their taskqueue functions.) scsi_cd.c: Remove sysctl variable creation code from cdregister(), and move it to a new function, cdsysctlinit(). Queue cdsysctlinit() to the taskqueue_thread taskqueue once we have fully registered the cd(4) driver instance. scsi_da.c: Remove sysctl variable creation code from daregister(), and move it to move it to a new function, dasysctlinit(). Queue dasysctlinit() to the taskqueue_thread taskqueue once we have fully registered the da(4) instance. taskqueue.h: Declare the new taskqueue_thread taskqueue, update some comments. subr_taskqueue.c: Create the new kernel thread taskqueue. This taskqueue runs outside of Giant, so any functions queued to it would need to explicitly acquire/release Giant if they need it. cd.4: Update the cd(4) man page to talk about the minimum command size sysctl/loader tunable. Also note that the changer variables are available as loader tunables as well. da.4: Update the da(4) man page to cover the retry_count, default_timeout and minimum_cmd_size sysctl variables/loader tunables. Remove references to /dev/r???, they aren't used any longer. cd.9: Update the cd(9) man page to describe the CD_Q_10_BYTE_ONLY quirk. taskqueue.9: Update the taskqueue(9) man page to describe the new thread task queue, and the taskqueue_swi_giant queue. MFC after: 3 days Notes: svn path=/head/; revision=119708
* Use __FBSDID().David E. O'Brien2003-06-111-2/+3
| | | | Notes: svn path=/head/; revision=116182
* Introduce a new taskqueue that runs completely free of Giant, and inScott Long2003-02-261-2/+19
| | | | | | | | | | turns runs its tasks free of Giant too. It is intended that as drivers become locked down, they will move out of the old, Giant-bound taskqueue and into this new one. The old taskqueue has been renamed to taskqueue_swi_giant, and the new one keeps the name taskqueue_swi. Notes: svn path=/head/; revision=111528
* If we fail to write to a vnode during a ktrace write, then we drop allJohn Baldwin2002-08-011-0/+1
| | | | | | | | | | | | other references to that vnode as a trace vnode in other processes as well as in any pending requests on the todo list. Thus, it is possible for a ktrace request structure to have a NULL ktr_vp when it is destroyed in ktr_freerequest(). We shouldn't call vrele() on the vnode in that case. Reported by: bde Notes: svn path=/head/; revision=101153
* Change callers of mtx_init() to pass in an appropriate lock type name. InJohn Baldwin2002-04-041-2/+2
| | | | | | | | | | most cases NULL is passed, but in some cases such as network driver locks (which use the MTX_NETWORK_LOCK macro) and UMA zone locks, a name is used. Tested on: i386, alpha, sparc64 Notes: svn path=/head/; revision=93818
* Change the preemption code for software interrupt thread schedules andJohn Baldwin2002-01-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mutex releases to not require flags for the cases when preemption is not allowed: The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent switching to a higher priority thread on mutex releease and swi schedule, respectively when that switch is not safe. Now that the critical section API maintains a per-thread nesting count, the kernel can easily check whether or not it should switch without relying on flags from the programmer. This fixes a few bugs in that all current callers of swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from fast interrupt handlers and the swi_sched of softclock needed this flag. Note that to ensure that swi_sched()'s in clock and fast interrupt handlers do not switch, these handlers have to be explicitly wrapped in critical_enter/exit pairs. Presently, just wrapping the handlers is sufficient, but in the future with the fully preemptive kernel, the interrupt must be EOI'd before critical_exit() is called. (critical_exit() can switch due to a deferred preemption in a fully preemptive kernel.) I've tested the changes to the interrupt code on i386 and alpha. I have not tested ia64, but the interrupt code is almost identical to the alpha code, so I expect it will work fine. PowerPC and ARM do not yet have interrupt code in the tree so they shouldn't be broken. Sparc64 is broken, but that's been ok'd by jake and tmm who will be fixing the interrupt code for sparc64 shortly. Reviewed by: peter Tested on: i386, alpha Notes: svn path=/head/; revision=88900
* - Change the taskqueue locking to protect the necessary parts of a taskJohn Baldwin2001-10-261-36/+7
| | | | | | | | | | | while it is on a queue with the queue lock and remove the per-task locks. - Remove TASK_DESTROY now that it is no longer needed. - Go back to inlining TASK_INIT now that it is short again. Inspired by: dfr Notes: svn path=/head/; revision=85560
* Add locking to taskqueues. There is one mutex per task, one mutex perJohn Baldwin2001-10-261-34/+80
| | | | | | | | | | | queue, and a mutex to protect the global list of taskqueues. The only visible change is that a TASK_DESTROY() macro has been added to mirror the TASK_INIT() macro to destroy a task before it is free'd. Submitted by: Andrew Reiter <awr@watson.org> Notes: svn path=/head/; revision=85521
* remove include of ipl.h because it no longer existsAlfred Perlstein2001-05-161-1/+0
| | | | Notes: svn path=/head/; revision=76666
* - Catch up to the new swi API changes:John Baldwin2001-02-091-4/+4
| | | | | | | | | | - Use swi_* function names. - Use void * to hold cookies to handlers instead of struct intrhand *. - In sio.c, use 'driver_name' instead of "sio" as the name of the driver lock to minimize diffs with cy(4). Notes: svn path=/head/; revision=72238
* Staticize some malloc M_ instances.Poul-Henning Kamp2000-12-081-1/+1
| | | | Notes: svn path=/head/; revision=69774
* - Overhaul the software interrupt code to use interrupt threads for eachJohn Baldwin2000-10-251-3/+6
| | | | | | | | | | | | | | | | | | | | | | type of software interrupt. Roughly, what used to be a bit in spending now maps to a swi thread. Each thread can have multiple handlers, just like a hardware interrupt thread. - Instead of using a bitmask of pending interrupts, we schedule the specific software interrupt thread to run, so spending, NSWI, and the shandlers array are no longer needed. We can now have an arbitrary number of software interrupt threads. When you register a software interrupt thread via sinthand_add(), you get back a struct intrhand that you pass to sched_swi() when you wish to schedule your swi thread to run. - Convert the name of 'struct intrec' to 'struct intrhand' as it is a bit more intuitive. Also, prefix all the members of struct intrhand with 'ih_'. - Make swi_net() a MI function since there is now no point in it being MD. Submitted by: cp Notes: svn path=/head/; revision=67551
* - Heavyweight interrupt threads on the alpha for device I/O interrupts.John Baldwin2000-10-051-1/+1
| | | | | | | | | | | | | | - Make softinterrupts (SWI's) almost completely MI, and divorce them completely from the x86 hardware interrupt code. - The ihandlers array is now gone. Instead, there is a MI shandlers array that just contains SWI handlers. - Most of the former machine/ipl.h files have moved to a new sys/ipl.h. - Stub out all the spl*() functions on all architectures. Submitted by: dfr Notes: svn path=/head/; revision=66698
* - Remove the inthand2_t type and use the equivalent driver_intr_t type fromJohn Baldwin2000-09-131-0/+1
| | | | | | | | | | | | | | | | newbus for referencing device interrupt handlers. - Move the 'struct intrec' type which describes interrupt sources into sys/interrupt.h instead of making it just be a x86 structure. - Don't create 'ithd' and 'intrec' typedefs, instead, just use 'struct ithd' and 'struct intrec' - Move the code to translate new-bus interrupt flags into an interrupt thread priority out of the x86 nexus code and into a MI ithread_priority() function in sys/kern/kern_intr.c. - Remove now-uneeded x86-specific headers from sys/dev/ata/ata-all.c and sys/pci/pci_compat.c. Notes: svn path=/head/; revision=65822
* Modify to use fixed STAILQ_LAST().Jeffrey Hsu2000-08-031-1/+1
| | | | | | | Reviewed by: dfr Notes: svn path=/head/; revision=64199
* Add taskqueue system for easy-to-use SWIs among other things.Doug Rabson2000-05-281-0/+203
Reviewed by: arch Notes: svn path=/head/; revision=61033