summaryrefslogtreecommitdiff
path: root/sys/kern/kern_switch.c
Commit message (Collapse)AuthorAgeFilesLines
* Remove debugging code that I originally only wanted to be there for a couple ↵Julian Elischer2002-07-111-23/+2
| | | | | | | | | of days after merge. Reminded with pointy stick by: jhb Notes: svn path=/head/; revision=99834
* Part 1 of KSE-IIIJulian Elischer2002-06-291-39/+623
| | | | | | | | | | | | | | | | The ability to schedule multiple threads per process (one one cpu) by making ALL system calls optionally asynchronous. to come: ia64 and power-pc patches, patches for gdb, test program (in tools) Reviewed by: Almost everyone who counts (at various times, peter, jhb, matt, alfred, mini, bernd, and a cast of thousands) NOTE: this is still Beta code, and contains lots of debugging stuff. expect slight instability in signals.. Notes: svn path=/head/; revision=99072
* Move the "- 1" into the RQB_FFS(mask) macro itself so thatPeter Wemm2002-06-201-2/+1
| | | | | | | | | | | | | | | | | implementations can provide a base zero ffs function if they wish. This changes #define RQB_FFS(mask) (ffs64(mask)) foo = RQB_FFS(mask) - 1; to #define RQB_FFS(mask) (ffs64(mask) - 1) foo = RQB_FFS(mask); On some platforms we can get the "- 1" for free, eg: those that use the C code for ffs64(). Reviewed by: jake (in principle) Notes: svn path=/head/; revision=98469
* Make the run queue parameters machine dependent. Optimize 64 bitJake Burkholder2002-05-251-0/+2
| | | | | | | | | | architectures by using a 64 bit word for the bit array which keeps track of non-empty queues. Reviewed by: peter Notes: svn path=/head/; revision=97261
* Remove runq_findproc. This never worked right in the first place and canJake Burkholder2002-05-081-23/+0
| | | | | | | be prohibitively expensive. Notes: svn path=/head/; revision=96209
* Stage-2 commit of the critical*() code. This re-inlines cpu_critical_enter()Matthew Dillon2002-04-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | and cpu_critical_exit() and moves associated critical prototypes into their own header file, <arch>/<arch>/critical.h, which is only included by the three MI source files that need it. Backout and re-apply improperly comitted syntactical cleanups made to files that were still under active development. Backout improperly comitted program structure changes that moved localized declarations to the top of two procedures. Partially re-apply one of the program structure changes to move 'mask' into an intermediate block rather then in three separate sub-blocks to make the code more readable. Re-integrate bug fixes that Jake made to the sparc64 code. Note: In general, developers should not gratuitously move declarations out of sub-blocks. They are where they are for reasons of structure, grouping, readability, compiler-localizability, and to avoid developer-introduced bugs similar to several found in recent years in the VFS and VM code. Reviewed by: jake Notes: svn path=/head/; revision=93607
* Compromise for critical*()/cpu_critical*() recommit. Cleanup the interruptMatthew Dillon2002-03-271-3/+4
| | | | | | | | | | | | | | | | | | | | | | disablement assumptions in kern_fork.c by adding another API call, cpu_critical_fork_exit(). Cleanup the td_savecrit field by moving it from MI to MD. Temporarily move cpu_critical*() from <arch>/include/cpufunc.h to <arch>/<arch>/critical.c (stage-2 will clean this up). Implement interrupt deferral for i386 that allows interrupts to remain enabled inside critical sections. This also fixes an IPI interlock bug, and requires uses of icu_lock to be enclosed in a true interrupt disablement. This is the stage-1 commit. Stage-2 will occur after stage-1 has stabilized, and will move cpu_critical*() into its own header file(s) + other things. This commit may break non-i386 architectures in trivial ways. This should be temporary. Reviewed by: core Approved by: core Notes: svn path=/head/; revision=93264
* Rename runq_find() to runq_findproc(), and hide it behind #ifdef DIAGNOSTIC,Dag-Erling Smørgrav2002-03-061-3/+5
| | | | | | | | as it can have a severe impact on performance under high load, and the bug it was meant to catch was fixed ages ago. Notes: svn path=/head/; revision=91751
* revert last commit temporarily due to whining on the lists.Matthew Dillon2002-02-261-7/+0
| | | | Notes: svn path=/head/; revision=91328
* STAGE-1 of 3 commit - allow (but do not require) interrupts to remainMatthew Dillon2002-02-261-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | enabled in critical sections and streamline critical_enter() and critical_exit(). This commit allows an architecture to leave interrupts enabled inside critical sections if it so wishes. Architectures that do not wish to do this are not effected by this change. This commit implements the feature for the I386 architecture and provides a sysctl, debug.critical_mode, which defaults to 1 (use the feature). For now you can turn the sysctl on and off at any time in order to test the architectural changes or track down bugs. This commit is just the first stage. Some areas of the code, specifically the MACHINE_CRITICAL_ENTER #ifdef'd code, is strictly temporary and will be cleaned up in the STAGE-2 commit when the critical_*() functions are moved entirely into MD files. The following changes have been made: * critical_enter() and critical_exit() for I386 now simply increment and decrement curthread->td_critnest. They no longer disable hard interrupts. When critical_exit() decrements the counter to 0 it effectively calls a routine to deal with whatever interrupts were deferred during the time the code was operating in a critical section. Other architectures are unaffected. * fork_exit() has been conditionalized to remove MD assumptions for the new code. Old code will still use the old MD assumptions in regards to hard interrupt disablement. In STAGE-2 this will be turned into a subroutine call into MD code rather then hardcoded in MI code. The new code places the burden of entering the critical section in the trampoline code where it belongs. * I386: interrupts are now enabled while we are in a critical section. The interrupt vector code has been adjusted to deal with the fact. If it detects that we are in a critical section it currently defers the interrupt by adding the appropriate bit to an interrupt mask. * In order to accomplish the deferral, icu_lock is required. This is i386-specific. Thus icu_lock can only be obtained by mainline i386 code while interrupts are hard disabled. This change has been made. * Because interrupts may or may not be hard disabled during a context switch, cpu_switch() can no longer simply assume that PSL_I will be in a consistent state. Therefore, it now saves and restores eflags. * FAST INTERRUPT PROVISION. Fast interrupts are currently deferred. The intention is to eventually allow them to operate either while we are in a critical section or, if we are able to restrict the use of sched_lock, while we are not holding the sched_lock. * ICU and APIC vector assembly for I386 cleaned up. The ICU code has been cleaned up to match the APIC code in regards to format and macro availability. Additionally, the code has been adjusted to deal with deferred interrupts. * Deferred interrupts use a per-cpu boolean int_pending, and masks ipending, spending, and fpending. Being per-cpu variables it is not currently necessary to lock; bus cycles modifying them. Note that the same mechanism will enable preemption to be incorporated as a true software interrupt without having to further hack up the critical nesting code. * Note: the old critical_enter() code in kern/kern_switch.c is currently #ifdef to be compatible with both the old and new methodology. In STAGE-2 it will be moved entirely to MD code. Performance issues: One of the purposes of this commit is to enhance critical section performance, specifically to greatly reduce bus overhead to allow the critical section code to be used to protect per-cpu caches. These caches, such as Jeff's slab allocator work, can potentially operate very quickly making the effective savings of the new critical section code's performance very significant. The second purpose of this commit is to allow architectures to enable certain interrupts while in a critical section. Specifically, the intention is to eventually allow certain FAST interrupts to operate rather then defer. The third purpose of this commit is to begin to clean up the critical_enter()/critical_exit()/cpu_critical_enter()/ cpu_critical_exit() API which currently has serious cross pollution in MI code (in fork_exit() and ast() for example). The fourth purpose of this commit is to provide a framework that allows kernel-preempting software interrupts to be implemented cleanly. This is currently used for two forward interrupts in I386. Other architectures will have the choice of using this infrastructure or building the functionality directly into critical_enter()/ critical_exit(). Finally, this commit is designed to greatly improve the flexibility of various architectures to manage critical section handling, software interrupts, preemption, and other highly integrated architecture-specific details. Notes: svn path=/head/; revision=91315
* In a threaded world, differnt priorirites become properties ofJulian Elischer2002-02-111-7/+3
| | | | | | | | | different entities. Make it so. Reviewed by: jhb@freebsd.org (john baldwin) Notes: svn path=/head/; revision=90538
* Modify the critical section API as follows:John Baldwin2001-12-181-0/+25
| | | | | | | | | | | | | | | | | | | | | | - The MD functions critical_enter/exit are renamed to start with a cpu_ prefix. - MI wrapper functions critical_enter/exit maintain a per-thread nesting count and a per-thread critical section saved state set when entering a critical section while at nesting level 0 and restored when exiting to nesting level 0. This moves the saved state out of spin mutexes so that interlocking spin mutexes works properly. - Most low-level MD code that used critical_enter/exit now use cpu_critical_enter/exit. MI code such as device drivers and spin mutexes use the MI wrappers. Note that since the MI wrappers store the state in the current thread, they do not have any return values or arguments. - mtx_intr_enable() is replaced with a constant CRITICAL_FORK which is assigned to curthread->td_savecrit during fork_exit(). Tested on: i386, alpha Notes: svn path=/head/; revision=88088
* Change p into ke->ke_proc, this was hidden behind INVARIANTS.Jonathan Lemon2001-09-181-1/+1
| | | | Notes: svn path=/head/; revision=83601
* KSE Milestone 2Julian Elischer2001-09-121-34/+48
| | | | | | | | | | | | | | | | | Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha Notes: svn path=/head/; revision=83366
* Spelling fix in a KASSERT: runq_chose -> runq_choose.John Baldwin2001-07-041-1/+1
| | | | Notes: svn path=/head/; revision=79246
* Catch up to header include changes:John Baldwin2001-03-281-0/+1
| | | | | | | | - <sys/mutex.h> now requires <sys/systm.h> - <sys/mutex.h> and <sys/sx.h> now require <sys/lock.h> Notes: svn path=/head/; revision=74914
* Jake essentially rewrote this. It is not by any stretch of thePeter Wemm2001-03-151-2/+0
| | | | | | | imagination a derivative of what I did before. Notes: svn path=/head/; revision=74282
* Assert that the process we're trying to enqueue isn't already there.Dag-Erling Smørgrav2001-03-111-0/+21
| | | | Notes: svn path=/head/; revision=74113
* Add a new informative KASSERT to ensure that a process is in the SRUN stateJohn Baldwin2001-03-091-0/+3
| | | | | | | before we return it to cpu_switch(). Notes: svn path=/head/; revision=74014
* - Assert that the proc to return is not NULL in runq_choose theJake Burkholder2001-02-241-0/+2
| | | | | | | | | same as runq_remove. - bzero the whole struct runq in runq_init just in case its not statically allocated. Notes: svn path=/head/; revision=72977
* Implement a unified run queue and adjust priority levels accordingly.Jake Burkholder2001-02-121-181/+163
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - All processes go into the same array of queues, with different scheduling classes using different portions of the array. This allows user processes to have their priorities propogated up into interrupt thread range if need be. - I chose 64 run queues as an arbitrary number that is greater than 32. We used to have 4 separate arrays of 32 queues each, so this may not be optimal. The new run queue code was written with this in mind; changing the number of run queues only requires changing constants in runq.h and adjusting the priority levels. - The new run queue code takes the run queue as a parameter. This is intended to be used to create per-cpu run queues. Implement wrappers for compatibility with the old interface which pass in the global run queue structure. - Group the priority level, user priority, native priority (before propogation) and the scheduling class into a struct priority. - Change any hard coded priority levels that I found to use symbolic constants (TTIPRI and TTOPRI). - Remove the curpriority global variable and use that of curproc. This was used to detect when a process' priority had lowered and it should yield. We now effectively yield on every interrupt. - Activate propogate_priority(). It should now have the desired effect without needing to also propogate the scheduling class. - Temporarily comment out the call to vm_page_zero_idle() in the idle loop. It interfered with propogate_priority() because the idle process needed to do a non-blocking acquire of Giant and then other processes would try to propogate their priority onto it. The idle process should not do anything except idle. vm_page_zero_idle() will return in the form of an idle priority kernel thread which is woken up at apprioriate times by the vm system. - Update struct kinfo_proc to the new priority interface. Deliberately change its size by adjusting the spare fields. It remained the same size, but the layout has changed, so userland processes that use it would parse the data incorrectly. The size constraint should really be changed to an arbitrary version number. Also add a debug.sizeof sysctl node for struct kinfo_proc. Notes: svn path=/head/; revision=72376
* Use PCPU_GET, PCPU_PTR and PCPU_SET to access all per-cpu variablesJake Burkholder2001-01-101-2/+2
| | | | | | | other then curproc. Notes: svn path=/head/; revision=70861
* Catch up to moving headers:John Baldwin2000-10-201-2/+1
| | | | | | | | - machine/ipl.h -> sys/ipl.h - machine/mutex.h -> sys/mutex.h Notes: svn path=/head/; revision=67365
* Idle processes are always runnable, so let them state at SRUN.John Baldwin2000-09-151-1/+0
| | | | Notes: svn path=/head/; revision=65900
* Fix some printf format string warnings due to sizeof(int) != sizeof(long) onJohn Baldwin2000-09-111-8/+8
| | | | | | | the alpha. Notes: svn path=/head/; revision=65764
* Major update to the way synchronization is done in the kernel. HighlightsJason Evans2000-09-071-25/+75
| | | | | | | | | | | | | | | | | | include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh Notes: svn path=/head/; revision=65557
* Commit major SMP cleanups and move the BGL (big giant lock) in theMatthew Dillon2000-03-281-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch Notes: svn path=/head/; revision=58717
* Fix a typo and a bug.Peter Wemm1999-08-191-10/+12
| | | | | | | | | - One RTP_PRIO_REALTIME was meant to be RTP_PRIO_IDLE. - RTP_PRIO_FIFO was not handled. - Move the usual case first for setrunqueue() etc. Notes: svn path=/head/; revision=50056
* Extract the next runnable process selection out of cpu_switch() into aPeter Wemm1999-08-191-0/+204
fairly machine independent C routine. gcc actually does a pretty good job of this. Reviewed by: msmith (in principle) Notes: svn path=/head/; revision=50027
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-08 03:19:55 +0000