19 years agoDEV messaging stage 1/4: Rearrange struct cdevsw and add a message port
Matthew Dillon [Mon, 21 Jul 2003 05:50:47 +0000 (05:50 +0000)]
DEV messaging stage 1/4: Rearrange struct cdevsw and add a message port
and auto-queueing mask.  The mask will tell us which message functions
can be safely queued to another thread and which still need to run in the
context of the caller.   Primary configuration fields (name, cmaj, flags,
port, autoq mask) are now at the head of the structure.  Function vectors,
which may eventually go away, are at the end.  The port and autoq fields
are non-functional in this stage.

The old BDEV device major number support has also been removed from cdevsw,
and code has been added to translate the bootdev passed from the boot code
(the boot code has always passed the now defunct block device major numbers
and we obviously need to keep that compatibility intact).

19 years agoFix a minor compile-time errors when INVARIANTS is not defined.
Matthew Dillon [Sun, 20 Jul 2003 07:46:19 +0000 (07:46 +0000)]
Fix a minor compile-time errors when INVARIANTS is not defined.

19 years agoMinor cleanups so GENERIC compiles. Fix underscores in assembly and an
Matthew Dillon [Sun, 20 Jul 2003 07:29:20 +0000 (07:29 +0000)]
Minor cleanups so GENERIC compiles.  Fix underscores in assembly and an
include file in if_fe.c

Report-by: Steve Mynott <steve@tighrope.demon.co.uk>
19 years agodoreti was improperly clearing the entire gd_reqflags field when, in
Matthew Dillon [Sun, 20 Jul 2003 07:14:27 +0000 (07:14 +0000)]
doreti was improperly clearing the entire gd_reqflags field when, in
fact, it was only supposed to clear RQF_INTPEND.  This resulted in
reschedules not being left flagged for userret.

19 years agoProfiling cleanup 1/2: fix crashes (all registers need to be left intact
Matthew Dillon [Sun, 20 Jul 2003 04:20:32 +0000 (04:20 +0000)]
Profiling cleanup 1/2: fix crashes (all registers need to be left intact
from assembly), and fix a few syntax issues, etc.   It isn't ticking away
properly yet but at least it isn't crashing.

Submitted-by: Kip Macy <kmacy@fsmware.com>
Additional-work: dillon

19 years agoAdd externs for *_nonlocked atomic extensions to avoid warning.
Matthew Dillon [Sun, 20 Jul 2003 03:55:25 +0000 (03:55 +0000)]
Add externs for *_nonlocked atomic extensions to avoid warning.

19 years agoThis is the initial implmentation of the LWKT messaging infrastructure.
Matthew Dillon [Sun, 20 Jul 2003 01:37:22 +0000 (01:37 +0000)]
This is the initial implmentation of the LWKT messaging infrastructure.
Messages are sent to message ports and typically replied to a message port
embedded in the originating thread's thread structure (td_msgport).
The port functions match up and optimization client sync/asynch requests
verses target synch/asynch responses.

In this initial implementation a port must be owned by a particular thread,
and we use *asynch* IPI messaging to forward queueing and dequeueing operations
to the correct cpu.  Most of the IPI overhead will be absorbed by the fact
that these same IPIs also tend to schedule the threads in question, which on
the correct cpu (which is the one it will be on) costs nothing.

Message ports have in-context dispatch functions for initiating, aborting,
and replying to a message which can be overriden and will queue by default.

This code compiles but is as yet unreferenced, and almost certainly needs more

19 years agozfreei->zfree (there is no zfreei anymore)
Matthew Dillon [Sun, 20 Jul 2003 00:41:28 +0000 (00:41 +0000)]
zfreei->zfree (there is no zfreei anymore)

Report-by: Kip Macy <kmacy@fsmware.com>
19 years agoRemove references to the no longer existant PZERO.
Matthew Dillon [Sat, 19 Jul 2003 22:21:21 +0000 (22:21 +0000)]
Remove references to the no longer existant PZERO.

19 years agoNuke huge mbuf macros stage 2/2: Cleanup the MCL*() cluster inlines by
Matthew Dillon [Sat, 19 Jul 2003 21:53:06 +0000 (21:53 +0000)]
Nuke huge mbuf macros stage 2/2: Cleanup the MCL*() cluster inlines by
moving them into procedures, similar to 1/2.

Suggested-by: Bosko Milekic <bmilekic@technokratis.com>
19 years agoRemove the priority part of the priority|flags argument to tsleep(). Only
Matthew Dillon [Sat, 19 Jul 2003 21:14:53 +0000 (21:14 +0000)]
Remove the priority part of the priority|flags argument to tsleep().  Only
flags are passed now.  The priority was a user scheduler thingy that is not
used by the LWKT subsystem.  For process statistics assume sleeps without
P_SINTR set to be disk-waits, and sleeps with it set to be normal sleeps.

This commit should not contain any operational changes.

19 years agoNuke huge mbuf macros stage 1/2: Remove massive inline mbuf macros to reduce
Matthew Dillon [Sat, 19 Jul 2003 21:09:27 +0000 (21:09 +0000)]
Nuke huge mbuf macros stage 1/2: Remove massive inline mbuf macros to reduce
L1/L2 cache pollution.  Est. performance improvement of 4-6% and the kernel
is 42KB smaller.

Submitted-by: Bosko Milekic <bmilekic@technokratis.com>
19 years agoRemove an unnecessary cli that was causing 'trap 12 with interrupts disabled'
Matthew Dillon [Sat, 19 Jul 2003 17:00:33 +0000 (17:00 +0000)]
Remove an unnecessary cli that was causing 'trap 12 with interrupts disabled'
warnings from start_init() (harmless, but annoying).

19 years agoMake the kernel load properly recognize ABS symbols (.SET assembly
Matthew Dillon [Fri, 18 Jul 2003 05:12:41 +0000 (05:12 +0000)]
Make the kernel load properly recognize ABS symbols (.SET assembly
instruction) whos value is 0.  It was improperly assuming that symbols with
a 0 value could not be found, and also confusing ABS symbols with COMMON

19 years agoGDB changes required for gdb -k kernel /dev/mem. Still selected by
Matthew Dillon [Sun, 13 Jul 2003 07:13:51 +0000 (07:13 +0000)]
GDB changes required for gdb -k kernel /dev/mem.  Still selected by
process, not thread.  A "thread" command still needs to be added.

19 years agoThe comment was wrong, ptmmap *is* used, put it back in (fix crash
Matthew Dillon [Sun, 13 Jul 2003 07:10:06 +0000 (07:10 +0000)]
The comment was wrong, ptmmap *is* used, put it back in (fix crash
accessing /dev/mem)

19 years agoFix minor compile warning.
Matthew Dillon [Sun, 13 Jul 2003 05:51:17 +0000 (05:51 +0000)]
Fix minor compile warning.

19 years agoFix minor buildworld issues, mainly #include file dependancies and fields
Matthew Dillon [Sun, 13 Jul 2003 05:45:17 +0000 (05:45 +0000)]
Fix minor buildworld issues, mainly #include file dependancies and fields
that have moved from struct proc to struct thread.

19 years agoAdd missing required '*' in indirect jmp (fix assembler warning).
Matthew Dillon [Sun, 13 Jul 2003 05:43:31 +0000 (05:43 +0000)]
Add missing required '*' in indirect jmp (fix assembler warning).

19 years agooops, fix bug in last commit, and adjust the p_slptime check.
Matthew Dillon [Sat, 12 Jul 2003 18:35:35 +0000 (18:35 +0000)]
oops, fix bug in last commit, and adjust the p_slptime check.

19 years agoMake the pigs display more meaningful by showing processes which haven't
Matthew Dillon [Sat, 12 Jul 2003 18:10:28 +0000 (18:10 +0000)]
Make the pigs display more meaningful by showing processes which haven't
slept recently even if they aren't using much cpu.  Things like buildworlds
were not showing up at all.  This combined with the graph display makes
pigs far more useful.

19 years agoCollapse gd_astpending and gd_reqpri together into gd_reqflags. gd_reqflags
Matthew Dillon [Sat, 12 Jul 2003 17:54:36 +0000 (17:54 +0000)]
Collapse gd_astpending and gd_reqpri together into gd_reqflags.  gd_reqflags
now rollsup requests made pending for doreti.  Cleanup a number of scheduling
primitives and note that we do not need to use locked bus cycles on per-cpu

Note that the aweful idelayed hack for certain softints (used only by the TTY
subsystem, BTW) gets slightly broken in this commit because idelayed has become
per-cpu and the clock ints aren't yet distributed.

19 years agoForward FAST interrupts to the MP lock holder + minor fixes.
Matthew Dillon [Sat, 12 Jul 2003 16:55:54 +0000 (16:55 +0000)]
Forward FAST interrupts to the MP lock holder + minor fixes.

19 years agopartially fix pctcpu and userland rescheduling. We really have to distribute
Matthew Dillon [Sat, 12 Jul 2003 03:10:35 +0000 (03:10 +0000)]
partially fix pctcpu and userland rescheduling.  We really have to distribute
hardclock/statclock to all cpus to make it work properly.

19 years agoaccount for the time array being in microseconds now, and allow the
Matthew Dillon [Sat, 12 Jul 2003 03:09:50 +0000 (03:09 +0000)]
account for the time array being in microseconds now, and allow the
interval to be specified as a fraction of a second.

19 years agoMake the cpu/stat display work properly again.
Matthew Dillon [Fri, 11 Jul 2003 23:34:08 +0000 (23:34 +0000)]
Make the cpu/stat display work properly again.

19 years agoFix overflow in delta percentage calculation due to the fact that our 32 bit
Matthew Dillon [Fri, 11 Jul 2003 23:33:24 +0000 (23:33 +0000)]
Fix overflow in delta percentage calculation due to the fact that our 32 bit
cp_time[] counters are now in microseconds.

19 years agoCleanup hardclock() and statclock(), making them work properly even though
Matthew Dillon [Fri, 11 Jul 2003 23:26:20 +0000 (23:26 +0000)]
Cleanup hardclock() and statclock(), making them work properly even though
we do not replicate the clock interrupts across all cpus.

19 years agomisc cleanup. Add a case where we don't want an idlethread to HLT (if there
Matthew Dillon [Fri, 11 Jul 2003 22:30:09 +0000 (22:30 +0000)]
misc cleanup.  Add a case where we don't want an idlethread to HLT (if there
are pending fast ints that need the MP lock).

19 years agoMP Implmentation 4/4: Final cleanup for this stage. Deal with a race
Matthew Dillon [Fri, 11 Jul 2003 17:42:11 +0000 (17:42 +0000)]
MP Implmentation 4/4: Final cleanup for this stage.  Deal with a race
that occurs due to not having to hold the MP lock through an lwkt_switch()
where another cpu may pull off a process from the userland scheduler and
schedule its thread before the original cpu has completely switched out it.
Oddly enough latencies were enough that this bug never caused a crash!

Cleanup the scheduling code and in particular the switch assembly code, save
and restore eflags (cli/sti state) when switching heavy weight processes
(this is already done for light weight threads), add some counters, and
optimize fork() to (statistically) stay on the current cpu for a short while
to take advantage of locality of cache reference, which greatly improves
fork/exec times.  Note that synchronous pipe operations between two procseses
already (statistically) stick to the same cpu (which is what we want).

19 years agoMP Implmentation 3B/4: Remove Xcpuast and Xforward_irq, replacing them
Matthew Dillon [Fri, 11 Jul 2003 01:23:24 +0000 (01:23 +0000)]
MP Implmentation 3B/4: Remove Xcpuast and Xforward_irq, replacing them
with IPI messaging functions.  Fix user scheduling issues so user processes
are dependably scheduled on available cpus.

19 years agoMP Implmentation 3A/4: Fix stupid bug introduced in last commit.
Matthew Dillon [Thu, 10 Jul 2003 18:36:13 +0000 (18:36 +0000)]
MP Implmentation 3A/4: Fix stupid bug introduced in last commit.

19 years agoMP Implmentation 3A/4: Cleanup MP lock operations to allow the MP lock to
Matthew Dillon [Thu, 10 Jul 2003 18:23:24 +0000 (18:23 +0000)]
MP Implmentation 3A/4: Cleanup MP lock operations to allow the MP lock to
occassionally be out of synch from td_mpcount, so we don't have to
disable interrupts and so we can call try_mplock() from the middle of a
switch function.

19 years agoMP Implmentation 3/4: MAJOR progress on SMP, full userland MP is now working!
Matthew Dillon [Thu, 10 Jul 2003 04:47:55 +0000 (04:47 +0000)]
MP Implmentation 3/4: MAJOR progress on SMP, full userland MP is now working!
A number of issues relating to MP lock operation have been fixed, primarily
that we have to read %cr2 before get_mplock() since get_mplock() may switch
away.  Idlethreads can now safely HLT without any performance detriment.
The userland scheduler has been almost completely rewritten and is now
using an extremely flexible abstraction with a lot of room to grow.  pgeflag
has been removed from mapdev (without per-page invalidation it isn't safe
to use PG_G even on UP).  Necessary locked bus cycles have been added for
the pmap->pm_active field in swtch.s.  CR3 has been unoptimized for the
moment (see comment in swtch.s).  Since the switch code runs without the
MP lock we have to adjust pm_active PRIOR to loading %cr3.
Additional sanity checks have been added to the code (see PARANOID_INVLTLB
and ONLY_ONE_USER_CPU in the code), plus many more in kern_switch.c.
A passive release mechanism has been implemented to optimize P_CURPROC/lwkt
priority shifting when going from user->kernel and kernel->user.
Note: preemptive interrupts don't care due to the way preemption works so
no additional complexity there.  non-locking atomic functions to protect
only against local interrupts have been added.  astpending now uses
non-locking atomic functions to set and clear bits.  private_tss has been
moved to a per-cpu variable.   The LWKT thread module has been considerably
enhanced and cleaned up, including some fixes to handle MPLOCKED vs td_mpcount
races (so eventually we can do MP locking without a pushfl/cli/popfl combo).
stopevent() needs critical section protection, maybe.

19 years agoThe syncer is not a process any more, deal with it as a thread.
Matthew Dillon [Tue, 8 Jul 2003 17:21:53 +0000 (17:21 +0000)]
The syncer is not a process any more, deal with it as a thread.

19 years agoMP Implmentation 2A/4: Post commit cleanup, fix missing token releases that
Matthew Dillon [Tue, 8 Jul 2003 09:57:14 +0000 (09:57 +0000)]
MP Implmentation 2A/4: Post commit cleanup, fix missing token releases that
were messing up td_pri's critical count, and add smart LWKT rescheduling
when a higher priority thread is scheduled (and could not preempt).  This
fixes the priority problems.

19 years agoMP Implementation 2/4: Implement a poor-man's IPI messaging subsystem,
Matthew Dillon [Tue, 8 Jul 2003 06:27:28 +0000 (06:27 +0000)]
MP Implementation 2/4: Implement a poor-man's IPI messaging subsystem,
get both cpus arbitrating the BGL for interrupts, IPIing foreign
cpu LWKT scheduling requests without crashing, and dealing with the cpl.

The APs are in a slightly less degenerate state now, but hardclock and
statclock distribution is broken, only one user process is being scheduled
at a time, and priorities are all messed up.

19 years agoMP Implementation 1/2: Get the APIC code working again, sweetly integrate the
Matthew Dillon [Sun, 6 Jul 2003 21:23:56 +0000 (21:23 +0000)]
MP Implementation 1/2: Get the APIC code working again, sweetly integrate the
MP lock into the LWKT scheduler, replace the old simplelock code with
tokens or spin locks as appropriate.  In particular, the vnode interlock
(and most other interlocks) are now tokens.  Also clean up a few curproc/cred
sequences that are no longer needed.

The APs are left in degenerate state with non IPI interrupts disabled as
additional LWKT work must be done before we can really make use of them,
and FAST interrupts are not managed by the MP lock yet.  The main thing
for this stage was to get the system working with an APIC again.

buildworld tested on UP and 2xCPU/MP (Dell 2550)

19 years agooops, forgot one. Remove another curproc/cred dependancy
Matthew Dillon [Sun, 6 Jul 2003 20:12:15 +0000 (20:12 +0000)]
oops, forgot one.  Remove another curproc/cred dependancy

19 years agoRemove td_proc dependancy on cred that is no longer used.
Matthew Dillon [Sun, 6 Jul 2003 20:11:25 +0000 (20:11 +0000)]
Remove td_proc dependancy on cred that is no longer used.

19 years agofix a bug in the exit td_switch function, curthread was not always being
Matthew Dillon [Sat, 5 Jul 2003 05:54:00 +0000 (05:54 +0000)]
fix a bug in the exit td_switch function, curthread was not always being
loaded correctly.

19 years agoIntegrate the interrupt related operations for /dev/random support
Matthew Dillon [Fri, 4 Jul 2003 05:57:27 +0000 (05:57 +0000)]
Integrate the interrupt related operations for /dev/random support
directly into kern/kern_intr.c to work around a bad design.  Eventually
the 5.x random code will be backported.

19 years agoGeneric MP rollup work.
Matthew Dillon [Fri, 4 Jul 2003 00:32:32 +0000 (00:32 +0000)]
Generic MP rollup work.

19 years agoadd gd_other_cpus
Matthew Dillon [Fri, 4 Jul 2003 00:26:00 +0000 (00:26 +0000)]
add gd_other_cpus

19 years agoDocument hardwired indexes for fields.
Matthew Dillon [Fri, 4 Jul 2003 00:25:48 +0000 (00:25 +0000)]
Document hardwired indexes for fields.

19 years agoMisc interrupts/LWKT 2/2: Fix a reentrancy issue with thread repriortization
Matthew Dillon [Thu, 3 Jul 2003 21:22:38 +0000 (21:22 +0000)]
Misc interrupts/LWKT 2/2: Fix a reentrancy issue with thread repriortization
when entering and leaving the kernel from userland.

19 years agomisc quota proc->thread
Matthew Dillon [Thu, 3 Jul 2003 18:35:27 +0000 (18:35 +0000)]
misc quota proc->thread

19 years agoSplit the struct vmmeter cnt structure into a global vmstats structure and
Matthew Dillon [Thu, 3 Jul 2003 18:34:00 +0000 (18:34 +0000)]
Split the struct vmmeter cnt structure into a global vmstats structure and
a per-cpu cnt structure.  Adjust the sysctls to accumulate statistics
over all cpus.

(userland syncup)

19 years agoSplit the struct vmmeter cnt structure into a global vmstats structure and
Matthew Dillon [Thu, 3 Jul 2003 18:20:03 +0000 (18:20 +0000)]
Split the struct vmmeter cnt structure into a global vmstats structure and
a per-cpu cnt structure.  Adjust the sysctls to accumulate statistics
over all cpus.

19 years agoproperly initialize ncpus for UP
Matthew Dillon [Thu, 3 Jul 2003 18:19:51 +0000 (18:19 +0000)]
properly initialize ncpus for UP

19 years agoSplit the struct vmmeter cnt structure into a global vmstats structure and
Matthew Dillon [Thu, 3 Jul 2003 17:24:04 +0000 (17:24 +0000)]
Split the struct vmmeter cnt structure into a global vmstats structure and
a per-cpu cnt structure.  Adjust the sysctls to accumulate statistics
over all cpus.

19 years agoAdd some temporary debugging.
Matthew Dillon [Thu, 3 Jul 2003 01:58:25 +0000 (01:58 +0000)]
Add some temporary debugging.

19 years agofix unused variable warning
Matthew Dillon [Thu, 3 Jul 2003 01:16:59 +0000 (01:16 +0000)]
fix unused variable warning

19 years agoRemove pre-ELF underscore prefix and asnames macro hacks.
Matthew Dillon [Tue, 1 Jul 2003 20:31:39 +0000 (20:31 +0000)]
Remove pre-ELF underscore prefix and asnames macro hacks.

19 years agoEnhance debugging (sync before MP work).
Matthew Dillon [Tue, 1 Jul 2003 18:49:52 +0000 (18:49 +0000)]
Enhance debugging (sync before MP work).

19 years agoFor the moment uio_td may sometimes be NULL -> nfsm_request -> nfs_request ->
Matthew Dillon [Tue, 1 Jul 2003 18:48:31 +0000 (18:48 +0000)]
For the moment uio_td may sometimes be NULL -> nfsm_request -> nfs_request ->
nfs_msg(), so allow td to be NULL there too.

19 years agoOperations on the user scheduler must be inside a critical section (fixes
Matthew Dillon [Tue, 1 Jul 2003 04:37:46 +0000 (04:37 +0000)]
Operations on the user scheduler must be inside a critical section (fixes
producable panic).

19 years agoSync userland up with the kernel. This primarily adjusts ps, etc to handle
Matthew Dillon [Tue, 1 Jul 2003 00:19:36 +0000 (00:19 +0000)]
Sync userland up with the kernel.  This primarily adjusts ps, etc to handle
information stored in the thread structure.

19 years agoAdd threads to the process-retrieval sysctls so they show up in top, ps, etc.
Matthew Dillon [Mon, 30 Jun 2003 23:54:04 +0000 (23:54 +0000)]
Add threads to the process-retrieval sysctls so they show up in top, ps, etc.
Reorder the boot sequence a little to add a TAILQ for all threads.   Add
a td_refs field to prevent a thread from disappearing on us.

19 years agoFix a race in sysctl_out_proc() vs copyout() that could crash the kernel.
Matthew Dillon [Mon, 30 Jun 2003 22:19:41 +0000 (22:19 +0000)]
Fix a race in sysctl_out_proc() vs copyout() that could crash the kernel.

19 years agoMisc interrupts/LWKT 1/2: threaded interrupts 2: Major work on the
Matthew Dillon [Mon, 30 Jun 2003 19:50:32 +0000 (19:50 +0000)]
Misc interrupts/LWKT 1/2: threaded interrupts 2: Major work on the
user scheduler, separate it completely from the LWKT scheduler and make
user priorities, including idprio, normal, and rtprio, work properly.
This includes fixing the priority inversion problem that 4.x had.
Also complete the work on interrupt preemption.  There were a few things
I wasn't doing correctly including not protecting the initial call
to cpu_heavy_restore when a process is just starting up.  Enhance DDB a
bit (threads don't show up in PS yet).

This is a major milestone.

19 years agoMisc interrupts/LWKT 1/2: interlock the idle thread. Put execution of
Matthew Dillon [Sun, 29 Jun 2003 07:37:07 +0000 (07:37 +0000)]
Misc interrupts/LWKT 1/2: interlock the idle thread.  Put execution of
fast interrupts inside a critical section.  Make the hardclock and statclock
INTR_FAST.  Implement the strict priority queue mechanism for LWKTs.
Implement prioritized preemption for interrupt and softint preemption.
Keep better stats.

Note: this commit hacks up the userland scheduler, in particular the
notion of 'curproc' because threaded interrupts really mess up the userland
scheduler's idea of curproc, which it uses to assume that the process is not
on a run queue even though it is runnable.  The next step will be to
separate out and cleanup the userland scheduler.

19 years agoAdd 64 bit display output support to sysctl plus convenient macros.
Matthew Dillon [Sun, 29 Jun 2003 06:48:32 +0000 (06:48 +0000)]
Add 64 bit display output support to sysctl plus convenient macros.

19 years agoImplement interrupt thread preemption + minor cleanup.
Matthew Dillon [Sun, 29 Jun 2003 05:29:31 +0000 (05:29 +0000)]
Implement interrupt thread preemption + minor cleanup.

19 years agothreaded interrupts 1: Rewrite the ICU interrupt code, splz, and doreti code.
Matthew Dillon [Sun, 29 Jun 2003 03:28:46 +0000 (03:28 +0000)]
threaded interrupts 1: Rewrite the ICU interrupt code, splz, and doreti code.
The APIC code hasn't been done yet.   Consolidate many interrupt thread
related functions into MI code, especially software interrupts.  All normal
interrupts and software interrupts are now threaded, and I'm almost ready
to deal with interrupt-thread-only preemption.  At the moment I run
interrupt threads in a critical section and probably will continue to do
so until I can make them MP safe.

19 years agoformat cleanup for readability. Tab out back-slashes.
Matthew Dillon [Sat, 28 Jun 2003 07:00:58 +0000 (07:00 +0000)]
format cleanup for readability.  Tab out back-slashes.

19 years agosmp/up collapse stage 2 of 2: cleanup the globaldata structure, cleanup
Matthew Dillon [Sat, 28 Jun 2003 04:16:05 +0000 (04:16 +0000)]
smp/up collapse stage 2 of 2:  cleanup the globaldata structure, cleanup
and separate machine dependant portions of thread, proc, and globaldata,
and reduce the need to include lots of MD header files.

19 years agoGive ps access to a process's thread structure.
Matthew Dillon [Sat, 28 Jun 2003 02:36:44 +0000 (02:36 +0000)]
Give ps access to a process's thread structure.

19 years agosmp/up collapse stage 1 of 2: Make UP use the globaldata structure the same
Matthew Dillon [Sat, 28 Jun 2003 02:09:52 +0000 (02:09 +0000)]
smp/up collapse stage 1 of 2: Make UP use the globaldata structure the same
way SMP does, and start removing all the bad macros and hacks that existed

19 years agogo back to using gd_cpuid instead of gd_cpu.
Matthew Dillon [Fri, 27 Jun 2003 20:27:19 +0000 (20:27 +0000)]
go back to using gd_cpuid instead of gd_cpu.

19 years agoCleanup lwkt threads a bit, change the exit/reap interlock.
Matthew Dillon [Fri, 27 Jun 2003 03:30:43 +0000 (03:30 +0000)]
Cleanup lwkt threads a bit, change the exit/reap interlock.

19 years agoproc->thread stage 6: kernel threads now create processless LWKT threads.
Matthew Dillon [Fri, 27 Jun 2003 01:53:26 +0000 (01:53 +0000)]
proc->thread stage 6: kernel threads now create processless LWKT threads.
A number of obvious curproc cases were removed, tsleep/wakeup was made to
work with threads (wmesg, ident, and timeout features moved to threads).
There are probably a few curproc cases left to fix.

19 years agocleanup some odd uses of curproc. Remove PHOLD/PRELE around physical I/O
Matthew Dillon [Thu, 26 Jun 2003 20:27:53 +0000 (20:27 +0000)]
cleanup some odd uses of curproc.  Remove PHOLD/PRELE around physical I/O
(our UPAGES can no longer be swapped out and if they eventually are made
to it will only be when the thread is sleeping on a particular address).

Also move the inblock/oublock accounting in vfs_busy_pages() allowing us
to remove additional curproc references from various filesystem code.  This
also makes inblock/oublock more consistent.

19 years agocleanup
Matthew Dillon [Thu, 26 Jun 2003 19:24:50 +0000 (19:24 +0000)]

19 years agosimple cleanups (removal of ancient macros)
Matthew Dillon [Thu, 26 Jun 2003 18:35:23 +0000 (18:35 +0000)]
simple cleanups (removal of ancient macros)

19 years agoproc->thread stage 5: BUF/VFS clearance! Remove the ucred argument from
Matthew Dillon [Thu, 26 Jun 2003 05:55:21 +0000 (05:55 +0000)]
proc->thread stage 5:  BUF/VFS clearance!  Remove the ucred argument from
vop_close, vop_getattr, vop_fsync, and vop_createvobject.  These VOPs can
be called from multiple contexts so the cred is fairly useless, and UFS
ignorse it anyway.  For filesystems (like NFS) that sometimes need a cred
we use proc0.p_ucred for now.

This removal also removed the need for a 'proc' reference in the related
VFS procedures, which greatly helps our proc->thread conversion.

bp->b_wcred and bp->b_rcred have also been removed, and for the same reason.
It makes no sense to have a particular cred when multiple users can
access a file.  This may create issues with certain types of NFS mounts
but if it does we will solve them in a way that doesn't pollute the
struct buf.

19 years agoIntroduce cratom(), remove crcopy().
Matthew Dillon [Thu, 26 Jun 2003 02:17:47 +0000 (02:17 +0000)]
Introduce cratom(), remove crcopy().

19 years agoproc->thread stage 4: post commit cleanup. Fix minor issues when recompiling
Matthew Dillon [Wed, 25 Jun 2003 05:22:33 +0000 (05:22 +0000)]
proc->thread stage 4: post commit cleanup.  Fix minor issues when recompiling

19 years agoproc->thread stage 4: post commit, introduce sys/file2.h. As with other header
Matthew Dillon [Wed, 25 Jun 2003 03:57:27 +0000 (03:57 +0000)]
proc->thread stage 4: post commit, introduce sys/file2.h.  As with other header
files I am separating the inline functions out, because interdependancies
are making limited header file inclusion and ordering a joke.

19 years agoproc->thread stage 4: rework the VFS and DEVICE subsystems to take thread
Matthew Dillon [Wed, 25 Jun 2003 03:56:13 +0000 (03:56 +0000)]
proc->thread stage 4: rework the VFS and DEVICE subsystems to take thread
pointers instead of process pointers as arguments, similar to what FreeBSD-5
did.  Note however that ultimately both APIs are going to be message-passing
which means the current thread context will not be useable for creds and
descriptor access.

19 years agoproc->thread stage 3.5: Add an IO_CORE flag so coda doesn't have to dig
Matthew Dillon [Tue, 24 Jun 2003 02:12:00 +0000 (02:12 +0000)]
proc->thread stage 3.5: Add an IO_CORE flag so coda doesn't have to dig
into the user process structure.

19 years agoproc->thread stage 3: synchronize ps, top, and libkvm, and add some convenience
Matthew Dillon [Mon, 23 Jun 2003 23:53:01 +0000 (23:53 +0000)]
proc->thread stage 3: synchronize ps, top, and libkvm, and add some convenience
fields to eproc in the kernel to help them along.

19 years agoproc->thread stage3: make time accounting threads based and rework it for
Matthew Dillon [Mon, 23 Jun 2003 23:36:14 +0000 (23:36 +0000)]
proc->thread stage3: make time accounting threads based and rework it for

Cleanup user/sys/interrupt time accounting.  Get rid of the microputime and
equivalent support code in mi_switch() (it was really a bad idea to put that
in the critical path IMHO).  Instead account for time statistically
from the statclock, which produce time accounting that is just as accurate
in the long haul.  Remove the u/s/iticks fields from the proc structure and
put a slightly different version in the thread structure, so time can be
accounted for both threads and processes.

19 years agoproc->thread stage2: post-commit fixes/cleanup(2)
Matthew Dillon [Mon, 23 Jun 2003 18:22:07 +0000 (18:22 +0000)]
proc->thread stage2: post-commit fixes/cleanup(2)

19 years agoproc->thread stage2: post-commit fixes/cleanup
Matthew Dillon [Mon, 23 Jun 2003 18:12:13 +0000 (18:12 +0000)]
proc->thread stage2: post-commit fixes/cleanup

19 years agoproc->thread stage 2: MAJOR revamping of system calls, ucred, jail API,
Matthew Dillon [Mon, 23 Jun 2003 17:55:51 +0000 (17:55 +0000)]
proc->thread stage 2: MAJOR revamping of system calls, ucred, jail API,
and some work on the low level device interface (proc arg -> thread arg).
As -current did, I have removed p_cred and incorporated its functions
into p_ucred.  p_prison has also been moved into p_ucred and adjusted
accordingly.  The jail interface tests now uses ucreds rather then processes.

The syscall(p,uap) interface has been changed to just (uap).  This is inclusive
of the emulation code.  It makes little sense to pass a proc pointer around
which confuses the MP readability of the code, because most system call code
will only work with the current process anyway.  Note that eventually
*ALL* syscall emulation code will be moved to a kernel-protected userland
layer because it really makes no sense whatsoever to implement these
emulations in the kernel.

suser() now takes no arguments and only operates with the current process.
The process argument has been removed from suser_xxx() so it now just takes
a ucred and flags.

The sysctl interface was adjusted somewhat.

19 years agoOptimize lwkt_rwlock.c a bit
Matthew Dillon [Sun, 22 Jun 2003 20:32:19 +0000 (20:32 +0000)]
Optimize lwkt_rwlock.c a bit

19 years agoproc->thread stage 1: change kproc_*() API to take and return threads. Note:
Matthew Dillon [Sun, 22 Jun 2003 17:39:48 +0000 (17:39 +0000)]
proc->thread stage 1: change kproc_*() API to take and return threads.  Note:
we won't be able to turn off the underlying proc until we have a clean thread
path all the way through, which aint now.

19 years agominor code optimization.
Matthew Dillon [Sun, 22 Jun 2003 17:08:39 +0000 (17:08 +0000)]
minor code optimization.

19 years agoFinish migrating the cpl into the thread structure.
Matthew Dillon [Sun, 22 Jun 2003 08:54:22 +0000 (08:54 +0000)]
Finish migrating the cpl into the thread structure.

19 years agoAdd parens for code readability (no functional change)
Matthew Dillon [Sun, 22 Jun 2003 04:31:04 +0000 (04:31 +0000)]
Add parens for code readability (no functional change)

19 years agothread stage 10: (note stage 9 was the kern/lwkt_rwlock commit). Cleanup
Matthew Dillon [Sun, 22 Jun 2003 04:30:43 +0000 (04:30 +0000)]
thread stage 10: (note stage 9 was the kern/lwkt_rwlock commit).  Cleanup
thread and process creation functions.  Check the spl against ipending in
cpu_lwkt_restore (so the idle loop does not lockup the machine).  Remove
the old VM object kstack allocation and freeing code.  Leave newly created
processes in a stopped state to fix wakeup/fork_handler races.  Normalize
the lwkt_init_*() functions.

Add a sysctl debug.untimely_switch which will cause the last crit_exit()
to yield, which causes a task switch to occur in wakeup() and catches a
lot of 4.x-isms that can be found and fixed on UP.

19 years agoAdd kern/lwkt_rwlock.c -- reader/writer locks. Clean up the process exit &
Matthew Dillon [Sat, 21 Jun 2003 17:31:22 +0000 (17:31 +0000)]
Add kern/lwkt_rwlock.c -- reader/writer locks.  Clean up the process exit &
reaping interlock code to allow context switches to occur.  Clean up and
make operational the lwkt_block/signaling code.

19 years agorename td_token to td_xtoken to deal with conflict against sys/thread.h
Matthew Dillon [Sat, 21 Jun 2003 17:27:24 +0000 (17:27 +0000)]
rename td_token to td_xtoken to deal with conflict against sys/thread.h

19 years agothread stage 8: add crit_enter(), per-thread cpl handling, fix deferred
Matthew Dillon [Sat, 21 Jun 2003 07:54:57 +0000 (07:54 +0000)]
thread stage 8: add crit_enter(), per-thread cpl handling, fix deferred
interrupt handling for critical sections, add some basic passive token code,
and blocking/signaling code.   Add structural definitions for additional
LWKT mechanisms.

Remove asleep/await.  Add generation number based xsleep/xwakeup.

Note that when exiting the last crit_exit() we run splz() to catch up
on blocked interrupts.  There is also some #if 0'd code that will cause
a thread switch to occur 'at odd times'... primarily wakeup()->
lwkt_schedule()->critical_section->switch.  This will be usefulf or testing
purposes down the line.

The passive token code is mostly disabled at the moment.  It's primary use
will be under SMP and its primary advantage is very low overhead on UP and,
if used properly, should also have good characteristics under SMP.

19 years agothread stage 7: Implement basic LWKTs, use a straight round-robin model for
Matthew Dillon [Fri, 20 Jun 2003 02:09:59 +0000 (02:09 +0000)]
thread stage 7: Implement basic LWKTs, use a straight round-robin model for
the moment.  Also continue consolidating the globaldata structure so both UP
and SMP use it with more commonality.  Temporarily match user processes up
with scheduled LWKTs on a 1:1 basis.  Eventually user processes will have
LWKTs, but they will not all be scheduled 1:1 with the user process's

With this commit work can potentially start to fan out, but I'm not ready
to announce yet.

19 years agothread stage 6: Move thread stack management from the proc structure to
Matthew Dillon [Thu, 19 Jun 2003 06:26:10 +0000 (06:26 +0000)]
thread stage 6:  Move thread stack management from the proc structure to
the thread structure, cleanup the pmap_new_*() and pmap_dispose_*()
functions, and disable UPAGES swapping (if we eventually separate the kstack
from the UPAGES we can reenable it).  Also LIFO/4 cache thread structures
which improves fork() performance by 40% (when used in typical fork/exec/exit
or fork/subshell/exit situations).

19 years agothread stage 5: Separate the inline functions out of sys/buf.h, creating
Matthew Dillon [Thu, 19 Jun 2003 01:55:08 +0000 (01:55 +0000)]
thread stage 5: Separate the inline functions out of sys/buf.h, creating
sys/buf2.h (A methodology that will continue as time passes).  This solves
inline vs struct ordering problems.

Do a major cleanup of the globaldata access methodology.  Create a
gcc-cacheable 'mycpu' macro & inline to access per-cpu data.  Atomicy is not
required because we will never change cpus out from under a thread, even if
it gets preempted by an interrupt thread, because we want to be able to
implement per-cpu caches that do not require locked bus cycles or special

19 years agoAdd missing prototype (fixes warning).
Matthew Dillon [Thu, 19 Jun 2003 01:04:34 +0000 (01:04 +0000)]
Add missing prototype (fixes warning).

19 years agoFix an argument mismatch in a debugging printf
Matthew Dillon [Thu, 19 Jun 2003 01:02:29 +0000 (01:02 +0000)]
Fix an argument mismatch in a debugging printf