Matthew Dillon [Sun, 4 Mar 2018 05:41:37 +0000 (21:41 -0800)]
Rune - StreamPair fixes
* Use separately allocated streams for StreamPair (prior commit).
* Clarify variable names.
Matthew Dillon [Sun, 4 Mar 2018 05:33:27 +0000 (21:33 -0800)]
Rune - Fix thread bugs, add lock debugging
* Fix a number of thread-tracking bugs and races. We have to set
RUNNING a little earlier to prevent stolen locks from being stolen
themselves while we are running.
* Fix a bug in the thread detach code which transfers some of the
old thread's locks over to the new thread. The lock count was
getting mangled.
* Add thread debugging.
* Add lock debugging.
Matthew Dillon [Sun, 4 Mar 2018 05:26:46 +0000 (21:26 -0800)]
Rune - Add Pointer.lockcount(), misc bug fixes
* Add Pointer.lockcount(). This returns the number of locks held on the
pointer target.
* Fix a bug in unbuffered fwrite() (M_NONE). The loop would sometimes
not terminate.
Matthew Dillon [Fri, 2 Mar 2018 06:31:02 +0000 (22:31 -0800)]
Rune - Threading fixes, cleanups
* Implement importdesc(). This sets O_NONBLOCK on the descriptor when
extpread()/extpwrite() are not available.
* Cleanup the Stream and StreamPair class a bit. Separate the allocation
for the read and write streams in StreamPair. This allows accessors to
separately thread the read and write side without deadlocking each other.
* Fix per-rgd TaskReadAry and TaskWriteAry handling. Recent threading
changes now allow all rgd's to maintain their own KQueue.
* Preallocate the array of lists instead of having an array of pointers
for TaskReadAry and TaskWriteAry. Its just easier, and the size of
the array is dynamic anyway (based on number of descriptors).
* Implement I/O support on systems which do not have extpread() and
extpwrite().
* Fix RunThreads accounting.
* Fix threadPushRunnable() use cases.
* Remove a bunch of LASTQUEUE related debugging.
Matthew Dillon [Thu, 1 Mar 2018 07:38:13 +0000 (23:38 -0800)]
Rune - Refactor runtime threading for generated code
* Get rid of a large number of bottlenecks by making the timer and
run queues per-cpu (well, per-GD).
* Fix numerous SMP races in the timer code.
* Fix a few pthread mutex deadlocks.
* Increase various table sizes.
* Do not blindly wakeup all threads on a Timer event's TWSleep[]
list. Just wakeup the thread associated with the Timer event.
* Test with 1 million Rune threads.
Matthew Dillon [Wed, 28 Feb 2018 18:25:08 +0000 (10:25 -0800)]
Rune - Check thread stats more often
* Check the thread stats more often to determine when more real pthreads
are needed.
Matthew Dillon [Mon, 26 Feb 2018 01:54:54 +0000 (17:54 -0800)]
Rune - Add 'mpsafe'
* Add the 'mpsafe' scope identifier (really storage qualifier but
as with 'unlocked', is parsed as a scope identifier to avoid
confusion in procedural declarations).
mpsafe scoped sources disable both the lock and the ref on the
stored version of the pointer or reference, and disables pointer
range checking on indirections.
mpsafe scoped destinations (e.g. of an indirection like (a->b) allow
accesses without any further locks or refs. If the source is also
scoped mpsafe, Rune will not bother generating any of its normal
pointer range checking or lock/ref code for such accesses.
* Clarify handling of 'unlocked', document handling of 'mpsafe'.
* Document 'noinline' scope. This prevents a procedure from being
inlined by Rune's code generator.
Matthew Dillon [Fri, 29 Dec 2017 23:44:20 +0000 (15:44 -0800)]
Rune - Fix global initialization bug
* Fix PointerInfo bug for global initializer. The info structure
contains only the pointer's bounds, not the pointer itself.
Fixes assembler syntax error.
* in_Refs field for argv and argv[n] elements, fixes assertion
for compiled execution.
Matthew Dillon [Fri, 29 Dec 2017 03:31:48 +0000 (19:31 -0800)]
Rune - Refactor storage space tracker
* Refactor the storage space tracker for primary and temporary
data.
* Fixes a bug where temporary storage was not being properly tracked,
causing some format calls to implode.
Matthew Dillon [Thu, 28 Dec 2017 20:15:37 +0000 (12:15 -0800)]
Rune - Fix posix_memalign() usage
* Fix posix_memalign() usage. Arguments were reversed in one case,
and the minimum allowed alignment is sizeof(void *).
* Fixes various bits of corruption.
Matthew Dillon [Thu, 28 Dec 2017 19:42:53 +0000 (11:42 -0800)]
Rune - Add bracketed array initializer support
* Add bracketed array initializer support. That is, something like
this:
int array[6] = [ 1, 2, 3, charlie() ];
* Supports compile-time (interpreted) resolvable functions and
constants like almost everything else.
Matthew Dillon [Thu, 28 Dec 2017 03:05:44 +0000 (19:05 -0800)]
Rune - Add tests for trig/log/exp functions
* Add test scripts for trig/log/exp functions.
Matthew Dillon [Thu, 28 Dec 2017 03:01:17 +0000 (19:01 -0800)]
Rune - Fix bug in FNEG generation
* Fix bug in FNEG -> x86 generation.
Matthew Dillon [Thu, 28 Dec 2017 03:00:40 +0000 (19:00 -0800)]
Rune - Add internal trig, log, pow, exp functions
* Add math support for trig, log, pow, and exp class functions
Matthew Dillon [Wed, 27 Dec 2017 23:35:43 +0000 (15:35 -0800)]
Rune - Fix debug directive
* Fix stack offset debug directive at procedure entry.
Matthew Dillon [Wed, 27 Dec 2017 23:34:02 +0000 (15:34 -0800)]
Rune - Flesh out compile-time global initializations
* Flesh out compile-time global initializations. Fix a recursion
into the same expression when the code generator invokes the
interpreter to resolve a global pointer assignment.
* Document more features.
Matthew Dillon [Wed, 27 Dec 2017 05:30:46 +0000 (21:30 -0800)]
Rune - Fix FP save and restore with mixed ABI calls
* Rune currently uses MSABI which requires XMM6-15, RDI, and RSI
to be saved by procedures in addition to the standard Linux/BSD
ABI which requires procedures to save RBX, RBP, and R12-R15.
We do this because Rune can't inline as well as a C compiler
and often has to make internal ABI calls (which also use MSABI)
in the middle of code the programmer might otherwise believe
does not have to make procedure calls. We want to be able to
maintain FP variables in XMM registers without repeated saving
and restoring. The addition of RBX and RBP to the callee-saves
set also helps Rune's register allocator perform better.
* Fix issues when Rune is calling into a linux/BSD ABI (i.e.
when interfacing with C, making system calls, etc). In this
situation, the Rune procedure was not saving registers that
could get trashed by the system call or C call).
Matthew Dillon [Wed, 27 Dec 2017 05:30:21 +0000 (21:30 -0800)]
Rune - Remove some debugging
* Remove a bit of debugging
Matthew Dillon [Sat, 15 Apr 2017 00:46:05 +0000 (17:46 -0700)]
Rune - Copy tracking minfo in GenPointerInd2()
* Copy tracking minfo in GenPointerInd2(). This removes a few
unnecessary get/put sequences in the generated output.
Matthew Dillon [Mon, 6 Mar 2017 03:41:57 +0000 (19:41 -0800)]
Rune - Change ABI
* Change the ABI to be more like the microsoft call ABI. The main reason
is that fewer registers are scratch across calls, which significantly
reduces the effort the RAS register allocator must make due to the large
number of RuneRunTime_*() calls that are made.
Matthew Dillon [Sat, 4 Mar 2017 00:57:01 +0000 (16:57 -0800)]
Rune - Clarify documentation for ops on reference types
* Clarify the documentation on the limitations of certain types of
operations on reference types.
Matthew Dillon [Sat, 4 Mar 2017 00:31:04 +0000 (16:31 -0800)]
Rune - Convert most library pointers to references
* In Rune we want to encourage the use of references over pointers
at every opportunity. Since no pointer arithmatic is allowed on a ref
and because it will either be NULL or valid, Rune can optimize refs
more easily.
Matthew Dillon [Sat, 4 Mar 2017 00:14:26 +0000 (16:14 -0800)]
RAS - Fix .zero alignment
* .zero alignment was misintpreted as a fill value. Generate a separate
.align pseudo-op for any alignment > 1.
Matthew Dillon [Sat, 11 Feb 2017 20:53:34 +0000 (12:53 -0800)]
Rune - save/restore errno in signal handlers
* Save and restore errno in signal handlers so as not to interfere
with mainline code.
Matthew Dillon [Sun, 11 Dec 2016 18:59:50 +0000 (10:59 -0800)]
Rune - Reorder weak bindings
* Reorder weak bindings for newer versions of GCC which require fully
formed types for weak bindings.
Matthew Dillon [Thu, 15 Sep 2016 02:39:07 +0000 (19:39 -0700)]
Rune - Consolidate most RefStor allocations
* Consolidate most RefStor allocations into allocRefStor().
* Start tracking potential PointerInfo extensions to stack frames and
RefStor's to allow PointerInfo structures to be made static. This
isn't set in stone yet and the counter / index fields are not yet
used.
Matthew Dillon [Thu, 15 Sep 2016 02:34:25 +0000 (19:34 -0700)]
Rune - Optimize getrgd()
* getrgd() has a specific problem in that GCC is badly broken when it comes
to accessing TLS variables in a NxM threading scheme. GCC will cache the
%fs:0 load into a register and nothing we tell it can get it to throw
away that cached value. In a NxM threading scheme, a micro-thread can
block and wind up on a different pthread when it wakes up.
This is particularly painful when GCC does heavy optimizations that
combine what would normally be separate procedural domains.
* The getrgd() I had before was portable, but painfully expensive (two
subroutine calls!). I've decided to optimize getrgd() into an inline
and use __asm to manually retrieve the per-pthread &RuneTLS. This
is not really portable, but it's very important the Rune be able to
switch micro-threads quickly.
Matthew Dillon [Wed, 14 Sep 2016 18:20:48 +0000 (11:20 -0700)]
Rune - Fix interpreter (tests/gfxinput2.d)
* Bring the interpreter's initialization of the PointerInfo structure
up-to-date. type->ty_Info was not being set to PINFO_STATIC. This
fixes interpretation of a number of test programs.
Matthew Dillon [Wed, 14 Sep 2016 06:45:16 +0000 (23:45 -0700)]
Rune - Fix deadlock in rune threading system
* Fix a deadlock in Rune's sideband locks. An unlock->wakeup->start
sequence needs to hold two sideband locks. To avoid the deadlock,
relock in address-sorted order if the initial attempt to obtain the
second lock fails (trying with a non-blocking call).
Matthew Dillon [Wed, 14 Sep 2016 04:52:53 +0000 (21:52 -0700)]
Rune - Start working on registerizing inlined procedure arguments (2)
* Fix a few bugs. It's a bit fragile so I put the important pieces
around #ifdef's.
* Enable the feature.
* Make sure that normal procedure call return values are not cached,
the result vector has to be memory.
Matthew Dillon [Tue, 13 Sep 2016 23:58:12 +0000 (16:58 -0700)]
Rune - Start working on registerizing inlined procedure arguments
* Start working on registerizing inlined procedure arguments. Currently
disabled (it almost works but the tests/gfx* programs have some problems
so clearly I'm missing something).
Matthew Dillon [Mon, 12 Sep 2016 23:13:20 +0000 (16:13 -0700)]
Rune - Make code PointerInfo-centric
* The PointerStor and LValueStor structures now contain only two elements,
the address pointer and a pointer to a PointerInfo structure.
* The PointerInfo structure is now the center of the universe. Make it
slightly larger to accomodate a in_Refs field, but ultimately we want
to incorporate it into related RefStor's and stack contexts and not
need any dynamic allocation.
* In this new scheme we have to pass a proper PointerInfo element in the
constructor/destructor (EmitRAS_CallConstructorDestructor()) code.
Use the method argument's type->ty_Info. TODO - missing RefStor for
re-ref case.
* PointerStor and LValueStor are now 16-byte-aligned. This significantly
reduces the cost of pointer copies (we can use %xmm0). There is still
a C compatibility structure somewhere that requires the __aligned(16)
extension (which is in-place with this commit) and is getting confused,
but I'd like to find and fix that with explicit pad.
* TODO - There's a memory leak as of this commit. PointerInfo's are being
allocated for lvalue args but not freed. Will have to change the
mechanic somewhat.
Matthew Dillon [Mon, 22 Aug 2016 02:12:43 +0000 (19:12 -0700)]
Rune - Cleanup PointerInfo use in interpreter
* Cleanup PointerInfo handling to track the structure and free it after
use.
* Optimize NULL handling.
* Scrap StrTable hacks previously used to track PointerInfo for quoted
strings.
Matthew Dillon [Sun, 21 Aug 2016 07:04:34 +0000 (00:04 -0700)]
Rune - Initial PointerStor/LValueStor rework
* Initial rework, make PointerStor and LValueStor much smaller structures.
Basically just { ptr, info } now, and move the fat fields one indirection
level down into the info.
* This commit I just mocked-up PointerInfo allocations so I could test with
the interpreter. They leak like crazy.
Matthew Dillon [Mon, 22 Aug 2016 02:10:10 +0000 (19:10 -0700)]
Rune - Fix interpreter's BOOL_BINARY_OP
* We can't use the 64-bit short-cut for floating point comparisons.
Fixed.
Matthew Dillon [Sat, 20 Aug 2016 06:24:52 +0000 (23:24 -0700)]
Rune - Add Unchecked option (-u)
* Add -u option. This disables all bounds checking in any generated
output. Interpreted programs will still do bounds-checking.
* Generate help if no arguments or unrecognized option specified.
Matthew Dillon [Sat, 20 Aug 2016 05:49:28 +0000 (22:49 -0700)]
Rune - Select stronger alignment when it makes sense
* Use a larger alignment for arrays, compound expressions, and procedure
arguments when the type is large enough.
* Improves copying and zeroing by allowing RAS to rollup fields into bigger
registers and/or use the %xmm registers.
Matthew Dillon [Sat, 20 Aug 2016 04:18:15 +0000 (21:18 -0700)]
Rune - Fix array passing, optimize constant array indices
* Fix array passing (arrays are pased by value), interpreter and code
generator.
* Optimize constant array indices, removing unnecessary instructions
which are only calculating or testing constants.
Matthew Dillon [Fri, 19 Aug 2016 23:38:12 +0000 (16:38 -0700)]
Rune - Change limport mechanic
* Use quoted import paths to mean as-self (aka limport), and angle-bracketed
paths to mean under an identifier. So e.g. one would normally import "sys";
and import <third_party_library>.
* Implement angle-bracket strings for import. This also allows ANSI
string concatenation, including with angle-brackets if you really want to.
* Fix string concatenation by limiting the successive concatenations to the
same quoting mechanism as the first one.
* Remove the 'limport' keyword.
* This mechanism is more readable in terms of alignment.
Matthew Dillon [Fri, 19 Aug 2016 22:37:56 +0000 (15:37 -0700)]
Rune - Add labeled blocks
* Allow statement blocks to be labeled. The statement block associated with
a procedure body cannot be labeled but may be referred to as "procedure".
* Allow block labels to be specified in the 'break' and 'continue'
statements.
Matthew Dillon [Fri, 19 Aug 2016 04:57:49 +0000 (21:57 -0700)]
RAS - Improve register allocator
* Remove the conditional bypass optimization for now, it actually makes
register allocation worse by messing up register life-time tests even
in simple procedures.
* Clarify comments a bit.
Matthew Dillon [Thu, 18 Aug 2016 18:01:47 +0000 (11:01 -0700)]
Rune - Pass-through -O[n], drop more insns in RAS
* The 'rune' command passes -O[n] through to RAS.
* Fix the drop-test in RAS. A non-WRITE EA could prevent an instruction
from being dropped.
This fixes unnecessary double initializations, primarily from pre-zeroing
of variables on semantic entry which are immediate initialized again via
the code.
Matthew Dillon [Thu, 18 Aug 2016 18:00:30 +0000 (11:00 -0700)]
Rune - Handle const array lookups.
* Array lookups on constants (aka strings) are now constant-optimized
in both the interpreter and the code generator.
Matthew Dillon [Thu, 18 Aug 2016 16:38:32 +0000 (09:38 -0700)]
Rune - Add strdup(), strcmp(), strncmp()
* Add strdup(), strcmp(), strncmp()
* Flag string functions which can be pure.
Matthew Dillon [Thu, 18 Aug 2016 16:36:59 +0000 (09:36 -0700)]
Rune - Fix ptr->int cast bug
* Fix var-args ptr-to-int cast, it was improperly taking the
address of the PointerStor instead of taking PointerStor.s_Addr.
Matthew Dillon [Thu, 18 Aug 2016 05:08:00 +0000 (22:08 -0700)]
Rune - Stabilization pass (CLOEXEC work, return-storage for internal calls)
* All system calls which create a new file descriptor always return the
descriptor with O_CLOEXEC set. open, pipe, socketpair, dup, dup2,
opendir.
* Implement proper return storage semantics for internal calls that return
non-trivial structures (aka pointers), and properly pre-initialize the
return storage.
* Rename some API calls in classes/* and clean-up.
* Fix the exec code for the case where the requested descriptors for
0, 1, and 2 are mis-ordered.
Matthew Dillon [Sat, 13 Aug 2016 04:16:01 +0000 (21:16 -0700)]
Rune - Rework resolver
* Change the first pass model to a multi-pass model that allows portions
of the tree to be requeued for resolve when they cannot be immediately
resolved.
* Make sure the dynamic methods are properly resolved. When a method
call is made through a reference the version of the method in any
subclass may end up being called and those procedures must be resolved.
However, we do not have to resolve method procedures in subclasses which
are never instantiated by the program, since those methods can never be
called.
* As part of the dynamic method work the various passes must handle
the recursion to resolve dynamic methods the same way, typically
by detecting a dynamic method call and iterating through d->d_Sub*
(recursively).
* Generate the dynamic index for method calls for all possible method
vectors for consistency. However, particular vectors which the resolver
knows will never be executed will be populated as NULL by the code
generator, significantly reducing the size of the code being output.
* Stabilization pass on some of the tests, fix a few issues here and
there such as missing ResolveDecl() calls when resolving operators
and an incorrect EXF_ADDRUSED flag when processing an array lookup.
* Rename functions for consistency.
Matthew Dillon [Sat, 13 Aug 2016 00:45:43 +0000 (17:45 -0700)]
Rune - Start work on inlining
* Start work on inlining procedures.
* Add ct_ArgBase, abstract-out the argument base.
* Optimize stack allocations a bit by flagging when a procedure might make
ABI calls.
* Get rid of the Exp 'copy' stuff, dup the Exp earlier rather than later
(fix several bugs where undup'd exp's were being unintentionally
modified).
* Detect dynamic method calls, index refined subclass methods via
d->d_SubBase and d->d_SubNext.
* gfxinput2.d test works but gfxbutton2.d can't resolve due to resolver
loops. Resolver is in a bit of a pickle and will have to be written
after this commit.
Matthew Dillon [Thu, 11 Aug 2016 03:18:31 +0000 (20:18 -0700)]
Rune - Adjust autocast code to fix implied compound expression
* Check for a cast operator for (scaler) -> (compound) before
automatically casting the scaler to compound. Before we automatically
cast the scaler first and the cast operator (if present) would never
be run. (tests/cast.d)
* (scaler) and scaler needs to be deterministic, so treat them both the
same - as a scaler. Otherwise simple parenthesis can cause confusion.
If doing a compound assignment this means we now require a named element
for a single-scaler compound if the programmer wants the scaler to use
an actual or implied scaler -> compound cast.
* Cleanup some debugging messages we no longer need.
Matthew Dillon [Thu, 11 Aug 2016 01:52:41 +0000 (18:52 -0700)]
Rune - Main should exit without waiting for threads
* The main() now exits when it returns, no longer waiting for
other threads to finish first.
The programmer can decide to wait for not with Thread.waitThreads();
Matthew Dillon [Thu, 11 Aug 2016 00:48:40 +0000 (17:48 -0700)]
Rune - refactor resolve-time constant evaluation
* Refactor resolve-time constant evaluations (e.g. in order to be able
to resolve array types and certain && and || optimizations).
Remove the conditionalized paths in resolve*() for RESOLVE_CONSTEXP.
Instead, we evaluate these issues in the storage-alignment pass.
* This makes tests/pure.d work.
Matthew Dillon [Thu, 11 Aug 2016 00:46:03 +0000 (17:46 -0700)]
Rune - minor cleanup
* Minor cleanup and fix a few issues in tests
Matthew Dillon [Wed, 10 Aug 2016 21:32:04 +0000 (14:32 -0700)]
Rune - Enhance && and || constant handling a bit
* The rhs can also short-cut an && or || if both the lhs and rhs are
constants.
Matthew Dillon [Wed, 10 Aug 2016 18:14:33 +0000 (11:14 -0700)]
Rune - Change thead bootstrapping, fix RunLVSAlloc()
* Thread bootstrapping now turns the calling thread into the first Rune
thread instead of pausing the calling thread.
Fixes issue with compile-time interpretation for constants.
* RunLVSAlloc() now locks the underlying refstor according to d->d_Storage.
Also simplify the API.
Matthew Dillon [Tue, 9 Aug 2016 06:51:39 +0000 (23:51 -0700)]
Rune - change 'locked' to 'hard'
* Change the confusing 'locked' scope qualifier to 'hard' which conforms
more to what it actually does.
Matthew Dillon [Tue, 9 Aug 2016 05:54:03 +0000 (22:54 -0700)]
rune - Fix tests/build1.d
* Fix tests/build1.d so it does something.
Matthew Dillon [Tue, 9 Aug 2016 05:52:50 +0000 (22:52 -0700)]
Rune - resolve the alignment for internal types immediately
* Internal types may be implied during resolution, be sure to
completely resolve its alignment too.
* Note that other (non-internal) types might have recursive dependencies,
and their alignment is not resolve in the normal resolveType() code.
Matthew Dillon [Tue, 9 Aug 2016 05:29:20 +0000 (22:29 -0700)]
rune - Fix -rpath and -L<libpath>
* Use the passed-in destination path to construct -rpath and -L when
linking a generated rune program, instead of hardwiring /usr/local/rune/*
Matthew Dillon [Tue, 9 Aug 2016 05:27:15 +0000 (22:27 -0700)]
rune - Fix generated (*pt).new for double-indirect
* Fix generated (*pt).new for double-indirect (tests/heap.d)
* Remove a test in the run-time that the generator does not have. I
think we don't need this type test, and it breaks the test.
* Fix tests/heap.d assertion, base must be unlocked.
Matthew Dillon [Fri, 15 Apr 2016 02:39:59 +0000 (19:39 -0700)]
Rune - Interpreter locking work 5/n
* Add missing locks in destructor
* Fix incorrect mrs lock (fixes test/mmap.d)
Matthew Dillon [Thu, 14 Apr 2016 23:01:08 +0000 (16:01 -0700)]
Rune - Update README
* Update README and make the perl and python loop tests conform to the
Rune 1-billion loop test (tests/loop1b.d).
Matthew Dillon [Thu, 14 Apr 2016 22:36:57 +0000 (15:36 -0700)]
Rune - Fix build without make obj
* When rune is built without make obj the .PATH libruntime was using
confuses the hell out of the build.
* Remove .PATH, use a source file #include trick instead to bring the
two source files from librune into libruntime that libruntime needs.
Reported-by: mneumann
Matthew Dillon [Thu, 14 Apr 2016 20:03:23 +0000 (13:03 -0700)]
Rune - Interpreter refactor ex_Run 3/n
* Fix DCOPY semantics for the interpreter. Flag that the rundata must be
thrown away and operate on ct_DCopyData instead of making a copy.
* Optimize Run64 for constant collapses. Also redo RunCachedConstantExp()
to simplify the code that uses it and to optimize more cases.
Matthew Dillon [Thu, 14 Apr 2016 17:31:29 +0000 (10:31 -0700)]
Rune - Interpreter refactor ex_Run 2/n
* Implement exp->ex_Run64() - A shortcut function which returns integral
rvalues up to 64-bits. This causes most expressions to self-optimize
significantly, to the point where the interpreter is able to omit a great
deal of locking in the critical path as well in some cases to omit nearly
all procedure calls other than the primary recursion when evaluating an
integral expression.
Cuts loop-1billion test from 20 seconds to 7 seconds on a 3.4 GHz Haswell
box. This is far better than the previous pointer-return optimization
which could only get it down to ~14 seconds or so. A native Rune compile
runs the same loop in 0.28 seconds. 25:1 is not bad for an interpreter
running a tight loop verses a compiled tight loop!
* Implement for nearly all internal integral operators.
* Implement for integral Declaration access, and implied void casts.
* Implement for integer-integer casts.
Matthew Dillon [Thu, 14 Apr 2016 05:55:19 +0000 (22:55 -0700)]
Rune - Interpreter refactor ex_Run 1/n
* Refactor ex_Run to not return a pointer to the object. This slows things
down a bit until the next commit.
Matthew Dillon [Thu, 14 Apr 2016 04:17:25 +0000 (21:17 -0700)]
Rune - Interpreter locking work 4/n
* This is a bit messy, but implement locking modes in the interpreter.
Fix a couple of bugs while doing so.
* Rune compiler needs to run in the rune threading environment now because
the lock code assumes it (the compiler can run the interpreter to resolve
constants).
* (temporary) don't lock declarations which already have an implied lock
whos addresses are not taken. Fixes performance issue with simple
expression. This is temporary, actual fix needs to test exp->ex_Flags
for use case.
Matthew Dillon [Tue, 12 Apr 2016 01:47:55 +0000 (18:47 -0700)]
Rune = Change ulong -> uintptr_t
* Change a non-standard use of ulong (in C) to uintptr_t, which is what it
was proxying for. If the atomic_*_long() functions aren't compatible with
uintptr_t we have bigger problems.
Matthew Dillon [Mon, 11 Apr 2016 01:07:05 +0000 (18:07 -0700)]
Rune - Interpreter locking work 3/n
* Abstract most manual copy fixups and pre-copy drops in preparation for
real locking.
Matthew Dillon [Sun, 10 Apr 2016 05:01:53 +0000 (22:01 -0700)]
Rune - Interpreter locking work 2/n
* Add fields to rundata_t (RunData) roughly similar to what we needed
in gendata_t (GenData) for locking support.
Matthew Dillon [Sun, 10 Apr 2016 01:20:34 +0000 (18:20 -0700)]
Rune - Interpreter locking work 1/n
* Revamp RunDiscard*() to take the sideband data pointer so we don't have
to load it into the rundata structure.
* Revamp DCopyData to allow RunGetExpTmpData() to be called at any point
instead of at the beginning of an ex_Run() procedure. This allows the
function to reuse the passed-in rundata_t structure.
* Other changes in preparation for expanding rundata_t for object locking
support.
Tomohiro Kusumi [Mon, 11 Apr 2016 08:23:46 +0000 (17:23 +0900)]
Rune - Fix compile errors/warnings
on FreeBSD/clang.
Other compile time errors that I didn't go far as to change are
1. cpu_ccfence() which is DragonFly specific function for inline asm
2. *(int*)0 = 1;
3. (StrTable *)(__DECONST(char *, str) - xxx);
Matthew Dillon [Fri, 8 Apr 2016 17:56:16 +0000 (10:56 -0700)]
Rune - Threaded function detach work
* General thread performance check. Co-routine switch overhead (e.g.
synchronous threaded call which does not detach) is around 250ns.
Threaded call with detach is 500ns to 1uS or so. Pretty good. I would
like to improve the co-routine switch overhead.
* Threaded procedures now actually run synchronously until they detach,
instead of using parent-blocking. Overhead is significantly reduced in
the SMP case.
* Finally add the code to transfer ownership of acquired locks when a
threaded procedure detaches, so the threaded procedure's thread properly
owns the procedure's locks.
* Code generator now implements the TSCHED instruction.
* Implement a GlobFreeQueue to shift excess cached entries from per-pthread
FreeQueue. Fixes a degenerate case where one pthread keeps producing
threads.
Matthew Dillon [Thu, 7 Apr 2016 07:21:27 +0000 (00:21 -0700)]
Rune - Content-locking work 1/2
* Separate gendata->implied_mask out from gendata->s_Beg so function is
more obvious. Fix two sbindex bugs while we are at it. One was a
missing increment, and the other a broken algorithm related to handling
compound types.
* For now treat compound types (that are not arguments) like classes and
do not content-lock their contents. The intended content-locking
recursion is more complex than expected.
* Clarify code documentation.
* When pointers or references are passed as lvalues, the resolver and
code generator now validate that the content-locking mode is compatible.
There is currently a hack for 'this' pointers (only used for e.g.
ptr.new()) to allow mismatches.
Matthew Dillon [Wed, 6 Apr 2016 23:38:25 +0000 (16:38 -0700)]
Rune - Correct type-o
* Cerroct tope-y
Reported-by: mneumann
Matthew Dillon [Wed, 6 Apr 2016 16:13:00 +0000 (09:13 -0700)]
Rune - do a pass on the README
* Do a pass on the README
Matthew Dillon [Wed, 6 Apr 2016 15:11:15 +0000 (08:11 -0700)]
Rune - retool rundata_t
* Interpreter needs to start catching up to the code generator in terms of
being able to lock data structures. I was able to punt on it before but
now it is beginning to matter. Even single-threaded there are
situations, such as in File.fflush(), where a synchronous switch due to
a blocking write() will cause problems.
* Do a major retool of rundata_t to allow the structure to be expanded.
We previously relied on its small size (two fields) and GCC's
pass-by-value optimizations. This made it impossible to expand since
pass-by-value, particularly pass-by-value returns, become a massive
optimization failure with larger structures. Change:
rundata_t ex_func(runctx_p ct, Exp *exp)
to:
void *ex_func(runctx_p ct, rundata_t *data, Exp *exp)
Where the data->data portion of data is returned directly as a pointer
(removing an indirection for quick accesses to the data). The rundata
structure itself can be passed down the stack.
An alternative is to also return the rundata_t * which can theoretically
remove a register push/pop for it around the procedure call. However
there are enough other procedure calls that this doesn't save us a whole
lot, and it also forces a sequential indirection which can cause cpu
stalls.
* Initially expand rundata_t by embedding temporary space which is easier
to get at than InterpGetExpTmpData(). This space can be used for most
of the canned operators.
* Reoptimize run_oper.c. Do a better job collapsing stack-variable cases
to remove additional calls and indirections. Unary ops on stack
variables and binary ops (stack-variable, constant) are much faster now
for both lvalue and non-lvalue ops. Also optimize the pointer arithmatic
ops.
This cuts my loop1b.d test times in half.
Matthew Dillon [Wed, 6 Apr 2016 01:33:57 +0000 (18:33 -0700)]
Rune - Tighten up parser, stabilization, optimization.
* Get const storage errors mostly working. Disallow assignments to constant
storage, disallow casts from pointer-to-const to pointer-to-nonconst.
Mostly a matter of fixing SimilarType() and adding additional cases
in similar routines. Fix a few CONST-vs-not tests at the same time.
* Generate better error message when any attempt is made to cast a pointer
to a complex class (one containing lvalues, pointers, or references) to
(void *). This is not allowed.
* Allow any pointer to be cast to size_t (so the address can be printed).
Before we were doing a double cast: (size_t)(void *)ptr, which no longer
works generically.
* Optimize initial loads to remove unnecessary PPUTs (and related). Mainly
effects the generation of procedure argument vectors for procedure calls.
* Fix missing GENSTAT_LOCKH settings in a few places (there are probably a
few more remaining).
Matthew Dillon [Tue, 5 Apr 2016 20:34:34 +0000 (13:34 -0700)]
Rune - Implement hard-locking model feature
* Add the 'locked' scope qualifier. This qualifier may be specified to the
left of 'method' in a method procedure to automatically hard-lock the
method object, and can be specified for argument and local variable
declarations to indicate that their content locks be hard rather than soft.
* Add a ton of supporting instructions. Basically add an 'H' version for
all the STK, STS, LV, P, and RS instructions which lock or unlock
storage. For example, PGETH rather than just PGET.
* Cleanup the documentation.
* Cleanup a few places in the class library that require hard locks
(e.g. File.fflush().
Matthew Dillon [Tue, 5 Apr 2016 16:11:28 +0000 (09:11 -0700)]
Rune - Enforce result; requirements
* Enforce result statement requirements in the resolver. The "result"
statement must be executed in the top-level of a procedure and once
executed access to any procedure argument or return storage is prohibited
(because it is no longer available now that the procedure has been
detached).
Matthew Dillon [Tue, 5 Apr 2016 06:39:46 +0000 (23:39 -0700)]
Rune - More autocasts for integer constants
* If the resolver is unable to find a matching operator and one side
of the expression is an integer or floating constant, the resolver
will now search for an operator with that side auto-cast to the
type of the other side.
If the resolver successfully finds an operator it will then test that
the value of the integer constant remains unchanged after the autocast.
If the value is unchanged the autocast is allowed. If the value has
changed the autocast is NOT allowed.
* This fixes a number of annoyances when the programmer desires to use
simple obvious integer constants with or without a suffix whos type does
not match the operator. For example, the expression (a + 1) would have
previously failed if the type of 'a' was not int32_t (because the integer
constant is of type int32_t). This expression will now succeed.
Matthew Dillon [Tue, 5 Apr 2016 05:38:40 +0000 (22:38 -0700)]
Rune - Documentation update
* Flesh out and revise.
Matthew Dillon [Tue, 5 Apr 2016 05:07:19 +0000 (22:07 -0700)]
Rune - Remove collapsed libraries
* Remove libraries no longer used from the rune link. Everything was
folded into libruntime.
Reported-by: lhmwzy
Matthew Dillon [Tue, 5 Apr 2016 04:54:37 +0000 (21:54 -0700)]
Rune - crater push test
* crater push test again.
Matthew Dillon [Tue, 5 Apr 2016 04:48:27 +0000 (21:48 -0700)]
Rune - crater push
* Edit README
Matthew Dillon [Tue, 5 Apr 2016 04:14:08 +0000 (21:14 -0700)]
Rune - cleanup obsolete doc files
* Cleanup old doc files and musings that are no longer applicable.
Matthew Dillon [Tue, 5 Apr 2016 04:03:59 +0000 (21:03 -0700)]
Rune - fix missing GROUP env
* Some shells don't set GROUP. Also add logic for USER, just in case.
Reported-by: lhmwzy
Matthew Dillon [Tue, 5 Apr 2016 02:30:39 +0000 (19:30 -0700)]
Rune - pretest crater repo
* pretest crater repo
Matthew Dillon [Tue, 5 Apr 2016 02:19:15 +0000 (19:19 -0700)]
Rune - Simplify the library topology
* Collapse libsupport, libruntime, and libdthread into just libruntime.
The libraries were already interdependent due to other work such as having
the interpreter use the runtime internal call linkages. We really only
need one library here. We keep libgen broken out for parse-time and the
interpreter. libruntime is used by everyone to varying degrees.
Matthew Dillon [Tue, 5 Apr 2016 01:38:03 +0000 (18:38 -0700)]
Rune - add *env(), fexec(), fwait().
* Add various environment handling functions, fexec(), and fwait().
* If envp is passed as NULL, environ is used for the environment.
Matthew Dillon [Mon, 4 Apr 2016 06:51:05 +0000 (23:51 -0700)]
Rune - Stabilization, system calls
* Flesh out FdLock and FdMap.
* Add Sys.panic(), Sys.exit()
* Allow procedure argument defaults to reference the procedure arguments.
In particular a method call can access the object via 'this' and use it
to set an argument default. At the moment this requires hacking
*GetDeclarationData() to detect the case.
* Fix two bugs in the code generator related to argument defaulting in
compound expressions (aka procedure argument declarations).
* Fix a bug in the calculation of temporary space for type defaults in
resolveStorageExpOnly() and resolveStorageType().
Matthew Dillon [Sun, 3 Apr 2016 07:14:06 +0000 (00:14 -0700)]
Rune - Flesh out core
* Buffered fopen/fdopen/freopen/fdreopen and other support functions.
automatic duplex detection.
* Use pipe2().
Matthew Dillon [Sun, 3 Apr 2016 03:36:13 +0000 (20:36 -0700)]
Rune - Stability, features
* Improve the resolver and run-time to optimize constants accessed via
class path. e.g. char buf[Fs.PATH_MAX] is allowed.
* Add Fs.PATH_MAX and a number of other constants. Also do some file
splitting and cleanup.
* Flesh out the BlockedPThreads mechanic for creating additional pthreads.
add threadSetBlocking() and theadClrBlocking().
* Add stat, chmod, chown, utimes family of functions.
Add symlink, link, readlink, unlink.
Add dup, dup2, flock, and umask.
Matthew Dillon [Sun, 3 Apr 2016 00:48:31 +0000 (17:48 -0700)]
Rune - Stabilization, add Directory
* Add the Directory class and associated directory iterator and system
calls.
* Fix a bug in SRSGET (rgd->FreeRS corruption)
* Fix a bug in argv setup for the interpreter (increment i before breaking
out on xopt + filename encountered to properly position argv).
* Fix a bug in GenSemanticExit() (improper call to SRSPUT and other issues).
Matthew Dillon [Sat, 2 Apr 2016 21:29:27 +0000 (14:29 -0700)]
Rune - Stabilization, flesh out core classes
* Add many more constants.
* Rename class Fd to class Fs and rearrange the methods. Add additional
classes. Start working on Stream and StreamPair. Somewhat hide the
low-level pipe() and socketpair() calls. Give StreamPair similar methods
that make life easier.
* Fix a few cases where SGF_ADDRUSED and EXF_ADDRUSED were not being
propagated properly by the resolver. Hack the context code to check
the arguments for ADDRUSED to trigger SRSGET/SRSPUT in the procedure's
main context. Fix a related bug.
* Properly propagate ADDRUSED and ADDROF in exp.exp (TOK_DOT) expressions.
This introduces a bit of a performance regression due to extra LVRSGETs
which will be fixed later. Fixes a number of lvalue-related issues.
* Fix a ref-count issue revealed by a more sophisticated tests/fd_pipe.d
Also test fgetbuf(), a high-performance descriptor data streamer.
Matthew Dillon [Sat, 2 Apr 2016 04:34:25 +0000 (21:34 -0700)]
Rune - RAS 3-op optimizations, fix dthread bug
* Optimize most 3-operand instructions to remove an intermediate move
if the target is a direct register.
* Fix a deadlock in libdthread's event-handling that could lockup the
timer code.
Matthew Dillon [Sat, 2 Apr 2016 02:52:18 +0000 (19:52 -0700)]
Rune - Features (scripting, constants, main)
* Add -x option that causes argument processing by the 'rune' binary to
stop when the first filename is hit. This can be used in scripts to
ensure that all arguments passed to the script by userland are passed
through to the rune program. Adjust the tests/*.
* Interpret the assigned expression for the 'global constant' case to
obtain an actual constant. Only implemented for base types for now
(allowing us to punt on dealing with lookups on constant arrays - which
will be placed in the global data space as before).
* Misc docs work.
* The 'rune' frontend is now set to dump core only if -d is specified.
Rune binaries will still dump core at run-time if a dpanic() occurs
regardless.
* Fix some weirdness in ParseExp() that was breaking sizeof(a.b)) and
arysize(a.b). Still not sure what I meant the original code to do!
* Add and document the 'Z' suffix to indicate 'size_t', aka the only
machine-specific integral type in Rune.
* Finish libruntime's main() support. Properly generate argc and argv.
Also make sure file descriptors 0, 1, and 2 exist.
Matthew Dillon [Fri, 1 Apr 2016 17:43:19 +0000 (10:43 -0700)]
Rune - Stabilization (constructors, destructors), misc.
* Stabilize the constructor and destructor call algorithm and fix a bug
where we were improperly diving SCOPE_LVALUE declarations. Mostly
in GenCallContentGroup() and RunCallContentGroup().
* Fix bugs in the Gen*ContentGroup() that was not handling SCOPE_UNLOCKED
properly. Document the cases to reduce future confusion (this is complex
stuff).
* Fix missing BEStackData in GeneratePushTmpDefaultContext().
* Run constructors after sub-constructors, run destructors before
sub-destructors.
* Clarify how partial assignments (e.g. Fubar futz = (x:23); ) are handled
verses complete replacements. Constructors are still run for partial
initializations, whereas with a complete replacement constructors on the
target should not be run because they've already been run on the source.
* No longer run constructors on persist mappings that have already been
initialized. This is for consistency but also because trying to separate
out the constructor and type-default initialization to handle the case
I originally desired got too messy.
* Fix a bug in the resolver where declarations were being added to the
various SemGroup->sg_*Base lists too late, preventing important type
flags from getting set in certain reentrant situations.
* Move libdthread/refstor.c to libruntime which is where it really belongs.
* Start fleshing out and documenting the core class hierarchy.
* Continue to flesh out stdio, including full duplex streaming support.
Matthew Dillon [Fri, 1 Apr 2016 04:08:58 +0000 (21:08 -0700)]
Rune - Syscall work, stabilization
* Fix bug in threadSleep*() where ms == 0 was not doing an indefinite
sleep. Mainly for WCHAN sleeping.
* Fix bug in EmitRAS_SemanticEnter() where LVALUE scope was being ignored,
causing the zeroing code to overflow the type.
* Fix bug in GenAllocReturnBase() where the data storage status was not
being properly set, resulting in a lock count mismatch.
* Remove some debugging, convert other debugging from stdout to stderr.
* Add a bunch of fd-related system calls. Also change ('fix') the default
behavior for read() and write() to use non-blocking I/O, loop internally
if necessary, and also handle EINTR restarts.
* Greatly simplify the Syscall structure. We no longer use sc_RetType
or sc_Args so get rid of them.
* Remove File.fgets(), add File.fgetln(). Also add File.fgetbuf()
which is similar to fgetln() in avoiding copies, but operates on
the whole buffer when possible instead of line-by-line.
* Fix fread(), fwrite(), and related stdio functions to work properly
with non-blocking I/O and EINTR.
Matthew Dillon [Thu, 31 Mar 2016 22:28:31 +0000 (15:28 -0700)]
Rune - Collapse system calls and external libraries
* Make the interpreter use the run-time system call API and remove all
interpreter-specific generation for same.
This makes interfacing to C via external libraries MUCH easier.
* The interpreter is now using libruntime a whole lot more, and this also
means that the interpreter is starting to use libruntime's ref and lock
code as well.
Matthew Dillon [Thu, 31 Mar 2016 02:23:53 +0000 (19:23 -0700)]
Rune - Stabilization
* Adjust interpreter to match the code generator w/regards to dropping
content. Remove the memset() that was used for debugging and properly
null-out pointers, references, and lvalues (leaving their s_Type field
intact).