Matthew Dillon [Wed, 27 Dec 2017 23:34:02 +0000 (15:34 -0800)]
Rune - Flesh out compile-time global initializations
* Flesh out compile-time global initializations. Fix a recursion
into the same expression when the code generator invokes the
interpreter to resolve a global pointer assignment.
* Document more features.
Matthew Dillon [Wed, 27 Dec 2017 05:30:46 +0000 (21:30 -0800)]
Rune - Fix FP save and restore with mixed ABI calls
* Rune currently uses MSABI which requires XMM6-15, RDI, and RSI
to be saved by procedures in addition to the standard Linux/BSD
ABI which requires procedures to save RBX, RBP, and R12-R15.
We do this because Rune can't inline as well as a C compiler
and often has to make internal ABI calls (which also use MSABI)
in the middle of code the programmer might otherwise believe
does not have to make procedure calls. We want to be able to
maintain FP variables in XMM registers without repeated saving
and restoring. The addition of RBX and RBP to the callee-saves
set also helps Rune's register allocator perform better.
* Fix issues when Rune is calling into a linux/BSD ABI (i.e.
when interfacing with C, making system calls, etc). In this
situation, the Rune procedure was not saving registers that
could get trashed by the system call or C call).
Matthew Dillon [Wed, 27 Dec 2017 05:30:21 +0000 (21:30 -0800)]
Rune - Remove some debugging
* Remove a bit of debugging
Matthew Dillon [Sat, 15 Apr 2017 00:46:05 +0000 (17:46 -0700)]
Rune - Copy tracking minfo in GenPointerInd2()
* Copy tracking minfo in GenPointerInd2(). This removes a few
unnecessary get/put sequences in the generated output.
Matthew Dillon [Mon, 6 Mar 2017 03:41:57 +0000 (19:41 -0800)]
Rune - Change ABI
* Change the ABI to be more like the microsoft call ABI. The main reason
is that fewer registers are scratch across calls, which significantly
reduces the effort the RAS register allocator must make due to the large
number of RuneRunTime_*() calls that are made.
Matthew Dillon [Sat, 4 Mar 2017 00:57:01 +0000 (16:57 -0800)]
Rune - Clarify documentation for ops on reference types
* Clarify the documentation on the limitations of certain types of
operations on reference types.
Matthew Dillon [Sat, 4 Mar 2017 00:31:04 +0000 (16:31 -0800)]
Rune - Convert most library pointers to references
* In Rune we want to encourage the use of references over pointers
at every opportunity. Since no pointer arithmatic is allowed on a ref
and because it will either be NULL or valid, Rune can optimize refs
more easily.
Matthew Dillon [Sat, 4 Mar 2017 00:14:26 +0000 (16:14 -0800)]
RAS - Fix .zero alignment
* .zero alignment was misintpreted as a fill value. Generate a separate
.align pseudo-op for any alignment > 1.
Matthew Dillon [Sat, 11 Feb 2017 20:53:34 +0000 (12:53 -0800)]
Rune - save/restore errno in signal handlers
* Save and restore errno in signal handlers so as not to interfere
with mainline code.
Matthew Dillon [Sun, 11 Dec 2016 18:59:50 +0000 (10:59 -0800)]
Rune - Reorder weak bindings
* Reorder weak bindings for newer versions of GCC which require fully
formed types for weak bindings.
Matthew Dillon [Thu, 15 Sep 2016 02:39:07 +0000 (19:39 -0700)]
Rune - Consolidate most RefStor allocations
* Consolidate most RefStor allocations into allocRefStor().
* Start tracking potential PointerInfo extensions to stack frames and
RefStor's to allow PointerInfo structures to be made static. This
isn't set in stone yet and the counter / index fields are not yet
used.
Matthew Dillon [Thu, 15 Sep 2016 02:34:25 +0000 (19:34 -0700)]
Rune - Optimize getrgd()
* getrgd() has a specific problem in that GCC is badly broken when it comes
to accessing TLS variables in a NxM threading scheme. GCC will cache the
%fs:0 load into a register and nothing we tell it can get it to throw
away that cached value. In a NxM threading scheme, a micro-thread can
block and wind up on a different pthread when it wakes up.
This is particularly painful when GCC does heavy optimizations that
combine what would normally be separate procedural domains.
* The getrgd() I had before was portable, but painfully expensive (two
subroutine calls!). I've decided to optimize getrgd() into an inline
and use __asm to manually retrieve the per-pthread &RuneTLS. This
is not really portable, but it's very important the Rune be able to
switch micro-threads quickly.
Matthew Dillon [Wed, 14 Sep 2016 18:20:48 +0000 (11:20 -0700)]
Rune - Fix interpreter (tests/gfxinput2.d)
* Bring the interpreter's initialization of the PointerInfo structure
up-to-date. type->ty_Info was not being set to PINFO_STATIC. This
fixes interpretation of a number of test programs.
Matthew Dillon [Wed, 14 Sep 2016 06:45:16 +0000 (23:45 -0700)]
Rune - Fix deadlock in rune threading system
* Fix a deadlock in Rune's sideband locks. An unlock->wakeup->start
sequence needs to hold two sideband locks. To avoid the deadlock,
relock in address-sorted order if the initial attempt to obtain the
second lock fails (trying with a non-blocking call).
Matthew Dillon [Wed, 14 Sep 2016 04:52:53 +0000 (21:52 -0700)]
Rune - Start working on registerizing inlined procedure arguments (2)
* Fix a few bugs. It's a bit fragile so I put the important pieces
around #ifdef's.
* Enable the feature.
* Make sure that normal procedure call return values are not cached,
the result vector has to be memory.
Matthew Dillon [Tue, 13 Sep 2016 23:58:12 +0000 (16:58 -0700)]
Rune - Start working on registerizing inlined procedure arguments
* Start working on registerizing inlined procedure arguments. Currently
disabled (it almost works but the tests/gfx* programs have some problems
so clearly I'm missing something).
Matthew Dillon [Mon, 12 Sep 2016 23:13:20 +0000 (16:13 -0700)]
Rune - Make code PointerInfo-centric
* The PointerStor and LValueStor structures now contain only two elements,
the address pointer and a pointer to a PointerInfo structure.
* The PointerInfo structure is now the center of the universe. Make it
slightly larger to accomodate a in_Refs field, but ultimately we want
to incorporate it into related RefStor's and stack contexts and not
need any dynamic allocation.
* In this new scheme we have to pass a proper PointerInfo element in the
constructor/destructor (EmitRAS_CallConstructorDestructor()) code.
Use the method argument's type->ty_Info. TODO - missing RefStor for
re-ref case.
* PointerStor and LValueStor are now 16-byte-aligned. This significantly
reduces the cost of pointer copies (we can use %xmm0). There is still
a C compatibility structure somewhere that requires the __aligned(16)
extension (which is in-place with this commit) and is getting confused,
but I'd like to find and fix that with explicit pad.
* TODO - There's a memory leak as of this commit. PointerInfo's are being
allocated for lvalue args but not freed. Will have to change the
mechanic somewhat.
Matthew Dillon [Mon, 22 Aug 2016 02:12:43 +0000 (19:12 -0700)]
Rune - Cleanup PointerInfo use in interpreter
* Cleanup PointerInfo handling to track the structure and free it after
use.
* Optimize NULL handling.
* Scrap StrTable hacks previously used to track PointerInfo for quoted
strings.
Matthew Dillon [Sun, 21 Aug 2016 07:04:34 +0000 (00:04 -0700)]
Rune - Initial PointerStor/LValueStor rework
* Initial rework, make PointerStor and LValueStor much smaller structures.
Basically just { ptr, info } now, and move the fat fields one indirection
level down into the info.
* This commit I just mocked-up PointerInfo allocations so I could test with
the interpreter. They leak like crazy.
Matthew Dillon [Mon, 22 Aug 2016 02:10:10 +0000 (19:10 -0700)]
Rune - Fix interpreter's BOOL_BINARY_OP
* We can't use the 64-bit short-cut for floating point comparisons.
Fixed.
Matthew Dillon [Sat, 20 Aug 2016 06:24:52 +0000 (23:24 -0700)]
Rune - Add Unchecked option (-u)
* Add -u option. This disables all bounds checking in any generated
output. Interpreted programs will still do bounds-checking.
* Generate help if no arguments or unrecognized option specified.
Matthew Dillon [Sat, 20 Aug 2016 05:49:28 +0000 (22:49 -0700)]
Rune - Select stronger alignment when it makes sense
* Use a larger alignment for arrays, compound expressions, and procedure
arguments when the type is large enough.
* Improves copying and zeroing by allowing RAS to rollup fields into bigger
registers and/or use the %xmm registers.
Matthew Dillon [Sat, 20 Aug 2016 04:18:15 +0000 (21:18 -0700)]
Rune - Fix array passing, optimize constant array indices
* Fix array passing (arrays are pased by value), interpreter and code
generator.
* Optimize constant array indices, removing unnecessary instructions
which are only calculating or testing constants.
Matthew Dillon [Fri, 19 Aug 2016 23:38:12 +0000 (16:38 -0700)]
Rune - Change limport mechanic
* Use quoted import paths to mean as-self (aka limport), and angle-bracketed
paths to mean under an identifier. So e.g. one would normally import "sys";
and import <third_party_library>.
* Implement angle-bracket strings for import. This also allows ANSI
string concatenation, including with angle-brackets if you really want to.
* Fix string concatenation by limiting the successive concatenations to the
same quoting mechanism as the first one.
* Remove the 'limport' keyword.
* This mechanism is more readable in terms of alignment.
Matthew Dillon [Fri, 19 Aug 2016 22:37:56 +0000 (15:37 -0700)]
Rune - Add labeled blocks
* Allow statement blocks to be labeled. The statement block associated with
a procedure body cannot be labeled but may be referred to as "procedure".
* Allow block labels to be specified in the 'break' and 'continue'
statements.
Matthew Dillon [Fri, 19 Aug 2016 04:57:49 +0000 (21:57 -0700)]
RAS - Improve register allocator
* Remove the conditional bypass optimization for now, it actually makes
register allocation worse by messing up register life-time tests even
in simple procedures.
* Clarify comments a bit.
Matthew Dillon [Thu, 18 Aug 2016 18:01:47 +0000 (11:01 -0700)]
Rune - Pass-through -O[n], drop more insns in RAS
* The 'rune' command passes -O[n] through to RAS.
* Fix the drop-test in RAS. A non-WRITE EA could prevent an instruction
from being dropped.
This fixes unnecessary double initializations, primarily from pre-zeroing
of variables on semantic entry which are immediate initialized again via
the code.
Matthew Dillon [Thu, 18 Aug 2016 18:00:30 +0000 (11:00 -0700)]
Rune - Handle const array lookups.
* Array lookups on constants (aka strings) are now constant-optimized
in both the interpreter and the code generator.
Matthew Dillon [Thu, 18 Aug 2016 16:38:32 +0000 (09:38 -0700)]
Rune - Add strdup(), strcmp(), strncmp()
* Add strdup(), strcmp(), strncmp()
* Flag string functions which can be pure.
Matthew Dillon [Thu, 18 Aug 2016 16:36:59 +0000 (09:36 -0700)]
Rune - Fix ptr->int cast bug
* Fix var-args ptr-to-int cast, it was improperly taking the
address of the PointerStor instead of taking PointerStor.s_Addr.
Matthew Dillon [Thu, 18 Aug 2016 05:08:00 +0000 (22:08 -0700)]
Rune - Stabilization pass (CLOEXEC work, return-storage for internal calls)
* All system calls which create a new file descriptor always return the
descriptor with O_CLOEXEC set. open, pipe, socketpair, dup, dup2,
opendir.
* Implement proper return storage semantics for internal calls that return
non-trivial structures (aka pointers), and properly pre-initialize the
return storage.
* Rename some API calls in classes/* and clean-up.
* Fix the exec code for the case where the requested descriptors for
0, 1, and 2 are mis-ordered.
Matthew Dillon [Sat, 13 Aug 2016 04:16:01 +0000 (21:16 -0700)]
Rune - Rework resolver
* Change the first pass model to a multi-pass model that allows portions
of the tree to be requeued for resolve when they cannot be immediately
resolved.
* Make sure the dynamic methods are properly resolved. When a method
call is made through a reference the version of the method in any
subclass may end up being called and those procedures must be resolved.
However, we do not have to resolve method procedures in subclasses which
are never instantiated by the program, since those methods can never be
called.
* As part of the dynamic method work the various passes must handle
the recursion to resolve dynamic methods the same way, typically
by detecting a dynamic method call and iterating through d->d_Sub*
(recursively).
* Generate the dynamic index for method calls for all possible method
vectors for consistency. However, particular vectors which the resolver
knows will never be executed will be populated as NULL by the code
generator, significantly reducing the size of the code being output.
* Stabilization pass on some of the tests, fix a few issues here and
there such as missing ResolveDecl() calls when resolving operators
and an incorrect EXF_ADDRUSED flag when processing an array lookup.
* Rename functions for consistency.
Matthew Dillon [Sat, 13 Aug 2016 00:45:43 +0000 (17:45 -0700)]
Rune - Start work on inlining
* Start work on inlining procedures.
* Add ct_ArgBase, abstract-out the argument base.
* Optimize stack allocations a bit by flagging when a procedure might make
ABI calls.
* Get rid of the Exp 'copy' stuff, dup the Exp earlier rather than later
(fix several bugs where undup'd exp's were being unintentionally
modified).
* Detect dynamic method calls, index refined subclass methods via
d->d_SubBase and d->d_SubNext.
* gfxinput2.d test works but gfxbutton2.d can't resolve due to resolver
loops. Resolver is in a bit of a pickle and will have to be written
after this commit.
Matthew Dillon [Thu, 11 Aug 2016 03:18:31 +0000 (20:18 -0700)]
Rune - Adjust autocast code to fix implied compound expression
* Check for a cast operator for (scaler) -> (compound) before
automatically casting the scaler to compound. Before we automatically
cast the scaler first and the cast operator (if present) would never
be run. (tests/cast.d)
* (scaler) and scaler needs to be deterministic, so treat them both the
same - as a scaler. Otherwise simple parenthesis can cause confusion.
If doing a compound assignment this means we now require a named element
for a single-scaler compound if the programmer wants the scaler to use
an actual or implied scaler -> compound cast.
* Cleanup some debugging messages we no longer need.
Matthew Dillon [Thu, 11 Aug 2016 01:52:41 +0000 (18:52 -0700)]
Rune - Main should exit without waiting for threads
* The main() now exits when it returns, no longer waiting for
other threads to finish first.
The programmer can decide to wait for not with Thread.waitThreads();
Matthew Dillon [Thu, 11 Aug 2016 00:48:40 +0000 (17:48 -0700)]
Rune - refactor resolve-time constant evaluation
* Refactor resolve-time constant evaluations (e.g. in order to be able
to resolve array types and certain && and || optimizations).
Remove the conditionalized paths in resolve*() for RESOLVE_CONSTEXP.
Instead, we evaluate these issues in the storage-alignment pass.
* This makes tests/pure.d work.
Matthew Dillon [Thu, 11 Aug 2016 00:46:03 +0000 (17:46 -0700)]
Rune - minor cleanup
* Minor cleanup and fix a few issues in tests
Matthew Dillon [Wed, 10 Aug 2016 21:32:04 +0000 (14:32 -0700)]
Rune - Enhance && and || constant handling a bit
* The rhs can also short-cut an && or || if both the lhs and rhs are
constants.
Matthew Dillon [Wed, 10 Aug 2016 18:14:33 +0000 (11:14 -0700)]
Rune - Change thead bootstrapping, fix RunLVSAlloc()
* Thread bootstrapping now turns the calling thread into the first Rune
thread instead of pausing the calling thread.
Fixes issue with compile-time interpretation for constants.
* RunLVSAlloc() now locks the underlying refstor according to d->d_Storage.
Also simplify the API.
Matthew Dillon [Tue, 9 Aug 2016 06:51:39 +0000 (23:51 -0700)]
Rune - change 'locked' to 'hard'
* Change the confusing 'locked' scope qualifier to 'hard' which conforms
more to what it actually does.
Matthew Dillon [Tue, 9 Aug 2016 05:54:03 +0000 (22:54 -0700)]
rune - Fix tests/build1.d
* Fix tests/build1.d so it does something.
Matthew Dillon [Tue, 9 Aug 2016 05:52:50 +0000 (22:52 -0700)]
Rune - resolve the alignment for internal types immediately
* Internal types may be implied during resolution, be sure to
completely resolve its alignment too.
* Note that other (non-internal) types might have recursive dependencies,
and their alignment is not resolve in the normal resolveType() code.
Matthew Dillon [Tue, 9 Aug 2016 05:29:20 +0000 (22:29 -0700)]
rune - Fix -rpath and -L<libpath>
* Use the passed-in destination path to construct -rpath and -L when
linking a generated rune program, instead of hardwiring /usr/local/rune/*
Matthew Dillon [Tue, 9 Aug 2016 05:27:15 +0000 (22:27 -0700)]
rune - Fix generated (*pt).new for double-indirect
* Fix generated (*pt).new for double-indirect (tests/heap.d)
* Remove a test in the run-time that the generator does not have. I
think we don't need this type test, and it breaks the test.
* Fix tests/heap.d assertion, base must be unlocked.
Matthew Dillon [Fri, 15 Apr 2016 02:39:59 +0000 (19:39 -0700)]
Rune - Interpreter locking work 5/n
* Add missing locks in destructor
* Fix incorrect mrs lock (fixes test/mmap.d)
Matthew Dillon [Thu, 14 Apr 2016 23:01:08 +0000 (16:01 -0700)]
Rune - Update README
* Update README and make the perl and python loop tests conform to the
Rune 1-billion loop test (tests/loop1b.d).
Matthew Dillon [Thu, 14 Apr 2016 22:36:57 +0000 (15:36 -0700)]
Rune - Fix build without make obj
* When rune is built without make obj the .PATH libruntime was using
confuses the hell out of the build.
* Remove .PATH, use a source file #include trick instead to bring the
two source files from librune into libruntime that libruntime needs.
Reported-by: mneumann
Matthew Dillon [Thu, 14 Apr 2016 20:03:23 +0000 (13:03 -0700)]
Rune - Interpreter refactor ex_Run 3/n
* Fix DCOPY semantics for the interpreter. Flag that the rundata must be
thrown away and operate on ct_DCopyData instead of making a copy.
* Optimize Run64 for constant collapses. Also redo RunCachedConstantExp()
to simplify the code that uses it and to optimize more cases.
Matthew Dillon [Thu, 14 Apr 2016 17:31:29 +0000 (10:31 -0700)]
Rune - Interpreter refactor ex_Run 2/n
* Implement exp->ex_Run64() - A shortcut function which returns integral
rvalues up to 64-bits. This causes most expressions to self-optimize
significantly, to the point where the interpreter is able to omit a great
deal of locking in the critical path as well in some cases to omit nearly
all procedure calls other than the primary recursion when evaluating an
integral expression.
Cuts loop-1billion test from 20 seconds to 7 seconds on a 3.4 GHz Haswell
box. This is far better than the previous pointer-return optimization
which could only get it down to ~14 seconds or so. A native Rune compile
runs the same loop in 0.28 seconds. 25:1 is not bad for an interpreter
running a tight loop verses a compiled tight loop!
* Implement for nearly all internal integral operators.
* Implement for integral Declaration access, and implied void casts.
* Implement for integer-integer casts.
Matthew Dillon [Thu, 14 Apr 2016 05:55:19 +0000 (22:55 -0700)]
Rune - Interpreter refactor ex_Run 1/n
* Refactor ex_Run to not return a pointer to the object. This slows things
down a bit until the next commit.
Matthew Dillon [Thu, 14 Apr 2016 04:17:25 +0000 (21:17 -0700)]
Rune - Interpreter locking work 4/n
* This is a bit messy, but implement locking modes in the interpreter.
Fix a couple of bugs while doing so.
* Rune compiler needs to run in the rune threading environment now because
the lock code assumes it (the compiler can run the interpreter to resolve
constants).
* (temporary) don't lock declarations which already have an implied lock
whos addresses are not taken. Fixes performance issue with simple
expression. This is temporary, actual fix needs to test exp->ex_Flags
for use case.
Matthew Dillon [Tue, 12 Apr 2016 01:47:55 +0000 (18:47 -0700)]
Rune = Change ulong -> uintptr_t
* Change a non-standard use of ulong (in C) to uintptr_t, which is what it
was proxying for. If the atomic_*_long() functions aren't compatible with
uintptr_t we have bigger problems.
Matthew Dillon [Mon, 11 Apr 2016 01:07:05 +0000 (18:07 -0700)]
Rune - Interpreter locking work 3/n
* Abstract most manual copy fixups and pre-copy drops in preparation for
real locking.
Matthew Dillon [Sun, 10 Apr 2016 05:01:53 +0000 (22:01 -0700)]
Rune - Interpreter locking work 2/n
* Add fields to rundata_t (RunData) roughly similar to what we needed
in gendata_t (GenData) for locking support.
Matthew Dillon [Sun, 10 Apr 2016 01:20:34 +0000 (18:20 -0700)]
Rune - Interpreter locking work 1/n
* Revamp RunDiscard*() to take the sideband data pointer so we don't have
to load it into the rundata structure.
* Revamp DCopyData to allow RunGetExpTmpData() to be called at any point
instead of at the beginning of an ex_Run() procedure. This allows the
function to reuse the passed-in rundata_t structure.
* Other changes in preparation for expanding rundata_t for object locking
support.
Tomohiro Kusumi [Mon, 11 Apr 2016 08:23:46 +0000 (17:23 +0900)]
Rune - Fix compile errors/warnings
on FreeBSD/clang.
Other compile time errors that I didn't go far as to change are
1. cpu_ccfence() which is DragonFly specific function for inline asm
2. *(int*)0 = 1;
3. (StrTable *)(__DECONST(char *, str) - xxx);
Matthew Dillon [Fri, 8 Apr 2016 17:56:16 +0000 (10:56 -0700)]
Rune - Threaded function detach work
* General thread performance check. Co-routine switch overhead (e.g.
synchronous threaded call which does not detach) is around 250ns.
Threaded call with detach is 500ns to 1uS or so. Pretty good. I would
like to improve the co-routine switch overhead.
* Threaded procedures now actually run synchronously until they detach,
instead of using parent-blocking. Overhead is significantly reduced in
the SMP case.
* Finally add the code to transfer ownership of acquired locks when a
threaded procedure detaches, so the threaded procedure's thread properly
owns the procedure's locks.
* Code generator now implements the TSCHED instruction.
* Implement a GlobFreeQueue to shift excess cached entries from per-pthread
FreeQueue. Fixes a degenerate case where one pthread keeps producing
threads.
Matthew Dillon [Thu, 7 Apr 2016 07:21:27 +0000 (00:21 -0700)]
Rune - Content-locking work 1/2
* Separate gendata->implied_mask out from gendata->s_Beg so function is
more obvious. Fix two sbindex bugs while we are at it. One was a
missing increment, and the other a broken algorithm related to handling
compound types.
* For now treat compound types (that are not arguments) like classes and
do not content-lock their contents. The intended content-locking
recursion is more complex than expected.
* Clarify code documentation.
* When pointers or references are passed as lvalues, the resolver and
code generator now validate that the content-locking mode is compatible.
There is currently a hack for 'this' pointers (only used for e.g.
ptr.new()) to allow mismatches.
Matthew Dillon [Wed, 6 Apr 2016 23:38:25 +0000 (16:38 -0700)]
Rune - Correct type-o
* Cerroct tope-y
Reported-by: mneumann
Matthew Dillon [Wed, 6 Apr 2016 16:13:00 +0000 (09:13 -0700)]
Rune - do a pass on the README
* Do a pass on the README
Matthew Dillon [Wed, 6 Apr 2016 15:11:15 +0000 (08:11 -0700)]
Rune - retool rundata_t
* Interpreter needs to start catching up to the code generator in terms of
being able to lock data structures. I was able to punt on it before but
now it is beginning to matter. Even single-threaded there are
situations, such as in File.fflush(), where a synchronous switch due to
a blocking write() will cause problems.
* Do a major retool of rundata_t to allow the structure to be expanded.
We previously relied on its small size (two fields) and GCC's
pass-by-value optimizations. This made it impossible to expand since
pass-by-value, particularly pass-by-value returns, become a massive
optimization failure with larger structures. Change:
rundata_t ex_func(runctx_p ct, Exp *exp)
to:
void *ex_func(runctx_p ct, rundata_t *data, Exp *exp)
Where the data->data portion of data is returned directly as a pointer
(removing an indirection for quick accesses to the data). The rundata
structure itself can be passed down the stack.
An alternative is to also return the rundata_t * which can theoretically
remove a register push/pop for it around the procedure call. However
there are enough other procedure calls that this doesn't save us a whole
lot, and it also forces a sequential indirection which can cause cpu
stalls.
* Initially expand rundata_t by embedding temporary space which is easier
to get at than InterpGetExpTmpData(). This space can be used for most
of the canned operators.
* Reoptimize run_oper.c. Do a better job collapsing stack-variable cases
to remove additional calls and indirections. Unary ops on stack
variables and binary ops (stack-variable, constant) are much faster now
for both lvalue and non-lvalue ops. Also optimize the pointer arithmatic
ops.
This cuts my loop1b.d test times in half.
Matthew Dillon [Wed, 6 Apr 2016 01:33:57 +0000 (18:33 -0700)]
Rune - Tighten up parser, stabilization, optimization.
* Get const storage errors mostly working. Disallow assignments to constant
storage, disallow casts from pointer-to-const to pointer-to-nonconst.
Mostly a matter of fixing SimilarType() and adding additional cases
in similar routines. Fix a few CONST-vs-not tests at the same time.
* Generate better error message when any attempt is made to cast a pointer
to a complex class (one containing lvalues, pointers, or references) to
(void *). This is not allowed.
* Allow any pointer to be cast to size_t (so the address can be printed).
Before we were doing a double cast: (size_t)(void *)ptr, which no longer
works generically.
* Optimize initial loads to remove unnecessary PPUTs (and related). Mainly
effects the generation of procedure argument vectors for procedure calls.
* Fix missing GENSTAT_LOCKH settings in a few places (there are probably a
few more remaining).
Matthew Dillon [Tue, 5 Apr 2016 20:34:34 +0000 (13:34 -0700)]
Rune - Implement hard-locking model feature
* Add the 'locked' scope qualifier. This qualifier may be specified to the
left of 'method' in a method procedure to automatically hard-lock the
method object, and can be specified for argument and local variable
declarations to indicate that their content locks be hard rather than soft.
* Add a ton of supporting instructions. Basically add an 'H' version for
all the STK, STS, LV, P, and RS instructions which lock or unlock
storage. For example, PGETH rather than just PGET.
* Cleanup the documentation.
* Cleanup a few places in the class library that require hard locks
(e.g. File.fflush().
Matthew Dillon [Tue, 5 Apr 2016 16:11:28 +0000 (09:11 -0700)]
Rune - Enforce result; requirements
* Enforce result statement requirements in the resolver. The "result"
statement must be executed in the top-level of a procedure and once
executed access to any procedure argument or return storage is prohibited
(because it is no longer available now that the procedure has been
detached).
Matthew Dillon [Tue, 5 Apr 2016 06:39:46 +0000 (23:39 -0700)]
Rune - More autocasts for integer constants
* If the resolver is unable to find a matching operator and one side
of the expression is an integer or floating constant, the resolver
will now search for an operator with that side auto-cast to the
type of the other side.
If the resolver successfully finds an operator it will then test that
the value of the integer constant remains unchanged after the autocast.
If the value is unchanged the autocast is allowed. If the value has
changed the autocast is NOT allowed.
* This fixes a number of annoyances when the programmer desires to use
simple obvious integer constants with or without a suffix whos type does
not match the operator. For example, the expression (a + 1) would have
previously failed if the type of 'a' was not int32_t (because the integer
constant is of type int32_t). This expression will now succeed.
Matthew Dillon [Tue, 5 Apr 2016 05:38:40 +0000 (22:38 -0700)]
Rune - Documentation update
* Flesh out and revise.
Matthew Dillon [Tue, 5 Apr 2016 05:07:19 +0000 (22:07 -0700)]
Rune - Remove collapsed libraries
* Remove libraries no longer used from the rune link. Everything was
folded into libruntime.
Reported-by: lhmwzy
Matthew Dillon [Tue, 5 Apr 2016 04:54:37 +0000 (21:54 -0700)]
Rune - crater push test
* crater push test again.
Matthew Dillon [Tue, 5 Apr 2016 04:48:27 +0000 (21:48 -0700)]
Rune - crater push
* Edit README
Matthew Dillon [Tue, 5 Apr 2016 04:14:08 +0000 (21:14 -0700)]
Rune - cleanup obsolete doc files
* Cleanup old doc files and musings that are no longer applicable.
Matthew Dillon [Tue, 5 Apr 2016 04:03:59 +0000 (21:03 -0700)]
Rune - fix missing GROUP env
* Some shells don't set GROUP. Also add logic for USER, just in case.
Reported-by: lhmwzy
Matthew Dillon [Tue, 5 Apr 2016 02:30:39 +0000 (19:30 -0700)]
Rune - pretest crater repo
* pretest crater repo
Matthew Dillon [Tue, 5 Apr 2016 02:19:15 +0000 (19:19 -0700)]
Rune - Simplify the library topology
* Collapse libsupport, libruntime, and libdthread into just libruntime.
The libraries were already interdependent due to other work such as having
the interpreter use the runtime internal call linkages. We really only
need one library here. We keep libgen broken out for parse-time and the
interpreter. libruntime is used by everyone to varying degrees.
Matthew Dillon [Tue, 5 Apr 2016 01:38:03 +0000 (18:38 -0700)]
Rune - add *env(), fexec(), fwait().
* Add various environment handling functions, fexec(), and fwait().
* If envp is passed as NULL, environ is used for the environment.
Matthew Dillon [Mon, 4 Apr 2016 06:51:05 +0000 (23:51 -0700)]
Rune - Stabilization, system calls
* Flesh out FdLock and FdMap.
* Add Sys.panic(), Sys.exit()
* Allow procedure argument defaults to reference the procedure arguments.
In particular a method call can access the object via 'this' and use it
to set an argument default. At the moment this requires hacking
*GetDeclarationData() to detect the case.
* Fix two bugs in the code generator related to argument defaulting in
compound expressions (aka procedure argument declarations).
* Fix a bug in the calculation of temporary space for type defaults in
resolveStorageExpOnly() and resolveStorageType().
Matthew Dillon [Sun, 3 Apr 2016 07:14:06 +0000 (00:14 -0700)]
Rune - Flesh out core
* Buffered fopen/fdopen/freopen/fdreopen and other support functions.
automatic duplex detection.
* Use pipe2().
Matthew Dillon [Sun, 3 Apr 2016 03:36:13 +0000 (20:36 -0700)]
Rune - Stability, features
* Improve the resolver and run-time to optimize constants accessed via
class path. e.g. char buf[Fs.PATH_MAX] is allowed.
* Add Fs.PATH_MAX and a number of other constants. Also do some file
splitting and cleanup.
* Flesh out the BlockedPThreads mechanic for creating additional pthreads.
add threadSetBlocking() and theadClrBlocking().
* Add stat, chmod, chown, utimes family of functions.
Add symlink, link, readlink, unlink.
Add dup, dup2, flock, and umask.
Matthew Dillon [Sun, 3 Apr 2016 00:48:31 +0000 (17:48 -0700)]
Rune - Stabilization, add Directory
* Add the Directory class and associated directory iterator and system
calls.
* Fix a bug in SRSGET (rgd->FreeRS corruption)
* Fix a bug in argv setup for the interpreter (increment i before breaking
out on xopt + filename encountered to properly position argv).
* Fix a bug in GenSemanticExit() (improper call to SRSPUT and other issues).
Matthew Dillon [Sat, 2 Apr 2016 21:29:27 +0000 (14:29 -0700)]
Rune - Stabilization, flesh out core classes
* Add many more constants.
* Rename class Fd to class Fs and rearrange the methods. Add additional
classes. Start working on Stream and StreamPair. Somewhat hide the
low-level pipe() and socketpair() calls. Give StreamPair similar methods
that make life easier.
* Fix a few cases where SGF_ADDRUSED and EXF_ADDRUSED were not being
propagated properly by the resolver. Hack the context code to check
the arguments for ADDRUSED to trigger SRSGET/SRSPUT in the procedure's
main context. Fix a related bug.
* Properly propagate ADDRUSED and ADDROF in exp.exp (TOK_DOT) expressions.
This introduces a bit of a performance regression due to extra LVRSGETs
which will be fixed later. Fixes a number of lvalue-related issues.
* Fix a ref-count issue revealed by a more sophisticated tests/fd_pipe.d
Also test fgetbuf(), a high-performance descriptor data streamer.
Matthew Dillon [Sat, 2 Apr 2016 04:34:25 +0000 (21:34 -0700)]
Rune - RAS 3-op optimizations, fix dthread bug
* Optimize most 3-operand instructions to remove an intermediate move
if the target is a direct register.
* Fix a deadlock in libdthread's event-handling that could lockup the
timer code.
Matthew Dillon [Sat, 2 Apr 2016 02:52:18 +0000 (19:52 -0700)]
Rune - Features (scripting, constants, main)
* Add -x option that causes argument processing by the 'rune' binary to
stop when the first filename is hit. This can be used in scripts to
ensure that all arguments passed to the script by userland are passed
through to the rune program. Adjust the tests/*.
* Interpret the assigned expression for the 'global constant' case to
obtain an actual constant. Only implemented for base types for now
(allowing us to punt on dealing with lookups on constant arrays - which
will be placed in the global data space as before).
* Misc docs work.
* The 'rune' frontend is now set to dump core only if -d is specified.
Rune binaries will still dump core at run-time if a dpanic() occurs
regardless.
* Fix some weirdness in ParseExp() that was breaking sizeof(a.b)) and
arysize(a.b). Still not sure what I meant the original code to do!
* Add and document the 'Z' suffix to indicate 'size_t', aka the only
machine-specific integral type in Rune.
* Finish libruntime's main() support. Properly generate argc and argv.
Also make sure file descriptors 0, 1, and 2 exist.
Matthew Dillon [Fri, 1 Apr 2016 17:43:19 +0000 (10:43 -0700)]
Rune - Stabilization (constructors, destructors), misc.
* Stabilize the constructor and destructor call algorithm and fix a bug
where we were improperly diving SCOPE_LVALUE declarations. Mostly
in GenCallContentGroup() and RunCallContentGroup().
* Fix bugs in the Gen*ContentGroup() that was not handling SCOPE_UNLOCKED
properly. Document the cases to reduce future confusion (this is complex
stuff).
* Fix missing BEStackData in GeneratePushTmpDefaultContext().
* Run constructors after sub-constructors, run destructors before
sub-destructors.
* Clarify how partial assignments (e.g. Fubar futz = (x:23); ) are handled
verses complete replacements. Constructors are still run for partial
initializations, whereas with a complete replacement constructors on the
target should not be run because they've already been run on the source.
* No longer run constructors on persist mappings that have already been
initialized. This is for consistency but also because trying to separate
out the constructor and type-default initialization to handle the case
I originally desired got too messy.
* Fix a bug in the resolver where declarations were being added to the
various SemGroup->sg_*Base lists too late, preventing important type
flags from getting set in certain reentrant situations.
* Move libdthread/refstor.c to libruntime which is where it really belongs.
* Start fleshing out and documenting the core class hierarchy.
* Continue to flesh out stdio, including full duplex streaming support.
Matthew Dillon [Fri, 1 Apr 2016 04:08:58 +0000 (21:08 -0700)]
Rune - Syscall work, stabilization
* Fix bug in threadSleep*() where ms == 0 was not doing an indefinite
sleep. Mainly for WCHAN sleeping.
* Fix bug in EmitRAS_SemanticEnter() where LVALUE scope was being ignored,
causing the zeroing code to overflow the type.
* Fix bug in GenAllocReturnBase() where the data storage status was not
being properly set, resulting in a lock count mismatch.
* Remove some debugging, convert other debugging from stdout to stderr.
* Add a bunch of fd-related system calls. Also change ('fix') the default
behavior for read() and write() to use non-blocking I/O, loop internally
if necessary, and also handle EINTR restarts.
* Greatly simplify the Syscall structure. We no longer use sc_RetType
or sc_Args so get rid of them.
* Remove File.fgets(), add File.fgetln(). Also add File.fgetbuf()
which is similar to fgetln() in avoiding copies, but operates on
the whole buffer when possible instead of line-by-line.
* Fix fread(), fwrite(), and related stdio functions to work properly
with non-blocking I/O and EINTR.
Matthew Dillon [Thu, 31 Mar 2016 22:28:31 +0000 (15:28 -0700)]
Rune - Collapse system calls and external libraries
* Make the interpreter use the run-time system call API and remove all
interpreter-specific generation for same.
This makes interfacing to C via external libraries MUCH easier.
* The interpreter is now using libruntime a whole lot more, and this also
means that the interpreter is starting to use libruntime's ref and lock
code as well.
Matthew Dillon [Thu, 31 Mar 2016 02:23:53 +0000 (19:23 -0700)]
Rune - Stabilization
* Adjust interpreter to match the code generator w/regards to dropping
content. Remove the memset() that was used for debugging and properly
null-out pointers, references, and lvalues (leaving their s_Type field
intact).
Matthew Dillon [Thu, 31 Mar 2016 01:56:27 +0000 (18:56 -0700)]
Rune - Stabilization
* Code generator must clean-out pointer, reference, and lvalue
elements when PUTting or RELeasing them. This is important for
consistency particularly during the destruction of dynamically
allocated structures.
* Implement in libruntime.
Matthew Dillon [Wed, 30 Mar 2016 23:47:34 +0000 (16:47 -0700)]
Rune - Add back-ticked strings and 'limport'
* Add back-ticked strings. These strings do not interpret escapes,
allowing you to embed single and double quotes, and backslashes.
Note that a back-ticked string may not contain a back-tick.
* Allow double-quoted and back-ticked strings to be mixed via concatenation.
i.e. "abc" `def` "ghi" `jkl\n` "\n".
* Add the 'limport' statement, which is a short-cut for 'import ... as self;'.
* Take a pass on grammer.html.
Matthew Dillon [Wed, 30 Mar 2016 21:05:02 +0000 (14:05 -0700)]
Rune - same treatment for RunFreeRefStor()
* Prevent freeRefStor() from chain-recursing releases by detecting the
recursion and linearizing. This prevents potential infinite-stack use.
* Undo the RefStor-in-mmap change. It will break badly due to pointers in
the embedded lock. We will need another solution.
Matthew Dillon [Wed, 30 Mar 2016 20:42:01 +0000 (13:42 -0700)]
Rune - Fix freeRefStor recursion
* Prevent freeRefStor() from chain-recursing releases by detecting the
recursion and linearizing. This prevents potential infinite-stack use.
* Take a pass on overview.html.
Matthew Dillon [Wed, 30 Mar 2016 06:23:19 +0000 (23:23 -0700)]
Rune - Optimization pass (DCOPY, NULL), cast ptr->bool feature
* Allow DCopyData to be chained and to survive simple casts and alias
recursions.
* Implement a simple optimization to allow DCOPY to be used for
assignments.
* Implement a hint called EX2F_CPTRONLY which the caller can set in a
sub-expression to indicate that it only needs the s_Addr field of the
pointer. This is rather dangerous since we must also be sure to not
attempt to PPUT or PREL or otherwise dereference/unlock the resulting
storage which might still be typed as a fat-pointer but has been
optimized to a thin-pointer.
This is used only for pointer comparisons for now.
* Optimize NULL to handle DCOPY and CPTRONLY.
* Add feature to auto-cast a pointer to a boolean, testing against NULL.
This allows e.g. if (ptr) ... rather than if (ptr != NULL) ...
Matthew Dillon [Wed, 30 Mar 2016 00:48:19 +0000 (17:48 -0700)]
Rune - Stabilization pass
* libdthread - Assert that the termination state is correct in libdthread.
* libdthread - Do not block in the event code if no threads remain.
* resolver - Tons of work to improve separation for the main resolve,
alignment, and storage assignment passes. Give them separate completion
flags.
(that might not seem like a lot but it took all day!).
* generator - Fix bug when initializing from a compound type's ed_AssExp.
Temporary storage is relative to the compound type's SemGroup. Also
fixe the run-time. This fixes issues where argument defaults for
procedure calls were imploding.
* Fix bug in gfx/window.d that was preventing the window from being
destroyed.
* Syscalls fleshing out. Use extpread() and extpwrite() for now (not
portable though).
* Separate out syscall categories into their own files.
Matthew Dillon [Tue, 29 Mar 2016 02:44:17 +0000 (19:44 -0700)]
Rune - Organizing pass
* Change dassert(0)'s to dpanic("msg");
* Reorganize libgen/run_syscalls.c and libgen/gen_syscalls.c. Combine the
run_* and gen_* arrays together then split the system calls into
categories (combining run_* and gen_* for each category).
Adds sys.c, sys_fd.c, sys_math.c, sys_misc.c, sys_thread.c.
* Start work on additional system support structures, system call
skeleton, and related constants. FdPair, FdLock, FdMap.
* Add stack RS instructions (STKRSGET, SRSGET, SRSPUT) and related code
to manage RefStors for stack contexts. Assert on semantic exit if any
related local variable address is still referenced.
Matthew Dillon [Mon, 28 Mar 2016 15:39:33 +0000 (08:39 -0700)]
Rune - Locked storage work 9/N
* Cleanup loose ends of last commit.
* Add BZero optimizations. Change the extension for BZERO to supply the
alignment (as ext2), issue discrete MOVEs if it makes sense to.
Matthew Dillon [Mon, 28 Mar 2016 02:41:38 +0000 (19:41 -0700)]
Rune - Locked storage work 8/N
* 'rune' frontend now understands output file (.m, .s, <executable>) and
automatically pipes to RAS, AS, and CC (for linking).
* Rune external libraries now register link libs to the frontend.
* Track SG->SG dependencies and attempt to order CTORs based on that.
This should fix issues where e.g. program global constructors are improperly
run before global constructors for things like stdin/stdout/stderr.
* Cleanup, reordering header files a bit.
* Add RSOP_THREAD for thread context (not yet used). Lock is painlessly
maintained for the duration of the thread.
Matthew Dillon [Sun, 27 Mar 2016 00:27:28 +0000 (17:27 -0700)]
Rune - Locked storage work 7/N
* Correct refs 1->0 transition test in libruntime.
* Correct a gfx/window bug that was destroying an X window too early.
Matthew Dillon [Sat, 26 Mar 2016 06:02:24 +0000 (23:02 -0700)]
Rune - Locked storage work 6/N
* Resolver asserts proper content-locked storage semantics.
* ptr.method() can handle locked and unlocked storage for internal
run-time calls and will assert if not locked otherwise.
* Cleanup eventThread() a bit.
* Cleanup ext_x11 - don't allow DCOPY on the return storage.
* Still have a ref-counting issues with gfx2.d's frame.
Matthew Dillon [Fri, 25 Mar 2016 16:22:57 +0000 (09:22 -0700)]
Rune - Locked storage work 5/N
* Resolver now complains if you try to take &content-locked-object.
Relax requirements when possible.
* Test 'unlocked' scope. Appears to work properly.
Matthew Dillon [Fri, 25 Mar 2016 08:22:23 +0000 (01:22 -0700)]
Rune - Locked storage work 4/N
* Keep track of whether an LValueStor is mrslocked or not. LValueStor's
(e.g.) as stack variables or as an argument, including the one used for
the method call, are mrslocked (meaning the callee can assume its already
locked).
* Greatly reduces overhead for method call recursions.
Matthew Dillon [Fri, 25 Mar 2016 07:50:27 +0000 (00:50 -0700)]
Rune - Locked storage work 3/N
* Get rid of SF_LOCKED, it created a big mess. Instead we now just use
Declaration->d_Scope.s_Flags & SCOPE_UNLOCKED to 'seed' the data state.
The natural locking state is determined by context and only exp.element
(GenDotExp) needing to track prior state for compound accesses.
* Get rid of related hacks.
* For the moment XXX figuring out whether ptr in ptr.new() is GENSTAT_LOCK
or GENSTAT_REFD (we assume it is for now).
* For the moment XXX figuring out whether *pptr (indirect through double ptr)
or *ptr-to-compound is GENSTAT_LOCK or GENSTAT_REFD (we assume not, for now).
* Track whether the RS lock in an LValueStor can be implied. GenDropContent*()
also needs to know so track it in a bitmap. Only applicable to procedure
arguments (limit 64 lvalue elements... seems ok to me).
Matthew Dillon [Thu, 24 Mar 2016 19:29:35 +0000 (12:29 -0700)]
Rune - synchronize for mobile work
* Misc WIP