Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge remote #1

Merged
merged 394 commits into from
Jan 9, 2020
Merged

merge remote #1

merged 394 commits into from
Jan 9, 2020

Conversation

gongchengra
Copy link
Owner

merge remote

dybvig and others added 30 commits October 11, 2017 18:57
    bintar BUILDING NOTICE makefiles/Mf-install.in scheme.1.in
    c/Makefile.i3nt c/Makefile.a6nt c/Makefile.ti3nt c/Makefile.ta6nt
    workarea c/scheme.rc s/7.ss s/cmacros.ss
    release_notes/release_notes.stex csug/csug.stex
  an existing setting will not interfere with the build process, and
  added a note to BUILDING that CHEZSCHEMELIBDIRS should be unset in
  Version 9.5 and before.
    s/Mf-base, mats/Mf-base, BUILDING
and curly braces, since the current cmtt font seems to support these
characters better than it once did.
  csug/scheme.sty
  release_notes/scheme.sty
  redundant keys and expands into exclusive-cond rather than cond.
  it catches references to => before expanding into exclusive-cond
  to avoid supporting => as an undocumented and useless extension
  of the case syntax.  the r6rs:case and case macros now require
  multiple clauses rather than leaving the enforcement to exclusive-cond,
  and the exclusive-cond macro now requires multiple clauses rather
  than leaving the enforcement to cond.
    syntax.ss,
    4.ms, root-experr*, patch*
  locale.h includes xlocale.h or, in glibc 2.26, its replacement.
    expeditor.c
  month and year at the time of generation.
    csug.stex, copyright.stex
- Updated configuration to set machine types in the CSUG and release notes
  make files, and updated distclean target to remove these files.
    configure, makefiles/Makefile-csug.in (renamed from csug/Makefile),
    makefiles/Makefile-release_notes.in
      (renamed from release_notes/Makefile),
    makefiles/Makefile
procedures with large numbers of variables:
- added pass-time tracking for pre-cpnanopass passes to compile.
    compile.ss
- added inline handler for fxdiv-and-mod
    cp0.ss, primdata.ss
- changed order in which return-point operations are done (adjust
  sfp first, then store return values, then restore local saves) to
  avoid storing return values to homes beyond the end of the stack
  in cases where adjusting sfp might result in a call to dooverflood.
    cpnanopass.ss, np-languages.ss
- removed unused {make-,}asm-return-registers bindings
    cpnanopass.ss
- corrected the max-fv value field of the lambda produced by the
  hand-coded bytevector=? handler.
    cpnanopass.ss
- reduced live-pointer and inspector free-variable mask computation
  overhead
    cpnanopass.ss
- moved regvec cset copies to driver so they aren't copied each
  time a uvar is assigned to a register.  removed checks for
  missing register csets, since registers always have csets.
    cpnanopass.ss
- added closure-rep else clause in record-inspector-information!.
    cpnanopass.ss
- augmented tree representation with a constant representation
  for full trees to reduce the overhead of manipulating trees or
  subtress with all bits set.
    cpnanopass.ss
- tree-for-each now takes start and end offsets; this cuts the
  cost of traversing and applying the action when the range of
  applicable offsets is other than 0..tree-size.
    cpnanopass.ss
- introduced the notion of poison variables to reduce the cost of
  register/frame allocation for procedures with large sets of local
  variables.  When the number of local variables exceeds a given
  limit (currently hardwired to 1000), each variable with a large
  live range is considered poison.  A reasonable set of variables
  with large live ranges (the set of poison variables) is computed
  by successive approximation to avoid excessive overhead.  Poison
  variables directly conflict with all spillables, and all non-poison
  spillables indirectly conflict with all poison spillables through
  a shared poison-cset.  Thus poison variables cannot live in the
  same location as any other variable, i.e., they poison the location.
  Conflicts between frame locations and poison variables are handled
  normally, which allows poison variables to be assigned to
  move-related frame homes.  Poison variables are spilled prior to
  register allocation, so conflicts between registers and poison
  variables are not represented.  move relations between poison
  variables and frame variables are recorded as usual, but other
  move relations involving poison variables are not recorded.
    cpnanopass.ss, np-languages.ss
- changed the way a uvar's degree is decremented by remove-victim!.
  instead of checking for a conflict between each pair of victim
  and keeper and decrementing when the conflict is found, remove-victim!
  now decrements the degree of each var in each victim's conflict
  set.  while this might decrement other victims' degrees unnecessarily,
  it can be much less expensive when large numbers of variables are
  involved, since the number of conflicts between two non-poison
  variables should be small due to the selection process for
  (non-)poison variables and the fact that the unspillables introduced
  by instruction selection should also have few conflicts.  That
  is, it reduces the worst-case complexity of decrementing degrees
  from O(n^2) to O(n).
    cpnanopass.ss
- took advice in compute-degree! comment to increment the uvars in
  each registers csets rather than looping over the registers for
  each uvar asking whether the register conflicts with the uvar.
    cpnanopass.ss
- assign-new-frame! now zeros out save-weight for local saves, since
  once they are explicitly saved and restored, they are no longer
  call-live and thus have no save cost.
    cpnanopass.ss
- desensitized the let-values source-caching timing test slightly
    8.ms
- updated allx, bullyx patches
    patch*
test and ephemeron gc test while resensitizing the former
  8.ms, 4.ms
various formatting and comment corrections
  workarea,
  s/Mf-base, bytevector.ss, cpnanopass.ss, date.ss,
  5_6.ms, examples.ms
entries x and y in the list produced by the sort call, if x's
bfp = y's bfp, x should come before y if x's efp < y's efp.
The idea is that enclosing entries should always come later
in the list.  this affects only languages where two expressions
can start at the same character position.
  pdhtml.ss
expanded capability of ez-grammar with support for simpl
parsing of binary operators w/precedence and associativity
and automatically generated markdown grammar descriptions.
ez-grammar-test.ss now also doubles as a test of pdhtml for
algebraic languages.
  mats/examples.ms,
  examples/ez-grammar.ss, examples/ez-grammar-test.ss,
  examples/Makefile
compilation now treat a malformed object file as if it were
not present and needs to be regenerated.  A malformed object
file (particularly a truncated one) might occur if the compiling
processes is killed or aborts before it has a chance to delete
a partial object file.
  syntax.ss,
  7.ms
for procedures with large numbers of variables
fix bytevector-[u/s]16-native-set! signature
mflatt and others added 29 commits July 2, 2019 13:17
Previously it was using #!/bin/bash as its interpreter. It is not
compatible with FreeBSD because bash is not installed by default.

The update-revision script is #!/bin/sh compatible and there are
other examples of #!/bin/sh scripts in the codebase so the change
is consistent with them.
and other platforms using non-GNU make systems.
Signed-off-by: Alexander Shopov <ash@kambanaria.org>
# Conflicts:
#	LOG
p is a pointer that iterates over path, which is buffer.
We should not try to get to an address preceding its start.
Since there was an execution path that leads to that,
guard against it with an additional check.

Signed-off-by: Alexander Shopov <ash@kambanaria.org>
Signed-off-by: Alexander Shopov <ash@kambanaria.org>
Ensure that floating-point registers used for arguments and results
are preserved while activating or deactivating a thread.
and functionality improvements (including support for measuring
coverage), primitive argument-checking fixes, and object-file changes
resulting in reduced load times (and some backward incompatibility):
- annotations are now preserved in object files for debug
  only, for profiling only, for both, or not at all, depending
  on the settings of generate-inspector-information and
  compile-profile.  in particular, when inspector information
  is not enabled but profiling is, source information does
  not leak into error messages and inspector output, though it is
  still available via the profile tools.  The mechanics of this
  involved repurposing the fasl a? parameter to hold an annotation
  flags value when it is not #f and remaking annotations with
  new flags if necessary before emitting them.
    compile.ss, fasl.ss, misc.ms
- altered a number of mats to produce correct results even
  when the 's' directory is profiled.
    misc.ms, cp0.ms, record.ms
- profile-release-counters is now generation-friendly; that is,
  it doesn't look for dropped code objects in generations that have
  not been collected since the last call to profile-release-counters.
  also, it no longer allocates memory when it releases counters.
    pdhtml.ss,
    gc.c, gcwrapper.c, globals.h, prim5.c
- removed unused entry points S_ifile, S_ofile, and S_iofile
    alloc.c, externs.h
- mats that test loading profile info into the compiler's database
  to guide optimization now weed out preexisting entries, in case
  the 's' directory is profiled.
    4.ms, mat.ss, misc.ms, primvars.ms
- counters for dropped code objects are now released at the start
  of each mat group.
    mat.ss
- replaced ehc (enable-heap-check) option with hci (heap-check-interval)
  option that allows heap checks to be performed periodically rather
  than on each collection.  hci=0 is equivalent to ehc=f (disabling
  heap checks) and hci=1 is equivalent to ehc=t (enabling heap
  checks every collection), while hci=100 enables heap checks only
  every 100th collection.  allx and bullyx mats use this feature
  to reduce heap-checking overhead to a more reasonable level.  this
  is particularly important when the 's' directory is profiled,
  since the amount of static memory to be checked is greatly increased
  due to the counters.
    mats/Mf-base, mat.ss, primvars.ms
- added a mat that calls #%show-allocation, which was otherwise not
  being tested.
    misc.ms
- removed a broken primvars mat and updated two others.  in each case,
  the mat was looking for information about primitives in the wrong
  (i.e., old) place and silently succeeding when it didn't find any
  primitives to tests.  the revised mats (along with a few others) now
  check to make sure at least one identifier has the information they
  look for.  the removed mat was checking for library information that
  is now compiled in, so the mat is now unnecessary.  the others were
  (not) doing argument-error checks.  fixing these turned up a handful of
  problems that have also been fixed: a couple of unbound variables in the
  mat driver, two broken primdata declarations, a tardy argument check
  by profile-load-data, and a bug in char-ready?, which was requiring
  an argument rather than defaulting it to the current input port.
    primdata.ss, pdhtml.ss, io.ms,
    primdvars.ms, 4.ms, 6.ms, misc.ms, patch*
- added initial support for recording coverage information.  when the
  new parameter generate-covin-files is set, the compiler generates
  .covin files containing the universe of all source objects for which
  profile forms are present in the expander output.  when profiling
  and generation of covin files are enabled in the 's' directory, the
  mats optionally generate .covout files for each mat file giving
  the subset of the universe covered by the mat file, along with an
  all.covout in each mat output directory aggregating the coverage
  for the directory and another all.covout in the top-level mat
  directory aggregating the coverage for all directories.
    back.ss, compile.ss, cprep.ss, primdata.ss, s/Mf-base,
    mat.ss, mats/Mf-base, mats/primvars.ms
- support for generating covout files is now built in.  with-coverage-output
  gathers and dumps coverage information, and aggregate-coverage-output
  combines (aggregates) covout files.
    pdhtml.ss, primdata.ss, compile.ss,
    mat.ss, mats/Mf-base, primvars.ms
- profile-clear now adjusts active coverage trackers to avoid losing
  coverage information.
    pdhtml.ss,
    prim5.c
- nested with-coverage calls are now supported.
    pdhtml.ss
- switched to a more compact representation for covin and covout files;
  reduces disk space (compressed or not) by about a factor of four
  and read time by about a factor of two with no increase in write time.
    primdata.ss, pdhtml.ss, cprep.ss, compile.ss,
    mat.ss, mats/Mf-base
- added support for determining coverage for an entire run, including
  coverage for expressions hit during boot time.  'all' mats now produce
  run.covout files in each output directory, and 'allx' mats produce
  an aggregate run.covout file in the mat directory.
    pdhtml.ss,
    mat.ss, mats/Mf-base
- profile-release-counters now adjusts active coverage trackers to
  account for the counters that have been released.
    pdhtml.ss,
    prim5.c
- replaced the artificial "examples" target with a real "build-examples"
  target so make won't think it always has to mats that depend upon
  the examples directory having been compiled.  mats make clean now
  runs make clean in the examples directory.
    mats/Mf-base
  importing a library from an object file now just visits the object
  file rather than doing a full load so that the run-time code for
  the library is not retained.  The run-time code is still read
  because the current fasl format forces the entire file to be read,
  but not retaining the code can lower heap size and garbage-collection
  cost, particularly when many object-code libraries are imported.
  The downside is that the file must be revisited if the run-time
  code turns out to be required.   This change exposed several
  places where the code was failing to check if a revisit is needed.
    syntax.ss,
    7.ms, 8.ms, misc.ms, root-experr*
- fixed typos: was passing unquoted load rather than quoted load
  to $load-library along one path (where it is loading source code
  and therefore irrelevant), and was reporting src-path rather than
  obj-path in a message about failing to define a library.
    syntax.ss
- compile-file and friends now put all recompile information in
  the first fasl object after the header so the library manager can
  find it without loading the entire fasl file.  The library manager
  now does so.  It also now checks to see if library object files
  need to be recreated before loading them rather than loading them and
  possibly recompiling them after discovering they are out of date, since
  the latter requires loading the full object file even if it's out of
  date, while the former takes advantage of the ability to extract just
  recompile information.  as well as reducing overhead, this eliminates
  possibly undesirable side effects, such as creation and registration
  of out-of-date nongenerative record-type descriptors.  because the
  library manager expects to find recompile information at the front of
  an object file, it will not find all recompile information if object
  files are "catted" together.  also, compile-file has to hold in memory
  the object code for all expressions in the file so that it can emit the
  unified recompile information, rather than writing to the object file
  incrementally, which can significantly increase the memory required
  to compile a large file full of individual top-level forms.  This does
  not affect top-level programs, which were already handled as a whole,
  or a typical library file that contains just a single library form.
    compile.ss, syntax.ss
- the library manager now checks include files before library dependencies
  when compile-imported-libraries is false (as it already did when
  compile-imported-libraries is true) in case a source change affects
  the set of imported libraries.  (A library change can affect the set
  of include files as well, but checking dependencies before include
  files can cause unneeded libraries to be loaded.)  The include-file
  check is based on recompile-info rather than dependencies, but the
  library checks are still based on dependencies.
    syntax.ss
- fixed check for binding of scheme-version. (the check prevents
  premature treatment of recompile-info records as Lexpand forms
  to be passed to $interpret-backend.)
    scheme.c
- strip-fasl-file now preserves recompile-info when compile-time info
  is stripped.
    strip.ss
- removed include-req* from library/ct-info and ctdesc records; it
  is no longer needed now that all recompile information is maintained
  separately.
    expand-lang.ss, syntax.ss, compile.ss, cprep.ss, syntax.ss
- changed the fasl format and reworked a lot of code in the expander,
  compiler, fasl writer, and fasl reader to allow the fasl reader
  to skip past run-time information when it isn't needed and
  compile-time information when it isn't needed.  Skipping past
  still involves reading and decoding when encrypted, but the fasl
  reader no longer parses or allocates code and data in the portions
  to be skipped.  Side effects of associating record uids with rtds
  are also avoided, as are the side effects of interning symbols
  present only in the skipped data.  Skipping past code objects
  also reduces or eliminates the need to synchronize data and
  instruction caches.  Since the fasl reader no longer returns
  compile-time (visit) or run-time (revisit) code and data when not
  needed, the fasl reader no longer wraps these objects in a pair
  with a 0 or 1 visit or revisit marker.  To support this change,
  the fasl writer generates separate top-level fasl entries (and
  graphs) for separate forms in the same top-level source form
  (e.g., begin or library).  This reliably breaks eq-ness of shared
  structure across these forms, which was previously broken only
  when visit or revisit code was loaded at different times (this
  is an incompatible change).  Because of the change, fasl "groups"
  are no longer needed, so they are no longer handled.
    7.ss, cmacros.ss, compile.ss, expand-lang.ss, strip.ss,
    externs.h, fasl.c, scheme.c,
    hash.ms
- the change above is surfaced in an optional fasl-read "situation"
  argument (visit, revisit, or load).  The default is load.  visit
  causes it to skip past revisit code and data; revisit causes it
  to skip past visit code and data; and load causes it not to skip
  past either.  visit-revisit data produced by (eval-when (visit
  revisit) ---) is never skipped.
    7.ss, primdata.ss,
    io.stex
- to improve compile-time and run-time error checking, the
  Lexpand recompile-info, library/rt-info, library-ct-info, and
  program-info forms have been replaced with list-structured forms,
  e.g., (recompile-info ,rcinfo).
    expand-lang.ss, compile.ss, cprep.ss, interpret.ss, syntax.ss
- added visit-compiled-from-port and revisit-compiled-from-port
  to complement the existing load-compiled-from-port.
    7.ss, primdata.ss,
    7.ms,
    system.stex
- increased amount read when seeking an lz4-encrypted input
  file from 32 to 1024 bytes at a time
    compress-io.c
- replaced the fasl a? parameter value #t with an "all" flag value
  so it's value is consistently a mask.
    cmacros.ss, fasl.ss, compile.ss
- split off profile mats into a separate file
    misc.ms, profile.ms (new), root-experr*, mats/Mf-base
- added coverage percent computations to mat allx/bullyx output
    mat.ss, mats/Mf-base, primvars.ms
- replaced coverage tables with more generic and generally useful
  source tables, which map source objects to arbitrary values.
    pdhtml.ss, compile.ss, cprep.ss, primdata.ss,
    mat.ss, mats/Mf-base, primvars.ms, profile.ms,
    syntax.stex
- reduced profile counting overhead by using calls to fold-left
  instead of calls to apply and map and by using fixnum operations
  for profile counts on 64-bit machines.
    pdhtml.ss
- used a critical section to fix a race condition in the calculations
  of profile counts that sometimes resulted in bogus (including
  negative) counts, especially when the 's' directory is profiled.
    pdhtml.ss
- added discard flag to declaration for hashtable-size
    primdata.ss
- redesigned the printed representation of source tables and rewrote
  get-source-table! to read and store incrementally to reduce memory
  overhead.
    compile.ss
- added generate-covin-files to the set of parameters preserved
  by compile-file, etc.
    compile.ss,
    system.stex
- moved covop argument before the undocumented machine and hostop
  arguments to compile-port and compile-to-port.  removed the
  undocumented ofn argument from compile-to-port; using
  (port-name ip) instead.
    compile.ss, primdata.ss,
    7.ms,
    system.stex
- compile-port now tries to come up with a file position to supply
  to make-read, which it can do if the port's positions are character
  positions (presently string ports) or if the port is positioned
  at zero.
    compile.ss
- audited the argument-type-error fuzz mat exceptions and fixed a
  host of problems this turned up (entries follow).  added #f as
  an invalid argument for every type for which #f is indeed invalid
  to catch places where the maybe- prefix was missing on the argument
  type.  the mat tries hard to determine if the condition raised
  (if any) as the result of an invalid argument is appropriate and
  redirects the remainder to the mat-output (.mo) file prefixed
  with 'Expected error', causing them to show up in the expected
  error output so developers will be encouraged to audit them in
  the future.
    primvars.ms, mat.ss
- added an initial symbol? test on machine type names so we produce
  an invalid machine type error message rather than something
  confusing like "machine type #f is not supported".
    compile.ss
- fixed declarations for many primitives that were specified as
  accepting arguments of more general types than they actually
  accept, such as number -> real for various numeric operations,
  symbol -> endianness for various bytevector operations,
  time -> time-utc for time-utc->date, and list -> list-of-string-pairs
  for default-library-search-handler.   also replaced some of the
  sub-xxxx types with specific types such as sub-symbol -> endianness
  in utf16->string, but only where they were causing issues with
  the primvars argument-type-error fuzz mat.  (this should be done
  more generally.)
    primdata.ss
- fixed incorrect who arguments (was map instead of fold-right,
  current-date instead of time-utc->date); switched to using
  define-who/set-who! generally.
    4.ss, date.ss
- append! now checks all arguments before any mutation
    5_2.ss
- with-source-path now properly supplies itself as who for the
  string? argument check; callers like load now do their own checks.
    7.ss
- added missing integer? check to $fold-bytevector-native-ref whose
  lack could have resulted in a compile-time error.
    cp0.ss
- fixed typo in output-port-buffer-mode error message
    io.ss
- fixed who argument (was fx< rather than fx<?)
    library.ss
- fixed declaration of first source-file-descriptor argument (was
  sfd, now string)
    primdata.ss
- added missing article 'a' in a few error messages
    prims.ss
- fixed the copy-environment argument-type error message for the list
  of symbols argument.
    syntax.ss
- the environment procedure now catches exceptions that occur and
  reraises the exception with itself as who if the condition isn't
  already a who condition.
    syntax.ss
- updated experr and allx patch files for changes to argument-count
  fuzz mat and fixes for problems turned up by them.
    root-experr*, patch*
- fixed a couple of issues setting port sizes: string and bytevector
  output port put handlers don't need room to store the character
  or byte, so they now set the size to the buffer length rather
  than one less.  binary-file-port-clear-output now sets the index
  rather than size to zero; setting the size to zero is inappropriate
  for some types of ports and could result in loss of buffering and
  even suppression of future output.  removed a couple of redundant
  sets of the size that occur immediately after setting the buffer.
    io.ss
- it is now possible to return from a call to with-profile-tracker
  multiple times and not double-count (or worse) any counts.
    pdhtml.ss, profile.ms
- read-token now requires a file position when it is handed a
  source-file descriptor (since the source-file descriptor isn't
  otherwise useful), and the source-file descriptor argument can
  no longer be #f.  the input file position plays the same role as
  the input file position in get-datum/annotations.  these extra
  read-token arguments are now documented.
    read.ss,
    6.ms,
    io.stex
- the source-file descriptor argument to get-datum/annotations can
  no longer be #f.  it was already documented that way.
    read.ss
- read-token and do-read now look for the character-positions port
  flag before asking if the port has port-position, since the latter
  is slightly more expensive.
    read.ss
- rd-error now reports the current port position if it can be determined
  when fp isn't already set, i.e., when reading from a port without
  character positions (presently any non string port) and fp has not
  been passed in explicitly (to read-token or get-datum/annotations).
  the port position might not be a character position, but it should be
  better than nothing.
    read.ss
- added comment noting an invariant for s_profile_release_counters.
    prim5.c
- restored accidentally dropped fasl-write formdef and dropped
  duplicate fasl-read formdef
    io.stex
- added a 'coverage' target that tests the coverage of the Scheme-code
  portions of Chez Scheme by the mats.
    Makefile.in, Makefile-workarea.in
- added .PHONY declarations for all of the targets in the top-level
  and workarea make files, and renamed the create-bintar, create-rpm,
  and create-pkg targets bintar, rpm, and pkg.
    Makefile.in, Makefile-workarea.in
- added missing --retain-static-relocation command-line argument and
  updated the date
    scheme.1.in
- removed a few redundant conditional variable settings
    configure
- fixed declaration of condition wait (timeout -> maybe-timeout)
    primdata.ss
argument-type-check error messages, for Windows multibyte->string
and string->multibyte.
    primvars.ms
gcc 9.1.0 fails to compile with -Werror and -O3 because it detects that `n` might be used uninitialize in line
`if (n != scheme_version) {`

We do know that `n` will be initialized in `if (zget_uptr(file, &n) != 0) {` but gcc doesn't know that unless compiled with LTO.
library-rt-info records in the code for compile-whole-xxx.
  compile.ss,
  7.ms
is enabled, so local transformer code can be profiled.
  syntax.ss,
  profile.ms
@gongchengra gongchengra merged commit 0bfe69c into gongchengra:master Jan 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.