v6.0
Major Changes Edited all benchmarks so that typed and untyped code are very similar.
If you compare any two typed/A.rkt
and untyped/A.rkt
files, the only differences should be the requires and the type annotations.
Example: gregor
In at least one place, the untyped gregor code had an extra assert. It's gone now.
diff --git a/benchmarks/gregor/untyped/date.rkt b/benchmarks/gregor/untyped/date.rkt
index a3102a9..6ceccb7 100644
--- a/benchmarks/gregor/untyped/date.rkt
+++ b/benchmarks/gregor/untyped/date.rkt
@@ -63,7 +64,6 @@
(define date->ymd Date-ymd)
;(: date->jdn (-> Any Integer))
(define (date->jdn d)
- (unless (Date? d) (error "date->jdn type error"))
(Date-jdn d))
Example: lnm
Typed lnm now uses asserts instead of casts to validate input data. Untyped lnm uses the same casts.
diff --git a/benchmarks/lnm/typed/spreadsheet.rkt b/benchmarks/lnm/typed/spreadsheet.rkt
index dd2dcbc..b869929 100644
--- a/benchmarks/lnm/typed/spreadsheet.rkt
+++ b/benchmarks/lnm/typed/spreadsheet.rkt
@@ -62,7 +62,7 @@
(void)
;; For each row, print the config ID and all the values
(for ([(row n) (in-indexed vec)])
- (void (natural->bitstring (cast n Index) #:pad (log2 num-configs)))
+ (void (natural->bitstring (assert n index?) #:pad (log2 num-configs)))
(for ([v row]) (void "~a~a" sep v))
(void)))
@@ -71,8 +71,18 @@
(define (rktd->spreadsheet input-filename
#:output [output #f]
#:format [format 'tab])
- (define vec (cast (file->value input-filename) (Vectorof (Listof Index))))
+ (define vec
+ (for/vector : (Vectorof (Listof Index)) ((x (in-vector (assert (file->value input-filename) vector?))))
+ (listof-index x)))
(define suffix (symbol->extension format))
(define out (or output (path-replace-suffix input-filename suffix)))
(define sep (symbol->separator format))
(vector->spreadsheet vec out sep))
+
+(: listof-index (-> Any (Listof Index)))
+(define (listof-index x)
+ (if (and (list? x)
+ (andmap index? x))
+ x
+ (error 'listof-index)))
diff --git a/benchmarks/lnm/untyped/spreadsheet.rkt b/benchmarks/lnm/untyped/spreadsheet.rkt
index 18be330..6466fb0 100644
--- a/benchmarks/lnm/untyped/spreadsheet.rkt
+++ b/benchmarks/lnm/untyped/spreadsheet.rkt
@@ -14,6 +14,7 @@
;; ----------------------------------------------------------------------------
(require
+ "../base/untyped.rkt"
(only-in racket/file file->value)
(only-in "bitstring.rkt" log2 natural->bitstring)
)
@@ -55,7 +56,7 @@
(void)
;; For each row, print the config ID and all the values
(for ([(row n) (in-indexed vec)])
- (void (natural->bitstring n #:pad (log2 num-configs)))
+ (void (natural->bitstring (assert n index?) #:pad (log2 num-configs)))
(for ([v row]) (void "~a~a" sep v))
(void)))
@@ -64,8 +65,16 @@
(define (rktd->spreadsheet input-filename
#:output [output #f]
#:format [format 'tab])
- (define vec (file->value input-filename))
+ (define vec
+ (for/vector ((x (in-vector (assert (file->value input-filename) vector?))))
+ (listof-index x)))
(define suffix (symbol->extension format))
(define out (or output (path-replace-suffix input-filename suffix)))
(define sep (symbol->separator format))
(vector->spreadsheet vec out sep))
+
+(define (listof-index x)
+ (if (and (list? x)
+ (andmap index? x))
+ x
+ (error 'listof-index)))
results (on Racket 7.7 BC release)
For most benchmarks, performance is the same before & after. But:
lnm
has lower overheadquadT
has higher overheadquadU
has higher overhead
lnm
typed code is much faster now (down from ~4.5s to 0.7s) because it uses assert
instead of cast
. The vector casts in spreadsheet.rkt
and summary.rkt
cost a little --- putting them back adds 1.5s and 0.5s, respectively. But the big savings comes from replacing (cast .... Index)
with (assert .... index?)
in bitstring.rkt
--- reverting adds almost 2.5s.
Both the untyped and fully-typed quad configurations run faster now, which likely makes the mixed configs. look worse. One reason for the change is that quad?
is a simple function instead of a define-predicate
... but things are harder to tease apart. (There are few changes to the main files, so things must be happening related to the base/
context, and that's hard to swap out & test.)
Full data & plots here:
gtp-benchmarks-v5-vs-v6.tar.gz
Raw gtp-measure output:
manifest-v6.tar.gz