On 06/05/17 15:31, Per Bothner wrote:
> On 05/06/2017 05:14 AM, Peter Lane wrote:
>> I (trivially) rewrote the chibi test suite to use SRFI 64 and checked
>> srfi 151 on the following 5 implementations, in R7RS mode. They all
>> passed the test suite.  For interest, I include the running times.
>>
>> Running times:  (all 558 passes)
>>
>> chibi (from master)         6m9.853s
>> gauche (0.9.6 from master)  4m55.710s
>> kawa 2.4                    4m24.362s [with test file split in 2]
>> larceny 0.99                37m6.170s
>> sagittarius 0.8.3           2m49.283s
>>

>
> Kawa is subject to the annoying JVM .bytecode size limitation, which is
> difficult
> to work around.  However, it can work to invoke Kawa in line-by-line mode:
>
>   kawa -f srfi-151-test.scm
>
> That only works for SRFI-64-styl;e tests, which do *not* require the whole
> file to be wrapped in a single form.

Thanks Per.  I didn't know that option.

>
> Are the runtimes measured using the reference implementation?

Yes, using the files on github, except for using srfi 64 for the tests.

> Of course most implementations would not use those.

Agreed.

If I replace bitwise-core for Larceny to use its (rnrs arithmetic
bitwise) library (which is implemented in terms of primitives), the
running time for the tests goes down to 14 seconds.

 >> Trying to use the chibi part of the cond-expand caused an error,
 >> ERROR: couldn't find include: "bit.so"

I found bit.so.  With this, running time for the test suite is almost
instantaneous, but there are errors:

bitwise:
.....................................................................
..............................................................................
.................................xxx.......xx................xxx..............
....................................x...x......................x..............
..............................................................................
..............................................................................
......xxx.............................................................x...x...
....................x
541 out of 558 (97.0%) tests passed in 0.101446866989136 seconds.
17 failures (3.0%).
FAIL: y
     expected -1 but got 18446744073709551615
FAIL: y
     expected -1 but got 18446744073709551615
FAIL: y
     expected -1 but got 340282366920938463481821351505477763071
FAIL: y
     expected -340282366920938463463374607431768211456 but got
6277101735386680763495507056286727952638980837032266301440
FAIL: y
     expected -340282366920938463463374607431768211456 but got
115792089237316195429848086744074588616765491721927291992096564844945395941376
FAIL: y
     expected 340282366920938463463374607431768211456 but got 0
FAIL: y
     expected 340282366920938463463374607431768211464 but got 8
FAIL: y
     expected 340282366920938463465680450440981905408 but got
2305843009213693952
FAIL: y
     expected -79 but got -78
FAIL: y
     expected -79 but got -78
FAIL: y
     expected 99 but got 28
FAIL: y
     expected -1 but got 18446744073709551615
FAIL: y
     expected -1 but got 18446744073709551615
FAIL: y
     expected -1 but got 340282366920938463481821351505477763071
FAIL: y
     expected -79 but got -78
FAIL: y
     expected -79 but got -78
FAIL: y
     expected 99 but got 28

--
Peter Lane
http://peterlane.info