CASBENCH Computer Algebra Benchmarks

This section was a proposal by Heinz Kredel (RZ Uni Mannheim kredel@rz.uni-mannheim.de) for the beginning of some computer algebra benchmarking. The contents of this section have been discussed by the Computer Algebra Fachgruppe. Its recommendations are to start with an informal collection of examples and timings. At first there should be no verification, only claims of groups based on their scientific reputation.

Classification of Benchmarks

Topics to be covered by the benchmarks.

Average Quality Benchmarks -- TPS, LIPS, FLOPS ? Mixture of typical application problems.
Graphical User Interface benchmarks ? assesments ? quality, ease of use, recomendations, desktop publishing quality.
Breadth and Scope Benchmarks -- mathematical quality, E.g. probabilistic prime number tests, precomputed group sizes.
Performance Benchmarks
- Low Level Benchmarks, -- Basic (Integer) and Polynomial Arithmetic.
- Medium Level Benchmarks, -- Factorization, Gröbner Bases, Resultants, etc.
- High Level Benchmarks, -- Integration, Systems of Equations, Cylindrical Algebraic Decomposition, other complex Applications.
Grand Challanges Benchmarks scalability, special hardware; e.g. van zur Gathens Factorization Challange.
Manufacturer Specific Benchmarks e.g. REDUCE test suite.

Methodology of Benchmarks

Free Availability but copyright and procedure protected.
Time Measurement in clock wall time.
Full disclosure agreement
- all parameters, hardware and software configuration (optional price)
- meta algorithm, used algorithm, or reference (if appropriate)
- no implementation details ?
- application submission form
Reproducibility, reproduction procedure, competitors.
Benchmark Database, design, features, availability. Scope of systems (What can be solved by which system ?)

References

CAFG93 V. Weispfenning, J. Grabmeier (eds.), Computeralgebra in Deutschland. Fachgruppe Computeralgebra der GI, DMV, GAMM, 1993. Erh\"altlich bei GI, Godesberger Allee 99, Bonn.
guide A guide to CA systems.
zGathen92 J. von zur Gathen, A Polynomial Factorization Challenge, SIGSAM Bulletin, Vol. 26, No. 2, April 1992, pp 22-24.
Souza93 Paulo Ney de Souza, Computer Algebra Systems, AMS Notices, Vol. 40, No. 6, July August 1993, pp 617-623.
Simon92 B. Simon, Comperative CAS Review, AMS Notices, Vol. 39, No. 7, September 1992, pp 700-710.
Parkbench93 PARKBENCH Committee, Public International Benchmarks for Parallel Computers, Uni Tennessee, 1993.
TPC93 Transaction Processing Performance Council (TPC), TPC Benchmark, Several Versions. San Jose, 1993.

Heinz Kredel <kredel@rz.uni-mannheim.de>, KRUM Server,

Last modification by GNU Emacs 19.27.1 (with html-mode) for OS/2 or HTML Wizard for OS/2 at 5. Mai 1995.