Directory of Activities

This page contains references to benchmark activities in computer algebra. The content was compiled from the results of the Benchmark Meeting at ISSAC'98. Please send an e-mail to me to get your activities included in this list.

Breadth and Scope Benchmarks

• A Review of CAS Mathematical Capabilities by Michael Wester.

  From the Abstract:
  In this paper, the capabilities of six major general purpose Computer Algebra Systems (CASs) (Axiom, Derive, Macsyma, Maple, Mathematica and Reduce) are reviewed on 123 short problems covering a broad range of (primarily) symbolic mathematics.
  A demo was developed for each CAS, run and the results evaluated. Problems were graded in terms of whether it was easy or difficult or possible to produce an answer and if an answer was produced, whether it was correct.

  Wester's test suite in MuPAD 1.2.2, Wester's test suite in MuPAD 1.3 by Paul Zimmermann

• Comparison of Mathematica on Various Machines by Karl Unterkofler.

  Timings for small Mathematica examples on various workstations.
  Latest timings can be obtained from Graz, Frankfurt or the Mathematica Benchmark Site.

• Comparison of Maple on Various Machines by Karl Unterkofler.

  Timings for small Maple examples on some workstations.

• Computer Challenge Problems by gjfee@jeeves.uwaterloo.ca

  14 problems to challenge your computer algebra system.

Average Quality Benchmarks

• Medicis Benchs by Joel Marchand and others.

  A collection of timings on various application problems, computer algebra systems and hardware platforms from the Medicis group. The timings are compiled from a user survey in a physics lab and are from codes in Maple, C, C++ and Fortran. Contact Joel Marchand for details.

Performance Benchmarks, Polynomial Systems

• PolyData Project by Olaf Bachmann, Hans Schoenemann, Hans-Gert Graebe, Jean-Charles Faugere and Michael Dengel.

  From the Web-Page:
  The PolyData project has the following two main goals:

  1. To provide a framework and general tools that are capable for
     • a systematic and uniform collection of problems (together with their solutions and other related background information) from different areas of Symbolic Computation
     • convenient extensions, manipulations, and categorizations of the collected data
     • the specification of (inter-)relations of the collected data
     • trusted benchmarking on various Computer Algebra Systems of (a significant part of) the collected problems
     • electronic transformation of the collected data into other representation formats, like HTML, or SQL, in order to conveniently view and/or (re)process the data
     • easy and comfortable reuse and extensions of the provided tools, so that people from all areas of Computer Algebra can conveniently contribute and use the results of our work
  2. To use these tools to systematically collect and maintain polynomial systems that were considered in papers or elsewhere, and to publish results of benchmark tests for computing various properties of these polynomial systems (like Groebner basis, szyzygies, free resolutions, decompositions, "solutions", etc).

• PoSSo Examples by Marco Silvestri.

  A collection of systems of polynomial equations used by the PoSSo Group for testing. Dated 28. Sept. 1994.

• Handbook of Polynomial Systems by D. Bini and B. Mourrain

  The aim of this data base is to provide a consequent list of polynomial systems which could be used to illustrate, compare, evaluate different methods for solving polynomial systems. We will gather polynomial systems (including the list of the PoSSo project) and add to each of these systems short descriptions of methods, results of computations, timing, ...

• Gb Benchs by Jean-Charles Faugere.

  Benchmarks used by the Gb, FGb and RS systems.

• ccNbody by Ilias Kotsireas.

  Polynomial systems arising in the study of central configurations in the N-body problem of Celestial Mechanics

• Some Examples for Solving Systems of Algebraic Equations by Calculating Groebner Bases by W. Boege, R. Gebauer and H. Kredel.

  One of the first papers with timings from 1984.

(Grand) Challenges Benchmarks

• Great Internet Mersenne Prime Search (GIMP)

  From the Background:
  The Great Internet Mersenne Prime Search (GIMPS) harnesses the power of thousands of small computers like yours to solve the seemingly intractable problem of finding HUGE prime numbers. Specifically, GIMPS looks for Mersenne Primes, expressed by the formula 2^P-1. Over 8,000 people have contributed computer time to help discover world-record Mersenne primes. Thoughout history, the largest known prime number has usually been a Mersenne prime.

• A Polynomial Factorisation Challenge by Joachim von zur Gathen.

  SIGSAM Bulletin, Vol. 26, No. 2, April 1992, Issue 100.
  MuPad results by Paul Zimmermann.

• Record Number Field Sieve Factorisations by Peter-Lawrence Montgomery.

  A team of researchers from Amsterdam and Oregon have factored the 162-digit Cunningham number (12^151 - 1)/11 using the Special Number Field Sieve (SNFS). This team also factored a 105-digit cofactor of 3^367 - 1 using the General Number Field Sieve (GNFS), These beat the prior records of 158 digits (SNFS) and 75 digits (GNFS). The Amsterdam group also factored the 123-digit cofactor of the Most Wanted number 2^511 - 1 using SNFS.

• Factor-by-mail project by Bob Silverman.

Others

• Comparison of mathematical programs for data analysis by Stefan Steinhaus. (New edition May 1999)

  From the Abstract:
  This comparison should give an overview about the functionality, the availability for different operating systems and the speed of mathematical programs for analysing huge or very huge data sets in mathematical, statistical or graphical ways. The focus of this test is therefor set to mathematical functions which are mainly used in economics, financial analysis, biology, chemistry, physics and some other subjects where the numerical analyse of data is very important.

• SATLIB - The Satisfiability Library by Holger Hoos and Thomas Stuetzle.

  From the Abstract:
  SATLIB is a collection of benchmark problems, solvers, and tools we are using for our own SAT related research. One strong motivation for creating SATLIB is to provide a uniform test-bed for SAT solvers as well as a site for collecting SAT problem instances, algorithms, and empirical characterisations of the algorithms' performance.

• Jacques Morgenstern Challenge