Methodology of Benchmarks
- Free Availability but copyright and procedure protected.
- Time Measurement in clock wall time.
- Full disclosure agreement
- all parameters, hardware and software configuration (optional price)
- meta algorithm, used algorithm, or reference (if appropriate)
- no implementation details ?
- application submission form
- Reproducibility, reproduction procedure,
- Benchmark Database, design, features, availability.
Scope of systems (What can be solved by which system ?)