Benchmark suite
We need to set up a benchmark suite for DOLFIN to track improvements (or regressions) in efficiency.
The benchmark suite will be run nightly on a dedicated server and the results summarized and plotted on fenics.org.
The benchmarks will also be backported (where possible) to earlier versions of DOLFIN.
Suggested categories and benchmarks:
1. la
la benchmarks should iterate over all backends and include the following benchmarks
1a. matrix-vector product
1b. gmres solve Poisson
1c. gmres + ilu solve Poisson
1d. gmres (or cg) + amg solve
1e-h. same as above in parallel
2. mesh
2a. reading mesh from file
2b. initializing topology (call to init)
2c. computing boundary
2d. iterating over entities
2e. mesh refinement
2f. mesh partitioning
2g. mesh intersection
3. fem
fem benchmarks should also iterate over all la backends
need timings for specific parts of assembly, in particular computing the dof map, tabulate_tensor, add, and apply
3a. serial assembly of Poisson with Lagrange elements degree 1, 2, 3, 4, 5
3b. serial assembly of DG Poisson formulation
3c-d. same as above in parallel
4. function
4a. interpolation / restriction of a Function
4b. interpoilation / restriction of an Expression
4c. evaluation at arbitrary points
5. ode
5a. initializing weights for q = 1, 2, 3, ..., 10
5b. solving Lorenz on [0, 200] fixed time step
5c. some mono-adaptive solve
5d. some multi-adaptive solve
6. misc
5a. jit compiler
5b. newton solver
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- High
- Drafter:
- None
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Approved
- Series goal:
- None
- Implementation:
- Implemented
- Milestone target:
- 1.0-beta2
- Started by
- Garth Wells
- Completed by
- Anders Logg
Related branches
Related bugs
Sprints
Whiteboard
GNW: Should also compare C++ versus Python performance.
AL: Good point
GNW: Poisson only is not a good test since it doesn't show up issues which arise for more complicated forms.
AL: Good point, please add a (limited) number of relevant test cases
AL: The list above is just a first iteration, please help out to expand and organize the test suite
AL: The benchbot is up and running. Please help out with providing relevant benchmarks before Johannes adds the final touches.
AL: I consider this implemented now (although with a somewhat different set of benchmarks). Johannes has backported quite a few of the benchmarks to DOLFIN 0.7.3