Benchmark suite

Registered by Anders Logg

We need to set up a benchmark suite for DOLFIN to track improvements (or regressions) in efficiency.

The benchmark suite will be run nightly on a dedicated server and the results summarized and plotted on fenics.org.
The benchmarks will also be backported (where possible) to earlier versions of DOLFIN.

Suggested categories and benchmarks:

1. la

  la benchmarks should iterate over all backends and include the following benchmarks

  1a. matrix-vector product
  1b. gmres solve Poisson
  1c. gmres + ilu solve Poisson
  1d. gmres (or cg) + amg solve
  1e-h. same as above in parallel

2. mesh

  2a. reading mesh from file
  2b. initializing topology (call to init)
  2c. computing boundary
  2d. iterating over entities
  2e. mesh refinement
  2f. mesh partitioning
  2g. mesh intersection

3. fem

  fem benchmarks should also iterate over all la backends

  need timings for specific parts of assembly, in particular computing the dof map, tabulate_tensor, add, and apply

  3a. serial assembly of Poisson with Lagrange elements degree 1, 2, 3, 4, 5
  3b. serial assembly of DG Poisson formulation
  3c-d. same as above in parallel

4. function

  4a. interpolation / restriction of a Function
  4b. interpoilation / restriction of an Expression
  4c. evaluation at arbitrary points

5. ode

  5a. initializing weights for q = 1, 2, 3, ..., 10
  5b. solving Lorenz on [0, 200] fixed time step
  5c. some mono-adaptive solve
  5d. some multi-adaptive solve

6. misc

  5a. jit compiler
  5b. newton solver

Blueprint information

Status:
Complete
Approver:
None
Priority:
High
Drafter:
None
Direction:
Needs approval
Assignee:
None
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
milestone icon 1.0-beta2
Started by
Garth Wells
Completed by
Anders Logg

Related branches

Sprints

Whiteboard

GNW: Should also compare C++ versus Python performance.

AL: Good point

GNW: Poisson only is not a good test since it doesn't show up issues which arise for more complicated forms.

AL: Good point, please add a (limited) number of relevant test cases

AL: The list above is just a first iteration, please help out to expand and organize the test suite

AL: The benchbot is up and running. Please help out with providing relevant benchmarks before Johannes adds the final touches.

AL: I consider this implemented now (although with a somewhat different set of benchmarks). Johannes has backported quite a few of the benchmarks to DOLFIN 0.7.3

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.