fbpx

Benchmarks

Rather than us saying our solver has a stellar performance record, we’ll let the benchmarks do the talking.

Third-party Benchmarks

Octeract Engine has been benchmarked by Prof. Hans Mittelmann of Arizona State University. The tests were run on QPLIB, a library of all types of quadratic problems. QPLIB is an interesting library because of how problems are selected:

“Our filtering process has then been in part based on the idea that, if a significant fraction of the solvers that can solve a QP instance do so in a “short” time, then the instance is not challenging enough to be included in the library.”

Fabio Furini, QPLIB paper

The tests were run using Octeract Engine 1.01.17 with 8 CPUs. As we see in the logs, other commercial solvers used commercial LP/MILP solvers such as CPLEX/GUROBI, as well as commercial NLP solvers. The Engine used open-source solvers to solve all LP, MILP, and NLP subproblems, so the results are closer to what you get out of the box.

Octeract Engine natively supports supercomputing. Now that we’ve established a performance baseline, let’s investigate what the Engine can do when given more computing power.

Bar chart showing time taken to solve QCQPs using 8 CPUs vs 172 CPUs (3)

This plot shows a CPU comparison for all problems that didn’t timeout in the original 8-core 3-hour runs and have been solved ten times faster with 172 CPUs.

This is, of course, not a fully rigorous comparison as we’re running a newer version of the Engine and on different hardware. However, the difference in computing power is overwhelming enough to make the comparison meaningful. We ran all non-convex continuous quadratic problems using 172 CPUs (~20x computing power).

Parallel Scaling Power

time-and-date (2)

With the CPU boost, Octeract Engine solved nearly twice as many problems (24 vs 14), and 80% of problems were solved at least 10 times faster (including perfect scaling of 20x for some problems). The rest were a bit too easy to fully benefit from parallel scaling and were between 3 and 5 times faster instead.

Octeract Engine is the only optimisation solver that uses the proprietary Kazazakis Distributed Algorithm (named after Octeract’s founder), so adding more computing power results in linear speed-ups for most problems. When the Engine is running in parallel mode, it uses this algorithm to close the optimality gap much more quickly and to find the global solution many times faster.

diagram (1)

Octeract Benchmark

While QPLIB is definitely interesting, Octeract Engine can do much more than just solve QPs or MILPs. It can solve any mathematical structure, no matter how obscure, to guaranteed global optimality. This includes all trigonometric functions, \(|f(x)|\), \(\max(f(x),g(x))\), \(\min(f(x),g(x))\), \(\sqrt{f(x)}\), and more. We put our solver to the test with test problems from the literature as well as our own problems (most of which are designed to break solvers in various ways).

Number of Problems

3016

Feasible Solutions

86%

Timeout (minutes)

30

Performance Records

So, let’s see some statistics on how the Engine performs on several industry-standard libraries, using just 1 CPU and a 30 min timeout. These libraries include a wide range of non-linear structures, including all of the ones mentioned above:

MINLPLib

This library consists of problems from the diverse classes of mixed integer non-linear programming (MINLP) and non-linear programming (NLP)

86

% Feasible Solutions

1464

Number of Problems

60

% Proved Global Optimality

CoconutLib

This library contains a test set of continuous global optimisation problems .

92

% Feasible Solutions

1285

Number of Problems

71

% Proved Global Optimality

QPLib

QPLib consists of a diverse class of quadratic programming problems. This includes both discrete and continuous problem instances of different characteristics.

62

% Feasible Solutions

360

Number of Problems

22

% Proved Global Optimality

Can the Engine Solve your Problem?

Yes! The Engine’s performance speaks for itself. It has the ability to solve any complex problem. In terms of problem size, the largest MINLP solved by Octeract Engine is:

110000

Variables

100000

Constraints

40

Seconds to Solve

Have you got a bigger problem? Bring it on. The Engine can solve any non-linear problem in its initial form and it always returns a reliable solution. Hit our solver with your best shot.

Trial Us Out

Take the Engine for a test drive.

Contact Us

We’re here to lend an ear. Get in touch.

Plexal, 14 East Bay Lane, London, E15 2GW