sampleTTSmetric#

class SampleTTSMetric(tau_attribute: str, percentile: float = 50.0, confidence: float = 0.99, num_bootstraps: int = 100, failure_fill_in_value: float = 1.7976931348623157e+308, tolerance: float = 1e-05, seed: int | None = None, **kwargs)[source]#

Bases: Metric

Time to solution (TTS) metric.

Time to solution (TTS) metric.

Parameters:

  • tau_attribute (str) – Name of result key to use as tau = the length of each

  • run –

  • results. (tau is then set to the mean over the) –

  • percentile (float, optional) – The percentile of the R99 distribution to

  • 50.0. (find. Defaults to) –

  • confidence (float, optional) – The required confidence such as 0.99 for 99%

  • 0.99. (confidence. Defaults to) –

  • num_bootstraps (int, optional) – Number of bootstrapped samples.

  • 100. (Defaults to) –

  • failure_fill_in_value (float, optional) – Fill in value to replace the mean

  • with (and std) –

  • sys.float_info.max. (Defaults to) –

  • tolerance (float, optional) – Tolerance for comparisons with

  • 1e-5. (best_known_energy. Defaults to) –

  • seed (int, optional) – Random seed for the bootstrapping process.

  • None. (Defaults to) –

Keyword Arguments:

method. (Additional arguments to pass to the parent __init__) –

Raises:

ValueError – Raises error if the confidence value is out of range.

calc(results: list[list], best_known_energies: list, **kwargs) → list[float][source]#

Calculate the mean and std of the sample TTS with the given percentile.

Parameters:

  • results (list[list]) – A result for each problem.

  • best_known_energies (list) – A best known energy for the problem associated with each result.

Keyword Arguments:

Metric.calc(). (Captures additional named arguments passed to) –

Returns:

The estimate of the given percentile of the TTS distribution, the std of the given percentile of the TTS distribution.

Return type:

list[float]

calc_R99(success_probability: float) → float[source]#

Calculate the R99.

Given a set of solutions and the best known solution, calculate the R99, defined as the number of independent runs that we need to call the solver to find a solution as good as the best_known_energy (or better) at least once with 99% confidence.

Parameters:

success_probability (float) – The success probability for solutions given best_known_energy.

Raises:

ValueError – Raises an error if the confidence value is out of range.

Returns:

The calcuated R99 value.

Return type:

float

calc_R99_distribution(success_probabilities: list[float], num_repeats: int) → ndarray[source]#

Returns the percentile-th R99 distribution for a sample of problems.

Note: problems should be of the same size.

Parameters:

  • success_probabilities (list[float]) – A success probability for each problem.

  • num_repeats (int) – Number of repeats in each result.

Returns:

An R99 distribution from bootstrapping.

Return type:

numpy.ndarray

calc_success_probabilities(results: list, best_known_energies: list) → ndarray[source]#

Calculate the success probabilities for all problems.

Parameters:

  • results (list) – Results for each problem.

  • best_known_energies (list) – A best known energy for the problem associated with each result

Returns:

A success probability for each problem.

Return type:

numpy.ndarray

calc_success_probability(solutions: list | dict, best_known_energy: float) → float[source]#

Calculate the success probability for a given problem.

Parameters:

  • solutions (Union[list, dict]) – The solutions.

  • best_known_energy (float) – The lowest energy known.

Returns:

Success probability.

Return type:

float