Tools for evaluating local search algorithms
More...
Go to the source code of this file.
Detailed Description
Tools for evaluating local search algorithms
 Todo:
 Evaluation tools for run_ubcsat

Functions are needed for standard evaluations.

At the Rlevel, we go for convenience, and overloading functions like "summary" or "plot" seems be the right way.

One needs to find out how to do this, how to recognise our specific data frames.

"summary" should show data, while "plot" might run through a serious of plots, showing the data from different angles.

Plotting an overview:

Given the dataframe in E, the best algorithms seems best graphed by which plots per algorithm the boxplots of the mindistribution.

However it is useless if on the xaxis not all algorithms are listed.

So the algorithmsnames needed to be printed vertically.

See https://stat.ethz.ch/pipermail/rhelp/2003May/033400.html .

And all names need to be shown.

And an option for cutoff_min is needed, which removes algorithms whose best minvalue is greater than the cutoffvalue.

DONE (eval_ubcsat_dataframe now prints the sorted tabledata) Sorting the algorithms:

Considering single algorithms by e.g.
> table(E$min[E$alg=="adaptnoveltyp"])
(note that currently algorithm names are inappropriately handled).

These tables can be put into a linear order by sorting first according to minvalue reached (the lower the better), and second by count obtained (the higher the better).

Also the time needed should at least be shown.

A function should be written which prints out the sorted tables in a nice way.

As a first attempt we have eval_ubcsat_dataframe, which just shows all results in table form (sorting needs to be added).

Perhaps then the (first) evaluation tool just uses plot(E$alg,E$min), followed by printing those *sorted* tables.

Also the average number of steps to the optimum is relevant: When it is closer to the cutoff, then increasing the cutoff likely might yield an improvement, while if not then the algorithm wanders randomly around without achieving something (so increasing the cutoff seems of little value).
Definition in file Evaluation.hpp.