Go to the documentation of this file.
00001 // Oliver Kullmann, 26.1.2008 (Swansea)
00002 /* Copyright 2008, 2009, 2011 Oliver Kullmann
00003 This file is part of the OKlibrary. OKlibrary is free software; you can redistribute
00004 it and/or modify it under the terms of the GNU General Public License as published by
00005 the Free Software Foundation and included in this library; either version 3 of the
00006 License, or any later version. */
00007 testobjects/*.mac"); do oklib --maxima --batch=${F} --very-quiet; if [[ $? != 0 ]]; then exit 1; fi; done)
00366      \endverbatim
00367      would work except that maxima returns 0 apparently in any case???
00368      It seems not possible to get a reaction on the error??? </li>
00369      <li> A possibility would be to set an environment-variable via "system".
00370      But apparently using for example system("export OKLIBMAXIMA=1") is not
00371      transported to the outer world, and thus is unusable. </li>
00372      <li> Or we use a different configuration file, as discussed above; but
00373      still the problem how to get informed about the error. </li>
00374      <li> Is it really necessary to write to a file via "system" ??? </li>
00375      <li> It seems that we should add the capability to the assert-function
00376      that in case of error and oklib_automatic_test=true (by default it's
00377      false) something is written to a file. </li>
00378      <li> For batch-running testobject-files, before the run the file is
00379      deleted, and if it has been created then the buildsystem issues an
00380      error. </li>
00381      <li> This assumes that every testobject-file is run on its own ---
00382      seems alright. </li>
00383      <li> The output of the batch-runs is all redirected into one log-file.
00384      One needs to look at it from time to time. </li>
00385      <li> Perhaps automatically one should parse the output for "error" etc.
00386      </li>
00387      <li> According to the e-mail reply by Mike Hansen to my request at the
00388      Maxima-mailing list (27.2.2008) we could use the Sage interface. </li>
00389     </ol>
00390    </li>
00391   </ul>
00394   \todo Testing the demos
00395   <ul>
00396    <li> Also the demos-files need to be run, via oklib_load, to see whether
00397    they still function correct. </li>
00398    <li> Since they contain asserts, this is also contains tests. </li>
00399    <li> A problem is that some demos run longer. </li>
00400    <li> So demos need to be qualified as "basic tests", "full tests",
00401    or "extensive tests". The problem how to do this.
00402     <ol>
00403      <li> The first line of a demos-file shall be
00404      \code
00405 if oklib_test_demos then
00406  if oklib_test_demos_level < 2 then return()$
00407      \endcode
00408      (where "2" here is the test-level of this file ("extensive" in
00409      this case)).
00410      </li>
00411      <li> Normally, oklib_test_demos=false. </li>
00412      <li> For running the demos in test-mode, oklib_test_demos=true,
00413      and then oklib_test_demos_level decides whether to run the demos-file
00414      or not.
00415      </li>
00416     </ol>
00417    </li>
00418    <li> And should these "tests" be included in the normal maxima-test? </li>
00419    <li> First we create a special target "maxima_check_demos", and then
00420    we'll decide. </li>
00421    <li> This target, as usual, loads all demos-files (.mac-files in
00422    demos-directories) below the given directory. </li>
00423    <li> So again a special maxima-init-file is created by the process,
00424    which contains all respective load-instructions. </li>
00425    <li> Again the question is whether we do this recursively. </li>
00426    <li> And again this appears to be superior. </li>
00427   </ul>
00430   \todo Improving the test system
00431   <ul>
00432    <li> It would be better to check whether all testobject-expressions actually
00433    evaluate to true (since tests might be broken, and for example simply
00434    nothing might be computed):
00435     <ol>
00436      <li> I remember that there is a Maxima function, which like "batch"
00437      executes all expressions in a file, and checks whether each evaluates
00438      to true? </li>
00439      <li> Apparently there is an undocumented feature of batch, namely it
00440      can be called as "batch(filename, 'test)", in which case it expects
00441      the file to be organised as a list of pairs of expressions
00442      \verbatim
00443 expression;
00444 expected_value$
00445      \endverbatim
00446      however this would be clumsy (since our expected value is always just
00447      "true"), and furthermore also some error-log is written to some file,
00448      which likely is (too) hard to control. </li>
00449     </ol>
00450    </li>
00451    <li> In oklib_test_level>=1 we must also additionally run the test
00452    with oklib_test_level-- and with oklib_monitor=true and
00453    oklib_monitor_level=0,1 (at least). </li>
00454    <li> return-expressions in functions to be tested:
00455     <ol>
00456      <li> If the function to be tested doesn't use a block, but uses a
00457      return-expression, then this just terminates the assert-evaluation
00458      itself. </li>
00459      <li> How can this be avoided? </li>
00460     </ol>
00461    </li>
00462    <li> DONE (now handled in 
00463    Buildsystem/MasterScript/SpecialProcessing/plans/general.hpp)
00464     We have the problem that the created file maxima-init.mac clashes
00465     with other such files created when running "oklib --maxima":
00466     <ol>
00467      <li> DONE (this is now handled by the userdir-directory)
00468      Is it possible to use for the test-runs a different initialisation
00469      file? Ask on the Maxima mailing list. </li>
00470      <li> See "Improve locality" in
00471      Buildsystem/MasterScript/SpecialProcessing/plans/general.hpp. </li>
00472     </ol>
00473    </li>
00474   </ul>
00477   \todo Handling floating and bigfloating numbers
00478   <ul>
00479    <li> It seems that assert_float_equal and assert_bfloat_equal work fine,
00480    however especially for assert_bfloat_equal this needs to be properly
00481    tested. </li>
00482   </ul>
00484 */