cctbx wiki baptiste addition

[Picca Frédéric-Emmanuel]

On Tue, 7 Aug 2012 11:30:49 +0200
Baptiste Carvello <devel at baptiste-carvello.net> wrote:

Hello

> > - a test failure should stop the build process
> 
> That's not how it works right now, upstream's way is to let the tests
> run to the end, then grep through the log. But I can easily modify the
> test runner in "cctbx_sources/libtbx/test_utils.py" to check the return
> codes and abort when appropriate.

Infact it is always nice to have all failed test at the end for all architectures.
So maybe the logic should be to run all test, check if the test fail or not
and at the end return True or False depending of the result.

this grep log thinks seems to me not really robust...

is it possible to have something like this in pseudo code

ok = True
for test in tests:
    ok &= test.run()

return ok

another concern about this is that the log of the test should be sent to stdout to be present in the debian build log.
if the log is only accessible from a file during the build, it is kind of useless...

> 
> > - "python tests that are not part of a package as "package_data" in setup.py. Thus they will be copied by distutils to its build directory, and also installed by our packages."
> > 
> > do you have exemple of theses test and how they will be usable once installed on the system ?
> 
> for example: "cctbx_sources/fable/test/*.py"
> or "cctbx_sources/cctbx/adptbx/boost_python/tst_adptbx.py"
> 
> An example of a test script which is in a package, but needs data which
> are not: "cctbx_sources/fable/tst_cout_compile.py"

ok and where are the data for this test ?
is it possible to create a cctbx-test-data package for this ?
licenses of the data ?
wiki :)

> All the tests are meant to be directly executed as python scripts, and
> should work the same on user systems, provided their data are installed too.
> 
> Logically, all the tests would belong in the cctbx-test package, but it
> may be easier to install them as part of the cctbx-python package.

> > - it would be nice to have a cctbx-test package with the binay test program.
> > but it whould imply having man pages for thoses scripts.
> > how many of them ?
> 
> A quick grep ("grep '$B' cctbx_sources/*/run_tests.py|wc -l") gives 24,
> in a half-dozen directories.

and if I correctly understand, thoses binaries are run from run_tests.py scripts.
what is the commad line used to run all the tests ?

We should imagine creating a cctbx-test package:
- providing a /usr/bin/cctbx-test script with a very simple
  man page explaining thaht cctbx-test execute all the cctbx tests.
- put all the binary test program under
  /usr/lib/cctbx/xxx this way no need to create a man page for all of them.

this cctbx-package should depends on a cctbx-test-data

> It's probably possible to write an "install_tests" command to copy them
> in place.

Yes it seems that you are great for this :)

Cheers

Fred

-- 
GPG public key 4096R/4696E015 2011-02-14
    fingerprint = E92E 7E6E 9E9D A6B1 AA31  39DC 5632 906F 4696 E015
uid  Picca Frédéric-Emmanuel <picca at synchrotron-soleil.fr>

GPG public key 1024D/A59B1171 2009-08-11
    fingerprint = 1688 A3D6 F0BD E4DF 2E6B  06AA B6A9 BA6A A59B 1171
uid  Picca Frédéric-Emmanuel <picca at debian.org>