Ralf Wildenhues wrote:

Hello there,

* Roberto Bagnara wrote on Wed, Oct 19, 2005 at 09:13:16PM CEST:

...

Instead, what I would like to have is to only say

test1_SOURCES = test1.cc test2_SOURCES = test2.cc ...

and then achieve the effect of (sorry for the pseudo-code)

for flags in $FLAGS_CHOICES do make check with CXXFLAGS="$flags" force recompilation at the next iteration (e.g., by erasing executables) done

How can I best obtain this effect without giving up (too many of) the advantages offered by Automake?

I believe something along these lines may be achieved by

| check_SCRIPTS = runtests | check_PROGRAMS = test1 test2 | test1_SOURCES = test1.cc | test2_SOURCES = test2.cc | TESTS = $(check_SCRIPTS) | my-check-clean: | rm -f $(check_PROGRAMS) *.o *.obj

in Makefile.am, and your pseudo code in `./runtests': for flags in $FLAGS_CHOICES do make check CXXFLAGS="$flags" TESTS='test1 test2' make my-check-clean # to force recompilation at the next iteration done

Surely the object removal is a bit of a hack, and should be adapted unless your test suite lives in its own directory/Makefile.am. I haven't tested this, by the way; please complain if it doesn't work. :)

Dear Ralf, thank you very much for your help (and sorry for the delay, but I wanted to make sure we did all our homework before getting back to you). Here is the script we are using: http://www.cs.unipr.it/cgi-bin/cvsweb.cgi/ppl/tests/BD_Shape/run_tests?rev=1... The dirty_marker trick allows us to avoid useless recompilations yet addressing the case where one run of tests is interrupted. We have also managed not to duplicate the list of tests and to deal with XFAILs in a satisfactory way. Any further advice you may have is very welcome.

If you want to go much further, you either end up creating more complex shell scripts, or using one of the more advanced test suite creation tools: Autoconf's Autotest, DejaGNU, ...

Autotest looks attractive. We may consider switching to it as soon as it stabilizes. All the best, and thanks again, Roberto -- Prof. Roberto Bagnara Computer Science Group Department of Mathematics, University of Parma, Italy http://www.cs.unipr.it/~bagnara/ mailto:bagnara@cs.unipr.it

Ralf Wildenhues wrote:

Ah, ok. A couple of comments. First, a bug I introduced by giving a non-complete example: Some `make' implementations will not allow you to override a macro on the command line iff it is also set in the Makefile. With `TESTS', that is the case in your script. Portable would be TESTS='...' make -e check

but `make -e' has its share of problems, too, depending on your environment (same issue with the other variables, of course). Another point where Autotest is more flexible.

Furthermore, you write

| check_PROGRAMS=$(MAKEFLAGS='' make -s print_check_PROGRAMS)

which I assume you need to avoid clutter in the output. I know many systems where it is very useful to override the `make' command used, so $MAKE would probably be better here and elsewhere in the script (unless that interferes with clutter in the output), e.g. to be able to use a make that allows macro override on the command line.

Thanks a lot, Ralf. I have done as you suggest.

Then, a comment to the Makefile.am: you don't need the lines | srcdir = @srcdir@ | VPATH = @VPATH@ | @SET_MAKE@ | SUBDIRS =

automake will take care of that by itself.

Is also abs_srcdir = @abs_srcdir@ redundant?

* Roberto Bagnara wrote on Sat, Jan 07, 2006 at 06:12:06PM CET:

...

Autotest looks attractive. We may consider switching to it as soon as it stabilizes.

The next Autoconf release should have a decently usable version of it.

I look forward to it. Thanks again, Roberto -- Prof. Roberto Bagnara Computer Science Group Department of Mathematics, University of Parma, Italy http://www.cs.unipr.it/~bagnara/ mailto:bagnara@cs.unipr.it