Test Cleanup
Test Cleanup Project
See the mission statement mail.
User Interface
CppUnit
For tests in C++, plain CppUnit (at least version 1.12.1) is available. (OOo bundles CppUnit 1.12.1 as an external module; alternatively, you can configure --with-system-cppunit
.)
Implement the test code as a dynamic library, following CppUnit conventions (i.e., using CPPUNIT_TEST_SUITE_REGISTRATION
and CPPUNIT_PLUGIN_IMPLEMENT
macros). The following settings are needed in the makefile.mk
:
ENABLE_EXCEPTIONS = TRUE
CFLAGSCXX += $(CPPUNIT_CFLAGS)
SHLnRPATH = NONE
SHLnSTDLIBS += $(CPPUNITLIB)
SHLnVERSIONMAP = version.map
, where that file must contain the following (see details below for improving this):
UDK_3_0_0 { global: cppunitTestPlugIn; local: *; };
To run tests, sal/cppunittester
delivers a cppunittester
executable that has some advantages over the plain CppUnit DllPlugInTester
:
- It uses
SAL_IMPLEMENT_MAIN
to properly set up the per-process settings required by URE. Test code as well as tested code based on URE will expect this and would fail in a process that is not set up in this way. - Getting CppUnit to build as an external module within the Windows OOo build is a non-trivial task. The less parts of it that need to be built, the better.
Something like the following in makefile.mk
will execute the tests:
ALLTAR : test test .PHONY : $(SHL1TARGETN) $(CPPUNITTESTER) $(SHL1TARGETN)
$(CPPUNITTESTER)
internally uses $(AUGMENT_LIBRARY_PATH_LOCAL)
to favor library instances in the local output tree over ones in solver. Alternatively to the above, .INCLUDE: _cppunit.mk
calls $(CPPUNITTESTER)
on each of $(SHL1TARGETN)
, …, $(SHL10TARGETN)
that is defined.
JUnit
For tests in Java, plain JUnit (at least version 4) is available. (OOo requires JUnit as a prerequisite, see junit.org for downloads; you can configure --with-junit=…
if it is not automatically found; alternatively, you can configure --without-junit
, disabling JUnit-based tests.)
subsequenttests
Some tests have build and/or run time dependencies on modules further up in the hierarchy, or modules that the rest of the module in question does not have dependencies on (and which are thus not listed in the module's build.lst
). The extreme case are tests that need a complete OOo installation as their test environment.
For such tests, the whole makefile.mk
should be wrapped in:
.IF "$(OOO_SUBSEQUENT_TESTS)" == "" nothing .PHONY : .ELSE ... .END
(and the directory included in the module's build.lst
).
The idea is to do a complete OOo build first (cd smoketestoo_native && build --all
) and then build and execute those tests either selectively or wholesale.
Selectively building a specific such test directory can be done with cd … && OOO_SUBSEQUENT_TESTS=TRUE dmake
.
To build and execute all these test directories, the tool subsequenttests
(in solenv/bin
) exists. It scans the build.lst
files of all modules for makefile.mk
s that mention OOO_SUBSEQUENT_TESTS
and simply executes dmake
in all those directories, with OOO_SUBSEQUENT_TESTS
set. It supports a -Pn
switch to build n directories in parallel.
Details
Working on CWS sb118 , CWS sb120 .
CppUnit 1.12.1
- Replace existing
cppunit
andtestshl2
combo with latest CppUnit 1.12.1. - OOo
configure --with-system-cppunit
(for now only if also--with-system-stl
!); minimal required version for now arbitrarily 1.12.1. - TODO: Upstream
PATCH_FILES
(windows.patch
?!?). - TODO:
RPATH
in generated libraries and executables. - TODO: Improve
CPPUNIT_PLUGIN_EXPORT
and get rid of version maps (GCC exception problems?). - TODO:
cppunit/makefile.mk
MY_LIBS = -lm
Solaris hack. - TODO: In general OOo build environment, no longer set
CFLAGS
etc. as environment variables; have package-specificOOO_package_CFLAGS
etc. and combine those intoCFLAGS
etc. where necessary (seeOOO_STLPORT_CXXFLAGS
etc. incppunit/makefile.mk
). - TODO: Why use
cppunittester
instead ofDllPlugInTester
? (Remove building/deliveringDllPlugInTester
.) - TODO: Make
CppUnit::assertion_traits<rtl::OUString>
fromsmoketestoo_native/smoketest.cxx
generally available.
testshl2
solenv/inc_cppunit.mk
/cppunit.mk
: “adaptedcppunit.mk
tocppunittester
and no longer require ‘dmake test
’ to run tests”.- TODO:
DEV300_m68
introducedTESTSHL2LIB
(which I removed again) and uses it in manyqa
makefiles (that I need to touch someday, anyway).
Unit Tests
OOO_SUBSEQUENT_TESTS
(build.lst
lists local dependencies of those directories, so that modifying a module and then calling “OOO_SUBSEQUENT_TESTS=x build
” rebuilds the module and executes the tests).- TODO: If Build Environment 2.0 would build directly to solver (instead of local output tree/deliver), that would simplify tests that currently need to take care to test local libraries instead of solver ones (see, for example,
OOO_TEST_PREFIX
in theservices.rdb
ofstoc/test/uriproc/makefile.mk
). - TODO: Constructs like “
-env:UNO_…=$(my_file)…
” would fail for problematic characters (spaces!) in paths.
Smoke Test
- Redesign
smoketestoo_native
to use CppUnit and pre-installed OOo. - TODO: For
smoketestoo_native/makefile.mk
“-env:arg-env=$(OOO_LIBRARY_PATH_VAR)"$${{$(OOO_LIBRARY_PATH_VAR)+=$$$(OOO_LIBRARY_PATH_VAR)}}"
” mechanism:osl_executeProcess
ustrEnvironments
members without “=
”: all platforms; documentation; documentation ofosl_executeProcess_WithRedicrectedIO
. - TODO: Dynamically create
smoketestdoc.sxw
? - TODO: “changed
tools/debug
to never fall back from writing to shell to writing to file, and readDBGSV_INIT
fromrtl::Bootstrap
; the latter causes problems iftools/debug
is called from static code executing beforeSAL_MAIN
has calledosl_setCommandArgs
→ removed statics from code and added assertions toosl/process
; various improvements tosmoketests.cxx
(incl. passing an appropriatedbgsv.ini
for non-pro builds)”: Instead of-env:DBGSV_INIT=
insmoketest.cxx
(and supportingDBGSV_INIT
as bootstrap variable), globally in the build environment letDBGSV_INIT
environment variable point to adbgsv.ini
witherror=coredump
, to catch all assertions thrown during build time (incl. ones from tests like smoke test; see Issue 109142 ). - TODO:
-headless
? (appears to still display windows on Windows and Mac OS X, however, see Issue 109052 )
Pre-Installed OOo
- For Sun Hamburg Release Engineering, take MWS archive installation sets from “S:” instead of solver: “take installation set from
SHIPDRIVE
if appropriate”. - TODO:
PKGFORMAT=installed
instead ofarchive
?
unoapi
Tests
- Known issues: Issue 109517 , Issue 109518 , Issue 109643 , Issue 109917 .
- Missing tests: Issue 109564 .
- Sporadic failures: Issue 109728 , Issue 109770 , Issue 109916 , Issue 109939 , Issue 110253 .