Welcome > CHERI > Beri > Cheri Source Code reading > Test

Test

Reference ¹

Files in test suite

cheritest/trunk. root of test suite

gxemul_log/ output of gxemul test
log/ holds output of test
obj/ holds obj files, memory images, and assembly dumps.
tests/ Individual tests and their matching Python Nose classes
tools/ Utility functions to perform common functions such as interpreting BERI simulator and gxemul output
fuzzing/ Scripts for fuzz testing the TLB

init.s

A thin loader to set up various aspects of CPU and memory configuration.

set up a stack at the top of memory.
install default before- and after-boot exception vectors and handlers, which will dump the register file and terminate if triggered.
Explicitly clear all general-purpose registers except stack-related registers that may have been modified during startup.
Invoke a user-provided test function using JAL; currently all test written in assembly, but the calling convention should support C as well.
On return from test, dump the register file and terminate.

Nose invocation

Run individual test: go to directory and run nosetests.

nosetests xxx.py
nosetests -a 'capabilities'

Automation in Makefile:

http://somethingaboutorange.com/mrl/projects/nose/

make test 
  ->: nosetest
     ->: nosetests_uncached.xml
        ->: $(CHERI_TEST_LOGS)  // RUN bluesim and generate log file
            -> $(LOGDIR)/test_raw_trace.log
            -> $(LOGDIR)/test_raw_trace_cached.log
            -> $(LOGDIR)/test_raw_trac%.log
                ->: $(OBJDIR)/test_raw_trac%.mem $(SIM) $(CHERICTL)
                -> $(call PREPARE_TEST, $<) && \
                    ((BERI_DEBUG_SOCKET_0=$$TMPDIR/sock \
                    CHERI_TRACE_FILE=$(CHERI_TRACE_FILE) \
                    $(call RUN_TEST,$*); \
                    $(CLEAN_TEST)) &) && \
                    $(call WAIT_FOR_SOCKET,$$TMPDIR/sock) && \
                    $(CHERICTL) setreg -r 12 -v 1 -p $$TMPDIR/sock && \
                    $(CHERICTL) memtrace -v 6 -p $$TMPDIR/sock      
            -> $(LOGDIR)/test_clang_dma%.log: 
                ->: $(OBJDIR)/startdramtest.mem $(OBJDIR)/test_clang_dma%.mem $(SIM)
                -> $(call PREPARE_TEST,$<) && \
                    cp $(PWD)/$(word 2, $^) . && \
                    $(call REPEAT_5, CHERI_KERNEL=$(notdir $(word 2, $^)) $(RUN_TEST_COMMAND)); \
                    $(CLEAN_TEST)
            -> $(LOGDIR)/%.log : $(OBJDIR)/%.mem $(SIM)
	            -> $(call PREPARE_TEST,$<) && $(call RUN_TEST,$*); $(CLEAN_TEST)

        -> $(TEST_PYTHON) // a list of python test files; no target for them.
        -> FORCE
        -> PYTHONPATH=tools/sim PERM_SIZE=31 CACHED=0 nosetests --with-xunit --xunit-file=nosetests.uncached.xml $(NOSEFLAGS_UNCACHED) $(TESTDIRS) || true
  ->: nosetest_cached
     ->: nosetests_cached.xml
        -> PYTHONPATH=tools/sim PERM_SIZE=31 CACHED=1 nosetests --with-xunit --xunit-file=nosetests_cached.xml $(NOSEFLAGS) $(TESTDIRS) || true


-> PREPARE_TEST, $< (*.mem)
   -> mktemp && cd tmp 
    && cp $(PWD)/$(1) mem.bin 
    && memConv.py bsim 
    && memConv.py bsimc2
    && $(COPY_PISM_CONFS)
      -> cp cheri/trunk/memoryconfig $TMPDIR/memoryconfig

-> RUN_TEST, $* ($* as the test name)
   -> $(call REPEAT_5, $(RUN_TEST_COMMAND))


-> RUN_TEST_COMMAND = \
	LD_LIBRARY_PATH=$(CHERILIBS_ABS)/peripherals \
    PISM_MODULES_PATH=$(PISM_MODULES_PATH) \
	CHERI_CONFIG=$$TMPDIR/simconfig \
	CHERI_DTB=$(DTB_FILE) \
	BERI_DEBUG_SOCKET_0=$(CHERISOCKET)  $(SIM) -w +regDump $(SIM_TRACE_OPTS) -m $(TEST_CYCLE_LIMIT) > \
	    $(PWD)/$@; \

-> REPEAT_5 = \
	for attempt in 0 1 2 4 5; do if \
	$(1) \
	then break; else false; fi; done

Note: Make Basics:

automatic variables:
- $< the first dependence; $^ all the dependences; $@ target;
- $* stem in pattern matching, or target name with a recognized extension removed, or empty.
- more on GNU manual.
$(word n,text): returns the nth word in the text, first index is 1.
$(basename, names...) (remove extension if exists), $(addprefix, prefix, names...), $(addsuffix, suffix, names...), apply the option for all names one by one and return a list of new names. more

nosetests --with-xunit --xunit-file=nosetests.uncached.xml: use xunit plugin, which write results in standard XUnit XML format. The result xml file is given via –xunit-file=xxx option, default nosetest.xml if not given.

$(NOSEFLAGS_UNCACHED): -A “$(NOSEPRED) and not cached”

reference ↩

If you could revise
the fundmental principles of
computer system design
to improve security...

... what would you change?