Skip to content

Latest commit

 

History

History
179 lines (129 loc) · 6.15 KB

unit-tests.adoc

File metadata and controls

179 lines (129 loc) · 6.15 KB
Raw

Unit Tests in Jason

Table of Contents

Introduction

Jason has a set of tools for testing Jason agents which includes a test management engine, a tester agent skeleton and plans for assertion and performance checks.

The test library is written and interpreted by Jason.

This test facilities can be used both using the management engine or as a simple library.

The test management engine

Jason provides an infrastructure to add testing agents to be launched in tests, which allows, for instance, continuous integration tests for Jason agents.

The test management engine is launched by $ ./gradlew test command (use just gradlew on MS Windows). It will launch all tester agents present in the default test folder for asl files.

In case of any assertion fails the exit status will inform failure to the operation system. If all tests pass, the exit status is normal, meaning success. These status inform integration tools (e.g. circle CI, github actions) about the results of tests.

The default command $ ./gradlew test prints out only warning and severe log messages. It also supports $ ./gradlew test --info which besides higher levels also prints info messages (which include the .print internal action). For even more deailed output $ ./gradlew test --debug can be used.

Jason tests is a NOT cacheable gradle task which means it runs completely every time and no clean command is needed.

The tester agents that imports the skeleton tester_agent.asl are equipped with the test library such as assertion plans.

All plans in which the label has the annotation test will be automatically launched by the test task.

Using the test engine

  1. Create an agent for testing your agent(s) in the folder src/test/jason/asl (all agents in this folder are automatically launched in the test task).

  2. Make your agent a tester agent (e.g.: test_bob.asl) including the tester_agent.asl skeleton ` { include("$jasonJar/test/jason/inc/tester_agent.asl") } `

  3. Add to your test agent support to the agent it is going to test, eg: ` { include("bob.asl") } `

  4. Run $ ./gradlew test to check the results.

Example 1

Let us say we have an agent bob which has a rule to sum two numbers. We want to check if bob can sum correctly. The content of bob.asl is:

sum(X,Y,R):-R = X + Y.

To test it, we can create a tester agent following the steps above. The content of src/test/jason/asl/test_bob.asl can be:

{ include("$jasonJar/test/jason/inc/tester_agent.asl") }
{ include("bob.asl") }

@[test] /* Notice the label with the annotation `test` will be automatically launched */
+!test_sum :
    true
    <-
    ?sum(1.3,2.6,R);
    !assert_equals(3.9,R,0.1);
.

Running ./gradlew test the result for this test should be:

> Task :testJason
BUILD SUCCESSFUL in 3s

Running ./gradlew test --info the result for this test should be:

[test_bob] assert_equals on goal 'test_sum' PASSED
[test_manager] #1 plans executed, #1 PASSED and #0 failed.
[test_manager] End of Jason unit tests: PASSED
BUILD SUCCESSFUL in 3s

Let us say, bob is not doing sums correctly, let us say the content of bob.asl is:

sum(X,Y,R):-R = X + Y - 1. /* Notice, this -1 will make all sums wrong!!! */

For this failure condition, running ./gradlew test the result for this test should be:

[test_bob] assert_equals on event 'test_sum' starting at line 1 FAILED! Expected 3.9+/-0.1, but had 2.9000000000000004
[test_manager] #1 plans executed, #0 passed and #1 FAILED.
[test_manager] End of Jason unit tests: FAILED!
> Task :testJason FAILED
BUILD FAILED in 3s

Notice that failures are severe messages that will be shown regardless '--info' is set or not on gradle test command.

Example 2

Let us say we have an agent alice as below in which the testing functionality is provided by the plan do_something_with_a_counter. However, the agent is using an artifact counter that is not focus of our tests.

In this sense, one may find a good approach bypassing inc external action using a mock plan which is useful to abstract complexity that is not necessary in the current context.

+!do_something_with_a_counter
    <-
    !do_inc;
    ?count(N);
    .log(warning,"Counter state ",N);
.

+!do_inc
    <-
    inc;
.

{ include("$jacamoJar/templates/common-cartago.asl") }
{ include("$jacamoJar/templates/common-moise.asl") }

To test it, we can create a tester agent src/test/jason/asl/test_alice.asl which the code is as below.

In this implementation, the plan !do_inc which is triggering the external action inc is being mocked.

We can bypass a plan adding a mock plan on the beginning of agent’s plans library. Since it has same functor and it is on the top, the agent by default will use the mock plan.

{ include("$jasonJar/test/jason/inc/tester_agent.asl") }
{ include("alice.asl") }

count(5).

@[test]
+!test_do_something_with_a_counter
    <-
    ?count(N0);

    !assert_equals(5,N0);
    .add_plan({ +!do_inc :
        count(N)
        <-
        -+count(N+1);
    }, self, begin);

    !do_something_with_a_counter;

    ?count(N1);
    !assert_equals(6,N1);
.

Additional info

  • The default test folder src/test/jason/asl should only have tester agents, otherwise statistics will fail.

  • Files you want to include into your tester agents can be placed into src/test/jason/inc, the content of this folder is not counted into statistics.

  • Two similar assertions done in the same plan will not be counted since they would have same signatures.

The library

The library provides assertion plans, which can be found in test_assert.asl file.

An alternative use of the test tools is to do not use the management engine, instead, just include test_assert.asl to any agent.

{ include("$jasonJar/test/jason/inc/test_assert.asl") }

The results for assertions will be displayed as ordinary printed messages generated by the agent.