wip: Merge into main the declarative testing feature (cmd to be renamed to suite)

[leogr]

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind documentation

/kind tests

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area commands

/area pkg

/area events

What this PR does / why we need it:

This PR merges the effort in the declarative-testing branch into the main branch. At a high level, it introduces the capability to generate suspicious actions by providing a YAML-formatted declarative description of them.
The PR introduces three commands under the declarative namespace (temporary solution to avoid overlapping with the already existing commands):

# Run test(s) specified via a YAML description.
event-generator declarative run [flags]
# Run test(s) specified via a YAML description and verify that they produce the expected outcomes.
event-generator declarative test [flags]
# Document test(s) YAML description properties
event-generator declarative explain [<propertyPathExpr>] [flags]

The test command is a superset of the run command: besides performing the described suspicious actions, it also connects to a Falco instance via gRPC and verifies that Falco correctly detected them as suspicious.
Notice: this requires Falco to be configured to output its alerts using the gRPC output and append to them the suspicious process environment variables:

sudo falco -c /etc/falco/falco.yaml -c /etc/falco/falco_rules.yaml -o grpc.enabled=true -o grpc_output.enabled=true -o 'append_output[]={"extra_fields": ["proc.env"]}'

Both run and test commands support loading test descriptions from multiple files and/or folders: this is achieved by using one or multiple times the flags --description-file and --description-dir. These flags accept also a comma-separated list of items. Loading a directory means loading all YAML files inside (notice: this is not done recursively). Lastly, they also support loading test descriptions from standard input; this is achieved by not providing any of the aforementioned flags.

The explain command can be used to explore the YAML properties used to describe the actions to be performed and the expected outcome from executing those actions (this latter has a meaning only for the test command). At a high level, each YAML file is a list of test descriptions. The following is an example of it:

tests:
  - name: "Test example"
    rule: "Reverse shell rule"
    description: "Testing reverse shell rule"
    runner: HostRunner
    context:
#      container:
#        image: imageName
#        name: containerName
      processes:
        - args: "arg1 arg2"
          name: "proc0"
          exe: "arg0"
          user: user1
        - user: user2
        - user: root
          capabilities: "cap_net_admin,cap_net_bind_service,cap_chown=ep"
    resources:
      - type: clientServer
        name: cs1
        l4Proto: "%{item.l4Proto}"
        address: "%{item.address}"
    steps:
      - type: syscall
        name: dup1
        syscall: dup2
        args:
          oldFd: "${cs1.client.fd}"
          newFd: 0
    expectedOutcome:
      source: "syscall"
    cases:
      - strategy: matrix
        values:
          l4Proto: ["tcp4", "udp4"]
          address: ["11.0.0.1:80"]
      - strategy: vector
        values:
          l4Proto: "unix"
          address: ""

This example contains 1 test description template: it is possible to recognize that it is a template because it specifies the cases keyword. This template will generate 3 tests in total: 2 for the first case and 1 for the last case. Practically speaking, the generated test will be generated by substituting to the %{item.<valueKey>} placeholders, following combinations of values:

• l4Proto="tcp4" and address="11.0.0.1:80"
• l4Proto="udp4" and address="11.0.0.1:80"
• l4Proto ="unix" and address=""

For more details regarding test cases, see:

• feat(decl/loader): add test cases support and fix doc generation for list items #266

For each of the produced test descriptions, the event-generator creates a fake process chain built using the user specified characteristics. In this case:

• the first process will be named proc0 will run as if it were spawned using the command line arg0 arg1 arg2 by user1 (which is created on the fly or reused if it already exists on the system
• the second process (child of the first process) will run as if spawned by user2
• the third process (child of the second process) will run as if spawned by root with the specified capabilities in its sets

The third process will perform the actual suspicious actions: resources and steps lists are used to describe them. resources are created in order before steps are run. In this case, the cs1 resource is of type clientServer: this means that a client and a server will be spawned using the specified l4 protocol and the specified address. In the case of stream-oriented protocols, the client and server will be automatically connected. Other resource types are provided for user convenience, such as fd (creating a file descriptor of the specified type) or process (spawning a process with the provided characteristics). See the following for a process resource use case leveraging the kill system call test step:

• feat(decl): add process test resource and kill system call test step #241

After the resources creation, the specified steps are executed. In this case, there is only 1 step, named dup1, of type syscall. This will perform a dup2 system call using the cs1 resource's client file descriptor as a parameter for the oldFd system call argument. The syntax used is called "field binding" and allows the exposed value of a previously executed step/resource to be used as the value for the current step field. The syntax format is ${<resourceName|stepName>.<pathToExposedField>}. Currently, the syscall step type is the only one that has been introduced. It supports about 20 system calls.

If the test command is used, after the test execution, the event-generator listens for alerts coming from the connected Falco instance, and matches them against the provided expectedOutcome. In this case, an alert is expected to be generated for the rule Reverse shell rule by the source syscall. The event-generator generates the results of these matchings to inform the user if the actions performed generated the expected results or not.

When running multiple tests, becomes relevant the grouping the event-generator performs of them. Tests are grouped in test suites based on the rule they are attempting to test. This means that a single test suite is associated with a single rule. Tests results are also grouped by test suite.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

• As discussed, we made the decision to rename the declarative command to suite because it clearly conveys that the command deals with a group of these and conceptually reflects that the YAML structure defines multiple tests. This also functions as a command namespace, allowing the co-existence of the old approach with the new functionality during the transition period (still TBD). The following steps would be to (a) rename the command from declarative to suite and add deprecation notices for old commands. These changes will be implemented in follow-up PRs.
• This PR has been prepared together with @ekoops, who has the credit for this terrific job! Thank you!

[poiana]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: leogr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment