Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bindgen cli/v6 #248

Closed
wants to merge 7 commits into from
Closed

bindgen cli/v6 #248

wants to merge 7 commits into from

Conversation

jasonish
Copy link
Owner

  • rust: integrate bindgen to generate rust bindings to C
    Add a minimal integration of bindgen to the build.

    This required some refactoring of the C so app-layer-events.h did not
    also include "rust.h", which causes issues for bindgen, probably
    related to circular references.

    AppLayerEventType was chosen as the first step as its an argument type
    some app-layer functions that we may want to use bindgen to export
    Rust, and one of the requirements of bindgen might be that C functions
    should only use datatypes defined in C, and not Rust. Following such a
    rule also prevents circular dependencies between Rust and C code.

    As bindgen can only accept one header file, we construct a
    pseudo-header file of all the headers that need to be
    consumed. Makefile dependency checking is done to make sure the
    pseudo-header is only generated as needed to avoid rebuilding every
    time.

    Special handling is required for Windows to use the Windows path.

    Ticket: http: configures libhtp to allow spaces in uri OISF/suricata#7341

  • bindgen: pre-generate rust bindings to C
    This simplifies building, as we don't have to worry about path and
    such under autoconf/automake. It does however mean when we update C
    headers that are exposed to Rust, we manually have to re-generate the
    bindings, but this is a check we can do in CI.

    It also eliminates the need for everyone who wants to build Suricata
    to have bindgen and clang tools installed.

  • app-layer: rename AppLayerEventType to SCAppLayerEventType
    Follow the naming scheme for public exports.

  • app-layer: remove prototypes from decode.h
    This lets us remove decode.h from app-layer-events.h as pulling in
    app-layer-events.h shouldn't result in pulling in dpdk, and other
    includes not related to app-layer-events.

    decode.h also doesn't need those forward declarations anymore due to
    previous changes.

  • bindgen: export SCAppLayerStateGetEventInfoByIdFn from C to Rust
    Instead of defining this function pointer in type in Rust, and having
    it in C signatures, create a type and export it to Rust.

    To facilitate this, and new header has been creates,
    "app-layer-types.h", this is to avoid the circular reference of C
    headers pulling in "rust.h" which are required to generate Rust
    bindings.

  • bindgen: export AppProto and AppProtoEnum to Rust
    This exposes the C define ALPROTO values to Rust without having to
    perform some runtime initialization with init_ffi.

    As app-layer-protos.h was clean of a circular reference to rust.h we
    could use it directly, it just needed the addition of
    suricata-common.h.

  • dhcp: use ALPROTO_DHCP from bindgen

  • github-ci: test if bindgen bindings are up to date
    This required us to remove the auto-generated by header as it can
    change depending on what version is used.

    I suppose different versions could generate slightly different output
    that would cause this error out. In which case we may want to use a
    custom script that can be a bit smarter output, or simply error out if
    any of the headers used in the bindings is newer than the generated
    _sys.rs file.

  • rust: add bindgen bindings check to make check
    If we have bindgen, and we're building in-tree, rebuild the bindings
    and compare to the previous bindings and print a warning if there is a
    difference.

    Only a warning for now, as I'm not sure where this might fail, and I
    don't want it to become a nuisance.

@jasonish jasonish force-pushed the bindgen-cli/v6 branch 3 times, most recently from 0a3cfe1 to 298db29 Compare January 22, 2025 20:39
Add a minimal integration of bindgen to the build.

This required some refactoring of the C so app-layer-events.h did not
also include "rust.h", which causes issues for bindgen, probably
related to circular references.

AppLayerEventType was chosen as the first step as its an argument type
some app-layer functions that we may want to use bindgen to export
Rust, and one of the requirements of bindgen might be that C functions
should only use datatypes defined in C, and not Rust. Following such a
rule also prevents circular dependencies between Rust and C code.

As bindgen can only accept one header file, we construct a
pseudo-header file of all the headers that need to be
consumed. Makefile dependency checking is done to make sure the
pseudo-header is only generated as needed to avoid rebuilding every
time.

Special handling is required for Windows to use the Windows path.

Ticket: OISF#7341

bindgen: pre-generate rust bindings to C

This simplifies building, as we don't have to worry about path and
such under autoconf/automake. It does however mean when we update C
headers that are exposed to Rust, we manually have to re-generate the
bindings, but this is a check we can do in CI.

It also eliminates the need for everyone who wants to build Suricata
to have bindgen and clang tools installed.
This lets us remove decode.h from app-layer-events.h as pulling in
app-layer-events.h shouldn't result in pulling in dpdk, and other
includes not related to app-layer-events.

decode.h also doesn't need those forward declarations anymore due to
previous changes.
Instead of defining this function pointer in type in Rust, and having
it in C signatures, create a type and export it to Rust.

To facilitate this, and new header has been creates,
"app-layer-types.h", this is to avoid the circular reference of C
headers pulling in "rust.h" which are required to generate Rust
bindings.
This exposes the C define ALPROTO values to Rust without having to
perform some runtime initialization with init_ffi.

As app-layer-protos.h was clean of a circular reference to rust.h we
could use it directly, it just needed the addition of
suricata-common.h.
This required us to remove the auto-generated by header as it can
change depending on what version is used.

I suppose different versions could generate slightly different output
that would cause this error out. In which case we may want to use a
custom script that can be a bit smarter output, or simply error out if
any of the headers used in the bindings is newer than the generated
_sys.rs file.
If we have bindgen, and we're building in-tree, rebuild the bindings
and compare to the previous bindings and print a warning if there is a
difference.

Only a warning for now, as I'm not sure where this might fail, and I
don't want it to become a nuisance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant