Ruby language bindings and technical questions

[thedavemarshall]

First, this is an awesome project! Thank you so much for sharing it with the world!

I am familiar with REGO, and I am interested in using https://github.com/matsadler/magnus to create a ruby gem to wrap the rust APIs. I see you've already done impressive work with structuring an FFI entrypoint and methods. I am new to rust, but I have extensive ruby experience, and I love learning. I'd love to help out with your efforts, could you please entertain a few questions for me?

• first, how would you feel about ruby bindings for regorus? I'm offering to support it myself if your capacity is an issue, but I wanted to run it by you first
• i noticed the other language bindings are not exported to package managers, but include instructions for developers to build the bindings themselves. Is this the intended approach for the forseeable future? Would you have a preference between if I published the ruby bindings as a separate gem (in a separate repository), or contributed a pull request to add ruby bindings to your existing repository? I'd be happy to sign any contributor or licensing agreements

at the technical level the FFI supports a lot of excellent functionality, but I had a few ideas I'd love to get your thoughts on

• could regorus::engine::eval_rule be supported in addition to the eval_query? this might have performance benefits in certain scenarios, I'd be happy to work with you on implementing support across the existing language bindings
• could there be a way to list the modules/policies currently loaded in the engine? either exposing parts of the AST, or maybe maintain a list of "path" strings from policies as they are loaded? This would help with testing, as I'm trying to iron out some of the memory and concurrency issues with Ruby<>Rust, and I wanted to check that I implemented clone() correctly. Again, I'd love to help out with supporting this in for language bindings as well
• I noticed the ast, lexer, parser are private and marked unstable. Is there criteria for when they would be considered stable, and can I help with those efforts?

Thank you for your time, and thanks again for such a cool project!

[anakrish]

@thedavemarshall Thanks for your interest in the project, and initiative for a Ruby binding.

first, how would you feel about ruby bindings for regorus? I'm offering to support it myself if your capacity is an issue, but I wanted to run it by you first

Ruby binding would be great. My focus for the near term is to make Regorus even more performant for certain use cases. I don't have much experience with Ruby either. Given your extensive experience with Ruby, it would be excellent if you could take ownership of the Ruby binding.

Would it be possible to share your usecase for a Ruby binding?

Is this the intended approach for the forseeable future?

See also #168. Publishing packages through official Microsoft pipelines takes quite a bit of effort. There is a chance that more dev resources are assigned to Regorus over the next few months. If that happens, we'd start publishing the packages to various repositories for use by the package managers

Would you have a preference between if I published the ruby bindings as a separate gem (in a separate repository), or contributed a pull request to add ruby bindings to your existing repository? I'd be happy to sign any contributor or licensing agreements.

My preference would be the latter. Mainly for the two reasons:

• Keeping bindings current with the Rust lib interface
• Ability to publish from Microsoft pipelines in the future.

However, the former approach can also work. The published gem needs to state that it is unofficial in the sense that it is not endorsed or published by Microsoft, given the prevalence and severity of supply chain attacks.

[anakrish]

could regorus::engine::eval_rule be supported in addition to the eval_query? this might have performance benefits in certain scenarios, I'd be happy to work with you on implementing support across the existing language bindings

That would be awesome! We have a related issue of keeping all the bindings up to date: #160

could there be a way to list the modules/policies currently loaded in the engine? either exposing parts of the AST, or maybe maintain a list of "path" strings from policies as they are loaded?

For now, I'd prefer exposing functions in Engine or in a new type CompiledPolicy (or some other name) to provide the needed information. I've been thinking of formalizing the notion of compiled policies (see #113 (comment)). We can share them efficiently between threads without any locks involved.

I noticed the ast, lexer, parser are private and marked unstable. Is there criteria for when they would be considered stable, and can I help with those efforts?

The reason they are marked unstable is that we want to try out a few approaches that can improve performance.

• Store Value directly for nodes like Var, Number etc instead of creating them on the fly each time during eval.
• Hoist loops and create nodes in the AST. We currently hoist loops on the fly each time a statement is executed.
• Add output expressions to the end of each query

I tried out some of these ideas in an F* reference implementation I'm writing (https://github.com/anakrish/Rego-FStar/blob/fc5e9de088ddefd523ce58e2fff05a1019f54b0c/Rego.Ast.fsti#L53).

There are likely other changes we could do to make evaluation faster. Hence my hesitation in exposing AST for now.

Having said that, I'd like to understand the use cases that would be served better by exposing the AST.

[anakrish]

@thedavemarshall It would be really cool if we could expose the Engine::add_extension capability in various bindings.
The extension method written in languages like Python, Ruby etc could either receive the arguments as JSON or as instantiations of types native to the language.

[thedavemarshall]

@anakrish awesome, thanks so much for the encouraging feedback! Here's a rough attempt at Ruby bindings #188

I am interested in a couple different use cases, the most simple is evaluating access controls and permissions as REGO policies from a Ruby on Rails web application. A more advanced use case I'm exploring is having a web application for editing REGO policies by manipulating the AST or some intermediate representation, allowing more intuitive graphical interfaces for users than writing raw REGO.

Ah, exposing Engine::add_extension would be an interesting feature! I might start with #160 and bindings for some of the easier functionality and build my way up to that lol.