System.ArgumentException: Number of observations can't be less than 1, despite multiple observations defined

[Nhyv]

Hi there,

I found out about RL Matrix from the youtube video https://youtu.be/Aoyj7qzyung as there aren't very many real world examples out there of this library in use (and the examples in the README seem to be out of date).

I'm trying to create an environment using the Source Gen (RLMatrix.Toolkit) and seem to be coming across an issue that I'm unsure how to resolve. I have defined multiple observation methods adorned with [RLMatrixObservation] and yet, when running the code, I get this exception

Unhandled exception. System.ArgumentException: Number of observations can't be less than 1
   at RLMatrix.DQN1D..ctor(String name, Int32 obsSize, Int32 width, Int32[] actionSizes, Int32 depth, Boolean noisyLayers, Single noiseScale)
   at RLMatrix.DQNNetProvider`1.CreateCriticNet(EnvSizeDTO`1 env, Boolean noisyLayers, Single noiseScale, Int32 numAtoms)
   at RLMatrix.DiscreteQAgentFactory`1.ComposeQAgent(DQNAgentOptions options, Int32[] ActionSizes, OneOf`2 StateSizes, IDQNNetProvider`1 netProvider, LRScheduler lrScheduler, IGAIL`1 gail)
   at RLMatrix.LocalDiscreteQAgent`1..ctor(DQNAgentOptions opts, Int32[] actionSizes, OneOf`2 stateSizes, IDiscreteQAgentFactory`1 agentComposer)
   at RLMatrix.Agents.Common.LocalDiscreteRolloutAgent`1..ctor(DQNAgentOptions options, IEnumerable`1 environments)
   at Program.<Main>$(String[] args) in E:\EaRL\Program.cs:line 21
   at Program.<Main>(String[] args)

Here is my full environment class below:
https://gist.github.com/Nhyv/cb81e0e51a2363953e0f7519df82c843

I am new to RL Matrix and machine learning in general so if I am missing anything obvious, please let me know. Thank you!

[asieradzk]

Hi @Nhyv thanks so much for trying out RLMatrix.

The examples in the main repo seem to be outdated, please refer to this one for more recent cases (I routinely use it for testing before I publish nuget packages)

https://github.com/asieradzk/CartPoleForTesting

[asieradzk]

First thing I notice is that your observations are integers, these should be floats as they get converted to 1d or 2d float array.

Thank you for demonstrating an interesting edge case! I will add more descriptive error message to toolkit when I work on next build.

Consider dropping by discord to get quicker feedback loop on your work.