Increased memory usage

[quasiquote1]

Hi,

We recently updated to the latest sparta master and noticed that our memory usage has increased just due to this change (for example sims that were previously taking 4GB now take 4.5GB). Everything else in our sims have remained the same (same memory pool sizes)

Do you guys know what might be accounting for this increase? Anything we can do to reduce it? We suspect it may be coming from the sparta scheduler but not sure how to narrow this down.

Thanks

[klingaard]

And you didn't change the compiler?

[klingaard]

Can you tell me from which versions of Sparta you moved from/to?

[quasiquote1]

We moved from b1cec4d75b7 to 64e119f0321

[klingaard]

I don't see anything that stands out as a memory hog, but I have to admit I'm really surprised your simulator is using so much memory. Our simulator might have crept to 1GB at one time, but I saw averages around 500MB.

Have you tried using valgrind memory tools to see where memory is being occupied? If in Sparta, let's fix that.

[quasiquote1]

We are currently heavily users of the preload functionality for Sparta Unit components, and our understanding is that the Yaml reader doesn't support any streaming functionality and instead buffers the input file in memory leading to the majority of the 4GB memory usage.

If you know of any streaming Yaml parser support please let us know and we can try it out.

[klingaard]

I don't have experience with event-based YAML parsers, but I'm bettin' they're out there. Wonder if it makes sense for you to move away from YAML preloading and something home-grown. An XML parser might be better -- their are many SAX-based parsers out there...

Are you using the PreloaderIF for cache/memory preloading? I guess it doesn't really matter what you're using it for; it looks like the interface uses YAML as a DOM, which would cause memory to explode on loading. But it looks like the YAML parser is destroyed after reading it. Curious how your model stays at 4.5GB...