Debug MacOS

[CyberKenneth]

MacOS Debug System Enhancements

Overview

Hello! I'm a computer engineering student working on my first significant open-source contribution. This pull request introduces an enhanced debug system for MacOS, specifically targeting version 10.6 and newer platforms.

A Student's Journey

🚧 Buyer Beware: This implementation is based on tutorials, documentation, and open-source resources. While I've done my best to create a robust solution, I acknowledge that comprehensive testing on real Mac hardware is crucial.

Motivation

After studying MacOS development tutorials and resources focused on version 10.6, I wanted to contribute to improving the debug infrastructure. The goal was to create a more flexible, informative logging system that could help developers track and diagnose issues more effectively.

Key Improvements

• Enhanced thread-safe logging mechanisms
• Configurable debug output with rich formatting
• Memory debugging tools with version-aware compatibility
• Flexible configuration through environment variables and programmatic controls

Learning Highlights

• Navigated platform-specific challenges in MacOS debugging
• Implemented thread safety and concurrency considerations
• Explored MacOS availability macros and version compatibility checks

Potential Limitations

• Developed without direct access to Mac hardware
• Primarily based on documentation for MacOS 10.6
• May require further refinement and testing on various MacOS versions

Enhancements from Original Implementation

Core Improvements

• 📝 Enhanced DebugPuts() functionality
• 🔍 Added comprehensive debug logging levels
• 🧵 Implemented advanced thread-safe logging mechanisms
• 🛠️ Integrated MacOS-specific memory debugging tools

Specific Enhancements

1. Debug Output Expansion

• Added multiple debug logging functions:
  • DebugTrace()
  • DebugVerbose()
  • DebugOutput()
  • DebugNotice()
  • DebugInfo()
  • DebugWarning()
  • DebugError()
  • DebugFatal()

2. Thread Safety Improvements

• Created thread-local logging tracking
• Developed optimized locking strategies
• Added thread mode configurations (Unsafe, Safe, Identify)

3. Configuration Capabilities

• Enabled environment variable-based debug configuration
• Added programmatic debug level and format control
• Implemented runtime debug settings modifications

4. MacOS-Specific Enhancements

• Added version-aware memory debugging tool support
• Integrated Objective-C runtime logging
• Implemented platform-specific compatibility checks

5. Logging Flexibility

• Introduced configurable output formatting
• Added timestamp and context tracking
• Created emoji-based log level indicators

Invitation to Collaborate

As a student contributor, I'm eager for feedback, suggestions, and guidance from more experienced developers. Your insights will be invaluable in improving this implementation.

Testing Appreciated

I encourage the community to:

• Review the code carefully
• Test on different MacOS versions
• Provide constructive feedback

Thank you for the opportunity to contribute to this project! 🙌

[CyberKenneth]

issue arose from a circular dependencies between header files. The Debug.h file includes platform-specific headers (like MacOSDebug.h), but the platform-specific implementation needs type definitions that should be accessible to all debug components.
The solution is to extract shared type definitions into a separate header file that has no dependencies of its own, creating a clean hierarchy:

DebugTypes.h (no dependencies) // add this maybe if we ant to do the system here something like

// DebugTypes.h
#ifndef LLGL_DEBUG_TYPES_H
#define LLGL_DEBUG_TYPES_H

namespace LLGL
{
    enum class DebugLevel
    {
        Trace,
        Verbose,
        Debug,
        Notice,
        Info,
        Warning,
        Error,
        Fatal,
        Silent
    };

    enum class DebugThreadMode
    {
        Unsafe,
        Safe,
        Identify
    };

    enum class DebugFormatFlags
    {
        Default = 0,
        IncludeTimestamp = 1 << 0,
        IncludeFunction = 1 << 1,
        IncludeFile = 1 << 2,
        IncludeLine = 1 << 3
    };

    // Enable bitwise operations for DebugFormatFlags
    inline DebugFormatFlags operator|(DebugFormatFlags a, DebugFormatFlags b)
    {
        return static_cast<DebugFormatFlags>(
            static_cast<int>(a) | static_cast<int>(b)
        );
    }

    inline DebugFormatFlags operator&(DebugFormatFlags a, DebugFormatFlags b)
    {
        return static_cast<DebugFormatFlags>(
            static_cast<int>(a) & static_cast<int>(b)
        );
    }
}

#endif // LLGL_DEBUG_TYPES_H

this would add a need for Debug.h to depend on DebugTypes.h
Platform-specific headers (depend on DebugTypes.h)
Platform-specific implementations (depend on their headers) this is my understanding

[LukasBanana]

Hello again and thank you for your first contribution.

I left several comments in your code. Most of these changes can be platform agnostic, they don't have to be limited to MacOS. I also think you should make use of the existing logging system and move the more advanced formatting such as timestamps and verbosity settings to the utility interface since those are not required for LLGL to work and you don't even use them in your PR, you only provide new functionality.

This implementation is based on tutorials, documentation, and open-source resources. While I've done my best to create a robust solution, I acknowledge that comprehensive testing on real Mac hardware is crucial.

I haven't tested your code on my MacBook yet, but I'd also hope that you are able to test your own code before filing a PR unless it's some obvious bug fixes. You are essentially leaving the responsibility to validate the correctness of your code to someone else.

If you don't have a Mac to test this, maybe start with another platform or some platform independent code changes.

Best regards
Laura

[LukasBanana]

This and all the other Debug... functions have the same interface. This adds an unnecessary amount of new functions to the interface, even though it's not in the public interface, it is still exposed to other backends. So this should be boiled down to only one function that takes your new enum as first parameter. You can still keep those functions separate in the .mm file, but the interface should be thinner.

[CyberKenneth]

I appreciate the feedback about the interface, but I want to clarify my intent. The different debug functions like DebugTrace(), DebugWarning(), etc., were designed to provide semantic clarity for developers. Each function represents a specific logging context that helps developers quickly understand the nature of the log message.

However, I understand the concern about interface complexity. My goal was to create expressive logging that helps developers quickly identify the context of a log message. The multiple functions allow for more readable code like:

🔧 [D] Loading graphics driver: NVIDIA GeForce RTX 3080
📝 [N] Detected 4K display resolution: 3840x2160
ℹ️ [I] Initializing render context
🔍 [V] Allocating 2GB video memory
🔬 [T] [ShaderCompiler::Compile] Compiling vertex shader
⚠️ [W] Shader optimization level set to default
🔧 [D] [TextureManager::Load] Loading texture: landscape_background.dds
📝 [N] Texture compression detected: BC7
ℹ️ [I] [RenderQueue::Prepare] Preparing render command buffer
🔍 [V] Sorting render passes
❌ [E] Vertex buffer allocation failed
💥 [F] Critical shader compilation error
🔬 [T] [RenderSystem::Render] Attempting fallback rendering mode
⚠️ [W] Performance degradation expected
📝 [N] Reduced shader complexity
ℹ️ [I] Rendering first frame
🔧 [D] Frame submission to GPU
❌ [E] Memory mapping error detected
💥 [F] Unrecoverable rendering state 

log end here 

ps
however now i know there is util area and I also dont have a good way of testing I probably would just delete this all. 

[LukasBanana]

I guess using emojis is a matter of preference, there might have been a time when I would have jumped on that idea, but I backed a bit away from adding too many colorful debug outputs, so I kept it with a few color options with text. I still mostly reject this idea due to the associated risks I mentioned.

[LukasBanana]

Is this all of your own documentation or it copied from some somewhere else? It looks like it's part of that header itself.

[CyberKenneth]

The documentation here was cited because I learned from it and was worried I may have used some code in a partially adapted state from existing Apple headers(I just had finished reading it and had it in the background), but I should have been clearer about its origin and why I had it mainly was the worry no one knew about it. I'll ensure any borrowed documentation is properly attributed or rewritten to be original.

I can remove the header here now that it was seen and figure out a better way that explain the purpose of the code and header?

[LukasBanana]

Yeah I don't think this header needs that amount of explanation.

[LukasBanana]

For flag enumerations, please use the same semantic as in the LLGL interface, i.e. a struct with only one anonymous enum such as CommandBufferFlags for instance. The rationale behind it is to have the same semantic as for scoped enums to reduce cluttering the global scope, i.e. you write the flags as DebugFormatFlags::Default instead of just Default.

[CyberKenneth]

Thanks for the feedback on the flag enumerations. As an embedded system and low-level programmer, I was raised by my uncle to primarily work in C and assembly language where the :: scope resolution operator isn't used. I am far newer to C++ than asm or C so it slipped my mind that's why I didn't initially use the CommandBufferFlags::flags syntax - it's just not a pattern I'm used to from my C programming background.
I appreciate you explaining the rationale behind using structs with anonymous enums in the LLGL interface to keep things consistent with the scoped enum semantics in C++. That makes a lot of sense to avoid cluttering the global namespace.
Moving forward, I'll work to keep this convention in mind and use the EnumName::ValueName syntax for flag enumerations in this project to maintain consistency and readable code. Thanks again for pointing this out and helping me better understand the preferred style here. I'm always happy to learn and adapt. Please let me know if you spot any other areas where I can improve my contributions!

[LukasBanana]

I originally started this project with the intent to keep this strictly C++, that's why I use std::uint32_t everywhere, but I slowly started going back to having plain C structs/types etc. in the code. So there are also some C-style enums in the code (you can find some here in D3D11BindingLocator.h:33 and GLResourceHeap.cpp:67 for instance) but those are not in the public interface and usually connected to low-level code that I like to call "bit-level hacking". Anything in the public interface of LLGL should make use of scoped enums or struct/enum combos for flags.

[LukasBanana]

This is a large number of debug levels and I think it goes a bit beyond the scope of LLGL. I personally see this level of fine-tuned logging mechanism on the application level but not in a low level library that acts as interface with the graphics APIs. There are currently two enums for error logging: LLGL::Log::ReportType and LLGL::ErrorType. The former one only has two entries, Default (for stdout output stream) and Error (for stderr output stream) to keep it simple. The latter one is solely for the RenderingDebugger.

It can't hurt to add more debugging functionality to LLGL, but I think a more extensive logging output with timestamps, verbosity setting and more should be kept to the application. If you still think this is something you want to see LLGL to provide out of the box, I suggest we move these features to the utility functions. Anything that is not mandatory for LLGL's core functionality is kept in the LLGL/Utils/ folder. You could add most of these features as an extension to the public LLGL::Log namespace, for example with a function that registers a more advanced logging callback, something similar to LLGL::RegisterCallbackStd() which registers a callback that forwards all log output to either stdout or stderr output streams.

[CyberKenneth]

That makes sense, I get the importance of keeping LLGL lightweight for application developers, but as contributors, having fine-grained debugging tools directly within LLGL would be a huge advantage. This may just be the bare metal nerd in me but being able to "pick at each grain of sand on the beach to see the bugs" with advanced logging and debugging features would make diagnosing issues much easier, especially if we use them in tests.

I like the idea of moving advanced logging to LLGL/Utils/ for optional use and keeping the original debug tools they way they are.
This way:

App developers can extend our logging system if they find it useful.
Contributors get powerful built-in debug tools without cluttering the core library.
Tests benefit from deeper logging insights without requiring external setups.
And who knows, some application developers might find our extended tools useful and either extend them further or integrate them into their debugging workflows. Keeping things structured this way lets LLGL stay clean and minimal while offering flexibility and power where needed.

Does this structure sound like a good balance between keeping LLGL lightweight and utility?

[LukasBanana]

As I mentioned, I'm not opposed to adding more debugging functionality to this project. I just think most of this doesn't have to be MacOS specific. Maybe start with writing your own LLGL::Log callback with more detailed output.

[LukasBanana]

I can't tell just by looking at it why the CIS builds failed, but they show a lot of compile errors, maybe due to the anonymous namespace which I usually don't use:

/Users/runner/work/LLGL/LLGL/sources/Platform/MacOS/MacOSDebug.mm:90:9: error: unknown type name 'DebugLevel'
        DebugLevel minLevel = DebugLevel::Info;
        ^

[CyberKenneth]

yea sorry again about this.

[LukasBanana]

No worries, that's what the CI builds are for. I mess this up often enough myself :)

[LukasBanana]

Not sure what performance impact it has to lock a default-initialized mutex, but it looks wrong to me locking and unlocking a mutex if the intention is to ignore thread safety. I think we should either have these mechanisms thread safe in general or not at all. If you want to make thread-safety configurable, I would maybe include/exclude them with a preprocessor macro that can be enabled/disabled via CMake, but think about why this should be thread safe in the first place. The current logging functions are always guarded with a mutex (see Log.cpp:106) and also a recursion lock (see Log.cpp:117) to avoid calling Log::Printf() inside the logging callbacks.

[CyberKenneth]

My goal was along those lines, but I did it dead tired where a preprocessor macro would set log-level thread safety and the like. I hadn't seen the LLGL/sources/Core
/Log.cpp because I hyper-focused on #129 ref "MacOS with the Metal renderer, memory leaks every frame.
In 3 minutes, the memory usage of a simple app reached 6 GB." one of my first thoughts was if I could write a workflow to probe the program with the Mac debugger.
that said I am open for your input across everything I posted tonight or i can delete if you want too..

[LukasBanana]

I hyper-focused on #129

I really appreciate someone trying to improve that situation, because I haven't been able to do much about it so far, but I think you would need a Mac to actually test your own code. Otherwise, this is a bit of a shot in the dark.

or i can delete if you want too..

As I said in a comment above, maybe try with a small extension of your own Log callback first. Maybe you find some other issues along the line you can fix and get to know the interface and architecture of LLGL a bit better along the way :)

[LukasBanana]

Please don't use Unicode characters in LLGL. Maybe I'm old-fashion in that regards, I'm aware some programming languages and even older ones like Java support Unicode characters not only in string literals but even in standard identifiers, but I don't see much value in that.

• How do you search for those string literals in the code base?
• Does this guarantee to work with older compilers or do they have to be configured differently to consume UTF-8 or Multibyte encoded source files?
• I remember when a very special Unicode character caused iPhones to freeze, so there is always a vulnerability concern regarding Unicode. I remember that because my phone didn't work that day when I was relying on its GPS navigation :-)

If you need those characters, please be so good to encode them with the old C++ escape sequence "\u....".

[CyberKenneth]

I see what you mean my idea was for you to be able to see in a command line the emoji and if it is what type of log you were looking for you could grab it even if it was scrolling fast. but I see how it could be a issue.

• remember when a very special Unicode character caused iPhones to freeze, so there is always a vulnerability concern regarding Unicode. I remember that because my phone didn't work that day when I was relying on its GPS navigation :-)

iOS 17 and a few others "::" was one and Unicode char from the Telugu language was another. Springboard seems to hate Unicode and stuff.

[LukasBanana]

I see you're familiar with that debacle 😄 It sure were fun times.

[CyberKenneth]

Yeah. diff note, I dont know how to revoke these propoged changes so we start at square one on this but pre-PR.