Backend as a separate process (sandboxing)

[athairus]

With no way to have multiple Libretro cores at once in a single address space and the fact the Libretro cores are free to do all kinds of bad things that can tank the program that we don't have much control over (calling abort(), accessing null pointers, etc.) it might be worth looking into making the backend a separate process. From there we can more or less act as a debugger to this child process. We can catch program-terminating signals and possibly read process memory directly to recover save data if we know the process is in an inconsistent state.

Qt has an API for pipes: http://doc.qt.io/qt-5/qlocalsocket.html

The interface between Phoenix and the external process will probably be input and control signals to the process and audio/video data from. This might get hairy with OpenGL as we'll end up copying completed frames from VRAM to RAM (only to end up back in VRAM when composited into the QML scene).

One thing that is a really nice advantage of this approach is we can launch as many of these instances as we want. The live preview idea lives on!

This shouldn't mean much to GStreamer other than adding a bit more complexity. GStreamer already has the ability to have multiple pipelines (TODO: make sure that's actually true). The GStreamer plugins aren't considered "external code" because we'll be writing them.

[athairus]

I've decided on a few tweaks:

• Audio will be handled by the sandboxed process directly, no need to pass it to the frontend for processing.
• Video:
  • OpenGL commands from 3D cores should be converted into some intermediate format along with whatever data their arguments may point to for efficient transmission across some IPC channel. This is the approach Chrome uses. For more details, look here in the section "Architectural Interlude: The GPU Process". We'll look into automated ways of generating the necessary code for each OpenGL revision Libretro supports
  • For 2D cores, we'll assume that the generated video will be small enough that passing entire frames via IPC won't be a huge deal.
• When the core supports it (when it makes us deal with the save data buffer), the save data buffer will be shared across processes so we can easily grab it when the core tanks

[athairus]

IPC libraries: https://github.com/fffaraz/awesome-cpp#inter-process-communication
IPC benchmarks: https://github.com/rigtorp/ipc-bench