gpu_hw_opengl: route batch programs through the same three-mode precompile machinery as D3D11/D3D12/Vulkan

This is the final backend in the swanstation_GPU_ShaderPrecompile
rollout. OpenGL is structurally different from the other three:
the libretro hardware-renderer protocol gives the core a single
GL context bound to the runloop thread, with no way to spin up a
worker thread that shares the context (no second context, no
glShareLists hand-off path). So OpenGL's 'Lazy' mode degrades to
the same shape as 'Disabled' - compile each batch program on the
runloop the first time the game dispatches a draw using it, then
cache it. No worker thread, no mutex, no atomic; everything runs
on one thread.

Adds GetBatchProgram(render_mode, texture_mode, dithering,
interlacing) as the single fault-in path, mirroring the
GetBatchPixelShader / GetBatchPipeline helpers on the other
backends. The matrix is the existing m_render_programs (array of
GL::Program, move-only); slots start as default-constructed
(m_program_id = 0) and GetBatchProgram fills them lazily.

The Reserved_*Direct16Bit dedup is handled differently from the
other three backends - GL::Program is move-only and owns its
GLuint via glDeleteProgram in its destructor, so we can't share
a single handle between two array slots without a double-delete
hazard. Instead, the helper feeds the canonical (lookup_mode)
fragment-shader source string into GetProgram for both slots.
The disk-backed program cache hits on the second call and
reloads the linked binary instead of recompiling, so the
duplicate is cheap (one glProgramBinary call rather than a full
GLSL compile + link).

Mode behaviour:
  - Enabled: synchronous walk of the matrix at CompilePrograms
    time, calling GetBatchProgram on each cell.
  - Lazy / Disabled: matrix stays default-constructed at
    CompilePrograms time; each slot is filled by DrawBatchVertices
    -> GetBatchProgram on first use of that combination.

Two small infrastructure additions to support the lazy path:

  - GL::Program::IsValid() accessor ('return m_program_id != 0').
    GetBatchProgram uses it to distinguish a never-touched slot
    from a compiled one. glUseProgram(0) un-binds; calling
    prog->Bind() on a slot with id 0 would silently un-bind the
    previous program and break rendering, so the explicit
    validity check is required.
  - GL::ShaderCache::IsOpen() accessor, mirroring D3D11 / D3D12
    parity. The shader cache becomes a member (m_shader_cache)
    so the lazy helper can reach it at draw time on
    UpdateSettings round-trips; without IsOpen the second Open()
    call would re-read the index and double-count entries.

GPU_HW_ShaderGen becomes a member (m_shadergen, unique_ptr since
the class is non-default-constructible) so GetBatchProgram can
generate batch shader source at draw time.
use_binding_layout is captured into m_use_binding_layout because
GetBatchProgram's link-callback needs it.

CompilePrograms now explicitly resets the m_render_programs
matrix at function entry. Without that, an UpdateSettings round-
trip (which calls CompilePrograms with new texture_filter /
scale / etc. without an explicit DestroyPrograms step) would
leave the previous programs in place; the lazy fault-in path's
IsValid() short-circuit would return them as if they were fresh,
and the game would render through programs built for the old
settings. The synchronous 'Enabled' precompile loop has the same
short-circuit so it needs the same guarantee. The reset is a
default-assign loop which invokes GL::Program's move-assign
operator, which calls Destroy() (i.e. glDeleteProgram) on each
slot before resetting to id = 0.

DrawBatchVertices routes its m_render_programs lookup through
GetBatchProgram so all three modes share the same fault-in path.

The OpenGL backend has historically had the longest cold-compile
stall in some configurations (long GLSL -> driver compile
pipeline on some Intel / Mesa stacks, no async API in core
OpenGL until ARB_parallel_shader_compile which not all drivers
implement). With 'Lazy' as the default mode, that stall moves
from CompilePrograms-on-init / CompilePrograms-on-settings-change
to a stutter on the first draw that touches each combination -
one per cell, capped at ~144 cells total over the session, often
only a handful in practice for most games. The trade-off is
preferable because it makes the UI feel responsive (RetroArch
keeps drawing frames while gameplay progresses) rather than
froze for the whole compile.

This commit completes the four-backend rollout. All of D3D11,
D3D12, Vulkan, and OpenGL now honour
swanstation_GPU_ShaderPrecompile = Lazy / Enabled / Disabled.

Build verification: linux platform=unix (which DOES compile
gpu_hw_opengl.cpp, unlike the D3D11/D3D12 cases): zero warnings,
zero errors. .so size 8040104 (+4240 vs commit a29d202's
8035864). The growth is the GetBatchProgram body, the matrix-
reset loop at CompilePrograms entry, and the persistent member
storage for the shader cache and shadergen. 25 retro_* exports
unchanged.

Files touched:
  - src/common/gl/program.h: IsValid() accessor.
  - src/common/gl/shader_cache.h: IsOpen() accessor.
  - src/core/gpu_hw_opengl.h: includes, m_shader_cache,
    m_shadergen, m_use_binding_layout members, GetBatchProgram
    declaration.
  - src/core/gpu_hw_opengl.cpp: CompilePrograms resets the matrix
    at entry, uses persistent member shader_cache / shadergen,
    three-way precompile branch; GetBatchProgram helper added;
    DrawBatchVertices routes through the helper.
This commit is contained in:
LibretroAdmin 2026-05-11 17:18:35 +00:00 • committed by U-DESKTOP-SPFP6AQ\twistedtechre
commit 5c9cf3ea9f
4 changed files with 211 additions and 85 deletions

View file

@ -34,6 +34,12 @@ public:
void Bind() const;
// Returns true if the program has a non-zero linked GL handle.
// Used by the lazy-compile path in GPU_HW_OpenGL to distinguish a
// never-touched matrix slot (program id 0) from one that has
// already been compiled and is ready for Bind().
bool IsValid() const { return m_program_id != 0; }
void Destroy();
int RegisterUniform(const char* name);

View file

@ -23,6 +23,14 @@ public:
void Open(bool is_gles, std::string_view base_path, u32 version);
// Returns whether Open() has already been called successfully on
// this instance. Used by the lazy-compile path in GPU_HW_OpenGL
// to avoid re-reading the same on-disk index on UpdateSettings
// round-trips through CompilePrograms, which would both leak
// file handles and double-count m_index entries. Mirrors the
// equivalent accessor on D3D11::ShaderCache and D3D12::ShaderCache.
bool IsOpen() const { return m_index_file != nullptr; }
std::optional<Program> GetProgram(const std::string_view vertex_shader, const std::string_view geometry_shader,
const std::string_view fragment_shader, const PreLinkCallback& callback = {});

View file

@ -1068,97 +1068,88 @@ bool GPU_HW_OpenGL::CreateTextureBuffer()
bool GPU_HW_OpenGL::CompilePrograms()
{
GL::ShaderCache shader_cache;
shader_cache.Open(IsGLES(), g_host_interface->GetShaderCacheBasePath(), SHADER_CACHE_VERSION);
// Reset every program object so the new compile starts from a
// clean slate. Without this, on an UpdateSettings round-trip
// through CompilePrograms (in Lazy / Disabled mode where the
// matrix isn't refilled here) the lazy fault-in path in
// GetBatchProgram would observe programs left over from the
// previous settings - built with the old texture filter / scale
// / etc. - and use them for the new draws. The synchronous
// 'Enabled' loop further down also relies on this: it calls
// GetBatchProgram which short-circuits on IsValid(), so a
// pre-populated slot would silently keep its old program.
//
// GL::Program's move-assign operator calls Destroy() on the
// destination before taking the source's state, so default-
// assigning each slot to a temporary {} both glDeleteProgram's
// the old handle and rewinds the slot to id = 0.
for (auto& a : m_render_programs)
for (auto& b : a)
for (auto& c : b)
for (auto& slot : c)
slot = GL::Program{};
const bool use_binding_layout = GPU_HW_ShaderGen::UseGLSLBindingLayout();
GPU_HW_ShaderGen shadergen(m_host_display->GetRenderAPI(), m_resolution_scale, m_multisamples, m_per_sample_shading,
m_true_color, m_scaled_dithering, m_texture_filtering, m_using_uv_limits,
m_pgxp_depth_buffer, m_disable_color_perspective, m_supports_dual_source_blend);
ShaderCompileProgressTracker progress("Compiling Programs", (4 * 9 * 2 * 2) + (2 * 3) + (2 * 2) + 1 + 1 + 1 + 1 + 1);
for (u32 render_mode = 0; render_mode < 4; render_mode++)
// Open the disk-backed program cache once. On subsequent calls
// (UpdateSettings -> CompilePrograms) the instance still holds
// the index from last time and the disk file hasn't moved, so
// re-opening would double-count m_index and leak file handles.
if (!m_shader_cache.IsOpen())
{
for (u32 texture_mode = 0; texture_mode < 9; texture_mode++)
m_shader_cache.Open(IsGLES(), g_host_interface->GetShaderCacheBasePath(), SHADER_CACHE_VERSION);
}
GL::ShaderCache& shader_cache = m_shader_cache;
m_use_binding_layout = GPU_HW_ShaderGen::UseGLSLBindingLayout();
const bool use_binding_layout = m_use_binding_layout;
m_shadergen = std::make_unique<GPU_HW_ShaderGen>(
m_host_display->GetRenderAPI(), m_resolution_scale, m_multisamples, m_per_sample_shading, m_true_color,
m_scaled_dithering, m_texture_filtering, m_using_uv_limits, m_pgxp_depth_buffer, m_disable_color_perspective,
m_supports_dual_source_blend);
GPU_HW_ShaderGen& shadergen = *m_shadergen;
// OpenGL is the odd one out: the libretro hardware-renderer
// protocol exposes only a single GL context bound to the
// runloop, so there's no way to spin up a worker that compiles
// in the background like D3D11/D3D12/Vulkan do. 'Lazy' therefore
// degrades to the same shape as 'Disabled': skip the matrix at
// CompilePrograms time, fault each combo in on the runloop the
// first time the game dispatches a draw using it.
//
// 'Enabled' still does the full synchronous walk via the
// GetBatchProgram helper, mirroring the D3D11/D3D12/Vulkan
// commits.
const GPUShaderPrecompileMode precompile_mode = g_settings.gpu_shader_precompile_mode;
const bool precompile_sync = (precompile_mode == GPUShaderPrecompileMode::Enabled);
const u32 batch_progress_units =
(precompile_mode == GPUShaderPrecompileMode::Enabled) ? static_cast<u32>(4 * 9 * 2 * 2) : 0u;
ShaderCompileProgressTracker progress("Compiling Programs",
batch_progress_units + (2 * 3) + (2 * 2) + 1 + 1 + 1 + 1 + 1);
if (precompile_sync)
{
for (u8 render_mode = 0; render_mode < 4; render_mode++)
{
// Reserved_Direct16Bit (3) and Reserved_RawDirect16Bit (7) produce
// shaders identical to Direct16Bit (2) and RawDirect16Bit (6) - see
// GPU_HW_D3D11::CompileShaders for the full explanation. For
// OpenGL we can't simply copy the resulting GL::Program object
// because it's move-only (it owns a GLuint program ID that the
// destructor will glDeleteProgram), so we have to re-link the
// program. We can, however, skip the ~50ms shader-source
// regeneration in GenerateBatchFragmentShader by re-using the
// already-built source string from the previous (source_mode)
// iteration. The shader cache will hit on the second GetProgram
// call and reload the binary instead of recompiling.
const u32 source_mode = (texture_mode == static_cast<u32>(GPUTextureMode::Reserved_Direct16Bit)) ? 2u :
(texture_mode == static_cast<u32>(GPUTextureMode::Reserved_RawDirect16Bit)) ? 6u :
texture_mode;
for (u8 dithering = 0; dithering < 2; dithering++)
for (u8 texture_mode = 0; texture_mode < 9; texture_mode++)
{
for (u8 interlacing = 0; interlacing < 2; interlacing++)
for (u8 dithering = 0; dithering < 2; dithering++)
{
const bool textured = (static_cast<GPUTextureMode>(texture_mode) != GPUTextureMode::Disabled);
const std::string batch_vs = shadergen.GenerateBatchVertexShader(textured);
// If we're at a Reserved_* texture_mode, build a fragment
// shader source for the equivalent non-reserved mode. The
// post-MD5 cache dedup will then short-circuit the GL
// compile/link work too.
const std::string fs = shadergen.GenerateBatchFragmentShader(
static_cast<BatchRenderMode>(render_mode), static_cast<GPUTextureMode>(source_mode),
ConvertToBoolUnchecked(dithering), ConvertToBoolUnchecked(interlacing));
const auto link_callback = [this, textured, use_binding_layout](GL::Program& prog) {
if (!use_binding_layout)
{
prog.BindAttribute(0, "a_pos");
prog.BindAttribute(1, "a_col0");
if (textured)
{
prog.BindAttribute(2, "a_texcoord");
prog.BindAttribute(3, "a_texpage");
prog.BindAttribute(4, "a_uv_limits");
}
if (!IsGLES() || m_supports_dual_source_blend)
{
if (m_supports_dual_source_blend)
{
prog.BindFragDataIndexed(0, "o_col0");
prog.BindFragDataIndexed(1, "o_col1");
}
else
{
prog.BindFragData(0, "o_col0");
}
}
}
};
std::optional<GL::Program> prog = shader_cache.GetProgram(batch_vs, {}, fs, link_callback);
if (!prog)
return false;
if (!use_binding_layout)
for (u8 interlacing = 0; interlacing < 2; interlacing++)
{
prog->BindUniformBlock("UBOBlock", 1);
if (textured)
{
prog->Bind();
prog->Uniform1i("samp0", 0);
}
const GL::Program* prog = GetBatchProgram(render_mode, texture_mode, ConvertToBoolUnchecked(dithering),
ConvertToBoolUnchecked(interlacing));
if (!prog)
return false;
progress.Increment();
}
m_render_programs[render_mode][texture_mode][dithering][interlacing] = std::move(*prog);
progress.Increment();
}
}
}
}
// For Lazy and Disabled: m_render_programs stays default-
// constructed (program id 0); each cell is filled on first draw
// by GetBatchProgram on the runloop thread.
for (u8 depth_24bit = 0; depth_24bit < 2; depth_24bit++)
{
@ -1303,11 +1294,100 @@ bool GPU_HW_OpenGL::CompilePrograms()
return true;
}
const GL::Program* GPU_HW_OpenGL::GetBatchProgram(u8 render_mode, u8 texture_mode, bool dithering, bool interlacing)
{
// Reserved_*Direct16Bit dedup. The fragment shader source for
// texture_mode 3 / 7 is byte-for-byte identical to 2 / 6 after
// macro expansion. Unlike the other backends we can't share a
// refcounted handle between slots (GL::Program is move-only and
// owns its GLuint via glDeleteProgram in its destructor); the
// best we can do is re-link the canonical source string into a
// separate program object. The disk-backed program cache hits on
// the second GetProgram call and reloads the linked binary
// instead of recompiling, so the duplicate is cheap.
const u8 lookup_mode = (texture_mode == static_cast<u8>(GPUTextureMode::Reserved_Direct16Bit)) ? 2u :
(texture_mode == static_cast<u8>(GPUTextureMode::Reserved_RawDirect16Bit)) ? 6u :
texture_mode;
GL::Program& slot = m_render_programs[render_mode][texture_mode][BoolToUInt8(dithering)][BoolToUInt8(interlacing)];
if (slot.IsValid())
return &slot;
if (!m_shadergen)
{
Log_ErrorPrint("GetBatchProgram called before CompilePrograms constructed the shadergen");
return nullptr;
}
const bool textured = (static_cast<GPUTextureMode>(lookup_mode) != GPUTextureMode::Disabled);
const std::string batch_vs = m_shadergen->GenerateBatchVertexShader(textured);
const std::string fs = m_shadergen->GenerateBatchFragmentShader(
static_cast<BatchRenderMode>(render_mode), static_cast<GPUTextureMode>(lookup_mode), dithering, interlacing);
const bool use_binding_layout = m_use_binding_layout;
const auto link_callback = [this, textured, use_binding_layout](GL::Program& prog) {
if (!use_binding_layout)
{
prog.BindAttribute(0, "a_pos");
prog.BindAttribute(1, "a_col0");
if (textured)
{
prog.BindAttribute(2, "a_texcoord");
prog.BindAttribute(3, "a_texpage");
prog.BindAttribute(4, "a_uv_limits");
}
if (!IsGLES() || m_supports_dual_source_blend)
{
if (m_supports_dual_source_blend)
{
prog.BindFragDataIndexed(0, "o_col0");
prog.BindFragDataIndexed(1, "o_col1");
}
else
{
prog.BindFragData(0, "o_col0");
}
}
}
};
std::optional<GL::Program> prog = m_shader_cache.GetProgram(batch_vs, {}, fs, link_callback);
if (!prog)
{
Log_ErrorPrintf("Lazy batch program compile failed for (rm=%u, tm=%u, d=%u, i=%u)", render_mode, texture_mode,
BoolToUInt8(dithering), BoolToUInt8(interlacing));
return nullptr;
}
if (!use_binding_layout)
{
prog->BindUniformBlock("UBOBlock", 1);
if (textured)
{
prog->Bind();
prog->Uniform1i("samp0", 0);
}
}
slot = std::move(*prog);
return &slot;
}
void GPU_HW_OpenGL::DrawBatchVertices(BatchRenderMode render_mode, u32 base_vertex, u32 num_vertices)
{
const GL::Program& prog = m_render_programs[static_cast<u8>(render_mode)][static_cast<u8>(m_batch.texture_mode)]
[BoolToUInt8(m_batch.dithering)][BoolToUInt8(m_batch.interlacing)];
prog.Bind();
// Fetch the batch program via the lazy helper. In 'Enabled'
// precompile mode every slot was filled at CompilePrograms time
// so this is a fast IsValid() check + array index. In 'Lazy' /
// 'Disabled' mode the slot is compiled on the first use of that
// combination, then cached. Single-threaded - no mutex / no
// atomic - because the libretro hardware-renderer protocol only
// gives us one GL context bound to this thread.
const GL::Program* prog = GetBatchProgram(static_cast<u8>(render_mode), static_cast<u8>(m_batch.texture_mode),
m_batch.dithering, m_batch.interlacing);
if (!prog)
return;
prog->Bind();
if (m_current_transparency_mode != m_batch.transparency_mode || m_current_render_mode != render_mode)
{

View file

@ -15,6 +15,7 @@
#include "core/host_display.h"
#include "glad.h"
#include "gpu_hw.h"
#include "gpu_hw_shadergen.h"
#include "texture_replacements.h"
#include <array>
#include <memory>
@ -144,6 +145,23 @@ private:
bool CompilePrograms();
// Lazy batch-program compile path. The libretro hardware-renderer
// protocol gives us a single GL context bound to the runloop
// thread; there's no way to compile shaders on a background
// worker (no second context, no glShareLists hand-off), so
// unlike the D3D11/D3D12/Vulkan backends OpenGL's 'Lazy' mode
// degrades to 'Disabled' - compile each combination on the
// runloop the first time the game dispatches a draw using it.
// No threading, no atomics, no mutex; everything runs on one
// thread.
//
// The matrix slots start as default-constructed GL::Programs
// (program id 0); GetBatchProgram fills a slot on demand,
// applying the Reserved_*Direct16Bit dedup at the matrix level
// by re-linking the program from the canonical mode's source
// string (the shader cache makes the second link cheap).
const GL::Program* GetBatchProgram(u8 render_mode, u8 texture_mode, bool dithering, bool interlacing);
void SetDepthFunc();
void SetDepthFunc(GLenum func);
void SetBlendMode();
@ -180,6 +198,20 @@ private:
GL::Program m_vram_copy_program;
GL::Program m_vram_update_depth_program;
// Persistent shader cache, shadergen, and the
// use_binding_layout flag the cold-start CompilePrograms picked
// up. All three used to be locals in CompilePrograms(); they
// now outlive that function so GetBatchProgram can call into
// them at draw time on a lazy miss. On UpdateSettings round-
// trips through CompilePrograms the cache instance persists
// (Open is guarded by the new GL::ShaderCache::IsOpen()
// accessor) so the in-memory index isn't re-read from disk each
// time. The shadergen is rebuilt each CompilePrograms because
// its baked-in settings may have changed.
GL::ShaderCache m_shader_cache;
std::unique_ptr<GPU_HW_ShaderGen> m_shadergen;
bool m_use_binding_layout = false;
u32 m_uniform_buffer_alignment = 1;
u32 m_texture_stream_buffer_size = 0;