From 5c9cf3ea9f5ce305ecbd97d18e38b3ac87ade616 Mon Sep 17 00:00:00 2001 From: LibretroAdmin Date: Mon, 11 May 2026 17:18:35 +0000 Subject: [PATCH] gpu_hw_opengl: route batch programs through the same three-mode precompile machinery as D3D11/D3D12/Vulkan This is the final backend in the swanstation_GPU_ShaderPrecompile rollout. OpenGL is structurally different from the other three: the libretro hardware-renderer protocol gives the core a single GL context bound to the runloop thread, with no way to spin up a worker thread that shares the context (no second context, no glShareLists hand-off path). So OpenGL's 'Lazy' mode degrades to the same shape as 'Disabled' - compile each batch program on the runloop the first time the game dispatches a draw using it, then cache it. No worker thread, no mutex, no atomic; everything runs on one thread. Adds GetBatchProgram(render_mode, texture_mode, dithering, interlacing) as the single fault-in path, mirroring the GetBatchPixelShader / GetBatchPipeline helpers on the other backends. The matrix is the existing m_render_programs (array of GL::Program, move-only); slots start as default-constructed (m_program_id = 0) and GetBatchProgram fills them lazily. The Reserved_*Direct16Bit dedup is handled differently from the other three backends - GL::Program is move-only and owns its GLuint via glDeleteProgram in its destructor, so we can't share a single handle between two array slots without a double-delete hazard. Instead, the helper feeds the canonical (lookup_mode) fragment-shader source string into GetProgram for both slots. The disk-backed program cache hits on the second call and reloads the linked binary instead of recompiling, so the duplicate is cheap (one glProgramBinary call rather than a full GLSL compile + link). Mode behaviour: - Enabled: synchronous walk of the matrix at CompilePrograms time, calling GetBatchProgram on each cell. - Lazy / Disabled: matrix stays default-constructed at CompilePrograms time; each slot is filled by DrawBatchVertices -> GetBatchProgram on first use of that combination. Two small infrastructure additions to support the lazy path: - GL::Program::IsValid() accessor ('return m_program_id != 0'). GetBatchProgram uses it to distinguish a never-touched slot from a compiled one. glUseProgram(0) un-binds; calling prog->Bind() on a slot with id 0 would silently un-bind the previous program and break rendering, so the explicit validity check is required. - GL::ShaderCache::IsOpen() accessor, mirroring D3D11 / D3D12 parity. The shader cache becomes a member (m_shader_cache) so the lazy helper can reach it at draw time on UpdateSettings round-trips; without IsOpen the second Open() call would re-read the index and double-count entries. GPU_HW_ShaderGen becomes a member (m_shadergen, unique_ptr since the class is non-default-constructible) so GetBatchProgram can generate batch shader source at draw time. use_binding_layout is captured into m_use_binding_layout because GetBatchProgram's link-callback needs it. CompilePrograms now explicitly resets the m_render_programs matrix at function entry. Without that, an UpdateSettings round- trip (which calls CompilePrograms with new texture_filter / scale / etc. without an explicit DestroyPrograms step) would leave the previous programs in place; the lazy fault-in path's IsValid() short-circuit would return them as if they were fresh, and the game would render through programs built for the old settings. The synchronous 'Enabled' precompile loop has the same short-circuit so it needs the same guarantee. The reset is a default-assign loop which invokes GL::Program's move-assign operator, which calls Destroy() (i.e. glDeleteProgram) on each slot before resetting to id = 0. DrawBatchVertices routes its m_render_programs lookup through GetBatchProgram so all three modes share the same fault-in path. The OpenGL backend has historically had the longest cold-compile stall in some configurations (long GLSL -> driver compile pipeline on some Intel / Mesa stacks, no async API in core OpenGL until ARB_parallel_shader_compile which not all drivers implement). With 'Lazy' as the default mode, that stall moves from CompilePrograms-on-init / CompilePrograms-on-settings-change to a stutter on the first draw that touches each combination - one per cell, capped at ~144 cells total over the session, often only a handful in practice for most games. The trade-off is preferable because it makes the UI feel responsive (RetroArch keeps drawing frames while gameplay progresses) rather than froze for the whole compile. This commit completes the four-backend rollout. All of D3D11, D3D12, Vulkan, and OpenGL now honour swanstation_GPU_ShaderPrecompile = Lazy / Enabled / Disabled. Build verification: linux platform=unix (which DOES compile gpu_hw_opengl.cpp, unlike the D3D11/D3D12 cases): zero warnings, zero errors. .so size 8040104 (+4240 vs commit a29d202's 8035864). The growth is the GetBatchProgram body, the matrix- reset loop at CompilePrograms entry, and the persistent member storage for the shader cache and shadergen. 25 retro_* exports unchanged. Files touched: - src/common/gl/program.h: IsValid() accessor. - src/common/gl/shader_cache.h: IsOpen() accessor. - src/core/gpu_hw_opengl.h: includes, m_shader_cache, m_shadergen, m_use_binding_layout members, GetBatchProgram declaration. - src/core/gpu_hw_opengl.cpp: CompilePrograms resets the matrix at entry, uses persistent member shader_cache / shadergen, three-way precompile branch; GetBatchProgram helper added; DrawBatchVertices routes through the helper. --- src/common/gl/program.h | 6 + src/common/gl/shader_cache.h | 8 ++ src/core/gpu_hw_opengl.cpp | 246 +++++++++++++++++++++++------------ src/core/gpu_hw_opengl.h | 32 +++++ 4 files changed, 209 insertions(+), 83 deletions(-) diff --git a/src/common/gl/program.h b/src/common/gl/program.h index 4f757a3a..4220932d 100644 --- a/src/common/gl/program.h +++ b/src/common/gl/program.h @@ -34,6 +34,12 @@ public: void Bind() const; + // Returns true if the program has a non-zero linked GL handle. + // Used by the lazy-compile path in GPU_HW_OpenGL to distinguish a + // never-touched matrix slot (program id 0) from one that has + // already been compiled and is ready for Bind(). + bool IsValid() const { return m_program_id != 0; } + void Destroy(); int RegisterUniform(const char* name); diff --git a/src/common/gl/shader_cache.h b/src/common/gl/shader_cache.h index 8a4b8ff3..93864a5b 100644 --- a/src/common/gl/shader_cache.h +++ b/src/common/gl/shader_cache.h @@ -23,6 +23,14 @@ public: void Open(bool is_gles, std::string_view base_path, u32 version); + // Returns whether Open() has already been called successfully on + // this instance. Used by the lazy-compile path in GPU_HW_OpenGL + // to avoid re-reading the same on-disk index on UpdateSettings + // round-trips through CompilePrograms, which would both leak + // file handles and double-count m_index entries. Mirrors the + // equivalent accessor on D3D11::ShaderCache and D3D12::ShaderCache. + bool IsOpen() const { return m_index_file != nullptr; } + std::optional GetProgram(const std::string_view vertex_shader, const std::string_view geometry_shader, const std::string_view fragment_shader, const PreLinkCallback& callback = {}); diff --git a/src/core/gpu_hw_opengl.cpp b/src/core/gpu_hw_opengl.cpp index 5483e64f..9a67bb8f 100644 --- a/src/core/gpu_hw_opengl.cpp +++ b/src/core/gpu_hw_opengl.cpp @@ -1068,97 +1068,88 @@ bool GPU_HW_OpenGL::CreateTextureBuffer() bool GPU_HW_OpenGL::CompilePrograms() { - GL::ShaderCache shader_cache; - shader_cache.Open(IsGLES(), g_host_interface->GetShaderCacheBasePath(), SHADER_CACHE_VERSION); + // Reset every program object so the new compile starts from a + // clean slate. Without this, on an UpdateSettings round-trip + // through CompilePrograms (in Lazy / Disabled mode where the + // matrix isn't refilled here) the lazy fault-in path in + // GetBatchProgram would observe programs left over from the + // previous settings - built with the old texture filter / scale + // / etc. - and use them for the new draws. The synchronous + // 'Enabled' loop further down also relies on this: it calls + // GetBatchProgram which short-circuits on IsValid(), so a + // pre-populated slot would silently keep its old program. + // + // GL::Program's move-assign operator calls Destroy() on the + // destination before taking the source's state, so default- + // assigning each slot to a temporary {} both glDeleteProgram's + // the old handle and rewinds the slot to id = 0. + for (auto& a : m_render_programs) + for (auto& b : a) + for (auto& c : b) + for (auto& slot : c) + slot = GL::Program{}; - const bool use_binding_layout = GPU_HW_ShaderGen::UseGLSLBindingLayout(); - GPU_HW_ShaderGen shadergen(m_host_display->GetRenderAPI(), m_resolution_scale, m_multisamples, m_per_sample_shading, - m_true_color, m_scaled_dithering, m_texture_filtering, m_using_uv_limits, - m_pgxp_depth_buffer, m_disable_color_perspective, m_supports_dual_source_blend); - - ShaderCompileProgressTracker progress("Compiling Programs", (4 * 9 * 2 * 2) + (2 * 3) + (2 * 2) + 1 + 1 + 1 + 1 + 1); - - for (u32 render_mode = 0; render_mode < 4; render_mode++) + // Open the disk-backed program cache once. On subsequent calls + // (UpdateSettings -> CompilePrograms) the instance still holds + // the index from last time and the disk file hasn't moved, so + // re-opening would double-count m_index and leak file handles. + if (!m_shader_cache.IsOpen()) { - for (u32 texture_mode = 0; texture_mode < 9; texture_mode++) + m_shader_cache.Open(IsGLES(), g_host_interface->GetShaderCacheBasePath(), SHADER_CACHE_VERSION); + } + GL::ShaderCache& shader_cache = m_shader_cache; + + m_use_binding_layout = GPU_HW_ShaderGen::UseGLSLBindingLayout(); + const bool use_binding_layout = m_use_binding_layout; + + m_shadergen = std::make_unique( + m_host_display->GetRenderAPI(), m_resolution_scale, m_multisamples, m_per_sample_shading, m_true_color, + m_scaled_dithering, m_texture_filtering, m_using_uv_limits, m_pgxp_depth_buffer, m_disable_color_perspective, + m_supports_dual_source_blend); + GPU_HW_ShaderGen& shadergen = *m_shadergen; + + // OpenGL is the odd one out: the libretro hardware-renderer + // protocol exposes only a single GL context bound to the + // runloop, so there's no way to spin up a worker that compiles + // in the background like D3D11/D3D12/Vulkan do. 'Lazy' therefore + // degrades to the same shape as 'Disabled': skip the matrix at + // CompilePrograms time, fault each combo in on the runloop the + // first time the game dispatches a draw using it. + // + // 'Enabled' still does the full synchronous walk via the + // GetBatchProgram helper, mirroring the D3D11/D3D12/Vulkan + // commits. + const GPUShaderPrecompileMode precompile_mode = g_settings.gpu_shader_precompile_mode; + const bool precompile_sync = (precompile_mode == GPUShaderPrecompileMode::Enabled); + const u32 batch_progress_units = + (precompile_mode == GPUShaderPrecompileMode::Enabled) ? static_cast(4 * 9 * 2 * 2) : 0u; + + ShaderCompileProgressTracker progress("Compiling Programs", + batch_progress_units + (2 * 3) + (2 * 2) + 1 + 1 + 1 + 1 + 1); + + if (precompile_sync) + { + for (u8 render_mode = 0; render_mode < 4; render_mode++) { - // Reserved_Direct16Bit (3) and Reserved_RawDirect16Bit (7) produce - // shaders identical to Direct16Bit (2) and RawDirect16Bit (6) - see - // GPU_HW_D3D11::CompileShaders for the full explanation. For - // OpenGL we can't simply copy the resulting GL::Program object - // because it's move-only (it owns a GLuint program ID that the - // destructor will glDeleteProgram), so we have to re-link the - // program. We can, however, skip the ~50ms shader-source - // regeneration in GenerateBatchFragmentShader by re-using the - // already-built source string from the previous (source_mode) - // iteration. The shader cache will hit on the second GetProgram - // call and reload the binary instead of recompiling. - const u32 source_mode = (texture_mode == static_cast(GPUTextureMode::Reserved_Direct16Bit)) ? 2u : - (texture_mode == static_cast(GPUTextureMode::Reserved_RawDirect16Bit)) ? 6u : - texture_mode; - - for (u8 dithering = 0; dithering < 2; dithering++) + for (u8 texture_mode = 0; texture_mode < 9; texture_mode++) { - for (u8 interlacing = 0; interlacing < 2; interlacing++) + for (u8 dithering = 0; dithering < 2; dithering++) { - const bool textured = (static_cast(texture_mode) != GPUTextureMode::Disabled); - const std::string batch_vs = shadergen.GenerateBatchVertexShader(textured); - // If we're at a Reserved_* texture_mode, build a fragment - // shader source for the equivalent non-reserved mode. The - // post-MD5 cache dedup will then short-circuit the GL - // compile/link work too. - const std::string fs = shadergen.GenerateBatchFragmentShader( - static_cast(render_mode), static_cast(source_mode), - ConvertToBoolUnchecked(dithering), ConvertToBoolUnchecked(interlacing)); - - const auto link_callback = [this, textured, use_binding_layout](GL::Program& prog) { - if (!use_binding_layout) - { - prog.BindAttribute(0, "a_pos"); - prog.BindAttribute(1, "a_col0"); - if (textured) - { - prog.BindAttribute(2, "a_texcoord"); - prog.BindAttribute(3, "a_texpage"); - prog.BindAttribute(4, "a_uv_limits"); - } - - if (!IsGLES() || m_supports_dual_source_blend) - { - if (m_supports_dual_source_blend) - { - prog.BindFragDataIndexed(0, "o_col0"); - prog.BindFragDataIndexed(1, "o_col1"); - } - else - { - prog.BindFragData(0, "o_col0"); - } - } - } - }; - - std::optional prog = shader_cache.GetProgram(batch_vs, {}, fs, link_callback); - if (!prog) - return false; - - if (!use_binding_layout) + for (u8 interlacing = 0; interlacing < 2; interlacing++) { - prog->BindUniformBlock("UBOBlock", 1); - if (textured) - { - prog->Bind(); - prog->Uniform1i("samp0", 0); - } + const GL::Program* prog = GetBatchProgram(render_mode, texture_mode, ConvertToBoolUnchecked(dithering), + ConvertToBoolUnchecked(interlacing)); + if (!prog) + return false; + progress.Increment(); } - - m_render_programs[render_mode][texture_mode][dithering][interlacing] = std::move(*prog); - - progress.Increment(); } } } } + // For Lazy and Disabled: m_render_programs stays default- + // constructed (program id 0); each cell is filled on first draw + // by GetBatchProgram on the runloop thread. for (u8 depth_24bit = 0; depth_24bit < 2; depth_24bit++) { @@ -1303,11 +1294,100 @@ bool GPU_HW_OpenGL::CompilePrograms() return true; } +const GL::Program* GPU_HW_OpenGL::GetBatchProgram(u8 render_mode, u8 texture_mode, bool dithering, bool interlacing) +{ + // Reserved_*Direct16Bit dedup. The fragment shader source for + // texture_mode 3 / 7 is byte-for-byte identical to 2 / 6 after + // macro expansion. Unlike the other backends we can't share a + // refcounted handle between slots (GL::Program is move-only and + // owns its GLuint via glDeleteProgram in its destructor); the + // best we can do is re-link the canonical source string into a + // separate program object. The disk-backed program cache hits on + // the second GetProgram call and reloads the linked binary + // instead of recompiling, so the duplicate is cheap. + const u8 lookup_mode = (texture_mode == static_cast(GPUTextureMode::Reserved_Direct16Bit)) ? 2u : + (texture_mode == static_cast(GPUTextureMode::Reserved_RawDirect16Bit)) ? 6u : + texture_mode; + + GL::Program& slot = m_render_programs[render_mode][texture_mode][BoolToUInt8(dithering)][BoolToUInt8(interlacing)]; + if (slot.IsValid()) + return &slot; + + if (!m_shadergen) + { + Log_ErrorPrint("GetBatchProgram called before CompilePrograms constructed the shadergen"); + return nullptr; + } + + const bool textured = (static_cast(lookup_mode) != GPUTextureMode::Disabled); + const std::string batch_vs = m_shadergen->GenerateBatchVertexShader(textured); + const std::string fs = m_shadergen->GenerateBatchFragmentShader( + static_cast(render_mode), static_cast(lookup_mode), dithering, interlacing); + + const bool use_binding_layout = m_use_binding_layout; + const auto link_callback = [this, textured, use_binding_layout](GL::Program& prog) { + if (!use_binding_layout) + { + prog.BindAttribute(0, "a_pos"); + prog.BindAttribute(1, "a_col0"); + if (textured) + { + prog.BindAttribute(2, "a_texcoord"); + prog.BindAttribute(3, "a_texpage"); + prog.BindAttribute(4, "a_uv_limits"); + } + + if (!IsGLES() || m_supports_dual_source_blend) + { + if (m_supports_dual_source_blend) + { + prog.BindFragDataIndexed(0, "o_col0"); + prog.BindFragDataIndexed(1, "o_col1"); + } + else + { + prog.BindFragData(0, "o_col0"); + } + } + } + }; + + std::optional prog = m_shader_cache.GetProgram(batch_vs, {}, fs, link_callback); + if (!prog) + { + Log_ErrorPrintf("Lazy batch program compile failed for (rm=%u, tm=%u, d=%u, i=%u)", render_mode, texture_mode, + BoolToUInt8(dithering), BoolToUInt8(interlacing)); + return nullptr; + } + + if (!use_binding_layout) + { + prog->BindUniformBlock("UBOBlock", 1); + if (textured) + { + prog->Bind(); + prog->Uniform1i("samp0", 0); + } + } + + slot = std::move(*prog); + return &slot; +} + void GPU_HW_OpenGL::DrawBatchVertices(BatchRenderMode render_mode, u32 base_vertex, u32 num_vertices) { - const GL::Program& prog = m_render_programs[static_cast(render_mode)][static_cast(m_batch.texture_mode)] - [BoolToUInt8(m_batch.dithering)][BoolToUInt8(m_batch.interlacing)]; - prog.Bind(); + // Fetch the batch program via the lazy helper. In 'Enabled' + // precompile mode every slot was filled at CompilePrograms time + // so this is a fast IsValid() check + array index. In 'Lazy' / + // 'Disabled' mode the slot is compiled on the first use of that + // combination, then cached. Single-threaded - no mutex / no + // atomic - because the libretro hardware-renderer protocol only + // gives us one GL context bound to this thread. + const GL::Program* prog = GetBatchProgram(static_cast(render_mode), static_cast(m_batch.texture_mode), + m_batch.dithering, m_batch.interlacing); + if (!prog) + return; + prog->Bind(); if (m_current_transparency_mode != m_batch.transparency_mode || m_current_render_mode != render_mode) { diff --git a/src/core/gpu_hw_opengl.h b/src/core/gpu_hw_opengl.h index df996748..7fb65033 100644 --- a/src/core/gpu_hw_opengl.h +++ b/src/core/gpu_hw_opengl.h @@ -15,6 +15,7 @@ #include "core/host_display.h" #include "glad.h" #include "gpu_hw.h" +#include "gpu_hw_shadergen.h" #include "texture_replacements.h" #include #include @@ -144,6 +145,23 @@ private: bool CompilePrograms(); + // Lazy batch-program compile path. The libretro hardware-renderer + // protocol gives us a single GL context bound to the runloop + // thread; there's no way to compile shaders on a background + // worker (no second context, no glShareLists hand-off), so + // unlike the D3D11/D3D12/Vulkan backends OpenGL's 'Lazy' mode + // degrades to 'Disabled' - compile each combination on the + // runloop the first time the game dispatches a draw using it. + // No threading, no atomics, no mutex; everything runs on one + // thread. + // + // The matrix slots start as default-constructed GL::Programs + // (program id 0); GetBatchProgram fills a slot on demand, + // applying the Reserved_*Direct16Bit dedup at the matrix level + // by re-linking the program from the canonical mode's source + // string (the shader cache makes the second link cheap). + const GL::Program* GetBatchProgram(u8 render_mode, u8 texture_mode, bool dithering, bool interlacing); + void SetDepthFunc(); void SetDepthFunc(GLenum func); void SetBlendMode(); @@ -180,6 +198,20 @@ private: GL::Program m_vram_copy_program; GL::Program m_vram_update_depth_program; + // Persistent shader cache, shadergen, and the + // use_binding_layout flag the cold-start CompilePrograms picked + // up. All three used to be locals in CompilePrograms(); they + // now outlive that function so GetBatchProgram can call into + // them at draw time on a lazy miss. On UpdateSettings round- + // trips through CompilePrograms the cache instance persists + // (Open is guarded by the new GL::ShaderCache::IsOpen() + // accessor) so the in-memory index isn't re-read from disk each + // time. The shadergen is rebuilt each CompilePrograms because + // its baked-in settings may have changed. + GL::ShaderCache m_shader_cache; + std::unique_ptr m_shadergen; + bool m_use_binding_layout = false; + u32 m_uniform_buffer_alignment = 1; u32 m_texture_stream_buffer_size = 0;