TsundereStation/data/shaders
LibretroAdmin e07ce046f0 core: pre-bake the textured xBR batch FS slice
Fifth and final batch FS template to be pre-baked, after
batch_untextured (9e6f933 + b6d1903 + c01f8ae),
batch_textured_nearest (9e4c33d + a7f5717),
batch_textured_bilinear (269a2a0 + f2ae92a), and
batch_textured_jinc2 (17a0c66 + 6afe8c2). Adds the HLSL template
+ regen recipe + 144 DXBC blobs for the (texture_mode !=
GPUTextureMode::Disabled AND m_texture_filter in {xBR,
xBRBinAlpha}) slice. No C++ wiring yet - the activation commit
lands next and completes the batch FS pre-bake arc.

Variant matrix is identical in shape to Bilinear / JINC2:

  * 6 (tex_mode) x 2 (dual) x 3 (interp) x 2 (persp) x 2 (BinAlpha)
    = 144 blobs.

Only the FilteredSampleFromVRAM body differs. xBR (eXtended Block
Resampling) is an edge-detection-based scaling algorithm:

  - Samples a 5x5 neighbourhood around each source texel via the
    P() helper macro that wraps SampleFromVRAM with UV-limit
    clamping.
  - Carries a per-texel "non-transparent" flag in the W field,
    repurposed from the VRAM "0000h" sentinel test.
  - Computes YCbCr distances (DistYCbCr) between adjacent samples
    using the rec.2020 W_YCBCR weights (0.2627, 0.6780, 0.0593,
    0.5).
  - For each of 4 quadrants (x = upper-left, y = upper-right, z =
    lower-right, w = lower-left), decides blend level: BLEND_NONE,
    BLEND_NORMAL, or BLEND_DOMINANT. The decision is gated by a
    gradient comparison: if the sum of YCbCr distances along one
    diagonal beats the sum along the other by
    DOMINANT_DIRECTION_THRESHOLD (3.6x), the dominant blend
    applies; otherwise normal blend.
  - Per-quadrant line-blend special case: if the gradient pattern
    matches a steep or shallow line shape (filtered by
    STEEP_DIRECTION_THRESHOLD = 2.2), adjust origin + direction
    vectors for the get_left_ratio mix factor.
  - Final colour is a per-quadrant lerp from E (centre) toward
    the chosen blend pixel, weighted by get_left_ratio which
    uses smoothstep on the projected distance to the blend line.

One documented behavioural quirk is preserved byte-for-byte: the
"non-transparent" flag for the I sample (lower-right of the 3x3)
is computed from H's vec4 rather than I's. This is a longstanding
shadergen bug in the original PCSX2 xBR implementation; we
reproduce it so the pre-baked DXBC matches the runtime
D3DCompile output exactly. Fixing it would be a semantic change
to xBR's visual output and is out of scope for the pre-bake arc.

The HLSL body is the largest of any filter template:
~250 lines of FilteredSampleFromVRAM body + helper functions,
producing ~50 KiB DXBC per variant on average (48-60 KiB across
the texture-mode / dual / persp / binalpha axis combos).
144 x ~50 KiB = ~7 MiB on disk and ~7 MiB embedded DXBC. Per-
variant compile time on Linux+wine is ~1s; the full 144-variant
regen takes 3-4 minutes uninterrupted (the wine-flake retry
logic at 269a2a0 catches transient misses).

Without the c532a34 TRANSPARENCY routing, this template would
have been 144 x 4 = 576 variants on the [render_mode] axis. Same
4x saving as Bilinear / JINC2.

Files in this commit (146 total, no C++ touched):

  data/shaders/d3d_common/batch_textured_xbr.ps.hlsl
    Hand-translated mirror of shadergen's textured-xBR-family
    output. Preamble identical to Bilinear / JINC2: BatchUBOData
    cbuffer, Texture2D + SamplerState bindings at register t0/s0,
    ApplyDithering / ApplyTextureWindow / ApplyUpscaledTextureWindow /
    FloatToIntegerCoords / RGBA8ToRGBA5551 / SampleFromVRAM
    helpers. FilteredSampleFromVRAM swaps in xBR's algorithm:
    DistYCbCr, IsPixEqual, get_left_ratio helpers + the 4-quadrant
    blend decision tree + per-quadrant blend application loops.
    main() is structurally identical to Bilinear / JINC2 - same
    `ialpha < 0.5` discard, same UV-limits unconditional clamp,
    same render_mode / pgxp_depth cbuffer-driven runtime branches.

  tools/regen_d3d_common_dxbc.py
    TEMPLATE_VARIANTS gains a batch_textured_xbr.ps.hlsl entry
    with all 144 variant tuples positioned after the JINC2 entry
    (chronological foundation-commit order: nearest, bilinear,
    jinc2, xbr, then the downsample templates). Variant suffix
    encoding identical to Bilinear / JINC2:
      pXrY_d{0,1}_{none,centroid,sample}_n{0,1}_b{0,1}
    where the b{0,1} digit encodes BINALPHA (0 = xBR,
    1 = xBRBinAlpha). Alphabetical sort matches the natural
    nested-loop iteration order in the matching picker.

  src/common/d3d_common/embedded_dxbc/batch_textured_xbr_ps_*.inc
    144 new .inc files, ~48-60 KiB DXBC per variant (~50 KiB
    average), totalling ~7 MiB on disk and ~7 MiB embedded DXBC
    in the .so. Per-variant size is the largest of any pre-baked
    batch FS template:
      Untextured:                 ~2-3 KiB
      Nearest:                    ~5 KiB
      Bilinear:                   ~6 KiB
      JINC2:                      ~17-24 KiB
      xBR:                        ~48-60 KiB
    The 5x5 sample fan-out + 4-quadrant blend decision tree + 4
    per-quadrant blend application blocks dominate the bytecode
    size; fxc /O3 cannot fold the data-dependent control flow
    even though many of the boolean inputs are compile-time
    constants from the BINALPHA macro.

What does NOT change in this commit:

  - GPU_HW_D3D12::GetBatchPipeline still routes xBR / xBRBinAlpha
    through the shadergen + D3DCompile fallback arm (the else
    branch at gpu_hw_d3d12.cpp after the JINC2 activation). The
    144 pre-baked xBR blobs are not yet consulted - they sit on
    disk waiting for the activation commit.

  - GPU_HW_D3D11::GetBatchPixelShader same: still falls through
    to the tmp_shadergen + D3DCompile path for xBR / xBRBinAlpha.

  - src/common/d3d_common/embedded_shaders.{h,cpp} untouched -
    they need 288 new extern declarations / includes to surface
    the .inc blob symbols to the C++ linker; the activation
    commit adds them along with the xBR-family picker.

  - m_texture_filter == xBR / xBRBinAlpha sessions still go
    through D3DCompile on both D3D11 and D3D12 backends until
    the activation commit lands.

  - The shadergen-fallback else arm in GetBatchPipeline and
    GetBatchPixelShader still exists. After the next activation
    commit it becomes unreachable and is deleted, together with
    the tmp_shadergen helper construction on the D3D11 side.

Test:
  - python3 tools/regen_d3d_common_dxbc.py --fxc /tmp/fxc_lite/
    fxc_lite.exe --wine /usr/bin/wine: 614 .inc files generated
    (470 pre-existing + 144 new xBR). Wine flakiness on Linux
    triggered transient retry warnings on the larger xBR variants
    (their longer fxc compile time leaves more room for wineserver
    state to drift); the retry logic added in 269a2a0 caught them.
  - Hand-compile of representative variants:
      no /D (p0r0_d0_none_n0_b0):                    49 692 bytes
      /D PALETTE_4_BIT=1:                            60 020 bytes
      /D PALETTE_4_BIT=1 /D BINALPHA=1:              60 048 bytes
      /D PALETTE_8_BIT=1 /D BINALPHA=1 /D RAW_TEXTURE=1
        /D USE_DUAL_SOURCE=1:                        58 912 bytes
      /D RAW_TEXTURE=1 /D INTERP_SAMPLE=1 /D NOPERSP=1
        /D BINALPHA=1 /D USE_DUAL_SOURCE=1:          48 584 bytes
    All compile cleanly; no fxc warnings or errors. Palette modes
    are noticeably larger than direct modes for the same reason
    as JINC2: the palette-index lookup adds another half-dozen
    SampleFromVRAM-equivalent dependent texture fetches per texel
    even after fxc's dead-code elimination, and xBR's 5x5
    neighbourhood multiplies the cost.

Cumulative pre-bake variant count for D3D batch FS:

  Pre-arc:                                    75 264 (every axis a -D)
  Post-cbuffer-routing arc:                   1 176
  Post-c01f8ae / a7f5717 (Untextured + Nearest, both backends):
                                              168 cell-equivalents
  Post-2fcf4cc (D3D11 downsample live):       22 additional cells
  Post-269a2a0 + f2ae92a (Bilinear-family live):
                                              +288 cell-equivalents
  Post-17a0c66 + 6afe8c2 (JINC2-family live):
                                              +288 cell-equivalents
  Post-this-commit (xBR slice on disk):       +144 baked
    (consumers wired in the activation commit)

Next:
  - Activation: 288 extern decls + 144 #include lines in
    embedded_shaders.{h,cpp}, plus a PickBatchTexturedXBRFS helper
    that takes the additional `binalpha` parameter, and the
    GetBatchPipeline / GetBatchPixelShader branches that route
    m_texture_filter in {xBR, xBRBinAlpha} + texture_mode !=
    Disabled to the new picker. Parallel to f2ae92a (Bilinear
    activation) and 6afe8c2 (JINC2 activation), but additionally
    this activation will:
      - Delete the now-unreachable shadergen-fallback else arm
        from GetBatchPipeline (D3D12) and GetBatchPixelShader
        (D3D11).
      - Delete the tmp_shadergen helper construction on the
        D3D11 side and the GetBatchFragmentShader call on the
        D3D12 side, since neither has any remaining consumer.
      - Tighten the D3D11 m_batch_pixel_shaders matrix
        dimensions if the [dithering] / [interlacing] dims
        become unused post-cbuffer-routing - they should, but
        the dim collapse is a separate concern from picker
        wiring and may land in a follow-up.
  - Independent: Phase 2 D3D12 downsample port - PSOs, explicit
    resource barriers, texture / mip-view creation. Uses the
    pre-baked blobs from 251e457 + the d3d_common rename at
    0ed7261.
2026-05-20 05:08:52 +02:00
..
d3d_common core: pre-bake the textured xBR batch FS slice 2026-05-20 05:08:52 +02:00
vulkan core: route TRANSPARENCY through the batch UBO 2026-05-19 08:13:03 +02:00