An honest technical retrospective. It is not about a successful method, but about an idea that reached the end of its road — and why reaching the end of the road is itself worth writing down.

Abstract

This article records a complete research exploration: it began with a naive tokenizer question (is the fixed-grid patch the image’s tokenizer?), was steered toward FH by a controlled comparison of four tokenizers (Grid/FH/SLIC/K-means), and then, starting from a “deep understanding of the essence of Felzenszwalb-Huttenlocher (FH) graph segmentation and CNN,” attempted to design a new computational paradigm that has both FH’s structural awareness and can be trained end-to-end by neural networks. The exploration path passed through three forms in sequence — the Neural Region Composer (NRC), the Region-First Vision Transformer (RF-ViT), and the Structural Aggregation Layer (SAL) — and finally converged on a “structure-aware computational primitive that can be embedded into any network.”

However, after strictly comparing the ideas against community work from 2024–2026, the conclusion is sobering: the core insight is correct, but it has been completely covered by simpler, more effective, and large-scale-verified works (SuiT, SPFormer, DART, BRDG). The value of this article is not in proposing a publishable new method, but in: (1) clarifying the essence of FH and CNN; (2) presenting a natural but ultimately failed design evolution; (3) honestly locating the prior art; (4) distilling transferable lessons.

1. Origin: A Deceptively Simple Question

The problem was originally just one sentence:

Deeply understand the essence of FH segmentation and CNN, then design a new algorithm suitable for neural network training.

On the surface this asks “how do you stitch FH and CNN together,” but the truly valuable reading is:

What “invariant truths” does each of them reveal about visual computation? Can these two truths be unified at a higher dimension?

Carrying the latter reading, all the subsequent reasoning unfolded.

1.1 One Step Earlier: From a Tokenizer Question to the First Controlled Experiment

Strictly speaking, the question above did not emerge from nowhere. Before it, there was a more naive and concrete exploration phase — and it was this phase that used experimental data to point the direction at FH.

The initial question was actually tiny:

Before deep learning, do we first cut the image into fixed grids of patches? Is this step equivalent to the tokenizer in NLP?

Following this analogy down, we find that the ways of “discretizing/serializing continuous images” are far more than fixed grids — they form an entire spectrum: fixed-grid patches (ViT/DeiT), learnable codebooks (VQ-VAE/VQGAN), superpixels/regions (SLIC, Farabet), CNN feature maps (DETR/Mask2Former), keypoints (SuperPoint), learnable queries (DETR/Perceiver), hierarchical pyramids (Swin/PVT), per-pixel (Segmenter). The core contradiction can be summarized in one sentence:

The fixed-grid patch is only the most “simple and crude” image tokenizer — it is content-agnostic; truly strong vision systems often let the model (or the image content itself) decide “how to tokenize.”

Along the “content-adaptive tokenizer” line, I hit an existing piece of code — Color-Block VSS: it uses classic FH graph segmentation to cut perceptually uniform color blocks in Lab space, then encodes each block as one token [symbol_id, residual, centroid]. It is naturally a “region-based adaptive tokenizer” — token boundaries hug object contours, and the token count adapts to content — but at the cost of FH being non-differentiable and untrainable end-to-end. This “elegant but non-differentiable” contradiction is exactly what all of the later NRC → RF-ViT → SAL work was trying to resolve.

1.2 First Experiments: Which of Four Tokenizers Is Better?

Before diving into “how to make it differentiable,” I did something more basic: on the same image (a 256×256 white cat), I pulled all four tokenizers to similar token counts (~200–300) for a horizontal comparison, with metrics covering reconstruction quality (SSIM/MSE), compression efficiency (SSIM/Token), and boundary alignment (distance and coverage of token boundaries against Canny ground-truth edges).

MethodTokensSSIM↑MSE↓SSIM/Token↑Boundary Distance↓Edge Coverage↑
K-means k=2562560.9880.0000423.86——
FH scale=5002950.8550.001222.9031.380.70
SLIC n=2001900.8160.001664.2929.870.43
Grid 16×162560.7380.003252.8834.220.13

Five key findings:

  1. K-means crushes reconstruction quality (SSIM 0.988) but has a fatal flaw: it clusters in color space without constraining spatial connectivity — the same color class can scatter across the image with no spatial structure, so it cannot be a tokenizer at all (tokens must be spatially contiguous regions). This immediately eliminates K-means from candidacy.
  2. FH has the best boundary alignment: edge coverage 0.70, far above Grid’s 0.13 and SLIC’s 0.43. In other words, FH’s token boundaries cling closest to real object contours — the most favorable property for downstream region feature extraction.
  3. SLIC produces the most uniform region sizes (area Std 24.9–47.2, vs. FH’s 181–474). Uniform regions are friendlier for training Transformers on variable-length sequences.
  4. Grid Patch lags across the board: nearly bottom on every metric, with its only advantages being zero compute overhead, fixed length, and differentiability — purely engineering properties.
  5. Compression efficiency: at very few tokens (~64), K-means(64) > SLIC(64) > Grid(32×32) — content-adaptive methods save tokens.

The direct conclusion of this experiment (reconstruction comparison, boundary visualization, region size distribution, efficiency-quality curves, and boundary alignment — five sets of figures):

On “boundaries aligning with real object contours,” FH clearly beats Grid. So the next hypothesis that must be validated naturally emerges: can FH’s precise boundaries translate into downstream performance gains (classification/segmentation)? Would CNN features extracted via Masked Pooling over FH regions be better than Grid patches?

It was this hypothesis that pushed the exploration from “which tokenizer reconstructs better” to “can FH’s structural awareness be embedded into a trainable network” — which in turn forced out the distilled question at the start of Section 1: deeply understand the essence of FH and CNN, and design an end-to-end trainable structure-aware paradigm.

Honesty statement: this comparison was run on a single cat image — directional/illustrative evidence, not statistically significant conclusions (at the time it was already annotated “requires statistical testing on an ImageNet subset”). Its role is to provide an intuition starting point, not a verdict.

2. Anatomy of the Two Paradigms

2.1 The Essence of FH: Bottom-Up Structural Decisions

FH is usually treated as an image segmentation algorithm, but what it really does can be said in one sentence:

Dynamically decide, based on local consistency, which information should be merged and which should not.

Its merge criterion is a relative standard, not an absolute threshold:

Merge(C1, C2)  ⟺  Dif(C1, C2) ≤ min( Int(C1) + τ(C1), Int(C2) + τ(C2) )

FH’s cost: non-differentiable, deterministic, purely bottom-up. It doesn’t learn; it directly computes.

FH’s Actual Computation Flow (and Why It’s Non-Differentiable)

1. Build graph: pixels are vertices, 8-neighborhood adjacent pixels get edges, edge weight = color difference |I(p) − I(q)|
2. Sort: sort all edges by weight ascending                     ← O(E log E)
3. Initialize: each pixel is its own region, Int(C) = 0
4. Greedy merge: traverse each edge (p, q) in ascending weight order:
      if p, q belong to different regions C1, C2, and
         w(p,q) ≤ min( Int(C1)+k/|C1|, Int(C2)+k/|C2| )
      then merge C1, C2 with union-find, and update Int = w(p,q)
5. Post-process: merge regions smaller than min_size

The roots of non-differentiability are threefold, each alone sufficient to sever the gradient:

Complexity O(E log E) ≈ O(HW log HW), runs on CPU. These three points are the real obstacles all later “differentiation” efforts had to route around — not simply “swap colors for CNN features.”

An easily overlooked detail: in τ(C) = k/|C|, the k makes small regions more inclined to merge (larger threshold), suppressing fragmentation. This “area-adaptive tolerance” is the hidden source of FH’s robustness — and it is exactly what NRC later lacked.

2.2 The Essence of CNN: Top-Down Feature Transformation

y_i = Σ_{j∈N(i)} w_ij · x_j

where w_ij are globally shared and fixed convolution kernel weights. This means:

CNN is differentiable and learns semantics from tasks, at the cost of being structurally blind — it is born unable to see that “the world’s building blocks are regions, not grids.”

Three more essential properties of CNN, used repeatedly later:

The last point is the heart of the contradiction: weight sharing makes CNN incapable of “position-dependent aggregation decisions” — which is precisely FH’s specialty.

2.3 Complementarity

DimensionFHCNN
Capability providedDecision (aggregation decisions)Transformation (feature transformation)
Boundary handlingSelectively preservedUniformly smoothed
DifferentiabilityNoYes
Optimization scopeGlobal (graph theory)Local (convolution)

In one sentence: FH gives “form,” CNN gives “spirit”; FH is non-differentiable but boundary-precise, CNN is differentiable but boundary-blurred. The temptation to fuse them comes from exactly this.

3. Design Evolution: Three Forms

3.1 Form One: NRC (Neural Region Composer)

Replace FH’s “sorted edges + hard merges” with “learnable seeds + iterative soft assignment (soft K-means-style growth)” to route around non-differentiability. The core is a “seed growing” paradigm: K learnable seeds (position + feature) “grow” into regions on the feature map through iterative soft assignment.

# NRC core iteration (simplified)
mu = seed_feats + seed_queries        # [K, C] seed features
center = seed_positions               # [K, 2] seed centroids
for t in range(T):                    # T = 3~5 steps
    sim     = F_flat @ mu.T / sqrt(C)             # [N, K] feature similarity
    dist    = cdist(pos, center)                  # [N, K] spatial distance
    spatial = exp(-dist**2 / (2*sigma**2))        # [N, K] spatial weight
    S       = softmax(alpha*sim + beta*spatial)   # [N, K] soft assignment
    mu      = (S.T @ F_flat)  / S.sum(0)[:,None]  # update region features
    center  = (S.T @ pos)     / S.sum(0)[:,None]  # update region centroids

The correspondence with FH is designed neatly: color difference → feature cosine similarity; spatial adjacency → spatial weight + centroid update; region merging → soft assignment update; internal consistency → region feature variance (usable as regularization).

A controlled experiment on a 256×170 cat image exposed three inherent defects with quantifiable metrics (FH as ground truth produced 83 regions):

MethodEffective RegionsBoundary PrecisionBoundary RecallBoundary F1
NRC (soft K-means, K=64)360.6590.2280.339
Improved (intra-variance normalization + edge-aware)640.5340.6240.576

Three defects:

  1. Boundary precision cliff: seeds initialized at 1/16 resolution mean one seed covers at least 16×16 original pixels — FH’s sub-pixel boundary advantage is gone.
  2. Missing FH’s “internal difference” dynamic mechanism: sim is an absolute similarity, with no Int(C) adaptive tolerance, causing over-segmentation in textured regions and under-segmentation in smooth ones. The improved version tries to add it: effective_dist = ||f_i − μ_k||² / (intra_var_k + τ) — regions with larger intra-variance become more tolerant.
  3. Softmax competition → seed death + blurred boundaries: under fixed K, seeds cannibalize each other (36 effective regions vs. FH’s 83), and boundary pixels “sit on the fence” (0.4/0.6).

These two rows of data tell a story by themselves: NRC is the “conservative” (high Precision, extremely low Recall — merging all the cat face details into large blocks); the improved version is the “aggressive” (high Recall but fragmented). No absolute winner, but both drift away from FH.

Side Discovery: The “Overall Color” Insight (Intrinsic Image Decomposition)

A valuable byproduct: edge-preserving smoothing of the L channel in Lab suppresses shadow-induced over-segmentation:

MethodRegionsEffect
FH Standard83Shadows cut into countless fragments
FH Robust (L smoothing)45Shadows merged, cat face more complete
FH Adaptive (edge-preserving L smoothing)53Balances shadow suppression and boundary preservation

The essence is naive intrinsic image decomposition (separating reflectance albedo from illumination shading) — elevating segmentation from “pixel color” to “surface material.” But when the same robust features were fed into NRC, the region count collapsed from 83 to 22 (seed competition death), and boundary F1 dropped from 0.60 to 0.26. A good idea diluted by a bad framework.

3.2 Form Two: RF-ViT (Region-First Vision Transformer)

The real goal surfaced: not a better segmentation, but a smarter basic image unit than the ViT patch.

First, look at the defects of existing token schemes, which explain the motivation:

SchemeBasic UnitCore Problem
ViT16×16 grid patchCuts through object boundaries — a cat and sky in one patch
SwinHierarchical windowsWindow boundaries are artificial, unrelated to image structure
SSN superpixelsSuperpixelBlurry boundaries, weak semantics, fixed region count
QuadTreeAdaptive quadtreeSubdivides spatially only, ignores content

RF-ViT’s four design pillars:

The proof of concept produced two numbers: (1) under the same token budget, region reconstruction PSNR (21.0 dB / 42 regions) significantly beat patches (13.7 dB / 35 patches), a +7.23 dB gap; (2) boundary gating suppressed cross-object attention from 0.163 to 0.000–0.002. The idea was beautiful — but it had no training script and no large-scale experiments, and the PSNR experiment was only an information-density demonstration of “reconstructing with region means,” not evidence of downstream gains.

3.3 Form Three: SAL (Structural Aggregation Layer)

Retreating from “overthrowing the architecture” to “reforming the computational primitive” — the most mature leap on the whole path:

Don’t build a new architecture; build a “smarter brick” that can be plugged into any network — like BatchNorm, ReLU, or Attention.

Extend the standard convolution to:

y_i = Σ_{j∈N(i)}  g(i,j) · w_ij · x_j
g(i,j) = σ( α·(Int(i,j) − Ext(i,j)) + β )

where Int is proxied by local standard deviation, Ext by feature Euclidean distance, and g is predicted by a tiny gating network (FHBGating). Auxiliary loss directly supervises “aggregate inside, isolate at boundaries.” This is SAL v6.

4. Honest Experimental Results

ModelTest AccuracyParameters
CNN baseline85%24,198
SAL v5 (no auxiliary supervision)35%12,298
SAL v6 (with auxiliary supervision)60%12,298

Auxiliary supervision did inflate the FH gate parameter change by ~9× (0.001→0.01), validating the diagnosis that “the gradient path is too long and needs direct supervision.” But 60% vs. 85% is a negative optimization, and it cannot be glossed over.

Why Auxiliary Supervision Was Needed: Gradient Path Analysis

The FH gate’s gradient must travel an extremely long path back from the task loss:

Loss → Classifier → ... → Conv1x1 → weighted sum → softmax → FH_Gate
         (task gradient, decays layer by layer; very weak by the time it reaches the gate)

SAL v5 had only this long path, so the gate barely moved (35%). v6 adds a short path:

Loss → FH_Loss → FH_Gate    (auxiliary gradient, reaches the gate directly)

This idea itself is correct — similar to GoogLeNet’s auxiliary classifiers and DETR’s per-layer decoder losses. But its implementation buried a fatal bug, below.

4.1 Three Engineering Defects That Must Be Acknowledged

Looking honestly at the code, part of the 60% is “crippled training signal,” not entirely a bad idea:

  1. Scalar collapse bug (most fatal): the auxiliary loss .mean()s ext_diff / int_diff across the whole batch and spatial dimensions into a single scalar before supervising. The FH gate outputs a spatial map of shape [B,9,H,W] but receives only one globally averaged push — it can never learn the position-wise “this is a boundary” signal. This is the direct reason the gate barely moved.
  2. Unbounded loss: fh_loss = -(int_diff - ext_diff) has no lower bound and can theoretically be pushed to negative infinity. Should be ReLU(ext_diff - int_diff) or clamped.
  3. Not vectorizable: FHBConv loops over 9 neighbors with Python for i in range(9), computing a local std and passing through the gate network separately for each neighbor — inefficient.

All of these are fixable. But fixing them only lifts 60% a bit — it cannot save the fundamental problem in Section 5 below.

The Correct Fix (Position-wise Supervision + Bounded Loss)

For completeness, here is how it should have been written (no longer collapsing the spatial map into a scalar):

def fh_criterion_fixed(x, gate_weights, logits, targets,
                       kernel_size=3, margin=0.1, lambda_fh=0.1):
    # gate_weights: [B, 9, H, W] (position-wise, no .mean() collapse)
    B, C, H, W = x.shape
    x_pad = F.pad(x, (kernel_size//2,)*4, mode='reflect')
    neigh = F.unfold(x_pad, kernel_size).view(B, C, kernel_size**2, H, W)
    diff  = (neigh - x.unsqueeze(2)).pow(2).sum(1)          # [B, 9, H, W]

    int_diff = (gate_weights       * diff).sum(1)           # position-wise [B, H, W]
    ext_diff = ((1 - gate_weights) * diff).sum(1)           # position-wise [B, H, W]

    # Bounded loss: penalize only when ext < int + margin; cannot diverge
    fh_loss   = F.relu(int_diff - ext_diff + margin).mean()
    task_loss = F.cross_entropy(logits, targets)
    return task_loss + lambda_fh * fh_loss

Two key changes: (1) int_diff/ext_diff are .mean()ed only after computing every position, not collapsed to a scalar first; (2) ReLU(· + margin) replaces the unbounded -(int - ext).

5. Hitting the Wall: The Community Already Did It

Strict comparison of the ideas against real literature (all papers below verified to exist):

5.1 “Regions Replace Patches” — Covered by SuiT / SPFormer

DimensionRF-ViT (this exploration)SuiT (2024) / SPFormer (2024)
Core ideaContent-consistent regions replace fixed patchesSuperpixels replace fixed patches (the exact same paradigm)
Region generationDRD: learnable seeds + iterative growth (heavy, hard to converge)Differentiable SLIC / superpixel cross-attention (one forward pass, simple)
ImageNet resultsNone (no training script)SPFormer: DeiT-S +1.1%, 22M params; SuiT also validated
Segmentation resultsNoneADE20K +4.2 mIoU

5.2 “Differentiable Adaptive Region Splitting” — Covered by DART

DART (arXiv 2506.10390, 2025): learnable region importance scores + differentiable quantization splitting, supporting both ViT and Mamba. Adds only ~1M parameters; DeiT-S + DART matches DeiT-B, FLOPs reduced 45%. Far simpler than FH gating + iterative growth, and verified.

5.3 “Differentiable Boundary Gating” — Covered by BRDG

BRDG (CVPR 2026, “Boundary-Responsive Differentiable Gating”): superpixel-level boundary gating + sparse refinement + boundary contrastive loss.

DimensionSAL v6 (this exploration)BRDG (CVPR 2026)
Gating granularityPixel-level 1×1 conv, per-pixel over its 3×3 neighborhood, no region conceptRegion-level: superpixel aggregation first, then boundary confidence per region
Refinement strategyNone, all pixels treated equallySparse refinement: boundary pixels go to fine-grained head, interiors to coarse-grained head
Contrastive learningNoneAdjacency-boosted boundary contrastive loss
Experiments100 synthetic samples, 60% vs CNN 85% (negative optimization)4 real surgical datasets, mIoU +4.5–7.0, boundary F1 +10, 150 FPS
PublicationNoneAccepted at CVPR 2026

Conclusion: SAL is essentially a degenerate subset of BRDG (pixel-level vs region-level, no contrastive learning, no sparse refinement, incomparable experiments).

5.4 The Last Gap: One Clean Closing Experiment

I once thought there was still a gap: boundary gating at the attention level (rather than the tokenizer level), i.e., A_ij = Content × Spatial × (1 − BoundaryGate). To leave no suspense, it was run as a clean closing experiment. The conclusion has two layers.

I. Even the “gap” doesn’t hold: prior-art check

A search revealed that “boundary/edge-aware attention bias” is a dense region in segmentation Transformers, not a gap:

So the “only gap” judgment was self-refuted: it wasn’t a gap — we just hadn’t run the experiment.

II. Correcting an Old Judgment: The Code Implemented It, but the Math Form Was Broken

The earlier claim of “degenerating into feature difference” was imprecise. In models/sart.py, the formula A_ij = C·S·(1−B) was actually fully implemented — the real problem was that the mathematical form was broken: it used “dot product × spatial weight × (1 − gate)” followed by L1 normalization instead of softmax — the dot product can be negative, the normalization denominator can approach zero, the probabilistic-attention semantics are lost, and the gating signal gets drowned out. The closing experiment first rewrote it in a mathematically correct form: add the boundary bias as an additive term to the logits before softmax (logit_ij = q_i·k_j/√d − λ·dist(e_i,e_j), broadcast per head, λ=softplus(·) learnable), and decouple it — attach it to verified patch tokens rather than the hard-to-converge DRD.

III. Experimental Setup

Script: train/exp_boundary_attn.py; raw results: train/boundary_attn_results.json.

IV. Results (CPU, 37.6 minutes)

ConfigurationTest Accuracy (3 seeds)ParametersLearned λ
none (standard ViT)77.00 ± 0.18%108,036—
learned (boundary gate)77.25 ± 0.85%111,1590.69 (did not grow)
random (random bias)76.69 ± 0.21%108,036—

All three readings point to the same conclusion:

  1. No statistically significant gain from gating. learned beats baseline by +0.25%, but its own variance is ±0.85 (nearly 5× the baseline’s 0.18) — the increment is drowned in noise, and gating actually made training less stable.
  2. The optimizer doesn’t want it. λ stayed frozen at its initial value of 0.69 (softplus(0)≈0.693). If gating were useful, gradients would push λ up; if clearly harmful, they’d push it to 0. It neither grew nor died — meaning on this task it was essentially inert and irrelevant.
  3. “Arbitrary structural bias” is about the same. The frozen random bias at 76.69% sits in the same band as both the learned gate and no gate, further undermining the narrative that “the gate is learning useful boundary structure” in this experiment.

V. Conclusion: No-Go

In this clean, controlled, three-seed small-scale comparison, boundary gating — corrected to a mathematically sound form and decoupled onto a healthy tokenizer — shows no statistically significant gain over a matched-parameter-count vanilla ViT, while consuming extra parameters and raising variance. Combined with the fact that prior art already covers this form, this “last gap” is now cleanly closed.

Honesty statement: synthetic data, small scale (CPU-only) — the conclusion is directional reference only, not equivalent to real benchmarks. But it at least upgraded “unverified” to “no observed benefit under controlled conditions” — closer to the truth than any assertion in the white paper.

6. Retrospective: Three Real Lessons

  1. “The direction is right” ≠ “it should be done.” Fixed patches being suboptimal — that intuition is completely correct, which is exactly why a bunch of strong teams were doing the same thing in 2024–2026. When a direction is correct enough, the question isn’t “has anyone done it” but “can my implementation beat the already-verified simplest solution.” Here the answer is no.

  2. Hand-designed complex primitives usually lose to simple modules learned end-to-end. FH gating needed a loop over 9 neighbors computing local std, and the signal was still weak; DART just needs a lightweight MLP scoring. Differentiating a beautiful classical criterion (FH) does not mean it provides a strong enough learning signal in end-to-end training — SAL v6’s 60% is empirical evidence of this warning.

  3. A “paradigm white paper” without large-scale experiments has near-zero persuasive power. No matter how elegant RF-ViT’s white paper read, without an ImageNet training script it is a castle in the air before reviewers and reality. Get it training and baseline-comparable first, then talk paradigms.

7. What Remains: What to Take Away, Where to Go Next

7.1 What Survives the Sunk Cost

7.2 Three More Pragmatic Directions

8. Conclusion

This exploration’s original question — “understand the essence of FH and CNN, design a new algorithm” — found an answer along the way that is deeper and cooler than expected:

The two paradigms can indeed be unified on the dimension of “structure-aware semantic aggregation,” and this direction is right; but the community has already implemented and verified it with simpler means. What is genuinely scarce has never been ideas — it is the evidence that something “trains through and beats the simplest baseline.”

Stopping the loss is not failure. It is moving limited time and compute from a road already walked to where there is still genuinely empty space. That, more than anything else, is the most valuable output of this exploration.

Appendix A: Three-Form Comparison Table

DimensionNRCRF-ViTSAL v6
PositioningDifferentiable segmenterRegion-tokenized ViTEmbeddable aggregation primitive
Core mechanismSoft K-means seed growthStructured triple tokens + boundary-gated attentiony_i = Σ g(i,j)·w_ij·x_j, g gated by FHBGating
DifferentiabilityYes (soft assignment)Yes (by design)Yes
Missing vs. FHInternal-difference adaptive tolerance, sub-pixel boundaries— (unimplemented)Gate degenerates to feature diff, disconnected from white paper
Training validationSingle-image comparison, boundary F1 up to 0.576No training script100 synthetic samples, 60% vs CNN 85%
Covered bySSN / differentiable SLICSuiT / SPFormerDART / BRDG

Appendix B: The Mathematical Form of Boundary-Gated Attention

RF-ViT’s envisioned attention (changes relative to standard ViT):

Standard ViT:   A_ij = softmax_j( q_i·k_j / √d )

Boundary-gated: A_ij = softmax_j( q_i·k_j / √d + log S_ij + log(1 − B_ij) )
                where S_ij = spatial proximity prior (Gaussian kernel of region-centroid distance)
                      B_ij = boundary strength between regions i, j ∈ [0,1]

Intuition: when a strong boundary exists between regions i and j (B_ij→1), log(1−B_ij)→−∞ and attention is crushed toward 0 — cross-object information flow is severed. This is semantically isomorphic to FH’s “never merge across strong boundaries,” but acting at the attention layer rather than the tokenizer layer. The closing experiment (Section 5.4) rewrote it in the mathematically correct additive-bias form and ran a three-seed comparison — the conclusion is No-Go, and this form was already covered by prior art.

Appendix C: SAL v6 Key Numbers

MetricCNN baselineSAL v5SAL v6
Synthetic accuracy85%35%60%
Parameters12,298—24,198
FH gate parameter change—~0.001 (barely moved)~0.01 (~9×)

Nearly doubling the parameters (24,198 vs 12,298) for a negative optimization is the most direct evidence that “complex primitives lose to simple modules.”

Reference Works (all verified to exist)

  1. Felzenszwalb & Huttenlocher. Efficient Graph-Based Image Segmentation. IJCV 2004.
  2. Lew et al. Superpixel Tokenization for Vision Transformers (SuiT). arXiv:2412.04680, 2024.
  3. Mei & Chen et al. SPFormer: Enhancing Vision Transformer with Superpixel Representation. arXiv:2401.02931, 2024.
  4. Yin et al. DART: Differentiable Dynamic Adaptive Region Tokenizer for Vision Transformer and Mamba. arXiv:2506.10390, 2025.
  5. BRDG: Boundary-Responsive Differentiable Gating for Superpixel-Based Segmentation. CVPR 2026.
  6. Superpixel-Aware Transformer. 2025. (encodes superpixel adjacency and spatial distance as learnable attention bias)
  7. Edge-Aware Self-Attention in Vision Transformer. CVPRW 2026.
  8. Boundary-Aware Vision Transformer. arXiv:2506.12980, 2025.
  9. Hybrid Transformer-CNN with Boundary-Aware Attention. 2026.
  10. Vaswani et al. Attention Is All You Need. NeurIPS 2017.

This article is an honest exploration retrospective; it claims no novel publishable contribution.