Odin (system name in the body: THEMIS): an open-source FHE inference implementation for Llama-3-8B

0. At a glance

One-line positioning: THEMIS (the workspace filename and the survey records call it Odin; the system name in the paper body is THEMIS; the real names of the open-source repositories are Homie/Holmes, see §2.0) is an implementation that uses GPU-accelerated CKKS to run the entire 32-layer forward pass of Llama-3-8B on ciphertext — but it only does prefill: 128 tokens, a single H100, 366.4 seconds, with no autoregressive decoding.

Three facts to remember first: (1) the body of the paper's contribution chapter §4 is empty; §5.3, §5.4.1 and §5.4.2 likewise contain only headings; and the PPL of the actual FHE execution in Table 10 is the placeholder XXX; (2) the denominator of 4.51× is the authors' own THOR adaptation, whose code "remain outside this release tree", so this number has exactly one source — the authors' self-report; (3) the system's real engineering detail is richer in the repository than in the paper — wherever this note discusses a mechanism, it marks whether the evidence comes from the "paper body" or from the "repository docs".

1. Problem and positioning

1.1 Why packing is the first-order problem when running LLMs on CKKS

The cost of homomorphic inference is determined by three things, and all three are driven by the packing scheme:

1. Ciphertext count: how many ciphertexts a 4096-dimensional token vector occupies determines the number of multiplications and the number of rotations.
2. Homomorphic cost of the linear layer: in CKKS a matrix multiplication must be expanded into a MAC (multiply-accumulate) sequence of "plaintext diagonal × ciphertext rotation"; whether weight encodings can be reused directly determines the size of this term.
3. Layout-conversion cost: the layout wanted by a linear layer, the layout wanted by attention, and the layout wanted by slot-wise non-linear operations all differ; every layout change is a rotation/permutation, and the deeper the model and the longer the sequence, the more this term accumulates.

For the second term the paper gives a very concrete diagnosis (§3.4, the most solid section in the whole paper): in the THOR-style page-diagonal representation, a single plaintext weight object is simultaneously bound to four physical attributes — input page, output group, real/imaginary part, and rotation copy — and the same pre-encoding can be reused only when all four attributes are identical; on Llama-3 with a 4096 hidden dimension + 14336 FFN, each layer needs 425,984 plaintext weight objects. One must therefore either cache pre-encodings (costing storage and residency) or repeatedly pack/encode on demand (costing time). Table 2 gives the consequence: weight pack + encode accounts for 28.02 s of the 44.10 s linear time (63.5%), of which encoding alone accounts for 38.6% of the linear time.

This is not a "not enough compute" problem but a "the same logical weight cannot amortize its encoding cost over multiple compatible input ciphertexts" problem. THEMIS's entire design starts from this diagnosis.

1.2 Where existing systems stand

§1.1 of the paper splits related work into two lines, a classification worth lifting directly into the survey's coordinate system:

• MPC / hybrid line: Iron (secure matrix multiplication with HE packing + protocols for Softmax/GELU/LayerNorm), PUMA (two-party LLaMA-7B), BOLT (joint optimization of secure matrix multiplication and non-linearities), BumbleBee (low-communication matrix multiplication + activation protocols). The shared cost is multi-round online communication and the assumption that the parties do not collude.
• Pure HE line: NEXUS (studies several packings and their conversion costs but does not put conversion cost on the end-to-end path), THOR (diagonal-major + compact packing, BERT-base on a single GPU), MOAI (coordinates packing across modules to reduce layout transitions and rotations), Park et al. (Llama-2-7B, distinguishing public from encrypted tokens to support longer inputs).

What the pure HE line lacks: an LLM-scale implementation that puts the packing contract, layout transitions and bootstrap scheduling on one end-to-end path. THOR achieved single-GPU pure HE but only up to BERT-base with sequence length 128; MOAI handled layout transitions but did not reach LLMs; NEXUS analysed conversion costs but did not run end to end. THEMIS's positioning is to fill this gap — at the price of filling only the prefill part.

1.3 The paper's three claims (and where they land in the body)

The abstract gives three: (a) a feature-major cross-layer layout that unifies the residual stream and the inter-layer interface; (b) short-lived intra-operator layouts matched to the access patterns of linear layers and attention, thereby reducing redundant plaintext weight encodings in the wide projections; (c) non-linearities via minimax polynomial approximation on an interval + input-range control + joint error allocation guided by model quality.

The problem is that the substance of (a) and (b) belongs in §4, and §4 is empty. (c) lands in §5.2, which is complete and quite well written; the performance evidence lands in §6. So §2 of this note will state explicitly which sentences I read from the paper body and which I read from the repository docs.

2. Anatomy of the method

2.0 Resolving the naming first: four names, one system

The paper gives no repository links because of double-blind review: the holmes README reads "This artifact has been anonymized for paper review... Upon paper acceptance, we will publish the original repository link to a verified release." (survey records). This also explains the naming confusion: the real system is the Homie runtime + Holmes multi-GPU (the H family), and Odin/THEMIS is the paper codename.

Two evidentiary disciplines, strictly observed for the rest of this note:

• "Paper body" = sentences citable from the PDF body, tagged with a section number.
• "Repository" = the authors' repository docs/DESIGN.md / docs/CODE_MAP.md / code comments, with the source tagged at the end of the sentence.

2.1 The canonical packing: a feature-major state — mostly from the repository

All the paper body offers is a functional description: §6.4 says that when 32 layers are chained, "each layer receives/sends FM ciphertexts, with no extra layout conversion between layers", and a layer's output is still 16 real FM ciphertexts; the abstract says this layout "unifies the residual stream and the inter-layer interface".

The exact contract is in the repository (docs/DESIGN.md):

FM[c, slot = 8*d + u] = H[token = 8*c + u, feature = d]
 
    c = 0..15      ciphertext index (token shard)
    u = 0..7       token offset within a shard
    d = 0..4095    feature (channel)
 
128 tokens = 16 shards * 8 tokens/shard
32768 slots = 8 tokens * 4096 features
128 * 4096 = 524288 = 16 * 32768

Each residual boundary is 16 real (non-complex) FM ciphertexts (docs/DESIGN.md). Adjacent token shards are temporarily paired into complex numbers z = x_even + i * x_odd inside the linear kernel, yielding 8 complex inputs — this is private to the operator and does not change the public cross-layer state (docs/DESIGN.md).

This design has three immediate consequences worth remembering:

1. The ciphertext count is fixed at 16, independent of layer width; it scales linearly with sequence length (exactly filling for S=128).
2. A plaintext weight diagonal aligns "8 tokens at once", so one encoding can be reused by several of the 16 shards — precisely the reuse chain that THOR cannot achieve and whose absence the diagnosis identifies.
3. Slot-wise non-linearities (SiLU, exp, rsqrt) need no permutation on FM; only attention needs one.

One terminology trap: §6.1 of the paper writes "ring dimension is N = 2^16", while docs/DESIGN.md writes "N = 32768". In CKKS, a ring dimension of 2^16 gives 32768 complex slots, so the two can be reconciled (128 * 4096 = 16 * 32768 is also self-consistent), but the two places use the same letter N for different quantities. When citing, either say ring dimension or say slot count; do not mix them. (This reconciliation is this note's own cross-check; neither the paper nor the repository states it.)

2.2 Linear layers: BSGS MAC + complex packing — skeleton in the paper, configuration in the repository

The paper body first clarifies a commonly confused point (§3.3.1): "BSGS is an evaluation strategy that reuses some rotations over a given layout rather than an independent packing scheme". That is, BSGS decides "how many rotations expand one matrix multiplication", but how many weight encodings can be saved is decided by packing. This distinction is the key to reading the paper's Method chapter (even though it is empty).

The performance evidence the paper body offers (§6.2): on the PC projection of Layer 0, the pack/enc share drops from 73% to 20% and the homomorphic evaluation itself from 10.03 s to 4.25 s (2.4×) — note that the latter is only 2.4×; the overall 7.8× speedup comes mainly from "packing less". On the static operand count, the number of weight encodings for QKV and gate/up drops to 1/16 of "diagonal packing without reuse", and W_O and down to 1/8 (the paper states this is a static comparison against the formulas, not a measured end-to-end ratio).

The repository gives the skeleton of "one linear engine, four invocation configurations" (docs/DESIGN.md):

The W_O / down column can be worked out by hand: if z = x_e + i*x_o and the weight W is real, then z * W = (W x_e) + i*(W x_o) — the real and the imaginary part are exactly the true results of the two token half-batches, so one ciphertext-plaintext multiplication buys both. The weights in the QKV column are themselves complex, which implies a further rearrangement inside the complex domain (a complex multiplication mixes x_e and x_o across real and imaginary parts), but the docs do not spell out the sign and conjugation conventions; confirming this requires reading operators/linear/qkv.py; this note does not speculate.

Policy ownership (docs/DESIGN.md): baby step, anchor step, hoist strategy, plaintext chunk size and cache policy all belong to LinearOperatorConfig and the operator instance; weights do not own runtime policy; baby rotations are reused across plaintext chunks.

2.3 Attention: the C–Δ layout and "no score repacking" — one sentence in the paper, one table in the repository

The only direct evidence for this mechanism in the paper body is a cost comparison (§6.2): besides CC time, THOR spends 0.54 s adjusting the attention input layout and 0.85 s repacking the output of diagonal Softmax into the probability copies that PV needs; THEMIS's S_Δ → P_Δ → PV path eliminates the latter while retaining the former (QKV input adaptation) and one PairFM conversion at the attention output. The abstract's own words are that QK^T inside attention produces scores Softmax can consume directly, and that PV consumes the probabilities directly.

The layout contract is in the repository (docs/DESIGN.md); note that _C is a persistent layout and _Δ a temporary layout indexed by relative displacement:

Q_C[d, q, lane]
K_Δ[d, q, lane] = K[(q + d) mod S, d, lane]
V_Δ[d, q, lane] = V[(q + d) mod S, d, lane]
S_Δ[delta, q, lane]
P_Δ[delta, q, lane]
O_C[d, q, lane]
PairFM[(2*d + lane), q]

After K and V are sheared by d positions along the query dimension, scores naturally appear indexed by delta = q - k; Softmax performs its row reduction on that delta axis, and PV reduces over the same axis, so the intermediate step of "converting the score array into the probability copies that PV needs" is simply unnecessary — the docs' own words are "never repacks scores" — and the hop from S_Δ to P_Δ needs no score permutation. The PairFM output converter performs one structured owner transpose, after which W_O returns to the persistent FM state (docs/DESIGN.md).

The conversion points inside one layer (docs/CODE_MAP.md, in data-flow order):

input RMSNorm (fused with the inter-layer bootstrap)
  → complex-token QKV diagonal linear
  → layout transform (lane/phase permutation, gathers, key shear)
  → QK kernel in Δ form
  → Δ-native Alg2 softmax
  → Δ PV kernel
  → PairFM output conversion
  → feature-major W_O + residual
  → post-attention bootstrap + RMSNorm (fused)
  → feature-major SwiGLU MLP + residual

How many rotations each delta of S_Δ/P_Δ needs, and the exact lane/phase arrangement — the docs give the contract but not the rotation counts; the static cost model is in operators/attention/audit.py (docs/CODE_MAP.md). Do not look for these numbers in the paper — the paper does not report them.

2.4 Non-linearities and normalization: the most complete section of the paper (body §5.2)

This part is complete in the paper body and can be cited directly. The non-linear operator layer assembles Softmax and SiLU from three building blocks — exp, reciprocal, rsqrt — and SiLU is then used adjacent to RMSNorm (docs/DESIGN.md). Below I follow the paper's order.

a) Starting point of the soft error budget: τ0 = 2⁻¹² (a 12-bit starting point, following Park et al.). §3.2.1 gives bounds in which the minimax error grows linearly with the interval length and as τ^(−1/s) with the tolerance (sufficient but not necessary); §3.2.3 shows that 12 bits is in fact conservative: even with δPPL pinned at 0.001, the shared tolerance can be relaxed to 2.258×10⁻³ (9.25×).

b) Input-range control (§3.2.2): a WikiText-2 vs C4 comparison was run first; the ratio of the p99 amplitude to the median at operator positions is 1.0, showing no global scale drift between the two corpora, but intervals are still set by calibration. Two kinds of outliers are handled separately: attention sinks (token dimension; a fixed sink prefix is prepended, and the KV can be precomputed as a public static cache) and QuaRot's orthogonal Hadamard rotation (feature dimension; can be folded offline into the public weights, adding no online homomorphic operations). The authors stress that the two are not substitutes for each other.

c) Softmax uses Alg1 for "doubling reduction" (§5.2.1; the wide/narrow track structure follows Cho et al.):

k = ceil( log2(M_hat) - log2(ln n) )
z = x - U_hat
u(0) = p_exp(z / 2^k)
for j = 1..k:
    s(j)      = sum_i ( u_i(j-1) )^2        # reduce to the auxiliary track
    lambda(j) = p_rsqrt( s(j) )             # narrow track: rsqrt on a few slots only, then BS
    u(j)      = ( lambda(j) * u(j-1) )^2    # wide track: slot-wise squaring
Softmax(x) = u(k)

Key points: underestimating U_hat produces out-of-range values, while overestimating it widens the exponent range and increases k; most layers share U_hat and k, Layer 31 uses a head-wise U_hat with k = max_h k_h, the goal being to keep the number of iterations consistent within a SIMD batch. Actual k per layer: 2 for most middle layers, 4 for Layers 0–1, 3 for Layers 2 and 14, and 5 for Layer 31 (§6.4).

d) Error allocation uses the conditional bisection of Alg2 (§5.2.3): the criterion is the PPL change after injecting a perturbation, ΔPPL_inj(τ) ≤ δPPL (Eq 9); following the priority π = [SiLU, RMSNorm2, RMSNorm1, Softmax], a conditional upper bound is found by bisection for each operator in turn (Eq 10). Why SiLU comes first: it sits on the gate branch of the 14336-wide FFN, and once its degree crosses a multiplicative-depth boundary it simultaneously lowers the cost of the non-linearity and of the adjacent wide linear layer, whereas lowering the degree of RMSNorm does not necessarily move the bootstrap position. Table 4 (δPPL = 0.001) results:

e) Polynomial generation (§5.2.4): since the degree cannot be solved inversely from τ, the procedure increments the degree step by step and checks the maximum error at the endpoints (criterion max |f − p| ≤ τ_i), and after the first satisfaction checks whether the adjacent lower degree also satisfies it. Gains (against the 12-bit starting point): over 32 layers the sum of SiLU degrees goes 2048 → 736 and the sum of multiplicative depths 199 → 155; over all non-linearities the degree count goes 24,448 → 22,694 and the depth 1,376 → 1,290 (Table 7). In online time, the four tunable operators together go 0.49 → 0.23 s, and the complete FFN 4.30 → 3.98 s (Table 8).

f) An honest boundary (§5.2.5): the absolute error of a single operation is about 10⁻⁴ (10⁻⁶ in the high-precision configuration), below the 10⁻³–2×10⁻² approximation budget — but the authors explicitly say this is a single-point result and cannot replace a whole-model accumulated-error evaluation, so it was not put into the search; only the Poly-only vs FHE differential is used to measure. This way of "knowing where one has no evidence" is rare in this paper and worth noting.

g) Different degrees per layer (repository supplement, not stated in the body): Layer 0 uses degree-16/32 RMSNorm and degree-48 SiLU, Layer 31 uses degree-128/128 and degree-384 (docs/DESIGN.md). In other words, "how many degrees each layer uses" is not a global constant but a per-layer entry parameter.

h) Fusion of RMSNorm and bootstrap (repository): the post-attention refresh and the RMSNorm immediately following it are combined into one explicit fused operator, the reason being to keep the temporary complex-pairing split scale from becoming a cross-operator contract; Alg2's Softmax owns its own λ bootstrap operator (docs/DESIGN.md).

2.5 Bootstrap: the layer obscured by §5.4

The paper gives only two sentences on bootstrap (§6.4): after the MLP residual completes, one inter-layer bootstrap restores the limb budget needed by the next layer's RMSNorm and its subsequent computations; for individual layers whose Softmax k is large, an extra in-layer bootstrap is added. §5.4.2 "Limb-Budget-Based Bootstrapping Schedule" has only a heading, no body. (paper body)

The repository fills in the mechanism (docs/DESIGN.md, docs/CODE_MAP.md):

• All layers share one CKKS context, one rotation-key union and one bootstrap program; a complete Layer 0 creates the modulus capacity its real schedule needs (the currently validated configuration is depth-41, while the paper body writes modulus chain depth 37 — the two figures may be "capacity" and "chain depth" respectively; this note does not merge them).
• drop_to_limbs / drop_levels only lower the modulus and do not consume depth; they are used to align the level before and after bootstrap.
• The two-pass correction can silently degrade (item 2 of "Known rough edges" in docs/CODE_MAP.md): _bootstrap_with_optional_correction wraps the whole iterations=2 correction in except ValueError: return first, so that if any align_to rejects the target state it returns the uncorrected single-pass ciphertext; the comment's own words are "That is a numerics change reported nowhere". And two passes are the production default for residual/softmax checkpoints (--softmax-bootstrap-iterations 2).
• The only per-context adjustment in Llama3Model.forward replaces qk_output_level with None, because "an absolute Layer0 alignment target does not survive a bootstrap". The sentence itself shows how fragile the level scheduling is.

Implication for reproduction: any change to the limb budget (a different CKKS parameter set, a different polynomial degree) requires re-validating this silent-degradation path first; the authors' smoke suite cannot see limb arithmetic (docs/CODE_MAP.md).

2.6 Engineering stack

There is a small gap in dependency attribution: §6.1 of the paper mentions only EasyFHE, and the name Homie never appears in the paper (confirmed by grep); in the repository, operators call the public API of homie.fhe directly (docs/DESIGN.md). When citing the engineering stack, write "the Homie runtime on top of EasyFHE", not "EasyFHE alone".

3. Table of key numbers

Discrepancies that only surface once you compute them yourself (this note's cross-check; the paper does not explain them): 366.4 / 32 = 11.45 s/layer, consistent with 11.12 s in Table 9; but 1651.9 / 32 = 51.6 s/layer, 15% below the 60.87 s baseline of Table 9 and also below the 59.24 s of Table 1. The gap on the baseline side may come from per-layer k differing or from the timing methodology (Table 1 takes three runs, the end-to-end number is a single run); the paper gives no explanation.

4. Evidence grading and reproducibility

This section is the focus of this note. When reading this paper, first decide which layer of evidence you are reading.

4.1 The paper's list of holes: more than Chapter 4

In other words: the method chapter (§4) as a whole + two key subsections of the implementation chapter (§5.3, §5.4) + the only end-to-end quality number (the FHE row of Table 10) are all absent. This is not "one section was left unfinished" but "the paper's contribution hole + implementation hole + quality hole" appearing simultaneously. What the authors did leave, §5.2 (error allocation), is conversely the most complete part — that section could stand alone as a paper.

Reading it as a "damaged manual" is the only workable approach: the body gives conclusions and numbers, the repository gives mechanisms.

4.2 Why 4.51× is not reproducible

Three reasons, each harder than the last:

1. The denominator is not THOR's published number. The original THOR paper reports 602.26 s, but that is BERT-base, 128 encrypted tokens, 1× A100 (see Table 11 of this paper). THEMIS's denominator is 1651.9 s, the authors' own THOR-style Llama-3-8B baseline. Mixing the two numbers yields a completely wrong ratio.
2. That baseline is outside the release. The last sentence of docs/DESIGN.md reads: "The old THOR and LD implementations remain outside this release tree." No code, no configuration, no per-layer logs: 1651.9 s cannot be audited.
3. Even the baseline's time is not self-consistent. As computed at the end of §3: 51.6 s/layer vs the 60.87 s/layer of Table 9, a 15% gap the paper does not explain.

So the accurate statement of 4.51× is: "a ratio self-reported by the authors against their internal THOR-adapted baseline". It cannot be recomputed by a third party, nor used as a cross-paper comparison between THEMIS and the THOR system. The paper itself concedes in §6.6 that the cross-system comparison of Table 11 is unusable (hardware, models, encryption intervals and timing boundaries all differ) — the authors are honest on this point; the problem is only that 4.51× is too easy to quote in isolation.

4.3 Three reproduction thresholds: weights, hardware, scope

• Weights: you must use the QuaRot-converted Meta-Llama-3-8B (§5.1 folds the Hadamard before O and down into the adjacent weights; calibration, polynomial generation and FHE execution all use the same converted model). The original Meta-Llama-3-8B weights cannot run 32 layers directly (survey records). "Clone and run" does not hold.
• Hardware: the paper reports a peak of 58.9 GiB → requiring the 80 GB tier; but the repository README lists two profiles, 32 GB and 80 GB, while the CI steady gate asserts running on an L40 (48 GB) (docs/CODE_MAP.md). The three tiers are mutually inconsistent, and how the 32 GB tier works under a 58.9 GiB peak is stated by neither the paper nor the repository docs.
• Scope: prefill only (128 tokens). Decoding lives in another repository, holmes (multi-GPU, 2026-10-14), outside this paper's measurement scope; therefore 366.4 s excludes autoregressive decoding, excludes the cost after KV cache growth, and excludes client-side encryption and decryption time (§3.1 explicitly excludes client-side preprocessing and encryption from FHE time).
• License: the repository holds LICENSE.TBD (survey records).

4.4 How to cite this paper

What you can write (supported by the body, with conditions):

1. "THEMIS reports an end-to-end CKKS forward pass of Llama-3-8B at 128-token prefill on a single H100: 366.4 s with a peak of 58.9 GiB (authors' self-report)."
2. "The work reports 4.51× against its own THOR-adapted baseline, but that baseline is outside its release, so the ratio cannot be independently reproduced."
3. "The work locates the bottleneck of end-to-end Llama-3-scale homomorphic inference in plaintext weight encoding (63.5% of its baseline's linear time), a diagnosis based on a recomputable per-operator decomposition (Tables 1–2)."
4. If you want to cite the packing contract at the method level, you must also note that "the method chapter body is missing; the contract is in the authors' repository docs/DESIGN.md".

What not to write:

• ✗ "Odin/THEMIS is 4.51× faster than THOR" — the 602.26 s of the THOR paper is BERT-base on different hardware; the two numbers cannot be multiplied or divided.
• ✗ "implements complete FHE inference for Llama-3-8B" — prefill only, no decode.
• ✗ "the first open-source Llama-3 FHE implementation" as an unconditional factual assertion — the authors' own words qualify it with "to the best of our knowledge", and the existence of Holmes shows that the system is still evolving.
• ✗ Any sentence citing an end-to-end FHE PPL — that cell is XXX.

4.5 Counter-evidence: why this does not look fabricated

A balancing point that is easy to overlook: the documentation proactively discloses its own numerical defects. docs/CODE_MAP.md has a "Known rough edges" section whose first item self-reports the silent degradation of the two-pass bootstrap correction ("That is a numerics change reported nowhere"); Table 9 honestly lists two items that got slower (RMSNorm 0.4×, layout conversion 0.7×); the references still contain local notes such as "Local reference PDF; final publication metadata to verify before submission" and "PDF: references/QuaRot.pdf".

Together these features point to one judgment: this is an uncleaned submission draft whose engineering is far more complete than its paper. The empty §4 is a real defect (a hard blow for both reviewability and citation), but it is a different kind of problem from "data fabrication"; the evidence in this note does not support the latter, and it should not be inferred from it.

"The paper is a damaged manual for the engineering" — this is my most condensed assessment of this material: to learn the method, read the repository; for a quick sense of positioning and ceiling, read §3 and §6 of the body.

4.6 Open-questions list (item by item)

5. Relation to the sibling papers

Three notes in this series form a progression from "classification" to "generation", and THEMIS stands at the deepest position:

• The survey (./01-private-transformer-inference-survey.mdx) provides the coordinate system. Its two main lines are "MPC interactive" and "HE non-interactive", and it stresses that the bottleneck migrates with the setup and that PTI cannot be measured in FLOPs. THEMIS belongs to the far end of the HE non-interactive side: zero online interaction, a single server, no collusion assumption — the price being traded for three new things: the complexity of the layout contract, the layered scheduling of bootstrap, and "prefill only". Seen through the survey's framework, THEMIS is one more instance of "bottleneck migration": it moves the bottleneck from the linear-layer multiplication of the BERT era to packing and bootstrap.
• BOLT (./02-bolt-mpc-he-inference.mdx) is its counter-reference. BOLT takes the 2PC hybrid route, jointly optimizing secure matrix multiplication and non-linear protocols, and still needs continuous communication between the two parties online. THEMIS's route is "replace all interaction with server-side rotations and bootstraps". The two give two cost structures for the same arithmetic problem: one pays in network round trips, the other in H100 seconds. The survey's section on "the essential difference between MPC and HE" can use exactly this pair as a concrete contrast.
• THOR (./03-thor-ckks-inference.mdx) is its direct predecessor. Diagonal-major + compact packing, BSGS deferred rotations in the rearrangement, and the PC-MM/CC-MM distinction are all THOR's legacy. THEMIS's contribution can be summed up in one sentence: when THOR's approach is pushed to LLM scale, the reuse granularity of packing is insufficient (425,984 plaintext objects per layer), so the design switches to feature-major + complex packing, bringing the encoding count down to 1/16 or 1/8. Conversely, THEMIS does not overturn THOR's mathematics; it changes the answer to the question "which physical attributes must coincide for an encoding to be reused".
• Another pure-HE branch: Park et al.'s Llama-2-7B takes coefficient encoding (together with slot encoding), at the cost of needing the homomorphic DFT pair SlotToCoeffs / CoeffsToSlots, which consumes level; §3.3.1 says EasyFHE has no complete path for this, so THEMIS only compares in the slot domain. "Coefficient-encoding Llama-2-7B" and "slot-domain THEMIS" are two technical routes for pure-HE LLMs, and deserve to be listed as a pair in the survey rather than as two points on the same route.
• MOAI likewise aims to reduce layout conversions but stops at BERT-base; THEMIS's Δ layout and MOAI's retention strategy are different solutions to the same problem and can be placed side by side.

6. Limitations and open questions

6.1 Limitations (by severity)

1. The method chapter is missing, damaging reviewability. §4 is empty, §5.3/§5.4 are empty, and the method can only be inferred backwards from the repository docs. Any reviewer would demand a resubmission on this ground.
2. No end-to-end quality number. The FHE row of Table 10 is XXX. The subsystems have three pieces of evidence (3 next-token matches, Poly-only PPL 12.79 vs Exact 12.67, single-operation error 10⁻⁴), but none of them constitutes proof that "end-to-end FHE PPL is close to the exact value". The ΔPPL of 0.119 is computed on the polynomially replaced model, not a result of FHE execution.
3. 4.51× is not auditable (§4.2).
4. Prefill only. 128 tokens fixed, no decode, KV cache growth outside the measured scope. For a title that says "end-to-end", this is a substantive gap in scope.
5. The two slowdowns are unexplained. RMSNorm 0.4× and layout conversion 0.7× — the paper lists them honestly but does not analyse why (possibly RMSNorm on FM requires a reduction across 16 ciphertexts, while THOR's layout is cheaper for some small operators). This precisely shows that "THEMIS wins on linear layers, not on conversions and normalization".
6. Numerical stability depends on a path that can silently degrade (§2.5). Changing the limb budget requires re-validation.
7. Narrow security boundary: it does not protect server weights, does not defend against a malicious server / side channels / DoS / access patterns; S is fixed at 128; the client must do embedding and final decoding — all of this is stated by the paper itself (§3.1) but is often ignored when citing.
8. The cross-system table is not comparable: the authors have declared it (§6.6), but Table 11 is still easily misread as a ranking.

6.2 Open questions

• What exactly are the rotation count per delta and the gather cost in operators/attention/audit.py? (not reported in the paper; the repository has a static model)
• What are the conjugation/sign conventions of the QKV complex carrier in operators/linear/qkv.py? Why does QKV use complex weights while W_O/down use real weights?
• The exact configuration of the 32 GB profile (smaller S? fewer layers? only some layers?) and its relation to the 58.9 GiB peak.
• How many bootstraps per layer, and how many in total? The paper gives this number nowhere.
• holmes's decode performance and KV cache strategy — if decode can also run, then the "prefill only" limitation downgrades from "a limitation of the system" to "a limitation of this paper's measurement scope".
• If δPPL were relaxed from 0.001 to 0.010, how much end-to-end time would be saved? Table 3 suggests the tolerance can go to 9.84×, but the paper does not run this experiment.
• End-to-end user-perceived time (including client-side encryption and decryption, and the network) — explicitly excluded by the paper, but this is the number that matters most in a real deployment.
• Does the measured wall clock of the 32-layer full gate agree with 366.4 s? The repository's full test asserts only the next token and a "wall-clock band", giving no number.

7. Self-test questions

Q1. The paper locates the end-to-end bottleneck in weight encoding — on which two numbers is this based?

A: weight pack + encode takes 28.02 s of the 44.10 s of the linear layer (63.5%), of which encoding alone accounts for 38.6% of linear time; the root cause is that under the page-diagonal representation each layer needs 425,984 plaintext weight objects, so the encoding cost of the same logical weight cannot be amortized over multiple compatible input ciphertexts (§3.4, Table 2).

Q2. Why is "16 real FM ciphertexts per layer" a meaningful constant? How is it consistent with 4096 dimensions and 128 tokens?

A: FM[c, slot=8*d+u] packs 8 tokens and 4096 features into one ciphertext, 8×4096 = 32768 = the slot count; 128 tokens need 16 shards, so 16 ciphertexts exactly tile the whole residual stream (128×4096 = 16×32768). It is a constant rather than a quantity that varies with layer width, which is what makes "no extra layout conversion between layers" possible (docs/DESIGN.md, §6.4).

Q3. Why can attention "not repack scores"? Which permutation is saved?

A: K/V have been sheared along the query dimension (K_Δ[d,q,lane] = K[(q+d) mod S, d, lane]), so scores naturally appear indexed by delta = q−k; Softmax's row reduction and PV's reduction share the same delta axis, so S_Δ → P_Δ needs no permutation; what is saved is precisely THOR's 0.85 s of "diagonal Softmax output → the probability copies PV needs". What is retained is the QKV input adaptation (0.54 s) and one PairFM conversion at the output (§6.2, docs/DESIGN.md).

Q4. Why does Softmax's rsqrt not take part in the tolerance relaxation?

A: each round of Alg1 uses p_rsqrt(s(j)) to normalize the magnitude, and this step necessarily bootstraps, so its depth is not affected by "polynomial degree"; since relaxing the tolerance cannot remove this bootstrap anyway, relaxing it brings no gain. Hence j1/j2 are fixed at 2⁻¹² (§5.2.3, Table 4).

Q5. Why is 4.51× not reproducible? Put it into one citable sentence.

A: the denominator (1651.9 s) is the authors' own THOR-style Llama-3-8B baseline, whose implementation "remain outside this release tree" and therefore cannot be audited; and it is a different object from the 602.26 s of BERT-base in the original THOR paper. "THEMIS reports 4.51× against its own baseline, which is outside the release scope."

Q6. How should this paper's "evidence grading" be remembered? Give the three layers.

A: the first layer is citable from the body (§3 diagnosis, §5.2 error allocation, §6 performance tables, Tables 3/4/7/8/9); the second layer is verifiable in the repository (the FM contract, the attention layout, carrier semantics, bootstrap fusion, the silent degradation of the two-pass correction); the third layer is self-reported only (the denominator of 4.51×, the configuration of the 32 GB profile, the next-token test of end-to-end correctness). The FHE row of Table 10 belongs to none of the three layers — it is empty.

8. Index of locations in the original