TOPLOC: A Local Sensitivity Hashing Scheme for Trusted Verification Inference
TOPLOC detects unauthorized modifications to models, prompts, or computational precision through local sensitivity hashing mechanism, verification speed is 100 times faster than original inference, and submission storage for every 32 new tokens only requires 258 bytes (compared to 262 KB for Llama-3.1-8B-Instruct). This solution maintains robustness across various hardware configurations, GPU types, tensor parallel dimensions, and attention kernel implementations, reducing memory overhead by 1000 times and is suitable for large-scale deployment.

