Model Detail
Llama-Prompt-Guard-2-86M
—Llama-Prompt-Guard-2-86M is a large language model with 139M parameters released by Meta. The model is registered under the text-classification pipeline tag on Hugging Face, distributed under a other license.
Llama-Prompt-Guard-2-86M ships with 139M parameters. Access is gated on Hugging Face under the other license, which means a manual approval step before weights can be downloaded.
Llama-Prompt-Guard-2-86M is best fit for general-purpose chat and instruction-following workloads. Treat this as a starting matrix rather than a benchmark verdict — the right deployment usually depends on the specific evaluation suite that mirrors your workload.
FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
arXiv:2610.23796v1 Announce Type: new Abstract: Transformer prefill reuses weights across tokens and is dominated by matrix multiplication and activation data movement; single-token decode has limited reuse and is dominated by weight and KV-cache reads. We present FUTRANSFORMER, a CUDA implementatio
MeddeX: A Retrieval-Augmented LLaMA-3 System with Gemini Vision for Evidence-Backed Medical Report Analysis
arXiv:2610.20632v1 Announce Type: new Abstract: Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual review processes are slow, error-prone, and difficult to scale. This paper presents MeddeX, a Medica
Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules
arXiv:2610.19210v1 Announce Type: new Abstract: Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches. Efficient recurrent alternatives reduce the asymptotic complexi
Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
arXiv:2608.21134v2 Announce Type: replace-cross Abstract: Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained hardware. Our approa
Transformers now runs llama.cpp quants
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
arXiv:2609.14864v1 Announce Type: new Abstract: We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configuratio
