Model Detail
Llama-4-Scout-17B-16E-Instruct
—Llama-4-Scout-17B-16E-Instruct is an AI model with 17B parameters released by Meta. And supports text+image->text inputs, distributed under a other license.
Llama-4-Scout-17B-16E-Instruct is priced at $0.2/M input tokens and $0.78/M output tokens. Operationally the model offers a 10000K-token context window, which matters when sizing it for prompt-heavy or latency-sensitive workloads. At this input rate the model sits in the commodity tier and is suitable for high-volume workloads where per-call cost dominates the decision.
Llama-4-Scout-17B-16E-Instruct ships with 17B parameters. The published knowledge cutoff is 2024-08-31, so newer events will not be reflected in zero-shot answers without retrieval. Total weight footprint is approximately 108.6 GB, which is the relevant figure when planning local-inference VRAM. Access is gated on Hugging Face under the other license, which means a manual approval step before weights can be downloaded.
Llama-4-Scout-17B-16E-Instruct is best fit for general-purpose AI workloads, high-volume batch jobs where per-call cost dominates the budget, and long-context tasks such as full-codebase analysis or book-length summarization (10000K tokens). Treat this as a starting matrix rather than a benchmark verdict — the right deployment usually depends on the specific evaluation suite that mirrors your workload.
FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
arXiv:2610.23796v1 Announce Type: new Abstract: Transformer prefill reuses weights across tokens and is dominated by matrix multiplication and activation data movement; single-token decode has limited reuse and is dominated by weight and KV-cache reads. We present FUTRANSFORMER, a CUDA implementatio
MeddeX: A Retrieval-Augmented LLaMA-3 System with Gemini Vision for Evidence-Backed Medical Report Analysis
arXiv:2610.20632v1 Announce Type: new Abstract: Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual review processes are slow, error-prone, and difficult to scale. This paper presents MeddeX, a Medica
Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules
arXiv:2610.19210v1 Announce Type: new Abstract: Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches. Efficient recurrent alternatives reduce the asymptotic complexi
Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
arXiv:2608.21134v2 Announce Type: replace-cross Abstract: Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained hardware. Our approa
Transformers now runs llama.cpp quants
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
arXiv:2609.14864v1 Announce Type: new Abstract: We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configuratio
