Model Detail
Llama-3.1-70B-LatamGPT-SFT-1.0
—Llama-3.1-70B-LatamGPT-SFT-1.0 is a large language model with 70B parameters released by latam-gpt. The model is registered under the text-generation pipeline tag on Hugging Face, released under the llama3.1 license.
Llama-3.1-70B-LatamGPT-SFT-1.0 ships with 70B parameters. Total weight footprint is approximately 70.6 GB, which is the relevant figure when planning local-inference VRAM. Distribution is governed by the llama3.1 license — review the exact terms before commercial deployment.
Llama-3.1-70B-LatamGPT-SFT-1.0 is best fit for general-purpose chat and instruction-following workloads. Treat this as a starting matrix rather than a benchmark verdict — the right deployment usually depends on the specific evaluation suite that mirrors your workload.
FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
arXiv:2610.23796v1 Announce Type: new Abstract: Transformer prefill reuses weights across tokens and is dominated by matrix multiplication and activation data movement; single-token decode has limited reuse and is dominated by weight and KV-cache reads. We present FUTRANSFORMER, a CUDA implementatio
MeddeX: A Retrieval-Augmented LLaMA-3 System with Gemini Vision for Evidence-Backed Medical Report Analysis
arXiv:2610.20632v1 Announce Type: new Abstract: Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual review processes are slow, error-prone, and difficult to scale. This paper presents MeddeX, a Medica
Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules
arXiv:2610.19210v1 Announce Type: new Abstract: Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches. Efficient recurrent alternatives reduce the asymptotic complexi
Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
arXiv:2608.21134v2 Announce Type: replace-cross Abstract: Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained hardware. Our approa
Transformers now runs llama.cpp quants
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
arXiv:2609.14864v1 Announce Type: new Abstract: We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configuratio
