goodput

Optimizing Large Language Model Serving: The Critical Shift from Throughput to Goodput

The rapid ascent of Large Language Models (LLMs) has revolutionized artificial intelligence, pushing the boundaries of what machines can achieve…