Yes, 2-5 minutes per pack is accurate for 5-inch FPV racing drones, and it's not a flaw -- it's a consequence of the power-to-weight ratio that makes these quads fast and responsive.
A 5-inch racing quad weighs 500-700g and pulls 30-60 amps continuous from a 4S 1500mAh pack at moderate flying. At full racing throttle, current draw can exceed 100 amps in bursts. That drains a 1500mAh pack in 2-3 minutes at race pace or 4-5 minutes at casual flying. By comparison, a DJI Mini 3 Pro sips current gently with efficient motors optimized for hover, not speed.
For a race session: buy 8-10 packs minimum. With a dual-port charger, you can charge two at a time while flying others -- a rolling rotation. Most pilots bring 10-12 packs to a full-day race event. Budget $200-280 for a 10-pack setup of CNHL Black batteries.
Shop FPV LiPo Battery Packs on Amazon