Looking for advice from people dealing with high LLM or AI API costs
by MutedMaintenance6420
Hi everyone, I’m from Korea, and English isn’t my first language, so I used AI to help translate this post. I’ve been experimenting with different AI tools and projects for a while. During that process, I ended up with a much larger token bill than I expected. That experience made me think there should be a better way for people and companies to use AI without constantly worrying about costs, while still keeping the system reliable and safe. At first, I started building something just for my own use. I added features one by one, and over time the project became much bigger than I originally planned. Eventually, I started wondering whether it could become a real business. But I’ve run into a problem. In Korea, I haven’t been able to find many companies that are openly talking about this issue or looking for a solution. I’m not sure whether the problem isn’t serious enough yet, or whether companies simply don’t want to share their AI cost problems publicly. So I’d really appreciate some honest feedback from developers, engineers, founders, or anyone operating AI products in production. Are AI or LLM costs a real problem for you? What part of cost management is the most difficult? How do you currently track costs by model, feature, customer, or request? What kind of tool or service would actually be useful? Have you tried solving this problem already? If so, what didn’t work? Any practical advice or real-world experience would be very helpful. I’m still trying to understand whether this is a problem worth solving and what people actually need. Thank you. submitted by /u/MutedMaintenance6420 to r/FinOps [link] [comments]
2 comments
If you're learning, building your own /r/localllm really is a great way to actually understand how LLMs work and why tokens are consumed for what reasons. Your first activity however, is getting observability set up. Larger orgs need to set up enterprise policies that block access to direct token consumption, and create a proxy system that everyone must go through (there are a few options out there). Once everyone is being funnelled into the proxy, you can observe, and eventually control token usage in a multitude of ways.
Thank you so much for your reply.