How I Built API Fallbacks for ApplyOnceAI

One problem showed up pretty quickly while I was building ApplyOnceAI. The AI part worked. Then the API limit got hit. That was not exactly what I wanted during a hackathon demo. ApplyOnceAI was built to help fill repetitive application forms. The extension needed an AI model for some of the work, so the first version depended on a single provider. It worked fine until it didn't. That made me rethink the whole approach.

One API is a single point of failure. If the only AI provider goes down, reaches its limit or starts returning errors, the user should not have to know why the application stopped working. At first, I could have just shown an error and moved on. But that felt like a poor experience. I already had access to several providers, so I started thinking about whether I could try another one when the first provider failed. That became the fallback system.

The basic idea: Instead of sending every request to one provider, I kept a list of providers in a specific order (Provider one, Provider two, Provider three, Provider four). The first provider gets the request. If the request succeeds, we are done. If it fails because of a temporary problem, the next provider gets a chance, then the next one if needed. The important part was not simply catching an error. I had to decide which errors should trigger a fallback and which ones should not. A temporary provider problem is different from a bad request. There is no point sending the exact same broken request to four different providers.

Timeouts matter too. Another thing I ran into was waiting. A request can technically still be running while the user is sitting there wondering what happened. So I added time limits around requests. If a provider takes too long, the system can stop waiting and move to the next option. That made the fallback system much more useful than simply doing try/catch error. The system needed to know when to stop waiting.

Keeping the providers separate: I also did not want the rest of the application to care which AI provider was being used. The extension should basically say: 'I need this task completed.' It should not need to know whether provider one, two or three handled it. That made the provider logic easier to change later. If one provider changed its API, I could work on that part without changing the rest of the application.

It was not perfect. The first version was definitely not perfect. There were cases where a provider returned an error that I did not expect. There were also limits that were not obvious until I actually started testing requests. That is one thing I like about building projects instead of only reading about them: You can plan the architecture for hours, but real problems usually show up when you start sending real requests.

What I took away from it: The biggest lesson was not about AI. It was about not depending too heavily on one thing when you have a reasonable alternative. For ApplyOnceAI, having multiple providers gave me a way to keep testing and demonstrating the project even when one provider reached its limit. It also made me think more carefully about failure. A successful request is easy to handle. The interesting engineering starts when the request fails. That is probably the part of ApplyOnceAI I learned the most from.

ApplyOnceAI: https://github.com/Surajmaurya1/ApplyonceAI