Mauricio is a Pontificia Universidad Católica de Chile (PUC) graduate pursuing a Master of Science (MSc) in Computer Science. He is researching automata theory and has worked with technologies like Serverless and NextJS. When he’s not working, he reads Star Wars books, participates in programming competitions, or plays board games.
The Project
Mauricio was assigned to design and implement a new Quota Manager service for the Nirvana platform.
At Nirvana, we integrate data from multiple sources, including telematics data and motor vehicle records from different providers. Clients can leverage various external APIs we manage on a usage-based pricing model for data access. Unfortunately, so many APIs create a vulnerability around unlimited API requests. Unbounded requests to any API by the client can result in bill shock for Nirvana. As a workaround, we implemented quota managers in the client code. However, quota managers are tightly coupled with individual clients, making the solution challenging to scale.
Mauricio’s project aimed to solve some of these challenges. The Quota Manager Service he was asked to work on had four main functional goals.
- Any client can self-configure quota limits and a quota management strategy like a fixed window, sliding window, or leaky bucket for any API.
- The service controls the API usage when multiple clients try to acquire a quota for the same API.
- Administrators can assign each external API a maximum periodic cost (hourly, daily, monthly, etc.)
- The service monitors metrics, raises alerts when the quota exceeds, and supports invoice verification from the external API providers.
The Challenges
Crafting a new service required a deep dive into the world of Quota Management. Mauricio had to study various real-world examples that demonstrated varying levels of complexity. He observed that each solution was customized to its unique context and couldn’t be duplicated within Nirvana.
However, Mauricio’s research helped him identify multiple quota management strategies using the API rate limiter feature to control request rates over short intervals. Every strategy has its pros and cons. To get it right, Mauricio had to interview different Nirvana team members further to understand how their code used the different APIs.
The detailed research and numerous discussions revealed that the Quota Manager service had to be extremely flexible to control requests from any internal client to any external service. The service was also required to perform at high speed, with very low latency as it is on the critical path of every client request, and performance issues could slow down the platform experience for every client.
The Solution
Mauricio’s project team defined a standard data representation called Resources to handle this.
For example, they could define the Resources “Google Maps” or “Terminal” for external services of the same name. They created an interface that allowed clients to
- Create Resources and configure them with specific quota limits and refresh intervals.
- Choose a quota management strategy like Fixed Window or Leaky Bucket when configuring a Resource.
The team identified that a Fixed Window strategy and a quota limit of 100 per month solved many use cases. Hence, they set these as the default values within the Service. Next, the team decided to implement the Quota Manager as a gRPC server, ensuring that the service is isolated from clients and is a singleton.
Today, the Quota Manager stores client quota configurations and the chosen management strategy in PostgreSQL but loads the data into memory within the gRPC server when it starts. Frequently accessed and constantly changing values, such as the current quota available, are stored in Redis. The service thus minimizes latency by running in-memory computations instead of PostgreSQL data pulls for every request.
The Quota Manager creates and updates Resources dynamically at runtime according to the chosen strategy. If clients need to make multiple requests for an extended job run, they can acquire a different quota percentage for each request. Each successful quota acquisition returns an allocation ID the client can use to restore or return any unused quota from their original request. This can happen for various reasons, such as if the request they are making to the third party fails or they just did not use the complete amount of the requested quota.
Metrics such as changes in quota availability for resources and calls with their error codes are sent to DataDog to monitor each resource’s activity and detect spikes in requests. DataDog triggers alerts and allows the Nirvana team to build dashboards to better understand our quotas.
Why Nirvana
“I like coding and solving technical challenges with impact in the real world. After reading up on Nirvana’s tech stack, I knew I would get to solve problems in efficiency and concurrency — topics I am passionate about. Working with cutting-edge technology like Go, collaborating with an international team of diverse backgrounds, and a startup environment — Nirvana was a no-brainer! It was simply too great an opportunity for my personal and professional growth as a software developer; I just couldn’t turn it down.”
Life at Nirvana
“The onboarding process exceeded my expectations. Throughout my project journey, I had a lot of support from everyone on the team. I could ask anyone in the office any question at any time. I learned new coding practices and the reason behind many technology decisions. We also had many fun in-office events — like happy hour, after-office, and team lunches. My internship also offered a hybrid work model that allowed me to go out with friends, attend workshops, and participate in coding competitions. All in all, I highly recommend Nirvana for an enriching and valuable internship experience!”
