Skip to content
APFerrer
M-IA-01 · ModuleBusiness software

Multi-provider LLM layer with customer-owned keys and quotas

A single gateway to OpenAI, Anthropic, Google Gemini and Azure AI Inference, with each customer's key stored encrypted, rate limits, monthly quotas and a log of every call.

Source codeEmbeddableOEM

Adding AI to software with many customers often ends in scattered calls, unprotected keys and no control over spending. This multi-tenant LLM gateway puts every call through a single point: each customer uses its own OpenAI, Anthropic, Google Gemini or Azure AI Inference key, which is stored encrypted, and the module applies limits, quotas and a log of every call.

Who it is for

Software companies with many customers that are adding AI and want each customer to use its own key, and compliance leads who need to know what was asked of the AI and by whom.

What a multi-provider LLM layer solves

  • Dependence on one provider. You switch providers without touching the rest of the application.
  • Who pays for the AI. Each customer uses its own key (BYOK) or, if agreed, the platform's.
  • Runaway spending. A single user or a malicious instruction cannot drain the budget.
  • No traceability. Every call is logged for audits and for the EU AI Act (Regulation (EU) 2024/1689).

What it does

  • Sends requests to the four providers through one interface and retries on transient errors.
  • Uses each provider's caching.
  • Picks the model by task and input size, and only moves up to a larger model if the answer fails.
  • Tells balance, key and rate limit errors apart, and lets you use data centers in the EU.
  • Stores each customer's key encrypted and shows only its last four characters. Optionally, it uses the platform's key.
  • Applies limits per user and per customer, per minute and per day, and monthly quotas of calls and tokens.
  • Tests a key without sending any customer data.
  • Logs every call: task, user, provider, model, tokens, estimated cost, response time and result. If there is no known rate for the model, the cost is recorded as unknown, never as zero.

Benefits

  • AI costs do not eat into your margin: each customer pays for its own usage or has a cap.
  • Your team does not have to build a component that every product with AI needs.

Industry context

General industry data with its public source. These are not results of this product.

What you get with each license

Price upon request, depending on license type and scope. Integration support and maintenance are contracted separately.

  • The module code, ready to add to your application.
  • Integration examples and documentation.
  • A warranty against defects for the agreed period.

Technical specifications

Language and runtime: TypeScript 5.9 on Node.js 22 (the version it ships with). Minimum supported version: Node.js 20.19 (required by Prisma 7 in its configuration).

Frameworks: It does not depend on a web framework. It uses Prisma 7.10 for configuration, limits and the call log, and AI SDK 6.0 for part of the connection with the providers.

Database: PostgreSQL.

Operating systems: A server with Node.js.

Interface languages: The module has no interface of its own.

Required infrastructure:

  • PostgreSQL.
  • Redis, optional, to share limits across several servers.
  • An encryption key in the server configuration.
  • An AI provider: OpenAI, Anthropic, Google Gemini or Azure AI Inference (requires the buyer's own account or each customer's).

Private deployment: It installs in your infrastructure, but it needs a connection to the AI provider each customer chooses.

Main libraries:

Library Version License
zod 4.4.3 MIT
@prisma/client 7.10.1 Apache-2.0
pg 8.22.0 MIT
ioredis 5.11.1 MIT
ai 6.0.308 Apache-2.0
@ai-sdk/openai 3.0.129 Apache-2.0
@ai-sdk/anthropic 3.0.132 Apache-2.0
@ai-sdk/google 3.0.134 Apache-2.0

Dependencies with specific license terms: None.

Compatibility and integrations: OpenAI Chat Completions API, Anthropic Messages API (version 2023-06-01), Google Gemini API v1beta, Azure AI Inference (including GitHub Models), PostgreSQL and Redis.

Redistribution terms

The asset's own code can be integrated into the licensee's commercial products under the embeddable module license, the source code license and the OEM license, on the terms of the contract. None of the module's production dependencies has a copyleft license or specific commercial terms.

Installation requirements

A Node.js application with PostgreSQL. Redis is optional, to share limits between servers. Each customer brings its own account with the AI provider, except in managed mode.

FAQ

Frequently asked questions

  • Which providers does it support?

    OpenAI, Anthropic, Google Gemini and Azure AI Inference, which includes GitHub Models.

  • Where is the customer's key stored?

    In your database, encrypted. The encryption key is in the server configuration.

  • Are prompts stored?

    A fingerprint (hash) of the input and the already redacted text are stored, with a retention period.

  • Is maintenance included?

    No. Providers change models, rates and formats often. Maintenance and support are contracted separately.

Request

Request information on M-IA-01

Fill in the form with the asset reference and the license type you're considering. I reply in writing. If you need to review code, tick the NDA box and we'll sign one before anything is shared.

Controller: APFerrer. Purpose: to answer your request. You can exercise your rights as described in the privacy policy.

Price upon request, depending on license type and scope. Maintenance and support are contracted separately.