VernLLM - lightweight resilience layer for OpenAI SDK
Introducing vernLLM: A Resilience Layer for LLM ApplicationsBuilding production-ready LLM applications is not just about sending prompts and receiving responses. Real-world AI systems need to handle timeouts,. . .
Introducing vernLLM: A Resilience Layer for LLM ApplicationsBuilding production-ready LLM applications is not just about sending prompts and receiving responses. Real-world AI systems need to handle timeouts, provider failures, rate limits, inconsistent outputs, and reliability issues. That is wherevernLLMcomes in. vernLLM is a lightweight resilience layer for OpenAI-compatible chat completion APIs, providing a single interface with built-in retries, timeouts, circuit breaking, caching, structured output, and usage tracking. Instead of rebuilding the same reliability features for every LLM project, vernLLM gives you the tools needed to make your AI integrations more robust from the start. FeaturesAutomatic retries with backoffTransient failures happen. vernLLM automatically retries recoverable errors while failing fast on validation errors and non-retryable responses. Timeouts & cancellationPrevent hanging requests with configurable timeouts and cancellation support. Circuit breaker protectionAutomatically stop sending requests to failing providers and recover when the service becomes healthy again. Structured output with type safetyPass a Zod schema and receive validated, typed results back. constresult=awaitllm. call({systemPrompt:'Return JSON: { "skills": string[] }',userContent:'Extract skills from: . . . ',schema:SkillsSchema// zod schema});Enter fullscreen modeExit fullscreen modeProvider-native JSON Schema supportConstrain model generation itself instead of only validating responses afterward. Built-in caching supportCache LLM responses using your own cache adapter withcachedCallandcachedLLMCall. One interface across providersUse the same API across multiple providers:OpenAIGroqMistralDeepSeekCerebrasTogether AIFireworks AIOllamaAnthropicGeminiAWS BedrockAny HTTP-compatible provider throughfromFetchWhy vernLLM?Many LLM applications end up creating their own wrappers around provider SDKs to handle:retry logicAPI failuresprovider switchingresponse validationcachingmonitoringvernLLM packages these patterns into a reusable, lightweight library so developers can focus on building AI features instead of maintaining infrastructure. Quick StartInstall:pnpm add vern-llm openaiEnter fullscreen modeExit fullscreen modeCreate your client:importOpenAIfrom'openai';import{VernLLM}from'vern-llm';exportconstllm=newVernLLM({client:newOpenAI({apiKey:process. env. OPENAI_API_KEY}),model:'gpt-4o',maxRetries:3,timeoutMs:10_000,circuitBreaker:true,onUsage:({totalTokens})=>{console. log(`Used${totalTokens}tokens`);},});// now do something with it!exportconstsummary=awaitllm. cachedLLMCall({cacheKey:`resume:${resumeId}`,ttl:3600,call:{systemPrompt:'Analyze this resume and return structured hiring insights. ',userContent:resumeText,schema:HiringSummarySchema,// Zod schema}});Enter fullscreen modeExit fullscreen modeBuilt for ProductionvernLLM is designed with production usage in mind:Zero bundled dependenciesTypeScript-first APIProvider adaptersExtensive test coverageMIT licensedWhether you are building AI agents, document processing pipelines, chat applications, or LLM-powered tools, vernLLM provides the reliability layer needed to ship with confidence. Learn MoreDocumentation:https://vernllm. vercel. app/GitHub:https://github. com/LakBud/vernLLMContributions, feedback, and ideas are welcome!