goai
Go AI SDK, the Go way. One unified API across 21+ providers. Streaming, structured output, MCP support, stdlib only. Go AI SDK for AI applications inspired by Vercel AI SDK.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/zendev-sh--goai)HTML
<a href="/listings/zendev-sh--goai"><img src="/api/listings/zendev-sh--goai/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
go get github.com/zendev-sh/goai@latest
Requires Go 1.25+.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
SKILL.md · Manifest · 627 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- # GoAI SDK - AI Agent Skill
- > Instructions for AI coding agents helping developers use the GoAI SDK.
- > Load this file when a user's project imports `github.com/zendev-sh/goai`.
- ## Overview
- GoAI is a Go SDK for AI applications. One unified API across 25+ LLM providers. Inspired by the Vercel AI SDK, adapted to Go idioms (generics, interfaces, channels).
- - **Package**: `github.com/zendev-sh/goai`
- - **Go version**: 1.25+
- - **Dependencies**: stdlib + `golang.org/x/oauth2` for Vertex AI. Optional `observability/otel` submodule adds OTel SDK (separate go.mod, not pulled unless imported).
- - **Docs**: https://goai.sh
- - **GoDoc**: https://pkg.go.dev/github.com/zendev-sh/goai
- ## Quick Reference
- ```
- go get github.com/zendev-sh/goai@latest
- ```
- ```go
- import (
- "github.com/zendev-sh/goai"
- "github.com/zendev-sh/goai/provider/openai" // or any provider
- )
- ```
- ## Core API
- ### 8 Top-Level Functions
- | Function | Purpose | Returns |
- | --------------------------------------------- | ------------------------------- | --------------------------- |
- | `goai.GenerateText(ctx, model, opts...)` | Non-streaming text generation | `(*TextResult, error)` |
- | `goai.StreamText(ctx, model, opts...)` | Streaming text via channels | `(*TextStream, error)` |
- | `goai.GenerateObject[T](ctx, model, opts...)` | Typed structured output (JSON) | `(*ObjectResult[T], error)` |
- | `goai.StreamObject[T](ctx, model, opts...)` | Streaming structured output | `(*ObjectStream[T], error)` |
- | `goai.Embed(ctx, model, text, opts...)` | Single text embedding | `(*EmbedResult, error)` |
- | `goai.EmbedMany(ctx, model, texts, opts...)` | Batch embeddings (auto-chunked) | `(*EmbedManyResult, error)` |
- | `goai.GenerateImage(ctx, model, imgOpts...)` | Image generation | `(*ImageResult, error)` |
- | `goai.GenerateVideo(ctx, model, vidOpts...)` | Video generation | `(*VideoResult, error)` |
- ### Model Constructors
- Each provider has `Chat()`, and optionally `Embedding()`, `Image()`, and `Video()`:
- ```go
- // Language models
- openai.Chat("gpt-4o")
- anthropic.Chat("claude-sonnet-4-20250514")
- google.Chat("gemini-2.5-flash")
- bedrock.Chat("anthropic.claude-sonnet-4-20250514-v1:0")
- azure.Chat("gpt-4o", azure.WithEndpoint("https://my-resource.openai.azure.com"))
- vertex.Chat("gemini-2.5-flash", vertex.WithProject("my-project"), vertex.WithLocation("us-central1"))
- groq.Chat("llama-3.3-70b-versatile")
- ollama.Chat("llama3.2")
- // Embedding models
- openai.Embedding("text-embedding-3-small")
- google.Embedding("text-embedding-004")
- cohere.Embedding("embed-english-v3.0")
- ollama.Embedding("nomic-embed-text")
- // Image models
- openai.Image("gpt-image-1")
- google.Image("imagen-4.0-generate-001")
- // Video models
- google.Video("veo-3.1-generate-preview")
- ```
- ### Auth - Auto-Resolved from Environment
- Providers auto-read API keys from env vars. No explicit config needed:
- | Provider | Env Var |
- | --------- | ------------------------------------------------------------ |
- | OpenAI | `OPENAI_API_KEY` |
- | Anthropic | `ANTHROPIC_API_KEY` |
- | Google | `GOOGLE_GENERATIVE_AI_API_KEY` or `GEMINI_API_KEY` |
- | Bedrock | `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` + `AWS_REGION` |
- | Azure | `AZURE_OPENAI_API_KEY` |
- | Vertex AI | Application Default Credentials (ADC) |
- | xAI | `XAI_API_KEY` |
- | Groq | `GROQ_API_KEY` |
- | Cohere | `COHERE_API_KEY` |
- | Mistral | `MISTRAL_API_KEY` |
- | DeepSeek | `DEEPSEEK_API_KEY` |
- Or set explicitly:
- ```go
- model := openai.Chat("gpt-4o", openai.WithAPIKey("sk-..."))
- ```
- Most providers support these options (Bedrock uses AWS credential options; Ollama requires no auth):
- ```go
- openai.WithAPIKey(key) // static API key
- openai.WithTokenSource(ts) // dynamic auth (OAuth, service accounts)
- openai.WithBaseURL(url) // override endpoint (Azure uses WithEndpoint)
- openai.WithHeaders(h) // custom HTTP headers
- openai.WithHTTPClient(c) // custom HTTP transport
- ```
- ---
- ## Patterns and Examples
- ### 1. Basic Text Generation
- ```go
- result, err := goai.GenerateText(ctx, openai.Chat("gpt-4o"),
- goai.WithSystem("You are a helpful assistant."),
- goai.WithPrompt("What is Go?"),
- )
- if err != nil {
- return err
- }
- fmt.Println(result.Text)
- ```
- ### 2. Streaming
- ```go
- stream, err := goai.StreamText(ctx, openai.Chat("gpt-4o"),
- goai.WithPrompt("Write a poem about Go."),
- )
- if err != nil {
- return err
- }
- // Option A: text-only channel
- for text := range stream.TextStream() {
- fmt.Print(text)
- }
- // Option B: raw chunks (mutually exclusive with TextStream)
- // for chunk := range stream.Stream() { ... }
- // Always available after streaming completes:
- result := stream.Result()
- fmt.Printf("\nTokens: %d in, %d out\n", result.TotalUsage.InputTokens, result.TotalUsage.OutputTokens)
- ```
- **Important**: `Stream()` and `TextStream()` are mutually exclusive. Only call one. `Result()` can always be called after either.
- ### 3. Structured Output (Generics)
- ```go
- type Recipe struct {
- Name string `json:"name" jsonschema:"description=Recipe name"`
- Ingredients []string `json:"ingredients"`
- Steps []string `json:"steps"`
- Difficulty string `json:"difficulty" jsonschema:"enum=easy|medium|hard"`
- }
- result, err := goai.GenerateObject[Recipe](ctx, openai.Chat("gpt-4o"),
- goai.WithPrompt("Chocolate chip cookies recipe"),
- )
- if err != nil {
- return err
- }
- fmt.Println(result.Object.Name) // typed access
- fmt.Println(result.Object.Difficulty) // "easy", "medium", or "hard"
- ```
- Schema is auto-generated from struct tags. Supported tags:
- - `json:"name"` - property name
- - `json:"-"` - exclude field
- - `jsonschema:"description=..."` - adds description
- - `jsonschema:"enum=a|b|c"` - restricts to enum values
- Supported types: string, bool, int/uint (all sizes), float32/64, slices, maps (string keys), structs (embedded structs flattened), pointers (nullable).
- ### 4. Streaming Structured Output
- ```go
- stream, err := goai.StreamObject[Recipe](ctx, model,
- goai.WithPrompt("Chocolate chip cookies recipe"),
- )
- if err != nil {
- return err
- }
- for partial := range stream.PartialObjectStream() {
- fmt.Printf("Name so far: %s\n", partial.Name)
- }
- result, err := stream.Result()
- // result.Object is the final validated Recipe
- ```
- ### 5. Tool Calling
- ```go
- weatherTool := goai.Tool{
- Name: "get_weather",
- Description: "Get weather for a city.",
- InputSchema: json.RawMessage(`{
- "type": "object",
- "properties": {
- "city": {"type": "string", "description": "City name"}
- },
- "required": ["city"]
- }`),
- Execute: func(ctx context.Context, input json.RawMessage) (string, error) {
- var params struct {
- City string `json:"city"`
- }
- if err := json.Unmarshal(input, ¶ms); err != nil {
- return "", err
- }
- return fmt.Sprintf("72F and sunny in %s", params.City), nil
- },
- }
- result, err := goai.GenerateText(ctx, model,
- goai.WithPrompt("What's the weather in Tokyo?"),
- goai.WithTools(weatherTool),
- goai.WithMaxSteps(3), // enable auto tool loop
- )
- ```
- **Key rules:**
- - `WithMaxSteps(1)` (default) = no auto loop, tool calls returned in `result.ToolCalls`
- - `WithMaxSteps(n)` where n > 1 = auto loop: generate -> execute tools -> re-generate -> repeat
- - Tools without `Execute` are sent as definitions only (for manual handling)
- - Tool errors are passed back to the model as `"error: ..."` messages
- ### 6. Conversation History
- ```go
- result, err := goai.GenerateText(ctx, model,
- goai.WithSystem("You are a helpful assistant."),
- goai.WithMessages(
- goai.UserMessage("Hi, my name is Alice."),
- goai.AssistantMessage("Hello Alice! How can I help you?"),
- goai.UserMessage("What's my name?"),
- ),
- )
- ```
- Message builders:
- - `goai.SystemMessage(text)` - system message
- - `goai.UserMessage(text)` - user message
- - `goai.AssistantMessage(text)` - assistant message
- - `goai.ToolMessage(toolCallID, toolName, output)` - tool result
- ### 7. Embeddings
- ```go
- // Single embedding
- model := openai.Embedding("text-embedding-3-small")
- result, err := goai.Embed(ctx, model, "Hello world")
- fmt.Printf("Dimensions: %d\n", len(result.Embedding))
- // Batch embeddings (auto-chunked, parallel)
- results, err := goai.EmbedMany(ctx, model, []string{"text1", "text2", "text3"},
- goai.WithMaxParallelCalls(4),
- )
- ```
- ### 8. Image Generation
- ```go
- result, err := goai.GenerateImage(ctx, openai.Image("gpt-image-1"),
- goai.WithImagePrompt("A gopher writing Go code"),
- goai.WithImageSize("1024x1024"),
- goai.WithImageCount(1),
- )
- // result.Images[0].Data = raw bytes, result.Images[0].MediaType = "image/png"
- ```
- **Note**: `GenerateImage` uses `ImageOption` (not `Option`): `WithImagePrompt`, `WithImageCount`, `WithImageSize`, `WithAspectRatio`, `WithImageMaxRetries`, `WithImageTimeout`, `WithImageProviderOptions`.
- ### 9. Video Generation
- ```go
- result, err := goai.GenerateVideo(ctx, google.Video("veo-3.1-generate-preview"),
- goai.WithVideoPrompt("A gopher writing Go code"),
- goai.WithVideoAspectRatio("16:9"),
- goai.WithVideoDuration(8*time.Second),
- )
- // result.Video.Data = raw bytes, result.Video.MediaType = "video/mp4"
- ```
- Video generation handles asynchronous operation polling and authenticated downloads. Use `WithVideoImage` for image-to-video and `WithVideoPollTimeout` to bound long-running operations.
- ### 10. Provider-Defined Tools (Server-Side)
- Built-in tools executed by the provider, not your code:
- ```go
- // OpenAI web search
- result, err := goai.GenerateText(ctx, openai.Chat("gpt-4o"),
- goai.WithPrompt("Latest Go release?"),
- goai.WithTools(providerTool(openai.Tools.WebSearch())),
- )
- // Google Search grounding
- result, err := goai.GenerateText(ctx, google.Chat("gemini-2.5-flash"),
- goai.WithPrompt("Latest news about Go"),
- goai.WithTools(providerTool(google.Tools.GoogleSearch())),
- )
- // Anthropic code execution
- result, err := goai.GenerateText(ctx, anthropic.Chat("claude-sonnet-4-20250514"),
- goai.WithPrompt("Calculate fibonacci(10)"),
- goai.WithTools(providerTool(anthropic.Tools.CodeExecution())),
- )
- // Helper to convert provider.ToolDefinition → goai.Tool:
- func providerTool(td provider.ToolDefinition) goai.Tool {
- return goai.Tool{
- Name: td.Name, ProviderDefinedType: td.ProviderDefinedType,
- ProviderDefinedOptions: td.ProviderDefinedOptions,
- }
- }
- ```
- Available provider tools:
- - **OpenAI**: `WebSearch()`, `CodeInterpreter()`, `ImageGeneration()`, `FileSearch(opts...)`
- - **Anthropic**: `WebSearch()`, `WebFetch()`, `Computer(opts)`, `Bash()`, `TextEditor()`, `CodeExecution()`, `ToolSearchToolRegex()`, `ToolSearchToolBM25()` (+ versioned variants)
- - **Google**: `GoogleSearch()`, `URLContext()`, `CodeExecution()`, `ComputerUse(opts...)`, `FileSearch(opts...)`
- - **xAI**: `WebSearch()`, `XSearch()`
- - **Groq**: `BrowserSearch()`
- ### 11. Observability Hooks
- ```go
- result, err := goai.GenerateText(ctx, model,
- goai.WithPrompt("Hello"),
- goai.WithOnRequest(func(info goai.RequestInfo) {
- log.Printf("Calling %s with %d messages", info.Model, info.MessageCount)
- }),
- goai.WithOnResponse(func(info goai.ResponseInfo) {
- log.Printf("Response in %v, tokens: %d", info.Latency, info.Usage.TotalTokens)
- }),
- goai.WithOnStepFinish(func(step goai.StepResult) {
- log.Printf("Step %d: %s, tools: %d", step.Number, step.FinishReason, len(step.ToolCalls))
- }),
- goai.WithOnToolCall(func(info goai.ToolCallInfo) {
- log.Printf("Tool %s (step %d) took %v, input: %s", info.ToolName, info.Step, info.Duration, info.Input)
- }),
- )
- ```
- ---
- ## Complete Options Reference
- ### Core Options (type `goai.Option`)
- ```go
- goai.WithSystem(s string) // system prompt
- goai.WithPrompt(s string) // single user message (shorthand)
- goai.WithMessages(msgs ...provider.Message) // conversation history
- goai.WithPromptCaching(bool) // enable prompt caching
- goai.WithTools(tools ...goai.Tool) // available tools
- goai.WithMaxSteps(n int) // auto tool loop iterations (default: 1)
- goai.WithToolChoice(tc string) // "auto" | "none" | "required" | "<tool_name>"
- goai.WithMaxOutputTokens(n int) // response length limit
- goai.WithTemperature(t float64) // randomness (0.0 - 2.0)
- goai.WithTopP(p float64) // nucleus sampling
- goai.WithTopK(k int) // top-K sampling
- goai.WithFrequencyPenalty(p float64) // frequency penalty
- goai.WithPresencePenalty(p float64) // presence penalty
- goai.WithSeed(s int) // deterministic generation
- goai.WithStopSequences(seqs ...string) // stop sequences
- goai.WithMaxRetries(n int) // retry count (default: 2)
- goai.WithRetryObserver(fn RetryObserver) // observe retry attempts
- goai.WithTimeout(d time.Duration) // request timeout
- goai.WithHeaders(h map[string]string) // additional HTTP headers
- goai.WithProviderOptions(opts map[string]any) // provider-specific params
- goai.WithOnRequest(fn func(RequestInfo)) // before each API call
- goai.WithOnResponse(fn func(ResponseInfo)) // after each API call
- goai.WithOnStepFinish(fn func(StepResult)) // after each step
- goai.WithOnToolCallStart(fn func(ToolCallStartInfo)) // before each tool execution
- goai.WithOnToolCall(fn func(ToolCallInfo)) // after each tool execution
- goai.WithOnFinish(fn func(FinishInfo)) // after generation terminates
- goai.WithOnPanic(fn func(PanicInfo)) // observe recovered callback panics
- goai.WithOnBeforeStep(fn func(BeforeStepInfo) BeforeStepResult)
- goai.WithOnBeforeToolExecute(fn func(BeforeToolExecuteInfo) BeforeToolExecuteResult)
- goai.WithOnAfterToolExecute(fn func(AfterToolExecuteInfo) AfterToolExecuteResult)
- goai.WithExplicitSchema(schema json.RawMessage) // override auto-generated schema
- goai.WithSchemaName(name string) // schema name (default: "response")
- goai.WithMaxParallelCalls(n int) // EmbedMany parallelism (default: 4)
- goai.WithEmbeddingProviderOptions(opts map[string]any) // embedding-specific params
- ```
- ### Image Options (type `goai.ImageOption`)
- ```go
- goai.WithImagePrompt(prompt string) // text prompt
- goai.WithImageCount(n int) // number of images (default: 1)
- goai.WithImageSize(size string) // e.g. "1024x1024"
- goai.WithAspectRatio(ratio string) // e.g. "16:9"
- goai.WithImageMaxRetries(n int) // retries on 429/5xx
- goai.WithImageTimeout(d time.Duration) // overall timeout
- goai.WithImageProviderOptions(opts map[string]any) // provider-specific params
- ```
- ### Video Options (type `goai.VideoOption`)
- ```go
- goai.WithVideoPrompt(prompt string)
- goai.WithVideoImage(image provider.MediaData)
- goai.WithVideoCount(n int)
- goai.WithVideoAspectRatio(ratio string)
- goai.WithVideoResolution(resolution string)
- goai.WithVideoDuration(d time.Duration)
- goai.WithVideoFPS(fps int)
- goai.WithVideoSeed(seed int64)
- goai.WithVideoFrameImages(frames ...provider.VideoFrame)
- goai.WithVideoInputReferences(refs ...provider.MediaData)
- goai.WithVideoAudio(enabled bool)
- goai.WithVideoProviderOptions(opts map[string]any)
- goai.WithVideoMaxRetries(n int)
- goai.WithVideoTimeout(d time.Duration)
- goai.WithVideoPollInterval(d time.Duration)
- goai.WithVideoPollTimeout(d time.Duration)
- ```
- ---
- ## Result Types
- ### TextResult (from GenerateText / StreamText)
- ```go
- type TextResult struct {
- Text string // accumulated generated text
- Reasoning string // accumulated reasoning text
- ToolCalls []provider.ToolCall // tool calls from final step
- Steps []StepResult // per-step results
- TotalUsage provider.Usage // aggregated token usage
- FinishReason provider.FinishReason // "stop", "tool-calls", "length", etc.
- Response provider.ResponseMetadata // provider metadata (ID, Model)
- ProviderMetadata map[string]map[string]any // provider-specific response data
- Sources []provider.Source // citations/references
- StepsExhausted bool // whether MaxSteps was exhausted
- ResponseMessages []provider.Message // messages for the next turn
- }
- ```
- ### ObjectResult[T] (from GenerateObject / StreamObject)
- ```go
- type ObjectResult[T any] struct {
- Object T // the parsed typed object
- Usage provider.Usage // total token consumption across all steps
- FinishReason provider.FinishReason
- Response provider.ResponseMetadata
- ProviderMetadata map[string]map[string]any // provider-specific response data
- ResponseMessages []provider.Message // messages for the next turn
- Steps []StepResult // results from each generation step (multi-step tool loops)
- }
- ```
- ### EmbedResult / EmbedManyResult
- ```go
- type EmbedResult struct {
- Embedding []float64
- Usage provider.Usage
- ProviderMetadata map[string]map[string]any
- Response provider.ResponseMetadata
- }
- type EmbedManyResult struct {
- Embeddings [][]float64
- Usage provider.Usage
- ProviderMetadata map[string]map[string]any
- Response provider.ResponseMetadata
- }
- ```
- ### ImageResult
- ```go
- type ImageResult struct {
- Images []provider.ImageData
- ProviderMetadata map[string]map[string]any
- Usage provider.Usage
- Response provider.ResponseMetadata
- }
- ```
- ### VideoResult
- ```go
- type VideoResult struct {
- Video provider.VideoData
- Videos []provider.VideoData
- ProviderMetadata map[string]map[string]any
- Response provider.ResponseMetadata
- }
- ```
- ### provider.Usage
- ```go
- type Usage struct {
- InputTokens int
- OutputTokens int
- TotalTokens int
- ReasoningTokens int
- CacheReadTokens int
- CacheWriteTokens int
- }
- ```
- ---
- ## Error Handling
- ```go
- import "errors"
- result, err := goai.GenerateText(ctx, model, goai.WithPrompt("..."))
- if err != nil {
- var apiErr *goai.APIError
- var overflowErr *goai.ContextOverflowError
- if errors.As(err, &overflowErr) {
- // Prompt too long for context window - truncate messages and retry
- log.Printf("Context overflow: %s", overflowErr.Message)
- } else if errors.As(err, &apiErr) {
- log.Printf("API error %d: %s (retryable: %v)", apiErr.StatusCode, apiErr.Message, apiErr.IsRetryable)
- } else {
- log.Printf("Error: %v", err)
- }
- }
- ```
- **Always use `errors.As()`, never type assertion.**
- Built-in retry: transient errors (429, 5xx) are retried automatically up to `WithMaxRetries(2)` times with exponential backoff.
- ---
- ## TokenSource (Dynamic Auth)
- For OAuth, service accounts, or rotating credentials:
- ```go
- ts := provider.CachedTokenSource(func(ctx context.Context) (*provider.Token, error) {
- // Fetch token from your auth system
- token, err := myAuthClient.GetToken(ctx)
- if err != nil {
- return nil, err
- }
- return &provider.Token{
- Value: token.AccessToken,
- ExpiresAt: token.ExpiresAt,
- }, nil
- })
- model := openai.Chat("gpt-4o", openai.WithTokenSource(ts))
- ```
- - `provider.StaticToken(key)` - wraps a static string
- - `provider.CachedTokenSource(fetchFn)` - caches until expiry, thread-safe
- ---
- ## Supported Providers
- ### Tier 1 (dedicated implementations)
- | Provider | Import | Chat | Embed | Image | Provider Tools |
- | ------------- | -------------------- | ---- | ----- | ----- | -------------- |
- | OpenAI | `provider/openai` | Yes | Yes | Yes | 4 |
- | Anthropic | `provider/anthropic` | Yes | - | - | 14 |
- | Google Gemini | `provider/google` | Yes | Yes | Yes | 5 |
- | AWS Bedrock | `provider/bedrock` | Yes | Yes | - | - |
- | Azure OpenAI | `provider/azure` | Yes | - | Yes | - |
- | Vertex AI | `provider/vertex` | Yes | Yes | Yes | - |
- ### Tier 2
- | Provider | Import | Chat | Embed | Provider Tools |
- | ---------- | ------------------- | ---- | ----- | -------------- |
- | Cohere | `provider/cohere` | Yes | Yes | - |
- | Mistral | `provider/mistral` | Yes | - | - |
- | xAI (Grok) | `provider/xai` | Yes | - | 2 |
- | Groq | `provider/groq` | Yes | - | 1 |
- | DeepSeek | `provider/deepseek` | Yes | - | - |
- ### Tier 3 (OpenAI-compatible)
- Fireworks, Together, DeepInfra, OpenRouter, Perplexity, Cerebras
- ### Local
- | Provider | Import | Chat | Embed | Default URL |
- | -------- | ----------------- | ---- | ----- | -------------------- |
- | Ollama | `provider/ollama` | Yes | Yes | `http://localhost:11434` |
- | vLLM | `provider/vllm` | Yes | Yes | `localhost:8000/v1` |
- | RunPod | `provider/runpod` | Yes | - | `https://api.runpod.ai/v2/{endpointID}/openai/v1` |
- | Custom | `provider/compat` | Yes | Yes | user-defined |
- ---
- ## Common Mistakes to Avoid
- 1. **Don't use `WithPrompt` and `WithMessages` with user message together carelessly** - `WithPrompt("x")` is auto-wrapped as a `UserMessage` and prepended to `Messages`. Use one or the other for the initial user input.
- 2. **Don't call both `Stream()` and `TextStream()`** - they are mutually exclusive on `TextStream`. The second call returns a closed channel.
- 3. **Don't forget `WithMaxSteps` for tool loops** - default is 1 (no loop). Set `WithMaxSteps(n)` where n > 1 to enable automatic tool execution.
- 4. **Don't type-assert errors** - always use `errors.As(err, &target)`, never `err.(*goai.APIError)`.
- 5. **Don't mix option families** - text/object/embed functions use `Option`, `GenerateImage` uses `ImageOption`, and `GenerateVideo` uses `VideoOption`.
- 6. **Don't forget error handling on streaming** - always check the error from `StreamText`/`StreamObject` before consuming the stream.
- 7. **`Result()` on streams is blocking** - it waits for the entire stream to complete. Call it after consuming `TextStream()`/`Stream()`, or on its own if you don't need incremental output.
README.md · README · 909 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- <p align="center">
- <img src="goai.png" alt="GoAI" width="400">
- </p>
- <h1 align="center">GoAI</h1>
- <p align="center"><em>AI SDK, the Go way.</em></p>
- <p align="center">Go SDK for building AI applications. One SDK, 25+ providers, MCP support.</p>
- <p align="center">
- <a href="bench/RESULTS.md"><img src="https://img.shields.io/badge/streaming-1.1x_faster-brightgreen" alt="Streaming"></a>
- <a href="bench/RESULTS.md"><img src="https://img.shields.io/badge/cold_start-24x_faster-brightgreen" alt="Cold Start"></a>
- <a href="bench/RESULTS.md"><img src="https://img.shields.io/badge/memory-3.1x_less-brightgreen" alt="Memory"></a>
- </p>
- <p align="center"><strong>1.1x faster streaming</strong>, <strong>24x faster cold start</strong>, <strong>3.1x less memory</strong> vs Vercel AI SDK (<a href="bench/RESULTS.md">benchmarks</a>)</p>
- <p align="center">
- <a href="https://goai.sh">Website</a> ·
- <a href="https://goai.sh/getting-started/installation">Docs</a> ·
- <a href="https://goai.sh/architecture">Architecture</a> ·
- <a href="https://goai.sh/providers/">Providers</a> ·
- <a href="https://goai.sh/examples">Examples</a> ·
- <a href="https://github.com/zendev-sh/zenflow">Orchestration (zenflow)</a>
- </p>
- ---
- Inspired by the [Vercel AI SDK](https://sdk.vercel.ai). The same clean abstractions, idiomatically adapted for Go with generics, interfaces, and functional options.
- ## Selected Releases
- > **v0.9.0** - Provider-neutral file upload and remote file references. `FileUploader` interface + `RemoteFileRef` for OpenAI (Files API), Anthropic (Files API), and Google Gemini (Files API). Compat providers map file parts to native OpenAI shapes. [Changelog →](https://github.com/zendev-sh/goai/releases)
- >
- > **v0.8.5** - New provider: [Requesty](https://goai.sh/providers/requesty) (OpenAI-compatible LLM gateway). Plus Anthropic native structured output, per-step `Reasoning`, and Gemini schema sanitization fixes. [Changelog →](https://github.com/zendev-sh/goai/releases)
- >
- > **v0.8.4** - `WithRetryObserver` callback for observing retry attempts, split tool-call stream metadata fix, and Anthropic `redacted_thinking` capture in streaming. [Changelog →](https://github.com/zendev-sh/goai/releases)
- >
- > **v0.8.3** - Native Ollama provider (`/api/chat` + `/api/embed`, think mode, no third-party SDK) and duplicate tool name detection. [Changelog →](https://github.com/zendev-sh/goai/releases)
- >
- > **v0.8.0** - MCP OAuth 2.1 + PKCE for remote servers, `NewTool[In]` typed constructor, and lifecycle hook panics surfaced as `*PanicError` via `WithOnPanic` (breaking). [Changelog →](https://github.com/zendev-sh/goai/releases)
- ## Features
- - **8 core functions**: `GenerateText`, `StreamText`, `GenerateObject[T]`, `StreamObject[T]`, `Embed`, `EmbedMany`, `GenerateImage`, `GenerateVideo`
- - **25+ providers**: OpenAI, Anthropic, Google, Bedrock, Azure, Vertex, Mistral, xAI, Groq, Cohere, DeepSeek, MiniMax, Fireworks, Together, DeepInfra, OpenRouter, Requesty, Perplexity, Cerebras, Ollama, vLLM, RunPod, Runcrate, Cloudflare Workers AI, FPT Smart Cloud, NVIDIA NIM, Heabsy, Opper, Cheaper Inference, CloudGPU, llama.cpp, + generic OpenAI-compatible
- - **Auto tool loop**: Define tools with `Execute` handlers, set `MaxSteps` for `GenerateText` and `StreamText`
- - **Structured output**: `GenerateObject[T]` auto-generates JSON Schema from Go types via reflection
- - **Streaming**: Real-time text and partial object streaming via channels
- - **Dynamic auth**: `TokenSource` interface for OAuth, rotating keys, cloud IAM, with `CachedTokenSource` for TTL-based caching
- - **Prompt caching**: Automatic cache control for supported providers (Anthropic, Bedrock, MiniMax)
- - **Citations/sources**: Grounding and inline citations from xAI, Perplexity, Google, OpenAI
- - **Web search**: Built-in web search tools for OpenAI, Anthropic, Google, Groq, xAI. Model decides when to search
- - **Code execution**: Server-side Python sandboxes via OpenAI, Anthropic, Google. No local setup
- - **Computer use**: Anthropic computer, bash, text editor tools for autonomous desktop interaction
- - **26 provider-defined tools**: Web fetch, file search, image generation, X search, and more - [full list](#provider-defined-tools)
- - **MCP client**: Connect to any MCP server (stdio, HTTP, SSE), auto-convert tools for use with GoAI
- - **Realtime voice** (experimental): Bidirectional audio sessions with interruption and automatic tool calls via Gemini Live, on the Gemini API or Vertex AI
- - **Observability**: Built-in Langfuse and OpenTelemetry (OTel) integrations for tracing generations, tools, and multi-step loops
- - **Multi-agent orchestration**: For declarative YAML workflows on top of GoAI, see [zenflow](https://github.com/zendev-sh/zenflow)
- - **10 lifecycle hooks**: Observability (`OnRequest`, `OnResponse`, `OnToolCallStart`, `OnToolCall`, `OnStepFinish`, `OnFinish`, `OnPanic`) and interceptor (`OnBeforeToolExecute`, `OnAfterToolExecute`, `OnBeforeStep`) hooks for permission gates, secret scanning, output transformation, and loop control
- - **Retry/backoff**: Automatic retry with exponential backoff on retryable HTTP errors (429/5xx)
- - **Minimal dependencies**: Core depends on `golang.org/x/oauth2` and `github.com/coder/websocket` (realtime sessions; no transitive dependencies) + one indirect (`cloud.google.com/go/compute/metadata`). Optional `observability/otel` and `observability/langfuse` submodules each use their own separate `go.mod`.
- ## Performance vs Vercel AI SDK
- | Metric | GoAI | Vercel AI SDK | Improvement |
- | -------------------- | ------ | ------------- | ----------- |
- | Streaming throughput | 1.46ms | 1.62ms | 1.1x faster |
- | Cold start | 569us | 13.9ms | 24x faster |
- | Memory (1 stream) | 220KB | 676KB | 3.1x less |
- | GenerateText | 56us | 79us | 1.4x faster |
- > Mock HTTP server, identical SSE fixtures, Apple M2. [Full report](bench/RESULTS.md)
- ## Install
- ```bash
- go get github.com/zendev-sh/goai@latest
- ```
- Requires Go 1.25+.
- ## Quick Start
- Most hosted providers auto-resolve API keys from environment variables. Local/custom providers may require explicit options:
- ```go
- package main
- import (
- "context"
- "fmt"
- "log"
- "github.com/zendev-sh/goai"
- "github.com/zendev-sh/goai/provider/openai"
- )
- func main() {
- // Reads OPENAI_API_KEY from environment automatically.
- model := openai.Chat("gpt-4o")
- result, err := goai.GenerateText(context.Background(), model,
- goai.WithPrompt("What is the capital of France?"),
- )
- if err != nil {
- log.Fatal(err)
- }
- fmt.Println(result.Text)
- }
- ```
- ## Streaming
- ```go
- ctx := context.Background()
- stream, err := goai.StreamText(ctx, model,
- goai.WithSystem("You are a helpful assistant."),
- goai.WithPrompt("Write a haiku about Go."),
- )
- if err != nil {
- log.Fatal(err)
- }
- for text := range stream.TextStream() {
- fmt.Print(text)
- }
- result := stream.Result()
- if err := stream.Err(); err != nil {
- log.Fatal(err)
- }
- fmt.Printf("\nTokens: %d in, %d out\n",
- result.TotalUsage.InputTokens, result.TotalUsage.OutputTokens)
- ```
- OpenAI Responses streams enforce a five-minute provider-event idle timeout.
- Configure this per model with `openai.WithResponsesStreamIdleTimeout`; pass `0`
- to explicitly disable the watchdog. Premature EOF and a bare `[DONE]` sentinel
- are rejected by default. Non-standard endpoints that require `[DONE]` can opt
- in explicitly with `openai.WithResponsesStreamDoneCompatibility(true)`; this
- option changes only terminal-sentinel handling and does not relax event schema
- validation. Every JSON data event must identify its type through either the JSON
- `type` property or the SSE `event:` field, and malformed recognized events
- terminate the stream. Endpoints should use SSE comments, not untyped data
- frames, for keepalives. Idle and protocol interruptions are exposed as typed
- `openai.StreamIdleTimeoutError` and `openai.StreamProtocolError` values.
- Streaming with tools:
- ```go
- import "github.com/zendev-sh/goai/provider"
- stream, err := goai.StreamText(ctx, model,
- goai.WithPrompt("What's the weather in Tokyo?"),
- goai.WithTools(weatherTool),
- goai.WithMaxSteps(5),
- )
- for chunk := range stream.Stream() {
- switch chunk.Type {
- case provider.ChunkText:
- fmt.Print(chunk.Text)
- case provider.ChunkStepFinish:
- fmt.Println("\n[step complete]")
- }
- }
- ```
- ## Structured Output
- Auto-generates JSON Schema from Go types. Works with OpenAI, Anthropic, and Google.
- ```go
- type Recipe struct {
- Name string `json:"name" jsonschema:"description=Recipe name"`
- Ingredients []string `json:"ingredients"`
- Steps []string `json:"steps"`
- Difficulty string `json:"difficulty" jsonschema:"enum=easy|medium|hard"`
- }
- result, err := goai.GenerateObject[Recipe](ctx, model,
- goai.WithPrompt("Give me a recipe for chocolate chip cookies"),
- )
- if err != nil {
- log.Fatal(err)
- }
- fmt.Printf("Recipe: %s (%s)\n", result.Object.Name, result.Object.Difficulty)
- ```
- Streaming partial objects:
- ```go
- stream, err := goai.StreamObject[Recipe](ctx, model,
- goai.WithPrompt("Give me a recipe for pancakes"),
- )
- if err != nil {
- log.Fatal(err)
- }
- for partial := range stream.PartialObjectStream() {
- fmt.Printf("\r%s (%d ingredients so far)", partial.Name, len(partial.Ingredients))
- }
- final, err := stream.Result()
- ```
- ## Tools
- Define a tool from a typed input struct with `goai.NewTool` - the JSON Schema is
- generated from the struct and arguments are unmarshaled for you. Set `MaxSteps`
- to enable the auto tool loop.
- ```go
- weatherTool := goai.NewTool("get_weather", "Get the current weather for a city.",
- func(ctx context.Context, args struct {
- City string `json:"city" jsonschema:"description=City name"`
- }) (string, error) {
- return fmt.Sprintf("22°C and sunny in %s", args.City), nil
- })
- result, err := goai.GenerateText(ctx, model,
- goai.WithPrompt("What's the weather in Tokyo?"),
- goai.WithTools(weatherTool),
- goai.WithMaxSteps(3),
- )
- if err != nil {
- log.Fatal(err)
- }
- fmt.Println(result.Text) // "It's 22°C and sunny in Tokyo."
- ```
- ## MCP (Model Context Protocol)
- Connect to any MCP server and use its tools with GoAI. Supports stdio, Streamable HTTP, and legacy SSE transports.
- ```go
- import "github.com/zendev-sh/goai/mcp"
- // Connect to any MCP server
- transport := mcp.NewStdioTransport("npx", []string{"-y", "@modelcontextprotocol/server-filesystem", "."})
- client := mcp.NewClient("my-app", "1.0", mcp.WithTransport(transport))
- _ = client.Connect(ctx)
- defer client.Close()
- // Use MCP tools with GoAI
- tools, _ := client.ListTools(ctx, nil)
- goaiTools := mcp.ConvertTools(client, tools.Tools)
- result, _ := goai.GenerateText(ctx, model,
- goai.WithTools(goaiTools...),
- goai.WithPrompt("List files in the current directory"),
- goai.WithMaxSteps(5),
- )
- ```
- For OAuth-protected remote servers, GoAI handles authorization-server discovery, dynamic client registration, PKCE, token refresh, and retry-on-401 natively — no external wrapper needed:
- ```go
- ts, _ := mcp.NewOAuthTokenSource(ctx, mcp.OAuthConfig{
- ServerURL: serverURL,
- RedirectURI: "http://127.0.0.1:8765/callback",
- Authorize: openBrowserAndAwaitRedirect, // returns the OAuth redirect URL
- })
- transport := mcp.NewHTTPTransport(serverURL, mcp.WithHTTPClient(mcp.NewOAuthHTTPClient(ts, nil)))
- client := mcp.NewClient("my-app", "1.0", mcp.WithTransport(transport))
- ```
- See [examples/mcp-tools](examples/mcp-tools/), [examples/mcp-oauth](examples/mcp-oauth/), and the [MCP documentation](https://goai.sh/concepts/mcp) for more.
- ## Companion Projects
- **[zenflow](https://github.com/zendev-sh/zenflow)** - multi-agent orchestration engine for Go. Declarative YAML workflows, typed inter-agent mailboxes, single static binary. Built on GoAI for the LLM layer; reach for it when one `MaxSteps` loop is no longer enough.
- ## Citations / Sources
- Providers that support grounding (Google, xAI, Perplexity) or inline citations (OpenAI) return sources:
- ```go
- result, err := goai.GenerateText(ctx, model,
- goai.WithPrompt("What were the major news events today?"),
- )
- if err != nil {
- log.Fatal(err)
- }
- if len(result.Sources) > 0 {
- for _, s := range result.Sources {
- fmt.Printf("[%s] %s - %s\n", s.Type, s.Title, s.URL)
- }
- }
- // Sources are also available per-step in multi-step tool loops.
- for _, step := range result.Steps {
- for _, s := range step.Sources {
- fmt.Printf(" Step source: %s\n", s.URL)
- }
- }
- ```
- ## Computer Use
- See [Provider-Defined Tools > Computer Use](#computer-use-1) and [examples/computer-use](examples/computer-use/) for Anthropic computer, bash, and text editor tools. Works with both Anthropic direct API and Bedrock.
- ## Embeddings
- ```go
- ctx := context.Background()
- model := openai.Embedding("text-embedding-3-small")
- // Single
- result, err := goai.Embed(ctx, model, "Hello world")
- if err != nil {
- log.Fatal(err)
- }
- fmt.Printf("Dimensions: %d\n", len(result.Embedding))
- // Batch (auto-chunked, parallel)
- many, err := goai.EmbedMany(ctx, model, []string{"foo", "bar", "baz"},
- goai.WithMaxParallelCalls(4),
- )
- if err != nil {
- log.Fatal(err)
- }
- ```
- ## Image Generation
- ```go
- ctx := context.Background()
- model := openai.Image("gpt-image-1")
- result, err := goai.GenerateImage(ctx, model,
- goai.WithImagePrompt("A sunset over mountains, oil painting style"),
- goai.WithImageSize("1024x1024"),
- )
- if err != nil {
- log.Fatal(err)
- }
- os.WriteFile("sunset.png", result.Images[0].Data, 0644)
- ```
- Also supported: Google Imagen (`google.Image("imagen-4.0-generate-001")`) and Vertex AI (`vertex.Image(...)`).
- ## Video Generation
- Google Veo video generation uses a long-running operation. GoAI starts the operation, polls until completion, and downloads the resulting video bytes.
- ```go
- ctx := context.Background()
- model := google.Video("veo-3.1-generate-preview")
- result, err := goai.GenerateVideo(ctx, model,
- goai.WithVideoPrompt("A paper plane taking flight at sunrise"),
- goai.WithVideoAspectRatio("16:9"),
- goai.WithVideoResolution("720p"),
- goai.WithVideoDuration(8*time.Second),
- goai.WithVideoAudio(true),
- )
- if err != nil {
- log.Fatal(err)
- }
- os.WriteFile("plane.mp4", result.Video.Data, 0644)
- ```
- Image-to-video is supported with `WithVideoImage`. First/last frames and image references are available through `WithVideoFrameImages` and `WithVideoInputReferences`.
- ## Realtime Voice
- Open a bidirectional voice session: stream audio in, play audio out, and let the model call tools mid-conversation. Experimental.
- ```go
- model := google.Live("gemini-3.8-live") // or vertex.Live(...) on Vertex AI
- session, err := goai.OpenRealtimeSession(ctx, model, goai.WithRealtimeTools(weatherTool))
- if err != nil {
- panic(err)
- }
- defer session.Close()
- go func() {
- for frame := range mic { // 16 kHz mono PCM16
- _ = session.InputAudio(ctx, frame)
- }
- }()
- for ev := range session.Events() {
- if ev.Type == provider.RealtimeOutputAudio {
- play(ev.Audio) // 24 kHz mono PCM16
- }
- }
- ```
- See [examples/realtime-voice](examples/realtime-voice/) and the [realtime documentation](https://goai.sh/concepts/realtime).
- ## Observability
- Built-in [Langfuse](https://langfuse.com) and [OpenTelemetry](https://opentelemetry.io) integrations. Ten lifecycle hooks cover the full generation pipeline -- observability providers use them to trace LLM calls, tool executions, panics, and multi-step agent loops:
- ```go
- import "github.com/zendev-sh/goai/observability/langfuse"
- // Credentials from env: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
- result, err := goai.GenerateText(ctx, model,
- goai.WithPrompt("Hello"),
- goai.WithTools(weatherTool),
- goai.WithMaxSteps(5),
- langfuse.WithTracing(langfuse.TraceName("my-agent")),
- )
- ```
- Interceptor hooks let you control tool execution without modifying core code:
- ```go
- // Permission gate: block dangerous tools
- goai.WithOnBeforeToolExecute(func(info goai.BeforeToolExecuteInfo) goai.BeforeToolExecuteResult {
- if info.ToolName == "delete_file" {
- return goai.BeforeToolExecuteResult{Skip: true, Result: "Permission denied."}
- }
- return goai.BeforeToolExecuteResult{}
- }),
- // Detect max-steps exhaustion
- goai.WithOnFinish(func(info goai.FinishInfo) {
- if info.StepsExhausted {
- log.Printf("Loop exhausted after %d steps", info.TotalSteps)
- }
- }),
- ```
- See [examples/hooks](examples/hooks/), [examples/langfuse](examples/langfuse/), [examples/otel](examples/otel/), and the [observability docs](https://goai.sh/concepts/observability) for details.
- ## Providers
- Many providers auto-resolve credentials from environment variables. Others (for example `ollama`, `vllm`, `compat`) use explicit options:
- ```go
- // Auto-resolved: reads OPENAI_API_KEY from env
- model := openai.Chat("gpt-4o")
- // Explicit key (overrides env)
- model := openai.Chat("gpt-4o", openai.WithAPIKey("sk-..."))
- // Cloud IAM auth (Vertex, Bedrock)
- model := vertex.Chat("gemini-2.5-pro",
- vertex.WithProject("my-project"),
- vertex.WithLocation("us-central1"),
- )
- // AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION from env)
- model := bedrock.Chat("anthropic.claude-sonnet-4-6-v1:0")
- // Local (Ollama, vLLM)
- model := ollama.Chat("llama3", ollama.WithBaseURL("http://localhost:11434"))
- result, err := goai.GenerateText(ctx, model, goai.WithPrompt("Hello"))
- ```
- ### Provider Table
- | Provider | Chat | Embed | Image | Auth | E2E | Import |
- | ---------- | ------------------------------------------------------------ | ---------------------------------------------------------- | ------------- | -------------------------------------------------------------------------------------------------- | ---- | --------------------- |
- | OpenAI | `gpt-4o`, `o3`, `codex-*` | `text-embedding-3-*` | `gpt-image-1` | `OPENAI_API_KEY`, `OPENAI_BASE_URL`, TokenSource | Full | `provider/openai` |
- | Anthropic | `claude-*` | - | - | `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, TokenSource | Full | `provider/anthropic` |
- | Google | `gemini-*` | `text-embedding-004` | `imagen-*` | `GOOGLE_GENERATIVE_AI_API_KEY` / `GEMINI_API_KEY`, TokenSource | Full | `provider/google` |
- | Bedrock | `anthropic.*`, `meta.*` | `titan-embed-*`, `cohere.embed-*`, `nova-2-*`, `marengo-*` | - | AWS keys, `AWS_BEARER_TOKEN_BEDROCK`, `AWS_BEDROCK_BASE_URL` | Full | `provider/bedrock` |
- | Vertex | `gemini-*` | `text-embedding-004` | `imagen-*` | TokenSource, ADC, or `GOOGLE_API_KEY` / `GEMINI_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY` fallback | Unit | `provider/vertex` |
- | Azure | `gpt-4o`, `claude-*` | - | via Azure | `AZURE_OPENAI_API_KEY`, TokenSource | Full | `provider/azure` |
- | OpenRouter | various | - | - | `OPENROUTER_API_KEY`, TokenSource | Unit | `provider/openrouter` |
- | Mistral | `mistral-large`, `magistral-*` | - | - | `MISTRAL_API_KEY`, TokenSource | Full | `provider/mistral` |
- | Groq | `mixtral-*`, `llama-*` | - | - | `GROQ_API_KEY`, TokenSource | Full | `provider/groq` |
- | xAI | `grok-*` | - | - | `XAI_API_KEY`, TokenSource | Unit | `provider/xai` |
- | Cohere | `command-r-*` | `embed-*` | - | `COHERE_API_KEY`, TokenSource | Unit | `provider/cohere` |
- | DeepSeek | `deepseek-*` | - | - | `DEEPSEEK_API_KEY`, TokenSource | Unit | `provider/deepseek` |
- | MiniMax | `MiniMax-M2.7`, `MiniMax-M2.5`, `MiniMax-M2.1`, `MiniMax-M2` | - | - | `MINIMAX_API_KEY`, `MINIMAX_BASE_URL`, TokenSource | Full | `provider/minimax` |
- | Fireworks | various | - | - | `FIREWORKS_API_KEY`, TokenSource | Unit | `provider/fireworks` |
- | Together | various | - | - | `TOGETHER_AI_API_KEY` (or `TOGETHER_API_KEY`), TokenSource | Unit | `provider/together` |
- | DeepInfra | various | - | - | `DEEPINFRA_API_KEY`, TokenSource | Unit | `provider/deepinfra` |
- | Perplexity | `sonar-*` | - | - | `PERPLEXITY_API_KEY`, TokenSource | Unit | `provider/perplexity` |
- | Cerebras | `llama-*` | - | - | `CEREBRAS_API_KEY`, TokenSource | Unit | `provider/cerebras` |
- | Ollama | local models | local models | - | none | Unit | `provider/ollama` |
- | vLLM | local models | local models | - | Optional auth via `WithAPIKey` / `WithTokenSource` | Unit | `provider/vllm` |
- | RunPod | any vLLM model | - | - | `RUNPOD_API_KEY`, TokenSource | Unit | `provider/runpod` |
- | Runcrate | open models (e.g. `deepseek-ai/DeepSeek-V4.1-Flash`) | - | - | `RUNCRATE_API_KEY`, `RUNCRATE_BASE_URL`, TokenSource | Unit | `provider/runcrate` |
- | Cloudflare | `@cf/meta/*`, `@cf/openai/gpt-oss-*`, `@cf/qwen/*` | `@cf/baai/bge-*` | - | `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_BASE_URL`, TokenSource | Unit | `provider/cloudflare` |
- | FPT Cloud | `Qwen3-*`, `Llama-*`, `gpt-oss-*`, `GLM-*`, `gemma-*` | `bge-*`, `gte-*`, `multilingual-e5-*` | - | `FPT_API_KEY`, `FPT_REGION` (`global`/`jp`), `FPT_BASE_URL`, TokenSource | Unit | `provider/fptcloud` |
- | NVIDIA NIM | `nvidia/llama-*`, `nvidia/nemotron-*` | `nvidia/nv-embed-*` | - | `NVIDIA_API_KEY`, `NVIDIA_BASE_URL`, TokenSource | Full | `provider/nvidia` |
- | llama.cpp | local models | local models | - | Optional auth via `WithAPIKey` / `WithTokenSource` | Unit | `provider/llamacpp` |
- | Requesty | `provider/model` (e.g. `openai/gpt-4o-mini`) | - | - | `REQUESTY_API_KEY`, `REQUESTY_BASE_URL`, TokenSource | Unit | `provider/requesty` |
- | Heabsy | open models (e.g. `qwen38`) | - | - | `HEABSY_API_KEY`, `HEABSY_BASE_URL`, TokenSource | Unit | `provider/heabsy` |
- | Opper | pool names (e.g. `claude-sonnet-4-6`) | - | - | `OPPER_API_KEY`, `OPPER_BASE_URL`, TokenSource | Unit | `provider/opper` |
- | Cheaper Inference | bare model IDs (e.g. `gpt-5.4-mini`) | - | - | `CHEAPER_INFERENCE_API_KEY`, `CHEAPER_INFERENCE_BASE_URL`, TokenSource | Unit | `provider/cheaperinference` |
- | CloudGPU | bare model IDs (e.g. `deepseek-v4.1-flash`) | - | - | `CLOUDGPU_API_KEY`, `CLOUDGPU_BASE_URL`, TokenSource | Unit | `provider/cloudgpu` |
- | Compat | any OpenAI-compatible | any | - | configurable | Unit | `provider/compat` |
- **E2E column**: "Full" = tested with real API calls. "Unit" = tested with mock HTTP servers (100% coverage).
- ### Tested Models
- <details>
- <summary><strong>E2E tested - 104 models across 8 providers</strong> (real API calls, click to expand)</summary>
- Last run: 2026-03-27. 104 models tested (generate + stream).
- | Provider | Models E2E tested (generate + stream) |
- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
- | Google (9) | `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.5-pro` (stream), `gemini-3-flash-preview`, `gemini-3-pro-preview`, `gemini-3.1-pro-preview`, `gemini-2.0-flash`, `gemini-flash-latest`, `gemini-flash-lite-latest` |
- | Azure (21) | `claude-opus-4-6`, `claude-sonnet-4-6`, `DeepSeek-V3.2`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-5`, `gpt-5-codex`, `gpt-5-mini`, `gpt-5-pro`, `gpt-5.1`, `gpt-5.1-codex`, `gpt-5.1-codex-max`, `gpt-5.1-codex-mini`, `gpt-5.2`, `gpt-5.2-codex`, `gpt-5.3-codex`, `gpt-5.4`, `gpt-5.4-pro`, `Kimi-K2.5`, `model-router`, `o3` |
- | Bedrock (61) | **Anthropic**: `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-opus-4-6-v1`, `claude-opus-4-5`, `claude-opus-4-1`, `claude-haiku-4-5`, `claude-3-5-sonnet`, `claude-3-5-haiku`, `claude-3-haiku` · **Amazon**: `nova-micro`, `nova-lite`, `nova-pro`, `nova-premier`, `nova-2-lite` · **Meta**: `llama4-scout`, `llama4-maverick`, `llama3-3-70b`, `llama3-2-{90,11,3,1}b`, `llama3-1-{70,8}b`, `llama3-{70,8}b` · **Mistral**: `mistral-large`, `mixtral-8x7b`, `mistral-7b`, `ministral-3-{14,8}b`, `voxtral-{mini,small}` · **Others**: `deepseek.v3`, `deepseek.r1`, `ai21.jamba-1-5-{mini,large}`, `cohere.command-r{-plus,}`, `google.gemma-3-{4,12,27}b`, `minimax.{m2,m2.1}`, `moonshotai.kimi-k2{-thinking,.5}`, `nvidia.nemotron-nano-{12,9}b`, `openai.gpt-oss-{120,20}b{,-safeguard}`, `qwen.qwen3-{32,235,coder-30,coder-480}b`, `qwen.qwen3-next-80b`, `writer.palmyra-{x4,x5}`, `zai.glm-4.7{,-flash}` |
- | Groq (2) | `llama-3.1-8b-instant`, `llama-3.3-70b-versatile` |
- | Mistral (5) | `mistral-small-latest`, `mistral-large-latest`, `devstral-small-2507`, `codestral-latest`, `magistral-medium-latest` |
- | Cerebras (1) | `llama3.1-8b` |
- | MiniMax (4) | `MiniMax-M2.7`, `MiniMax-M2.5`, `MiniMax-M2.1`, `MiniMax-M2` (generate + stream + tools + thinking) |
- | NVIDIA (1) | `meta/llama-3.3-70b-instruct` |
- </details>
- <details>
- <summary><strong>Unit tested</strong> (mock HTTP server, 100% coverage, click to expand)</summary>
- | Provider | Models in unit tests |
- | ---------- | -------------------------------------------------------------------------------------------------- |
- | OpenAI | `gpt-4o`, `o3`, `text-embedding-3-small`, `dall-e-3`, `gpt-image-1` |
- | Anthropic | `claude-sonnet-4-20250514`, `claude-sonnet-4-5-20241022`, `claude-sonnet-4-6-20260310` |
- | Google | `gemini-2.5-flash`, `gemini-2.5-flash-image`, `imagen-4.0-fast-generate-001`, `text-embedding-004` |
- | Bedrock | `us.anthropic.claude-sonnet-4-6`, `anthropic.claude-sonnet-4-20250514-v1:0`, `meta.llama3-70b` |
- | Azure | `gpt-4o`, `gpt-5.2-chat`, `dall-e-3`, `claude-sonnet-4-6` |
- | Vertex | `gemini-2.5-pro`, `imagen-3.0-generate-002`, `text-embedding-004` |
- | Cohere | `command-r-plus`, `command-a-reasoning`, `embed-v4.0` |
- | Mistral | `mistral-large-latest` |
- | Groq | `llama-3.3-70b-versatile` |
- | xAI | `grok-3` |
- | DeepSeek | `deepseek-chat` |
- | DeepInfra | `meta-llama/Llama-3.3-70B-Instruct` |
- | Fireworks | `accounts/fireworks/models/llama-v3p3-70b-instruct` |
- | OpenRouter | `anthropic/claude-sonnet-4` |
- | Perplexity | `sonar-pro` |
- | Together | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
- | Cerebras | `llama-3.3-70b` |
- | NVIDIA | `meta/llama-3.3-70b-instruct` |
- | Ollama | `llama3`, `llama3.2:1b`, `nomic-embed-text` |
- | vLLM | `meta-llama/Llama-3-8b` |
- | RunPod | `meta-llama/Llama-3.3-70B-Instruct` |
- | Cloudflare | `@cf/meta/llama-3.1-8b-instruct`, `@cf/baai/bge-base-en-v1.5` |
- | FPT Cloud | `Qwen3-32B` |
- </details>
- ### Custom / Self-Hosted
- Use the `compat` provider for any OpenAI-compatible endpoint:
- ```go
- model := compat.Chat("my-model",
- compat.WithBaseURL("https://my-api.example.com/v1"),
- compat.WithAPIKey("..."),
- )
- ```
- ### Dynamic Auth with TokenSource
- For OAuth, rotating keys, or cloud IAM:
- ```go
- ts := provider.CachedTokenSource(func(ctx context.Context) (*provider.Token, error) {
- tok, err := fetchOAuthToken(ctx)
- return &provider.Token{
- Value: tok.AccessToken,
- ExpiresAt: tok.Expiry,
- }, err
- })
- model := openai.Chat("gpt-4o", openai.WithTokenSource(ts))
- ```
- `CachedTokenSource` handles TTL-based caching (zero ExpiresAt = cache forever), thread-safe refresh without holding locks during network calls, and manual token invalidation via the `InvalidatingTokenSource` interface.
- ### AWS Bedrock
- Native Converse API with SigV4 signing (no AWS SDK dependency). Supports cross-region inference fallback, extended thinking, and image/document input:
- ```go
- model := bedrock.Chat("anthropic.claude-sonnet-4-6-v1:0",
- bedrock.WithRegion("us-west-2"),
- bedrock.WithReasoningConfig(bedrock.ReasoningConfig{
- Type: bedrock.ReasoningEnabled,
- BudgetTokens: 4096,
- }),
- )
- ```
- Auto-resolves `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` from environment. Cross-region fallback retries with `us.` prefix on model ID mismatch errors.
- ### Azure OpenAI
- Supports both OpenAI models (GPT, o-series) and Claude models (routed to Azure Anthropic endpoint automatically):
- ```go
- // OpenAI models
- model := azure.Chat("gpt-4o",
- azure.WithEndpoint("https://my-resource.openai.azure.com"),
- )
- // Claude models (auto-routed to Anthropic endpoint)
- model := azure.Chat("claude-sonnet-4-6",
- azure.WithEndpoint("https://my-resource.openai.azure.com"),
- )
- ```
- Auto-resolves `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT` (or `AZURE_RESOURCE_NAME`) from environment.
- ### Response Metadata
- Every result includes provider response metadata:
- ```go
- result, _ := goai.GenerateText(ctx, model, goai.WithPrompt("Hello"))
- fmt.Printf("Request ID: %s\n", result.Response.ID)
- fmt.Printf("Model used: %s\n", result.Response.Model)
- ```
- ## Options Reference
- ### Generation Options
- | Option | Description | Default |
- | ------------------------------- | ---------------------------------------- | ---------------- |
- | `WithSystem(s)` | System prompt | - |
- | `WithPrompt(s)` | Single user message | - |
- | `WithMessages(...)` | Conversation history | - |
- | `WithTools(...)` | Available tools | - |
- | `WithMaxOutputTokens(n)` | Response length limit | provider default |
- | `WithTemperature(t)` | Randomness (0.0-2.0) | provider default |
- | `WithTopP(p)` | Nucleus sampling | provider default |
- | `WithTopK(k)` | Top-K sampling | provider default |
- | `WithFrequencyPenalty(p)` | Frequency penalty | provider default |
- | `WithPresencePenalty(p)` | Presence penalty | provider default |
- | `WithSeed(s)` | Deterministic generation | - |
- | `WithStopSequences(...)` | Stop triggers | - |
- | `WithMaxSteps(n)` | Tool loop iterations | 1 (no loop) |
- | `WithSequentialToolExecution()` | Execute tools one at a time | parallel |
- | `WithMaxRetries(n)` | Retries on 429/5xx | 2 |
- | `WithTimeout(d)` | Overall timeout | none |
- | `WithHeaders(h)` | Per-request HTTP headers | - |
- | `WithProviderOptions(m)` | Provider-specific params | - |
- | `WithPromptCaching(b)` | Enable prompt caching | false |
- | `WithToolChoice(tc)` | "auto", "none", "required", or tool name | - |
- ### Lifecycle Hooks
- | Option | Description |
- | ----------------------------- | --------------------------------------------------------------- |
- | `WithOnRequest(fn)` | Called before each API call |
- | `WithOnResponse(fn)` | Called after each API call |
- | `WithOnToolCallStart(fn)` | Called before each tool execution begins |
- | `WithOnToolCall(fn)` | Called after each tool execution |
- | `WithOnStepFinish(fn)` | Called after each tool loop step |
- | `WithOnFinish(fn)` | Called once after all steps complete (carries `StepsExhausted`) |
- | `WithOnBeforeToolExecute(fn)` | Intercept before tool Execute -- can skip, override ctx/input |
- | `WithOnAfterToolExecute(fn)` | Intercept after tool Execute -- can modify output/error |
- | `WithOnBeforeStep(fn)` | Intercept before step 2+ -- can inject messages or stop loop |
- ### Structured Output Options
- | Option | Description |
- | ----------------------- | --------------------------------------------- |
- | `WithExplicitSchema(s)` | Override auto-generated JSON Schema |
- | `WithSchemaName(n)` | Schema name for provider (default "response") |
- ### Embedding Options
- | Option | Description | Default |
- | --------------------------------- | ------------------------- | ------- |
- | `WithMaxParallelCalls(n)` | Batch parallelism | 4 |
- | `WithEmbeddingProviderOptions(m)` | Embedding provider params | - |
- ### Image Options
- | Option | Description |
- | ----------------------------- | ------------------------------ |
- | `WithImagePrompt(s)` | Text description |
- | `WithImageCount(n)` | Number of images |
- | `WithImageSize(s)` | Dimensions (e.g., "1024x1024") |
- | `WithAspectRatio(s)` | Aspect ratio (e.g., "16:9") |
- | `WithImageMaxRetries(n)` | Retries on 429/5xx |
- | `WithImageTimeout(d)` | Overall timeout |
- | `WithImageProviderOptions(m)` | Image provider params |
- ## Error Handling
- GoAI generation and image APIs return typed errors for actionable failure modes (MCP client APIs return `*mcp.MCPError`):
- ```go
- result, err := goai.GenerateText(ctx, model, goai.WithPrompt("..."))
- if err != nil {
- var overflow *goai.ContextOverflowError
- var apiErr *goai.APIError
- switch {
- case errors.As(err, &overflow):
- // Prompt too long - truncate and retry
- case errors.As(err, &apiErr):
- if apiErr.IsRetryable {
- // 429 rate limit, 503 - already retried MaxRetries times
- }
- fmt.Printf("API error %d: %s\n", apiErr.StatusCode, apiErr.Message)
- // HTTP API errors include ResponseBody and ResponseHeaders for debugging
- default:
- // Network error, context cancelled, etc.
- }
- }
- ```
- Error types:
- | Type | Fields | When |
- | ---------------------- | ------------------------------------------------------------------------- | ----------------------------------- |
- | `APIError` | `StatusCode`, `Message`, `IsRetryable`, `ResponseBody`, `ResponseHeaders` | Non-2xx API responses |
- | `ContextOverflowError` | `Message`, `ResponseBody` | Prompt exceeds model context window |
- Retry behavior: automatic exponential backoff on retryable HTTP errors (429/5xx, plus OpenAI 404 propagation). `retry-after-ms` and `Retry-After` (both delay-seconds and HTTP-date) are respected. Retries apply to request-level failures (including initial stream connection), not mid-stream error events.
- ## Provider-Defined Tools
- Providers expose built-in tools that the model can invoke server-side. GoAI supports 26 provider-defined tools across 5 providers:
- | Provider | Tools | Import |
- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
- | Anthropic | `Computer`, `Computer_20251124`, `Bash`, `TextEditor`, `TextEditor_20250728`, `WebSearch`, `WebSearch_20260209`, `WebSearch_20260318`, `WebFetch`, `CodeExecution`, `CodeExecution_20250522`, `CodeExecution_20250825`, `ToolSearchToolRegex`, `ToolSearchToolBM25` | `provider/anthropic` |
- | OpenAI | `WebSearch`, `CodeInterpreter`, `FileSearch`, `ImageGeneration` | `provider/openai` |
- | Google | `GoogleSearch`, `URLContext`, `CodeExecution`, `ComputerUse`, `FileSearch` | `provider/google` |
- | xAI | `WebSearch`, `XSearch` | `provider/xai` |
- | Groq | `BrowserSearch` | `provider/groq` |
- All tools follow the same pattern: create a definition with `<provider>.Tools.ToolName()` (e.g., `openai.Tools`, `anthropic.Tools`), then pass it as a `goai.Tool`:
- ```go
- // Example: def := openai.Tools.WebSearch(openai.WithSearchContextSize("medium"))
- def := <provider>.Tools.ToolName(options...)
- result, _ := goai.GenerateText(ctx, model,
- goai.WithTools(goai.Tool{
- Name: def.Name,
- ProviderDefinedType: def.ProviderDefinedType,
- ProviderDefinedOptions: def.ProviderDefinedOptions,
- }),
- )
- ```
- ### Web Search
- The model searches the web and returns grounded responses. Available from OpenAI, Anthropic, Google, and Groq.
- ```go
- // OpenAI (via Responses API) - also works via Azure
- def := openai.Tools.WebSearch(openai.WithSearchContextSize("medium"))
- // Anthropic (via Messages API) - also works via Bedrock
- def := anthropic.Tools.WebSearch(anthropic.WithMaxUses(5))
- // Google (grounding with Google Search) - returns Sources
- def := google.Tools.GoogleSearch()
- // result.Sources contains grounding URLs from Google Search
- // Groq (interactive browser search)
- def := groq.Tools.BrowserSearch()
- ```
- ### Code Execution
- The model writes and runs code in a sandboxed environment. Server-side, no local setup needed.
- ```go
- // OpenAI Code Interpreter - Python sandbox via Responses API
- def := openai.Tools.CodeInterpreter()
- // Anthropic Code Execution - Python sandbox via Messages API
- def := anthropic.Tools.CodeExecution() // v20260120, GA, no beta needed
- // Google Code Execution - Python sandbox via Gemini API
- def := google.Tools.CodeExecution()
- ```
- ### Web Fetch
- Claude fetches and processes content from specific URLs directly.
- ```go
- def := anthropic.Tools.WebFetch(
- anthropic.WithWebFetchMaxUses(3),
- anthropic.WithCitations(true),
- )
- ```
- ### File Search
- Semantic search over uploaded files in vector stores (OpenAI Responses API).
- ```go
- def := openai.Tools.FileSearch(
- openai.WithVectorStoreIDs("vs_abc123"),
- openai.WithMaxNumResults(5),
- )
- ```
- ### Image Generation
- LLM generates images inline during conversation (different from `goai.GenerateImage()` which calls the Images API directly).
- ```go
- def := openai.Tools.ImageGeneration(
- openai.WithImageQuality("low"),
- openai.WithImageSize("1024x1024"),
- )
- // On Azure, also set: azure.WithHeaders(map[string]string{
- // "x-ms-oai-image-generation-deployment": "gpt-image-1.5",
- // })
- ```
- ### Computer Use
- Anthropic computer, bash, and text editor tools for autonomous desktop interaction. Client-side execution required.
- ```go
- computerDef := anthropic.Tools.Computer(anthropic.ComputerToolOptions{
- DisplayWidthPx: 1920, DisplayHeightPx: 1080,
- })
- bashDef := anthropic.Tools.Bash()
- textEditorDef := anthropic.Tools.TextEditor()
- // Wrap each with an Execute handler for client-side execution
- ```
- ### URL Context
- Gemini fetches and processes web content from URLs in the prompt.
- ```go
- def := google.Tools.URLContext()
- ```
- See [examples/](examples/) for complete runnable examples of each tool.
- ## Examples
- See the [examples/](examples/) directory:
- - [chat](examples/chat/) - Non-streaming generation
- - [streaming](examples/streaming/) - Real-time text streaming
- - [streaming-tools](examples/streaming-tools/) - Streaming with multi-step tool loops
- - [structured](examples/structured/) - Structured output with Go generics
- - [tools](examples/tools/) - Single tool call
- - [agent-loop](examples/agent-loop/) - Multi-step agent with callbacks
- - [multi-turn](examples/multi-turn/) - Multi-turn conversation with ResponseMessages
- - [citations](examples/citations/) - Accessing sources and citations
- - [hooks](examples/hooks/) - Lifecycle hooks: permission gates, secret scanning, loop control, OnFinish
- - [langfuse](examples/langfuse/) - Langfuse tracing integration
- - [otel](examples/otel/) - OpenTelemetry tracing and metrics
- - [computer-use](examples/computer-use/) - Anthropic computer, bash, and text editor tools
- - [embedding](examples/embedding/) - Embeddings with similarity search
- - [web-search](examples/web-search/) - Web search across providers (OpenAI, Anthropic, Google)
- - [web-fetch](examples/web-fetch/) - Anthropic web fetch tool
- - [code-execution](examples/code-execution/) - Anthropic code execution tool
- - [code-interpreter](examples/code-interpreter/) - OpenAI code interpreter tool
- - [google-search](examples/google-search/) - Google Search grounding with Gemini
- - [google-code-execution](examples/google-code-execution/) - Google Gemini code execution tool
- - [file-search](examples/file-search/) - OpenAI file search tool
- - [image-generation](examples/image-generation/) - OpenAI image generation via Responses API
- - [mcp-tools](examples/mcp-tools/) - MCP tools with GoAI LLM integration
- - [mcp-filesystem](examples/mcp-filesystem/) - Filesystem MCP server via stdio
- - [mcp-github](examples/mcp-github/) - GitHub MCP server via stdio
- - [mcp-playwright](examples/mcp-playwright/) - Playwright MCP server for browser automation
- - [mcp-remote](examples/mcp-remote/) - MCP over Streamable HTTP transport
- - [mcp-sse](examples/mcp-sse/) - MCP over legacy SSE transport
- - [mcp-local](examples/mcp-local/) - MCP client basics (no LLM needed)
- ## Project Structure
- ```
- goai/ # Core SDK
- ├── provider/ # Provider interface + shared types
- │ ├── provider.go # LanguageModel, EmbeddingModel, ImageModel interfaces
- │ ├── types.go # Message, Part, Usage, StreamChunk, etc.
- │ ├── token.go # TokenSource, CachedTokenSource
- │ ├── openai/ # OpenAI (Chat Completions + Responses API)
- │ ├── anthropic/ # Anthropic (Messages API)
- │ ├── google/ # Google Gemini (REST API)
- │ ├── bedrock/ # AWS Bedrock (Converse API + SigV4 + EventStream)
- │ ├── vertex/ # Google Vertex AI (OpenAI-compat)
- │ ├── azure/ # Azure OpenAI
- │ ├── cohere/ # Cohere (Chat v2 + Embed)
- │ ├── minimax/ # MiniMax (Anthropic-compatible API)
- │ ├── compat/ # Generic OpenAI-compatible
- │ └── ... # 23 OpenAI-compatible provider implementation files
- ├── internal/
- │ ├── openaicompat/ # Shared codec for 23 OpenAI-compatible provider implementation files
- │ ├── gemini/ # Schema sanitization (Vertex, Google)
- │ ├── sse/ # SSE line parser
- │ └── httpc/ # HTTP utilities
- ├── examples/ # Usage examples
- └── bench/ # Performance benchmarks (GoAI vs Vercel AI SDK)
- ├── fixtures/ # Shared SSE test fixtures
- ├── go/ # Go benchmarks (go test -bench)
- ├── ts/ # TypeScript benchmarks (Bun + Tinybench)
- ├── collect.sh # Result aggregation → report
- └── Makefile # make bench-all
- ```
- ## Contributing
- See [CONTRIBUTING.md](CONTRIBUTING.md).
- ## License
- [MIT](LICENSE)
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.