Introduction

Anyone who has shipped a non-trivial Gen AI feature to production has hit the same wall: the happy path is fun to demo, but the unhappy path is brutal. Model providers throw 5xx errors. Rate limits kick in at the worst possible moment. A long-running agent gets halfway through a 12-step plan and the pod is restarted by Kubernetes. A user closes the tab in the middle of a streaming response and you have no idea what state the flow ended up in.

Genkit gives you a clean way to author those LLM-orchestrating pipelines as flows, but a Genkit flow, by itself, is just a function. It lives and dies with the process that invokes it. There is no durable history, no automatic retry policy, no built-in cancellation, no operations UI.

This is exactly the gap that Temporal was designed to close for general-purpose backend code, and it is the gap that the new genkitx-temporal plugin closes for Genkit flows.

In this article we will look at:

• What the plugin actually does.
• Why Temporal and Genkit are a particularly good match.
• How to use it end to end: define, register, run.
• When you should reach for it (and when you probably shouldn’t).

What is genkitx-temporal?

genkitx-temporal is a Genkit plugin that lets you execute any Genkit flow inside a Temporal Workflow, transparently. You keep writing flows the way you always have; the plugin takes care of:

• Registering the flow so a Temporal Worker can pick it up.
• Wrapping each execution in a deterministic Temporal Workflow.
• Running the non-deterministic LLM/tool/RAG work inside a Temporal Activity, so retries and timeouts are safe.
• Giving you helpers to start workflows from any client.

Under the hood, the plugin ships a generic, deterministic workflow called runGenkitFlow that invokes a single Temporal Activity called runGenkitFlowActivity. The activity looks up your Genkit flow by name in an in-process registry and runs it inside a full Node environment, where all the messy, non-deterministic things (network calls, model calls, tool calls, RAG lookups) are perfectly fine.

┌────────────┐  start workflow   ┌──────────────────────┐
│  Client    │ ────────────────▶ │  Temporal Server     │
└────────────┘                   └─────────┬────────────┘
                                           │ task
                                           ▼
                                ┌──────────────────────┐
                                │  Worker process      │
                                │  ┌────────────────┐  │
                                │  │ runGenkitFlow  │  │  (workflow, sandboxed)
                                │  │       │        │  │
                                │  │       ▼        │  │
                                │  │ runGenkit-     │  │  (activity, full Node)
                                │  │ FlowActivity   │  │
                                │  │       │        │  │
                                │  │       ▼        │  │
                                │  │  your Genkit   │  │
                                │  │  flow (LLM…)   │  │
                                │  └────────────────┘  │
                                └──────────────────────┘

This split is the whole trick. Temporal Workflows are required to be deterministic so that they can be replayed from event history after a crash, deploy or scale event. LLM calls obviously are not deterministic. Putting the LLM work inside an Activity is the canonical Temporal pattern, and the plugin does it for you so you don’t have to think about it.

Why Temporal and Genkit are a great match

It is easy to underestimate how much engineering is hiding behind “just call the model again if it fails”. Once you start building real agents, the list of things you want from your runtime grows quickly. Genkit gives you the authoring primitives, and Temporal gives you the runtime guarantees. Together:

• Automatic retries for transient errors. LLM providers regularly return 429s and 5xx. Tool calls hit the network. Vector stores time out. Temporal lets you express retry policies (exponential backoff, max attempts, non-retryable error types) declaratively, applied to every flow execution.
• Durable history. Every event in your flow’s execution is persisted by the Temporal Server. If your Worker pod is killed mid-flow, another Worker picks up the workflow exactly where it left off, with all prior activity results intact. No partial state, no double-charging the user, no orphan jobs.
• Timeouts, heartbeats and cancellation. Long-running agents that browse, plan and call tools for minutes are a nightmare to control with plain HTTP. Temporal models start-to-close, schedule-to-close and heartbeat timeouts as first-class concepts, plus explicit cancellation semantics you can wire to a user closing a tab.
• Operational visibility out of the box. The Temporal UI gives you a per-execution timeline of every workflow and activity. You can see inputs, outputs, retries, failures and stack traces, search by workflow id, terminate or signal running workflows. This complements the Genkit Developer UI nicely: Genkit’s UI is your local debugging tool during development; Temporal’s UI is your operational dashboard in staging and production.
• Horizontal scalability for free. Need more throughput? Run more Worker processes pointing at the same task queue. The Temporal Server load-balances workflows and activities across them. Your flows did not need to change.
• Long-running, human-in-the-loop friendly. Temporal workflows can sleep for days, wait for signals from external systems, and resume cleanly. Perfect for agents that need approval before executing a sensitive tool, or RAG pipelines that wait for a document indexing job to finish.

In other words, genkitx-temporal turns your Genkit flows from “smart functions inside a Node process” into first-class durable workloads without changing the way you write them.

Installation

npm install genkitx-temporal genkit

The Temporal SDK packages are peer-installed automatically as dependencies of the plugin.

You also need a running Temporal Server. For local development, the easiest path is:

brew install temporal
temporal server start-dev

This starts a Temporal Server on localhost:7233 and the UI on http://localhost:8233.

Usage end to end

The plugin exposes a small, focused API. Three things are enough to ship a durable Genkit flow: define it with the Temporal-aware helper, start a Worker, and execute it from a client.

1. Define a flow with defineTemporalFlow

defineTemporalFlow is a drop-in replacement for ai.defineFlow. The returned object is a normal Genkit flow, so you can still call it directly, expose it via the Developer UI, run it from tests, etc. The only difference is that it is also registered internally so a Temporal Worker can find it by name.

// flows.ts
import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';
import { defineTemporalFlow, temporal } from 'genkitx-temporal';

export const ai = genkit({
  plugins: [
    googleAI(),
    temporal({ taskQueue: 'my-queue' }),
  ],
  model: googleAI.model('gemini-flash-latest'),
});

export const jokeFlow = defineTemporalFlow(
  ai,
  {
    name: 'jokeFlow',
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (subject) => {
    const { text } = await ai.generate(`Tell me a joke about ${subject}`);
    return text;
  },
);

A few things to notice:

• You configure the plugin like any other Genkit plugin, passing the Temporal task queue (and optionally address, namespace, etc.).
• The flow body is plain Genkit code. No Temporal-specific imports, no proxyActivities, no determinism gymnastics. The plugin handles all that for you.
• The flow’s name is also its Temporal registration key. Keep it unique within your Worker.

2. Start a Worker

Workers are the processes that actually execute your flows. A Worker imports your flows (so the registry is populated) and then calls startTemporalWorker:

// worker.ts
import './flows';   // side-effect import: registers the flows
import { startTemporalWorker } from 'genkitx-temporal';

startTemporalWorker({ taskQueue: 'my-queue' })
  .catch((e) => { console.error(e); process.exit(1); });

node ./dist/worker.js

You can run as many Worker processes as you want against the same task queue. Temporal will distribute work across them automatically. If a Worker dies mid-flow, another one picks up.

3. Execute a flow as a Temporal Workflow

From any client (an HTTP handler, a CLI, a cron job, another workflow), use executeTemporalFlow to start a workflow and await its result:

// client.ts
import { executeTemporalFlow } from 'genkitx-temporal';
import { jokeFlow } from './flows';

const result = await executeTemporalFlow(jokeFlow, 'cats', {
  taskQueue: 'my-queue',
});
console.log(result);

If you don’t want to block on the result, use startTemporalFlow instead. It returns the raw Temporal WorkflowHandle, which lets you query, signal, or cancel the running workflow later. This is the building block for human-in-the-loop scenarios, scheduled flows, fan-out/fan-in patterns, and so on.

Configuration

temporal(options) and every helper accept the same connection options. Anything you don’t pass falls back to environment variables, then to sensible defaults:

This makes it straightforward to run the same code locally against a dev server and in production against Temporal Cloud or a self-hosted cluster.

Advanced: combine with your own workflows and activities

The bundled runGenkitFlow workflow is enough for the common case. If you want to mix Genkit flows with your own existing Temporal workflows and activities, the plugin lets you bring your own:

await startTemporalWorker({
  taskQueue: 'my-queue',
  workflowsPath: require.resolve('./my-workflows'),
  activities: { ...require('./my-activities') },
});

// my-activities.ts
export { runGenkitFlowActivity } from 'genkitx-temporal/activities';
export async function myOtherActivity(/* ... */) { /* ... */ }

Re-exporting runGenkitFlowActivity keeps the built-in workflow working, so you can compose Genkit flows alongside hand-written activities for the parts of your system that aren’t AI.

API summary

The full public surface is tiny:

• temporal(options?) — the Genkit plugin.
• defineTemporalFlow(ai, config, fn) — define a flow and register it for Temporal execution.
• startTemporalWorker(options?) — start a Worker process.
• executeTemporalFlow(flow, input, options?) — run a flow inside a Workflow and await the result.
• startTemporalFlow(flow, input, options?) — same, but returns the WorkflowHandle for fire-and-forget / signalling.
• runGenkitFlowActivity — the underlying activity, re-exported so you can combine it with your own activities.
• registerTemporalFlow(name, flow) — manually register a flow that was defined elsewhere (useful when wrapping flows you don’t own).

A small API surface is the point. The plugin is intentionally a thin bridge between two well-designed systems; it does not try to reinvent either of them.

When to reach for genkitx-temporal

Not every flow needs a durable runtime. A streaming chat response that takes 800ms and either succeeds or is retried by the user is fine running on a plain HTTP handler.

You will feel the benefits the moment your flows look like one of these:

• Multi-step agents that orchestrate several model calls and tool invocations, where partial progress is expensive to throw away.
• Long-running pipelines (document ingestion, batch summarization, fine-tuning prep) where individual steps can take minutes and the process must survive deploys.
• Critical business workflows (refunds, account changes, contract generation) where you cannot afford to lose state or accidentally execute a step twice.
• Human-in-the-loop agents that need to pause for approval, an external webhook, or a manual review before proceeding.
• Anything you currently glue together with a job queue, a retry library, and a state machine. Temporal subsumes all three, and genkitx-temporal plugs Genkit straight into it.

If your team already runs Temporal for non-AI workloads, this plugin is a no-brainer: it lets your Gen AI features inherit all the operational maturity your platform team has already built around it.

How it pairs with the rest of the Genkit ecosystem

The thing I like the most about this plugin is that it composes cleanly with everything else Genkit gives you, not just one or two features:

• Genkit Developer UI. Because defineTemporalFlow returns a normal Genkit flow, you can still iterate on it locally with the Genkit Developer UI: fast feedback loop during development, durable execution in production.
• Genkit middleware. Middleware (in-call retries, fallbacks, skill injection, tool approval, prompt rewriting, etc.) runs inside the activity. You get two complementary layers of resilience: middleware for fine-grained in-call recovery, Temporal for whole-flow durability and replay.
• Tools and function calling. Tools defined with ai.defineTool are just normal flow code from Temporal’s perspective. Their calls, retries and outputs appear in both the Genkit trace and the Temporal event history.
• RAG primitives. Retrievers, indexers, embedders and rerankers all live inside the activity. That means heavy ingestion jobs (chunking, embedding, upserting into a vector store) inherit Temporal’s retry policies and survive restarts mid-batch.
• Evaluators and datasets. Genkit’s evaluators are flows like any other, so you can run eval jobs as Temporal Workflows, schedule them, fan them out across Workers, and inspect every run in the Temporal UI.
• Prompts and Dotprompt. Versioned .prompt files, prompt registries and structured outputs all work unchanged. The flow body is plain Genkit.
• Plugins and model providers. Any Genkit plugin (Google AI, Vertex AI, OpenAI, Anthropic, Ollama, local models, vector stores, etc.) plugs in as usual; the Temporal layer doesn’t care which provider is on the other side of the call.
• Telemetry and tracing. Genkit’s OpenTelemetry traces continue to be emitted from inside the activity, so they show up in whatever observability backend you already use, alongside Temporal’s own event history.
• Deployment surfaces. Flows can still be exposed as HTTP endpoints, Cloud Functions, Firebase callable functions or Express handlers; executeTemporalFlow is just one more entry point, and you can mix and match (e.g. quick chat requests over HTTP, long agent runs through Temporal).
• Multi-language story. Genkit is available in JS/TS, Go, Python (preview), Dart/Flutter (preview) and through a community Java SDK. This particular plugin targets JS/TS, but the architectural pattern — define your AI logic in Genkit, run it as a Temporal Workflow — is reusable across runtimes thanks to Temporal’s polyglot SDKs.

Conclusion

Genkit is a great way to author Gen AI logic. Temporal is a great way to run any long-running, failure-prone workload. genkitx-temporal is the missing adapter between the two: a small, focused plugin that turns every Genkit flow into a durable, retryable, observable Temporal Workflow without asking you to rewrite a single line of business logic.

If you are building anything more ambitious than a single-turn chat endpoint, give it a try:

• Source: github.com/xavidop/genkitx-temporal
• Docs: xavidop.github.io/genkitx-temporal
• Runnable example: examples/test-app

Your future on-call self will thank you the first time a model provider has a bad afternoon and your agents keep humming along regardless.

Introduction

If you have ever shipped a backend service without a debugger, a hot-reload dev server or decent logs, you remember how painful it was. Now imagine doing it in a world where your “function” is a non-deterministic black box that can rewrite its own output every time you call it. Welcome to building Gen AI applications.

The single biggest productivity multiplier I have found in the last two years of shipping AI apps is a good local debugging tool. Not a hosted dashboard. Not a cloud trace viewer with a 30-second propagation delay. A local tool that runs next to your code, picks up your edits, lets you replay a request with a tweaked prompt, and shows the full trace, tool call by tool call. We will get into the local-vs-hosted trade-off later in the article.

This article is about why that matters and how the three leading JS/TS Gen AI frameworks approach it:

• Genkit Developer UI
• Vercel AI SDK DevTools
• Mastra Studio

They are three very different products solving overlapping problems, and each one makes different trade-offs that are worth understanding before you commit to one.

Shift-left, applied to Gen AI

“Shift-left” comes from the world of testing and security: the earlier in the development lifecycle you catch a problem, the cheaper it is to fix. Bugs found in production cost orders of magnitude more than bugs found while you’re typing.

Gen AI apps make this principle existential, not just economical, because the failure modes are weirder:

• A prompt regression doesn’t show up as a stack trace. It shows up as the wrong tone, a hallucinated fact or a tool call that misfires once every twenty runs.
• A new model version can quietly change behavior across thousands of code paths with no warning.
• A tool that returns slightly different JSON can cause silent downstream breakage.

The only way to keep your sanity is to collapse the feedback loop to seconds. You want to be able to:

1. Change a prompt or a piece of orchestration code.
2. Re-run that exact unit, with that exact input.
3. See the model’s input, its output, every tool call, every retry, every token used.
4. Compare it to the previous run.
5. Decide if it’s better, worse or different.

If any one of those steps requires deploying, redeploying, opening a cloud console or grep’ing logs, you have already lost. Cycle time is the metric. A debugging tool is what makes that cycle time low enough to iterate productively.

Local vs hosted

It is worth being explicit about this trade-off, because all three tools in this article are primarily local:

• A local debugging tool runs on your machine, next to your code, with zero network latency between your edits and what you see. Iteration is fast, traces are immediate, and there is no risk of leaking prompts/responses to a third party. The downside is that it’s just for you.
• A hosted observability platform (Langfuse, LangSmith, cloud-native APM tools, etc.) is shared by your team, persists data long-term and is essential for production monitoring. The trade-off is propagation delay, configuration overhead and, in some cases, data residency concerns.

These are complementary, not competing. The argument here is that the local part of the loop is the one that’s still often missing, and that’s where shift-left lives.

What “good” looks like

After working with all three of the tools below, I would summarize the qualities of a good Gen AI debugging tool as:

• Zero-config or near-zero-config — it should pick up your code, not the other way around.
• Live reload — edit your code, save, hit “run” again. No restart.
• Interactive runners — invoke any prompt, tool, model, or higher-level primitive (agent, workflow, etc.) directly with arbitrary input.
• Full traces — every step of the generation loop, with input/output/usage at each node.
• Replay and modify — re-run any past trace with a tweak.
• Minimal code changes — the framework should be observable by default, ideally without you having to remember to wrap each model call.

Let’s see how each tool stacks up.

Genkit Developer UI

The Genkit team treats the Developer UI as a first-class part of the framework. It ships with the genkit-cli package and requires no code changes in your application to attach to your running process.

You install the CLI once globally:

npm install -g genkit-cli

And then start your app under it:

genkit start -- npx tsx --watch src/index.ts

That’s it. Genkit attaches to your running Node process, discovers every flow, prompt, model, tool, retriever, indexer, embedder and evaluator you have defined, and exposes them all in a local web app at http://localhost:4000.

Telemetry API running on http://localhost:4033
Genkit Developer UI: http://localhost:4000

What you actually get inside:

• Interactive runners for every primitive. Flows, prompts, tools, models, retrievers, indexers, embedders and evaluators all get an interactive panel where you fill out the input (validated against the Zod schema) and hit run.
• Full traces with the entire generation graph: every model call, every tool invocation, every middleware, with input/output/usage tokens.
• Live reload. Combined with --watch, edits to your code show up without restarting the UI.
• Prompt iteration. Tweak prompts and re-run inline.
• Evals. Run evaluators against datasets directly from the UI.
• Open by default. Add -o to auto-open in your browser.

What stands out here is that observability comes for free. You do not wrap your model and you do not add a middleware just to see traces. Anything defined with the Genkit primitives is automatically introspected and traced, which makes the “let me see what’s happening” cost essentially zero.

If you also use the middleware system, every middleware also shows up in the trace, which makes debugging things like retries and fallbacks a lot easier.

Vercel AI SDK DevTools

Vercel introduced the AI SDK DevTools more recently, and it is a meaningful step up from having nothing. The design philosophy is different from Genkit’s: it is a middleware-based capture tool that focuses tightly on the language-model layer, rather than an integrated development environment for the whole AI app.

You opt in by wrapping your model:

import { wrapLanguageModel, gateway } from 'ai';
import { devToolsMiddleware } from '@ai-sdk/devtools';

const model = wrapLanguageModel({
  model: gateway('anthropic/claude-sonnet-4.5'),
  middleware: devToolsMiddleware(),
});

And then run the viewer in another terminal:

npx @ai-sdk/devtools
# open http://localhost:4983

What it captures:

• Input parameters and prompts.
• Output content and tool calls.
• Token usage and timing.
• Raw provider request/response payloads.
• Multi-step interactions are grouped into “runs” with multiple “steps”.

Things to keep in mind when comparing it with the others:

• It is opt-in per model. Every model you want to observe has to be wrapped explicitly. In a real codebase with many models, this usually means a shared factory or repeated wrapping. If you forget on one path, you have a blind spot.
• It focuses on the language-model layer. Anything happening above that layer (your own application logic, custom orchestration, business steps) is not captured unless you add it yourself.
• It is a viewer, not an interactive playground. You cannot “re-run this with a different prompt” from the UI; you go back to your code, edit and re-call.
• It does not enumerate AI primitives because, in the AI SDK, those are just functions in your codebase. There is nothing to enumerate at runtime.
• Storage is a JSON file (.devtools/generations.json). The team is clear that this is local-only, never for production, and the middleware automatically appends .devtools to your .gitignore.

The upside is that it is probably the quickest of the three to drop into an existing project: install the package, wrap a model, run the viewer. If you live inside Next.js and the AI SDK and you mainly want a “what just happened?” panel for your model calls, it does the job well. If you are debugging a multi-step agent with custom orchestration, you will probably want to combine it with something that sees more of the picture.

Mastra Studio

Mastra Studio is the most ambitious of the three in terms of surface area. It is bundled with mastra dev and runs at http://localhost:4111, doubling as a local development tool and a deployable team console (you can ship Studio to production for non-developers).

Out of the box, it gives you:

• Agents tab — chat with your agents, hot-swap models, tweak temperature/top-p, view traces, attach scorers.
• Workflows tab — visualize workflows as graphs, run them step by step with custom inputs, watch the active step in real time, inspect tool calls and JSON outputs.
• Processors tab — see input/output processors and guardrails wired to each agent.
• Tools tab — run tools in isolation to debug them.
• Workspaces tab — file browser into the agent’s workspace filesystem, with a Skills tab listing discovered skills.
• MCP servers tab — list attached MCP servers and their tools.
• Request context — set runtime variables that flow into agent instructions/tools through DI, with schema-driven forms.
• Evaluation suite — Scorers, Datasets and Experiments tabs to run datasets through agents/workflows, attach scorers and compare experiments side-by-side.
• Observability with traces and logs.
• Editor integration for non-technical teammates to iterate on agents and version every change without redeploying.

This is more than a debugging tool — it is closer to a development environment plus a team-facing console. The natural trade-off is that Studio is tightly coupled to Mastra’s primitives (Agents, Workflows, Processors, Workspaces), so it shines the most when your application is structured the Mastra way.

If you build non-trivial agentic systems and you want evaluation and dataset management baked into the same tool, Studio is genuinely very good. For a quick “see my prompt and the model’s response” loop, it can feel like more than you need.

Side-by-side comparison

A quick note on the Vercel column: simplicity of integration is a real feature for many teams, and it is fair to call this the lightest-touch option. The trade-off is that observability is opt-in per model, so you have to be deliberate to avoid blind spots.

When I would pick which

After using all three, this is roughly how I’d think about it:

• Building a serious agent or multi-step pipeline with tools, retries, fallbacks and evals → Genkit Developer UI is hard to beat on pure iteration speed. Zero-config tracing of the full pipeline, every primitive runnable, and middleware shows up in the trace for free.
• Working primarily inside Next.js with the Vercel AI SDK and you mainly want better visibility into model calls → Vercel AI SDK DevTools is a quick win. Lightweight setup and it does what it says on the tin; pair it with something else if you need higher-level orchestration views.
• Building agentic systems with workflows, datasets and scorers, and you want a UI your PM or domain expert can also use → Mastra Studio. The evaluation suite and the deployability are real differentiators.
• Multi-language stack (JS/TS plus Go, Python or Dart) → Genkit is the only one of the three with first-party multi-language support; the local tooling story is consistent across runtimes.

The moral of the story: pick a debugging tool, any debugging tool

In a normal backend project, you can sometimes get away with a weak dev loop because the code is deterministic. Bad UX, but it works.

In an AI project, the code is not deterministic. Every iteration is also a small experiment. If your loop is slow or your visibility is poor, you don’t just iterate slower, you iterate worse, because you can’t tell whether your changes are improvements or regressions. Shift-left in this world is not a virtue, it is a survival strategy.

A good local debugging tool:

• Turns “I think the prompt got better” into “I can see the trace, the tokens, the eval score, side by side”.
• Catches regressions before they reach a user.
• Lets you onboard new engineers in hours instead of days, because they can see the system.
• Lets non-engineers (PMs, designers, domain experts) participate in iteration when the UI is friendly enough — Mastra Studio is explicit about this.

If you are starting a Gen AI project today and you are picking a framework, the local tooling story should weigh as much as the model abstraction or the tool API. It is the difference between writing AI code and engineering AI features.

Conclusion

All three tools are good in their lane:

• Genkit Developer UI is the most introspective of the three and requires zero code changes to be useful. It’s the one I reach for when I want to move fast and see every step of the pipeline.
• Vercel AI SDK DevTools is the lightest-touch option to drop into an existing AI SDK app, with a clear focus on the model-call layer. Think of it as a focused viewer rather than a full development environment.
• Mastra Studio is the most ambitious, closer to a full IDE-meets-console for agentic systems, with first-class evaluation, datasets and a deployable team UI. It’s at its best when your app is structured around Mastra’s primitives.

Pick the one that matches the shape of your project, but please pick something. Building Gen AI without a local debugging tool in 2026 is engineering with the lights off.

Further reading:

• Genkit Developer Tools
• Vercel AI SDK DevTools
• Mastra Studio overview
• Top JS/TS Gen AI Frameworks for 2026
• Genkit Middleware deep dive

Introduction

If you have been building anything non-trivial with Genkit, you have probably bumped into the same set of cross-cutting concerns over and over again: retrying transient model errors, falling back to a cheaper model when quota explodes, gating tool execution behind human approval, injecting filesystem access for coding agents, logging every request and response for observability…

Until now, you ended up either wrapping ai.generate() calls by hand or writing ad-hoc helpers that ended up duplicated across flows. The new Genkit Middleware changes that. It introduces a first-class, composable middleware layer for the generate() pipeline, with hooks for the model, the tool execution and the high-level generation loop, plus a small but very useful set of official middlewares published in the brand new @genkit-ai/middleware package.

This article is a practical tour of what the new middleware system gives you, the built-in middlewares you can drop in today, and how to write your own with generateMiddleware.

The official documentation lives at Genkit Middleware. All examples below assume the JavaScript/TypeScript SDK.

A quick reminder: although this article focuses on the JS/TS middleware API, Genkit is a multi-language framework. The official SDKs cover JavaScript/TypeScript (primary, stable), Go, Python (preview) and Dart/Flutter (preview), and there is a community-maintained Java SDK used in production. The middleware concepts described here are JS/TS-specific today, but the underlying generate() pipeline exists across all SDKs and the same patterns are landing on the other runtimes.

What is middleware in Genkit

Conceptually, Genkit middleware behaves like the middleware you already know from Express or Koa, only applied to the LLM lifecycle instead of HTTP requests:

1. A generate() call is intercepted before it reaches the model.
2. Each middleware can inspect or modify the request, decide whether to call next(), and inspect or modify the response on the way back.
3. Multiple middlewares can be chained. They run in the order they are declared and unwind in reverse order, exactly like an onion.

What makes Genkit’s design interesting is that it does not give you a single chokepoint, it gives you three orthogonal interception phases:

• model — wraps the call to the underlying model. Perfect for retries, fallbacks, request/response logging or response transformations.
• tool — wraps tool execution. Ideal for approvals, sandboxing, audit logs or input/output validation.
• generate — wraps the whole high-level generation loop (prompting, tool calling, output parsing). Best for things like injecting tools or system instructions before the loop starts.

You opt in per call via a use: array, which keeps things explicit and avoids global side effects:

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Hello',
  use: [retry({ maxRetries: 3 }), loggerMiddleware({ verbose: true })],
});

Installation

The official middlewares ship in their own package, decoupled from the Genkit core:

npm install @genkit-ai/middleware
# or
pnpm add @genkit-ai/middleware

You still need genkit itself and a model provider plugin (for example @genkit-ai/google-genai).

The built-in middleware catalogue

Let’s go through the five middlewares the Genkit team ships out of the box.

filesystem — give the model a sandboxed file system

filesystem injects a standard set of file manipulation tools (list_files, read_file, write_file, search_and_replace) into the generation loop, restricted to a root directory of your choice.

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';
import { filesystem } from '@genkit-ai/middleware';

const ai = genkit({ plugins: [googleAI()] });

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Create a hello world Node app in the workspace',
  use: [
    filesystem({
      rootDirectory: './workspace',
      allowWriteAccess: true,
    }),
  ],
});

Useful options:

• rootDirectory (required) — sandbox root, all paths are confined to it.
• allowWriteAccess — defaults to false. Read-only by default is a sane choice for safety.
• toolNamePrefix — namespace the injected tools to avoid collisions with your own.

This is essentially the building block for a “coding agent” pattern, without you having to write tool definitions or path validation logic.

skills — auto-load Markdown skills as system context

skills scans a directory for SKILL.md files (plus their YAML frontmatter), injects relevant ones into the system prompt, and exposes a use_skill tool the model can call when it needs more specific guidance.

import { skills } from '@genkit-ai/middleware';

const response = await ai.generate({
  prompt: 'How do I run tests in this repo?',
  use: [skills({ skillPaths: ['./skills'] })],
});

Think of it as a lightweight, file-based knowledge layer: every skill is a self-contained Markdown file with metadata, and the middleware decides when to surface them. It is a really clean alternative to ad-hoc system prompt soup.

toolApproval — human-in-the-loop for tool calls

toolApproval enforces an allowlist of tools the model is allowed to execute autonomously. Anything outside the list raises a ToolInterruptError, so you can pause execution, ask the user, and resume.

import { genkit, restartTool } from 'genkit';
import { toolApproval } from '@genkit-ai/middleware';

const response = await ai.generate({
  prompt: 'write a file',
  tools: [writeFileTool],
  use: [toolApproval({ approved: [] })], // empty list -> always interrupt
});

if (response.finishReason === 'interrupted') {
  const interrupt = response.interrupts[0];

  // ... ask the user, then mark the tool call as approved
  const approvedPart = restartTool(interrupt, { toolApproved: true });

  const resumedResponse = await ai.generate({
    messages: response.messages,
    resume: { restart: [approvedPart] },
    use: [toolApproval({ approved: [] })],
  });
}

This is exactly the pattern you want for any agent that touches the real world (filesystem writes, payments, sending emails). No more home-grown approval flags scattered across the codebase.

retry — exponential backoff with jitter for transient errors

The retry middleware retries failed model calls on transient status codes (UNAVAILABLE, DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, ABORTED, INTERNAL) using exponential backoff with jitter.

import { retry } from '@genkit-ai/middleware';

const response = await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: 'Heavy reasoning task...',
  use: [
    retry({
      maxRetries: 3,
      initialDelayMs: 1000,
      backoffFactor: 2,
    }),
  ],
});

Knobs you actually care about:

• maxRetries (default 3)
• statuses — which status codes to retry on
• initialDelayMs / maxDelayMs / backoffFactor
• noJitter — if you really want deterministic delays

This is one of those things every team writes once, badly. Having it in the framework is a very welcome change.

fallback — gracefully degrade to a different model

fallback switches to an alternate model when the primary one fails on configurable status codes. The classic use case is “try Pro first, fall back to Flash when quota is exhausted”:

import { fallback } from '@genkit-ai/middleware';

const response = await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: 'Try the pro model first...',
  use: [
    fallback({
      models: [googleAI.model('gemini-flash-latest')],
      statuses: ['RESOURCE_EXHAUSTED'],
    }),
  ],
});

You can chain multiple fallback models, and isolateConfig lets you decide whether the fallback inherits the original request configuration or starts clean (handy when the fallback model does not support the same options as the primary).

Building your own middleware with generateMiddleware

The same primitive that powers all the built-ins is exposed for you. The generateMiddleware helper gives you typed config schemas (via Zod) and access to the ai instance.

Here is the canonical “logger” example, straight from the docs but lightly annotated:

import { generateMiddleware, z } from 'genkit';

export const loggerMiddleware = generateMiddleware(
  {
    name: 'loggerMiddleware',
    description: 'Logs requests and responses',
    configSchema: z.object({
      verbose: z.boolean().optional(),
    }),
  },
  ({ config, ai }) => {
    return {
      // Phase 1: intercept the model call
      model: async (req, ctx, next) => {
        if (config?.verbose) {
          console.log('Request:', JSON.stringify(req));
        }
        const resp = await next(req, ctx);
        if (config?.verbose) {
          console.log('Response:', JSON.stringify(resp));
        }
        return resp;
      },
      // You could also add `tool: ...` and `generate: ...` hooks here.
    };
  }
);

Using it is identical to the official ones:

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Hello',
  use: [loggerMiddleware({ verbose: true })],
});

A few patterns I have found very useful:

• PII redaction — implement a model hook that scrubs the request prompt and the response text against a regex/dictionary, returning the cleaned version.
• Cost accounting — wrap the model hook to read usage tokens from the response, and emit them to your metrics backend tagged by user/feature.
• Per-tenant quotas — use the generate hook to check a counter (Redis, Firestore…) before calling next(); throw your own custom error if the tenant is over quota.
• Caching — keyed on a hash of the model + request, return a cached response if hit, otherwise call next() and persist the result.

For more inspiration, the source of the official middlewares is open in the Genkit GitHub repository, and reading them is genuinely educational.

Composition: stacking middlewares

Middlewares compose in array order. A reasonable production stack might look like this:

const response = await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: userPrompt,
  tools: myTools,
  use: [
    loggerMiddleware({ verbose: false }),       // outermost: see everything
    retry({ maxRetries: 3 }),                   // recover from transient failures
    fallback({                                  // degrade if Pro is overloaded
      models: [googleAI.model('gemini-flash-latest')],
      statuses: ['RESOURCE_EXHAUSTED'],
    }),
    toolApproval({ approved: ['searchDocs'] }), // gate dangerous tools
  ],
});

The order matters: outer middlewares see the result of the inner ones. Put logging on the outside if you want it to record the final state after retries and fallbacks; put it on the inside if you want to see every individual model attempt.

The importance of middleware for production agents

Genkit Middleware is one of those features that does not look flashy in a changelog but quietly fixes a lot of real-world friction. It pushes Genkit closer to a “batteries-included” framework for production agents:

• Cross-cutting concerns are no longer copy-pasted across flows.
• Safety-critical behavior (approvals, sandboxes, fallbacks) is declarative.
• The model / tool / generate split gives you precise control without forcing you to monkey-patch.
• The middleware contract is small enough that the community can ship plugins that interoperate.

If you maintain any non-trivial Genkit application, the upgrade is a no-brainer. Drop in retry and fallback first, you will probably see incidents disappear within the week. Then start writing your own middlewares for the things that are unique to your domain.

Conclusion

Middleware turns Genkit’s generate() from “a function you call” into “a pipeline you compose”. The official @genkit-ai/middleware package covers the most common production needs (filesystem access, skills, tool approval, retries, fallbacks), and generateMiddleware makes writing your own a 20-line affair instead of a refactor.

For the next steps, take a look at:

• Genkit Middleware documentation
• Genkit middleware source on GitHub
• Genkit flows — middleware composes especially well with typed flows
• Tool calling and Interrupts — the foundation that toolApproval builds on

Happy hacking, and may your fallback models always be cheaper than your primary one.

Introduction

Two of the most popular Gen AI frameworks in JavaScript/TypeScript, Vercel AI SDK and Genkit, both ship a middleware system to extend their model calls with cross-cutting behavior: logging, caching, RAG, retries, fallbacks, guardrails, tool approval, etc.

On the surface they look very similar. In practice, they sit at different abstraction levels and embody different philosophies. This article puts them side by side using their official docs as the source of truth:

• Vercel AI SDK — Language Model Middleware
• Genkit — Middleware

I’ll walk through the API surface, the built-ins, how composition works, how you write your own, and finish with a concrete decision matrix.

One scope note before we dive in: Vercel AI SDK is JavaScript/TypeScript only, while Genkit is a multi-language framework with official SDKs for JavaScript/TypeScript (primary, stable), Go, Python (preview) and Dart/Flutter (preview), plus a community-maintained Java SDK. The middleware comparison below is JS/TS-to-JS/TS, but if you also need to share patterns with Go, Python, Dart or Java services, that is a Genkit-only conversation.

TL;DR

Both are good. They are designed for different mental models and you can absolutely use both in the same codebase if you mix the two SDKs.

The mental model

Vercel AI SDK: wrap the model

In Vercel’s world, middleware is a decorator over the language model itself. You take a model, wrap it, and the result is “still a model”. It plugs into generateText, streamText, generateObject, etc. without those functions even knowing middleware exists.

import { wrapLanguageModel, streamText } from 'ai';

const wrappedLanguageModel = wrapLanguageModel({
  model: yourModel,
  middleware: yourLanguageModelMiddleware,
});

const result = streamText({
  model: wrappedLanguageModel,
  prompt: 'What cities are in the United States?',
});

This is very clean: the middleware travels with the model and is transparent to the rest of your code.

Genkit: opt in per call

Genkit takes the opposite stance. Instead of wrapping the model, you pass middlewares as part of each generate() call via the use: array, and you can intercept three different phases of the pipeline: the model, the tools, and the high-level generate loop.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Hello',
  use: [retry({ maxRetries: 3 }), loggerMiddleware({ verbose: true })],
});

Trade-off: it is more explicit (you see exactly what runs at each call site), at the cost of being noisier when you want a global behavior. Both styles are easy to wrap in a small helper.

The hooks side by side

Vercel AI SDK

Three hooks, all centered on the language model contract:

• transformParams — mutate the request before it hits doGenerate or doStream.
• wrapGenerate — wrap the non-streaming call, observe and modify the result.
• wrapStream — wrap the streaming call. You typically pipe the stream through a TransformStream to inspect or rewrite chunks.

Example: the canonical logging middleware, including streaming:

import type {
  LanguageModelV3Middleware,
  LanguageModelV3StreamPart,
} from '@ai-sdk/provider';

export const yourLogMiddleware: LanguageModelV3Middleware = {
  wrapGenerate: async ({ doGenerate, params }) => {
    console.log('doGenerate called', params);
    const result = await doGenerate();
    console.log('generated text:', result.text);
    return result;
  },

  wrapStream: async ({ doStream, params }) => {
    const { stream, ...rest } = await doStream();
    let generatedText = '';

    const transformStream = new TransformStream<
      LanguageModelV3StreamPart,
      LanguageModelV3StreamPart
    >({
      transform(chunk, controller) {
        if (chunk.type === 'text-delta') generatedText += chunk.delta;
        controller.enqueue(chunk);
      },
      flush() {
        console.log('stream finished:', generatedText);
      },
    });

    return { stream: stream.pipeThrough(transformStream), ...rest };
  },
};

Genkit

Three hooks too, but cutting along execution phase instead of stream vs non-stream:

• model — wraps the model invocation. Streaming and non-streaming go through the same hook.
• tool — wraps tool execution. This has no equivalent in Vercel’s spec (tools are not part of the language-model middleware contract there).
• generate — wraps the entire high-level generation loop, including tool calling and output parsing.

import { generateMiddleware, z } from 'genkit';

export const loggerMiddleware = generateMiddleware(
  {
    name: 'loggerMiddleware',
    description: 'Logs requests and responses',
    configSchema: z.object({ verbose: z.boolean().optional() }),
  },
  ({ config }) => ({
    model: async (req, ctx, next) => {
      if (config?.verbose) console.log('Request:', JSON.stringify(req));
      const resp = await next(req, ctx);
      if (config?.verbose) console.log('Response:', JSON.stringify(resp));
      return resp;
    },
  })
);

Two important differences:

1. Tools are first-class citizens in Genkit middleware. You can intercept tool execution itself, which is exactly what powers toolApproval. In Vercel AI SDK, tool gating typically lives in your application code or inside experimental_prepareStep for agents, not in the middleware spec.
2. Genkit middleware factories carry a Zod config schema, so misconfigured middleware fails fast with a clear error. Vercel middleware is a plain object that conforms to LanguageModelV3Middleware; configuration is your responsibility.

Built-ins, head to head

Vercel AI SDK

Centered on adapting the model contract to real-world quirks:

• extractReasoningMiddleware — pulls <think>...</think> blocks (and similar) out of the text and surfaces them as a reasoning property. Crucial for DeepSeek R1 and friends.
• extractJsonMiddleware — strips Markdown code fences from JSON outputs so Output.object() keeps working with chatty models.
• simulateStreamingMiddleware — fakes a streaming interface on top of a non-streaming model, so your UI code stays consistent.
• defaultSettingsMiddleware — pins default settings (temperature, max output tokens, provider options).
• addToolInputExamplesMiddleware — for providers that don’t support inputExamples natively, serializes them into the tool description.

The theme is clear: smooth over differences between providers. Vercel runs an SDK that has to talk to dozens of model providers, so its middleware library reflects that.

Genkit

Centered on production patterns and agentic behavior:

• filesystem — sandboxed file tools (list_files, read_file, write_file, search_and_replace) restricted to a root directory.
• skills — auto-injects SKILL.md files into the system prompt and exposes a use_skill tool.
• toolApproval — human-in-the-loop gating for tool calls, with first-class ToolInterruptError and resume support.
• retry — exponential backoff with jitter for transient errors.
• fallback — automatic switch to a different model on failure.

Theme: production hardening and agent ergonomics. Genkit assumes you’ll plug in your own providers but want batteries for retry/fallback/approval/skills.

Notice the very small overlap. If you wanted “extract reasoning” in Genkit you’d write it as a custom middleware (10-20 lines). If you wanted “retry with backoff” in Vercel AI SDK, same thing. Each ecosystem chose to ship the middlewares its users were asking for the most.

Composition

Vercel AI SDK

You pass an array to wrapLanguageModel. The order is “outermost first”: the first middleware in the array runs outside the second one.

const wrappedLanguageModel = wrapLanguageModel({
  model: yourModel,
  middleware: [firstMiddleware, secondMiddleware],
});
// applied as: firstMiddleware(secondMiddleware(yourModel))

Composition is static. The wrapped model is a long-lived value. Great for “configure once at startup” scenarios.

Genkit

You pass an array to use: on every call. Order is the same onion model: outer middlewares wrap inner ones.

const response = await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: userPrompt,
  use: [
    loggerMiddleware({ verbose: false }),
    retry({ maxRetries: 3 }),
    fallback({
      models: [googleAI.model('gemini-flash-latest')],
      statuses: ['RESOURCE_EXHAUSTED'],
    }),
    toolApproval({ approved: ['searchDocs'] }),
  ],
});

Composition is dynamic. You can change the stack per request based on user, tenant, A/B test, environment, etc. without rebuilding model instances.

Per-request metadata

A common need is to pass per-request context (user id, tenant, trace id…) into a middleware.

Vercel AI SDK uses providerOptions, namespaced by the middleware name:

export const yourLogMiddleware: LanguageModelV3Middleware = {
  wrapGenerate: async ({ doGenerate, params }) => {
    console.log('METADATA', params?.providerMetadata?.yourLogMiddleware);
    return doGenerate();
  },
};

await generateText({
  model: wrapLanguageModel({ model: 'anthropic/claude-sonnet-4.5', middleware: yourLogMiddleware }),
  prompt: 'Invent a new holiday and describe its traditions.',
  providerOptions: {
    yourLogMiddleware: { hello: 'world' },
  },
});

Genkit is more direct: middleware is invoked as a factory, so you pass config when you instantiate it for that call:

use: [loggerMiddleware({ verbose: true, requestId: 'abc-123' })],

Both work, but the Genkit approach is type-checked end-to-end thanks to the Zod config schema, while Vercel’s providerOptions is more loosely typed.

Streaming

Vercel AI SDK has separate hooks for streaming and non-streaming, which is honest because the two have very different semantics. You almost always need to pipe the stream through a TransformStream.

Genkit folds streaming into the same model hook. The middleware sees the request and the response, and the underlying engine handles whether it was streamed. This is more ergonomic when your middleware doesn’t need per-chunk logic, but if you do need to inspect chunks (say, for guardrails on partial output), you’ll need to drop down to lower-level APIs.

Tools and agents

This is where the two systems diverge most.

In Vercel AI SDK, tools live above the middleware layer. The LanguageModelV3Middleware spec sits at the model level; agentic loops are handled by higher-level abstractions like generateText, streamText and the agent APIs (experimental_prepareStep, etc.).

In Genkit, the tool and generate hooks make tool execution and the agent loop first-class targets for middleware. This is what enables clean implementations of:

• Tool approval (toolApproval)
• Tool sandboxing (filesystem)
• Agent-level skill injection (skills)

If you build a lot of agents with tool calling, Genkit’s middleware surface is genuinely more expressive.

Observability and tracing

Both frameworks are observable but in different shapes:

• Genkit has a built-in Developer UI that shows traces of every generate() call, including middleware, tools, and nested flows. Middleware automatically participates in traces.
• Vercel AI SDK is OpenTelemetry-friendly and integrates naturally with Vercel’s @vercel/otel, AI Gateway and Vercel Observability. You can wire your middleware into spans yourself.

If “open the Dev UI and see what every middleware did” is high on your list, Genkit wins. If you already live inside the Vercel platform, the AI SDK plays beautifully with the rest of the stack.

A concrete decision guide

I would reach for Vercel AI SDK middleware when:

• I want a very small, well-defined surface area: tweak params, wrap generate, wrap stream.
• I am bouncing between many model providers and need normalization (reasoning extraction, JSON cleanup, default settings).
• I prefer “configure the model once at startup” over “compose per call”.
• I am already deep in the Vercel ecosystem (Next.js, AI Gateway, Vercel Observability).
• Streaming-aware behavior with per-chunk transforms is a hard requirement.

I would reach for Genkit middleware when:

• I am building agents with tool calling, and I need tool approval, sandboxing or skill injection.
• I want production-grade retry and fallback behavior out of the box.
• I want to compose middleware per request based on user, tenant or experiment.
• I value the Developer UI and rich tracing over my entire pipeline.
• I want typed, validated middleware configuration via Zod.

And honestly, in many real systems you might use both: Vercel AI SDK on the front-end / edge for chat UX, and Genkit on the backend for orchestration, tool calling and agentic flows. The middleware contracts are scoped, you won’t get the kind of conflicts you would expect.

Conclusion

Vercel AI SDK middleware and Genkit middleware are two well-thought-out answers to the same question, asked from different vantage points.

• Vercel AI SDK treats middleware as a provider abstraction layer: you adapt and normalize what providers give you.
• Genkit treats middleware as a production-pipeline composition layer: you orchestrate models, tools and agentic loops with reusable building blocks.

Pick the one that matches the shape of your problem, and don’t be afraid to mix them. The best part of 2026 in JS/TS Gen AI is that both are mature, both are open source, and both let you write your own middleware in a handful of lines.

Further reading:

• Vercel AI SDK — Language Model Middleware
• Genkit Middleware documentation
• Top JS/TS Gen AI Frameworks for 2026
• Genkit GitHub repository
• Vercel AI SDK GitHub repository

Introduccion

Si vienes del ecosistema Node.js y quieres construir funcionalidades de IA generativa sin pelearte con demasiada infraestructura, Genkit es una de las mejores opciones ahora mismo.

En esta guia vas a crear tu primer proyecto con JavaScript/TypeScript, definir un flow tipado, ejecutarlo en local y probarlo en la Developer UI.

Esta introduccion esta basada en la guia oficial de Genkit para JS/TS: Get started with Genkit

Que es Genkit y por que usarlo

Genkit es un framework open source para construir aplicaciones de IA con una capa de desarrollo muy práctica para backend:

1. Define inputs y outputs con esquemas (Zod).
2. Crea flows reutilizables y tipados.
3. Prueba todo en local con trazas en la Developer UI.
4. Te deja cambiar de proveedor/modelo con menos fricción.

Prerrequisitos

Antes de empezar, necesitas:

1. Node.js v20 o superior.
2. npm.
3. Una API key de Gemini desde Google AI Studio.

1) Crear el proyecto TypeScript

Desde terminal:

mkdir my-genkit-app
cd my-genkit-app

npm init -y
npm pkg set type=module

npm install -D typescript tsx
npx tsc --init

mkdir src
touch src/index.ts

Con esto ya tienes la base de un proyecto TS en Node.

2) Instalar Genkit

Instala primero la CLI (necesaria para la Developer UI):

npm install -g genkit-cli

Instala despues las dependencias del proyecto:

npm install genkit @genkit-ai/google-genai

3) Configurar la API key

Exporta la variable de entorno en tu shell:

export GEMINI_API_KEY=<tu_api_key>

Si usas zsh, tambien puedes dejarla en tu .zshrc para no repetir este paso cada vez.

4) Crear tu primer flow con Genkit

Edita src/index.ts y pega este ejemplo:

import { googleAI } from '@genkit-ai/google-genai';
import { genkit, z } from 'genkit';

// Inicializa Genkit con el plugin de Google AI y modelo por defecto.
const ai = genkit({
  plugins: [googleAI()],
  model: googleAI.model('gemini-3-pro', {
    temperature: 0.8,
  }),
});

const RecipeInputSchema = z.object({
  ingredient: z.string().describe('Ingrediente principal o tipo de cocina'),
  dietaryRestrictions: z
    .string()
    .optional()
    .describe('Restricciones alimentarias, si existen'),
});

const RecipeSchema = z.object({
  title: z.string(),
  description: z.string(),
  prepTime: z.string(),
  cookTime: z.string(),
  servings: z.number(),
  ingredients: z.array(z.string()),
  instructions: z.array(z.string()),
  tips: z.array(z.string()).optional(),
});

export const recipeGeneratorFlow = ai.defineFlow(
  {
    name: 'recipeGeneratorFlow',
    inputSchema: RecipeInputSchema,
    outputSchema: RecipeSchema,
  },
  async (input) => {
    const prompt = `Create a recipe with the following requirements:\nMain ingredient: ${input.ingredient}\nDietary restrictions: ${input.dietaryRestrictions || 'none'}`;

    const { output } = await ai.generate({
      prompt,
      output: { schema: RecipeSchema },
    });

    if (!output) {
      throw new Error('Failed to generate recipe');
    }

    return output;
  }
);

async function main() {
  const recipe = await recipeGeneratorFlow({
    ingredient: 'aguacate',
    dietaryRestrictions: 'vegetariana',
  });

  console.log(recipe);
}

main().catch(console.error);

Este ejemplo muestra tres conceptos clave de Genkit:

1. Esquemas de input/output con Zod para evitar respuestas ambiguas.
2. Flow reutilizable que despues puedes exponer como API.
3. Structured Output desde el modelo (no solo texto libre).

5) Ejecutar la aplicacion

ejecuta el ejemplo:

npx tsx src/index.ts

Si todo va bien, veras por consola un objeto JSON con la receta generada.

6) Probar en la Developer UI

Arranca la UI de desarrollo de Genkit:

genkit start -- npx tsx --watch src/index.ts

Por defecto estara en:

• http://localhost:4000

Dentro de la UI:

1. Selecciona recipeGeneratorFlow.
2. Introduce un input como este:

{
  "ingredient": "aguacate",
  "dietaryRestrictions": "vegetariana"
}

1. Pulsa Run.

Veras el structured output y la traza de ejecucion para debuggear prompts y tiempos.

Script opcional en package.json

Para no recordar el comando largo, puedes añadir este script:

{
  "scripts": {
    "genkit:ui": "genkit start -- npx tsx --watch src/index.ts"
  }
}

Y ejecutarlo con:

npm run genkit:ui

Errores tipicos al empezar

1. GEMINI_API_KEY no definida.
2. Usar Node < 20.
3. Intentar correr la UI sin genkit-cli instalado globalmente.

Siguientes pasos recomendados

Cuando ya tengas este primer flow funcionando, te recomiendo seguir por este orden:

1. Developer tools
2. Generating content
3. Creating flows
4. Tool calling
5. Dotprompt

Conclusiones

Genkit en JS/TS te permite pasar de idea a prototipo funcional en muy poco tiempo, manteniendo un codigo limpio, tipado y facil de evolucionar.

Si trabajas en Node.js y quieres construir features con IA de forma seria (pero sin sobreingenieria), este stack es una apuesta muy solida.

Introduction

For the last few years, every Gen AI tutorial, framework, and “hello world” has assumed one thing: that you are writing Python. It made sense at the start. The research community lives in Python, the model providers ship Python SDKs first, and the notebook culture is hard to beat for prototyping. But there is a quiet, important shift happening in 2026: the teams actually shipping AI features at scale are increasingly moving their production Gen AI workloads off Python, and onto languages built for services.

Go is at the center of that shift. And Genkit Go, the Go flavor of Google’s open-source Gen AI framework, is the cleanest path I have seen to build production-ready AI services in Go: typed flows, structured output, built-in HTTP serving, observability, and a Developer UI, all from a single binary.

This article is two things at once. First, an honest argument about why Python is a poor fit for production Gen AI services. Second, a hands-on getting-started with Genkit Go so you can replace that Python microservice this week.

Why Python Hurts in Production Gen AI

Python is great for research and prototyping. But Gen AI applications are not really “AI code”, they are mostly I/O-heavy network services that happen to call a model. And that is exactly where Python struggles.

Concurrency is a constant fight

Gen AI workloads are dominated by long, concurrent network calls: streaming completions, tool calls, embedding requests, vector DB lookups, MCP servers. Go’s goroutines and channels were literally designed for this. In Python you have a choice between three uncomfortable options: threads (limited by the GIL), asyncio (which infects your entire codebase and breaks the moment one library is sync), or multiprocessing (heavy, awkward, and unfriendly to shared state). None of them feel native. All of them leak through your abstractions.

Cold starts and memory footprint

A Python AI service typically pulls in pydantic, httpx, an SDK or two, and a tokenizer. You are easily looking at 200, 400 MB of resident memory and several seconds of cold start before you serve a single request. A Go binary doing the same job is one statically linked file, tens of MB of RAM, and starts in milliseconds. On Cloud Run, Lambda, Azure Functions, or any autoscaling platform, this difference is not a micro-optimization, it is the difference between a service that scales to zero gracefully and one that does not.

Dependency hell is worse for AI

pip, poetry, uv, conda, venv, requirements.txt, pyproject.toml. Pin a Torch version, break a transitive dep. Upgrade an SDK, break Pydantic v1 vs v2. Every Python AI repo I have inherited has spent at least a day fixing the environment before running a single prompt. Go’s module system, with a single go.mod and go.sum, is boring, reproducible, and just works.

Types are optional, and it shows

Structured output, tool calling, and MCP all rely on schemas. In Python, the schema lives in Pydantic models, in docstrings, in comments, and sometimes in your head. In Go, the schema is the struct. The compiler enforces it. Genkit picks it up automatically via JSON schema tags. You cannot ship a flow whose input does not match what the model returns, because it will not compile.

Deployment is a packaging exercise

Python deployments are Dockerfiles full of system packages, base images that drift, and “works on my machine” surprises. Go deploys as a single static binary. FROM scratch, copy the binary, done. For AI services that need to run on Cloud Run, on Kubernetes, on the edge, or as a sidecar, that is a massive operational win.

The performance ceiling is real

Yes, the heavy lifting happens on the model provider’s GPUs. But your service still has to parse tokens off a streaming response, fan out tool calls, merge results, enforce timeouts, and push telemetry, per request, at concurrency. Go does that work an order of magnitude more efficiently than CPython, and without you having to think about it.

None of this means Python is wrong for research. It means Python is the wrong default for the service that exposes that research to your users.

Go Is the Best Language for Agentic Coders

There is one more reason to pick Go in 2026 that did not really exist two years ago: agentic coders. Tools like Claude Code, Cursor’s agent mode, GitHub Copilot’s agent, Gemini Code Assist, Codex, Aider, and the growing ecosystem of autonomous coding agents are now a real part of how software gets written. And it turns out that Go is the language they thrive in.

Why? It comes down to three properties of the language that align almost perfectly with how an LLM-based agent reasons about code:

Strong, static typing closes the feedback loop

Agentic coders work in a tight loop: write code, compile, read the error, fix, repeat. Go’s compiler is fast, strict, and brutally honest. When an agent generates a wrong call, the compiler tells it exactly what is wrong and where, in seconds. In Python, the same mistake might only surface at runtime, three layers deep, with a stack trace that requires the agent to spend tokens reasoning about dynamic behavior. Strong typing turns “guess and pray” into “verify and continue”.

There is usually one obvious way to do something

Python has at least four HTTP clients, three async paradigms, two type systems, and an opinion war about every major design decision. An agent has to choose, and choices cost tokens and increase the chance of going off the rails. Go is famously opinionated: one formatter (gofmt), one module system, one idiomatic way to handle errors, one standard layout. Less surface area means less ambiguity, which means less token consumption and more correct code per iteration.

Tooling is built for machines, not just humans

go build, go test, go vet, gopls, and staticcheck produce structured, parseable output. Agents can read it directly without heuristics. Combine that with go doc and the standard library being uniformly documented, and you give an agent a self-describing environment it can navigate without hallucinating.

And then Genkit Go takes it one level further

Genkit Go leans into the same properties:

• Flow inputs and outputs are Go structs, the schema is the type. An agent generating a new flow knows exactly what shape the data has, because the compiler will reject anything else.
• The API surface is small and consistent: genkit.Init, genkit.DefineFlow, genkit.DefineTool, genkit.GenerateData, genkit.Handler. There is one obvious way to define a flow, one obvious way to expose it, one obvious way to call a model.
• Tool definitions are typed end-to-end, so an agent writing a new tool gets compile-time guarantees that its signature matches what the runtime expects.

The net effect is that an agentic coder pointed at a Genkit Go codebase will produce more correct code, in fewer iterations, with fewer tokens than the same agent pointed at an equivalent Python codebase. In a world where you are increasingly going to be the reviewer of agent-generated code rather than the author, that compounds fast.

Why Genkit Go Specifically

If you accept the premise that Go is the better runtime for Gen AI services, the next question is: which framework? You can absolutely call the Gemini, OpenAI, or Anthropic SDKs directly from Go. But you will quickly end up rebuilding the same primitives every Genkit user already has for free.

Here is what Genkit Go gives you out of the box, and what you would otherwise have to write yourself:

It is the same philosophy as Genkit Java and the JavaScript flavor I covered in my 2026 JS/TS Gen AI frameworks comparison: a thin, opinionated, cloud-agnostic layer that turns “AI logic” into a typed function you can call, test, deploy, and observe.

What We Are Going to Build

A Go service exposing a single AI flow that generates a structured recipe from a main ingredient and optional dietary restrictions. It will:

• Accept a typed RecipeInput as input.
• Call Gemini 3 Pro via the Google AI plugin.
• Return a strongly-typed Recipe struct (no manual JSON parsing).
• Be served as an HTTP endpoint on :3400.
• Be testable visually in the Genkit Developer UI.

All in a single main.go file. No web framework. No code generation. Just Go.

Prerequisites

• Go 1.24+ (install)
• Node.js 18+ (only required for the Genkit CLI / Developer UI)
• A Google GenAI API key (free, no credit card, from Google AI Studio)

Install the Genkit CLI

The Genkit CLI is your local companion for running and inspecting flows in the Developer UI:

curl -sL cli.genkit.dev | bash

Verify it:

genkit --version

Set Up the Project

Create a fresh module:

mkdir genkit-go-recipes && cd genkit-go-recipes
go mod init example/genkit-go-recipes

Install the Genkit Go package:

go get github.com/firebase/genkit/go

Set your API key:

export GEMINI_API_KEY=<your API key>

The Code: a Single main.go

Create main.go with the following content. This is the entire service.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "log"
    "net/http"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
    "github.com/firebase/genkit/go/plugins/server"
)

// Input schema, picked up automatically by Genkit and the Dev UI.
type RecipeInput struct {
    Ingredient          string `json:"ingredient" jsonschema:"description=Main ingredient or cuisine type"`
    DietaryRestrictions string `json:"dietaryRestrictions,omitempty" jsonschema:"description=Any dietary restrictions"`
}

// Output schema, returned directly by the model as a typed Go struct.
type Recipe struct {
    Title        string   `json:"title"`
    Description  string   `json:"description"`
    PrepTime     string   `json:"prepTime"`
    CookTime     string   `json:"cookTime"`
    Servings     int      `json:"servings"`
    Ingredients  []string `json:"ingredients"`
    Instructions []string `json:"instructions"`
    Tips         []string `json:"tips,omitempty"`
}

func main() {
    ctx := context.Background()

    // Initialize Genkit with the Google AI plugin and a default model.
    g := genkit.Init(ctx,
        genkit.WithPlugins(&googlegenai.GoogleAI{}),
        genkit.WithDefaultModel("googleai/gemini-3-pro"),
    )

    // Define a typed flow: (RecipeInput) -> (Recipe, error)
    recipeGeneratorFlow := genkit.DefineFlow(g, "recipeGeneratorFlow",
        func(ctx context.Context, input *RecipeInput) (*Recipe, error) {
            dietary := input.DietaryRestrictions
            if dietary == "" {
                dietary = "none"
            }

            prompt := fmt.Sprintf(`Create a recipe with the following requirements:
                Main ingredient: %s
                Dietary restrictions: %s`, input.Ingredient, dietary)

            // Structured generation: Gemini returns a Recipe directly.
            recipe, _, err := genkit.GenerateData[Recipe](ctx, g,
                ai.WithPrompt(prompt),
            )
            if err != nil {
                return nil, fmt.Errorf("failed to generate recipe: %w", err)
            }
            return recipe, nil
        },
    )

    // Smoke-test the flow once at boot.
    recipe, err := recipeGeneratorFlow.Run(ctx, &RecipeInput{
        Ingredient:          "avocado",
        DietaryRestrictions: "vegetarian",
    })
    if err != nil {
        log.Fatalf("could not generate recipe: %v", err)
    }
    out, _ := json.MarshalIndent(recipe, "", "  ")
    fmt.Println("Sample recipe generated:")
    fmt.Println(string(out))

    // Expose the flow as an HTTP endpoint.
    mux := http.NewServeMux()
    mux.HandleFunc("POST /recipeGeneratorFlow", genkit.Handler(recipeGeneratorFlow))

    log.Println("Starting server on http://localhost:3400")
    log.Println("Flow available at: POST http://localhost:3400/recipeGeneratorFlow")
    log.Fatal(server.Start(ctx, "127.0.0.1:3400", mux))
}

Take a moment to count what is not in this file:

• No web framework.
• No JSON parsing of the model output.
• No manual OpenTelemetry setup.
• No request/response DTO duplication.
• No Dockerfile yet (we will not need much).

The struct is the contract. The flow is the endpoint. The compiler enforces both.

Run It

go run .

You should see a structured recipe printed as JSON, then the server logging that it is listening on :3400.

In another terminal, hit it with curl:

curl -X POST "http://localhost:3400/recipeGeneratorFlow" \
  -H "Content-Type: application/json" \
  -d '{"data": {"ingredient": "tomato", "dietaryRestrictions": "vegan"}}'

You will get back a fully structured JSON recipe. That is it, you have a production-shaped Gen AI microservice in one file.

Test It Visually with the Developer UI

The Genkit Developer UI is one of the strongest reasons to adopt Genkit, regardless of language. It gives you a local web app to run flows, inspect traces, tweak prompts, and debug tool calls.

From the project root:

genkit start -- go run .

Open http://localhost:4000, pick recipeGeneratorFlow, paste:

{
  "ingredient": "avocado",
  "dietaryRestrictions": "vegetarian"
}

Click Run. You will see the typed output and a full trace of the model call: tokens, latency, prompt, response. This is the kind of inner loop Python frameworks are still catching up on.

Deploying It

Because it is Go, deployment is almost anticlimactic. A minimal Dockerfile:

FROM golang:1.24 AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -o /out/server .

FROM gcr.io/distroless/static
COPY --from=build /out/server /server
ENV PORT=3400
EXPOSE 3400
ENTRYPOINT ["/server"]

That is your entire production image. Deploy it to Cloud Run, Cloud Run Jobs, Kubernetes, AWS Lambda (via container image), Azure Container Apps, or any platform that runs containers. No Python runtime to vendor. No pip install at build time. No virtual environment. Just a binary.

If you want to see the same pattern applied to other clouds and languages, I have already covered:

• Genkit + AWS Lambda + Bedrock
• Genkit + Azure Functions + AI Foundry
• Genkit Java 101

Genkit Go fits the same mold, with the smallest runtime footprint of all of them.

“But What About…”

A few honest objections worth addressing.

• “All the cool research libraries are in Python.” True. Keep them in Python, behind a small Python service that does only the research-y bit. Put your product surface (the part your users actually call) in Go. That separation is healthy.
• “My team only knows Python.” Go is famously the easiest “real” backend language to learn. A Python developer can be productive in Go in days, and Genkit’s API surface is small enough that the learning curve is mostly Go itself, not the framework.
• “What about LangChain / LlamaIndex features?” Most of what those frameworks give you (flows, tools, RAG, prompts, evaluation, observability) Genkit Go gives you too, with a fraction of the surface area and without the abstraction tax. See my 2026 frameworks comparison for the long version.
• “Is Genkit Go production-ready?” It powers Gen AI features at Google and a growing list of companies. The Go SDK shares the same core philosophy and plugin model as the JS and Java SDKs. It is stable enough to bet on, and the iteration speed is high.

Wrapping Up

Python earned its place as the language of AI research. It did not earn its place as the language of AI services. Those are different problems with different constraints, and the constraints of production services, concurrency, footprint, deployment, types, observability, all favor Go.

Genkit Go is the framework that finally makes that switch painless. You get a typed, observable, multi-provider Gen AI service in one file, one binary, and one deploy. If you are still maintaining a Python microservice whose only job is to call an LLM and return structured JSON, you are paying a tax you do not need to pay.

Try it on your next flow. Replace one Python service. See how much smaller the resulting system is, in code, in memory, and in operational surface area.

References

• Genkit Go, Get Started
• Genkit Go, Flows
• Genkit Go, Tool Calling
• Genkit Go, Deployment on Cloud Run
• Genkit GitHub

Introduction

Java has always been a serious language for production systems, and in 2026, the Generative AI ecosystem has finally caught up. For years, Java developers watched from the sidelines as Python and TypeScript accumulated framework after framework for building LLM-powered applications. Today, the picture is very different. Java has multiple mature, actively maintained AI frameworks, each with its own philosophy and trade-offs.

This article covers the four frameworks I have personally used to ship Java AI applications: Genkit Java, Spring AI, LangChain4j, and Google ADK Java. Each one represents a meaningfully different bet on what a Java AI framework should be, and understanding those differences will save you from picking the wrong tool.

Genkit Java

History and Direction

Genkit started life as a TypeScript-first framework launched by Google at I/O 2024. The Java SDK arrived as a community-maintained effort, built and maintained by developers within the Google ecosystem who wanted to bring the same developer experience to Java that Genkit had established in TypeScript. As of 2026, Genkit Java is unofficial, it is not an official Google product, but it is actively maintained, follows the core Genkit design closely, and ships its own plugin ecosystem.

The framework’s first stable release landed in early 2026 after months of preview use. Its ambition mirrors the TypeScript SDK’s: bring Genkit’s multi-level abstractions (vanilla generation, typed flows, agents), its broad provider-neutral plugin model, and, crucially, the Genkit Developer UI to Java developers. The Java SDK ships with Spring Boot and Jetty server plugins, making it a natural fit for teams that already run Java services in production. The Javadoc and architecture are clean and idiomatic Java, this does not feel like a port; it feels designed for the language.

The direction is clear: maintain parity with the TypeScript Genkit SDK’s abstractions while embracing Java idioms (builder patterns, typed schemas via Java classes, annotation-free configuration). Support for evaluation, MCP (Model Context Protocol), RAG with pgvector and Pinecone, and multi-agent patterns is already in place.

What Makes Genkit Java Stand Out

Like its TypeScript counterpart, Genkit Java provides three levels of abstraction in a single SDK: direct model calls, typed flows (observable pipelines), and agents. This is unique in the Java AI space, no other Java framework gives you all three in one coherent API.

Supported languages: Java 21+ (primary). Deploys to Spring Boot, Jetty, or Firebase Cloud Functions.

Vanilla Generation

import com.google.genkit.Genkit;
import com.google.genkit.ai.GenerateOptions;
import com.google.genkit.plugins.googlegenai.GoogleGenAIPlugin;

Genkit genkit = Genkit.builder()
    .plugin(GoogleGenAIPlugin.create())
    .build();

String text = genkit.generate(GenerateOptions.builder()
    .model("googleai/gemini-flash-latest")
    .prompt("Explain the CAP theorem in two sentences.")
    .build()).getText();

System.out.println(text);

Typed Flows — Observable Pipelines

Flows are the heart of Genkit Java. They wrap your AI logic in a named, typed, traceable unit that is automatically exposed as an HTTP endpoint and visible in the Dev UI.

import com.google.genkit.Genkit;
import com.google.genkit.flow.FlowOptions;
import com.google.genkit.plugins.googlegenai.GoogleGenAIPlugin;
import com.google.genkit.plugins.jetty.JettyPlugin;
import com.google.genkit.plugins.jetty.JettyPluginOptions;

record TranslateRequest(String text, String targetLanguage) {}
record TranslateResponse(String translation, String detectedLanguage) {}

JettyPlugin jetty = new JettyPlugin(JettyPluginOptions.builder().port(8080).build());

Genkit genkit = Genkit.builder()
    .plugin(GoogleGenAIPlugin.create())
    .plugin(jetty)
    .build();

genkit.defineFlow(
    FlowOptions.<TranslateRequest, TranslateResponse>builder()
        .name("translateText")
        .inputClass(TranslateRequest.class)
        .outputClass(TranslateResponse.class)
        .build(),
    (ctx, request) -> {
        var response = genkit.generate(GenerateOptions.builder()
            .model("googleai/gemini-flash-latest")
            .prompt("Translate '%s' to %s. Return JSON with 'translation' and 'detectedLanguage'."
                .formatted(request.text(), request.targetLanguage()))
            .outputClass(TranslateResponse.class)
            .build());
        return response.getOutput(TranslateResponse.class);
    }
);

jetty.start();

Tools and Agents

import com.google.genkit.ai.tool.ToolDefinition;

var weatherTool = genkit.defineTool(
    ToolDefinition.<String, String>builder()
        .name("getWeather")
        .description("Returns current weather for a city.")
        .inputClass(String.class)
        .outputClass(String.class)
        .build(),
    (ctx, city) -> "Sunny, 24°C in " + city
);

// Use the tool inside a flow or agent
var result = genkit.generate(GenerateOptions.builder()
    .model("googleai/gemini-flash-latest")
    .prompt("What's the weather like in Tokyo?")
    .tools(List.of(weatherTool))
    .build());

The Dev UI — Same Power as TypeScript

One of Genkit Java’s most compelling features is that the same Genkit Developer UI used by the TypeScript SDK works directly with Java applications. You install the Genkit CLI (Node.js-based) and start your Java app through it:

npm install -g genkit
genkit start -- mvn exec:java

The Dev UI opens at http://localhost:4000 and gives you:

• Flow runner — execute any flow interactively with custom inputs and inspect typed outputs.
• Trace explorer — full OpenTelemetry traces for every generate and flow call, showing latency, token counts, and exact prompts.
• Model playground — test any registered model directly.
• Tool testing — stub and test tools in isolation.
• Dotprompt editor — edit .prompt files live with variable injection.

This is the single biggest advantage Genkit Java has over every other Java AI framework: a zero-config, local developer UI that replaces the need for LangSmith or Grafana during development.

Provider Support

Genkit Java ships plugins for: Google GenAI (Gemini), OpenAI, Anthropic (Claude), AWS Bedrock, Azure AI Foundry, Ollama, xAI (Grok), DeepSeek, Cohere, Mistral, and Groq. All accessed through the same genkit.generate() interface.

Vector store plugins cover: Firebase Firestore, Weaviate, PostgreSQL (pgvector), Pinecone, and a local in-memory store.

Pros and Cons

Spring AI

History and Direction

Spring AI was announced by the Spring team (Broadcom) in mid-2023 and reached its 1.0 GA release in mid-2024. It is the most enterprise-grade option in this comparison, built by the same team that maintains Spring Framework, Spring Boot, and Spring Data, which together underpin a vast proportion of the world’s Java server-side applications.

The founding premise of Spring AI is that AI integration in Java applications should feel like every other Spring integration: auto-configured, testable, portable, and production-ready out of the box. The project draws inspiration from LangChain and LlamaIndex but explicitly avoids being a port, it is designed from the ground up to be idiomatic Spring. If you have written Spring applications, Spring AI will feel immediately familiar: @Autowired AI clients, Spring Boot starters, application.properties configuration, and Advisor patterns that mirror Spring’s existing interception model.

Spring AI’s direction through 2025 and into 2026 has been to deepen its observability story (Micrometer-native metrics and traces), expand its ChatClient fluent API, and ship more vector store integrations. The framework is now the de facto standard for teams that are already invested in the Spring ecosystem and want to add AI capabilities without introducing a foreign dependency philosophy.

What Makes Spring AI Stand Out

Spring AI’s killer feature is Spring Boot integration depth. There is no framework on this list, in any language, that integrates AI capabilities as seamlessly into an existing application framework as Spring AI does with Spring Boot. Auto-configuration, conditional beans, health indicators, Actuator endpoints for AI metrics, everything a Spring developer expects, applied to AI.

Supported languages: Java (primary). Also supports Kotlin (via Spring’s Kotlin DSL). Runs anywhere Spring Boot runs: embedded Tomcat, Jetty, Undertow, GraalVM native images.

// application.properties
// spring.ai.openai.api-key=${OPENAI_API_KEY}
// spring.ai.openai.chat.options.model=gpt-4o

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.*;

@RestController
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/chat")
    public String chat(@RequestParam String message) {
        return chatClient.prompt()
            .user(message)
            .call()
            .content();
    }
}

Structured Output

Spring AI’s BeanOutputConverter maps model responses directly to Java POJOs, using the class schema to generate format instructions automatically.

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.converter.BeanOutputConverter;

record MovieReview(String title, int rating, String summary, List<String> pros) {}

BeanOutputConverter<MovieReview> converter = new BeanOutputConverter<>(MovieReview.class);

MovieReview review = chatClient.prompt()
    .user(u -> u.text("Review the movie Inception. {format}")
        .param("format", converter.getFormat()))
    .call()
    .entity(MovieReview.class);

System.out.println(review.title() + " — " + review.rating() + "/10");

RAG with Advisors

Spring AI’s Advisors API is one of its most elegant features. Advisors wrap ChatClient calls with cross-cutting concerns, RAG retrieval, chat memory, logging, guardrails, in a declarative, composable way.

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.QuestionAnswerAdvisor;
import org.springframework.ai.vectorstore.VectorStore;

@Service
public class DocumentQAService {

    private final ChatClient chatClient;

    public DocumentQAService(ChatClient.Builder builder, VectorStore vectorStore) {
        this.chatClient = builder
            .defaultAdvisors(new QuestionAnswerAdvisor(vectorStore))
            .build();
    }

    public String answerQuestion(String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
    }
}

Observability

Spring AI ships with Micrometer integration out of the box. Every chat call generates spans (Spring Boot tracing) and metrics (prompt token count, completion token count, model latency) visible in any Micrometer-compatible backend: Prometheus, Grafana, Zipkin, or Datadog. There is no separate Dev UI, observability is handled by your existing Spring Boot infrastructure.

Broad Vector Store and Model Support

Spring AI supports 10+ model providers (OpenAI, Anthropic, Google Vertex AI, Amazon Bedrock, Azure OpenAI, Mistral, Ollama, Groq, and more) and 20+ vector stores (PGVector, Pinecone, Weaviate, Redis, Elasticsearch, MongoDB Atlas, Chroma, and more), the broadest integration coverage of any Java AI framework.

Pros and Cons

LangChain4j

History and Direction

LangChain4j was started in early 2023 by a small community of Java developers who noticed that the LLM framework explosion happening in Python had no Java equivalent. Despite the name, the project is not a mechanical port of LangChain Python, it is a fusion of ideas from LangChain, Haystack, LlamaIndex, and original innovation, packaged in a way that makes sense for Java.

It grew quickly through 2023 and 2024, driven by its comprehensive integration list (20+ LLM providers, 30+ vector stores) and its clean two-level abstraction model: low-level primitives for maximum control and high-level AI Services for rapid development. The AI Services pattern, where you define an interface with annotations and LangChain4j implements it for you at runtime, became the framework’s signature feature and arguably the most Java-idiomatic approach to LLM integration in the ecosystem.

By 2025, LangChain4j had formal integrations with Quarkus, Spring Boot, Micronaut, and Helidon, covering every major Java application framework. The team’s direction in 2026 is focused on deepening agentic capabilities (multi-step tools, planning loops, MCP support) and improving the observability story, which has historically been a weaker point compared to Spring AI’s Micrometer integration or Genkit’s Dev UI.

What Makes LangChain4j Stand Out

LangChain4j’s AI Services pattern is its defining feature. Instead of writing imperative LLM call code, you declare an interface, annotate it with @SystemMessage, @UserMessage, and memory annotations, and LangChain4j generates the implementation. The result is AI code that reads like a Java service contract, clean, testable, and completely familiar to Java developers.

Supported languages: Java (primary). Kotlin extensions available (coroutine-based async support). Integrates with Spring Boot, Quarkus, Micronaut, Helidon.

import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.model.openai.OpenAiChatModel;

interface TranslationAssistant {
    @SystemMessage("You are a professional translator. Translate text accurately and naturally.")
    String translate(@UserMessage String text, @V("language") String targetLanguage);
}

var model = OpenAiChatModel.withApiKey(System.getenv("OPENAI_API_KEY"));

TranslationAssistant assistant = AiServices.builder(TranslationAssistant.class)
    .chatLanguageModel(model)
    .build();

String result = assistant.translate("The quick brown fox jumps over the lazy dog", "Spanish");
System.out.println(result);

Memory and Streaming

import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.service.MemoryId;

interface ConversationalAssistant {
    @SystemMessage("You are a helpful assistant.")
    String chat(@MemoryId String userId, @UserMessage String message);
}

ConversationalAssistant assistant = AiServices.builder(ConversationalAssistant.class)
    .chatLanguageModel(model)
    .chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(20))
    .build();

// Each userId gets its own isolated memory
assistant.chat("user-42", "My name is Alice.");
String response = assistant.chat("user-42", "What's my name?");
// Returns: "Your name is Alice."

RAG Pipeline

import dev.langchain4j.data.document.loader.UrlDocumentLoader;
import dev.langchain4j.data.document.splitter.DocumentSplitters;
import dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;

// Ingest documents
var documents = UrlDocumentLoader.load("https://example.com/docs");
var splitter = DocumentSplitters.recursive(500, 50);
var segments = splitter.splitAll(documents);

var embeddingModel = OpenAiEmbeddingModel.withApiKey(apiKey);
var embeddingStore = new InMemoryEmbeddingStore<TextSegment>();
EmbeddingStoreIngestor.ingest(segments, embeddingStore, embeddingModel);

// Build RAG-enabled assistant
interface DocsAssistant {
    String answer(@UserMessage String question);
}

var retriever = EmbeddingStoreContentRetriever.builder()
    .embeddingStore(embeddingStore)
    .embeddingModel(embeddingModel)
    .maxResults(3)
    .build();

DocsAssistant assistant = AiServices.builder(DocsAssistant.class)
    .chatLanguageModel(model)
    .contentRetriever(retriever)
    .build();

Two Abstraction Levels

LangChain4j explicitly offers two levels:

• Low level — ChatModel, UserMessage, AiMessage, EmbeddingStore: full control, more code.
• High level — AiServices: declarative interfaces, minimal boilerplate.

This mirrors what Genkit Java achieves differently. Where Genkit gives you flows and agents as pipeline concepts, LangChain4j uses interface-based AI Services as its high-level abstraction, very idiomatic in Java terms.

Pros and Cons

Google ADK Java

History and Direction

Google ADK (Agent Development Kit) launched in 2024 as a Python-first agent framework targeting enterprise deployments on Google Cloud. Java was a late addition to the multi-language roadmap, with ADK Java 1.0 shipping in early 2026 alongside ADK Go 1.0. The Java SDK arrival was significant: it signaled that Google views ADK as a serious enterprise runtime, not just a Python scripting tool.

ADK Java follows the same design philosophy as the Python SDK: everything is an agent, workflow, or tool. The framework is optimized for building reliable, evaluatable, production-grade multi-agent systems and deploying them to Google Cloud infrastructure, primarily Vertex AI Agent Engine, Cloud Run, and GKE. Like its Python counterpart, ADK Java carries the weight of Google Cloud gravity. The best developer experience, the smoothest deployment path, and the most mature observability story all assume you are running on GCP.

ADK Java 1.0 includes the full agent runtime (LLM agents, sequential/loop/parallel workflow agents), tool calling, MCP support, A2A (Agent-to-Agent) protocol, session/memory management, and streaming. The Java API closely mirrors the Python API in structure, which means the mental model transfers well, but also means the Java SDK carries a style that reflects Python-first design decisions.

ADK Java’s Position: Agent-Only, Enterprise-Grade

Like its Python counterpart, ADK Java is an agent framework, it has no vanilla generation primitive or flow abstraction outside the agent model. Its raison d’être is spinning up reliable, evaluatable agents and deploying them at enterprise scale. If you are building a multi-agent system on Google Cloud and Java is your language of choice, ADK Java 1.0 is Google’s recommended path.

Supported languages: Java (with ADK Java 1.0). Also: Python (primary), TypeScript, Go.

import com.google.adk.agents.LlmAgent;
import com.google.adk.tools.GoogleSearchTool;
import com.google.adk.runner.InMemoryRunner;
import com.google.genai.types.Content;

var researchAgent = LlmAgent.builder()
    .name("researcher")
    .model("gemini-flash-latest")
    .instruction("You help users research topics thoroughly and accurately.")
    .tools(List.of(new GoogleSearchTool()))
    .build();

var runner = new InMemoryRunner(researchAgent);

var session = runner.sessionService().createSession(
    researchAgent.name(), "user-1"
).blockingGet();

var userMessage = Content.fromParts(Part.fromText(
    "What are the latest developments in fusion energy?"
));

runner.runAsync(researchAgent.name(), session.id(), userMessage)
    .blockingForEach(event -> {
        if (event.finalResponse()) {
            System.out.println(event.stringifyContent());
        }
    });

Multi-Agent Orchestration

ADK Java’s multi-agent capabilities match the Python SDK’s, including sequential, parallel, and loop orchestration.

import com.google.adk.agents.SequentialAgent;

var researcher = LlmAgent.builder()
    .name("researcher")
    .model("gemini-flash-latest")
    .instruction("Research the given topic and provide key facts.")
    .build();

var writer = LlmAgent.builder()
    .name("writer")
    .model("gemini-flash-latest")
    .instruction("Write a clear, well-structured article from the research provided.")
    .build();

var editor = LlmAgent.builder()
    .name("editor")
    .model("gemini-flash-latest")
    .instruction("Polish and format the article for publication.")
    .build();

var pipeline = SequentialAgent.builder()
    .name("contentPipeline")
    .subAgents(List.of(researcher, writer, editor))
    .build();

Vertex AI Lock-In

ADK Java’s production deployment story is built around Vertex AI Agent Engine and Google Cloud. While you can run ADK Java locally (via the ADK CLI or directly) and deploy to Cloud Run or GKE independently, the managed evaluation tools, performance dashboards, and enterprise support all assume GCP. This is the clearest example in the Java AI space of a framework built to serve a platform rather than being platform-neutral.

Pros and Cons

Head-to-Head Comparison

Developer Experience

Abstraction Levels

Genkit Java is the only Java AI framework that provides all three levels: vanilla generation, typed flows (pipelines), and agents. Spring AI covers generation and a basic agent model via tools, but lacks a flow abstraction. LangChain4j provides two levels (low-level primitives and high-level AI Services) but is agent/service focused. ADK Java is agent-only.

Observability

Framework Neutrality

Genkit Java and LangChain4j are built to be provider-neutral: they support every major model and deploy to any infrastructure. Spring AI is similarly neutral on model providers, though it carries Spring’s opinionated application framework as a dependency, a worthwhile trade for most Java shops. ADK Java carries the heaviest platform dependency: its full value is unlocked on Google Cloud.

Java Ecosystem Fit

Which Framework Should You Choose?

Choose Genkit Java if:

• You want to iterate on your AI fast and get feedback with less back and forth — Genkit was built from the ground up for powerful local tooling and observability, and the Dev UI is genuinely transformative.
• You need multiple abstraction levels (vanilla calls, typed flows, and agents) in one SDK.
• Provider neutrality matters: you need to swap or mix Gemini, Claude, OpenAI, and Bedrock.
• Your team also writes TypeScript and wants a consistent framework story across both stacks.

Choose Spring AI if:

• You are already running Spring Boot and want AI to feel like any other Spring integration.
• Micrometer-native metrics and traces plugging into your existing Prometheus/Grafana stack are a priority.
• You need the broadest model and vector store coverage with production-grade auto-configuration.
• GraalVM native images are a requirement for your deployment targets.

Choose LangChain4j if:

• You want the most Java-idiomatic high-level AI abstraction: interface-based AI Services with annotations.
• You need the largest integration ecosystem and don’t want to be tied to any application framework.
• Your team works across Spring Boot, Quarkus, Micronaut, and Helidon, LangChain4j is the most framework-agnostic.
• RAG pipelines with rich document ingestion and retrieval are a core use case.

Choose ADK Java if:

• You are building enterprise-grade multi-agent systems and Google Cloud is your runtime.
• You need official Google support and SLA-backed infrastructure for agent deployment.
• Multi-agent orchestration (sequential, parallel, loop) and the A2A interoperability protocol matter.
• Your team is already using the ADK Python SDK and wants to extend to Java services.

Conclusion

Java’s AI framework landscape in 2026 is surprisingly rich. The four frameworks covered here serve genuinely different needs, and unlike in the JavaScript world where Genkit, Vercel, Mastra, LangChain, and ADK overlap significantly, the Java options each occupy a clearer niche.

For enterprise Spring Boot teams, Spring AI is the obvious choice, zero friction, production-ready observability via Micrometer, and the broadest integration matrix. For teams that value developer experience above all, Genkit Java’s Dev UI is a category apart and worth the unofficial status trade-off. For framework-agnostic Java developers who want the most idiomatic Java AI service abstraction, LangChain4j’s AI Services pattern is hard to beat. And for Google Cloud enterprise workloads that need reliable multi-agent orchestration at scale, ADK Java 1.0 is where Google is putting its weight.

The most important thing is that you no longer have an excuse to reach for Python just because it has better AI tooling. Java’s time in generative AI has arrived.

Last updated: April 2026. Framework versions referenced: Genkit Java 1.0.0-SNAPSHOT, Spring AI 1.x, LangChain4j 0.36.x, Google ADK Java 1.0.

Introduction

The Generative AI tooling ecosystem has exploded over the past two years. What started as a handful of Python libraries has grown into a rich, opinionated landscape of frameworks spanning multiple languages, deployment targets, and philosophy bets. As a developer who has shipped production applications using all five of the frameworks covered in this article, Genkit, Vercel AI SDK, Mastra, LangChain, and Google ADK, I want to offer a practical, hands-on view of where each one excels, where each one falls short, and what I would reach for depending on the project I’m building.

This is not a benchmark post. Tokens per second and latency numbers go stale within weeks. Instead, this is a developer experience and architecture comparison, the kind of thing that matters when you’re deciding what framework will carry your product through 2026 and beyond.

A quick note on scope: all five frameworks are in active development and moving fast. Code samples in this article use the APIs as of April 2026.

Genkit

History and Direction

Genkit was announced by Google at Google I/O 2024 as an open-source framework designed to bring production-ready AI tooling to full-stack developers, regardless of their cloud provider. At the time, the JavaScript/TypeScript ecosystem lacked a coherent story for building AI-powered features with the kind of developer ergonomics you’d expect from, say, a Next.js app. Firebase’s team set out to fix that, building Genkit not as a proprietary Firebase product but as a cloud-agnostic SDK with first-class support for plugins.

By mid-2024, Genkit had already attracted a community plugin ecosystem covering AWS Bedrock, Azure OpenAI, Ollama, Cohere, and a growing list of vector stores. The framework reached its 1.0 milestone in late 2024 and shipped major expansions in 2025, most notably adding Python (preview), Go, and Dart (preview) SDKs alongside the primary TypeScript runtime. This multi-language vision is central to Genkit’s story: it aspires to be the framework you reach for no matter what stack you’re running. As of 2026, the Dart SDK has matured notably, making Genkit one of the very few AI frameworks with meaningful Flutter support, giving mobile developers a first-class path into generative AI that no other framework on this list can match. It is also important to note that Genkit has a unofficial Java SDK, maintained by the community, which has been used in production but is not officially supported by the Genkit team.

The team’s declared direction is to deepen Genkit’s role as a full-stack AI layer: strong observability primitives baked into the runtime, composable workflow abstractions (flows), and an expanding model plugin ecosystem. The ambition is not just to be a bridge to a single model provider but to be the connective tissue that lets you swap providers, mix modalities, and trace every hop in your pipeline, all from one coherent API. Of course, Adding more campabilities to its DEV UI is also a major focus, with the goal of making it the best local development experience for AI applications, regardless of where they deploy.

What Makes Genkit Stand Out

Genkit occupies a unique position among the frameworks in this comparison: it is the only one that provides multiple levels of abstraction in a single, coherent API. You can call a model directly (vanilla generation), compose steps into a typed flow, or wire up a fully autonomous agent, and you can mix all three in the same application. Most other frameworks force you to choose a lane.

Supported languages: TypeScript/JavaScript (primary, stable), Python (preview), Go, Dart/Flutter (preview)

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

// Vanilla generation — no abstraction needed
const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'What is the capital of France?',
});
console.log(text);

Flows — Composable, Typed Pipelines

Flows are Genkit’s first-class pipeline primitive. They are strongly typed, observable end-to-end, and automatically traced in the Dev UI. You define them once and can invoke them from CLI, HTTP, or the Dev UI without any extra scaffolding.

import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const summarizeFlow = ai.defineFlow(
  {
    name: 'summarizeArticle',
    inputSchema: z.object({ url: z.string().url() }),
    outputSchema: z.object({ summary: z.string(), keyPoints: z.array(z.string()) }),
  },
  async ({ url }) => {
    const { output } = await ai.generate({
      model: googleAI.model('gemini-flash-latest'),
      prompt: `Summarize the article at ${url} and list the key points.`,
      output: {
        schema: z.object({ summary: z.string(), keyPoints: z.array(z.string()) }),
      },
    });
    return output!;
  }
);

Agent Abstractions

For agents, Genkit uses definePrompt with tools and a system prompt to define specialized agents, along with tool calling via defineTool and conversation memory, all integrated with the same tracing and observability infrastructure that flows use. The agent model is deliberate: it gives you control over how much autonomy you hand over to the model.

import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const weatherTool = ai.defineTool(
  {
    name: 'getWeather',
    description: 'Returns current weather conditions for a given city.',
    inputSchema: z.object({ city: z.string() }),
    outputSchema: z.object({ temperature: z.number(), condition: z.string() }),
  },
  async ({ city }) => {
    // Real implementation would call a weather API
    return { temperature: 22, condition: 'Sunny' };
  }
);

const travelAgent = ai.definePrompt(
  {
    name: 'travelAdvisor',
    description: 'Travel Advisor can help with trip planning and weather-based advice',
    model: googleAI.model('gemini-flash-latest'),
    tools: [weatherTool],
    system: 'You are a helpful travel advisor. Use available tools to give accurate advice.',
  }
);

// Start a chat session with the agent
const chat = ai.chat(travelAgent);
const response = await chat.send('Should I pack a jacket for my trip to Lisbon?');
console.log(response.text);

The Dev UI — Where Genkit Truly Shines

The Genkit Developer UI is, frankly, the killer feature. No other framework in this comparison comes close to what Genkit offers locally. You launch it with a single command:

npx genkit start

The Dev UI gives you:

• Flow runner — execute any flow with a custom input, inspect the typed output, and view the full execution trace.
• Model playground — invoke any registered model directly, tweak prompt templates, compare outputs.
• Tool testing — stub and test individual tools in isolation before wiring them into an agent.
• Trace explorer — every generate, flow, and agent call is traced with latency breakdowns, token counts, and the exact prompts and completions sent to the model. This is OpenTelemetry-compatible telemetry, exportable to Cloud Trace, Langfuse, or any OTEL collector.
• Dotprompt editor — Genkit’s .prompt files (Dotprompt) are editable live in the UI, with real-time preview and variable injection.
• Session replay — replay any traced session end-to-end to reproduce bugs without re-running the full application.

This local observability loop collapses what normally requires a deployed tracing backend (LangSmith, Langfuse, Weave) into a zero-config experience that runs entirely offline. For development speed, this is enormous.

Vercel’s Developer Tool, by comparison, is a lightweight panel primarily for inspecting HTTP streaming responses. It doesn’t offer flow visualization, trace exploration, or tool testing. It’s functional but basic, the kind of thing you’d expect as a starting point, not a full developer experience.

Broad Model Support — Provider Neutral by Design

Genkit ships official plugins for Google AI (Gemini), Google Vertex AI, OpenAI, Anthropic Claude, Cohere, Mistral, Ollama (local models), AWS Bedrock, and more. The community has extended this to xAI, DeepSeek, Perplexity, and Azure OpenAI. Every model, regardless of provider, is accessed through the same ai.generate() interface, and every call is automatically traced.

import { genkit } from 'genkit';
import { anthropic } from 'genkitx-anthropic';
import { openAI } from 'genkitx-openai';

const ai = genkit({ plugins: [anthropic(), openAI()] });

// Switch between providers without changing downstream code
const { text: claudeResponse } = await ai.generate({
  model: anthropic.model('claude-sonnet-4-5'),
  prompt: 'Explain transformer attention in one paragraph.',
});

const { text: gptResponse } = await ai.generate({
  model: openAI.model('gpt-4o'),
  prompt: 'Explain transformer attention in one paragraph.',
});

Pros and Cons

Vercel AI SDK

History and Direction

The Vercel AI SDK was born out of a practical need: Vercel builds the infrastructure that powers a large portion of the modern web, and as developers started shipping AI features inside Next.js apps in 2023, the friction of integrating streaming LLM responses into React was painfully apparent. Vercel released the initial AI SDK as an open-source library to standardize streaming, provider integration, and UI hooks across their ecosystem.

The SDK grew quickly, adding support for Vue, Svelte, SolidJS, and plain Node.js, but its DNA remains deeply tied to the Vercel and Next.js stack. Version 3 in 2024 introduced streamUI, which lets you stream React components as model output, a paradigm-shift for building truly generative user interfaces. Version 4, shipping in late 2024, brought generateObject and streamObject with Zod schemas, structured output across all providers, and an expanded agent API. By 2026, AI SDK v6 has established itself as the go-to choice for teams that live in the Vercel/React ecosystem and want the lowest-friction path from a prompt to a production UI.

Vercel’s direction is clear: deeper integration between AI, edge compute, and the frontend. The AI Gateway, launched in 2025, acts as a provider proxy with load balancing and fallback, another layer of lock-in dressed as a convenience. The SDK is intentionally lower-level than Genkit or Mastra, favoring simplicity and composability over opinionated abstractions.

What Makes the Vercel AI SDK Stand Out

The Vercel AI SDK’s greatest strength is its seamless integration with React and the web UI layer. useChat, useCompletion, and useObject hooks wire directly into streaming AI responses with built-in state management, loading indicators, and error boundaries. If you’re building a Next.js app and want to add a chat interface or a streaming form, nothing gets you there faster.

Supported languages: TypeScript/JavaScript (primary). Node.js, React, Next.js, Nuxt, SvelteKit, SolidStart, Expo (React Native).

// app/api/chat/route.ts (Next.js App Router)
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openai('gpt-4o'),
    messages,
  });

  return result.toDataStreamResponse();
}

// app/page.tsx — chat UI with one hook
'use client';
import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}><b>{m.role}:</b> {m.content}</div>
      ))}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Say something..." />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

Structured Generation and Agent Patterns

The SDK provides clean primitives for structured output and tool use, though the abstractions are deliberately minimal. You get generateText, streamText, generateObject, streamObject, and a simple maxSteps loop for agentic behavior. There is no high-level “flow” abstraction or graph, you compose these primitives yourself.

import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const { object } = await generateObject({
  model: openai('gpt-4o'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.object({ name: z.string(), amount: z.string() })),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a recipe for a vegan chocolate cake.',
});

Genkit vs. Vercel AI SDK — Abstraction Levels

Compared to Genkit, the Vercel AI SDK operates at a lower level of abstraction. This is by design, Vercel wants to give you sharp, composable tools, not an opinionated framework. The trade-off is that you assemble more boilerplate yourself. Want to trace a multi-step agent? Wire up OpenTelemetry manually. Want a typed pipeline? Build it yourself. Genkit bakes these in.

Conversely, Vercel’s deep UI integration, streaming RSC, useChat, generative UI patterns, is something Genkit does not attempt to own. For Flutter-based applications, Genkit’s Dart SDK fills this role, but in the web domain, Vercel wins on integration depth.

Pros and Cons

Mastra

History and Direction

Mastra is the youngest framework in this comparison, founded in 2024 by the team behind Gatsby (Cade Diehm and Sam Bhagwat). Coming from a background of developer experience tooling and static-site generation, Mastra’s founders approached AI framework design with a strong bias toward TypeScript ergonomics, workflow-first thinking, and integrated tooling. The name “Mastra” (Swahili for “master”) reflects the team’s ambition to be the definitive TypeScript-native AI orchestration layer.

Mastra reached public beta in late 2024 and gained significant traction in early 2025 among TypeScript developers frustrated with LangChain’s Python-ported patterns. The framework’s distinct feature, a built-in Studio UI, arrived in early 2025 and quickly became its marquee differentiator. Mastra Studio is a web-based visual interface for defining, testing, and running agents and workflows, accessible locally or in the cloud. By mid-2025, Mastra had secured seed funding and announced hosted cloud infrastructure for deploying Mastra agents directly from the Studio.

Mastra’s direction is firmly in the TypeScript/JavaScript ecosystem. The team has shown no signs of pursuing multi-language support; instead, they are doubling down on deep integrations with popular TypeScript meta-frameworks like Next.js, Astro, SvelteKit, and Hono. Think of Mastra as the opinionated, batteries-included agent framework for TypeScript developers who want to spin up production agents as fast as possible, without writing any platform glue.

What Makes Mastra Stand Out

Mastra is purpose-built for one thing: spinning up agents fast. It is an agent-only framework, you will not find vanilla model calls or a “flow” primitive. Everything in Mastra is modelled around agents, tools, memory, and workflows. If you know exactly what you need (an agent with memory and tool access), Mastra gets you there in fewer lines of code than any other framework here.

Supported languages: TypeScript/JavaScript exclusively. Integrations with Next.js, Astro, SvelteKit, Hono, Express.

import { Mastra, Agent } from '@mastra/core';
import { openai } from '@mastra/openai';

const researchAgent = new Agent({
  name: 'researcher',
  model: openai('gpt-4o'),
  instructions: `You are a research assistant. 
    Find relevant information, synthesize key points, 
    and present clear, well-structured summaries.`,
  tools: {
    // Tools added here
  },
});

const mastra = new Mastra({ agents: { researchAgent } });

const response = await mastra.getAgent('researcher').generate([
  { role: 'user', content: 'Summarize the latest developments in quantum computing.' },
]);

console.log(response.text);

Workflows

Mastra’s workflow primitive lets you chain agent steps into typed, directed graphs, useful when you need a mix of deterministic logic and LLM reasoning.

import { Workflow, Step } from '@mastra/core';
import { z } from 'zod';

const contentPipeline = new Workflow({
  name: 'contentPipeline',
  triggerSchema: z.object({ topic: z.string() }),
});

contentPipeline
  .step({
    id: 'research',
    execute: async ({ context }) => {
      const { topic } = context.triggerData;
      // Agent call to research the topic
      return { research: `Key facts about ${topic}` };
    },
  })
  .then({
    id: 'draft',
    execute: async ({ context }) => {
      const { research } = context.getStepResult('research');
      // Agent call to draft the article
      return { draft: `Article draft using: ${research}` };
    },
  })
  .commit();

Pros and Cons

LangChain

History and Direction

LangChain is, by a significant margin, the most widely used AI framework in the world, but its story is complicated. Harrison Chase created LangChain in October 2022 as a Python library for chaining LLM calls, and it spread virally through the developer community in early 2023 as everyone scrambled to experiment with GPT-3 and GPT-4. Its key insight, that useful AI applications require structured chains of calls, retrieval augmentation, and tool integration, was correct and arrived at the right moment. GitHub stars and npm downloads shot to the top of every chart.

The JavaScript port, langchain on npm, arrived shortly after and has tracked the Python library closely in both API design and feature parity. This is the source of one of LangChain’s most persistent criticisms: the JavaScript SDK feels like Python idioms force-translated into TypeScript. Patterns like BaseChain, runnable pipelines with .pipe(), and the LCEL (LangChain Expression Language) make perfect sense coming from Python’s compositional patterns but feel unnatural to TypeScript developers accustomed to async/await and module-based composition.

LangChain the company raised $35M in 2023 and has since built a growing platform around LangSmith (observability and evaluation) and LangGraph (graph-based orchestration). This is where the tension lies: LangChain’s open-source SDK and LangSmith are designed to complement each other. Getting the best observability experience requires using LangSmith. While you can configure other backends, the seamless experience is on their platform. The framework is excellent and featureful, but its commercial direction is unmistakably pointed toward LangSmith adoption.

In 2025, LangChain reorganized its JavaScript library around a cleaner agent API (create_agent) and introduced Deep Agents, pre-built agent implementations with built-in context compression and subagent spawning. LangGraph remains the recommended framework for complex multi-step workflows, and LangSmith continues to be the best-in-class platform for production LLM observability.

LangChain’s Position: Agent-First, Platform-Tied

LangChain is squarely an agent framework. Its sweet spot is spinning up capable agents quickly, particularly for teams coming from the Python AI ecosystem who want to move to or stay in JavaScript without losing the LangChain mental model. It is the most feature-complete framework here in terms of raw agent capabilities, RAG patterns, and integrations, but that breadth comes with complexity.

Supported languages: Python (primary, feature-complete), JavaScript/TypeScript (JS port, near-parity). Note: the JS SDK carries Python-style patterns.

import { createAgent } from 'langchain/agents';
import { ChatOpenAI } from '@langchain/openai';

function getWeather(city: string): string {
  // Real implementation would call a weather API
  return `It's always sunny in ${city}!`;
}

const model = new ChatOpenAI({ model: 'gpt-4o', temperature: 0 });

const agent = createAgent({
  model,
  tools: [
    {
      name: 'get_weather',
      description: 'Get weather for a given city.',
      func: getWeather,
    },
  ],
  systemPrompt: 'You are a helpful assistant.',
});

const result = await agent.invoke({
  messages: [{ role: 'user', content: 'What is the weather in Madrid?' }],
});
console.log(result.messages.at(-1)?.content);

LangSmith Observability

LangSmith is LangChain’s answer to the observability problem. It provides trace visualization, dataset management, prompt versioning, and LLM evaluation, all polished and production-grade. The integration with LangChain is seamless: set LANGSMITH_TRACING=true and every run is captured automatically.

The catch is that LangSmith is a SaaS platform. Genkit’s Dev UI provides comparable local observability with zero cloud dependency. If you need hosted, team-scale observability, LangSmith is arguably the best option in the market. If you need local, zero-config development tracing, Genkit wins.

Pros and Cons

Google ADK (Agent Development Kit)

History and Direction

Google ADK was announced at Google Cloud Next 2024 as Google’s opinionated take on a production-grade agent framework, specifically targeting enterprise deployments on Google Cloud. Unlike Genkit, which is cloud-agnostic and full-stack, ADK was designed from day one around Vertex AI and Google Cloud’s agent infrastructure, including Agent Engine, Cloud Run, and GKE. It is the framework Google recommends when you’re building agents that will live in a Google Cloud environment at scale.

ADK’s initial release was Python-only, which told the story clearly: this was a framework for the enterprise Python AI developer, data scientists, ML engineers, and cloud architects who think in agents and workflows and are already committed to Google Cloud. The TypeScript, Go, and Java SDKs followed in 2025, with ADK Go 1.0 and ADK Java 1.0 shipping in early 2026. This multi-language expansion signals that Google is positioning ADK as more than a Python script runner, it wants to be the enterprise agent runtime for any Google Cloud workload.

ADK 2.0, released in 2026, brought significant refinements: graph-based workflow APIs, a visual Web UI builder, enhanced evaluation tooling (including user simulation and environment simulation for testing agents end-to-end), and deeper A2A (Agent-to-Agent) protocol support. The A2A protocol is an open standard that allows ADK agents to communicate with agents built on other frameworks, a meaningful interoperability effort in a fragmented ecosystem.

Google’s direction with ADK is unmistakable: this is enterprise AI infrastructure for Google Cloud customers. If your organization runs on GCP and needs reliable, scalable, observable agent deployments with enterprise support, ADK is Google’s answer. If you need to be cloud-agnostic, look elsewhere.

ADK’s Position: Agent-First, Enterprise-Grade

Like LangChain and Mastra, ADK is an agent-only framework, its reason for existing is to make building, evaluating, and deploying agents fast and reliable. Unlike Mastra (which targets indie developers and startups), ADK is purpose-built for enterprise scenarios: multi-agent systems, graph-based orchestration, agent evaluation at scale, and deployment to Google’s managed infrastructure.

Supported languages: Python (primary, feature-complete), TypeScript/JavaScript, Go, Java. Note: the API design and documentation are heavily Python-first; TypeScript and other SDKs track but sometimes lag the Python feature set.

# Python — ADK's primary language
from google.adk import Agent
from google.adk.tools import google_search

research_agent = Agent(
    name="researcher",
    model="gemini-flash-latest",
    instruction="You help users research topics thoroughly and accurately.",
    tools=[google_search],
)

# Run locally
result = research_agent.run("What are the latest developments in fusion energy?")
print(result.text)

// TypeScript ADK
import { Agent } from '@google/adk';
import { googleSearch } from '@google/adk/tools';

const researchAgent = new Agent({
  name: 'researcher',
  model: 'gemini-flash-latest',
  instruction: 'You help users research topics thoroughly and accurately.',
  tools: [googleSearch],
});

const result = await researchAgent.run(
  'What are the latest developments in fusion energy?'
);
console.log(result.text);

Multi-Agent Systems

ADK’s multi-agent support is one of its strongest features. You can compose agents hierarchically, assign them different models, and let them collaborate via the A2A protocol.

from google.adk import Agent
from google.adk.agents import SequentialAgent, ParallelAgent

researcher = Agent(name="researcher", model="gemini-flash-latest", instruction="Research the topic.")
writer = Agent(name="writer", model="gemini-pro-latest", instruction="Write a clear article from the research.")
editor = Agent(name="editor", model="gemini-flash-latest", instruction="Polish and format the article.")

content_pipeline = SequentialAgent(
    name="contentPipeline",
    agents=[researcher, writer, editor],
)

result = content_pipeline.run("Write an article about the impact of quantum computing on cryptography.")

Vertex AI Lock-In

ADK’s evaluation, deployment, and production observability features lean heavily on Vertex AI Agent Engine, Cloud Trace, and Google’s managed infrastructure. You can run ADK locally and even deploy to Cloud Run or GKE independently, but to get the full ADK experience, including agent evaluation, performance dashboards, and managed scaling, you’re on Google Cloud. This is similar to how LangSmith is the intended observability backend for LangChain: technically optional, practically expected.

Frameworks like Genkit, Vercel AI SDK, and Mastra were designed from the ground up to be cloud-neutral. ADK and LangChain, by contrast, have strong ecosystem gravity toward their respective platforms.

Pros and Cons

Head-to-Head Comparison

Developer Experience

Abstraction Levels

Genkit is the only framework that gives you all three levels in one SDK: vanilla generation, typed flows (pipelines), and agents. Vercel AI SDK lives at the lower end, it gives you clean generation and tool-calling primitives but no flow abstraction. Mastra, LangChain, and ADK are agent frameworks: they optimize for spinning up agents quickly but don’t offer a coherent story for when you just want to generate text or structure a pipeline without agent autonomy.

Observability

Language Support

Framework Neutrality

Genkit, Vercel AI SDK, and Mastra were built from the ground up to be provider-neutral. They support OpenAI, Anthropic, Google, and others through a unified API, and they deploy to any infrastructure.

LangChain and ADK are platform-influenced. LangChain’s full power unlocks with LangSmith; ADK’s full power unlocks on Google Cloud. This is not a dealbreaker, both platforms are excellent, but it is an architectural commitment you should make consciously.

Idiom and Code Style

Genkit, Mastra, and Vercel AI SDK feel natively TypeScript: async/await everywhere, Zod schemas for validation, module-based composition, and no runtime class inheritance chains to navigate.

LangChain and ADK’s TypeScript SDKs carry the weight of their Python origins. You’ll find class-heavy APIs, .pipe() chains, and patterns that feel natural if you’ve written LangChain Python but unfamiliar if you’re coming from the TypeScript world. This is not a quality judgment, it’s a cultural fit question.

Which Framework Should You Choose?

After building with all five, here’s my honest take:

Choose Genkit if:

• You want to iterate on your AI fast and get feedback with less back and forth — Genkit was built from the ground up for powerful local tooling and observability.
• You need to mix vanilla generation, typed pipelines (flows), and agents in the same app.
• Provider neutrality is important now or likely to be important later.
• You’re building a Flutter/Dart mobile app and need AI capabilities.
• You want OpenTelemetry-compatible tracing without configuring a separate backend.

Choose Vercel AI SDK if:

• You’re building a React/Next.js app and want the lowest-friction path to streaming AI UI.
• Simplicity and minimal API surface matter more than built-in abstractions.
• You’re already on the Vercel platform and want native integration.
• Your use case maps well to the UI hooks (useChat, useCompletion, generative UI).

Choose Mastra if:

• You’re a TypeScript developer who wants to spin up a production agent as fast as possible.
• You want a clean, idiomatic TypeScript agent API without Python-ported patterns.
• The visual Studio UI for workflow design appeals to your team.
• You’re building in the Next.js/SvelteKit/Hono ecosystem.

Choose LangChain if:

• Your team is coming from the Python AI ecosystem and wants cross-language continuity.
• You need the broadest possible integration ecosystem (the most integrations of any framework).
• You’re investing in LangSmith for production observability and want a cohesive platform.
• LangGraph’s graph-based orchestration matches your workflow complexity.

Choose ADK if:

• You’re building enterprise-grade multi-agent systems on Google Cloud.
• Vertex AI’s infrastructure (Agent Engine, Cloud Trace, Vertex Search) is already in your stack.
• You need battle-tested multi-language support including Go and Java.
• Agent evaluation at scale (user simulation, custom metrics) is a core requirement.

Conclusion

The Generative AI framework landscape in 2026 is not a winner-take-all market. Each of the five frameworks covered here has a legitimate use case, a growing community, and an active development team.

If I had to crown one framework as the most versatile choice for teams that haven’t already committed to a cloud platform, it would be Genkit. Its combination of multi-level abstractions, provider neutrality, and, above all, the Developer UI creates a development experience that genuinely accelerates iteration. The fact that it is expanding to Dart/Flutter, Python, and Go while keeping its TypeScript SDK as the best-in-class experience is a sign of a team thinking about the long game.

That said, none of these frameworks is going away. LangChain’s ecosystem depth, ADK’s enterprise footprint, Vercel’s UI ergonomics, and Mastra’s TypeScript-native speed all serve real needs. The most important thing is to make the choice deliberately, understanding what you’re trading when you pick a platform-tied framework, and what you’re gaining when you pick a more opinionated one.

Happy building.

Last updated: April 2026. Framework versions referenced: Genkit 1.x, Vercel AI SDK 6.x, Mastra 0.x (latest), LangChain JS 0.3.x, Google ADK 2.0.

Introduction

One of the most powerful things about Genkit is that it is cloud-agnostic. You are not locked into a single provider. In this post, we will explore how to run Genkit flows on AWS Lambda using the AWS Bedrock plugin, deploying a full AI-powered story and joke generator with streaming support, all managed by the Serverless Framework.

The project uses the onCallGenkit helper from the AWS Bedrock plugin, which wraps any Genkit flow as a Lambda handler automatically, handling CORS, request parsing, error formatting, and even streaming via Lambda Function URLs.

Why Genkit on AWS?

If your infrastructure lives on AWS, you might think Genkit is not for you. Think again. The community-maintained AWS Bedrock plugin brings first-class Bedrock support to Genkit, giving you:

• Access to Amazon Nova, Anthropic Claude, and other Bedrock models
• The onCallGenkit helper for zero-boilerplate Lambda handlers
• Full compatibility with the Genkit Dev UI for local development
• Streaming support via Lambda Function URLs

Prerequisites

• Node.js 20 or later
• AWS Account with AWS CLI configured and access to AWS Bedrock
• Serverless Framework (installed as dev dependency)
• Genkit CLI installed globally

npm install -g genkit-cli

Project Structure

genkit-aws-lambda-bedrock/
├── src/
│   └── index.ts          # Genkit flows + Lambda handlers via onCallGenkit
├── serverless.yml        # Serverless Framework configuration
├── tsconfig.json         # TypeScript configuration
├── package.json          # Dependencies and scripts
└── README.md

Initializing Genkit with Bedrock

The setup is minimal. Import the plugin, initialize Genkit, and pick your model:

import { genkit, z } from 'genkit';
import { awsBedrock, amazonNovaProV1, onCallGenkit } from 'genkitx-aws-bedrock';

const ai = genkit({
  plugins: [awsBedrock()],
  model: amazonNovaProV1(),
});

That is it. Genkit is now configured to use Amazon Nova Pro via Bedrock. You can swap to anthropicClaude35SonnetV2 or any other supported model with a single line change.

Defining Flows

Genkit flows are the core building block. Each flow has typed input and output schemas using Zod, making everything type-safe from end to end.

Story Generator Flow

const StoryInputSchema = z.object({
  topic: z.string().describe('The main topic or theme for the story'),
  style: z.string().optional().describe('Writing style (e.g., adventure, mystery, sci-fi)'),
  length: z.enum(['short', 'medium', 'long']).default('medium'),
});

const StoryOutputSchema = z.object({
  title: z.string(),
  genre: z.string(),
  story: z.string(),
  wordCount: z.number(),
  themes: z.array(z.string()),
});

const storyGeneratorFlow = ai.defineFlow(
  {
    name: 'storyGeneratorFlow',
    inputSchema: StoryInputSchema,
    outputSchema: StoryOutputSchema,
  },
  async (input) => {
    const lengthMap = { short: '200-300', medium: '500-700', long: '1000-1500' };
    const wordCount = lengthMap[input.length];

    const prompt = `Create a creative ${input.style || 'fictional'} story with the following requirements:
      Topic: ${input.topic}
      Length: ${wordCount} words
      
      Please provide a captivating story with a clear beginning, middle, and end.
      Include rich descriptions and engaging characters.`;

    const { output } = await ai.generate({
      prompt,
      output: { schema: StoryOutputSchema },
    });

    if (!output) {
      throw new Error('Failed to generate story');
    }

    return output;
  }
);

Notice how ai.generate returns a fully structured, typed object. No JSON parsing, no string manipulation. Genkit handles all of that for you.

Joke Flow with Streaming

Genkit also supports streaming responses. The jokeStreamingFlow uses ai.generateStream and sendChunk to emit text chunks as they arrive from the LLM:

const jokeStreamingFlow = ai.defineFlow(
  {
    name: 'jokeStreamingFlow',
    inputSchema: z.object({
      subject: z.string().describe('The subject to tell a joke about'),
    }),
    outputSchema: z.object({
      joke: z.string(),
      type: z.string().optional(),
    }),
    streamSchema: z.string(),
  },
  async (input, sendChunk) => {
    const { stream, response } = await ai.generateStream({
      prompt: `Tell me a funny joke about ${input.subject}. Make it clever and appropriate for all ages.`,
      output: {
        schema: z.object({
          joke: z.string(),
          type: z.string().optional(),
        }),
      },
    });

    for await (const chunk of stream) {
      sendChunk(chunk.text);
    }

    const result = await response;
    return result.output || { joke: result.text };
  }
);