Plugin SDK reference

Plugin runtime media helpers

Speech, media understanding, generation, web search, and the low-level media utilities. Part of the Plugin runtime helpers reference.

Media and generation namespaces

api.runtime.tts

Text-to-speech synthesis.

typescript
// Standard TTSconst clip = await api.runtime.tts.textToSpeech({  text: "Hello from OpenClaw",  cfg: api.config,}); // Telephony-optimized TTSconst telephonyClip = await api.runtime.tts.textToSpeechTelephony({  text: "Hello from OpenClaw",  cfg: api.config,}); // List available voicesconst voices = await api.runtime.tts.listVoices({  provider: "elevenlabs",  cfg: api.config,});

Uses core tts configuration and provider selection. Returns PCM audio buffer + sample rate. textToSpeechStream is also available for streaming synthesis.

api.runtime.mediaUnderstanding

Image, audio, and video analysis.

typescript
// Describe an imageconst image = await api.runtime.mediaUnderstanding.describeImageFile({  filePath: "/tmp/inbound-photo.jpg",  cfg: api.config,  agentDir: "/tmp/agent",}); // Prepare a capture limit before installing audio receive listeners.const budget = await api.runtime.mediaUnderstanding.resolveAudioInputBudget({  cfg: api.config,});// budget.enabled is false when audio understanding is disabled. Otherwise,// budget.maxBytes includes the container header and covers the largest fallback. // Transcribe audioconst { text } = await api.runtime.mediaUnderstanding.transcribeAudioFile({  filePath: "/tmp/inbound-audio.ogg",  cfg: api.config,  mime: "audio/ogg", // optional, for when MIME cannot be inferred}); // Describe a videoconst video = await api.runtime.mediaUnderstanding.describeVideoFile({  filePath: "/tmp/inbound-video.mp4",  cfg: api.config,}); // Generic file analysisconst result = await api.runtime.mediaUnderstanding.runFile({  filePath: "/tmp/inbound-file.pdf",  cfg: api.config,}); // Structured image extraction through a specific provider/model.// Include at least one image; text inputs are supplemental context.const evidence = await api.runtime.mediaUnderstanding.extractStructuredWithModel({  provider: "codex",  model: "gpt-5.6-sol",  input: [    {      type: "image",      buffer: receiptImageBuffer,      fileName: "receipt.png",      mime: "image/png",    },    { type: "text", text: "Prefer the printed total over handwritten notes." },  ],  instructions: "Extract vendor, total, and searchable tags.",  schemaName: "receipt.evidence",  jsonSchema: {    type: "object",    properties: {      vendor: { type: "string" },      total: { type: "number" },      tags: { type: "array", items: { type: "string" } },    },    required: ["vendor", "total"],  },  cfg: api.config,});

Returns { text: undefined } when no output is produced (e.g. skipped input).

describeImageFileWithModel(...) describes an already-known image through a specific provider/model, bypassing the default active-model resolution that describeImageFile(...) uses.

api.runtime.imageGeneration

Image generation.

typescript
const result = await api.runtime.imageGeneration.generate({  prompt: "A robot painting a sunset",  cfg: api.config,}); const providers = api.runtime.imageGeneration.listProviders({ cfg: api.config });
api.runtime.videoGeneration

Video generation, mirroring the image generation shape.

typescript
const result = await api.runtime.videoGeneration.generate({  prompt: "A drone shot flying over a coastline at sunrise",  cfg: api.config,}); const providers = api.runtime.videoGeneration.listProviders({ cfg: api.config });
api.runtime.musicGeneration

Music generation, mirroring the image generation shape.

typescript
const result = await api.runtime.musicGeneration.generate({  prompt: "An upbeat lo-fi track for a coding session",  cfg: api.config,}); const providers = api.runtime.musicGeneration.listProviders({ cfg: api.config });
api.runtime.webSearch

Web search.

typescript
const providers = api.runtime.webSearch.listProviders({ config: api.config }); const result = await api.runtime.webSearch.search({  config: api.config,  args: { query: "OpenClaw plugin SDK", count: 5 },});
api.runtime.media

Low-level media utilities.

typescript
const webMedia = await api.runtime.media.loadWebMedia(url);const mime = await api.runtime.media.detectMime(buffer);const kind = api.runtime.media.mediaKindFromMime("image/jpeg"); // "image"const isVoice = api.runtime.media.isVoiceCompatibleAudio(filePath);const metadata = await api.runtime.media.getImageMetadata(filePath);const resized = await api.runtime.media.resizeToJpeg(buffer, { maxWidth: 800 });const terminalQr = await api.runtime.media.renderQrTerminal("https://openclaw.ai");const pngQr = await api.runtime.media.renderQrPngBase64("https://openclaw.ai", {  scale: 6, // 1-12  marginModules: 4, // 0-16});const pngQrDataUrl = await api.runtime.media.renderQrPngDataUrl("https://openclaw.ai");const tmpRoot = resolvePreferredOpenClawTmpDir();const pngQrFile = await api.runtime.media.writeQrPngTempFile("https://openclaw.ai", {  tmpRoot,  dirPrefix: "my-plugin-qr-",  fileName: "qr.png",});
Was this useful?
On this page

On this page