Stability AI Integration with SmythOS

Want to embed cutting-edge image generation into your workflows? Connect Stability AI to SmythOS and empower your agents to create and edit images with state-of-the-art models like Stable Diffusion.

TL;DR

Securely link your Stability AI account to SmythOS using an API key. Then, use our suite of components to automate a wide range of generative image tasks, from text-to-image and image-to-image to inpainting, outpainting, and upscaling.

List of Stability AI Components

Quickly compare Stability AI components by what they do and their key I/O. Click any component name to jump directly to its detailed guide.

Component	Action	What it Does	Inputs	Key Outputs	Use Case
Text-to-Image	Generate	Creates an image from a text prompt.	required `prompt_text`	`url`	Generating marketing assets from a description.
Image-to-Image	Generate	Transforms a source image based on a text prompt.	required `image_binary`, `prompt`	`url`	Applying new styles to existing images.
Image Upscale	Enhance	Increases the resolution and quality of an image.	required `image_binary`, `width`	`url`	Improving low-quality user-uploaded photos.
Inpaint	Edit	Edits a masked area of an image based on a prompt.	required `image_binary`, `prompt` optional `mask_image_binary`	`url`	Removing or adding objects in a photo.
Outpaint	Edit	Extends the borders of an image with AI-generated content.	required `image_binary` optional `prompt`	`url`	Changing an image's aspect ratio.
Search and Replace	Edit	Finds an object in an image and replaces it based on a prompt.	required `image_binary`, `search_prompt`, `prompt`	`url`	Swapping products in a lifestyle photo.
Remove Background	Edit	Automatically removes the background from an image.	required `image_binary`	`url`	Creating product cutouts for e-commerce.
Image-to-Video	Generate	Creates a short video by animating a still image.	required `image_binary`	`id` → `url`	Adding subtle motion to static images for ads.
Creative Upscale	Enhance	Intelligently enhances and upscales degraded images.	required `image_binary`, `prompt`	`id` → `url`	Restoring old photos or low-res art.

INFO

Why Integrate Stability AI with Your Agent?

Stability AI is at the forefront of open-source generative AI. Integrating its models with SmythOS allows you to build powerful, automated visual content pipelines.

Automate Content Creation: Generate unique, high-quality images for blog posts, social media, or ad campaigns on the fly, based on text descriptions or other data.
Dynamic Image Editing: Build agents that can programmatically edit images. Automate tasks like removing backgrounds from product photos, replacing objects in lifestyle shots, or upscaling entire image libraries.
Enhance User-Generated Content: Create workflows that automatically improve the quality of images uploaded by users, using upscaling and restoration models.
Unlock Creative Exploration: Rapidly prototype visual concepts. Use agents to generate dozens of variations of an image based on different prompts, styles, or seeds, helping you find the perfect creative direction faster.

Prerequisites

Before you begin, please ensure you have the following:

An active SmythOS account. (Sign up here).
A Stability AI Platform account.
Your Stability AI API Key.

Getting Started With Stability AI

The connection between SmythOS and Stability AI is configured using a secure API Key.

Step 1: Get Your Stability AI API Key

Log in to your Stability AI Platform account.
Navigate to your account page and click on the API Keys section. You can go there directly via https://platform.stability.ai/account/keys.
Copy your API key. If you don't have one, create one first.

Step 2: Store Your API Key in SmythOS Vault

Your API Key is a sensitive credential. Use the SmythOS Vault to store it securely.

In your SmythOS dashboard, navigate to the Vault.
Create a new secret and paste your Stability AI API Key as the value. Give it a memorable name, like stability_api_key.
For more details, see the Vault Documentation.

Step 3: Configure a Stability AI Component

In your SmythOS agent graph, drag and drop any Stability AI component.
Click the component to open its Settings panel.
In the API Key field, select the secret you saved in the Vault (e.g., stability_api_key).
Your connection is now configured for that component.

Heads-up

You must add the API Key from the Vault to each Stability AI component you use. This ensures all your API calls are properly authenticated.

Which Stability AI Component Should I Use?

If you need to…	Target	Use this Component	Why this one?
Create an image from a description	A text prompt	Text-to-Image	The fundamental component for generating new images from scratch.
Modify an existing image with a prompt	An image file and a text prompt	Image-to-Image	Perfect for applying new styles or making significant changes.
Remove a distracting object from a photo	An image file and a mask	Inpaint	Offers precise control over which part of the image to regenerate.
Extend the canvas of an image	An image file and direction	Outpaint	Intelligently generates new content to expand an image's borders.
Make a low-resolution image sharp	An image file	Image Upscale	A fast and effective way to double the resolution of an image.
Add subtle motion to a static picture	An image file	Image to Video	Creates an engaging short video clip from a single frame.

Component Details

This section provides detailed information for each Stability AI component.

Text-to-Image Models

Generates an image from a text prompt using various Stable Diffusion models. Components include Text-to-Image SD 1.6, SD3 Text to Image, SD3 Turbo Text to Image, and Stable Image Core.

INFO

These components require an API Key for authentication, as detailed in the Getting Started section.

Component-Specific Settings

Style Preset: Select a predefined style like photographic, anime, or cinematic.
Sampler/Clip Guidance: Advanced settings to control the diffusion process.
Image Dimensions/Aspect Ratio: Set the size and shape of the output image.

Inputs

Field	Type	Required	Notes
`prompt_text` / `prompt`	string	Yes	The descriptive text to guide image generation.
`negative_prompt`	string	Optional	Describe elements to avoid in the image.
`seed`	integer	Optional	A number to ensure reproducible results. Default: `0` (random).
`output_format`	string	Optional	The format of the generated image. Default: `png`.

Outputs

Field	Type	Description
`url`	string	A direct URL to the generated image.
`Response`	object	The raw JSON response from the Stability AI API.
`Headers`	object	The HTTP headers from the API response.

Use Case

An agent generates a "thought of the day" quote, then passes it to the Stable Image Core component with a prompt like "A beautiful, serene landscape with the quote quote elegantly written, digital art" to create a shareable social media image.

{
  "component": "stabilityai.sd3TextToImage",
  "prompt": "a photorealistic image of an astronaut riding a horse on Mars",
  "aspect_ratio": "16:9"
}

Prompt Is Everything

The quality and relevance of the output image are highly dependent on the detail and clarity of your prompt.

Image-to-Image Models

Transforms a source image based on a text prompt. Components include SD3 Image to Image and SD3 Turbo Image to Image.

INFO

These components require an API Key for authentication, as detailed in the Getting Started section.

Inputs

Field	Type	Required	Notes
`image_binary`	binary	Yes	The source image file to be transformed.
`prompt`	string	Yes	A description of the desired output image.
`strength`	float	Yes	Controls how much the original image is preserved. A lower value gives the AI more creative freedom. Default: `0.5`.
`seed`	integer	Optional	A number for reproducible results. Default: `0`.

Outputs

Field	Type	Description
`url`	string	A direct URL to the transformed image.
`Response`	object	The raw JSON response from the Stability AI API.
`Headers`	object	The HTTP headers from the API response.

Use Case

An agent takes a real estate photo and uses a prompt like "sunny day, vibrant green grass, blue sky" to enhance the weather and mood of the image.

{
  "component": "stabilityai.sd3ImageToImage",
  "image_binary": "path/to/your/image.png",
  "prompt": "a cyberpunk city at night, neon lights, rain",
  "strength": 0.7
}

Image Editing Tools

Components like Inpaint, Outpaint, Search and Replace, and Remove Background provide powerful editing capabilities.

INFO

These components require an API Key for authentication, as detailed in the Getting Started section.

Common Inputs

Field	Type	Required	Notes
`image_binary`	binary	Yes	The source image file to be edited.
`prompt`	string	Varies	A description to guide the edit (e.g., what to add, what to replace an object with).
`mask_image_binary`	binary	Optional	(Inpaint) A black and white image where white areas indicate where to edit.
`search_prompt`	string	Optional	(Search and Replace) A description of the object to find and replace.

Outputs

Field	Type	Description
`url`	string	A direct URL to the edited image.
`Response`	object	The raw JSON response from the Stability AI API.
`Headers`	object	The HTTP headers from the API response.

Use Case: Background Removal

An agent watches a folder of new product photos, runs each one through the Remove Background component, and saves the resulting transparent PNG to a different folder, ready for use on an e-commerce website.

{
  "component": "stabilityai.inpaint",
  "image_binary": "path/to/your/image.png",
  "mask_image_binary": "path/to/your/mask.png",
  "prompt": "a small, red bird sitting on the branch"
}

Image Enhancement & Video Tools

Components like Image to Image Upscale, Creative Upscale, and Image to Video enhance or animate your images. Note that some of these are asynchronous.

INFO

These components require an API Key for authentication, as detailed in the Getting Started section.

Common Inputs & Asynchronous Flow

Inputs: Typically require an image_binary and may have other parameters like width or creativity.
Asynchronous Flow: Some processes (like Creative Upscale and Image to Video) are asynchronous.
1. You first call the main component (e.g., Image to Video). It returns an id.
2. You must then use this id as input to the corresponding "Get Result" component (e.g., Get Image to Video Result).
3. The "Get Result" component will return the final url once processing is complete.

Outputs

Field	Type	Description
`id`	string	(For async jobs) The ID of the generation process.
`url`	string	(For sync jobs or "Get Result") The direct URL to the final asset.
`Response`	object	The raw JSON response from the Stability AI API.
`Headers`	object	The HTTP headers from the API response.

Use Case: Upscaling

An agent takes a low-resolution user avatar, runs it through the Image to Image Upscale component with a target width of 1024, and updates the user's profile with the new high-resolution version.

// Step 1: Start the Job
{
  "component": "stabilityai.imageToVideo",
  "image_binary": "path/to/your/image.png",
  "motion_bucket_id": 150
}
// Output from Step 1 will contain an "id"

// Step 2: Get the Result
{
  "component": "stabilityai.getImageToVideoResult",
  "id": "id-from-previous-step"
}

Handle Asynchronous Jobs

For components that return an id, you must build a two-step process in your agent. A simple approach is to add a delay after the first call, then call the "Get Result" component.

Best Practices & Advanced Tips

Secure Your API Key: Always store your Stability AI API key in the SmythOS Vault.
Understand Model Differences: The SD3 and Core models have different strengths. Experiment to see which model and style preset works best for your prompts.
Master Prompting: For all generative tasks, the prompt is the most important input. Be descriptive. Include details about subject, style, lighting, and composition. Use the negative_prompt to exclude unwanted elements.
Check API Costs: Image generation and editing consume credits from your Stability AI account. Be aware of the cost of different operations and monitor your usage in your Stability AI dashboard.

Troubleshooting Common Issues

Error: 401 Unauthorized
- Cause: The API Key is incorrect, missing, or invalid.
- Solution: Verify the API Key in your SmythOS Vault is correct and active in your Stability AI account.
Error: 400 Bad Request / Validation Error
- Cause: An input is missing, has the wrong data type, or is outside the allowed range (e.g., image dimensions not divisible by 64).
- Solution: Carefully check all inputs against the component's requirements. For image dimensions, ensure they are valid. For prompts, ensure they are not empty.
Asynchronous Job Never Completes
- Cause: The generation task may have failed on Stability AI's end, or the job is taking a very long time.
- Solution: Check the status of the job via the raw Response from the "Get Result" component. If it remains in a "processing" state for an extended period, try the request again. Check Stability AI's status page for any ongoing issues.

What's Next?

You are now ready to build powerful visual automation workflows with the SmythOS Stability AI Integration!

Consider these ideas:

Build an Agent That...
- Automatically generates featured images for new blog posts based on the post's title and a summary.
- Creates a product personalization tool. A user uploads a photo of their room, and the agent uses Search and Replace to show how a new piece of furniture would look in their space.
- Monitors a folder for new product photos, automatically removes the background, upscales the image, and then creates a short animated video of the product for social media.
Explore Other Integrations:
- Use an LLM from OpenRouter to generate creative, detailed prompts to feed into Stability AI's text-to-image components.
- Generate an image with Stability AI and then use a voice from Elevenlabs to describe it.
- Create a workflow where users can describe an image in a chat, and the agent generates it and sends it back via Slack.

Was this page helpful?

List of Stability AI Components​

Prerequisites​

Getting Started With Stability AI​

Step 1: Get Your Stability AI API Key​

Step 2: Store Your API Key in SmythOS Vault​

Step 3: Configure a Stability AI Component​

Which Stability AI Component Should I Use?​

Component Details​

Text-to-Image Models​

Component-Specific Settings​

Inputs​

Outputs​

Image-to-Image Models​

Inputs​

Outputs​

Image Editing Tools​

Common Inputs​

Outputs​

Image Enhancement & Video Tools​

Common Inputs & Asynchronous Flow​

Outputs​

Best Practices & Advanced Tips​

Troubleshooting Common Issues​

What's Next?​

List of Stability AI Components

Prerequisites

Getting Started With Stability AI

Step 1: Get Your Stability AI API Key

Step 2: Store Your API Key in SmythOS Vault

Step 3: Configure a Stability AI Component

Which Stability AI Component Should I Use?

Component Details

Text-to-Image Models

Component-Specific Settings

Inputs

Outputs

Image-to-Image Models

Inputs

Outputs

Image Editing Tools

Common Inputs

Outputs

Image Enhancement & Video Tools

Common Inputs & Asynchronous Flow

Outputs

Best Practices & Advanced Tips

Troubleshooting Common Issues

What's Next?