Replicate Integration with SmythOS
Want to run any open-source AI model with a simple API? Connect Replicate to SmythOS and empower your agents to perform cutting-edge AI tasks, from image generation to audio transcription.
List of Replicate Components
Quickly compare Replicate components by what they do and their key I/O. Click any component name to jump directly to its detailed guide.
Component | Action | What it Does | Inputs | Key Outputs | Use Case |
---|---|---|---|---|---|
Llama 2 Chat | Generate | Interacts with the Llama 2 AI model for text generation. | required prompt | id | Conversational AI, Q&A, and text completion. |
Music Generator | Generate | Creates music from a text prompt. | required prompt , duration | id | Generating background music for videos. |
Image Restoration | Enhance | Restores and enhances old or degraded images. | required image_url | id | Improving the quality of scanned old photos. |
Face Restoration | Enhance | Specifically restores and clarifies faces within an image. | required image_url | id | Fixing blurry faces in photographs. |
Background Remover | Edit | Removes the background from a given image. | required image_url | id | Creating product images with transparent backgrounds. |
Answer Image Questions | Analyze | Provides a text answer to a question about an image. | required image_url , question | id | Visual Q&A and image understanding. |
Llama Text Generator | Generate | Generates text content based on a prompt using a Llama model. | required prompt | id | Drafting articles, emails, or creative writing. |
Face Swap | Edit | Swaps a face from a source image onto a target image. | required swap_image_url , target_image_url | id | Creating memes or visual effects. |
Upscale Image | Enhance | Enhances the resolution of an image with AI upscaling. | required image_url , prompt | id | Making low-resolution images sharp and clear. |
Stability Text Generation | Generate | Generates text using a Stability AI model. | required prompt | id | An alternative for high-quality text generation. |
Stability AI Image Generation | Generate | Creates an image from a prompt using a Stable Diffusion model. | required prompt | id | Generating marketing assets or concept art. |
Get Prediction | Fetch | Retrieves the final result of an asynchronous Replicate job. | required prediction_id | Response | The essential second step for all Replicate workflows. |
Prerequisites
Before you begin, please ensure you have the following:
- An active SmythOS account. (Sign up here).
- A Replicate account.
- Your Replicate API Token.
Getting Started With Replicate
The connection between SmythOS and Replicate is configured using a secure API Token.
Step 1: Get Your Replicate API Token
- Log in to your Replicate account.
- Navigate to your Account settings by clicking your profile icon.
- Your API token is displayed on this page. Copy it to your clipboard.
Step 2: Store Your API Token in SmythOS Vault
Your API Token is a sensitive credential. Use the SmythOS Vault
to store it securely.
- In your SmythOS dashboard, navigate to the Vault.
- Create a new secret and paste your Replicate API Token as the value. Give it a memorable name, like
replicate_api_token
. - For more details, see the Vault Documentation.
Step 3: Configure a Replicate Component
- In your SmythOS agent graph, drag and drop any Replicate component.
- Click the component to open its Settings panel.
- In the
API Token
field, select the secret you saved in the Vault (e.g.,replicate_api_token
). - Your connection is now configured for that component.
Which Replicate Component Should I Use?
If you need to… | Target | Use this Component | Why this one? |
---|---|---|---|
Generate an image from a text description | A text prompt | Stability AI Image Generation | The standard way to create images using Stable Diffusion models. |
Generate a piece of music | A text prompt | Music Generator | Specifically designed to create audio files from a musical description. |
Transcribe a meeting recording | An audio file URL | Use the openai/whisper model with the universal "Run Model" component. | The best-in-class model for speech-to-text. |
Fix an old, blurry photo | An image URL | Image Restoration | A specialized model for enhancing the overall quality of old images. |
Get the final result of any generation job | A prediction id | Get Prediction | This is the mandatory second step for every Replicate workflow to retrieve the output. |
Component Details
This section provides detailed information for each Replicate component.
Llama 2 Chat
Interact with the Llama 2 AI model for conversational AI and text generation tasks.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
prompt | string | Yes | The text prompt or question to send to the Llama 2 model. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the prediction job. This is essential for retrieving the result. |
Response | object | The initial API response confirming the job has started. |
Headers | object | The HTTP headers from the API response. |
Music Generator
Generate a piece of music from a text description.
Component-Specific Settings
- Model Version: Choose between different model sizes like
stereo-large
ormelody-large
. - Output Format: Select
wav
ormp3
.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
prompt | string | Yes | A description of the music you want to create (e.g., "An upbeat, 8-bit chiptune track"). |
duration | integer | Yes | The duration of the generated audio in seconds. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the music generation job. |
Response | object | The initial API response confirming the job has started. |
Image Restoration
Restores and enhances old or low-quality images using advanced AI models.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
image_url | string | Yes | The public URL of the image to be restored. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the image restoration job. |
Response | object | The initial API response confirming the job has started. |
Face Restoration
A specialized model that focuses on restoring and clarifying human faces within an image.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
image_url | string | Yes | The public URL of the image containing faces to restore. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the face restoration job. |
Response | object | The initial API response confirming the job has started. |
Background Remover
Removes the background from an image, leaving the foreground subject with a transparent background.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
image_url | string | Yes | The public URL of the image to process. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the background removal job. |
Response | object | The initial API response confirming the job has started. |
Answer Image Questions
Analyzes an image and provides a text-based answer to a question about it.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
image_url | string | Yes | The public URL of the image to analyze. |
question | string | Yes | The question you want to ask about the image. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the visual Q&A job. |
Response | object | The initial API response confirming the job has started. |
Llama Text Generator
Generates text content based on a provided prompt using a Llama family model.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
prompt | string | Yes | The starting point or context to guide the text generation. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the text generation job. |
Response | object | The initial API response confirming the job has started. |
Face Swap
Swaps the face from a source image onto a target image.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
swap_image_url | string | Yes | The URL of the image containing the face you want to use. |
target_image_url | string | Yes | The URL of the image where the face will be placed. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the face swap job. |
Response | object | The initial API response confirming the job has started. |
Upscale Image
Enhances the resolution of an image using AI-powered upscaling.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
image_url | string | Yes | The URL of the low-resolution image to be upscaled. |
prompt | string | Yes | A prompt to guide the upscaling process. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the upscaling job. |
Response | object | The initial API response confirming the job has started. |
Stability AI Image Generation
Generates a high-quality image from a text prompt using a Stable Diffusion model.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
prompt | string | Yes | A description of the image you want to generate. |
Outputs
Field | Type | Description |
---|---|---|
id | string | The unique ID of the image generation job. |
Response | object | The initial API response confirming the job has started. |
Get Prediction
Retrieves the final result of an asynchronous job started by any other Replicate component.
Inputs
Field | Type | Required | Notes |
---|---|---|---|
prediction_id | string | Yes | The id obtained from the output of another Replicate component (e.g., Llama 2 Chat, Music Generator). |
Outputs
Field | Type | Description |
---|---|---|
Response | object | The full JSON response from the API, which includes the status of the job and the final output . |
Headers | object | The HTTP headers from the API response. |
Best Practices & Advanced Tips
- Secure Your API Token: Always store your Replicate API token in the SmythOS
Vault
. - Master the Asynchronous Flow: The key to using Replicate is the two-step process. First, call a generative component to get a prediction
id
. Second, use theGet Prediction
component (often in a loop with a delay) to poll for the final result. Check thestatus
field in the response until it is "succeeded". - Check Model Inputs on Replicate: Before using a new model, visit its page on the Replicate website and click the "API" tab. This will show you the exact structure and required parameters for the
input
object. - Use Public URLs: For all components that take an image or audio file as input, the URL must be publicly accessible for Replicate's servers to download it.
Troubleshooting Common Issues
-
Error:
401 Unauthorized
- Cause: The API Token is incorrect, invalid, or has been revoked.
- Solution: Verify that the API Token in your SmythOS Vault is correct and matches the one in your Replicate account settings.
-
Error:
402 Payment Required
- Cause: You have run out of funds in your Replicate account.
- Solution: Check your billing status and add credits to your Replicate account.
-
Prediction Fails or is Stuck "processing"
- Cause: The model may have encountered an error with your specific input, or there could be a temporary issue with the model on Replicate's side.
- Solution: Check the
error
field in the response from theGet Prediction
component for a detailed message. Try a different, simpler input to see if the model is working. Check the Replicate status page for any known issues.
-
Incorrect Input Error
- Cause: The JSON object passed as the
input
does not match the schema the model expects. - Solution: Go to the model's page on Replicate and carefully review the "API" tab. Ensure every parameter name and data type in your input matches their example exactly.
- Cause: The JSON object passed as the
What's Next?
You are now ready to build agents with a virtually limitless set of AI capabilities using the SmythOS Replicate Integration!
Consider these ideas:
-
Build an Agent That...
- Creates a full video short. It uses an LLM to write a script, Elevenlabs to generate a voiceover, a Replicate text-to-image model to create scenes, and a Replicate image-to-video model to animate them.
- Acts as an "image expert." A user provides an image URL, and the agent uses
Answer Image Questions
to describe it,Face Restoration
to enhance it, andBackground Remover
to create a cutout version. - Transcribes a podcast episode using the
openai/whisper
model on Replicate, then uses an LLM from OpenRouter to generate show notes and chapter timestamps.
-
Explore Other Integrations:
- Generate images with Replicate and store them in OneDrive or another cloud storage service.
- Use Replicate's speech-to-text models to process audio from customer support calls, then log the transcript as a note in GoHighLevel or HubSpot.