LLM Assistant Component
Use the LLM Assistant component to create stateful, multi-turn chat experiences. It automatically tracks conversation history, allowing your agent to provide coherent and context-aware responses over multiple interactions.
Why this matters
What You’ll Configure
- Model Selection
- Define Behavior
- Configure Inputs
- Advanced Settings
- Handle the Output
- Best Practices
- Troubleshooting Tips
- What to Try Next
Step 1: Select a Model
Choose the language model that will power your assistant. You can use built-in models or connect to your own.
Field | Required? | Description | Tips |
---|---|---|---|
Model | Yes | The LLM used for generating responses. | Defaults to SmythOS-provided models (e.g., OpenAI). You can also select other shared models like Claude or Together AI. For more, see Model Rates. |
Custom Model | No | Connect to your own LLM provider, such as Amazon Bedrock or Google Vertex AI. | This is an enterprise feature. You will need to provide your own credentials and select a foundation model. Contact us to enable it. |
Step 2: Define the Behavior
The Behavior
field acts as the system prompt, giving the assistant its core instructions, personality, and constraints.
Setting | Required? | Description | Default Value |
---|---|---|---|
Behavior | No | The system prompt that guides the assistant's tone and actions. | You are a helpful assistant that helps people with their questions |
Crafting a Good Persona
Step 3: Configure Inputs
These inputs are essential for tracking the conversation and capturing the user's message.
Input | Required? | Description | Notes |
---|---|---|---|
UserId | Yes | A unique identifier for the end-user. | Used to group all conversations for a specific user. |
ConversationId | Yes | A unique identifier for a single conversation thread. | Allows a single user to have multiple, separate conversations. |
UserInput | Yes | The message or prompt submitted by the user. | This is what the assistant will respond to. |
Step 4: Configure Advanced Settings
Fine-tune the assistant's streaming behavior.
Setting | Description |
---|---|
Passthrough Mode | Controls how responses are streamed. When disabled (default), the response streams automatically. When enabled, you get manual control over the output, which is useful for post-processing or custom streaming logic. |
Step 5: Handle the Output
The component produces a single output containing the assistant's reply.
Output | Description | Data Structure |
---|---|---|
Response | The complete message generated by the LLM Assistant. | String |
Best Practices
- Use Stable IDs: Ensure that
UserId
andConversationId
are consistent across interactions to maintain conversation history correctly. - Set a Clear Behavior: A well-defined system prompt in the
Behavior
field is the key to a reliable and predictable assistant. - Manage Context: For very long conversations, be aware of the model's context window. The assistant will automatically handle history, but extremely long threads may eventually lose early context.
- Use Passthrough for Complex Logic: If you need to validate, modify, or log the assistant's reply before showing it to the user, enable
Passthrough Mode
.
Troubleshooting Tips
If your assistant is not working as expected...
What to Try Next
- Use an Agent Skill to provide a user-friendly interface for your
LLM Assistant
. - Pass the
Response
output into a RAG Remember Component to store key facts from the conversation. - Use a Classifier Component on the
UserInput
before it reaches the assistant to detect user intent and route the conversation.