Custom Storage

Custom Storage lets you connect your own vector database, such as Pinecone, to handle document embeddings outside of SmythOS. This gives you full control over storage, scaling, and retrieval logic, especially in enterprise or multi-agent environments.

Why use Custom Storage?

Use Custom Storage if you need external access to your vector data, want to scale embedding limits, or plan to share the same data across multiple agents or systems.

How it works

When Custom Storage is enabled, all indexing operations from your Data Spaces write directly to the external vector store. SmythOS fetches context from there during retrieval-augmented generation (RAG).

This setup is managed from the Data Pool.

Pinecone setup

To use Pinecone:

1. Create an account at pinecone.io
2. Create an index with:
   • Dimension: 1536
   • Environment: e.g. us-west4-gcp
3. Copy the following values from your Pinecone dashboard:
   • API Key
   • Index Name
   • Index Host URL
   • Region/Environment
4. In SmythOS Studio, open Data Pool
5. Click Customize Storage
6. Paste your Pinecone credentials into the form
7. Save configuration

Paste your Pinecone credentials and save to activate external storage.

Vector dimension must be 1536

SmythOS requires a vector dimension of 1536 to match the embedding model format. Other values will not work.

When to use internal vs custom storage

Use Case	Internal Storage	Custom Storage (Pinecone)
Quick setup and testing	✓
Production-grade indexing		✓
Control over storage and cost		✓
Multi-agent shared knowledge base		✓
Maximum retrieval speed within SmythOS	✓

Most users can start with internal storage

If you are unsure whether you need Custom Storage, begin with the default setup. You can migrate later if needed.

What's Next

• Data Pool overview
• Create and manage Data Spaces
• Connect data to agents
• Monitor usage in Analytics