Think Models
evroc Think Models is a Model-as-a-Service inference platform where evroc hosts leading open source AI models on NVIDIA H100 and B200 GPUs. You get reliable, performant inference without managing infrastructure.
evroc Think Models offers two ways to run models, each suited to different workloads:
- Shared Models — shared endpoints managed by evroc, billed per token. Best for most use cases.
- Dedicated Model Instances — your own isolated GPU instance, billed per hour. Best when you need full control or predictable cost.
Shared Models
Shared Models are managed by evroc. You call an OpenAI-compatible API endpoint and pay only for the tokens you use. evroc handles scaling, availability, and GPU scheduling — you don't need to provision or manage instances.
Shared Models are the fastest way to start. You create an API key, pick a model from
the supported models list, and send requests to
https://models.think.evroc.com/v1. There's no instance to configure, no GPU to
choose, and no minimum spend.
You're billed per million tokens (input and output separately). Some models also support cached input tokens at a reduced rate, so repeated prompts cost less.
When to use Shared Models
- You want to get started quickly without managing infrastructure
- Your traffic is variable or unpredictable
- You want to try different models before committing to one
- You're building an application that calls an OpenAI-compatible API
Tokens and billing
When you send a request to a language model, the text is converted into tokens — the basic unit a model processes. A token is roughly four characters of text, or about three-quarters of a word. "Hello, world" might be three tokens; a longer sentence like "evroc Think Models runs on European infrastructure" might be eight.
Input and output tokens
Every API request involves two types of tokens:
- Input tokens — the text you send to the model. This includes your system prompt, user messages, conversation history, and any context you pass in. Input tokens are typically cheaper.
- Output tokens — the text the model generates in response. Output tokens are more expensive because they require the model to do generative work, not just read context.
Both are counted and billed separately. A request with a long prompt and a short answer costs mostly in input tokens. A request with a short prompt and a long response costs mostly in output tokens.
Why this matters for cost
Because input and output are priced differently, the structure of your requests directly affects your bill:
- A chatbot with long system prompts and short answers is input-heavy
- A code generation tool with short prompts and long outputs is output-heavy
- A RAG pipeline that passes large document chunks as context uses a lot of input tokens on every request
Dedicated Model Instances
A Dedicated Model Instance is your own isolated deployment of a model on GPUs allocated exclusively to you. You choose the model, the instance size (GPU type and count), and the arguments. The instance runs continuously until you delete it.
When to use Dedicated Model Instances
- You need guaranteed latency and throughput for production workloads
- You want to run a model that isn't available as a Shared Model
- You need full control over instance configuration (arguments, GPU type, memory)
- Your usage is high enough that per-hour billing is cheaper than per-token
Model
The specification of the model to run. In general, a HuggingFace handle, such as
mistralai/Ministral-8B-Instruct-2410.
ModelInstance
A dedicated instance of a model, created by a user. This will be associated with a name and user-specified options (e.g. size, arguments).
ModelInstance size
The set of resources allocated for a specific instance, for instance
1-b200-27c-240g, with the most important of it being the GPU used (assigned in a
static and isolated way, e.g. each GPU resource could be used by at most one model
instance).
Choosing between Shared and Dedicated
| Shared Models | Dedicated Model Instances | |
|---|---|---|
| Billing | Per million tokens (input/output) | Per GPU-hour |
| Scaling | Automatic, handled by evroc | Fixed to your instance size |
| Setup | API key + endpoint | Create instance, choose GPU size |
| Latency | Variable, depends on shared load | Predictable, isolated resources |
| Model selection | Curated catalogue of shared models | Any model you specify |
| Best for | Getting started, variable traffic, experimentation | Production workloads, custom models, high throughput |
See Also
- Supported models — Available models, model cards, and capabilities
- evroc Think Models CLI — Manage model instances and API keys from the command line
- Inference API — OpenAI-compatible API reference