Source: https://docs.nygen.io/CyteOnto/guides/models-and-providers/
Revision: cb82698da3fc9e2bc385eb9a70e07f9304835eba
Structured page: https://docs.nygen.io/CyteOnto/guides.models-and-providers.json

Models and providers

CyteOnto uses two models:

An LLM, through a pydantic-ai <https://ai.pydantic.dev/> agent that you create, splits compound labels and writes a description of each label and each Cell Ontology term. LlmConfig names this model in the cache files.

An embedding model, which CyteOnto calls over HTTP, embeds the descriptions. EmbdConfig configures it.

Together they form a model pair. The descriptions of the ontology terms are stored per LLM, and their embeddings per model pair. Changing either model changes the ontology files that CyteOnto needs.

Use a precomputed model pair

python -m cyteonto.setup downloads the ontology files of two model pairs. With either pair, from_config() loads the files and calls no model.

Pair	LlmConfig	EmbdConfig

Default	provider="nebius", model="moonshotai/Kimi-K2.6"	provider="nebius", model="Qwen/Qwen3-Embedding-8B"

Fallback	provider="fireworks", model="accounts/fireworks/models/kimi-k2p6"	provider="openrouter", model="Qwen/Qwen3-Embedding-8B"

The same values are in cyteonto.config.Config, as PRIMARY_LLM_PROVIDER, PRIMARY_LLM_MODEL, PRIMARY_EMBEDDING_PROVIDER, PRIMARY_EMBEDDING_MODEL and the matching FALLBACK_ names.

The file names are built from these strings. The provider is lowercased, but the model strings must match exactly, including their case. EmbdConfig() without arguments selects qwen/qwen3-embedding-8b on OpenRouter, which matches neither pair.

With another model pair, from_config() generates the missing files once and saves them in data_dir:

With another LLM, it describes all 2,902 terms and embeds the descriptions: 2,902 descriptions of one LLM request each, plus one for each PubMed search, and 2,902 embedding requests, plus retries.

With the default LLM and another embedding model, it embeds the 2,902 existing descriptions.

Compare results only between runs that used the same model pair: the scores depend on the models.

Create the agent

CyteOnto uses only the model of your agent, with its own instructions and output types; the agent’s instructions, tools and settings are not used. The model must support tool calling, which pydantic-ai uses for structured output. These providers serve OpenAI-compatible APIs, so their agents use pydantic-ai’s OpenAI model with the provider’s base URL:

Provider	Base URL

Nebius Token Factory	https://api.tokenfactory.nebius.com/v1/

Fireworks	https://api.fireworks.ai/inference/v1

OpenRouter	https://openrouter.ai/api/v1/

Together	https://api.together.xyz/v1/

import os

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

from cyteonto import LlmConfig

# Kimi K2.6 on Fireworks, the LLM of the fallback pair.
fireworks_agent = Agent(
    OpenAIChatModel(
        "accounts/fireworks/models/kimi-k2p6",
        provider=OpenAIProvider(
            api_key=os.environ["FIREWORKS_API_KEY"],
            base_url="https://api.fireworks.ai/inference/v1",
        ),
    )
)
fireworks_llm = LlmConfig(
    provider="fireworks", model="accounts/fireworks/models/kimi-k2p6"
)

LlmConfig.model must equal the model name of the agent, agent.model.model_name, or CyteOnto raises a ValueError. LlmConfig.provider is only a label for the cache files: the agent decides where the requests go.

Choose the embedding provider

EmbdConfig.provider selects the endpoint. CyteOnto sends one request per text, with up to maxConcurrent requests at a time, 10 by default.

provider	Endpoint	API key variable

deepinfra	https://api.deepinfra.com/v1/openai/embeddings	DEEPINFRA_API_KEY

fireworks	https://api.fireworks.ai/inference/v1/embeddings	FIREWORKS_API_KEY

google	https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:embedContent	GOOGLE_API_KEY

nebius	https://api.tokenfactory.nebius.com/v1/embeddings	NEBIUS_API_KEY

openai	https://api.openai.com/v1/embeddings	OPENAI_API_KEY

openrouter	https://openrouter.ai/api/v1/embeddings	OPENROUTER_API_KEY

together	https://api.together.xyz/v1/embeddings	TOGETHER_API_KEY

CyteOnto reads the key from EmbdConfig.apiKey, then from the provider’s variable, then from EMBEDDING_MODEL_API_KEY. Without a key, CyteOnto raises a ValueError. The ollama provider sends the requests to a local Ollama server at http://localhost:11434/api/embed and needs no key.

For openrouter, modelSettings is added to each request body. By default it routes requests to DeepInfra first, with fallbacks to other providers allowed:

from cyteonto import EmbdConfig

# The default routing for OpenRouter.
EmbdConfig(
    provider="openrouter",
    model="Qwen/Qwen3-Embedding-8B",
    modelSettings={"provider": {"order": ["deepinfra"], "allow_fallbacks": True}},
)

# No routing: OpenRouter picks the provider.
EmbdConfig(provider="openrouter", model="Qwen/Qwen3-Embedding-8B", modelSettings={})

Other providers ignore modelSettings.

Fall back to other models

Pass a second agent and embedding configuration, and CyteOnto uses them when the primary models fail:

from cyteonto import CyteOnto, EmbdConfig

cyto = await CyteOnto.from_config(
    agent=agent,
    embedding=embedding,
    llm=llm,
    fallback_agent=fireworks_agent,
    fallback_llm=fireworks_llm,
    fallback_embedding=EmbdConfig(
        provider="openrouter", model="Qwen/Qwen3-Embedding-8B"
    ),
)

fallback_agent describes the labels whose description is still blank after the primary agent’s retries. Its descriptions are cached in the files of the primary llm. Compound labels are always split by the primary agent.

fallback_llm names the fallback agent’s model in cyto.model_pair_usage. It does not change the cache files.

fallback_embedding takes over when a request of a batch fails with the primary provider after its retries. It embeds the batch of labels again; in from_config(), the terms get the ontology embeddings of the fallback model, which are made if they are missing. From then on, the instance embeds with the fallback configuration, saves the label embeddings under that model pair, and matches the labels against the ontology embeddings of the primary llm with the fallback embedding model. In the call that fails over, the labels embedded before the failover are embedded again with the fallback model, so that all the scores of the call come from one model. A fallback configuration equal to embedding is not used.

setup does not download the ontology embeddings of the default LLM with the fallback embedding model. At the first failover, CyteOnto creates them by embedding the 2,902 existing term descriptions with the fallback model before it switches, which delays that call. If that fails, the call fails and the instance keeps the primary model. To avoid the delay during an outage, create them in advance:

# Embed the 2,902 term descriptions of the default LLM with the fallback model.
await CyteOnto.from_config(
    agent=agent,
    embedding=EmbdConfig(provider="openrouter", model="Qwen/Qwen3-Embedding-8B"),
    llm=llm,
)

After a run, cyto.model_pair_usage.to_status_dict() reports which models were used. llmTier becomes mixed once the fallback agent was asked to describe labels. embeddingTier tells which models made the label embeddings that were matched, new or cached: primary, fallback, or mixed for both.

Reasoning

Some models, such as Kimi K2.6, can reason before they answer. With reasoning=False, the default, CyteOnto’s requests ask the provider to turn reasoning off, which makes the descriptions faster and cheaper. It does so with settings that Fireworks, Together and OpenRouter read; other providers may ignore them. Pass reasoning=True to leave reasoning as the provider configures it. The ontology embeddings record whether they were made with reasoning.

The PubMed tool

With use_pubmed_tool=True, the default, the agent that describes a label can search PubMed: each search returns up to 5 abstracts by default from NCBI’s E-utilities, and the prompt allows at most 3 searches per description. Set NCBI_API_KEY to raise NCBI’s rate limit. If PubMed cannot be reached, the search returns no abstracts and the description continues. Pass use_pubmed_tool=False for faster and cheaper descriptions; the hosted API turns the tool off by default.

Concurrency

max_description_concurrency, 100 by default, limits the LLM requests that run at the same time, and EmbdConfig.maxConcurrent, 10 by default, limits the embedding requests. Lower them if your provider rate-limits you. Failed requests are retried; see Environment and limits <https://docs.nygen.io/CyteOnto/reference/configuration> for the retry rules.