arch.ai

Package arch.ai · Version 0.1.0

The ML / AI systems vocabulary — the platform an ML or LLM product runs on, which arch.backend’s vector stores only begin to cover. Provides serving (model_server, llm_gateway), the ML platform (model_registry, feature_store, training_platform), and LLM application pieces (agent_runtime).

Model an inference path like any other: a service calls a model_server’s interface; the model reads features from a feature_store and embeddings from a vector store (arch.backend). LLM apps route through an llm_gateway, which fronts external providers (external_system).

Import the kinds you use, e.g. use model_server, llm_gateway, feature_store from arch.ai.

Types

Type	Extends	Description
`model_registry`	`module`	A model registry — versioned trained-model artifacts and the experiments and lineage behind them
`mlflow`	`model_registry`	Open-source tracking, registry and packaging for the ML lifecycle; a common default
`wandb`	`model_registry`	Experiment tracking and model registry with rich visualization; choose for collaborative experimentation
`neptune`	`model_registry`	Experiment metadata store for tracking runs and models at scale
`feature_store`	`module`	A feature store — serves consistent features to training and inference from paired offline and online stores, preventing train/serve skew
`feast`	`feature_store`	Open-source feature store coordinating offline and online feature serving; a common default
`tecton`	`feature_store`	Managed feature platform with feature pipelines and low-latency serving
`hopsworks`	`feature_store`	Feature store and ML platform with an integrated offline/online store
`training_platform`	`module`	An ML training platform — orchestrates distributed training jobs, hyperparameter search and GPU scheduling
`kubeflow`	`training_platform`	Kubernetes-native ML toolkit with pipelines and training operators
`determined`	`training_platform`	Distributed training platform with built-in hyperparameter search and checkpointing
`metaflow`	`training_platform`	Code-first ML workflow framework from Python; the workflow-shaped member of this family — choose for a code-first path to production
`flyte`	`training_platform`	Kubernetes-native workflow engine for ML and data pipelines with strong typing and versioning
`agent_runtime`	`module`	An LLM agent runtime — runs tool-using agents with planning loops, tool calls and memory
`langgraph`	`agent_runtime`	Graph-structured framework for stateful, multi-step LLM agents
`crewai`	`agent_runtime`	Framework for orchestrating collaborating role-based agents
`model_server`	`module`	A model-inference server — answers predictions behind an API, with batching and versioned endpoints
`triton`	`model_server`	GPU-optimized multi-framework inference server; choose for high-throughput mixed-model serving
`torchserve`	`model_server`	Serving runtime for PyTorch models
`tf_serving`	`model_server`	Serving runtime for TensorFlow models
`vllm`	`model_server`	High-throughput LLM inference server with paged attention; a widely-used default for self-hosted LLMs
`tgi`	`model_server`	Text Generation Inference — a production LLM serving runtime
`ray_serve`	`model_server`	Framework-agnostic serving on Ray; choose to compose multi-step inference graphs
`kserve`	`model_server`	Kubernetes-native model serving with autoscaling and canaries
`bentoml`	`model_server`	Packaging-and-serving framework that bundles models into deployable services
`ollama`	`model_server`	Local-first runtime for running open models on a workstation or single host; choose for local and edge inference
`sglang`	`model_server`	High-throughput LLM serving runtime with structured-output and caching optimizations
`llm_gateway`	`module`	An LLM gateway — a routing/proxy tier fronting model providers with caching, rate limits, cost tracking and failover
`litellm`	`llm_gateway`	Open-source proxy unifying many LLM providers behind one API; choose for self-hosted routing and cost control
`portkey`	`llm_gateway`	Managed LLM gateway with routing, caching and observability