Data Model Layer (ORM)
Tip
When to read this: When you need data models, automatic table creation, or relationship mapping. Skip it if not needed—it won't affect the use of other features.
Note
The ORM is async-native: All CRUD methods (create / get / save / delete / create_table / QuerySet terminal operations) are async def and can simply be awaited inside the event loop—the underlying drivers are aiosqlite / aiomysql / asyncpg (switched along with the storage backend), so there is no synchronous blocking. The synchronous get/set of KV storage is a compatibility layer, unrelated to the ORM.
Since 2.9.0, the framework ships with a built-in declarative data model layer: inherit Model and declare fields with Field to gain automatic table creation and full CRUD capabilities. Models are built directly on top of the built-in storage layer—SQLite / MySQL / PostgreSQL is determined by the ErisPulse.storage.backend configuration, and switching backends requires no changes to model code.
Declaring Models
from ErisPulse.Core.Bases import Model, Field
class User(Model):
id: int = Field(primary_key=True, autoincrement=True)
name: str = Field(max_length=64)
age: int = Field(default=0, ge=0, le=150)
role: str = Field(default="user", choices=["user", "admin"])
tags: list = Field(default_factory=list) # JSON column, automatically serialized on read/write
bio: str = Field(default="", description={"i18n": "user.bio", "default": "Bio"})
- The table name defaults to the snake_case form of the class name (
UserProfile → user_profile); override it with__tablename__ = "xxx" - The
Fieldconstraint vocabulary is identical to that of declarative config classes (choices/ge/le/max_length;descriptionlikewise supports i18n dictionaries); the validator engine shares the same origin as config validation
Field Parameters
| Parameter | Description |
|---|---|
default |
Default value (when not provided and not autoincrement → required NOT NULL) |
default_factory |
Factory for mutable default values (e.g., list) |
primary_key |
Primary key (multiple ones form a composite primary key, consolidated into a table-level PRIMARY KEY (a, b)) |
autoincrement |
Auto-increment primary key (implies a primary key; automatically backfilled after create; dedicated to single-column primary keys, mutually exclusive with composite primary keys) |
max_length |
Maximum string length (generates VARCHAR(n), validated on write) |
nullable |
Whether NULL is allowed (default False) |
index |
Creates a regular index |
unique |
Unique constraint |
choices / ge / le |
Enumeration / numeric range (validated on write) |
description |
Description (i18n dictionary, same format as config classes) |
column_type |
Overrides the SQL column type definition |
foreign_key |
Column-level foreign key constraint ("table.column", generates REFERENCES) |
on_delete |
Foreign key delete action: CASCADE / SET NULL / RESTRICT / NO ACTION / SET DEFAULT |
auto_now_add |
Automatically writes the current timestamp (float epoch seconds) on insert |
auto_now |
Automatically written on both insert and instance save() updates (not triggered by the QuerySet.update bulk path) |
Composite Primary Keys and Unique-Together Constraints
Multiple primary_key=True fields form a composite primary key; get / save / delete locate rows by all primary key columns;
__unique_together__ declares unique-together groups (multiple groups allowed):
class Membership(Model):
__tablename__ = "memberships"
__unique_together__ = (("user_id", "group_id"),)
user_id: int = Field(primary_key=True)
group_id: int = Field(primary_key=True)
role: str = Field(default="member")
m = await Membership.get(user_id=1, group_id=2)
autoincrementis mutually exclusive with composite primary keys; having both raises an error right at model definition__unique_together__is implemented as unique indexes (index names prefixed withuq_): generated together with newly created tables, and for existing tables, runningcreate_table()adds them automatically—no rebuild needed- Composite primary key models do not yet support relationship mapping (has-many / belongs-to); accessing relationships on their instances raises a clear error
Table Creation and CRUD
await User.create_table() # idempotent (IF NOT EXISTS)
user = await User.create(name="Alice", age=20) # insert (auto-increment primary key backfilled automatically)
got = await User.get(id=user.id) # first row of an equality query
users = await User.where(User.age > 18).order_by("-age").limit(10).all()
first = await User.where(User.name == "Alice").first()
total = await User.count(User.age > 18)
user.age = 21
await user.save() # update by primary key (constraints validated first); falls back to insert when the primary key is missing (auto-increment keys are likewise backfilled onto the instance)
await user.save(update_fields=["age"]) # partial column update (whitelist; other columns are not written)
await user.delete() # delete by primary key
user, created = await User.get_or_create({"age": 20}, name="Alice") # returns it if found; creates it otherwise
user, created = await User.update_or_create({"age": 21}, name="Alice") # updates defaults and saves if found; creates otherwise
await User.update_all(User.age > 18, role="adult") # bulk update
await User.delete_all(User.age > 100) # bulk delete
Note
Under concurrent contention, two coroutines in get_or_create / update_or_create may both reach the creation branch—unique constraints (primary key / unique / __unique_together__) will make the later one receive a StorageOperationError; when idempotent semantics are needed, catch it and re-query once.
Query expressions support > >= < <= == !=, in_([...]), and combinations with & (AND) / | (OR):
await User.where((User.age > 18) & User.name.in_(["Alice", "Bob"])).all()
Transactions
ORM reads and writes share the same transaction routing as the framework's storage layer: when inside an ambient storage.atransaction() transaction, create / save / delete and queries automatically reuse the transaction connection and are committed or rolled back together with the transaction, never committing independently.
async with storage.atransaction():
await User.create(name="Alice")
... # if an exception is raised inside the block, the INSERT above is rolled back as well
Relationship with Declarative Config Classes
Model declarations and ConfigClass (@dataclass + field(metadata=...)) share the same underlying machinery—the constraint vocabulary, the validator engine (validate_field_constraints), and the type category registry (python_type_category); but the class foundations are deliberately separate: config fields are plain values (TOML round-trip, hot-reloaded on load), while model fields are column descriptors (class access = query expressions, one instance per row). One declaration syntax, two foundations each serving its own purpose.
When to Use Which Declaration
| Dimension | Config class field(metadata=...) |
Model field Field() |
|---|---|---|
| Applicable scenarios | Module behavior parameters (few, human-readable, hot-reloadable) | Business data records (many rows, programmatic read/write, queryable) |
| Storage form | config.toml (comments preserved, TOML round-trip) |
Database tables (automatic table creation, SQL dialects) |
| Value form | Plain values (read directly as dataclass attributes) |
Descriptors (class access = column expression, instance access = row value) |
| Constraint declaration | metadata={"choices": ..., "min": ..., "max": ...} |
Field(choices=..., ge=..., le=..., max_length=...) |
| Shared foundation | Validator engine + constraint vocabulary + type category registry (the same set) | Same as left |
Rule of thumb: use config classes for "how a module behaves"; use models for "what data users produce".
Boundaries and Caveats
- The backend is determined by the global storage configuration; a model may override it with a custom
BaseStorageinstance via the__storage__class attribute (for test injection) list/dictfields (including parameterized generics such aslist[int],dict[str, int]) are stored as JSON text columns by container category, automatically serialized on read/write- Constraint validation runs automatically before writes (
create/save); failures raiseValueError(localized messages) - Automatic migration already covers the add-column scenario (see below); column type changes and column drops require manual handling
- Behavior on storage connection failure is consistent with the storage layer: the framework does not crash and automatically reconnects after a cooldown
Automatic Migration (Phase Two)
When the table already exists, create_table() automatically compares the existing columns against the model fields: newly added fields are automatically applied via ALTER TABLE ADD COLUMN (migrated columns drop the NOT NULL constraint; existing rows are backfilled with NULL), and new fields declared with index=True get their indexes created at the same time. No need to write migration scripts by hand.
# Model evolution after v1 ships: adding the email / bio fields
class User(Model):
__tablename__ = "orm_users"
id: int = Field(primary_key=True, autoincrement=True)
name: str = Field(max_length=64)
age: int = Field(default=0)
email: str = Field(default="") # New: ADD COLUMN runs automatically on the next create_table()
bio: str = Field(default="")
await User.create_table() # idempotent: only migrates newly added columns
Boundaries: only adding columns is supported; primary key changes, column type changes, and column drops require manual handling (to avoid destructive ALTER mistakes).
Foreign Keys (Foundation for Relationship Mapping)
foreign_key="table.column" declares a column-level foreign key constraint, and the DDL generates a REFERENCES clause; on_delete declares the delete action (e.g., "CASCADE" to cascade-delete referencing rows):
class Post(Model):
__tablename__ = "posts"
id: int = Field(primary_key=True, autoincrement=True)
author: int = Field(foreign_key="orm_users.id", on_delete="CASCADE")
content: str = Field(max_length=255)
Relationship Mapping (relationship)
Declare relationship() as class attributes in the model class body; the direction is determined automatically by which table the foreign key column is declared in:
from ErisPulse.Core.Bases import Model, Field, relationship
class User(Model):
__tablename__ = "orm_users"
id: int = Field(primary_key=True, autoincrement=True)
name: str = Field(max_length=64)
posts = relationship("Post", foreign_key="author") # foreign key in the other table → has-many
class Post(Model):
__tablename__ = "posts"
id: int = Field(primary_key=True, autoincrement=True)
author: int = Field(foreign_key="orm_users.id")
content: str = Field(max_length=255)
writer = relationship("User", foreign_key="author") # foreign key in this table → belongs-to
has-many: the instance attribute returns a queryset with all QuerySet chaining capabilities available, and create automatically backfills this table's primary key into the other table's foreign key column:
alice = await User.get(name="Alice")
posts = await alice.posts.all() # all posts by this user
latest = await alice.posts.order_by("-id").first()
total = await alice.posts.count()
hot = await alice.posts.where(Post.content != "").all() # appending conditions on the other table's fields
await alice.posts.delete() # deletes only this user's posts
new_post = await alice.posts.create(content="hi") # author automatically = alice.id
belongs-to: directly await to obtain the other side's instance (returns None when there is no match or the foreign key is NULL):
post = await Post.get(id=1)
writer = await post.writer # a User instance or None
await writer.posts.count() # works in both directions
Key points:
relatedaccepts a class-name string (resolved lazily via the model class-name registry, regardless of definition order on either side) or the model class directly- Relationship queries and the related model each use their own storage backend (cross-backend JOIN is not supported—a relationship query is two independent SQL statements)
- Relationship queries are not cached; every access performs a live query. For arbitrary custom queries, you can still use
User.where(...)
Next Steps
Switching Databases or Deploying to a Cloud Database
- Storage Backends — Configuration and switching for SQLite / MySQL / PostgreSQL, with zero changes to model code
No Table Schema Needed, Just Storing a Few Key-Value Pairs
- Module Core Concepts - Storage System — Lightweight key-value storage usage, no model declaration required