Make default link types required

[martastain]

This pull request introduces logic to ensure that all required default link types are present when loading or deploying a project. The main focus is on improving data integrity by automatically adding any missing default link types to a project's configuration, both during project loading and deployment. The changes also improve logging for better traceability.

Ensuring presence of default link types:

• Added a new function ensure_required_project_link_types in project.py to check and create any missing default link types for a project during the loading process. This function is called in the _load method of ProjectEntity to guarantee required link types exist.
• Updated deploy_project.py to iterate through default_link_types and append any missing ones to the project's anatomy during deployment, with debug logging for visibility.

[kalisp]

How this could be actually tested? Trying to restore dump with missing link types?

[martastain]

You could just delete some link types in develop and then switch to this branch... it should be repopulated 🤞

[BigRoy]

When should this trigger and recreate them?

I I delete Link Types from anatomy from a project, then restarting server - it doesn't recreate them.

Doesn't seem to work for me @martastain

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[martastain]

Fixed the check when cache is used

[martastain]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

🖕

[Copilot]

🟡 Changes recommended

Cached loading can bypass validation, and conflict handling can cache a link type that is not present in the database.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Conflict handling and cache invalidation leave required link types vulnerable to errors or stale data.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

ayon_server/entities/project.py:108

• When the canonical name is already occupied by a different link type, this branch only logs and continues, so the project still lacks the required default link type despite this function's guarantee. This collision needs to be surfaced or reconciled rather than silently returning an invalid configuration.

                logger.warning(
                    f"Cannot create required link type {candidate.name} in "
                    f"project {project_name}: name is taken by a conflicting "
                    "link type"
                )

ayon_server/entities/project.py:111

• If the insert was skipped because an equal row already exists, appending candidate discards that row's stored data in the payload. A stale project-data cache can hit this path after a link-type update, causing customized color/style values to be replaced by defaults until the cache is rebuilt; append the fetched existing row instead.

        link_types.append(candidate.dict())

ayon_server/entities/project.py:190

• The backfill is also skipped for a warm project-data cache because this call checks only the cached list, not the database. Since link-type PUT/DELETE handlers do not invalidate project-data, deleting a default leaves the row absent while _load returns the stale cached entry; invalidate this cache on mutations or re-read link types before accepting the cache.

                if await ensure_required_project_link_types(
                    project_name, payload["link_types"]
                ):

• Files reviewed: 5/5 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Conflict handling can omit an existing required type, and transactional backfills may publish uncommitted data to Redis.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

ayon_server/entities/project.py:311

• This backfill can run inside an existing Postgres.transaction() for for_update=True loads, but _load writes project-data before that transaction commits. If the caller rolls back without reaching ProjectEntity.save()'s cache cleanup, the cache will claim these defaults exist while the database does not; later cache hits then skip the backfill. Avoid publishing this payload until commit, or skip caching transactional loads.

ayon_server/entities/project.py:101

• The conflict lookup returns only one row for an OR predicate. Because name and the identity triple are separate unique constraints, the database can contain one row matching candidate.name and another matching the candidate identity; fetchrow may return the former, so this branch omits the required identity even though it already exists. Query the constraints separately (or inspect all matches) and prefer the row matching the identity before declaring the default unavailable.

            conflicting = await Postgres.fetchrow(
                f"""
                SELECT name, link_type, input_type, output_type, data
                FROM project_{project_name}.link_types
                WHERE name = $1

• Files reviewed: 6/6 changed files
• Comments generated: 0 new
• Review effort level: Lite

[BigRoy]

It seems to work now - whenever I try to remove the existing links, it seems to save with success, but upon refresh they are back. That does make it a bit confusing possibly, because it doesn't tell you in frontend that you can't seem to delete them?

Which makes me think, should we also show these entries differently in anatomy? And, could we?

These are the links it seems to enforce: