Skip to content

missingFields on scoped responses (scaffold-only)

Date: 2026-04-27 Status: §11 phases A+C shipped 2026-04-27 with curation populated for the four registered person-entities; Phase B (Guardian / Files specs) and Phase D (E2E + frontend wiring) outstanding. Earlier sections describe the original scaffold iteration as merged. Iteration: Scaffold + alignment lift (see §11)


1. Goal

Surface a per-scope missingFields: string[] array on every GET response for entities that opt into completeness tracking, so the frontend can render a "profile completion" status without a second API round-trip and without re-implementing field-readability rules client-side.

The list contains the field names — native and custom — that are (a) declared as required for profile completeness AND (b) currently empty on the record. The RBAC contract is: a caller only ever sees missingFields for scopes they can read, because the existing FieldFilterInterceptor strips entire scope groups (including their missingFields) when the caller lacks READ.

This iteration ships the mechanism only. The curated list of fields is intentionally empty per scope; populating it is a follow-up iteration that depends on the "Grace period" user story (see §10).


2. Non-goals

  • Grace period model. A separate user story owns gracePeriodEndsAt per academic year (default = firstTermStart - 14d, configurable by admin). That feature governs when missingness becomes critical/blocking. This spec does not introduce the grace-period date, an admin endpoint for it, or any blocking semantics tied to it.
  • Severity / criticality flag on missing fields. No critical: boolean, no severity grading. The frontend reads grace-period state separately when that feature lands.
  • Population of the curated list. Empty arrays ship; the curation iteration sources fields from the ClickUp SMS user-stories document.
  • Relation-based completeness rules (e.g. "must have at least one document"). The empty-value rule covers scalars only.
  • E2E tests for completeness behavior. Unit and interceptor coverage suffice while values are empty; E2E lands with the curation iteration.

3. Decisions

# Decision Rationale
D1 "Missing" = field on the curated list whose value is null, undefined, or "". Catches realistic empty-string slip-throughs without overreaching into relations. 0 and false are valid filled values.
D2 Curated list lives in a new src/common/constants/completion-required-fields.ts, parallel to scope-fields.ts. Least invasive; avoids refactoring the existing scope-fields shape consumed by seed and base service.
D3 Custom field definitions with isRequired=true auto-participate. isRequired already means "expected to be filled"; duplicating that signal client-side would be wasted work.
D4 Custom fields are listed by their flat key (bloodType), not namespaced (customFields.bloodType). Custom-field keys cannot collide with native fields — custom_field_definitions is unique by (tenant, entity, field_key), and any clash with a native field is a separate bug class.
D5 Computation happens in BaseTenantedCrudService.toScopedResponse() via a shared pure utility; hand-rolled toScopedResponses call the same utility. The base service has record, raw, and defsByScope already in scope — no second pass, no duplicated DB lookups.
D6 FieldFilterInterceptor runs an assertMissingFieldsPresence drift check (dev-throws, prod-logs) for registered entities only. Mirrors the existing assertScopedShape / assertAggregateShape precedent. Catches hand-rolled-service drift the moment an entity opts in.
D7 Opt-in per entity via presence in COMPLETION_REQUIRED_REGISTRY. Unregistered entities ship byte-for-byte unchanged. Lets us roll out incrementally without a big-bang DTO migration; future entities can opt in one at a time.
D8 missingFields: [] is always emitted (never omitted) for registered entities. Predictable shape for the frontend; no ?: checks in the consumer.
D9 Build-time spec asserts every registered entity-scope DTO declares missingFields: string[]. Runtime drift check catches hand-rolled service bugs; build-time check catches DTO/Swagger drift before merge.

4. Architecture overview

A new declarative registry keyed by entity → scope → field-name list expresses which native fields are required for profile completeness. Custom field definitions with isRequired=true are pulled in at runtime alongside the native list.

Inside BaseTenantedCrudService.toScopedResponse(), after each scope group is built, a shared pure utility produces a string[] of empty fields and attaches it to the group as missingFields. The hand-rolled ReferentsService.toScopedResponse calls the same utility.

FieldFilterInterceptor is extended with a drift-check that asserts each kept scope group of a registered entity carries the key. Entities not in the registry are unaffected.

Response shape

{
  "id": "uuid",
  "anagraphic": {
    "firstName": "Marco",
    "lastName": "Rossi",
    "dateOfBirth": null,
    "customFields": { "nickname": null },
    "missingFields": []
  },
  "sensitive": { "...": "...", "missingFields": [] },
  "createdAt": "...",
  "updatedAt": "..."
}

missingFields: [] is emitted for every visible scope group of a registered entity. Once the curation iteration populates the registry, the array contains the field names whose values fail D1.


5. Components

5.1 src/common/constants/completion-required-fields.ts (new)

export const STUDENT_COMPLETION_REQUIRED = {
  anagraphic: [] as readonly string[],
  contacts: [] as readonly string[],
  enrollment: [] as readonly string[],
  sensitive: [] as readonly string[],
  documents: [] as readonly string[],
} as const;
// + TEACHER_, STAFF_, REFERENT_, DEPARTMENT_, GRADE_, ROOM_ counterparts.

export const COMPLETION_REQUIRED_REGISTRY: Record<
  string,
  Record<string, readonly string[]>
> = {
  [EntityKey.STUDENTS]: STUDENT_COMPLETION_REQUIRED,
  [EntityKey.TEACHERS]: TEACHER_COMPLETION_REQUIRED,
  [EntityKey.STAFF]: STAFF_COMPLETION_REQUIRED,
  [EntityKey.REFERENTS]: REFERENT_COMPLETION_REQUIRED,
  [EntityKey.DEPARTMENTS]: DEPARTMENT_COMPLETION_REQUIRED,
  [EntityKey.GRADES]: GRADE_COMPLETION_REQUIRED,
  [EntityKey.ROOMS]: ROOM_COMPLETION_REQUIRED,
};

Presence in COMPLETION_REQUIRED_REGISTRY is the opt-in switch for both the runtime computation and the drift check. All arrays ship empty.

5.2 src/common/utils/compute-missing-fields.ts (new)

Pure function, no Nest/Prisma deps:

export function computeMissingFields(
  record: Record<string, unknown>,
  customFieldsRaw: Record<string, unknown>,
  entityKey: string,
  scopeKey: string,
  defsByScope: DefinitionsByScope,
): string[];
  • Returns [] if entityKey is not in COMPLETION_REQUIRED_REGISTRY.
  • Native: from COMPLETION_REQUIRED_REGISTRY[entityKey][scopeKey], keep names whose value on record is null, undefined, or "".
  • Custom: from defsByScope[scopeKey], keep field_keys where isRequired === true and customFieldsRaw[field_key] is null, undefined, or "".
  • Order: native first (registry order), then custom (defsByScope order). Stable.
  • Note: the registry-not-present fast path returns []. Callers (the base service, ReferentsService) decide whether to emit the key based on registry presence — see §5.3.

5.3 BaseTenantedCrudService.toScopedResponse modification

Inside the existing for (const [scopeKey, mapping] of Object.entries(this.scopeFieldMappings)) loop, after nativeFields and the custom-fields block are merged, attach missingFields only when this.entityKey is in COMPLETION_REQUIRED_REGISTRY. This keeps unregistered entities byte-for-byte identical to today. mergeDynamicScopes (custom-fields-only scopes) gets the same conditional treatment.

5.4 ReferentsService.toScopedResponse modification

Same conditional call to the utility, manually wired since this service does not extend the base. Required because REFERENTS is registered.

5.5 FieldFilterInterceptor drift check

A new private assertMissingFieldsPresence(filtered, entity, handlerLabel) runs after filterScopes. For entities in COMPLETION_REQUIRED_REGISTRY only, it walks the kept scope groups and asserts each one has a missingFields key. Dev-throws / prod-logs, mirroring assertScopedShape. Platform-admin path runs the assertion against the unfiltered response so DTO drift surfaces in dev for admin requests too.

5.6 Response DTO updates

The missingFields property is added to *ResponseDto classes only — never to Create*Dto or Update*Dto (which derive from Create via PartialType). It's a server-computed, response-only field; keeping it off write DTOs means inbound payloads carrying missingFields are rejected by the existing ValidationPipe (forbidNonWhitelisted) without any new code.

Each scope sub-DTO under dto/scopes/<scope>.dto.ts for registered entities gains:

@ApiProperty({ type: [String], description: 'Field names empty and required for profile completeness.' })
missingFields!: string[];

Mechanical change across approximately 22 files (7 registered entities × scope counts: students/teachers/staff = 5 each, referents = 4, departments/grades/rooms = 1 each). Each diff is a single property addition on the scope's *ResponseDto.

5.7 ALWAYS_ALLOWED_RESPONSE_FIELDS — no change

missingFields lives inside scope groups, not at the response root. The interceptor's existing logic passes nested objects through unchanged once a scope is kept.


6. Data flow

Single GET (e.g. GET /students/:id):

  1. Controller invokes StudentsService.findOne(id, tenantId).
  2. Base service queries Prisma (via students.queries.ts includes), then loads defsByScope from CustomFieldsService.
  3. toScopedResponse(record, defsByScope):
  4. Per scope mapping: builds nativeFields, picks customFields, conditionally calls computeMissingFields(...), assembles { ...nativeFields, ...customFields, missingFields }.
  5. mergeDynamicScopes does the same for custom-fields-only scopes.
  6. FieldFilterInterceptor strips scope groups the caller cannot read; for each kept group, assertMissingFieldsPresence confirms missingFields is present (registered entities only).
  7. Final JSON ships. Each visible scope group carries missingFields: [].

Paginated list (GET /students): - Each item in data already passed through toScopedResponse. The interceptor's existing per-item filterScopes runs the new check per item.

Platform admin: - Filtering bypassed, but the base service still attaches missingFields. The drift check runs against the unfiltered response.


7. Error handling

missingFields is purely additive and computed from already-fetched data; the runtime path introduces no new failure modes. The error surface is the two drift checks.

Source Trigger Dev Prod
Runtime drift (FieldFilterInterceptor) A registered entity returns a kept scope group missing the missingFields key Throws with handler / entity / scope name Logs error-level entry, returns response unchanged
Build-time DTO check (new spec) A scope sub-DTO of a registered entity does not declare missingFields: string[] Spec fails in CI N/A (caught before merge)

Out of scope (intentional non-errors): - missingFields: [] is not an error — it indicates a complete scope group. - A field on the curated list that doesn't exist on the record reads as undefined and counts as missing per D1. With an empty registry this cannot happen yet; when curation lands, typos surface as permanently-missing names and are caught during that iteration's review. - Custom fields with isRequired=true on scopes the caller cannot read are stripped by FieldFilterInterceptor together with their scope group. The user never sees a missing-field name from a scope they're unauthorized to read.

No new ErrorCode values, no AppException throws.


8. Testing

Suite File Coverage
Unit (utility) compute-missing-fields.spec.ts (new) Empty for unregistered entities; empty-list scaffold base case; native fields with null / undefined / "" are listed; 0 and false are not; isRequired=true custom fields with empty values are listed; isRequired=false custom fields are not; native and custom in one array, no namespacing; stable ordering (native registry order → custom defs order).
Unit (base service) base-tenanted-crud.service.spec.ts (extend) Registered test entity emits missingFields: [] on every scope group of findOne and findAll; unregistered test entity is byte-for-byte identical to today; dynamic scopes via mergeDynamicScopes carry the key when registered.
Unit (hand-rolled) referents.service.spec.ts (extend) toScopedResponse produces missingFields: [] on every scope group.
Unit (interceptor) field-filter.interceptor.spec.ts (extend) Registered entity + missing key → throws in dev / logs in prod; unregistered entity → no-op even when key absent; stripping a scope group also strips its missingFields.
Build-time DTO completion-required-fields.spec.ts (new) For every entity in COMPLETION_REQUIRED_REGISTRY and every scope key, the matching scope sub-DTO class declares missingFields: string[] (reflect-metadata design:type === Array) and carries the @ApiProperty({ type: [String] }) Swagger metadata.

E2E tests are deferred to the curation iteration when missingFields carries non-empty values.


9. Open questions

None blocking. Surface for review during implementation:

  • Caching of COMPLETION_REQUIRED_REGISTRY lookups — the registry is a flat object accessed once per scope group. No memoization needed at scaffold scale; revisit only if profiling shows hot-path cost during curation.
  • Order of native vs custom in the array — committed to "native first, custom second". If the frontend prefers alphabetical or category-grouped order, change is local to computeMissingFields.

10. Dependencies and follow-ups

Depends on (deferred to a future iteration): - Curated list iteration — sources field lists from the ClickUp SMS user-stories document. Blueprint locked in §11 (data-model alignment, engine extensions, target registry, rollout phases). No code changes outside completion-required-fields.ts and the artifacts §11 names. - Grace-period user story (column + endpoint shipped 2026-04-27)AcademicYear.gracePeriodEnding column, default = earliest Period.startDate of type TERM minus 14 days, mutable via PATCH /academic-years/:id. Blocking semantics (when missingness becomes critical) still TBD.

Follow-ups expected after curation: - E2E tests for representative scenarios per entity. - Frontend integration: profile-completion badge consuming missingFields per scope. - Severity/blocking integration once US-23 blocking semantics ship.


11. Data-model alignment for curation (US-13 / US-17 / US-18 / US-19 / US-23 / US-32)

Source: ClickUp doc "Grace Period — Required Fields by Entity" (page 8cnxf2d-5875 in doc 8cnxf2d-7815). User stories are authoritative for required-field content; this section codifies the schema and engine deltas needed before the curation iteration can populate COMPLETION_REQUIRED_REGISTRY faithfully.

11.1 Precedence rule

User stories win for content. One exception: AcademicYear.gracePeriodEnding ships with default = earliest Period.startDate of type TERM minus 14 days, overriding the doc's "1 week before Academic Year start" wording (US-23 Scenario 1). The shipped default and PATCH /academic-years/:id (2026-04-27) are the canonical specification of this date. Both magnitude (14d, not 7d) and anchor (first TERM start, not academic-year start) are explicit engineering choices.

11.2 Schema deltas — implementable now

Isolated nullable additions on existing tables. Single migration, safe ADD COLUMNs, no backfill.

Entity Field Type Scope Source US Notes
Department studentPlatformAccess Boolean @default(false) configuration US-4 S6 Gates conditional Student.schoolEmail requirement (§11.4).
Teacher extraHours Int? employment US-19 Hours beyond totalHoursPerWeek.
Teacher daysOn String? employment US-19 CSV of Weekday enum names — days the teacher is available. Format "MON,TUE,WED". Validated DTO-side via @IsIn over comma-split tokens. The Weekday enum (MON, TUE, WED, THU, FRI, SAT, SUN) is a TypeScript enum (e.g. src/common/constants/weekday.ts) — Prisma stores the raw string; no enum Weekday in schema.prisma.
Staff extraHours Int? employment US-19
Staff iban String? @db.VarChar(34) employment US-19 Mirrors Teacher.iban shape.
Student passportFileId String? documents US-13 File-storage mock — placeholder string column. The real Files model (S3 storage, lifecycle, signed URLs, virus scanning) lands in a later spec; until then this column accepts an opaque string written by an interim upload mechanism (TBD). Naming aligns with the eventual FK target.
Student identityCardFileId String? documents US-13 Same as above.
Teacher passportFileId String? documents US-19 Same. (Doc text says passport_file without _id — normalized for consistency.)
Teacher identityCardFileId String? documents US-19 Same.
Staff passportFileId String? documents US-19 Same.
Staff identityCardFileId String? documents US-19 Same.
Referent passportFileId String? documents US-17 Same.
Referent identityCardFileId String? documents US-17 Same.

Each addition cascades to: prisma/schema.prisma, src/common/constants/scope-fields.ts, the entity's scope sub-DTOs (Create*/Response*), BaseTenantedCrudService scope-field mappings, RBAC seed (prisma/seed/rbac-catalogue.ts FIELD_MAPPINGS if the scope's field list is materialized in seed).

File-mock invariant. The string columns are an interim shape; when the Files spec lands, they will be retyped to String? @db.Uuid and an FK constraint added. Treat them as opaque tokens in code; do not derive structure from the value or write callers that assume specific formats.

11.3 Schema deltas — require a prerequisite spec

Each item below is too cross-cutting to fold into the missingFields curation iteration. Each becomes its own design doc under docs/superpowers/specs/.

Topic Affected entities Why a separate spec
Guardian model Guardian (currently absent) US-18 defines a separate Guardian record — Guardian is not a Referent; the two are distinct domain concepts (confirmed 2026-04-27). Today the schema has only Referent + StudentReferentLink; Guardian needs its own model with its own document requirements. Spec to write.
Files model (deferred — non-blocking) All entities with file FKs Real S3-backed storage with lifecycle, signed URLs, virus scanning, MIME/size limits. The *FileId columns from §11.2 are interim string mocks until this spec lands; they will be retyped to String? @db.Uuid and gain an FK constraint at that point. Curation work is not blocked on this spec.

11.3.1 Out-of-scope — Teacher Academic Assignments

US-19 lists departments, grades, and subjects as Teacher completeness-required fields ("Admin only (R)"). These are fully deferred from the missingFields curation — not just "needs a prereq spec," but out of scope until product re-prioritizes. Implications:

  • No prereq specs are drafted for TeacherSubject or TeacherGradeAssignment.
  • The TEACHER_COMPLETION_REQUIRED.employment registry omits departments, grades, and subjects entirely.
  • The TeacherDepartment join table that already exists is left as-is; no per-grade granularity work happens.
  • The product story for Teacher Academic Assignment completeness remains in US-19 / the ClickUp doc but is unenforced by missingFields until it's re-scoped.

If product re-asks: lift this section, add the prereq specs back to §11.3, retag the §11.5 entries from [deferred] to [grades] / [subjects], and put Phase B back on its prior shape.

11.4 Engine extensions for computeMissingFields

The current utility (D1 in §3) treats every listed field as an unconditional empty-check. Three new shapes are needed for faithful curation:

Capability Use case (from doc) Suggested registry shape
XOR groups Documents: "passport set OR identity_card set" satisfies completeness (Student / Teacher / Staff / Referent / Guardian) List entries can be string (current shape) or { anyOf: string[][] } — group is satisfied if any inner array's fields are all non-empty. If none satisfied, all fields of the first group go into missingFields (deterministic — frontend prompts the canonical path).
Conditional rules Student schoolEmail required only when the linked Department has studentPlatformAccess === true (US-4 S6) Entry shape { field: string, requireWhen: (record, ctx) => boolean } evaluated against the record + a small context bag containing already-joined parents (Department for Student). Context loading is the service's responsibility — utility stays pure.
Cross-entity (per-link) propagation Student record completeness includes its Referent's cellPhone (R (Referent)); Referent record completeness includes per-link relationshipType The Student referents scope's missingFields enumerates per-link gaps as referents[i].cellPhone; same shape on the Referent students scope. Registry shape { perLink: string[] } on the relational scope. Computation reads the joined link rows.

All three are additive on top of D1 — entries that are plain strings keep current semantics. Migration of existing tests is mechanical.

11.5 Target curated registry (post-deltas)

The shape after §11.2–§11.4 land. Field names use camelCase Prisma names. Items gated on a prerequisite spec are tagged [subjects], [grades], [guardian] — file FKs are not tagged because §11.2 ships them as string mocks; the eventual Files retypes are transparent to the registry.

STUDENT_COMPLETION_REQUIRED:
  anagraphic:  ['nationality']
  contacts:    [{ field: 'schoolEmail',
                  requireWhen: 'department.studentPlatformAccess === true' }]
  enrollment:  ['enrollmentDate', 'gradeId']
  sensitive:   ['dietType']                      // already defaults to STANDARD; missing only if explicitly cleared
  documents:   [{ anyOf: [
                   ['passportNumber', 'passportExpiryDate', 'passportFileId'],
                   ['identityCardNumber', 'identityCardExpiryDate', 'identityCardFileId'],
                 ]}]
  referents:   [{ perLink: ['cellPhone'] }]      // cross-entity from Referent

TEACHER_COMPLETION_REQUIRED:
  anagraphic:  ['gender', 'nationality']
  contacts:    ['cellPhone', 'personalEmail',
                'emergencyContactFullname', 'emergencyContactPhone']
  documents:   [{ anyOf: [
                   ['passportNumber', 'passportExpiryDate', 'passportFileId'],
                   ['identityCardNumber', 'identityCardExpiryDate', 'identityCardFileId'],
                 ]}]
  employment:  ['totalHoursPerWeek', 'extraHours', 'iban', 'daysOn']
                // 'departments' / 'grades' / 'subjects' [deferred] — see §11.3.1
  sensitive:   []

STAFF_COMPLETION_REQUIRED:
  anagraphic:  ['gender', 'placeOfBirth', 'nationality']
  contacts:    ['cellPhone', 'personalEmail',
                'emergencyContactFullname', 'emergencyContactPhone']
  documents:   [{ anyOf: [
                   ['passportNumber', 'passportExpiryDate', 'passportFileId'],
                   ['identityCardNumber', 'identityCardExpiryDate', 'identityCardFileId'],
                 ]}]
  employment:  ['totalHoursPerWeek', 'extraHours', 'iban']
  sensitive:   []

REFERENT_COMPLETION_REQUIRED:
  anagraphic:  ['firstName', 'lastName', 'dateOfBirth']
  contacts:    ['cellPhone']
  documents:   [{ anyOf: [
                   ['passportFileId'],                  // US-17 only requires file, not number+expiry
                   ['identityCardFileId'],
                 ]}]
  sensitive:   []
  students:    [{ perLink: ['relationshipType'] }]      // join-table required-per-link

GUARDIAN_COMPLETION_REQUIRED:                          // [guardian] — registry entry lands when the Guardian model spec ships
  // anagraphic: name + surname + dateOfBirth (R-creation, so always present at completeness check)
  // documents:  [{ anyOf: [['passportFileId'], ['identityCardFileId']] }]

DEPARTMENT_/GRADE_/ROOM_COMPLETION_REQUIRED:           // unchanged — doc lists no rules

11.6 Rollout phases

Phase Status Scope Outcome
A — schema lift ✅ shipped 2026-04-27 All §11.2 deltas in one Prisma migration: Department.studentPlatformAccess, Teacher.{extraHours, daysOn, passportFileId, identityCardFileId}, Staff.{extraHours, iban, passportFileId, identityCardFileId}, Student.{passportFileId, identityCardFileId}, Referent.{passportFileId, identityCardFileId}. DTOs, scope-fields.ts, RBAC seed FIELD_MAPPINGS, and BaseTenantedCrudService mappings updated. New TS Weekday enum + DTO validator for daysOn. All native columns and DTO surfaces aligned with US-13 / US-17 / US-19 required-field expectations.
C — engine extensions ✅ shipped 2026-04-27 Type-discriminated MissingFieldRule union added to computeMissingFields (string / { anyOf } / { field, requireWhen } / { perLink, itemsKey?, itemPath? }). COMPLETION_REQUIRED_REGISTRY retyped to Record<string, Record<string, readonly MissingFieldRule[]>>. Plain-string semantics preserved. Student include now selects Department.studentPlatformAccess for the conditional predicate, plus referent passportFileId / identityCardFileId for the perLink rule. All §11.5 rule shapes expressible. Curation lifted from "unconditional-string subset" to the full §11.5 target registry minus the prereq-spec entries.
D — full curation 🟡 backend complete; product-side pending §11.5 registry populated for Student / Teacher / Staff / Referent (Guardian deferred to Phase B). documents uses anyOf, Student.contacts uses requireWhen, Student.referents and Referent.students use perLink. E2E coverage shipped in test/missing-fields.e2e-spec.ts (plain-string + anyOf + requireWhen + perLink). missingFields ships with real values for the four registered entities. Pending outside this repo: frontend wiring (profile-completion badge consumes missingFields); severity/blocking semantics once the US-23 grace-period blocking story lands.
B — prerequisite specs 📝 drafted 2026-04-27 — implementation pending Two design docs landed: 2026-04-27-guardian-model-design.md (blocking) and 2026-04-27-files-model-design.md (non-blocking, retypes *FileId from String? to String? @db.Uuid + FK constraint). Teacher Academic Assignment specs are out of scope per §11.3.1. Guardian spec unblocks GUARDIAN_COMPLETION_REQUIRED registration once implemented. Files spec retypes the eight interim string columns once implemented. Both implementations are sequenced after Phase B sign-off.

Phases A and C shipped concurrently in one branch with the curation populated alongside. Phase B specs are drafted and awaiting implementation sign-off. Once Guardian lands, Phase D wraps with E2E tests + frontend wiring.