Syncable Entity Cache And Transform

Syncable Entity: Cache & Transform (Step 2/6)

Purpose: Create cache layer and transformation utilities to convert between different entity representations.

When to use: After completing Step 1 (Types & Constants). Required before building validators and action handlers.

Quick Start

This step creates:

1. Cache service for flat entity maps
2. Entity-to-flat conversion utility
3. Input transform utils (DTO → Universal Flat Entity)

Key principle: Input transform utils must output universal flat entities (with universalIdentifier and foreign keys mapped to universal identifiers).

Step 1: Create Cache Service

File: src/engine/metadata-modules/flat-my-entity/services/flat-my-entity-cache.service.ts

import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { v4 } from 'uuid';

import { WorkspaceCache } from 'src/engine/twenty-orm/decorators/workspace-cache.decorator';
import { MyEntityEntity } from 'src/engine/metadata-modules/my-entity/entities/my-entity.entity';
import { type FlatMyEntityMaps } from 'src/engine/metadata-modules/flat-my-entity/types/flat-my-entity-maps.type';
import { fromMyEntityEntityToFlatMyEntity } from 'src/engine/metadata-modules/flat-my-entity/utils/from-my-entity-entity-to-flat-my-entity.util';

@Injectable()
export class FlatMyEntityCacheService {
  constructor(
    @InjectRepository(MyEntityEntity, 'metadata')
    private readonly myEntityRepository: Repository<MyEntityEntity>,
  ) {}

  @WorkspaceCache({ flatMapsKey: 'flatMyEntityMaps' })
  async getFlatMyEntityMaps(): Promise<FlatMyEntityMaps> {
    const myEntities = await this.myEntityRepository.find({
      withDeleted: true, // CRITICAL: Include soft-deleted entities
    });

    const flatMyEntities = myEntities.map((entity) =>
      fromMyEntityEntityToFlatMyEntity(entity),
    );

    return {
      byId: Object.fromEntries(flatMyEntities.map((e) => [e.id, e])),
      byName: Object.fromEntries(flatMyEntities.map((e) => [e.name, e])),
    };
  }
}

Critical rules:

• Use @WorkspaceCache decorator with unique flatMapsKey
• Always use withDeleted: true to include soft-deleted entities
• Cache key pattern: flat{EntityName}Maps (camelCase)

Step 2: Entity-to-Flat Conversion

File: src/engine/metadata-modules/flat-my-entity/utils/from-my-entity-entity-to-flat-my-entity.util.ts

import { v4 } from 'uuid';
import { type MyEntityEntity } from 'src/engine/metadata-modules/my-entity/entities/my-entity.entity';
import { type FlatMyEntity } from 'src/engine/metadata-modules/flat-my-entity/types/flat-my-entity.type';

export const fromMyEntityEntityToFlatMyEntity = (
  entity: MyEntityEntity,
): FlatMyEntity => {
  return {
    id: entity.id,
    // Critical: generate a new UUID for universalIdentifier
    universalIdentifier: v4(),
    workspaceId: entity.workspaceId,
    applicationId: entity.applicationId,
    name: entity.name,
    label: entity.label,
    description: entity.description,
    isCustom: entity.isCustom,
    parentEntityId: entity.parentEntityId,
    settings: entity.settings,
    createdAt: entity.createdAt.toISOString(),
    updatedAt: entity.updatedAt.toISOString(),
    deletedAt: entity.deletedAt?.toISOString() ?? null,
  };
};

Critical: universalIdentifier must be a new UUID generated with v4() (not entity.id)

Step 3: Input Transform Utils (DTO → Universal Flat Entity)

File: src/engine/metadata-modules/flat-my-entity/utils/from-create-my-entity-input-to-universal-flat-my-entity.util.ts

import { v4 } from 'uuid';
import { sanitizeString } from 'twenty-shared/string';
import { type CreateMyEntityInput } from 'src/engine/metadata-modules/my-entity/dtos/create-my-entity.input';
import { type UniversalFlatMyEntity } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-my-entity.type';
import { resolveEntityRelationUniversalIdentifiers } from 'src/engine/metadata-modules/flat-entity/utils/resolve-entity-relation-universal-identifiers.util';
import { type AllFlatEntityMapsByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps-by-metadata-name.type';

export const fromCreateMyEntityInputToUniversalFlatMyEntity = ({
  input,
  workspaceId,
  flatEntityMaps,
}: {
  input: CreateMyEntityInput;
  workspaceId: string;
  flatEntityMaps?: AllFlatEntityMapsByMetadataName;
}): UniversalFlatMyEntity => {
  const id = v4();
  const universalIdentifier = v4();

  // 1. Extract foreign key IDs BEFORE sanitization
  const parentEntityId = input.parentEntityId ?? null;

  // 2. Sanitize string properties
  const name = sanitizeString(input.name);
  const label = sanitizeString(input.label);
  const description = input.description ? sanitizeString(input.description) : null;

  // 3. Build base flat entity
  const baseFlatEntity = {
    id,
    universalIdentifier,
    workspaceId,
    applicationId: null,
    name,
    label,
    description,
    isCustom: true,
    parentEntityId,
    settings: input.settings ?? null,
    createdAt: new Date().toISOString(),
    updatedAt: new Date().toISOString(),
    deletedAt: null,
  };

  // 4. Resolve foreign keys to universal identifiers (if flatEntityMaps provided)
  if (flatEntityMaps) {
    return resolveEntityRelationUniversalIdentifiers({
      metadataName: 'myEntity',
      flatEntity: baseFlatEntity,
      flatEntityMaps,
    });
  }

  // 5. Return with null universal foreign keys if no maps
  return {
    ...baseFlatEntity,
    parentEntityUniversalIdentifier: null,
  };
};

Key steps:

1. Generate IDs (id and universalIdentifier with v4())
2. Extract foreign keys before sanitization
3. Sanitize all string properties
4. Build base flat entity
5. Resolve foreign keys → universal identifiers

Step 4: Create Flat Entity Module

File: src/engine/metadata-modules/flat-my-entity/flat-my-entity.module.ts

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';

import { MyEntityEntity } from 'src/engine/metadata-modules/my-entity/entities/my-entity.entity';
import { FlatMyEntityCacheService } from 'src/engine/metadata-modules/flat-my-entity/services/flat-my-entity-cache.service';

@Module({
  imports: [TypeOrmModule.forFeature([MyEntityEntity], 'metadata')],
  providers: [FlatMyEntityCacheService],
  exports: [FlatMyEntityCacheService],
})
export class FlatMyEntityModule {}

Rules:

• Import entity with 'metadata' datasource
• Export cache service for use in other modules

Common Patterns

Pattern: Foreign Key Resolution

// Extract foreign keys BEFORE sanitization
const parentEntityId = input.parentEntityId ?? null;

// After building base entity, resolve to universal identifiers
const universalFlatEntity = resolveEntityRelationUniversalIdentifiers({
  metadataName: 'myEntity',
  flatEntity: baseFlatEntity,
  flatEntityMaps,
});

Pattern: JSONB with SerializedRelation

// For JSONB properties containing foreign keys
const settings = input.settings
  ? {
      ...input.settings,
      fieldMetadataId: input.settings.fieldMetadataId,
    }
  : null;

// After resolution, JSONB foreign keys become universal identifiers
return resolveEntityRelationUniversalIdentifiers({
  metadataName: 'myEntity',
  flatEntity: { ...baseFlatEntity, settings },
  flatEntityMaps,
});

Pattern: Update Transform

// from-update-my-entity-input-to-universal-flat-my-entity-updates.util.ts
export const fromUpdateMyEntityInputToUniversalFlatMyEntityUpdates = ({
  input,
  flatEntityMaps,
}: {
  input: UpdateMyEntityInput;
  flatEntityMaps?: AllFlatEntityMapsByMetadataName;
}): Partial<UniversalFlatMyEntity> => {
  const updates: Partial<UniversalFlatMyEntity> = {};

  if (input.name !== undefined) {
    updates.name = sanitizeString(input.name);
  }

  if (input.parentEntityId !== undefined) {
    updates.parentEntityId = input.parentEntityId;
  }

  updates.updatedAt = new Date().toISOString();

  // Resolve foreign keys if maps provided
  if (flatEntityMaps) {
    return resolveEntityRelationUniversalIdentifiers({
      metadataName: 'myEntity',
      flatEntity: updates as any,
      flatEntityMaps,
    });
  }

  return updates;
};

Checklist

Before moving to Step 3:

• Cache service created with @WorkspaceCache decorator
• Cache uses withDeleted: true
• Cache key follows flat{EntityName}Maps pattern
• Entity-to-flat conversion implemented
• universalIdentifier set correctly (generated with v4())
• Create input transform implemented
• Update input transform implemented (if needed)
• Foreign keys extracted before sanitization
• String properties sanitized
• Foreign keys resolved to universal identifiers
• Flat entity module created and exports cache service

Next Step

Once cache and transform utilities are complete, proceed to: Syncable Entity: Builder & Validation (Step 3/6)

For complete workflow, see @creating-syncable-entity rule.