Cia Data Integration

CIA Data Integration Skill

Purpose

Guide the integration of Swedish political data into the CIA platform's data model, covering entity mapping, data transformation, and consistency rules for Riksdagen, election, and financial data sources.

When to Use

• ✅ Mapping new Riksdagen data types to JPA entities
• ✅ Processing parliamentary voting records
• ✅ Integrating election result data
• ✅ Adding new political entity relationships
• ✅ Maintaining data quality and consistency rules

Do NOT use for:

• ❌ Generic API integration (use api-integration skill)
• ❌ UI/dashboard development (use vaadin-component-design skill)

CIA Data Model Overview

┌──────────────────────────────────────────────────────┐
│                   External Data Sources               │
├──────────────┬────────────┬──────────┬───────────────┤
│  Riksdagen   │  Val.se    │  World   │     ESV       │
│  Open Data   │  Elections │  Bank    │   Finances    │
└──────┬───────┴─────┬──────┴────┬─────┴───────┬───────┘
       │             │           │             │
       ▼             ▼           ▼             ▼
┌──────────────────────────────────────────────────────┐
│              Service Layer (Spring)                   │
│  service.external.riksdagen                          │
│  service.external.val                                │
│  service.external.worldbank                          │
│  service.external.esv                                │
└──────────────────────┬───────────────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────────────┐
│            Internal Data Model (JPA)                  │
│  model.internal.application.data                     │
│    ├── Politicians     ├── Committees                │
│    ├── Votes           ├── Documents                 │
│    ├── Parties         ├── Elections                 │
│    └── Risk Scores     └── Financial Data            │
└──────────────────────────────────────────────────────┘

Key Module Structure

Entity Mapping Patterns

Riksdagen Person to Internal Politician

@Entity
@Table(name = "person_data")
public class PersonData implements Serializable {

    @Id
    @Column(name = "id")
    private String id;              // Riksdagen person ID (intressent_id)

    @Column(name = "first_name")
    private String firstName;       // förnamn

    @Column(name = "last_name")
    private String lastName;        // efternamn

    @Column(name = "party")
    private String party;           // parti

    @Column(name = "born_year")
    private int bornYear;           // född_ar

    @Column(name = "gender")
    private String gender;          // kön

    @Column(name = "status")
    private String status;          // status (Tjänstgörande, etc.)
}

Vote Record Processing

@Entity
@Table(name = "vote_data")
public class VoteData implements Serializable {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;

    @Column(name = "intressent_id")
    private String intressentId;    // Links to PersonData

    @Column(name = "rm")
    private String riksmote;        // Parliamentary session (e.g., "2023/24")

    @Column(name = "beteckning")
    private String designation;     // Vote designation

    @Column(name = "rost")
    private String vote;            // Ja/Nej/Avstår/Frånvarande

    @Column(name = "votering_id")
    private String voteringId;      // Unique voting session ID
}

Data Quality Rules

Validation Constraints

public class DataQualityValidator {

    // Valid Swedish Parliament parties
    private static final Set<String> VALID_PARTIES = Set.of(
        "S", "M", "SD", "C", "V", "KD", "L", "MP", "-"
    );

    // Valid vote values from Riksdagen
    private static final Set<String> VALID_VOTES = Set.of(
        "Ja", "Nej", "Avstår", "Frånvarande"
    );

    public void validatePersonData(PersonData person) {
        Objects.requireNonNull(person.getId(), "Person ID is required");
        if (!VALID_PARTIES.contains(person.getParty())) {
            LOG.warn("Unknown party code: {} for person: {}",
                person.getParty(), person.getId());
        }
    }

    public void validateVoteData(VoteData vote) {
        Objects.requireNonNull(vote.getIntressentId(), "Intressent ID is required");
        if (!VALID_VOTES.contains(vote.getVote())) {
            throw new DataValidationException(
                "Invalid vote value: " + vote.getVote());
        }
    }
}

Data Freshness Requirements

Riksdagen API Integration

API Endpoints

GET /personlista/?iid={id}&utformat=json     → Person details
GET /voteringlista/?rm={session}&utformat=json → Voting list
GET /dokumentlista/?sok={query}&utformat=json  → Document search
GET /dokumentstatus/{doc_id}.json              → Document status

Data Import Workflow

@Service
public class RiksdagDataImportService {

    @Transactional
    public ImportResult importPersonData() {
        // 1. Fetch from Riksdagen API
        PersonLista personList = riksdagClient.getPersonLista();

        // 2. Validate each record
        List<PersonData> validated = personList.getPersons().stream()
            .filter(this::isValid)
            .map(this::mapToInternal)
            .collect(Collectors.toList());

        // 3. Upsert into database
        int updated = personRepository.batchUpsert(validated);

        // 4. Log import metrics
        return new ImportResult(validated.size(), updated);
    }
}

Security Considerations

• Sanitize all imported data — external data is untrusted input
• Use parameterized queries — never concatenate API data into SQL
• Log data lineage — track source and import timestamp for audit
• GDPR compliance — politician data is public, but handle personal data carefully
• Validate XML/JSON — protect against XXE and injection attacks

ISMS Alignment