Java Data Engineering And Integration Services
Overview
Use this skill when Java is the main implementation language for data-adjacent services or processing components. It helps agents design operationally safe JVM services for ingestion, metadata, contracts, stream handling, connectors, and control-plane style data tooling with deliberate resource, dependency, and concurrency management.
When to Use
- building ingestion or connector services in
Java
- implementing JVM-based stream processors or integration utilities
- exposing data-platform metadata, contract, or control services
- managing
Maven or Gradle builds for data-related services
- debugging resource, thread, serialization, or connection-pool behavior in JVM services
Do not treat Java services as generic app code when they carry data-delivery, contract, or pipeline semantics.
Workflow
Define the service role and operational boundary.
Clarify:
- request or event model
- upstream and downstream systems
- throughput and latency expectations
- delivery guarantees
- retry and failure behavior
Make contracts explicit.
Include:
- payload schemas
- versioning behavior
- idempotency rules
- error model
- compatibility with downstream consumers
Design resource and concurrency behavior deliberately.
Review:
- thread pools
- blocking versus async paths
- connection management
- backpressure
- graceful shutdown and restart behavior
Package and configure for operations.
Decide:
Maven or Gradle conventions- dependency version strategy
- environment configuration
- secrets handling
- observability and health signals
Validate service behavior under realistic load and failure conditions.
Require:
- contract checks
- retry and timeout tests
- connection and resource sanity
- release and rollback readiness
Common Rationalizations
| Rationalization |
Reality |
| "The framework defaults are good enough." |
Defaults for thread pools, connection pools, and retries often fail under data-heavy or bursty workloads. |
| "It is just a connector wrapper." |
Connectors still define contracts, error handling, retries, and downstream correctness. |
| "Java is verbose but safe by default." |
JVM services still fail due to resource leaks, blocking calls, schema drift, and weak operational boundaries. |
Red Flags
- retry behavior can duplicate writes or downstream side effects
- thread, connection, or shutdown behavior is undocumented
- service config and secrets are embedded in code or build files
- payload compatibility or schema evolution is undefined
- observability is limited to basic process health with no data-path evidence
Verification
1---2name: java-data-engineering-and-integration-services3description: Guides agents through Java-based data engineering services and processors. Use when building connectors, ingestion services, stream processors, metadata services, JVM batch tools, or operational integrations in Java.4---56# Java Data Engineering And Integration Services78## Overview910Use this skill when `Java` is the main implementation language for data-adjacent services or processing components. It helps agents design operationally safe JVM services for ingestion, metadata, contracts, stream handling, connectors, and control-plane style data tooling with deliberate resource, dependency, and concurrency management.1112## When to Use1314- building ingestion or connector services in `Java`15- implementing JVM-based stream processors or integration utilities16- exposing data-platform metadata, contract, or control services17- managing `Maven` or `Gradle` builds for data-related services18- debugging resource, thread, serialization, or connection-pool behavior in JVM services1920Do not treat Java services as generic app code when they carry data-delivery, contract, or pipeline semantics.2122## Workflow23241. Define the service role and operational boundary.25 Clarify:26 - request or event model27 - upstream and downstream systems28 - throughput and latency expectations29 - delivery guarantees30 - retry and failure behavior31322. Make contracts explicit.33 Include:34 - payload schemas35 - versioning behavior36 - idempotency rules37 - error model38 - compatibility with downstream consumers39403. Design resource and concurrency behavior deliberately.41 Review:42 - thread pools43 - blocking versus async paths44 - connection management45 - backpressure46 - graceful shutdown and restart behavior47484. Package and configure for operations.49 Decide:50 - `Maven` or `Gradle` conventions51 - dependency version strategy52 - environment configuration53 - secrets handling54 - observability and health signals55565. Validate service behavior under realistic load and failure conditions.57 Require:58 - contract checks59 - retry and timeout tests60 - connection and resource sanity61 - release and rollback readiness6263## Common Rationalizations6465| Rationalization | Reality |66| --- | --- |67| "The framework defaults are good enough." | Defaults for thread pools, connection pools, and retries often fail under data-heavy or bursty workloads. |68| "It is just a connector wrapper." | Connectors still define contracts, error handling, retries, and downstream correctness. |69| "Java is verbose but safe by default." | JVM services still fail due to resource leaks, blocking calls, schema drift, and weak operational boundaries. |7071## Red Flags7273- retry behavior can duplicate writes or downstream side effects74- thread, connection, or shutdown behavior is undocumented75- service config and secrets are embedded in code or build files76- payload compatibility or schema evolution is undefined77- observability is limited to basic process health with no data-path evidence7879## Verification8081- [ ] The Java service role and delivery semantics are explicit82- [ ] Contract, retry, and idempotency behavior are defined83- [ ] Resource, concurrency, and shutdown behavior are reviewed84- [ ] Build, config, and observability expectations are operationally real
vaquarkhan