Integrating with Systems That Have No API

Older and specialist systems often expose no supported HTTP interface. They may exchange scheduled files, permit read access to reporting tables, accept records through an import directory or offer only a user interface. The dangerous response is to treat any technical opening as an integration contract. Direct writes can violate hidden invariants, screen automation can break after a label change and files can be replayed without anyone noticing. A sound design identifies what the vendor supports, who owns the data, how transactions are acknowledged and how both sides reconcile. The aim is not to imitate a modern API at any cost, but to build a controlled boundary around the capabilities that genuinely exist.

Rank interfaces by support and semantics

Start with vendor-supported batch imports, exports, message queues, command-line tools or extension points. These often enforce business validation that raw database access bypasses. Read-only database integration can be appropriate for reporting or change capture when schemas and isolation effects are understood. Writes to application tables should be avoided unless explicitly supported, because triggers, derived values, caches and audit records may be missed. UI automation is usually the last resort for low-volume operations where no safer route exists. Document the chosen interface’s transaction semantics, limits, schedule, encoding, versioning and support position. “It worked in a test copy” is not a contract for production behaviour.

Put an anti-corruption layer at the boundary

Legacy codes and record shapes should not leak throughout the new estate. A boundary service can translate the external representation into a canonical command or event, validate required meaning and retain the original payload for controlled troubleshooting. Keep mappings versioned and explicit, including defaults and rejected values. The layer should know which system owns each field and avoid sending stale data back over authoritative changes. It also isolates consumers from fixed-width files, unusual date formats and schema drift. This is not a licence to invent a universal enterprise model; canonical forms should be scoped to the integration domain and preserve identifiers needed for lineage and reconciliation.

Design idempotency before retries

Networks fail, scheduled jobs overlap and operators resend files. Every consumer must assume a message or row can arrive more than once. Use a stable source transaction identifier, not a randomly generated identifier per attempt, and record processing outcome atomically with the business change where possible. File-level checksums can detect exact replays, while row-level keys handle corrected or partially repeated files. Define whether an update replaces, reverses or amends the earlier transaction. A retry should be safe after an uncertain timeout. If the target cannot provide atomicity, use an inbox or staging ledger and reconciliation to manage ambiguity rather than pretending exactly-once delivery exists across independent systems.

• Supported import or export facility with documented validation.
• Read-only database view or change feed with controlled resource use.
• Managed file exchange with atomic delivery, schema and acknowledgements.
• UI automation only when volume and change risk make it supportable.
• Manual controlled hand-off when automation risk exceeds its value.

Files need protocol, not just a folder

Define naming, sequence, schema version, character encoding, delimiters, time zone and trailer totals. Producers should write to a temporary name and rename atomically when complete, or use a separate manifest, so consumers never read a partial file. Encrypt transport and storage as required, restrict service accounts and avoid sensitive data in filenames. Move processed and rejected files to retained locations with checksums and timestamps. An acknowledgement should distinguish accepted delivery from successful business posting. For large batches, row-level error files allow valid work to proceed only if the business rules permit partial acceptance; otherwise reject the batch consistently. Monitor missing expected files as well as failed ones.

Make reconciliation a first-class flow

Technical success does not prove both systems represent the same business state. Compare sent, received, accepted, rejected and posted counts and values by batch and accounting period. Preserve source and target identifiers so unmatched items can be traced. For mutable master data, periodic snapshots or hashes can identify drift that event processing missed. Assign ownership for breaks and provide a replay mechanism that cannot bypass approval or duplicate settled work. Dashboards should age unresolved differences and show the last successful exchange. Reconciliation is especially important where acknowledgement is weak, a database poll can miss updates, or screen automation cannot distinguish a committed action from a timed-out display.

Contain change and operational risk

Contract-test representative files, views and screens, and alert on unexpected schema or layout changes before corrupting data. Throttle database reads and automation so integration does not degrade the source system. Secrets belong in managed stores, service accounts need least privilege and personal data should be minimised in logs and retained payloads. Define support ownership across organisational boundaries: who investigates, who can replay and who decides that a mapping has changed. Planned vendor upgrades should trigger compatibility testing. Where UI automation is unavoidable, use stable selectors when available, capture evidence carefully and stop on ambiguity rather than clicking through an unknown state.

Polling and change capture need precise watermarks

When the only option is periodic database polling, a timestamp greater than the last observed value is often unsafe. Clocks can differ, several rows can share a timestamp, and transactions can commit after a later-timestamped row has already advanced the watermark. Prefer a monotonically increasing sequence or supported change-data-capture mechanism. If only timestamps exist, use an overlap window and deduplicate by stable record and version identifiers. Persist the watermark only after the corresponding batch is durably staged, then separate extraction progress from downstream processing progress. Deletes require tombstones, an audit table or periodic full comparison because a missing row cannot be found by querying updated records. Database isolation and source load must be agreed with its operator. Long scans may block application work or bloat version storage, while dirty reads can publish states that never commit. Fetch bounded pages in a deterministic order and record query duration, lag and row counts. Schema changes should fail visibly at validation rather than shift columns silently. For records updated more than once between polls, decide whether consumers need every transition or only the latest state; an audit or log source is required for the former. A poller is a small replication system, and deserves checkpoints, replay, backpressure and reconciliation rather than an unattended timer around an ad hoc query.

Polling deserves the discipline of a replication system. A timestamp watermark alone can miss rows when clocks differ, values tie or transactions commit out of order. Prefer a supported change feed or monotonic sequence; otherwise overlap polling windows and deduplicate using stable record and version identifiers. Persist progress only after a batch is durably staged, and track extraction separately from downstream processing. Deletes need tombstones, audit data or periodic comparison. Use bounded, deterministic pages under an agreed isolation level so the poller does not damage source performance. Monitor source lag as well as job success, and run backfills through the same validation and idempotency path as ordinary changes.

Choose the least fragile truthful design

Inventory supported mechanisms and test their semantics with the system owner. Prefer a vendor-enforced import over table writes, and a structured file over screen automation. Wrap the choice in validation, canonical translation, stable identifiers, idempotent processing and observable queues. Define reconciliation and recovery before building the happy path, then test duplicates, partial files, timeouts, late arrivals and upgrades. Sometimes a controlled manual exchange is more responsible than brittle automation. The correct architecture acknowledges the limitations of the legacy system and compensates with explicit controls, rather than hiding those limitations behind an API-shaped façade.