tinylink/docs/user_manual.md

TinyLink User Manual

Welcome to TinyLink: the friendly lab middleware that never sleeps, rarely panics, and keeps your results moving even when downstream systems are having a bad day.

What TinyLink Can Do

TinyLink sits between laboratory instruments and CLQMS, then handles the heavy lifting:

• Receives messages from instruments over three connector types:
  • http-json (HTTP endpoint)
  • hl7-tcp (TCP socket)
  • astm-serial (physical serial port/COM)
• Parses incoming payloads and normalizes them into one canonical JSON format.
• Stores raw and normalized data in SQLite for durability.
• Deduplicates repeated payloads using a hash key.
• Sends results to CLQMS with automatic retry and backoff.
• Moves non-deliverable payloads to dead letter for review.
• Exposes operational endpoints for health and metrics.

Think of it as a reliable translator + traffic controller for instrument data.

Default Ports and Endpoints

By default (from middleware/config/app.yaml):

• Instrument config + health + metrics API: 4001 (from host.port)
• Instrument connectors are configured per instrument entity (inst1, inst2, ...)

Useful endpoints:

• GET http://localhost:4001/health
• GET http://localhost:4001/health/ready
• GET http://localhost:4001/metrics
• POST http://localhost:3001/messages (for JSON instrument payloads)

Add a New Instrument (New "Inst")

TinyLink now uses a single file-based configuration. There is no POST /instruments write flow.

To add an instrument, keep one config file in:

middleware/
  config/
    app.yaml

Step 1: Edit app.yaml

Example:

host:
  url: http://localhost:4000/api/results
  apikey: ""
  port: 4001

inst1:
  enabled: true
  connector:
    type: serial
    port: COM1
    baudRate: 9600
  config:
    location: lab-a
  translator:
    parser: astm
    forceInstrumentId: true
    meta:
      profile: astm-default

What it does:

• host contains upstream endpoint and API settings (url, apikey, port).
• Every top-level entity other than host is an instrument (inst1, inst2, ...).
• connector is embedded per instrument, so each instrument has its own connector type/settings.
• translator stays embedded per instrument, so each instrument can define parser and metadata.

Step 2: Restart TinyLink

TinyLink validates instrument files before startup. Restart after adding or editing files. Use this preflight command before npm start:

npm run instrument:check

Step 3: Verify instrument is loaded

You can still use read-only endpoints:

curl http://localhost:4001/instruments
curl http://localhost:4001/instruments/inst1

If your instrument does not appear, run npm run instrument:check and fix reported errors.

Matching rules (important)

TinyLink picks one instrument by matching connector + instruments[].match rules.

• 0 matches: payload is dropped (no matching instrument config).
• More than 1 match: payload is dropped (ambiguous instrument match).
• Exactly 1 match: translator runs and message continues through queue + delivery.

TinyLink is strict because your audit trail deserves peace and quiet.

Canonical Payload Shape

After parsing and normalization, TinyLink expects this shape.

Required fields:

• instrument_id
• sample_id
• result_time (ISO timestamp)
• results[] with at least one item containing:
  • test_code
  • value

Optional fields:

• unit, flag, patient_id, operator_id, meta

Example:

{
  "instrument_id": "SYSMEX_XN1000",
  "sample_id": "SMP-20260326-001",
  "result_time": "2026-03-26T10:20:00Z",
  "results": [
    {
      "test_code": "WBC",
      "value": "8.2",
      "unit": "10^3/uL",
      "flag": "N"
    }
  ],
  "meta": {
    "source_protocol": "HL7",
    "connector": "hl7-tcp"
  }
}

Delivery and Retry Behavior

TinyLink does not give up easily:

• Retries transient failures (timeouts, DNS/connection issues, HTTP 5xx).
• Uses backoff schedule: 30s -> 2m -> 10m -> 30m -> 2h -> 6h.
• Maximum attempts: 10.
• HTTP 400 and 422 are treated as non-retriable and moved to dead letter immediately.

Health, Readiness, and Metrics

Use these to check system status:

• GET /health: connector status + queue counts (pending, retrying, deadLetters).
• GET /health/ready: confirms SQLite readiness.
• GET /metrics: Prometheus-style metrics, including attempts, latency, and success timestamp.

Daily Operations (Quick Checklist)

• Start app: npm start
• Validate instrument files: npm run instrument:check
• Run migrations manually: npm run migrate
• Backup DB: npm run maintenance -- backup
• Vacuum DB: npm run maintenance -- vacuum
• Prune old delivery logs: npm run maintenance -- prune --days=30

Troubleshooting New Instrument Onboarding

Instrument returns 404

• Confirm exact instrument_id spelling and casing.
• Verify the instrument exists in middleware/config/app.yaml as its own top-level key (inst1, inst2, ...).
• Verify that instrument has connector.type, connector settings, and translator.parser.

Data is not flowing

• Confirm instrument has enabled: true.
• Confirm connector matches incoming protocol.
• Confirm match rules fit connector metadata (localPort, remoteAddress, remotePort, or comPort).
• Check /health and /health/ready.

Backlog or dead letters are growing

• Check /metrics for queue and delivery trends.
• Validate CLQMS URL/token/timeouts and downstream availability.

Final Tip

If you are adding many instruments, standardize your naming (for example: LAB1_XN1000, LAB1_COBAS_E411) and keep connector mappings documented. Future-you will send present-you a thank-you card.