tinylink/docs/user_manual.md

TinyLink User Manual

Welcome to TinyLink: the friendly lab middleware that never sleeps, rarely panics, and keeps your results moving even when downstream systems are having a bad day.

What TinyLink Can Do

TinyLink sits between laboratory instruments and CLQMS, then handles the heavy lifting:

• Receives messages from instruments over three connector types:
  • http-json (HTTP endpoint)
  • hl7-tcp (TCP socket)
  • astm-serial (physical serial port/COM)
• Wraps incoming payloads as raw content, then normalizes them into one canonical JSON format via the translator pipeline.
• Stores raw and normalized data in SQLite for durability.
• Deduplicates repeated payloads using a hash key.
• Sends results to CLQMS with automatic retry and backoff.
• Moves non-deliverable payloads to dead letter for review.
• Exposes operational endpoints for health and metrics.

Think of it as a reliable translator + traffic controller for instrument data.

Default Ports and Endpoints

By default (from config/app.yaml):

• Instrument config + health + metrics API: 4001 (from host.port)
• Instrument connectors are configured per instrument entity (inst1, inst2, ...)

Useful endpoints:

• GET http://localhost:4001/health
• GET http://localhost:4001/health/ready
• GET http://localhost:4001/metrics
• GET http://localhost:4001/dashboard
• POST http://localhost:3001/messages (for JSON instrument payloads)

Architecture reference:

• docs/separated_architecture.md

Add a New Instrument (New "Inst")

TinyLink now uses a single file-based configuration. There is no POST /instruments write flow.

To add an instrument, keep one config file in:

core/
  config/
    app.yaml

Step 1: Edit app.yaml

Example:

host:
  url: http://localhost:4000/api/results
  apikey: ""
  port: 4001

inst1:
  enabled: true
  connector:
    type: serial
    port: COM1
    baudRate: 9600
  config:
    location: lab-a
  translator:
    engine: overrides
    forceInstrumentId: true
    meta:
      profile: astm-default

Optional: Use per-instrument .map translator files

If you want a shorter per-instrument translator config, set translator to the map name.

Example: translator: inst1 loads config/inst1.map.

config/app.yaml example:

inst1:
  enabled: true
  connector:
    type: serial
    port: COM1
  translator: inst1

config/inst1.map example:

# HEADER
<VT>H|\^&|||WST^P1|||||{instrument_id}^System1||P|1|{sample_id}<CR>

SAMPLE_ID = O[3]
CHEMNUM = R[3]
RESULT = R[4]

# ORDER
O|1|{SAMPLE_ID}||{order_tests}||||||{specimen_type}||||{tube_type}||||||||||O|<CR>

# RESULT
<<
@for R
R|1|^^^{CHEMNUM}|{RESULT}<CR>
@end
>>

# TERMINATOR
L|1|N<FS><CR>

Notes:

• Preferred style is section-based: # MESSAGE_NAME plus one body line.
• Use << and >> for multiline section bodies.
• KEY = value remains supported for backwards compatibility.
• ALIAS = R[3] (or O[11], P[2], OBX[5.2]) defines readable field aliases.
• You can reference selectors directly as placeholders: {R[3]}.
• Control tokens in message bodies are decoded: <VT>, <FS>, <STX>, <ETX>, <CR>, <LF>.
• @for R ... @end repeats a multiline block for each incoming record of type R.
• If records are not pre-parsed, TinyLink auto-extracts records from raw_payload/meta.raw_payload for selector usage.
• Placeholders use {name} and missing values default to empty strings.
• translator.messages controls output order. If omitted, messages render in map file order.

What it does:

• host contains upstream endpoint and API settings (url, apikey, port).
• Every top-level entity other than host is an instrument (inst1, inst2, ...).
• connector is embedded per instrument, so each instrument has its own connector type/settings.
• translator stays embedded per instrument, so each instrument can define mapping rules and metadata.

Step 2: Restart TinyLink

TinyLink validates instrument files before startup. Restart after adding or editing files. Use this preflight command before npm start:

npm run instrument:check

Step 3: Verify instrument is loaded

You can still use read-only endpoints:

curl http://localhost:4001/instruments
curl http://localhost:4001/instruments/inst1

If your instrument does not appear, run npm run instrument:check and fix reported errors.

Matching rules (important)

TinyLink picks one instrument by matching connector + instruments[].match rules.

• 0 matches: payload is dropped (no matching instrument config).
• More than 1 match: payload is dropped (ambiguous instrument match).
• Exactly 1 match: translator runs and message continues through queue + delivery.

TinyLink is strict because your audit trail deserves peace and quiet.

Canonical Payload Shape

After wrapping and normalization, TinyLink expects this shape.

Required fields:

• instrument_id
• sample_id
• result_time (ISO timestamp)
• results[] with at least one item containing:
  • test_code
  • value

Optional fields:

• unit, flag, patient_id, operator_id, meta

Example:

{
  "instrument_id": "SYSMEX_XN1000",
  "sample_id": "SMP-20260326-001",
  "result_time": "2026-03-26T10:20:00Z",
  "results": [
    {
      "test_code": "WBC",
      "value": "8.2",
      "unit": "10^3/uL",
      "flag": "N"
    }
  ],
  "meta": {
    "source_protocol": "HL7",
    "connector": "hl7-tcp"
  }
}

Delivery and Retry Behavior

TinyLink does not give up easily:

• Retries transient failures (timeouts, DNS/connection issues, HTTP 5xx).
• Uses backoff schedule: 30s -> 2m -> 10m -> 30m -> 2h -> 6h.
• Maximum attempts: 10.
• HTTP 400 and 422 are treated as non-retriable and moved to dead letter immediately.

Health, Readiness, and Metrics

Use these to check system status:

• GET /health: connector status + queue counts (pending, retrying, deadLetters).
• GET /health/ready: confirms SQLite readiness.
• GET /metrics: Prometheus-style metrics, including attempts, latency, and success timestamp.

Daily Operations (Quick Checklist)

• Start app: npm start
• Validate instrument files: npm run instrument:check
• Run migrations manually: npm run migrate
• Backup DB: npm run maintenance -- backup
• Vacuum DB: npm run maintenance -- vacuum
• Prune old delivery logs: npm run maintenance -- prune --days=30

Troubleshooting New Instrument Onboarding

Instrument returns 404

• Confirm exact instrument_id spelling and casing.
• Verify the instrument exists in config/app.yaml as its own top-level key (inst1, inst2, ...).
• Verify that instrument has connector.type, connector settings, and translator.

Data is not flowing

• Confirm instrument has enabled: true.
• Confirm connector matches incoming protocol.
• Confirm match rules fit connector metadata (localPort, remoteAddress, remotePort, or comPort).
• Check /health and /health/ready.

Backlog or dead letters are growing

• Check /metrics for queue and delivery trends.
• Validate CLQMS URL/token/timeouts and downstream availability.

Final Tip

If you are adding many instruments, standardize your naming (for example: LAB1_XN1000, LAB1_COBAS_E411) and keep connector mappings documented. Future-you will send present-you a thank-you card.