26 KiB
26 KiB
Database Schema Redesign Proposal: Test, OrderTest & RefRange Modules
Date: 2025-12-16
Status: Draft / Proposal
Author: Development Team
Purpose: Propose cleaner, more maintainable table structure
The Problem: Current Design Issues
1. Test Module - Confusing Table Split
Current Structure:
testdefsite → Basic test info (code, name, description)
testdeftech → Technical info (result type, units, specimen)
testdefcal → Calculation formula
testgrp → Test grouping/panels
testmap → External system mapping
Issues:
| Problem | Description |
|---|---|
| ❌ Artificial separation | testdefsite and testdeftech are 1:1 relationship - why separate them? |
| ❌ Confusing naming | "def" prefix is redundant, "site" suffix is misleading |
| ❌ Redundant columns | SiteID, DisciplineID, DepartmentID duplicated across tables |
| ❌ Hard to query | Need multiple JOINs just to get basic test info |
2. OrderTest Module - Unnecessary Normalization
Current Structure:
ordertest → Main order
ordercom → Comments (separate table)
orderatt → Attachments (separate table)
orderstatus → Status history (separate table)
Issues:
| Problem | Description |
|---|---|
| ❌ Over-normalized | Comments/attachments could be JSON or simpler structure |
| ❌ Status as separate table | If you only need current status, this adds complexity |
| ❌ Missing link | No link between order and actual tests ordered |
3. RefRange Module - Too Many Similar Tables
Current Structure:
refnum → Numeric ranges (Low, High, Critical)
refthold → Threshold (single cutoff value)
reftxt → Text reference
refvset → Value set reference
Issues:
| Problem | Description |
|---|---|
| ❌ 4 tables for same concept | All are "reference ranges" with slight variations |
| ❌ Duplicated columns | Same columns repeated: TestSiteID, SpcType, Sex, AgeStart, AgeEnd |
| ❌ Hard to maintain | Adding a new field means updating 4 tables |
Proposed Redesign
Part A: Test Module - Consolidated Design
BEFORE (5 tables):
testdefsite + testdeftech + testdefcal + testgrp + testmap
AFTER (3 tables):
tests → All test definition in ONE table
test_panels → Panel/group membership
test_mappings → External system mapping
A1. tests (Consolidated Test Definition)
Merge testdefsite, testdeftech, and testdefcal into ONE table:
┌─────────────────────────────────────────────────────────────────┐
│ tests │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ site_id INT UNSIGNED -- Which lab site │
├─────────────────────────────────────────────────────────────────┤
│ -- Basic Info (from testdefsite) -- │
│ code VARCHAR(10) -- Test code │
│ name VARCHAR(100) -- Test name │
│ description VARCHAR(255) │
│ test_type ENUM('single','panel','calculated') │
├─────────────────────────────────────────────────────────────────┤
│ -- Technical Info (from testdeftech) -- │
│ discipline_id INT UNSIGNED -- Chemistry, Hematology │
│ department_id INT UNSIGNED │
│ result_type ENUM('numeric','text','coded') │
│ specimen_type VARCHAR(20) │
│ specimen_qty DECIMAL(10,2) │
│ specimen_unit VARCHAR(20) │
│ unit VARCHAR(20) -- Result unit │
│ decimal_places TINYINT │
│ method VARCHAR(100) │
│ expected_tat INT -- Turnaround time (mins) │
├─────────────────────────────────────────────────────────────────┤
│ -- Calculated Test Info (from testdefcal) -- │
│ formula TEXT -- NULL if not calculated │
│ formula_inputs JSON -- List of input test IDs │
├─────────────────────────────────────────────────────────────────┤
│ -- Display Order -- │
│ sort_order_screen INT │
│ sort_order_report INT │
│ visible_screen BOOLEAN DEFAULT 1 │
│ visible_report BOOLEAN DEFAULT 1 │
├─────────────────────────────────────────────────────────────────┤
│ -- Audit -- │
│ created_at DATETIME │
│ updated_at DATETIME │
│ deleted_at DATETIME -- Soft delete │
└─────────────────────────────────────────────────────────────────┘
Benefits:
- ✅ One query to get all test info
- ✅ No redundant columns
- ✅ Clear naming
- ✅
test_typetells you if it's a panel or calculated test
A2. test_panels (Panel Membership)
For tests that are panels (groups of other tests):
┌─────────────────────────────────────────────────────────────────┐
│ test_panels │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ panel_test_id INT UNSIGNED FK → tests.id -- The panel │
│ member_test_id INT UNSIGNED FK → tests.id -- Member test │
│ sort_order INT │
│ created_at DATETIME │
└─────────────────────────────────────────────────────────────────┘
Example: CBC panel contains: WBC, RBC, HGB, HCT, PLT
panel_test_id=1 (CBC), member_test_id=2 (WBC)
panel_test_id=1 (CBC), member_test_id=3 (RBC)
panel_test_id=1 (CBC), member_test_id=4 (HGB)
...
A3. test_mappings (External System Mapping)
Keep this separate (good design):
┌─────────────────────────────────────────────────────────────────┐
│ test_mappings │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ test_id INT UNSIGNED FK → tests.id │
│ external_system VARCHAR(50) -- 'LIS', 'HIS', 'Analyzer'│
│ external_code VARCHAR(50) -- Code in that system │
│ external_name VARCHAR(100) │
│ connection_id INT UNSIGNED -- Which connection/device │
│ direction ENUM('inbound','outbound','both') │
│ created_at DATETIME │
│ updated_at DATETIME │
└─────────────────────────────────────────────────────────────────┘
Part B: Reference Range - Unified Design
BEFORE (4 tables):
refnum + refthold + refvset + reftxt
AFTER (1 table):
reference_ranges → All reference types in ONE table
B1. reference_ranges (Unified)
┌─────────────────────────────────────────────────────────────────┐
│ reference_ranges │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ test_id INT UNSIGNED FK → tests.id │
├─────────────────────────────────────────────────────────────────┤
│ -- Criteria (same across all old tables) -- │
│ specimen_type VARCHAR(20) │
│ sex ENUM('M','F','A') -- A = All/Any │
│ age_min INT -- In days for precision │
│ age_max INT -- In days │
│ age_unit ENUM('days','months','years') │
│ criteria VARCHAR(100) -- Additional criteria │
├─────────────────────────────────────────────────────────────────┤
│ -- Reference Type -- │
│ ref_type ENUM('numeric','threshold','text','coded') │
├─────────────────────────────────────────────────────────────────┤
│ -- Numeric Range (when ref_type = 'numeric') -- │
│ critical_low DECIMAL(15,4) │
│ normal_low DECIMAL(15,4) │
│ normal_high DECIMAL(15,4) │
│ critical_high DECIMAL(15,4) │
├─────────────────────────────────────────────────────────────────┤
│ -- Threshold (when ref_type = 'threshold') -- │
│ threshold_value DECIMAL(15,4) │
│ threshold_operator ENUM('<','<=','>','>=','=') │
│ below_text VARCHAR(50) -- "Negative", "Normal" │
│ above_text VARCHAR(50) -- "Positive", "Abnormal" │
│ gray_zone_low DECIMAL(15,4) │
│ gray_zone_high DECIMAL(15,4) │
│ gray_zone_text VARCHAR(50) -- "Equivocal" │
├─────────────────────────────────────────────────────────────────┤
│ -- Text/Coded (when ref_type = 'text' or 'coded') -- │
│ reference_text TEXT -- Expected values or desc │
│ value_set JSON -- For coded: list of valid│
├─────────────────────────────────────────────────────────────────┤
│ -- Audit -- │
│ created_at DATETIME │
│ updated_at DATETIME │
└─────────────────────────────────────────────────────────────────┘
Benefits:
- ✅ One table instead of 4
- ✅ Easy to add new reference types
- ✅ Single query with
ref_typefilter - ✅ No duplicated criteria columns
Part C: OrderTest - Cleaner Design
BEFORE (4 tables):
ordertest + ordercom + orderatt + orderstatus
AFTER (3 tables):
orders → Main order with current status
order_tests → Individual tests in the order (MISSING before!)
order_history → Status changes + comments combined
C1. orders (Main Order)
┌─────────────────────────────────────────────────────────────────┐
│ orders │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ order_number VARCHAR(30) UNIQUE -- Display order ID │
│ accession_number VARCHAR(30) -- Lab accession │
├─────────────────────────────────────────────────────────────────┤
│ -- Patient & Visit -- │
│ patient_id INT UNSIGNED FK → patients.id │
│ visit_id INT UNSIGNED FK → visits.id │
│ site_id INT UNSIGNED │
├─────────────────────────────────────────────────────────────────┤
│ -- Order Details -- │
│ priority ENUM('routine','urgent','stat') │
│ status ENUM('pending','collected','received', │
│ 'in_progress','completed','cancelled') │
│ ordered_by INT UNSIGNED -- Doctor/User ID │
│ ordered_at DATETIME │
│ collected_at DATETIME │
│ received_at DATETIME │
│ completed_at DATETIME │
├─────────────────────────────────────────────────────────────────┤
│ -- Audit -- │
│ created_at DATETIME │
│ updated_at DATETIME │
│ deleted_at DATETIME │
└─────────────────────────────────────────────────────────────────┘
C2. order_tests (Tests in Order) — NEW TABLE!
This was MISSING in original design! How do you know what tests are in an order?
┌─────────────────────────────────────────────────────────────────┐
│ order_tests │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ order_id INT UNSIGNED FK → orders.id │
│ test_id INT UNSIGNED FK → tests.id │
├─────────────────────────────────────────────────────────────────┤
│ status ENUM('ordered','in_progress','resulted', │
│ 'verified','cancelled') │
│ result_value VARCHAR(255) -- The actual result │
│ result_flag ENUM('N','L','H','LL','HH','A') -- Normal/Abn│
│ result_comment TEXT │
│ resulted_by INT UNSIGNED -- Tech who entered result │
│ resulted_at DATETIME │
│ verified_by INT UNSIGNED -- Supervisor who verified │
│ verified_at DATETIME │
├─────────────────────────────────────────────────────────────────┤
│ created_at DATETIME │
│ updated_at DATETIME │
└─────────────────────────────────────────────────────────────────┘
C3. order_history (Combined Audit Trail)
Combine ordercom, orderatt, orderstatus into one audit table:
┌─────────────────────────────────────────────────────────────────┐
│ order_history │
├─────────────────────────────────────────────────────────────────┤
│ id INT UNSIGNED PK AUTO_INCREMENT │
│ order_id INT UNSIGNED FK → orders.id │
│ order_test_id INT UNSIGNED FK → order_tests.id (nullable) │
├─────────────────────────────────────────────────────────────────┤
│ event_type ENUM('status_change','comment','attachment', │
│ 'result_edit','verification') │
│ old_value TEXT │
│ new_value TEXT │
│ comment TEXT │
│ attachment_path VARCHAR(255) -- For attachments │
├─────────────────────────────────────────────────────────────────┤
│ created_by INT UNSIGNED │
│ created_at DATETIME │
└─────────────────────────────────────────────────────────────────┘
Summary: Before vs After
| Module | Before | After | Change |
|---|---|---|---|
| Test | 5 tables | 3 tables | -2 tables |
| RefRange | 4 tables | 1 table | -3 tables |
| OrderTest | 4 tables | 3 tables | -1 table, +1 essential table |
| Total | 13 tables | 7 tables | -6 tables |
New ERD
┌─────────────────────────────────────────────────────────────────────────────┐
│ PROPOSED ERD │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │
│ │ tests │◄────────│ test_panels │ │ test_mappings │ │
│ │ (All tests) │ │ (Panel→Test) │ │ (Ext. systems) │ │
│ └──────┬──────┘ └──────────────┘ └─────────────────┘ │
│ │ │
│ │ 1:N │
│ ▼ │
│ ┌──────────────────┐ │
│ │ reference_ranges │ (All ref types in one table) │
│ └──────────────────┘ │
│ │
│ │
│ ┌──────────┐ 1:N ┌─────────────┐ 1:N ┌───────────────┐ │
│ │ patients │◄──────────│ orders │◄──────────│ order_history │ │
│ └──────────┘ └──────┬──────┘ └───────────────┘ │
│ │ │
│ │ 1:N │
│ ▼ │
│ ┌─────────────┐ │
│ │ order_tests │ (What tests are in order) │
│ └──────┬──────┘ │
│ │ │
│ │ N:1 │
│ ▼ │
│ ┌─────────────┐ │
│ │ tests │ │
│ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Migration Strategy
Since this is a major restructure:
- Create new migration files (don't modify old ones)
- Write data migration script to move data from old to new tables
- Update Models, Controllers, Views to use new table names
- Test thoroughly before dropping old tables
Questions for Discussion
- Is storing
formulaas TEXT acceptable, or need a more structured approach? - Should
order_historystore ALL changes, or just important ones? - Any additional fields needed that I missed?
Next Steps
- ✅ Review and approve this proposal
- 🔲 Create new migration files
- 🔲 Write data migration scripts
- 🔲 Update Models to use new tables
- 🔲 Update Controllers and Services
- 🔲 Deprecate old tables