433 lines
26 KiB
Markdown
433 lines
26 KiB
Markdown
# Database Schema Redesign Proposal: Test, OrderTest & RefRange Modules
|
|
|
|
**Date:** 2025-12-16
|
|
**Status:** Draft / Proposal
|
|
**Author:** Development Team
|
|
**Purpose:** Propose cleaner, more maintainable table structure
|
|
|
|
---
|
|
|
|
## The Problem: Current Design Issues
|
|
|
|
### 1. Test Module - Confusing Table Split
|
|
|
|
**Current Structure:**
|
|
```
|
|
testdefsite → Basic test info (code, name, description)
|
|
testdeftech → Technical info (result type, units, specimen)
|
|
testdefcal → Calculation formula
|
|
testgrp → Test grouping/panels
|
|
testmap → External system mapping
|
|
```
|
|
|
|
**Issues:**
|
|
| Problem | Description |
|
|
|:--------|:------------|
|
|
| ❌ Artificial separation | `testdefsite` and `testdeftech` are 1:1 relationship - why separate them? |
|
|
| ❌ Confusing naming | "def" prefix is redundant, "site" suffix is misleading |
|
|
| ❌ Redundant columns | `SiteID`, `DisciplineID`, `DepartmentID` duplicated across tables |
|
|
| ❌ Hard to query | Need multiple JOINs just to get basic test info |
|
|
|
|
---
|
|
|
|
### 2. OrderTest Module - Unnecessary Normalization
|
|
|
|
**Current Structure:**
|
|
```
|
|
ordertest → Main order
|
|
ordercom → Comments (separate table)
|
|
orderatt → Attachments (separate table)
|
|
orderstatus → Status history (separate table)
|
|
```
|
|
|
|
**Issues:**
|
|
| Problem | Description |
|
|
|:--------|:------------|
|
|
| ❌ Over-normalized | Comments/attachments could be JSON or simpler structure |
|
|
| ❌ Status as separate table | If you only need current status, this adds complexity |
|
|
| ❌ Missing link | No link between order and actual tests ordered |
|
|
|
|
---
|
|
|
|
### 3. RefRange Module - Too Many Similar Tables
|
|
|
|
**Current Structure:**
|
|
```
|
|
refnum → Numeric ranges (Low, High, Critical)
|
|
refthold → Threshold (single cutoff value)
|
|
reftxt → Text reference
|
|
refvset → Value set reference
|
|
```
|
|
|
|
**Issues:**
|
|
| Problem | Description |
|
|
|:--------|:------------|
|
|
| ❌ 4 tables for same concept | All are "reference ranges" with slight variations |
|
|
| ❌ Duplicated columns | Same columns repeated: TestSiteID, SpcType, Sex, AgeStart, AgeEnd |
|
|
| ❌ Hard to maintain | Adding a new field means updating 4 tables |
|
|
|
|
---
|
|
|
|
## Proposed Redesign
|
|
|
|
### Part A: Test Module - Consolidated Design
|
|
|
|
**BEFORE (5 tables):**
|
|
```
|
|
testdefsite + testdeftech + testdefcal + testgrp + testmap
|
|
```
|
|
|
|
**AFTER (3 tables):**
|
|
```
|
|
tests → All test definition in ONE table
|
|
test_panels → Panel/group membership
|
|
test_mappings → External system mapping
|
|
```
|
|
|
|
#### A1. `tests` (Consolidated Test Definition)
|
|
|
|
Merge `testdefsite`, `testdeftech`, and `testdefcal` into ONE table:
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ tests │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ site_id INT UNSIGNED -- Which lab site │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Basic Info (from testdefsite) -- │
|
|
│ code VARCHAR(10) -- Test code │
|
|
│ name VARCHAR(100) -- Test name │
|
|
│ description VARCHAR(255) │
|
|
│ test_type ENUM('single','panel','calculated') │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Technical Info (from testdeftech) -- │
|
|
│ discipline_id INT UNSIGNED -- Chemistry, Hematology │
|
|
│ department_id INT UNSIGNED │
|
|
│ result_type ENUM('numeric','text','coded') │
|
|
│ specimen_type VARCHAR(20) │
|
|
│ specimen_qty DECIMAL(10,2) │
|
|
│ specimen_unit VARCHAR(20) │
|
|
│ unit VARCHAR(20) -- Result unit │
|
|
│ decimal_places TINYINT │
|
|
│ method VARCHAR(100) │
|
|
│ expected_tat INT -- Turnaround time (mins) │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Calculated Test Info (from testdefcal) -- │
|
|
│ formula TEXT -- NULL if not calculated │
|
|
│ formula_inputs JSON -- List of input test IDs │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Display Order -- │
|
|
│ sort_order_screen INT │
|
|
│ sort_order_report INT │
|
|
│ visible_screen BOOLEAN DEFAULT 1 │
|
|
│ visible_report BOOLEAN DEFAULT 1 │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Audit -- │
|
|
│ created_at DATETIME │
|
|
│ updated_at DATETIME │
|
|
│ deleted_at DATETIME -- Soft delete │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Benefits:**
|
|
- ✅ One query to get all test info
|
|
- ✅ No redundant columns
|
|
- ✅ Clear naming
|
|
- ✅ `test_type` tells you if it's a panel or calculated test
|
|
|
|
---
|
|
|
|
#### A2. `test_panels` (Panel Membership)
|
|
|
|
For tests that are panels (groups of other tests):
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ test_panels │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ panel_test_id INT UNSIGNED FK → tests.id -- The panel │
|
|
│ member_test_id INT UNSIGNED FK → tests.id -- Member test │
|
|
│ sort_order INT │
|
|
│ created_at DATETIME │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Example:** CBC panel contains: WBC, RBC, HGB, HCT, PLT
|
|
```
|
|
panel_test_id=1 (CBC), member_test_id=2 (WBC)
|
|
panel_test_id=1 (CBC), member_test_id=3 (RBC)
|
|
panel_test_id=1 (CBC), member_test_id=4 (HGB)
|
|
...
|
|
```
|
|
|
|
---
|
|
|
|
#### A3. `test_mappings` (External System Mapping)
|
|
|
|
Keep this separate (good design):
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ test_mappings │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ test_id INT UNSIGNED FK → tests.id │
|
|
│ external_system VARCHAR(50) -- 'LIS', 'HIS', 'Analyzer'│
|
|
│ external_code VARCHAR(50) -- Code in that system │
|
|
│ external_name VARCHAR(100) │
|
|
│ connection_id INT UNSIGNED -- Which connection/device │
|
|
│ direction ENUM('inbound','outbound','both') │
|
|
│ created_at DATETIME │
|
|
│ updated_at DATETIME │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
### Part B: Reference Range - Unified Design
|
|
|
|
**BEFORE (4 tables):**
|
|
```
|
|
refnum + refthold + refvset + reftxt
|
|
```
|
|
|
|
**AFTER (1 table):**
|
|
```
|
|
reference_ranges → All reference types in ONE table
|
|
```
|
|
|
|
#### B1. `reference_ranges` (Unified)
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ reference_ranges │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ test_id INT UNSIGNED FK → tests.id │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Criteria (same across all old tables) -- │
|
|
│ specimen_type VARCHAR(20) │
|
|
│ sex ENUM('M','F','A') -- A = All/Any │
|
|
│ age_min INT -- In days for precision │
|
|
│ age_max INT -- In days │
|
|
│ age_unit ENUM('days','months','years') │
|
|
│ criteria VARCHAR(100) -- Additional criteria │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Reference Type -- │
|
|
│ ref_type ENUM('numeric','threshold','text','coded') │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Numeric Range (when ref_type = 'numeric') -- │
|
|
│ critical_low DECIMAL(15,4) │
|
|
│ normal_low DECIMAL(15,4) │
|
|
│ normal_high DECIMAL(15,4) │
|
|
│ critical_high DECIMAL(15,4) │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Threshold (when ref_type = 'threshold') -- │
|
|
│ threshold_value DECIMAL(15,4) │
|
|
│ threshold_operator ENUM('<','<=','>','>=','=') │
|
|
│ below_text VARCHAR(50) -- "Negative", "Normal" │
|
|
│ above_text VARCHAR(50) -- "Positive", "Abnormal" │
|
|
│ gray_zone_low DECIMAL(15,4) │
|
|
│ gray_zone_high DECIMAL(15,4) │
|
|
│ gray_zone_text VARCHAR(50) -- "Equivocal" │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Text/Coded (when ref_type = 'text' or 'coded') -- │
|
|
│ reference_text TEXT -- Expected values or desc │
|
|
│ value_set JSON -- For coded: list of valid│
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Audit -- │
|
|
│ created_at DATETIME │
|
|
│ updated_at DATETIME │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Benefits:**
|
|
- ✅ One table instead of 4
|
|
- ✅ Easy to add new reference types
|
|
- ✅ Single query with `ref_type` filter
|
|
- ✅ No duplicated criteria columns
|
|
|
|
---
|
|
|
|
### Part C: OrderTest - Cleaner Design
|
|
|
|
**BEFORE (4 tables):**
|
|
```
|
|
ordertest + ordercom + orderatt + orderstatus
|
|
```
|
|
|
|
**AFTER (3 tables):**
|
|
```
|
|
orders → Main order with current status
|
|
order_tests → Individual tests in the order (MISSING before!)
|
|
order_history → Status changes + comments combined
|
|
```
|
|
|
|
#### C1. `orders` (Main Order)
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ orders │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ order_number VARCHAR(30) UNIQUE -- Display order ID │
|
|
│ accession_number VARCHAR(30) -- Lab accession │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Patient & Visit -- │
|
|
│ patient_id INT UNSIGNED FK → patients.id │
|
|
│ visit_id INT UNSIGNED FK → visits.id │
|
|
│ site_id INT UNSIGNED │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Order Details -- │
|
|
│ priority ENUM('routine','urgent','stat') │
|
|
│ status ENUM('pending','collected','received', │
|
|
│ 'in_progress','completed','cancelled') │
|
|
│ ordered_by INT UNSIGNED -- Doctor/User ID │
|
|
│ ordered_at DATETIME │
|
|
│ collected_at DATETIME │
|
|
│ received_at DATETIME │
|
|
│ completed_at DATETIME │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ -- Audit -- │
|
|
│ created_at DATETIME │
|
|
│ updated_at DATETIME │
|
|
│ deleted_at DATETIME │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
#### C2. `order_tests` (Tests in Order) — **NEW TABLE!**
|
|
|
|
**This was MISSING in original design!** How do you know what tests are in an order?
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ order_tests │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ order_id INT UNSIGNED FK → orders.id │
|
|
│ test_id INT UNSIGNED FK → tests.id │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ status ENUM('ordered','in_progress','resulted', │
|
|
│ 'verified','cancelled') │
|
|
│ result_value VARCHAR(255) -- The actual result │
|
|
│ result_flag ENUM('N','L','H','LL','HH','A') -- Normal/Abn│
|
|
│ result_comment TEXT │
|
|
│ resulted_by INT UNSIGNED -- Tech who entered result │
|
|
│ resulted_at DATETIME │
|
|
│ verified_by INT UNSIGNED -- Supervisor who verified │
|
|
│ verified_at DATETIME │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ created_at DATETIME │
|
|
│ updated_at DATETIME │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
#### C3. `order_history` (Combined Audit Trail)
|
|
|
|
Combine `ordercom`, `orderatt`, `orderstatus` into one audit table:
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ order_history │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ id INT UNSIGNED PK AUTO_INCREMENT │
|
|
│ order_id INT UNSIGNED FK → orders.id │
|
|
│ order_test_id INT UNSIGNED FK → order_tests.id (nullable) │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ event_type ENUM('status_change','comment','attachment', │
|
|
│ 'result_edit','verification') │
|
|
│ old_value TEXT │
|
|
│ new_value TEXT │
|
|
│ comment TEXT │
|
|
│ attachment_path VARCHAR(255) -- For attachments │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ created_by INT UNSIGNED │
|
|
│ created_at DATETIME │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Summary: Before vs After
|
|
|
|
| Module | Before | After | Change |
|
|
|:-------|:-------|:------|:-------|
|
|
| **Test** | 5 tables | 3 tables | -2 tables |
|
|
| **RefRange** | 4 tables | 1 table | -3 tables |
|
|
| **OrderTest** | 4 tables | 3 tables | -1 table, +1 essential table |
|
|
| **Total** | 13 tables | 7 tables | **-6 tables** |
|
|
|
|
### New ERD
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ PROPOSED ERD │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │
|
|
│ │ tests │◄────────│ test_panels │ │ test_mappings │ │
|
|
│ │ (All tests) │ │ (Panel→Test) │ │ (Ext. systems) │ │
|
|
│ └──────┬──────┘ └──────────────┘ └─────────────────┘ │
|
|
│ │ │
|
|
│ │ 1:N │
|
|
│ ▼ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ reference_ranges │ (All ref types in one table) │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ │
|
|
│ ┌──────────┐ 1:N ┌─────────────┐ 1:N ┌───────────────┐ │
|
|
│ │ patients │◄──────────│ orders │◄──────────│ order_history │ │
|
|
│ └──────────┘ └──────┬──────┘ └───────────────┘ │
|
|
│ │ │
|
|
│ │ 1:N │
|
|
│ ▼ │
|
|
│ ┌─────────────┐ │
|
|
│ │ order_tests │ (What tests are in order) │
|
|
│ └──────┬──────┘ │
|
|
│ │ │
|
|
│ │ N:1 │
|
|
│ ▼ │
|
|
│ ┌─────────────┐ │
|
|
│ │ tests │ │
|
|
│ └─────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Migration Strategy
|
|
|
|
Since this is a major restructure:
|
|
|
|
1. **Create new migration files** (don't modify old ones)
|
|
2. **Write data migration script** to move data from old to new tables
|
|
3. **Update Models, Controllers, Views** to use new table names
|
|
4. **Test thoroughly** before dropping old tables
|
|
|
|
---
|
|
|
|
## Questions for Discussion
|
|
|
|
1. Is storing `formula` as TEXT acceptable, or need a more structured approach?
|
|
2. Should `order_history` store ALL changes, or just important ones?
|
|
3. Any additional fields needed that I missed?
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. ✅ Review and approve this proposal
|
|
2. 🔲 Create new migration files
|
|
3. 🔲 Write data migration scripts
|
|
4. 🔲 Update Models to use new tables
|
|
5. 🔲 Update Controllers and Services
|
|
6. 🔲 Deprecate old tables
|