# Database Schema Redesign Proposal: Test, OrderTest & RefRange Modules **Date:** 2025-12-16 **Status:** Draft / Proposal **Author:** Development Team **Purpose:** Propose cleaner, more maintainable table structure --- ## The Problem: Current Design Issues ### 1. Test Module - Confusing Table Split **Current Structure:** ``` testdefsite → Basic test info (code, name, description) testdeftech → Technical info (result type, units, specimen) testdefcal → Calculation formula testgrp → Test grouping/panels testmap → External system mapping ``` **Issues:** | Problem | Description | |:--------|:------------| | ❌ Artificial separation | `testdefsite` and `testdeftech` are 1:1 relationship - why separate them? | | ❌ Confusing naming | "def" prefix is redundant, "site" suffix is misleading | | ❌ Redundant columns | `SiteID`, `DisciplineID`, `DepartmentID` duplicated across tables | | ❌ Hard to query | Need multiple JOINs just to get basic test info | --- ### 2. OrderTest Module - Unnecessary Normalization **Current Structure:** ``` ordertest → Main order ordercom → Comments (separate table) orderatt → Attachments (separate table) orderstatus → Status history (separate table) ``` **Issues:** | Problem | Description | |:--------|:------------| | ❌ Over-normalized | Comments/attachments could be JSON or simpler structure | | ❌ Status as separate table | If you only need current status, this adds complexity | | ❌ Missing link | No link between order and actual tests ordered | --- ### 3. RefRange Module - Too Many Similar Tables **Current Structure:** ``` refnum → Numeric ranges (Low, High, Critical) refthold → Threshold (single cutoff value) reftxt → Text reference refvset → Value set reference ``` **Issues:** | Problem | Description | |:--------|:------------| | ❌ 4 tables for same concept | All are "reference ranges" with slight variations | | ❌ Duplicated columns | Same columns repeated: TestSiteID, SpcType, Sex, AgeStart, AgeEnd | | ❌ Hard to maintain | Adding a new field means updating 4 tables | --- ## Proposed Redesign ### Part A: Test Module - Consolidated Design **BEFORE (5 tables):** ``` testdefsite + testdeftech + testdefcal + testgrp + testmap ``` **AFTER (3 tables):** ``` tests → All test definition in ONE table test_panels → Panel/group membership test_mappings → External system mapping ``` #### A1. `tests` (Consolidated Test Definition) Merge `testdefsite`, `testdeftech`, and `testdefcal` into ONE table: ``` ┌─────────────────────────────────────────────────────────────────┐ │ tests │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ site_id INT UNSIGNED -- Which lab site │ ├─────────────────────────────────────────────────────────────────┤ │ -- Basic Info (from testdefsite) -- │ │ code VARCHAR(10) -- Test code │ │ name VARCHAR(100) -- Test name │ │ description VARCHAR(255) │ │ test_type ENUM('single','panel','calculated') │ ├─────────────────────────────────────────────────────────────────┤ │ -- Technical Info (from testdeftech) -- │ │ discipline_id INT UNSIGNED -- Chemistry, Hematology │ │ department_id INT UNSIGNED │ │ result_type ENUM('numeric','text','coded') │ │ specimen_type VARCHAR(20) │ │ specimen_qty DECIMAL(10,2) │ │ specimen_unit VARCHAR(20) │ │ unit VARCHAR(20) -- Result unit │ │ decimal_places TINYINT │ │ method VARCHAR(100) │ │ expected_tat INT -- Turnaround time (mins) │ ├─────────────────────────────────────────────────────────────────┤ │ -- Calculated Test Info (from testdefcal) -- │ │ formula TEXT -- NULL if not calculated │ │ formula_inputs JSON -- List of input test IDs │ ├─────────────────────────────────────────────────────────────────┤ │ -- Display Order -- │ │ sort_order_screen INT │ │ sort_order_report INT │ │ visible_screen BOOLEAN DEFAULT 1 │ │ visible_report BOOLEAN DEFAULT 1 │ ├─────────────────────────────────────────────────────────────────┤ │ -- Audit -- │ │ created_at DATETIME │ │ updated_at DATETIME │ │ deleted_at DATETIME -- Soft delete │ └─────────────────────────────────────────────────────────────────┘ ``` **Benefits:** - ✅ One query to get all test info - ✅ No redundant columns - ✅ Clear naming - ✅ `test_type` tells you if it's a panel or calculated test --- #### A2. `test_panels` (Panel Membership) For tests that are panels (groups of other tests): ``` ┌─────────────────────────────────────────────────────────────────┐ │ test_panels │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ panel_test_id INT UNSIGNED FK → tests.id -- The panel │ │ member_test_id INT UNSIGNED FK → tests.id -- Member test │ │ sort_order INT │ │ created_at DATETIME │ └─────────────────────────────────────────────────────────────────┘ ``` **Example:** CBC panel contains: WBC, RBC, HGB, HCT, PLT ``` panel_test_id=1 (CBC), member_test_id=2 (WBC) panel_test_id=1 (CBC), member_test_id=3 (RBC) panel_test_id=1 (CBC), member_test_id=4 (HGB) ... ``` --- #### A3. `test_mappings` (External System Mapping) Keep this separate (good design): ``` ┌─────────────────────────────────────────────────────────────────┐ │ test_mappings │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ test_id INT UNSIGNED FK → tests.id │ │ external_system VARCHAR(50) -- 'LIS', 'HIS', 'Analyzer'│ │ external_code VARCHAR(50) -- Code in that system │ │ external_name VARCHAR(100) │ │ connection_id INT UNSIGNED -- Which connection/device │ │ direction ENUM('inbound','outbound','both') │ │ created_at DATETIME │ │ updated_at DATETIME │ └─────────────────────────────────────────────────────────────────┘ ``` --- ### Part B: Reference Range - Unified Design **BEFORE (4 tables):** ``` refnum + refthold + refvset + reftxt ``` **AFTER (1 table):** ``` reference_ranges → All reference types in ONE table ``` #### B1. `reference_ranges` (Unified) ``` ┌─────────────────────────────────────────────────────────────────┐ │ reference_ranges │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ test_id INT UNSIGNED FK → tests.id │ ├─────────────────────────────────────────────────────────────────┤ │ -- Criteria (same across all old tables) -- │ │ specimen_type VARCHAR(20) │ │ sex ENUM('M','F','A') -- A = All/Any │ │ age_min INT -- In days for precision │ │ age_max INT -- In days │ │ age_unit ENUM('days','months','years') │ │ criteria VARCHAR(100) -- Additional criteria │ ├─────────────────────────────────────────────────────────────────┤ │ -- Reference Type -- │ │ ref_type ENUM('numeric','threshold','text','coded') │ ├─────────────────────────────────────────────────────────────────┤ │ -- Numeric Range (when ref_type = 'numeric') -- │ │ critical_low DECIMAL(15,4) │ │ normal_low DECIMAL(15,4) │ │ normal_high DECIMAL(15,4) │ │ critical_high DECIMAL(15,4) │ ├─────────────────────────────────────────────────────────────────┤ │ -- Threshold (when ref_type = 'threshold') -- │ │ threshold_value DECIMAL(15,4) │ │ threshold_operator ENUM('<','<=','>','>=','=') │ │ below_text VARCHAR(50) -- "Negative", "Normal" │ │ above_text VARCHAR(50) -- "Positive", "Abnormal" │ │ gray_zone_low DECIMAL(15,4) │ │ gray_zone_high DECIMAL(15,4) │ │ gray_zone_text VARCHAR(50) -- "Equivocal" │ ├─────────────────────────────────────────────────────────────────┤ │ -- Text/Coded (when ref_type = 'text' or 'coded') -- │ │ reference_text TEXT -- Expected values or desc │ │ value_set JSON -- For coded: list of valid│ ├─────────────────────────────────────────────────────────────────┤ │ -- Audit -- │ │ created_at DATETIME │ │ updated_at DATETIME │ └─────────────────────────────────────────────────────────────────┘ ``` **Benefits:** - ✅ One table instead of 4 - ✅ Easy to add new reference types - ✅ Single query with `ref_type` filter - ✅ No duplicated criteria columns --- ### Part C: OrderTest - Cleaner Design **BEFORE (4 tables):** ``` ordertest + ordercom + orderatt + orderstatus ``` **AFTER (3 tables):** ``` orders → Main order with current status order_tests → Individual tests in the order (MISSING before!) order_history → Status changes + comments combined ``` #### C1. `orders` (Main Order) ``` ┌─────────────────────────────────────────────────────────────────┐ │ orders │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ order_number VARCHAR(30) UNIQUE -- Display order ID │ │ accession_number VARCHAR(30) -- Lab accession │ ├─────────────────────────────────────────────────────────────────┤ │ -- Patient & Visit -- │ │ patient_id INT UNSIGNED FK → patients.id │ │ visit_id INT UNSIGNED FK → visits.id │ │ site_id INT UNSIGNED │ ├─────────────────────────────────────────────────────────────────┤ │ -- Order Details -- │ │ priority ENUM('routine','urgent','stat') │ │ status ENUM('pending','collected','received', │ │ 'in_progress','completed','cancelled') │ │ ordered_by INT UNSIGNED -- Doctor/User ID │ │ ordered_at DATETIME │ │ collected_at DATETIME │ │ received_at DATETIME │ │ completed_at DATETIME │ ├─────────────────────────────────────────────────────────────────┤ │ -- Audit -- │ │ created_at DATETIME │ │ updated_at DATETIME │ │ deleted_at DATETIME │ └─────────────────────────────────────────────────────────────────┘ ``` --- #### C2. `order_tests` (Tests in Order) — **NEW TABLE!** **This was MISSING in original design!** How do you know what tests are in an order? ``` ┌─────────────────────────────────────────────────────────────────┐ │ order_tests │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ order_id INT UNSIGNED FK → orders.id │ │ test_id INT UNSIGNED FK → tests.id │ ├─────────────────────────────────────────────────────────────────┤ │ status ENUM('ordered','in_progress','resulted', │ │ 'verified','cancelled') │ │ result_value VARCHAR(255) -- The actual result │ │ result_flag ENUM('N','L','H','LL','HH','A') -- Normal/Abn│ │ result_comment TEXT │ │ resulted_by INT UNSIGNED -- Tech who entered result │ │ resulted_at DATETIME │ │ verified_by INT UNSIGNED -- Supervisor who verified │ │ verified_at DATETIME │ ├─────────────────────────────────────────────────────────────────┤ │ created_at DATETIME │ │ updated_at DATETIME │ └─────────────────────────────────────────────────────────────────┘ ``` --- #### C3. `order_history` (Combined Audit Trail) Combine `ordercom`, `orderatt`, `orderstatus` into one audit table: ``` ┌─────────────────────────────────────────────────────────────────┐ │ order_history │ ├─────────────────────────────────────────────────────────────────┤ │ id INT UNSIGNED PK AUTO_INCREMENT │ │ order_id INT UNSIGNED FK → orders.id │ │ order_test_id INT UNSIGNED FK → order_tests.id (nullable) │ ├─────────────────────────────────────────────────────────────────┤ │ event_type ENUM('status_change','comment','attachment', │ │ 'result_edit','verification') │ │ old_value TEXT │ │ new_value TEXT │ │ comment TEXT │ │ attachment_path VARCHAR(255) -- For attachments │ ├─────────────────────────────────────────────────────────────────┤ │ created_by INT UNSIGNED │ │ created_at DATETIME │ └─────────────────────────────────────────────────────────────────┘ ``` --- ## Summary: Before vs After | Module | Before | After | Change | |:-------|:-------|:------|:-------| | **Test** | 5 tables | 3 tables | -2 tables | | **RefRange** | 4 tables | 1 table | -3 tables | | **OrderTest** | 4 tables | 3 tables | -1 table, +1 essential table | | **Total** | 13 tables | 7 tables | **-6 tables** | ### New ERD ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ PROPOSED ERD │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │ │ │ tests │◄────────│ test_panels │ │ test_mappings │ │ │ │ (All tests) │ │ (Panel→Test) │ │ (Ext. systems) │ │ │ └──────┬──────┘ └──────────────┘ └─────────────────┘ │ │ │ │ │ │ 1:N │ │ ▼ │ │ ┌──────────────────┐ │ │ │ reference_ranges │ (All ref types in one table) │ │ └──────────────────┘ │ │ │ │ │ │ ┌──────────┐ 1:N ┌─────────────┐ 1:N ┌───────────────┐ │ │ │ patients │◄──────────│ orders │◄──────────│ order_history │ │ │ └──────────┘ └──────┬──────┘ └───────────────┘ │ │ │ │ │ │ 1:N │ │ ▼ │ │ ┌─────────────┐ │ │ │ order_tests │ (What tests are in order) │ │ └──────┬──────┘ │ │ │ │ │ │ N:1 │ │ ▼ │ │ ┌─────────────┐ │ │ │ tests │ │ │ └─────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` --- ## Migration Strategy Since this is a major restructure: 1. **Create new migration files** (don't modify old ones) 2. **Write data migration script** to move data from old to new tables 3. **Update Models, Controllers, Views** to use new table names 4. **Test thoroughly** before dropping old tables --- ## Questions for Discussion 1. Is storing `formula` as TEXT acceptable, or need a more structured approach? 2. Should `order_history` store ALL changes, or just important ones? 3. Any additional fields needed that I missed? --- ## Next Steps 1. ✅ Review and approve this proposal 2. 🔲 Create new migration files 3. 🔲 Write data migration scripts 4. 🔲 Update Models to use new tables 5. 🔲 Update Controllers and Services 6. 🔲 Deprecate old tables