mahdahar/5panda.11ty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(clqms): renumber audit logging plan document with consistent prefix

Browse Source
This commit is contained in:
mahdahar 2026-02-20 10:09:30 +07:00
parent a69b2fc7d8
commit cc7f101932
3 changed files with 773 additions and 484 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

771
src/projects/clqms01/suggestion/004-audit-logging-plan.md Normal file
View File

@ -0,0 +1,771 @@
---
layout: clqms-post.njk
tags: clqms
title: "Audit Logging Architecture Plan"
description: "Comprehensive audit trail implementation based on 5W1H principles across four specialized log types"
date: 2026-02-20
order: 4
---
# Audit Logging Architecture Plan for CLQMS
> **Clinical Laboratory Quality Management System (CLQMS)** - Comprehensive audit trail implementation based on section 4.2.1.20 Error Management requirements, implementing 5W1H audit principles across four specialized log types.
---
## Executive Summary
This document defines the audit logging architecture for CLQMS, implementing the **5W1H audit principle** (What, When, Who, How, Where, Why) across four specialized log tables. The design supports both **manual** (user-initiated) and **automatic** (instrument/service-initiated) operations with complete traceability.
---
## 1. Requirements Analysis (Section 4.2.1.20)
### 5W1H Audit Principles
| Dimension | Description | Captured Fields |
|-----------|-------------|-----------------|
| **What** | Data changed, operation performed | `operation`, `table_name`, `field_name`, `previous_value`, `new_value` |
| **When** | Timestamp of activity | `created_at` |
| **Who** | User performing operation | `user_id` |
| **How** | Mechanism, application, session | `mechanism`, `application_id`, `web_page`, `session_id`, `event_type` |
| **Where** | Location of operation | `site_id`, `workstation_id`, `pc_name`, `ip_address` |
| **Why** | Reason for operation | `reason` |
### Four Log Types
| Log Type | Description | Examples |
|----------|-------------|----------|
| **Data Log** | Events related to data operations | Patient demographics, visits, test orders, samples, results, user data, master data, archiving, transaction errors |
| **Service Log** | Background service events | Host communication, instrument communication, printing, messaging, resource access, system errors |
| **Security Log** | Security and access events | Logins/logouts, file access, permission changes, password failures, system changes |
| **Error Log** | Error events by entity | Instrument errors, integration errors, validation errors |
### Mechanism Types
- **MANUAL**: User-initiated actions via web interface
- **AUTOMATIC**: System/instrument-initiated (duplo/repeated operations)
---
## 2. Table Architecture
### 2.1 Overview
Four separate tables optimized for different volumes and retention:
| Table | Volume | Retention | Partitioning |
|-------|--------|-----------|--------------|
| `data_audit_log` | Medium | 7 years | Monthly |
| `service_audit_log` | Very High | 2 years | Monthly |
| `security_audit_log` | Low | Permanent | No |
| `error_audit_log` | Variable | 5 years | Monthly |
---
### 2.2 Table: data_audit_log
```sql
CREATE TABLE data_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
-- WHAT: Operation and data details
operation VARCHAR(50) NOT NULL, -- 'CREATE', 'UPDATE', 'DELETE', 'ARCHIVE', etc.
entity_type VARCHAR(50) NOT NULL, -- 'patient', 'visit', 'test_order', 'sample', 'user', etc.
entity_id VARCHAR(36) NOT NULL, -- ID of affected entity
table_name VARCHAR(100), -- Database table name
field_name VARCHAR(100), -- Specific field changed (NULL if multiple)
previous_value JSON, -- Value before change
new_value JSON, -- Value after change
-- HOW: Mechanism details
mechanism ENUM('MANUAL', 'AUTOMATIC') NOT NULL DEFAULT 'MANUAL',
application_id VARCHAR(50), -- Application identifier
web_page VARCHAR(500), -- URL/endpoint accessed
session_id VARCHAR(100), -- Session identifier
event_type VARCHAR(100), -- Event classification
-- WHERE: Location information
site_id VARCHAR(36), -- Site/location ID
workstation_id VARCHAR(36), -- Workstation ID
pc_name VARCHAR(100), -- Computer name
ip_address VARCHAR(45), -- IP address (IPv6 compatible)
-- WHO: User information
user_id VARCHAR(36) NOT NULL, -- User ID or 'SYSTEM' for automatic
-- WHEN: Timestamp
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
-- WHY: Reason
reason TEXT, -- User-provided reason
-- Context: Additional flexible data
context JSON, -- Log-type-specific extra data
-- Indexes
INDEX idx_operation_created (operation, created_at),
INDEX idx_entity (entity_type, entity_id, created_at),
INDEX idx_user_created (user_id, created_at),
INDEX idx_mechanism (mechanism, created_at),
INDEX idx_table (table_name, created_at),
INDEX idx_site (site_id, created_at),
INDEX idx_created (created_at),
INDEX idx_session (session_id, created_at)
) ENGINE=InnoDB
PARTITION BY RANGE (YEAR(created_at) * 100 + MONTH(created_at)) (
PARTITION p202601 VALUES LESS THAN (202602),
PARTITION p202602 VALUES LESS THAN (202603),
PARTITION p_future VALUES LESS THAN MAXVALUE
);
```
**Data Log Examples:**
- Patient registration: `operation='CREATE'`, `entity_type='patient'`, `mechanism='MANUAL'`
- Sample result from instrument: `operation='UPDATE'`, `entity_type='result'`, `mechanism='AUTOMATIC'`
- User profile update: `operation='UPDATE'`, `entity_type='user'`, `mechanism='MANUAL'`
---
### 2.3 Table: service_audit_log
```sql
CREATE TABLE service_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
-- WHAT: Service operation details
operation VARCHAR(50) NOT NULL, -- 'COMMUNICATION', 'PRINT', 'BACKUP', 'MESSAGE', etc.
entity_type VARCHAR(50) NOT NULL, -- 'host', 'instrument', 'database', 'network', etc.
entity_id VARCHAR(36) NOT NULL, -- Service identifier
service_class VARCHAR(50), -- 'communication', 'printing', 'messaging', 'resource'
resource_type VARCHAR(100), -- 'database_access', 'backup', 'network', 'internet'
resource_details JSON, -- IP, port, connection details
previous_value JSON, -- State before
new_value JSON, -- State after
-- HOW: Mechanism and context
mechanism ENUM('MANUAL', 'AUTOMATIC') NOT NULL DEFAULT 'AUTOMATIC',
application_id VARCHAR(50), -- Service application ID
service_name VARCHAR(100), -- Background service name
session_id VARCHAR(100), -- Service session
event_type VARCHAR(100), -- Event classification
-- WHERE: Location and resources
site_id VARCHAR(36),
workstation_id VARCHAR(36),
pc_name VARCHAR(100),
ip_address VARCHAR(45),
port INT, -- Port number for network
-- WHO: System or user
user_id VARCHAR(36) NOT NULL, -- 'SYSTEM' for automatic services
-- WHEN: Timestamp
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
-- WHY: Reason if manual
reason TEXT,
-- Context: Service-specific data
context JSON, -- Communication details, error codes, etc.
-- Indexes
INDEX idx_operation_created (operation, created_at),
INDEX idx_entity (entity_type, entity_id, created_at),
INDEX idx_service_class (service_class, created_at),
INDEX idx_user_created (user_id, created_at),
INDEX idx_mechanism (mechanism, created_at),
INDEX idx_site (site_id, created_at),
INDEX idx_created (created_at)
) ENGINE=InnoDB
PARTITION BY RANGE (YEAR(created_at) * 100 + MONTH(created_at)) (
PARTITION p202601 VALUES LESS THAN (202602),
PARTITION p202602 VALUES LESS THAN (202603),
PARTITION p_future VALUES LESS THAN MAXVALUE
);
```
**Service Log Examples:**
- Instrument communication: `operation='COMMUNICATION'`, `entity_type='instrument'`, `service_class='communication'`
- Database backup: `operation='BACKUP'`, `entity_type='database'`, `service_class='resource'`
- Automatic print: `operation='PRINT'`, `service_class='printing'`, `mechanism='AUTOMATIC'`
---
### 2.4 Table: security_audit_log
```sql
CREATE TABLE security_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
-- WHAT: Security event details
operation VARCHAR(50) NOT NULL, -- 'LOGIN', 'LOGOUT', 'ACCESS_DENIED', 'PASSWORD_FAIL', etc.
entity_type VARCHAR(50) NOT NULL, -- 'user', 'file', 'folder', 'setting', 'application'
entity_id VARCHAR(36) NOT NULL, -- Target entity ID
security_class VARCHAR(50), -- 'authentication', 'authorization', 'system_change'
resource_path VARCHAR(500), -- File/folder path accessed
previous_value JSON, -- Previous security state
new_value JSON, -- New security state
-- HOW: Access details
mechanism ENUM('MANUAL', 'AUTOMATIC') NOT NULL DEFAULT 'MANUAL',
application_id VARCHAR(50),
web_page VARCHAR(500),
session_id VARCHAR(100),
event_type VARCHAR(100), -- 'SUCCESS', 'FAILURE', 'WARNING'
-- WHERE: Access location
site_id VARCHAR(36),
workstation_id VARCHAR(36),
pc_name VARCHAR(100),
ip_address VARCHAR(45),
-- WHO: User attempting action
user_id VARCHAR(36) NOT NULL, -- User ID or 'UNKNOWN' for failed attempts
-- WHEN: Timestamp
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
-- WHY: Reason if provided
reason TEXT,
-- Context: Security-specific data
context JSON, -- Permission changes, failure counts, etc.
-- Indexes
INDEX idx_operation_created (operation, created_at),
INDEX idx_entity (entity_type, entity_id, created_at),
INDEX idx_security_class (security_class, created_at),
INDEX idx_user_created (user_id, created_at),
INDEX idx_event_type (event_type, created_at),
INDEX idx_site (site_id, created_at),
INDEX idx_created (created_at),
INDEX idx_session (session_id, created_at)
) ENGINE=InnoDB;
```
**Security Log Examples:**
- User login: `operation='LOGIN'`, `entity_type='user'`, `security_class='authentication'`, `event_type='SUCCESS'`
- Failed password: `operation='PASSWORD_FAIL'`, `entity_type='user'`, `security_class='authentication'`, `event_type='FAILURE'`
- Permission change: `operation='UPDATE'`, `entity_type='user'`, `security_class='authorization'`
- File access: `operation='ACCESS'`, `entity_type='file'`, `security_class='authorization'`
---
### 2.5 Table: error_audit_log
```sql
CREATE TABLE error_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
-- WHAT: Error details
operation VARCHAR(50) NOT NULL, -- 'ERROR', 'WARNING', 'CRITICAL'
entity_type VARCHAR(50) NOT NULL, -- 'instrument', 'integration', 'database', 'validation'
entity_id VARCHAR(36) NOT NULL, -- Entity where error occurred
error_code VARCHAR(50), -- Specific error code
error_message TEXT, -- Error message
error_details JSON, -- Stack trace, context
previous_value JSON, -- State before error
new_value JSON, -- State after error (if recovered)
-- HOW: Error context
mechanism ENUM('MANUAL', 'AUTOMATIC') NOT NULL DEFAULT 'MANUAL',
application_id VARCHAR(50),
web_page VARCHAR(500),
session_id VARCHAR(100),
event_type VARCHAR(100), -- 'TRANSACTION_ERROR', 'SYSTEM_ERROR', 'VALIDATION_ERROR'
-- WHERE: Error location
site_id VARCHAR(36),
workstation_id VARCHAR(36),
pc_name VARCHAR(100),
ip_address VARCHAR(45),
-- WHO: User or system
user_id VARCHAR(36) NOT NULL, -- User ID or 'SYSTEM'
-- WHEN: Timestamp
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
-- WHY: Error context
reason TEXT, -- Why the error occurred
-- Context: Additional error data
context JSON, -- Related IDs, transaction info, etc.
-- Indexes
INDEX idx_operation_created (operation, created_at),
INDEX idx_entity (entity_type, entity_id, created_at),
INDEX idx_error_code (error_code, created_at),
INDEX idx_event_type (event_type, created_at),
INDEX idx_user_created (user_id, created_at),
INDEX idx_site (site_id, created_at),
INDEX idx_created (created_at)
) ENGINE=InnoDB
PARTITION BY RANGE (YEAR(created_at) * 100 + MONTH(created_at)) (
PARTITION p202601 VALUES LESS THAN (202602),
PARTITION p202602 VALUES LESS THAN (202603),
PARTITION p_future VALUES LESS THAN MAXVALUE
);
```
**Error Log Examples:**
- Transaction error: `operation='ERROR'`, `entity_type='database'`, `event_type='TRANSACTION_ERROR'`
- Instrument error: `operation='ERROR'`, `entity_type='instrument'`, `event_type='SYSTEM_ERROR'`
- Integration error: `operation='ERROR'`, `entity_type='integration'`, `event_type='SYSTEM_ERROR'`
- Validation error: `operation='ERROR'`, `entity_type='validation'`, `event_type='VALIDATION_ERROR'`
---
## 3. Example Audit Entries
### 3.1 Data Log Entry (Patient Update)
```json
{
"id": 15243,
"operation": "UPDATE",
"entity_type": "patient",
"entity_id": "PAT-2026-001234",
"table_name": "patients",
"field_name": null,
"previous_value": {
"NameFirst": "John",
"NameLast": "Doe",
"Phone": "+1-555-0100"
},
"new_value": {
"NameFirst": "Johnny",
"NameLast": "Doe-Smith",
"Phone": "+1-555-0199"
},
"mechanism": "MANUAL",
"application_id": "CLQMS-WEB",
"web_page": "/api/patient/PAT-2026-001234",
"session_id": "sess_abc123",
"event_type": "PATIENT_UPDATE",
"site_id": "SITE-001",
"workstation_id": "WS-001",
"pc_name": "LAB-PC-01",
"ip_address": "192.168.1.100",
"user_id": "USR-001",
"created_at": "2026-02-19T14:30:00Z",
"reason": "Patient requested name change after marriage",
"context": {
"changed_fields": ["NameFirst", "NameLast", "Phone"],
"validation_status": "PASSED"
}
}
```
### 3.2 Service Log Entry (Instrument Communication)
```json
{
"id": 89345,
"operation": "COMMUNICATION",
"entity_type": "instrument",
"entity_id": "INST-001",
"service_class": "communication",
"resource_type": "instrument_communication",
"resource_details": {
"protocol": "HL7",
"port": 2575,
"direction": "INBOUND"
},
"previous_value": { "status": "IDLE" },
"new_value": { "status": "RECEIVING" },
"mechanism": "AUTOMATIC",
"application_id": "INSTRUMENT-SERVICE",
"service_name": "instrument-listener",
"session_id": "svc_inst_001",
"event_type": "RESULT_RECEIVED",
"site_id": "SITE-001",
"workstation_id": "WS-LAB-01",
"pc_name": "LAB-SERVER-01",
"ip_address": "192.168.1.10",
"port": 2575,
"user_id": "SYSTEM",
"created_at": "2026-02-19T14:35:22Z",
"reason": null,
"context": {
"sample_id": "SMP-2026-004567",
"test_count": 5,
"bytes_received": 2048
}
}
```
### 3.3 Security Log Entry (Failed Login)
```json
{
"id": 4521,
"operation": "PASSWORD_FAIL",
"entity_type": "user",
"entity_id": "USR-999",
"security_class": "authentication",
"resource_path": "/api/auth/login",
"previous_value": { "failed_attempts": 2 },
"new_value": { "failed_attempts": 3 },
"mechanism": "MANUAL",
"application_id": "CLQMS-WEB",
"web_page": "/login",
"session_id": "sess_fail_789",
"event_type": "FAILURE",
"site_id": "SITE-002",
"workstation_id": "WS-RECEPTION",
"pc_name": "RECEPTION-PC-02",
"ip_address": "203.0.113.45",
"user_id": "USR-999",
"created_at": "2026-02-19T15:10:05Z",
"reason": null,
"context": {
"lockout_threshold": 5,
"remaining_attempts": 2,
"username_attempted": "john.doe"
}
}
```
### 3.4 Error Log Entry (Database Transaction Failure)
```json
{
"id": 1203,
"operation": "ERROR",
"entity_type": "database",
"entity_id": "DB-PRIMARY",
"error_code": "DB_TXN_001",
"error_message": "Transaction rollback due to deadlock",
"error_details": {
"sql_state": "40001",
"error_number": 1213,
"deadlock_victim": true
},
"previous_value": { "transaction_status": "ACTIVE" },
"new_value": { "transaction_status": "ROLLED_BACK" },
"mechanism": "AUTOMATIC",
"application_id": "CLQMS-WEB",
"web_page": "/api/orders/batch-update",
"session_id": "sess_xyz789",
"event_type": "TRANSACTION_ERROR",
"site_id": "SITE-001",
"workstation_id": "WS-001",
"pc_name": "LAB-PC-01",
"ip_address": "192.168.1.100",
"user_id": "USR-001",
"created_at": "2026-02-19T15:15:30Z",
"reason": "Deadlock detected during batch update",
"context": {
"affected_tables": ["orders", "order_tests"],
"retry_count": 0,
"transaction_id": "txn_20260219151530"
}
}
```
---
## 4. Implementation Strategy
### 4.1 Central Audit Service
```php
<?php
namespace App\Services;
class AuditService
{
/**
* Log a DATA audit event
*/
public static function logData(
string $operation,
string $entityType,
string $entityId,
?string $tableName = null,
?string $fieldName = null,
?array $previousValue = null,
?array $newValue = null,
?string $reason = null,
?array $context = null
): void {
self::log('data_audit_log', [
'operation' => $operation,
'entity_type' => $entityType,
'entity_id' => $entityId,
'table_name' => $tableName,
'field_name' => $fieldName,
'previous_value' => $previousValue ? json_encode($previousValue) : null,
'new_value' => $newValue ? json_encode($newValue) : null,
'mechanism' => 'MANUAL',
'application_id' => 'CLQMS-WEB',
'web_page' => $_SERVER['REQUEST_URI'] ?? null,
'session_id' => session_id(),
'event_type' => strtoupper($entityType) . '_' . strtoupper($operation),
'site_id' => session('site_id'),
'workstation_id' => session('workstation_id'),
'pc_name' => gethostname(),
'ip_address' => $_SERVER['REMOTE_ADDR'] ?? null,
'user_id' => auth()->id() ?? 'SYSTEM',
'reason' => $reason,
'context' => $context ? json_encode($context) : null,
'created_at' => date('Y-m-d H:i:s')
]);
}
/**
* Log a SERVICE audit event
*/
public static function logService(
string $operation,
string $entityType,
string $entityId,
string $serviceClass,
?string $resourceType = null,
?array $resourceDetails = null,
?array $previousValue = null,
?array $newValue = null,
?string $serviceName = null,
?array $context = null
): void {
self::log('service_audit_log', [
'operation' => $operation,
'entity_type' => $entityType,
'entity_id' => $entityId,
'service_class' => $serviceClass,
'resource_type' => $resourceType,
'resource_details' => $resourceDetails ? json_encode($resourceDetails) : null,
'previous_value' => $previousValue ? json_encode($previousValue) : null,
'new_value' => $newValue ? json_encode($newValue) : null,
'mechanism' => 'AUTOMATIC',
'application_id' => $serviceName ?? 'SYSTEM-SERVICE',
'service_name' => $serviceName,
'session_id' => session_id() ?: 'service_session',
'event_type' => strtoupper($serviceClass) . '_' . strtoupper($operation),
'site_id' => session('site_id'),
'workstation_id' => session('workstation_id'),
'pc_name' => gethostname(),
'ip_address' => $_SERVER['REMOTE_ADDR'] ?? null,
'port' => $resourceDetails['port'] ?? null,
'user_id' => 'SYSTEM',
'reason' => null,
'context' => $context ? json_encode($context) : null,
'created_at' => date('Y-m-d H:i:s')
]);
}
/**
* Log a SECURITY audit event
*/
public static function logSecurity(
string $operation,
string $entityType,
string $entityId,
string $securityClass,
?string $eventType = 'SUCCESS',
?string $resourcePath = null,
?array $previousValue = null,
?array $newValue = null,
?string $reason = null,
?array $context = null
): void {
self::log('security_audit_log', [
'operation' => $operation,
'entity_type' => $entityType,
'entity_id' => $entityId,
'security_class' => $securityClass,
'resource_path' => $resourcePath,
'previous_value' => $previousValue ? json_encode($previousValue) : null,
'new_value' => $newValue ? json_encode($newValue) : null,
'mechanism' => 'MANUAL',
'application_id' => 'CLQMS-WEB',
'web_page' => $_SERVER['REQUEST_URI'] ?? null,
'session_id' => session_id(),
'event_type' => $eventType,
'site_id' => session('site_id'),
'workstation_id' => session('workstation_id'),
'pc_name' => gethostname(),
'ip_address' => $_SERVER['REMOTE_ADDR'] ?? null,
'user_id' => auth()->id() ?? 'UNKNOWN',
'reason' => $reason,
'context' => $context ? json_encode($context) : null,
'created_at' => date('Y-m-d H:i:s')
]);
}
/**
* Log an ERROR audit event
*/
public static function logError(
string $entityType,
string $entityId,
string $errorCode,
string $errorMessage,
string $eventType,
?array $errorDetails = null,
?array $previousValue = null,
?array $newValue = null,
?string $reason = null,
?array $context = null
): void {
self::log('error_audit_log', [
'operation' => 'ERROR',
'entity_type' => $entityType,
'entity_id' => $entityId,
'error_code' => $errorCode,
'error_message' => $errorMessage,
'error_details' => $errorDetails ? json_encode($errorDetails) : null,
'previous_value' => $previousValue ? json_encode($previousValue) : null,
'new_value' => $newValue ? json_encode($newValue) : null,
'mechanism' => 'AUTOMATIC',
'application_id' => 'CLQMS-WEB',
'web_page' => $_SERVER['REQUEST_URI'] ?? null,
'session_id' => session_id() ?: 'system',
'event_type' => $eventType,
'site_id' => session('site_id'),
'workstation_id' => session('workstation_id'),
'pc_name' => gethostname(),
'ip_address' => $_SERVER['REMOTE_ADDR'] ?? null,
'user_id' => auth()->id() ?? 'SYSTEM',
'reason' => $reason,
'context' => $context ? json_encode($context) : null,
'created_at' => date('Y-m-d H:i:s')
]);
}
/**
* Generic log method with async support
*/
private static function log(string $table, array $data): void
{
// For high-volume operations, dispatch to queue
if (in_array($table, ['service_audit_log', 'error_audit_log'])) {
self::dispatchAuditJob($table, $data);
} else {
// Direct insert for data and security logs
\Config\Database::connect()->table($table)->insert($data);
}
}
private static function dispatchAuditJob(string $table, array $data): void
{
// Implementation: Queue the audit entry for async processing
// This prevents blocking user operations during high-volume logging
}
}
```
---
## 5. Query Patterns
### 5.1 Common Audit Queries
```sql
-- View patient history (DATA log)
SELECT * FROM data_audit_log
WHERE entity_type = 'patient'
AND entity_id = 'PAT-2026-001234'
ORDER BY created_at DESC;
-- User activity report
SELECT operation, entity_type, COUNT(*) as count
FROM data_audit_log
WHERE user_id = 'USR-001'
AND created_at > DATE_SUB(NOW(), INTERVAL 7 DAY)
GROUP BY operation, entity_type;
-- Instrument communication history (SERVICE log)
SELECT * FROM service_audit_log
WHERE entity_type = 'instrument'
AND entity_id = 'INST-001'
AND operation = 'COMMUNICATION'
ORDER BY created_at DESC;
-- Failed login attempts (SECURITY log)
SELECT * FROM security_audit_log
WHERE operation IN ('PASSWORD_FAIL', 'ACCESS_DENIED')
AND event_type = 'FAILURE'
AND created_at > DATE_SUB(NOW(), INTERVAL 24 HOUR)
ORDER BY created_at DESC;
-- Recent errors (ERROR log)
SELECT * FROM error_audit_log
WHERE created_at > DATE_SUB(NOW(), INTERVAL 1 HOUR)
AND event_type = 'CRITICAL'
ORDER BY created_at DESC;
-- Find changes to specific field
SELECT * FROM data_audit_log
WHERE table_name = 'patients'
AND field_name = 'Phone'
AND entity_id = 'PAT-2026-001234'
ORDER BY created_at DESC;
```
---
## 6. Migration Plan
### Phase 1: Foundation (Week 1)
- [ ] Drop existing unused tables (patreglog, patvisitlog, specimenlog)
- [ ] Create 4 new audit tables with partitioning
- [ ] Create AuditService class
- [ ] Add database indexes
### Phase 2: Core Implementation (Week 2)
- [ ] Integrate data_audit_log into Patient model
- [ ] Integrate data_audit_log into Order/Test models
- [ ] Integrate data_audit_log into Master data models
- [ ] Integrate security_audit_log into authentication
### Phase 3: Service & Error Logging (Week 3)
- [ ] Implement service_audit_log for instrument communication
- [ ] Implement service_audit_log for printing/messaging
- [ ] Implement error_audit_log for database errors
- [ ] Implement error_audit_log for instrument errors
- [ ] Implement error_audit_log for integration errors
### Phase 4: API & Optimization (Week 4)
- [ ] Create unified API endpoint for querying all log types
- [ ] Add filters by log_type, date range, user, entity
- [ ] Implement async logging queue
- [ ] Add log export functionality (CSV/PDF)
---
## 7. Retention Strategy (TBD)
| Table | Proposed Retention | Notes |
|-------|-------------------|-------|
| `data_audit_log` | 7 years | Patient data compliance |
| `service_audit_log` | 2 years | High volume, operational only |
| `security_audit_log` | Permanent | Compliance and forensics |
| `error_audit_log` | 5 years | Debugging and incident analysis |
---
## 8. Key Design Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| **Table Count** | 4 tables | Separates by log type, different retention needs |
| **5W1H** | All 6 dimensions captured | Complete audit trail per section 4.2.1.20 |
| **Mechanism** | MANUAL vs AUTOMATIC | Distinguishes user vs instrument operations |
| **User for AUTO** | 'SYSTEM' | Clear identification of automatic operations |
| **JSON Storage** | previous_value, new_value, context | Flexible schema evolution |
| **Partitioning** | Monthly for high-volume tables | Manage service and error log growth |
| **Async Logging** | Yes for service/error logs | Don't block user operations |
---
*Document Version: 2.0*
*Based on: Section 4.2.1.20 Error Management*
*Date: February 20, 2026*

482
src/projects/clqms01/suggestion/audit-logging-plan.md
View File

@ -1,482 +0,0 @@
---
layout: clqms-post.njk
tags: clqms
title: "CLQMS: Audit Logging Architecture Plan"
description: "Comprehensive audit trail strategy for tracking changes across master data, patient records, and laboratory operations"
date: 2026-02-19
order: 4
---
# Audit Logging Architecture Plan for CLQMS
> **Clinical Laboratory Quality Management System (CLQMS)** - A comprehensive audit trail strategy for tracking changes across master data, patient records, and laboratory operations.
---
## Executive Summary
This document outlines a unified audit logging architecture for CLQMS, designed to provide complete traceability of data changes while maintaining optimal performance and maintainability. The approach separates audit logs into three domain-specific tables, utilizing JSON for flexible value storage.
---
## 1. Current State Analysis
### Existing Audit Infrastructure
| Aspect | Current Status |
|--------|---------------|
| **Database Tables** | 3 tables exist in migrations (patreglog, patvisitlog, specimenlog) |
| **Implementation** | Tables created but not actively used |
| **Structure** | Fixed column approach (FldName, FldValuePrev) |
| **Code Coverage** | No models or controllers implemented |
| **Application Logging** | Basic CodeIgniter file logging for debug/errors |
### Pain Points Identified
- ❌ **3 separate tables** with nearly identical schemas
- ❌ **Fixed column structure** - rigid and requires schema changes for new entities
- ❌ **No implementation** - audit tables exist but aren't populated
- ❌ **Maintenance overhead** - adding new entities requires new migrations
---
## 2. Proposed Architecture
### 2.1 Domain Separation
We categorize audit logs by **data domain** and **access patterns**:
| Table | Domain | Volume | Retention | Use Case |
|-------|--------|--------|-----------|----------|
| `master_audit_log` | Reference Data | Low | Permanent | Organizations, Users, ValueSets |
| `patient_audit_log` | Patient Records | Medium | 7 years | Demographics, Contacts, Insurance |
| `order_audit_log` | Operations | High | 2 years | Orders, Tests, Specimens, Results |
### 2.2 Unified Table Structure
#### Master Audit Log
```sql
CREATE TABLE master_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
entity_type VARCHAR(50) NOT NULL, -- 'organization', 'user', 'valueset'
entity_id VARCHAR(36) NOT NULL, -- UUID or primary key
action ENUM('CREATE', 'UPDATE', 'DELETE', 'PATCH') NOT NULL,
old_values JSON NULL, -- Complete snapshot before change
new_values JSON NULL, -- Complete snapshot after change
changed_fields JSON, -- Array of modified field names
-- Context
user_id VARCHAR(36),
site_id VARCHAR(36),
ip_address VARCHAR(45),
user_agent VARCHAR(500),
app_version VARCHAR(20),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_entity (entity_type, entity_id),
INDEX idx_created (created_at),
INDEX idx_user (user_id, created_at)
) ENGINE=InnoDB;
```
#### Patient Audit Log
```sql
CREATE TABLE patient_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
entity_type VARCHAR(50) NOT NULL, -- 'patient', 'contact', 'insurance'
entity_id VARCHAR(36) NOT NULL,
patient_id VARCHAR(36), -- Context FK for patient
action ENUM('CREATE', 'UPDATE', 'DELETE', 'MERGE', 'UNMERGE') NOT NULL,
old_values JSON NULL,
new_values JSON NULL,
changed_fields JSON,
reason TEXT, -- Why the change was made
-- Context
user_id VARCHAR(36),
site_id VARCHAR(36),
ip_address VARCHAR(45),
session_id VARCHAR(100),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_entity (entity_type, entity_id),
INDEX idx_patient (patient_id, created_at),
INDEX idx_created (created_at),
INDEX idx_user (user_id, created_at)
) ENGINE=InnoDB;
```
#### Order/Test Audit Log
```sql
CREATE TABLE order_audit_log (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
entity_type VARCHAR(50) NOT NULL, -- 'order', 'test', 'specimen', 'result'
entity_id VARCHAR(36) NOT NULL,
-- Context FKs
patient_id VARCHAR(36),
visit_id VARCHAR(36),
order_id VARCHAR(36),
action ENUM('CREATE', 'UPDATE', 'DELETE', 'CANCEL', 'REORDER', 'COLLECT', 'RESULT') NOT NULL,
old_values JSON NULL,
new_values JSON NULL,
changed_fields JSON,
status_transition VARCHAR(100), -- e.g., 'pending->collected'
-- Context
user_id VARCHAR(36),
site_id VARCHAR(36),
device_id VARCHAR(36), -- Instrument/edge device
ip_address VARCHAR(45),
session_id VARCHAR(100),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_entity (entity_type, entity_id),
INDEX idx_order (order_id, created_at),
INDEX idx_patient (patient_id, created_at),
INDEX idx_created (created_at),
INDEX idx_user (user_id, created_at)
) ENGINE=InnoDB;
```
---
## 3. JSON Value Structure
### Example Audit Entry
```json
{
"id": 15243,
"entity_type": "patient",
"entity_id": "PAT-2026-001234",
"action": "UPDATE",
"old_values": {
"NameFirst": "John",
"NameLast": "Doe",
"Gender": "M",
"BirthDate": "1990-01-15",
"Phone": "+1-555-0100"
},
"new_values": {
"NameFirst": "Johnny",
"NameLast": "Doe-Smith",
"Gender": "M",
"BirthDate": "1990-01-15",
"Phone": "+1-555-0199"
},
"changed_fields": ["NameFirst", "NameLast", "Phone"],
"user_id": "USR-001",
"site_id": "SITE-001",
"created_at": "2026-02-19T14:30:00Z"
}
```
### Benefits of JSON Approach
✅ **Schema Evolution** - Add new fields without migrations
✅ **Complete Snapshots** - Reconstruct full record state at any point
✅ **Flexible Queries** - MySQL 8.0+ supports JSON indexing and extraction
✅ **Audit Integrity** - Store exactly what changed, no data loss
---
## 4. Implementation Strategy
### 4.1 Central Audit Service
```php
<?php
namespace App\Services;
class AuditService
{
/**
* Log an audit event to the appropriate table
*/
public static function log(
string $category, // 'master', 'patient', 'order'
string $entityType, // e.g., 'patient', 'order'
string $entityId,
string $action,
?array $oldValues = null,
?array $newValues = null,
?string $reason = null,
?array $context = null
): void {
$changedFields = self::calculateChangedFields($oldValues, $newValues);
$data = [
'entity_type' => $entityType,
'entity_id' => $entityId,
'action' => $action,
'old_values' => $oldValues ? json_encode($oldValues) : null,
'new_values' => $newValues ? json_encode($newValues) : null,
'changed_fields' => json_encode($changedFields),
'user_id' => auth()->id() ?? 'SYSTEM',
'site_id' => session('site_id') ?? 'MAIN',
'created_at' => date('Y-m-d H:i:s')
];
// Route to appropriate table
$table = match($category) {
'master' => 'master_audit_log',
'patient' => 'patient_audit_log',
'order' => 'order_audit_log',
default => throw new \InvalidArgumentException("Unknown category: $category")
};
// Async logging recommended for high-volume operations
self::dispatchAuditJob($table, $data);
}
private static function calculateChangedFields(?array $old, ?array $new): array
{
if (!$old || !$new) return [];
$changes = [];
$allKeys = array_unique(array_merge(array_keys($old), array_keys($new)));
foreach ($allKeys as $key) {
if (($old[$key] ?? null) !== ($new[$key] ?? null)) {
$changes[] = $key;
}
}
return $changes;
}
}
```
### 4.2 Model Integration
```php
<?php
namespace App\Models;
use App\Services\AuditService;
class PatientModel extends BaseModel
{
protected $table = 'patients';
protected $primaryKey = 'PatientID';
protected function logAudit(
string $action,
?array $oldValues = null,
?array $newValues = null
): void {
AuditService::log(
category: 'patient',
entityType: 'patient',
entityId: $this->getPatientId(),
action: $action,
oldValues: $oldValues,
newValues: $newValues
);
}
// Override save method to auto-log
public function save($data): bool
{
$oldData = $this->find($data['PatientID'] ?? null);
$result = parent::save($data);
if ($result) {
$this->logAudit(
$oldData ? 'UPDATE' : 'CREATE',
$oldData?->toArray(),
$this->find($data['PatientID'])->toArray()
);
}
return $result;
}
}
```
---
## 5. Query Patterns & Performance
### 5.1 Common Queries
```sql
-- View entity history
SELECT * FROM patient_audit_log
WHERE entity_type = 'patient'
AND entity_id = 'PAT-2026-001234'
ORDER BY created_at DESC;
-- User activity report
SELECT entity_type, action, COUNT(*) as count
FROM patient_audit_log
WHERE user_id = 'USR-001'
AND created_at > DATE_SUB(NOW(), INTERVAL 7 DAY)
GROUP BY entity_type, action;
-- Find all changes to a specific field
SELECT * FROM order_audit_log
WHERE JSON_CONTAINS(changed_fields, '"result_value"')
AND patient_id = 'PAT-001'
AND created_at > '2026-01-01';
```
### 5.2 Partitioning Strategy (Order/Test)
For high-volume tables, implement monthly partitioning:
```sql
CREATE TABLE order_audit_log (
-- ... columns
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
) ENGINE=InnoDB
PARTITION BY RANGE (YEAR(created_at) * 100 + MONTH(created_at)) (
PARTITION p202601 VALUES LESS THAN (202602),
PARTITION p202602 VALUES LESS THAN (202603),
PARTITION p_future VALUES LESS THAN MAXVALUE
);
```
---
## 6. Soft Delete Handling
Soft deletes ARE captured as audit entries with complete snapshots:
```php
// When soft deleting a patient:
AuditService::log(
category: 'patient',
entityType: 'patient',
entityId: $patientId,
action: 'DELETE',
oldValues: $fullRecordBeforeDelete, // Complete last known state
newValues: null,
reason: 'Patient requested data removal'
);
```
This ensures:
- ✅ Full audit trail even for deleted records
- ✅ Compliance with "right to be forgotten" (GDPR)
- ✅ Ability to restore accidentally deleted records
---
## 7. Migration Plan
### Phase 1: Foundation (Week 1)
- [ ] Drop existing unused tables (patreglog, patvisitlog, specimenlog)
- [ ] Create new audit tables with JSON columns
- [ ] Create AuditService class
- [ ] Add database indexes
### Phase 2: Core Implementation (Week 2)
- [ ] Integrate AuditService into Patient model
- [ ] Integrate AuditService into Order model
- [ ] Integrate AuditService into Master data models
- [ ] Add audit trail to authentication events
### Phase 3: API & UI (Week 3)
- [ ] Create API endpoints for querying audit logs
- [ ] Build admin interface for audit review
- [ ] Add audit export functionality (CSV/PDF)
### Phase 4: Optimization (Week 4)
- [ ] Implement async logging queue
- [ ] Add table partitioning for order_audit_log
- [ ] Set up retention policies and archiving
- [ ] Performance testing and tuning
---
## 8. Retention & Archiving Strategy
| Table | Retention Period | Archive Action |
|-------|---------------|----------------|
| `master_audit_log` | Permanent | None (keep forever) |
| `patient_audit_log` | 7 years | Move to cold storage after 7 years |
| `order_audit_log` | 2 years | Partition rotation: drop old partitions |
### Automated Maintenance
```sql
-- Monthly job: Archive old patient audit logs
INSERT INTO patient_audit_log_archive
SELECT * FROM patient_audit_log
WHERE created_at < DATE_SUB(NOW(), INTERVAL 7 YEAR);
DELETE FROM patient_audit_log
WHERE created_at < DATE_SUB(NOW(), INTERVAL 7 YEAR);
-- Monthly job: Drop old order partitions
ALTER TABLE order_audit_log DROP PARTITION p202501;
```
---
## 9. Questions for Stakeholders
Before implementation, please confirm:
1. **Retention Policy**: Are the proposed retention periods (master=forever, patient=7 years, order=2 years) compliant with your regulatory requirements?
2. **Async vs Sync**: Should audit logging be synchronous (block on failure) or asynchronous (queue-based)? Recommended: async for order/test operations.
3. **Archive Storage**: Where should archived audit logs be stored? Options: separate database, file storage (S3), or compressed tables.
4. **User Access**: Which user roles need access to audit trails? Should users see their own audit history?
5. **Compliance**: Do you need specific compliance features (e.g., HIPAA audit trail requirements, 21 CFR Part 11 for FDA)?
---
## 10. Key Design Decisions Summary
| Decision | Choice | Rationale |
|----------|--------|-----------|
| **Table Count** | 3 tables | Separates concerns, optimizes queries, different retention |
| **JSON vs Columns** | JSON for values | Flexible, handles schema changes, complete snapshots |
| **Full vs Diff** | Full snapshots | Easier to reconstruct history, no data loss |
| **Soft Deletes** | Captured in audit | Compliance and restore capability |
| **Partitioning** | Order table only | High volume, time-based queries |
| **Async Logging** | Recommended | Don't block user operations |
---
## Conclusion
This unified audit logging architecture provides:
✅ **Complete traceability** across all data domains
✅ **Regulatory compliance** with proper retention
✅ **Performance optimization** through domain separation
✅ **Flexibility** via JSON value storage
✅ **Maintainability** with centralized service
The approach balances audit integrity with system performance, ensuring CLQMS can scale while maintaining comprehensive audit trails.
---
*Document Version: 1.0*
*Author: CLQMS Development Team*
*Date: February 19, 2026*

4
src/projects/clqms01/suggestion/index.md
View File

@ -41,7 +41,7 @@ When documenting suggestions, please include:
### [High] Audit Logging Architecture Plan
**Description:** Comprehensive audit trail strategy for tracking changes across master data, patient records, and laboratory operations.
[View Plan](./audit-logging-plan.md)
[View Plan](./004-audit-logging-plan.md)
### [TBD] Add new suggestions here
@ -59,4 +59,4 @@ When documenting suggestions, please include:
-->
---
_Last updated: 2026-01-09 08:40:21_
_Last updated: 2026-02-20 00:00:00_