clqms-be/.serena/memories/important_patterns.md

# CLQMS Important Patterns & Special Considerations

## JWT Authentication Pattern

### How Authentication Works
1. User logs in via `/api/v2/auth/login` or `/api/login`
2. Server generates JWT token with payload containing user info
3. Token stored in HTTP-only cookie named `token`
4. `AuthFilter` checks for token on protected routes
5. Token decoded using `JWT_SECRET` from environment
6. If invalid/missing, returns 401 Unauthorized

### AuthFilter Location
- File: `app/Filters/AuthFilter.php`
- Registered in: `app/Config/Filters.php` as `'auth'` alias

### Protected Routes
Routes are protected by adding `'filter' => 'auth'` to route group:
```php
$routes->group('api', ['filter' => 'auth'], function ($routes) {
    $routes->get('patient', 'Patient\PatientController::index');
    $routes->post('patient', 'Patient\PatientController::create');
});
```

### JWT Token Structure
```php
$payload = [
    'iss' => 'localhost',           // Issuer
    'aud' => 'localhost',           // Audience
    'iat' => time(),                // Issued at
    'nbf' => time(),                // Not before
    'exp' => time() + 3600,         // Expiration (1 hour)
    'uid' => 1,                     // User ID
    'email' => 'admin@admin.com'    // User email
];
```

### Generating JWT Token (for Tests)
```php
$key = getenv('JWT_SECRET') ?: 'my-secret-key';
$payload = [
    'iss' => 'localhost',
    'aud' => 'localhost',
    'iat' => time(),
    'exp' => time() + 3600,
    'uid' => 1,
    'email' => 'admin@admin.com'
];
$token = \Firebase\JWT\JWT::encode($payload, $key, 'HS256');
```

### Injecting Token in Tests
```php
protected $token;

protected function setUp(): void {
    parent::setUp();
    // Generate token as shown above
    $this->token = $encodedToken;
}

public function testEndpoint() {
    $this->withHeaders(['Cookie' => 'token=' . $this->token])
        ->get('api/patient')
        ->assertStatus(200);
}
```

### Public Routes (No Authentication)
```php
$routes->group('api', function ($routes) {
    $routes->post('login', 'AuthController::login');  // No auth filter
});

$routes->group('api/demo', function ($routes) {
    $routes->post('order', 'Test\DemoOrderController::createDemoOrder');  // No auth
});
```

## UTC Date Handling Pattern

### BaseModel Date Normalization
`BaseModel` automatically handles UTC date conversion:

**Before Insert/Update:**
```php
protected $beforeInsert = ['normalizeDatesToUTC'];
protected $beforeUpdate = ['normalizeDatesToUTC'];
```
- Converts local dates to UTC before database operations
- Uses helper: `convert_array_to_utc($data)`

**After Find/Insert/Update:**
```php
protected $afterFind = ['convertDatesToUTCISO'];
protected $afterInsert = ['convertDatesToUTCISO'];
protected $afterUpdate = ['convertDatesToUTCISO'];
```
- Converts UTC dates to ISO 8601 format for API responses
- Uses helper: `convert_array_to_utc_iso($data)`

### Date Formats

**Database Format (UTC):**
- Format: `Y-m-d H:i:s`
- Timezone: UTC
- Example: `2026-02-11 23:55:08`

**API Response Format (ISO 8601):**
- Format: `Y-m-d\TH:i:s\Z`
- Example: `2026-02-11T23:55:08Z`

### Manual Date Conversion
```php
// Convert to UTC for storage
$birthdate = new \DateTime('1990-05-15', new \DateTimeZone('Asia/Jakarta'));
$utcDate = $birthdate->setTimezone(new \DateTimeZone('UTC'))->format('Y-m-d H:i:s');

// Format for display (ISO 8601)
$displayDate = $utcDate->format('Y-m-d\TH:i:s\Z');
```

### Date Fields in Database
- `CreateDate` - Record creation timestamp
- `DelDate` - Soft delete timestamp (null if not deleted)
- `TimeOfDeath` - Death timestamp (patient)
- Other date fields vary by table

## Soft Delete Pattern

### How Soft Deletes Work
- All transactional tables use `DelDate` field for soft deletes
- Setting `DelDate` to current timestamp marks record as deleted
- Queries automatically exclude records where `DelDate` is not null
- Data remains in database for audit trail

### BaseModel Soft Delete Configuration
```php
protected $useSoftDeletes = true;
protected $deletedField = 'DelDate';
```

### Manual Soft Delete
```php
// In controller
$this->db->table('patient')
    ->where('InternalPID', $InternalPID)
    ->update(['DelDate' => date('Y-m-d H:i:s')]);

// Or using model
$this->where('InternalPID', $id)->delete(); // BaseModel handles this
```

### Querying Deleted Records
If you need to include soft-deleted records:
```php
$this->withDeleted()->findAll();
```

### Including Deleted Records in Join
When joining with tables that might have soft-deleted records:
```php
->join('patatt', 'patatt.InternalPID = patient.InternalPID and patatt.DelDate is null', 'left')
```

## ValueSet Pattern

### ValueSet Library Location
- File: `app/Libraries/ValueSet.php`
- Data directory: `app/Libraries/Data/valuesets/`

### Getting ValueSet Data
```php
use App\Libraries\ValueSet;

// Get all values for a lookup (formatted for dropdowns)
$gender = ValueSet::get('sex');
// Returns: [{"value"=>"1","label":"Female"},{"value"=>"2","label":"Male"},...]

// Get raw data without formatting
$raw = ValueSet::getRaw('sex');
// Returns: [{"key":"1","value":"Female"},{"key":"2","value":"Male"},...]

// Get single label by key
$label = ValueSet::getLabel('sex', '1'); // Returns 'Female'

// Get key/value pairs for select inputs
$options = ValueSet::getOptions('sex');
// Returns: [["key"=>"1","value"=>"Female"],...]
```

### Transforming Database Results
```php
$patients = $this->model->getPatients();
$patients = ValueSet::transformLabels($patients, [
    'Sex' => 'sex',
    'Priority' => 'order_priority',
    'MaritalStatus' => 'marital_status',
]);
// Adds fields: SexLabel, PriorityLabel, MaritalStatusLabel
```

### ValueSet JSON File Format
```json
{
  "name": "sex",
  "description": "Patient gender",
  "values": [
    {"key": "1", "value": "Female"},
    {"key": "2", "value": "Male"},
    {"key": "3", "value": "Unknown"}
  ]
}
```

### Clearing ValueSet Cache
After modifying ValueSet JSON files:
```php
ValueSet::clearCache();
```

### Common ValueSets
| Name | Description | Example Values |
|------|-------------|----------------|
| `sex` | Patient gender | Female, Male, Unknown |
| `marital_status` | Marital status | Single, Married, Divorced |
| `race` | Ethnicity | Jawa, Sunda, Batak, etc. |
| `order_priority` | Order priority | Stat, ASAP, Routine, Preop |
| `order_status` | Order lifecycle | STC, SCtd, SArrv, SRcvd |
| `specimen_type` | Specimen types | BLD, SER, PLAS, UR, CSF |
| `specimen_status` | Specimen status | Ordered, Collected, Received |
| `result_status` | Result validation | Preliminary, Final, Corrected |
| `test_type` | Test definition types | TEST, PARAM, CALC, GROUP |

## Error Handling Pattern

### Controller Error Handling
```php
public function create() {
    $input = $this->request->getJSON(true);

    if (!$this->validateData($input, $this->rules)) {
        return $this->failValidationErrors($this->validator->getErrors());
    }

    try {
        $result = $this->model->createPatient($input);
        return $this->respondCreated([
            'status' => 'success',
            'message' => "data {$result} created successfully"
        ]);
    } catch (\Exception $e) {
        return $this->failServerError('Something went wrong: ' . $e->getMessage());
    }
}
```

### Model Error Handling
```php
public function createPatient(array $input) {
    $db = \Config\Database::connect();
    $db->transBegin();

    try {
        $this->insert($input);
        $newId = $this->getInsertID();
        $this->checkDbError($db, 'Insert patient');
        
        $db->transCommit();
        return $newId;
    } catch (\Exception $e) {
        $db->transRollback();
        throw $e;
    }
}

private function checkDbError($db, string $context) {
    $error = $db->error();
    if (!empty($error['code'])) {
        throw new \Exception(
            "{$context} failed: {$error['code']} - {$error['message']}"
        );
    }
}
```

### ResponseTrait Methods
```php
// Success (200)
return $this->respond(['status' => 'success', 'data' => $data], 200);

// Created (201)
return $this->respondCreated(['status' => 'success', 'message' => 'Created']);

// Validation Error (400)
return $this->failValidationErrors($errors);

// Not Found (404)
return $this->failNotFound('Resource not found');

// Server Error (500)
return $this->failServerError('Error message');

// Unauthorized (401) - use in AuthFilter
return Services::response()
    ->setStatusCode(401)
    ->setJSON(['status' => 'failed', 'message' => 'Unauthorized']);
```

## Database Transaction Pattern

### Standard Transaction Pattern
```php
$db = \Config\Database::connect();
$db->transBegin();

try {
    // Insert/Update main record
    $this->insert($data);
    $id = $this->getInsertID();
    $this->checkDbError($db, 'Insert main');

    // Insert related records
    $relatedModel->insert($relatedData);
    $this->checkDbError($db, 'Insert related');

    $db->transCommit();
    return $id;
} catch (\Exception $e) {
    $db->transRollback();
    throw $e;
}
```

### When to Use Transactions
- Multi-table operations (main record + related records)
- Operations that must be atomic
- When you need to rollback all changes if any operation fails

## Nested Data Handling Pattern

### Extracting Nested Data
```php
// Extract nested data before filtering
$patIdt = $input['PatIdt'] ?? null;
$patCom = $input['PatCom'] ?? null;
$patAtt = $input['PatAtt'] ?? null;

// Remove nested arrays that don't belong to parent table
unset($input['PatIdt'], $input['PatCom'], $input['PatAtt']);

// Now $input only contains fields for main table
```

### Processing Nested Data
```php
// Insert main record
$this->insert($input);
$mainId = $this->getInsertID();

// Process related records
if (!empty($patIdt)) {
    $modelPatIdt->createPatIdt($patIdt, $mainId);
}

if (!empty($patCom)) {
    $modelPatCom->createPatCom($patCom, $mainId);
}

if (!empty($patAtt) && is_array($patAtt)) {
    foreach ($patAtt as $address) {
        $modelPatAtt->createPatAtt($address, $mainId);
    }
}
```

## Foreign Key Handling Pattern

### Array-Based Foreign Keys
```php
// Handle array of related records
if (!empty($input['LinkTo']) && is_array($input['LinkTo'])) {
    $internalPids = array_column($input['LinkTo'], 'InternalPID');
    $input['LinkTo'] = implode(',', $internalPids);
}
$input['LinkTo'] = empty($input['LinkTo']) ? null : $input['LinkTo'];
```

### Single Record Foreign Key
```php
// Handle single related record
if (!empty($input['Custodian']) && is_array($input['Custodian'])) {
    $input['Custodian'] = $input['Custodian']['InternalPID'] ?? null;
    if ($input['Custodian'] !== null) {
        $input['Custodian'] = (int) $input['Custodian'];
    }
}
```

## Validation Rules Pattern

### Dynamic Validation Rules
```php
// Override validation rules based on input type
$type = $input['PatIdt']['IdentifierType'] ?? null;
$identifierRulesMap = [
    'KTP'  => 'required|regex_match[/^[0-9]{16}$/]',      // 16 digits
    'PASS' => 'required|regex_match[/^[A-Za-z0-9]{1,9}$/]',   // alphanumeric max 9
    'SSN'  => 'required|regex_match[/^[0-9]{9}$/]',          // numeric 9 digits
    'SIM'  => 'required|regex_match[/^[0-9]{19,20}$/]',       // numeric 19-20 digits
    'KTAS' => 'required|regex_match[/^[0-9]{11}$/]',          // numeric 11 digits
];

if ($type && is_string($type)) {
    $identifierRule = $identifierRulesMap[$type] ?? 'permit_empty|max_length[255]';
    $this->rules['PatIdt.IdentifierType'] = 'required';
    $this->rules['PatIdt.Identifier'] = $identifierRule;
} else {
    $this->rules['PatIdt.IdentifierType'] = 'permit_empty';
    $this->rules['PatIdt.Identifier'] = 'permit_empty|max_length[255]';
}
```

### Common Validation Rules
```php
// Required field
'NameFirst' => 'required'

// String with letters, apostrophes, spaces
'NameFirst' => 'regex_match[/^[A-Za-z\'\. ]+$/]'

// Alphanumeric
'PatientID' => 'regex_match[/^[A-Za-z0-9]+$/]'

// Numeric
'InternalPID' => 'is_natural'

// Email
'EmailAddress1' => 'valid_email'

// Phone number with optional + and 8-15 digits
'Phone' => 'regex_match[/^\\+?[0-9]{8,15}$/]'

// Date
'Birthdate' => 'required'

// Permit empty
'AlternatePID' => 'permit_empty'
```

## Edge API Pattern

### Edge API Purpose
Integration with laboratory instruments via `tiny-edge` middleware for:
- Receiving instrument results
- Sending pending orders to instruments
- Acknowledging order delivery
- Logging instrument status

### Edge API Endpoints
```php
// Receive instrument results
POST /api/edge/results

// Fetch pending orders for instrument
GET /api/edge/orders?instrument=coulter_counter

// Acknowledge order delivery
POST /api/edge/orders/:id/ack

// Log instrument status
POST /api/edge/status
```

### Edge API Workflow
```
Instrument → tiny-edge → POST /api/edge/results → edgeres table
                                                           ↓
                                                    [Manual/Auto Processing]
                                                           ↓
                                                  patres table (patient results)
```

### Staging Table Pattern
- Raw results stored in `edgeres` table first
- Allows validation before processing to main tables
- Rerun handling via `AspCnt` field (attempt count)

## Security Considerations

### Environment Variables (NEVER COMMIT)
- `JWT_SECRET` - JWT signing key
- Database credentials (username, password)
- API keys for external services

### Input Validation
- Always validate user input with `$this->validateData($input, $rules)`
- Use CodeIgniter validation rules
- Custom regex patterns for specific formats

### SQL Injection Prevention
- Use Query Builder (parameter binding)
- Never concatenate user input into raw SQL
- If using raw SQL, escape inputs: `$this->db->escape($value)`

### Output Escaping
- ResponseTrait automatically handles JSON encoding
- For HTML output (if needed), use `esc()` helper

## API Documentation Pattern

### Critical Requirement
After modifying ANY controller, **MUST** update `public/api-docs.yaml`:
- Add new endpoints
- Update existing endpoint schemas
- Document request/response formats
- Include field names, types, and validation rules
- Add example requests/responses

### API Documentation Format
```yaml
paths:
  /api/patient:
    get:
      summary: List patients
      parameters:
        - name: InternalPID
          in: query
          schema:
            type: integer
      responses:
        '200':
          description: Success
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: string
                  message:
                    type: string
                  data:
                    type: array
                    items:
                      $ref: '#/components/schemas/Patient'
```

## Testing Pattern

### Feature Test Structure
```php
<?php
namespace Tests\Feature;

use CodeIgniter\Test\FeatureTestTrait;
use CodeIgniter\Test\CIUnitTestCase;

class PatientCreateTest extends CIUnitTestCase
{
    use FeatureTestTrait;

    protected $token;

    protected function setUp(): void {
        parent::setUp();
        // Generate JWT token
        $this->token = $this->generateToken();
    }

    public function testCanCreatePatient() {
        $this->withHeaders(['Cookie' => 'token=' . $this->token])
            ->post('api/patient', [
                'PatientID' => 'PT001',
                'NameFirst' => 'John',
                'NameLast' => 'Doe',
                'Sex' => '2',
                'Birthdate' => '1990-05-15'
            ])
            ->assertStatus(200)
            ->assertJSONFragment(['status' => 'success']);
    }
}
```

### Test Commands
```bash
# Run all tests
vendor/bin/phpunit

# Run specific test file (IMPORTANT for debugging)
vendor/bin/phpunit tests/feature/Patient/PatientCreateTest.php

# Run with coverage
vendor/bin/phpunit --coverage-text=tests/coverage.txt --coverage-html=tests/coverage/ -d memory_limit=1024m
```

## Common Pitfalls

### Forgetting to Clear ValueSet Cache
**Problem:** Modified ValueSet JSON file but changes not reflected
**Solution:** Call `ValueSet::clearCache()` after modifications

### Not Updating api-docs.yaml
**Problem:** API documentation out of sync with implementation
**Solution:** Always update `public/api-docs.yaml` after controller changes

### Missing Soft Delete Filter in Joins
**Problem:** Query returns soft-deleted records from joined tables
**Solution:** Add `and table.DelDate is null` to join condition

### Incorrect Date Format
**Problem:** Dates not in UTC format causing issues
**Solution:** BaseModel handles this, but manual dates must be in `Y-m-d H:i:s` UTC

### Validation Rule Not Applied
**Problem:** Input not validated, invalid data inserted
**Solution:** Always call `$this->validateData($input, $rules)` before processing

### Transaction Not Rolled Back on Error
**Problem:** Partial data left in database on error
**Solution:** Always use try-catch with `$db->transRollback()`

### Not Using BaseModel for Date Handling
**Problem:** Dates not normalized to UTC
**Solution:** All models must extend `BaseModel`

## Debugging Tips

### Enable Detailed Error Messages
In development environment (`.env`):
```env
CI_ENVIRONMENT = development
```

### Log SQL Queries
Add to database config or temporarily enable:
```php
$db->setQueryLog(true);
$log = $db->getQueryLog();
```

### Check Application Logs
```bash
# Windows
type writable\logs\log-*.log

# Unix/Linux
tail -f writable/logs/log-*.log
```

### Add Temporary Debug Output
```php
var_dump($variable); die();
// or
log_message('debug', 'Debug info: ' . json_encode($data));
```

### Run Specific Test for Debugging
```bash
vendor/bin/phpunit tests/feature/SpecificTest.php --filter testMethodName
```

### Check Database State
```bash
php spark db:table patient
# or use MySQL Workbench, phpMyAdmin, etc.
```