navidocs/STACKCP_ARCHITECTURE_ANALYSIS.md
ggq-admin eaf9fae275 docs: Add complete NaviDocs handover documentation and StackCP analysis
This commit finalizes the NaviDocs MVP documentation with comprehensive handover materials.

## Documentation Added:

1. **NAVIDOCS_HANDOVER.md** - Complete project handover (65% MVP complete)
   - Executive summary and current status
   - Repository structure and component details
   - Testing results and known issues
   - Deployment options (StackCP vs VPS)
   - Next steps and risk assessment
   - Success metrics and recommendations

2. **StackCP Analysis Documents**:
   - ANALYSIS_INDEX.md - Master overview
   - STACKCP_ARCHITECTURE_ANALYSIS.md - Technical deep-dive
   - STACKCP_DEBATE_BRIEF.md - Deployment decision framework
   - STACKCP_QUICK_REFERENCE.md - Fast decision-making tool

## Current State Summary:

**Completed** (65% MVP):
-  Database schema (13 tables, fully normalized)
-  OCR pipeline (3 options: Tesseract 85%, Google Drive, Google Vision)
-  Upload endpoint with background processing
-  StackCP deployment fully evaluated
-  Local development environment operational

**Pending** (35% to MVP):
- ⚠️ Meilisearch authentication (15-min fix)
- ⚠️ Frontend UI incomplete (1-2 days)
- ⚠️ Authentication not implemented (1 day)
- ⚠️ Tests needed (2-3 days)

## Deployment Options:

**StackCP Shared Hosting**: /bin/bash infrastructure, suitable for <5K docs/month
**VPS Alternative**: /month, better for scale

## Key Findings:

- Upload + OCR pipeline:  Working (85% confidence)
- Database: 184KB with test data
- Services: Redis , Meilisearch ⚠️ (auth issue), API , Worker 
- Git: 18 commits, all code committed

Ready for: Development continuation, deployment preparation
Not ready for: Production (needs auth + testing)

🚀 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-19 13:19:42 +02:00

1078 lines
32 KiB
Markdown

# StackCP Deployment Architecture Analysis for NaviDocs
**Tech Lead Assessment**: FastFile/NaviDocs Autonomous Development Prompt Evaluation
**Date**: 2025-10-19
**Analyst**: Tech Lead
**Target Environment**: StackCP (20i) Shared Hosting with `/tmp` executable directory
**Status**: Critical architectural revisions required
---
## Executive Summary
The existing NaviDocs development roadmap assumes a **standard deployment environment** with normal filesystem permissions, local services, and standard CI/CD. The StackCP discovery of `/tmp` as the only executable directory fundamentally breaks multiple architectural assumptions.
**Verdict**: The current autonomous development prompt and mission statement are **NOT viable** for StackCP deployment without significant architectural rewrites.
---
## Critical Issues (Blockers)
### 1. Repository Structure Assumes Home Directory Execution
**Current Assumption** (from README.md, roadmap):
```
navidocs/
├── server/ # Code + node_modules in home directory
├── client/ # Build output in home directory
├── uploads/ # Data in home directory
└── db/ # Database in home directory
```
**StackCP Reality**:
```
/tmp/navidocs/ # MUST be here (executable)
├── server/ # Code + native modules (better-sqlite3)
├── client/ # Build tools need execution
└── node_modules/ # Native binaries need exec
~/navidocs/ # Data only (noexec is fine)
├── uploads/ # File storage
├── db/ # SQLite database
├── logs/ # Application logs
└── .env # Configuration (secrets)
```
**Impact**:
- Entire codebase deployment location changes
- Path references in code must support dual locations
- Build process must handle `/tmp` compilation
- Git workflows must account for `/tmp` volatility
**Required Changes**:
- All absolute paths in code must use environment variables
- Database path must be configurable to `~/navidocs/db/`
- Upload path must be configurable to `~/navidocs/uploads/`
- Log path must be configurable to `~/navidocs/logs/`
---
### 2. Native Module Compilation Strategy Broken
**Current Approach** (package.json dependencies):
```json
"dependencies": {
"better-sqlite3": "^11.0.0", // Native module
"bcrypt": "^5.1.0", // Native module
}
```
**StackCP Reality**:
- `npm install` in home directory: **FAILS** (noexec prevents compilation)
- `npm install` in `/tmp`: **WORKS** (executable permissions)
- Must use `/tmp/node /path/to/npm-cli.js` wrapper
**Impact**:
- Standard `npm install` commands fail
- CI/CD scripts must be rewritten
- Developer workflow documentation is incorrect
- Deployment scripts need custom npm wrapper
**Required Changes**:
- Document MUST specify npm execution method: `/tmp/npm`
- All package installation must happen in `/tmp/navidocs`
- Deployment scripts must check for `/tmp` vs home directory
- Add validation that node_modules are in executable location
---
### 3. Process Persistence and Management Assumptions
**Current Assumption** (2-week-launch-plan.md):
```bash
# Uses PM2, systemd, or screen
pm2 start server/workers/ocr-worker.js
systemctl --user enable navidocs
```
**StackCP Reality**:
- PM2 **may not work** (depends on StackCP's Node.js manager)
- systemd **user services uncertain** (shared hosting restrictions)
- `/tmp` contents **may be cleared on reboot**
- StackCP Node.js Manager is the **recommended** approach
**Impact**:
- Process management strategy must be completely rewritten
- Code persistence in `/tmp` is questionable
- Application restart procedures are different
- Monitoring and health checks need rethinking
**Required Changes**:
- Primary: Use StackCP Node.js Application Manager (GUI-based)
- Fallback: `nohup` scripts with manual management
- Add checkpoint/restore mechanism for `/tmp` code
- Document StackCP-specific process management
- Create health check that detects `/tmp` code loss
---
### 4. File Persistence and Backup Strategy
**Current Assumption** (2-week-launch-plan.md):
```bash
# Assumes everything is in one location
tar -czf ~/backups/navidocs-$(date +%Y%m%d).tar.gz ~/navidocs/
```
**StackCP Reality**:
- Application code: `/tmp/navidocs/` (may be volatile)
- Application data: `~/navidocs/` (persistent)
- Backups must handle split locations
- `/tmp` may be cleared without warning
**Impact**:
- Backup scripts must handle two locations
- Restore procedure is more complex
- Code must be re-deployable from Git, not backups
- Data-only backups for database/uploads
**Required Changes**:
- Separate code backups (Git) from data backups (SQLite, uploads)
- Backup script: Only backup `~/navidocs/` (data)
- Deployment script: Always deploy code fresh to `/tmp/navidocs/`
- Checkpoint script: Sync `/tmp/navidocs` to `~/navidocs/code-checkpoint/` daily
- Restore script: Clone from Git to `/tmp`, restore data from backup
---
### 5. Development Workflow Misalignment
**Current Assumption** (QUICKSTART.md):
```bash
cd ~/navidocs/server
npm install
npm run dev
```
**StackCP Reality**:
```bash
cd /tmp/navidocs/server
/tmp/npm install
/tmp/node index.js
```
**Impact**:
- Developer documentation is incorrect
- Local development doesn't match production
- Deployment instructions are wrong
- Testing procedures need revision
**Required Changes**:
- Rewrite QUICKSTART.md with StackCP-specific paths
- Add "Local Development" vs "StackCP Deployment" sections
- Document npm wrapper usage
- Provide helper script (`stackcp-setup.sh`) installation steps
---
## Major Revisions Needed
### Section 1: Mission Statement (ARCHITECTURE-SUMMARY.md)
**Current State**: No mention of deployment constraints
**Required Additions**:
```markdown
## Deployment Constraints
### StackCP Shared Hosting
NaviDocs is designed to run on StackCP (20i) shared hosting with the following constraints:
- **Executable code location**: `/tmp/navidocs/` (only location with exec permissions)
- **Data storage location**: `~/navidocs/` (database, uploads, logs, config)
- **Code persistence**: `/tmp` may be cleared on reboot - deploy from Git
- **Native modules**: Must be compiled in `/tmp` using `/tmp/npm`
- **Process management**: Use StackCP Node.js Application Manager (primary)
- **External services required**:
- Redis Cloud (free tier: 30MB)
- Google Cloud Vision API (free tier: 1K pages/month)
- Meilisearch (already running on StackCP server)
### Path Configuration
All file paths must be configurable via environment variables:
- `CODE_PATH=/tmp/navidocs` (application code)
- `DATA_PATH=~/navidocs` (persistent data)
- `DATABASE_PATH=${DATA_PATH}/db/navidocs.db`
- `UPLOAD_DIR=${DATA_PATH}/uploads`
- `LOG_DIR=${DATA_PATH}/logs`
```
---
### Section 2: Technology Stack (ARCHITECTURE-SUMMARY.md)
**Current State**: Lists local dependencies (Redis, Tesseract)
**Required Revisions**:
```markdown
### Backend (StackCP Deployment)
- **Node.js 20** - Express 5 (via `/tmp/node`)
- **SQLite** - better-sqlite3 with WAL mode (compiled in `/tmp`)
- **Meilisearch** - Running on StackCP server (port 7700)
- **Redis Cloud** - External service (free 30MB tier)
- **Google Cloud Vision API** - OCR engine (free 1K pages/month)
- **BullMQ** - Background job processing (with Redis Cloud)
### StackCP-Specific Components
- **npm wrapper**: `/tmp/npm` (executes via `/tmp/node`)
- **Helper scripts**: `~/stackcp-setup.sh` (environment setup)
- **StackCP Node.js Manager**: Process supervision
- **Checkpoint script**: Daily sync `/tmp``~/navidocs/code-checkpoint/`
```
---
### Section 3: Deployment Strategy (docs/roadmap/v1.0-mvp.md)
**Current State**: Generic deployment steps
**Required Complete Rewrite**:
```markdown
## StackCP Deployment Strategy
### Prerequisites
1. StackCP account with SSH access
2. Redis Cloud account (free tier)
3. Google Cloud account with Vision API enabled
4. Git repository with NaviDocs code
### Phase 1: Environment Setup (30 minutes)
1. SSH into StackCP server
2. Install helper scripts (`stackcp-setup.sh`)
3. Create directory structure:
- `/tmp/navidocs/` (code)
- `~/navidocs/` (data)
4. Configure external services:
- Redis Cloud connection
- Google Vision API credentials
5. Verify Meilisearch is running
### Phase 2: Code Deployment (15 minutes)
1. Clone repository to `/tmp/navidocs`
2. Install dependencies using `/tmp/npm install`
3. Build client using `/tmp/npm run build`
4. Configure `.env` with StackCP paths
5. Initialize database in `~/navidocs/db/`
### Phase 3: Service Startup (10 minutes)
1. Use StackCP Node.js Application Manager:
- Add application: `/tmp/navidocs/server`
- Set startup file: `index.js`
- Set Node version: 20.x
- Set port: 3001
2. Start OCR worker (via nohup or screen)
3. Configure reverse proxy in StackCP
4. Copy client build to `~/public_html/`
### Phase 4: Operational Setup (15 minutes)
1. Create checkpoint script (daily `/tmp``~/code-checkpoint/`)
2. Create data backup script (daily `~/navidocs/``~/backups/`)
3. Set up health checks (monitor `/tmp` code presence)
4. Document restart procedure (code loss recovery)
```
---
### Section 4: Development Workflow (QUICKSTART.md)
**Current State**: Assumes standard environment
**Required Rewrite**:
```markdown
## Quick Start
### Local Development (Standard Environment)
```bash
# Clone repository
git clone <repo-url> ~/navidocs
cd ~/navidocs/server
npm install
npm run dev
```
### StackCP Deployment (Shared Hosting)
```bash
# SSH into StackCP
ssh stackcp
# Load StackCP helpers
source ~/stackcp-setup.sh
# Deploy to /tmp (executable location)
cd /tmp
git clone <repo-url> navidocs
cd navidocs/server
/tmp/npm install # Must use wrapper
# Configure environment
cat > .env << 'EOF'
DATABASE_PATH=/home/sites/7a/c/cb8112d0d1/navidocs/db/navidocs.db
UPLOAD_DIR=/home/sites/7a/c/cb8112d0d1/navidocs/uploads
REDIS_HOST=redis-xxxxx.redis.cloud
# ... (Redis Cloud credentials)
EOF
# Initialize database (in persistent location)
/tmp/node db/init.js
# Start via StackCP Manager (GUI)
# OR use helper function:
navidocs-start
```
### Key Differences: Local vs StackCP
| Aspect | Local Dev | StackCP Deploy |
|--------|-----------|----------------|
| Code location | `~/navidocs/` | `/tmp/navidocs/` |
| npm command | `npm install` | `/tmp/npm install` |
| Node command | `node index.js` | `/tmp/node index.js` |
| Data location | `./db/`, `./uploads/` | `~/navidocs/db/`, `~/navidocs/uploads/` |
| Process manager | PM2 / nodemon | StackCP Node.js Manager |
| Redis | Local (redis-server) | Redis Cloud |
| OCR engine | Tesseract (local) | Google Cloud Vision API |
```
---
### Section 5: Backup and Recovery (2-week-launch-plan.md)
**Current State**: Single-location backup
**Required Revisions**:
```markdown
## Backup Strategy for StackCP
### What to Backup
1. **Data** (critical, daily):
- SQLite database: `~/navidocs/db/navidocs.db`
- Uploaded PDFs: `~/navidocs/uploads/`
- Configuration: `~/navidocs/.env` (encrypted)
- Logs (last 7 days): `~/navidocs/logs/`
2. **Code checkpoint** (daily):
- `/tmp/navidocs/` → `~/navidocs/code-checkpoint/`
- This is a snapshot, not the source of truth
- Real source of truth: Git repository
### Backup Script (StackCP-specific)
```bash
#!/bin/bash
# ~/bin/backup-navidocs-data
BACKUP_DIR=~/backups/navidocs
DATE=$(date +%Y%m%d-%H%M%S)
# 1. Backup persistent data only
tar -czf ${BACKUP_DIR}/data-${DATE}.tar.gz \
~/navidocs/db \
~/navidocs/uploads \
~/navidocs/.env \
~/navidocs/logs
# 2. Checkpoint /tmp code (may be lost on reboot)
rsync -av --delete /tmp/navidocs/ ~/navidocs/code-checkpoint/
# 3. Verify backup integrity
tar -tzf ${BACKUP_DIR}/data-${DATE}.tar.gz > /dev/null
# 4. Rotate old backups (keep 30 days)
find ${BACKUP_DIR} -name "data-*.tar.gz" -mtime +30 -delete
echo "Backup complete: ${BACKUP_DIR}/data-${DATE}.tar.gz"
```
### Restore Procedure
```bash
#!/bin/bash
# ~/bin/restore-navidocs
# 1. Stop application
navidocs-stop
# 2. Deploy fresh code from Git (to /tmp)
cd /tmp
rm -rf navidocs
git clone <repo-url> navidocs
cd navidocs/server
/tmp/npm install
# 3. Restore data from backup
cd ~
tar -xzf backups/navidocs/data-YYYYMMDD-HHMMSS.tar.gz
# 4. Verify database integrity
/tmp/node -e "const db = require('better-sqlite3')('~/navidocs/db/navidocs.db'); console.log(db.pragma('integrity_check'));"
# 5. Restart application
navidocs-start
```
### Health Check for /tmp Code Loss
```bash
#!/bin/bash
# ~/bin/check-navidocs-code
if [ ! -f /tmp/navidocs/server/index.js ]; then
echo "ERROR: Code missing from /tmp/navidocs"
echo "Restoring from checkpoint..."
# Restore from checkpoint
cp -r ~/navidocs/code-checkpoint/* /tmp/navidocs/
# Verify restoration
if [ -f /tmp/navidocs/server/index.js ]; then
echo "Code restored from checkpoint"
navidocs-start
else
echo "CRITICAL: Code checkpoint also missing, deploy from Git"
exit 1
fi
fi
```
### Cron Jobs (StackCP-specific)
```cron
# Data backup (daily at 3am)
0 3 * * * ~/bin/backup-navidocs-data
# Code checkpoint (daily at 4am)
0 4 * * * rsync -av --delete /tmp/navidocs/ ~/navidocs/code-checkpoint/
# Health check for code loss (every 6 hours)
0 */6 * * * ~/bin/check-navidocs-code
```
```
---
## Minor Tweaks (Sections That Mostly Work)
### 1. Database Schema (docs/architecture/database-schema.sql)
**Status**: No changes needed
**Reason**: Schema is platform-agnostic, only paths change
### 2. Meilisearch Configuration (docs/architecture/meilisearch-config.json)
**Status**: Minor tweaks
**Changes**: Document that Meilisearch is already running on StackCP, just need master key
### 3. API Routes (server/routes/)
**Status**: Minor tweaks
**Changes**: Ensure all file paths use environment variables, no hardcoded paths
### 4. Frontend Code (client/)
**Status**: No changes needed
**Reason**: Frontend is platform-agnostic, just needs correct API base URL
### 5. Security Hardening Guide (docs/architecture/hardened-production-guide.md)
**Status**: Minor additions
**Changes**: Add StackCP-specific security notes (file permissions, StackCP Node.js Manager)
---
## Proposed Repository Structure
### Git Repository (Source of Truth)
```
navidocs/
├── .env.example # Template with StackCP paths
├── .env.stackcp.example # StackCP-specific template
├── README.md # General overview
├── QUICKSTART.md # Local development
├── QUICKSTART-STACKCP.md # StackCP deployment
├── ARCHITECTURE-SUMMARY.md # Updated with constraints
├── server/
│ ├── index.js
│ ├── config/
│ │ └── paths.js # Centralized path configuration
│ ├── db/
│ │ └── init.js
│ ├── routes/
│ ├── services/
│ └── workers/
├── client/
│ └── ...
├── scripts/
│ ├── stackcp/
│ │ ├── setup.sh # One-time environment setup
│ │ ├── deploy.sh # Deploy code to /tmp
│ │ ├── backup-data.sh # Backup persistent data
│ │ ├── checkpoint-code.sh # Checkpoint /tmp to ~
│ │ ├── restore.sh # Restore from backup + Git
│ │ └── health-check.sh # Detect /tmp code loss
│ └── local/
│ └── dev.sh # Local development setup
└── docs/
├── deployment/
│ ├── local.md
│ ├── stackcp.md # Complete StackCP guide
│ └── vps.md # Future: standard VPS
└── architecture/
└── ...
```
### StackCP Server Layout (Deployed)
```
/tmp/navidocs/ # Application code (volatile)
├── server/
│ ├── index.js
│ ├── node_modules/ # Native modules compiled here
│ │ └── better-sqlite3/
│ ├── config/
│ ├── db/
│ ├── routes/
│ ├── services/
│ └── workers/
└── client/
└── dist/
~/navidocs/ # Application data (persistent)
├── db/
│ └── navidocs.db # SQLite database
├── uploads/
│ └── *.pdf # Uploaded documents
├── logs/
│ ├── server.log
│ └── worker.log
├── .env # Configuration (secrets)
├── google-credentials.json # Google Cloud Vision credentials
└── code-checkpoint/ # Daily snapshot of /tmp code
└── (copy of /tmp/navidocs)
~/backups/navidocs/ # Data backups
├── data-20251019-030000.tar.gz
└── ...
~/public_html/ # Web root (frontend)
├── index.html
└── assets/
~/bin/ # Helper scripts
├── backup-navidocs-data
├── checkpoint-navidocs-code
├── check-navidocs-code
└── restore-navidocs
~/stackcp-setup.sh # Environment helpers (source this)
```
---
## Revised Deployment Steps (StackCP-Specific)
### Step 1: Prerequisites (One-Time)
```bash
# 1. Sign up for external services
- Redis Cloud: https://redis.com/try-free/
- Google Cloud: Enable Vision API
# 2. SSH into StackCP
ssh your-username@ssh-node-gb.lhr.stackcp.net
# 3. Verify Node.js and Meilisearch
/tmp/node --version # Should be v20.19.5
curl http://127.0.0.1:7700/health # Should return "available"
# 4. Install helper scripts
cd ~
git clone <navidocs-repo> /tmp/navidocs-setup
cp /tmp/navidocs-setup/scripts/stackcp/setup.sh ~/stackcp-setup.sh
source ~/stackcp-setup.sh
```
### Step 2: Initial Deployment
```bash
# 1. Deploy code to /tmp
cd /tmp
git clone <navidocs-repo> navidocs
cd navidocs/server
/tmp/npm install --production
# 2. Build frontend
cd /tmp/navidocs/client
/tmp/npm install
/tmp/npm run build
# 3. Create data directories
mkdir -p ~/navidocs/{db,uploads,logs,code-checkpoint}
mkdir -p ~/backups/navidocs
# 4. Configure environment
cp /tmp/navidocs/.env.stackcp.example ~/navidocs/.env
# Edit ~/navidocs/.env with:
# - Redis Cloud credentials
# - Google Vision credentials path
# - StackCP-specific paths
# Symlink .env to code location
ln -s ~/navidocs/.env /tmp/navidocs/server/.env
# 5. Upload Google credentials
# (From local machine)
scp google-credentials.json stackcp:~/navidocs/
# 6. Initialize database
cd /tmp/navidocs/server
/tmp/node db/init.js
```
### Step 3: Start Services
```bash
# Option A: StackCP Node.js Manager (Recommended)
# 1. Log into StackCP control panel
# 2. Advanced → Node.js Applications → Add Application
# - Path: /tmp/navidocs/server
# - Startup: index.js
# - Node: 20.x
# - Port: 3001
# 3. Start application
# 4. Configure reverse proxy: yoursite.com → http://127.0.0.1:3001
# Option B: Manual (for testing)
source ~/stackcp-setup.sh
navidocs-start
```
### Step 4: Operational Setup
```bash
# 1. Install backup scripts
cp /tmp/navidocs/scripts/stackcp/* ~/bin/
chmod +x ~/bin/backup-navidocs-data
chmod +x ~/bin/checkpoint-navidocs-code
chmod +x ~/bin/check-navidocs-code
chmod +x ~/bin/restore-navidocs
# 2. Set up cron jobs
crontab -e
# Add:
0 3 * * * ~/bin/backup-navidocs-data
0 4 * * * ~/bin/checkpoint-navidocs-code
0 */6 * * * ~/bin/check-navidocs-code
# 3. Test backup/restore
~/bin/backup-navidocs-data
~/bin/restore-navidocs
```
### Step 5: Updates and Maintenance
```bash
# Update code (from Git)
cd /tmp/navidocs
git pull
cd server && /tmp/npm install
cd ../client && /tmp/npm run build
cp -r dist/* ~/public_html/
# Restart services
navidocs-stop
navidocs-start
# Or use StackCP Manager:
# Advanced → Node.js Applications → NaviDocs → Restart
```
---
## Path Configuration Strategy
### Central Configuration File
**server/config/paths.js**:
```javascript
import path from 'path';
import { fileURLToPath } from 'url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
// Detect environment
const isStackCP = process.env.STACKCP_DEPLOYMENT === 'true';
const homeDir = process.env.HOME || '/home/sites/7a/c/cb8112d0d1';
// Base paths
export const CODE_PATH = isStackCP
? '/tmp/navidocs'
: path.join(__dirname, '../..');
export const DATA_PATH = isStackCP
? path.join(homeDir, 'navidocs')
: path.join(__dirname, '../..');
// Derived paths
export const DATABASE_PATH = process.env.DATABASE_PATH ||
path.join(DATA_PATH, 'db', 'navidocs.db');
export const UPLOAD_DIR = process.env.UPLOAD_DIR ||
path.join(DATA_PATH, 'uploads');
export const LOG_DIR = process.env.LOG_DIR ||
path.join(DATA_PATH, 'logs');
export const CONFIG_DIR = process.env.CONFIG_DIR ||
path.join(DATA_PATH, 'config');
// Validate paths exist (create if needed)
import fs from 'fs';
[DATA_PATH, path.dirname(DATABASE_PATH), UPLOAD_DIR, LOG_DIR, CONFIG_DIR]
.forEach(dir => {
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir, { recursive: true });
}
});
```
### Environment Variables (.env.stackcp.example)
```bash
# StackCP Deployment Configuration
STACKCP_DEPLOYMENT=true
NODE_ENV=production
PORT=3001
# Paths (StackCP-specific)
# Code is in /tmp/navidocs (executable)
# Data is in ~/navidocs (persistent)
HOME=/home/sites/7a/c/cb8112d0d1
DATA_PATH=${HOME}/navidocs
DATABASE_PATH=${DATA_PATH}/db/navidocs.db
UPLOAD_DIR=${DATA_PATH}/uploads
LOG_DIR=${DATA_PATH}/logs
CONFIG_DIR=${DATA_PATH}/config
# External Services
REDIS_HOST=redis-xxxxx.c1.us-east-1-2.ec2.redis.cloud
REDIS_PORT=12345
REDIS_PASSWORD=your-password-here
# Meilisearch (already running on StackCP)
MEILISEARCH_HOST=http://127.0.0.1:7700
MEILISEARCH_MASTER_KEY=find-on-server
MEILISEARCH_INDEX_NAME=navidocs-pages
# OCR - Google Cloud Vision API
PREFERRED_OCR_ENGINE=google-vision
GOOGLE_APPLICATION_CREDENTIALS=${CONFIG_DIR}/google-credentials.json
# Security
JWT_SECRET=generate-with-openssl-rand-base64-32
JWT_EXPIRES_IN=7d
```
---
## Operational Concerns
### 1. /tmp Persistence Risk
**Problem**: `/tmp` may be cleared on reboot or system maintenance
**Mitigation**:
1. **Daily checkpoint**: `rsync /tmp/navidocs/ → ~/navidocs/code-checkpoint/`
2. **Health check**: Detect missing `/tmp/navidocs` and restore from checkpoint
3. **Automated recovery**: Script that detects loss and re-deploys from Git
4. **Monitoring**: Alert if `/tmp/navidocs` disappears
**Implementation**:
```bash
#!/bin/bash
# ~/bin/check-navidocs-code (runs every 6 hours)
CODE_DIR=/tmp/navidocs/server/index.js
CHECKPOINT_DIR=~/navidocs/code-checkpoint
if [ ! -f "$CODE_DIR" ]; then
logger "NaviDocs code missing from /tmp, attempting recovery"
# Try checkpoint first (faster)
if [ -d "$CHECKPOINT_DIR" ]; then
cp -r $CHECKPOINT_DIR /tmp/navidocs
logger "NaviDocs code restored from checkpoint"
navidocs-start
exit 0
fi
# Checkpoint missing, deploy from Git
cd /tmp
git clone <repo-url> navidocs
cd navidocs/server
/tmp/npm install --production
logger "NaviDocs code deployed from Git"
navidocs-start
fi
```
### 2. Node Module Recompilation
**Problem**: Native modules must be recompiled on each deployment
**Mitigation**:
1. **Cache node_modules in checkpoint**: Include in daily checkpoint
2. **Verify after restore**: Run health check that validates better-sqlite3 works
3. **Document recompilation time**: ~2 minutes for full npm install
**Implementation**:
```bash
#!/bin/bash
# ~/bin/checkpoint-navidocs-code
rsync -av --delete /tmp/navidocs/ ~/navidocs/code-checkpoint/
# Verify native modules work
cd ~/navidocs/code-checkpoint/server
/tmp/node -e "require('better-sqlite3'); console.log('Native modules OK');"
```
### 3. StackCP Node.js Manager Integration
**Problem**: StackCP's GUI-based manager may conflict with manual process management
**Mitigation**:
1. **Choose one approach**: Either StackCP Manager OR manual (nohup/screen)
2. **Document clearly**: QUICKSTART must specify which to use
3. **Provide both**: Helper functions for manual, instructions for GUI
**Recommendation**: Use StackCP Manager for API server, manual for OCR worker
### 4. External Service Dependencies
**Problem**: Redis Cloud and Google Vision introduce network dependencies
**Mitigation**:
1. **Graceful degradation**: Queue fails → log error but don't crash
2. **Health checks**: Monitor external service connectivity
3. **Fallback options**: Document SQLite-based queue as fallback
4. **Rate limiting**: Respect free tier limits (1K Google Vision pages/month)
---
## Revised Mission Statement
**Original** (ARCHITECTURE-SUMMARY.md):
> "A future-proof, multi-vertical document management platform for boat owners, marinas, and property managers."
**Revised for StackCP Reality**:
> "A future-proof, multi-vertical document management platform for boat owners, marinas, and property managers, architected for constrained shared hosting environments with split executable/data directories."
**Key Additions to Mission**:
1. **Deployment flexibility**: Design for StackCP constraints while maintaining portability
2. **Path abstraction**: All file operations use configurable paths
3. **External service integration**: Cloud-first approach (Redis Cloud, Google Vision)
4. **Operational resilience**: Handle `/tmp` volatility with checkpoints and auto-recovery
5. **Developer experience**: Provide environment-specific tooling (helpers, wrappers)
---
## Sections Requiring Complete Rewrite
### 1. QUICKSTART.md
**Reason**: Assumes standard environment, all commands are wrong for StackCP
**Action**: Split into QUICKSTART-LOCAL.md and QUICKSTART-STACKCP.md
### 2. docs/roadmap/2-week-launch-plan.md
**Reason**: Deployment steps assume local services, PM2, systemd
**Action**: Rewrite deployment phase with StackCP-specific steps
### 3. server/.env.example
**Reason**: Uses relative paths, assumes local Redis/Tesseract
**Action**: Create .env.stackcp.example with absolute paths, cloud services
### 4. Any deployment scripts in scripts/
**Reason**: Assumes standard environment
**Action**: Create scripts/stackcp/ directory with StackCP-specific scripts
### 5. docs/DEPLOYMENT_STACKCP.md (existing)
**Reason**: Outdated, assumes VPS-like environment
**Action**: Complete rewrite based on verified StackCP constraints
---
## Sections Requiring Minor Revisions
### 1. README.md
**Changes**:
- Add "Deployment Options" section (Local, StackCP, VPS)
- Link to deployment-specific guides
- Update prerequisites with "OR" options (local Redis OR Redis Cloud)
### 2. ARCHITECTURE-SUMMARY.md
**Changes**:
- Add "Deployment Constraints" section
- Update "Technology Stack" with StackCP variants
- Add "Path Configuration Strategy"
### 3. server/package.json
**Changes**:
- Add scripts for StackCP: `"stackcp:install": "/tmp/npm install"`
- Document that native modules need executable directory
### 4. docs/architecture/hardened-production-guide.md
**Changes**:
- Add StackCP-specific security considerations
- Document StackCP Node.js Manager security model
- Note that `/tmp` code is readable by other StackCP users (security risk)
---
## Testing Strategy Additions
### 1. StackCP Environment Tests
```bash
# Verify deployment environment
test-stackcp-env() {
# Check /tmp is executable
touch /tmp/test-exec && chmod +x /tmp/test-exec
/tmp/test-exec && echo "OK: /tmp is executable" || echo "FAIL: /tmp not executable"
# Check home is noexec
touch ~/test-exec && chmod +x ~/test-exec
~/test-exec && echo "FAIL: Home should be noexec" || echo "OK: Home is noexec"
# Check npm wrapper works
/tmp/npm --version || echo "FAIL: npm wrapper broken"
# Check Meilisearch reachable
curl -s http://127.0.0.1:7700/health | grep available || echo "FAIL: Meilisearch not running"
}
```
### 2. Path Configuration Tests
```javascript
// server/config/paths.test.js
import { DATABASE_PATH, UPLOAD_DIR, LOG_DIR } from './paths.js';
import fs from 'fs';
describe('Path Configuration', () => {
test('DATABASE_PATH is in data directory (not /tmp)', () => {
expect(DATABASE_PATH).not.toContain('/tmp/');
expect(DATABASE_PATH).toContain('/navidocs/db/');
});
test('All data directories exist', () => {
expect(fs.existsSync(path.dirname(DATABASE_PATH))).toBe(true);
expect(fs.existsSync(UPLOAD_DIR)).toBe(true);
expect(fs.existsSync(LOG_DIR)).toBe(true);
});
});
```
### 3. Checkpoint/Restore Tests
```bash
# Test checkpoint and restore procedure
test-checkpoint-restore() {
echo "Testing checkpoint/restore..."
# 1. Backup current /tmp/navidocs
mv /tmp/navidocs /tmp/navidocs-backup
# 2. Restore from checkpoint
~/bin/check-navidocs-code
# 3. Verify code is restored
[ -f /tmp/navidocs/server/index.js ] || { echo "FAIL: Code not restored"; exit 1; }
# 4. Verify native modules work
/tmp/node -e "require('/tmp/navidocs/server/node_modules/better-sqlite3')" || {
echo "FAIL: Native modules broken after restore"
exit 1
}
# 5. Cleanup
rm -rf /tmp/navidocs
mv /tmp/navidocs-backup /tmp/navidocs
echo "OK: Checkpoint/restore works"
}
```
---
## Recommended Implementation Priority
### Phase 1: Critical Path Fixes (Week 1)
1. Create `server/config/paths.js` (centralized path configuration)
2. Update all code to use path configuration (database, uploads, logs)
3. Create `.env.stackcp.example` with correct paths
4. Write StackCP deployment scripts (`scripts/stackcp/`)
5. Test deployment on StackCP server
### Phase 2: Operational Resilience (Week 2)
1. Write checkpoint script (`checkpoint-navidocs-code`)
2. Write health check script (`check-navidocs-code`)
3. Write backup/restore scripts (`backup-navidocs-data`, `restore-navidocs`)
4. Test recovery scenarios (/tmp loss, data corruption)
5. Set up cron jobs
### Phase 3: Documentation (Week 2)
1. Rewrite QUICKSTART-STACKCP.md
2. Update ARCHITECTURE-SUMMARY.md
3. Revise 2-week-launch-plan.md for StackCP
4. Create deployment runbook
5. Document troubleshooting procedures
### Phase 4: Developer Experience (Week 3)
1. Create `stackcp-setup.sh` helper script
2. Create npm wrapper (`/tmp/npm`)
3. Add management functions (navidocs-start, stop, status)
4. Test on fresh StackCP account
5. Gather feedback from beta deployment
---
## Risks and Mitigations
| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| `/tmp` cleared during production | **CRITICAL** | Medium | Daily checkpoint + auto-recovery script |
| Native module compilation fails | **HIGH** | Low | Document known-good Node version, keep checkpoint |
| Redis Cloud free tier exhausted | **MEDIUM** | Low | Monitor usage, implement SQLite queue fallback |
| Google Vision free tier exhausted | **MEDIUM** | Medium | Rate limit uploads, warn users, offer paid tier |
| StackCP restricts long-running processes | **HIGH** | Low | Use StackCP Manager, contact support if issues |
| `/tmp` code readable by other users | **MEDIUM** | High | Document security risk, recommend VPS for sensitive data |
| Data path misconfiguration | **CRITICAL** | Medium | Add startup validation, fail fast if paths wrong |
| Checkpoint script fails silently | **HIGH** | Medium | Add monitoring, alert on checkpoint failures |
---
## Conclusion
The existing NaviDocs development roadmap and documentation **requires substantial revisions** to accommodate StackCP's unique constraints. The discovery that only `/tmp` has executable permissions fundamentally changes:
1. **Repository structure**: Code vs data must be split
2. **Deployment process**: Custom npm wrappers, StackCP Manager
3. **Backup strategy**: Code from Git, data from backups
4. **Operational resilience**: Checkpoint scripts, auto-recovery
5. **Developer workflow**: Different commands for local vs StackCP
**Estimated effort to revise**:
- Code changes: 2-3 days (path configuration, testing)
- Documentation: 1-2 days (rewrites)
- Operational scripts: 1 day (backup, checkpoint, health checks)
- Testing: 1 day (deployment verification)
**Total**: 5-7 days of focused work to make NaviDocs production-ready for StackCP.
**Recommendation**: Treat this as a **StackCP-specific deployment variant**, not the primary development environment. Maintain standard local development workflow, add StackCP deployment as a supported option with its own documentation and tooling.
---
**Next Steps for Debate**:
1. Review this analysis with Security team (`.tmp` code readability risk)
2. Review with OCI team (container deployment alternative)
3. Decide: Proceed with StackCP adaptations OR migrate to VPS
4. If proceeding: Implement Phase 1 (Critical Path Fixes) first
5. Test on StackCP, validate assumptions, iterate
---
**Document Status**: Ready for technical review
**Prepared by**: Tech Lead
**Date**: 2025-10-19
**Version**: 1.0