This commit finalizes the NaviDocs MVP documentation with comprehensive handover materials. ## Documentation Added: 1. **NAVIDOCS_HANDOVER.md** - Complete project handover (65% MVP complete) - Executive summary and current status - Repository structure and component details - Testing results and known issues - Deployment options (StackCP vs VPS) - Next steps and risk assessment - Success metrics and recommendations 2. **StackCP Analysis Documents**: - ANALYSIS_INDEX.md - Master overview - STACKCP_ARCHITECTURE_ANALYSIS.md - Technical deep-dive - STACKCP_DEBATE_BRIEF.md - Deployment decision framework - STACKCP_QUICK_REFERENCE.md - Fast decision-making tool ## Current State Summary: **Completed** (65% MVP): - ✅ Database schema (13 tables, fully normalized) - ✅ OCR pipeline (3 options: Tesseract 85%, Google Drive, Google Vision) - ✅ Upload endpoint with background processing - ✅ StackCP deployment fully evaluated - ✅ Local development environment operational **Pending** (35% to MVP): - ⚠️ Meilisearch authentication (15-min fix) - ⚠️ Frontend UI incomplete (1-2 days) - ⚠️ Authentication not implemented (1 day) - ⚠️ Tests needed (2-3 days) ## Deployment Options: **StackCP Shared Hosting**: /bin/bash infrastructure, suitable for <5K docs/month **VPS Alternative**: /month, better for scale ## Key Findings: - Upload + OCR pipeline: ✅ Working (85% confidence) - Database: 184KB with test data - Services: Redis ✅, Meilisearch ⚠️ (auth issue), API ✅, Worker ✅ - Git: 18 commits, all code committed Ready for: Development continuation, deployment preparation Not ready for: Production (needs auth + testing) 🚀 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
32 KiB
StackCP Deployment Architecture Analysis for NaviDocs
Tech Lead Assessment: FastFile/NaviDocs Autonomous Development Prompt Evaluation
Date: 2025-10-19
Analyst: Tech Lead
Target Environment: StackCP (20i) Shared Hosting with /tmp executable directory
Status: Critical architectural revisions required
Executive Summary
The existing NaviDocs development roadmap assumes a standard deployment environment with normal filesystem permissions, local services, and standard CI/CD. The StackCP discovery of /tmp as the only executable directory fundamentally breaks multiple architectural assumptions.
Verdict: The current autonomous development prompt and mission statement are NOT viable for StackCP deployment without significant architectural rewrites.
Critical Issues (Blockers)
1. Repository Structure Assumes Home Directory Execution
Current Assumption (from README.md, roadmap):
navidocs/
├── server/ # Code + node_modules in home directory
├── client/ # Build output in home directory
├── uploads/ # Data in home directory
└── db/ # Database in home directory
StackCP Reality:
/tmp/navidocs/ # MUST be here (executable)
├── server/ # Code + native modules (better-sqlite3)
├── client/ # Build tools need execution
└── node_modules/ # Native binaries need exec
~/navidocs/ # Data only (noexec is fine)
├── uploads/ # File storage
├── db/ # SQLite database
├── logs/ # Application logs
└── .env # Configuration (secrets)
Impact:
- Entire codebase deployment location changes
- Path references in code must support dual locations
- Build process must handle
/tmpcompilation - Git workflows must account for
/tmpvolatility
Required Changes:
- All absolute paths in code must use environment variables
- Database path must be configurable to
~/navidocs/db/ - Upload path must be configurable to
~/navidocs/uploads/ - Log path must be configurable to
~/navidocs/logs/
2. Native Module Compilation Strategy Broken
Current Approach (package.json dependencies):
"dependencies": {
"better-sqlite3": "^11.0.0", // Native module
"bcrypt": "^5.1.0", // Native module
}
StackCP Reality:
npm installin home directory: FAILS (noexec prevents compilation)npm installin/tmp: WORKS (executable permissions)- Must use
/tmp/node /path/to/npm-cli.jswrapper
Impact:
- Standard
npm installcommands fail - CI/CD scripts must be rewritten
- Developer workflow documentation is incorrect
- Deployment scripts need custom npm wrapper
Required Changes:
- Document MUST specify npm execution method:
/tmp/npm - All package installation must happen in
/tmp/navidocs - Deployment scripts must check for
/tmpvs home directory - Add validation that node_modules are in executable location
3. Process Persistence and Management Assumptions
Current Assumption (2-week-launch-plan.md):
# Uses PM2, systemd, or screen
pm2 start server/workers/ocr-worker.js
systemctl --user enable navidocs
StackCP Reality:
- PM2 may not work (depends on StackCP's Node.js manager)
- systemd user services uncertain (shared hosting restrictions)
/tmpcontents may be cleared on reboot- StackCP Node.js Manager is the recommended approach
Impact:
- Process management strategy must be completely rewritten
- Code persistence in
/tmpis questionable - Application restart procedures are different
- Monitoring and health checks need rethinking
Required Changes:
- Primary: Use StackCP Node.js Application Manager (GUI-based)
- Fallback:
nohupscripts with manual management - Add checkpoint/restore mechanism for
/tmpcode - Document StackCP-specific process management
- Create health check that detects
/tmpcode loss
4. File Persistence and Backup Strategy
Current Assumption (2-week-launch-plan.md):
# Assumes everything is in one location
tar -czf ~/backups/navidocs-$(date +%Y%m%d).tar.gz ~/navidocs/
StackCP Reality:
- Application code:
/tmp/navidocs/(may be volatile) - Application data:
~/navidocs/(persistent) - Backups must handle split locations
/tmpmay be cleared without warning
Impact:
- Backup scripts must handle two locations
- Restore procedure is more complex
- Code must be re-deployable from Git, not backups
- Data-only backups for database/uploads
Required Changes:
- Separate code backups (Git) from data backups (SQLite, uploads)
- Backup script: Only backup
~/navidocs/(data) - Deployment script: Always deploy code fresh to
/tmp/navidocs/ - Checkpoint script: Sync
/tmp/navidocsto~/navidocs/code-checkpoint/daily - Restore script: Clone from Git to
/tmp, restore data from backup
5. Development Workflow Misalignment
Current Assumption (QUICKSTART.md):
cd ~/navidocs/server
npm install
npm run dev
StackCP Reality:
cd /tmp/navidocs/server
/tmp/npm install
/tmp/node index.js
Impact:
- Developer documentation is incorrect
- Local development doesn't match production
- Deployment instructions are wrong
- Testing procedures need revision
Required Changes:
- Rewrite QUICKSTART.md with StackCP-specific paths
- Add "Local Development" vs "StackCP Deployment" sections
- Document npm wrapper usage
- Provide helper script (
stackcp-setup.sh) installation steps
Major Revisions Needed
Section 1: Mission Statement (ARCHITECTURE-SUMMARY.md)
Current State: No mention of deployment constraints
Required Additions:
## Deployment Constraints
### StackCP Shared Hosting
NaviDocs is designed to run on StackCP (20i) shared hosting with the following constraints:
- **Executable code location**: `/tmp/navidocs/` (only location with exec permissions)
- **Data storage location**: `~/navidocs/` (database, uploads, logs, config)
- **Code persistence**: `/tmp` may be cleared on reboot - deploy from Git
- **Native modules**: Must be compiled in `/tmp` using `/tmp/npm`
- **Process management**: Use StackCP Node.js Application Manager (primary)
- **External services required**:
- Redis Cloud (free tier: 30MB)
- Google Cloud Vision API (free tier: 1K pages/month)
- Meilisearch (already running on StackCP server)
### Path Configuration
All file paths must be configurable via environment variables:
- `CODE_PATH=/tmp/navidocs` (application code)
- `DATA_PATH=~/navidocs` (persistent data)
- `DATABASE_PATH=${DATA_PATH}/db/navidocs.db`
- `UPLOAD_DIR=${DATA_PATH}/uploads`
- `LOG_DIR=${DATA_PATH}/logs`
Section 2: Technology Stack (ARCHITECTURE-SUMMARY.md)
Current State: Lists local dependencies (Redis, Tesseract)
Required Revisions:
### Backend (StackCP Deployment)
- **Node.js 20** - Express 5 (via `/tmp/node`)
- **SQLite** - better-sqlite3 with WAL mode (compiled in `/tmp`)
- **Meilisearch** - Running on StackCP server (port 7700)
- **Redis Cloud** - External service (free 30MB tier)
- **Google Cloud Vision API** - OCR engine (free 1K pages/month)
- **BullMQ** - Background job processing (with Redis Cloud)
### StackCP-Specific Components
- **npm wrapper**: `/tmp/npm` (executes via `/tmp/node`)
- **Helper scripts**: `~/stackcp-setup.sh` (environment setup)
- **StackCP Node.js Manager**: Process supervision
- **Checkpoint script**: Daily sync `/tmp` → `~/navidocs/code-checkpoint/`
Section 3: Deployment Strategy (docs/roadmap/v1.0-mvp.md)
Current State: Generic deployment steps
Required Complete Rewrite:
## StackCP Deployment Strategy
### Prerequisites
1. StackCP account with SSH access
2. Redis Cloud account (free tier)
3. Google Cloud account with Vision API enabled
4. Git repository with NaviDocs code
### Phase 1: Environment Setup (30 minutes)
1. SSH into StackCP server
2. Install helper scripts (`stackcp-setup.sh`)
3. Create directory structure:
- `/tmp/navidocs/` (code)
- `~/navidocs/` (data)
4. Configure external services:
- Redis Cloud connection
- Google Vision API credentials
5. Verify Meilisearch is running
### Phase 2: Code Deployment (15 minutes)
1. Clone repository to `/tmp/navidocs`
2. Install dependencies using `/tmp/npm install`
3. Build client using `/tmp/npm run build`
4. Configure `.env` with StackCP paths
5. Initialize database in `~/navidocs/db/`
### Phase 3: Service Startup (10 minutes)
1. Use StackCP Node.js Application Manager:
- Add application: `/tmp/navidocs/server`
- Set startup file: `index.js`
- Set Node version: 20.x
- Set port: 3001
2. Start OCR worker (via nohup or screen)
3. Configure reverse proxy in StackCP
4. Copy client build to `~/public_html/`
### Phase 4: Operational Setup (15 minutes)
1. Create checkpoint script (daily `/tmp` → `~/code-checkpoint/`)
2. Create data backup script (daily `~/navidocs/` → `~/backups/`)
3. Set up health checks (monitor `/tmp` code presence)
4. Document restart procedure (code loss recovery)
Section 4: Development Workflow (QUICKSTART.md)
Current State: Assumes standard environment
Required Rewrite:
## Quick Start
### Local Development (Standard Environment)
```bash
# Clone repository
git clone <repo-url> ~/navidocs
cd ~/navidocs/server
npm install
npm run dev
StackCP Deployment (Shared Hosting)
# SSH into StackCP
ssh stackcp
# Load StackCP helpers
source ~/stackcp-setup.sh
# Deploy to /tmp (executable location)
cd /tmp
git clone <repo-url> navidocs
cd navidocs/server
/tmp/npm install # Must use wrapper
# Configure environment
cat > .env << 'EOF'
DATABASE_PATH=/home/sites/7a/c/cb8112d0d1/navidocs/db/navidocs.db
UPLOAD_DIR=/home/sites/7a/c/cb8112d0d1/navidocs/uploads
REDIS_HOST=redis-xxxxx.redis.cloud
# ... (Redis Cloud credentials)
EOF
# Initialize database (in persistent location)
/tmp/node db/init.js
# Start via StackCP Manager (GUI)
# OR use helper function:
navidocs-start
Key Differences: Local vs StackCP
| Aspect | Local Dev | StackCP Deploy |
|---|---|---|
| Code location | ~/navidocs/ |
/tmp/navidocs/ |
| npm command | npm install |
/tmp/npm install |
| Node command | node index.js |
/tmp/node index.js |
| Data location | ./db/, ./uploads/ |
~/navidocs/db/, ~/navidocs/uploads/ |
| Process manager | PM2 / nodemon | StackCP Node.js Manager |
| Redis | Local (redis-server) | Redis Cloud |
| OCR engine | Tesseract (local) | Google Cloud Vision API |
---
### Section 5: Backup and Recovery (2-week-launch-plan.md)
**Current State**: Single-location backup
**Required Revisions**:
```markdown
## Backup Strategy for StackCP
### What to Backup
1. **Data** (critical, daily):
- SQLite database: `~/navidocs/db/navidocs.db`
- Uploaded PDFs: `~/navidocs/uploads/`
- Configuration: `~/navidocs/.env` (encrypted)
- Logs (last 7 days): `~/navidocs/logs/`
2. **Code checkpoint** (daily):
- `/tmp/navidocs/` → `~/navidocs/code-checkpoint/`
- This is a snapshot, not the source of truth
- Real source of truth: Git repository
### Backup Script (StackCP-specific)
```bash
#!/bin/bash
# ~/bin/backup-navidocs-data
BACKUP_DIR=~/backups/navidocs
DATE=$(date +%Y%m%d-%H%M%S)
# 1. Backup persistent data only
tar -czf ${BACKUP_DIR}/data-${DATE}.tar.gz \
~/navidocs/db \
~/navidocs/uploads \
~/navidocs/.env \
~/navidocs/logs
# 2. Checkpoint /tmp code (may be lost on reboot)
rsync -av --delete /tmp/navidocs/ ~/navidocs/code-checkpoint/
# 3. Verify backup integrity
tar -tzf ${BACKUP_DIR}/data-${DATE}.tar.gz > /dev/null
# 4. Rotate old backups (keep 30 days)
find ${BACKUP_DIR} -name "data-*.tar.gz" -mtime +30 -delete
echo "Backup complete: ${BACKUP_DIR}/data-${DATE}.tar.gz"
Restore Procedure
#!/bin/bash
# ~/bin/restore-navidocs
# 1. Stop application
navidocs-stop
# 2. Deploy fresh code from Git (to /tmp)
cd /tmp
rm -rf navidocs
git clone <repo-url> navidocs
cd navidocs/server
/tmp/npm install
# 3. Restore data from backup
cd ~
tar -xzf backups/navidocs/data-YYYYMMDD-HHMMSS.tar.gz
# 4. Verify database integrity
/tmp/node -e "const db = require('better-sqlite3')('~/navidocs/db/navidocs.db'); console.log(db.pragma('integrity_check'));"
# 5. Restart application
navidocs-start
Health Check for /tmp Code Loss
#!/bin/bash
# ~/bin/check-navidocs-code
if [ ! -f /tmp/navidocs/server/index.js ]; then
echo "ERROR: Code missing from /tmp/navidocs"
echo "Restoring from checkpoint..."
# Restore from checkpoint
cp -r ~/navidocs/code-checkpoint/* /tmp/navidocs/
# Verify restoration
if [ -f /tmp/navidocs/server/index.js ]; then
echo "Code restored from checkpoint"
navidocs-start
else
echo "CRITICAL: Code checkpoint also missing, deploy from Git"
exit 1
fi
fi
Cron Jobs (StackCP-specific)
# Data backup (daily at 3am)
0 3 * * * ~/bin/backup-navidocs-data
# Code checkpoint (daily at 4am)
0 4 * * * rsync -av --delete /tmp/navidocs/ ~/navidocs/code-checkpoint/
# Health check for code loss (every 6 hours)
0 */6 * * * ~/bin/check-navidocs-code
---
## Minor Tweaks (Sections That Mostly Work)
### 1. Database Schema (docs/architecture/database-schema.sql)
**Status**: No changes needed
**Reason**: Schema is platform-agnostic, only paths change
### 2. Meilisearch Configuration (docs/architecture/meilisearch-config.json)
**Status**: Minor tweaks
**Changes**: Document that Meilisearch is already running on StackCP, just need master key
### 3. API Routes (server/routes/)
**Status**: Minor tweaks
**Changes**: Ensure all file paths use environment variables, no hardcoded paths
### 4. Frontend Code (client/)
**Status**: No changes needed
**Reason**: Frontend is platform-agnostic, just needs correct API base URL
### 5. Security Hardening Guide (docs/architecture/hardened-production-guide.md)
**Status**: Minor additions
**Changes**: Add StackCP-specific security notes (file permissions, StackCP Node.js Manager)
---
## Proposed Repository Structure
### Git Repository (Source of Truth)
navidocs/ ├── .env.example # Template with StackCP paths ├── .env.stackcp.example # StackCP-specific template ├── README.md # General overview ├── QUICKSTART.md # Local development ├── QUICKSTART-STACKCP.md # StackCP deployment ├── ARCHITECTURE-SUMMARY.md # Updated with constraints ├── server/ │ ├── index.js │ ├── config/ │ │ └── paths.js # Centralized path configuration │ ├── db/ │ │ └── init.js │ ├── routes/ │ ├── services/ │ └── workers/ ├── client/ │ └── ... ├── scripts/ │ ├── stackcp/ │ │ ├── setup.sh # One-time environment setup │ │ ├── deploy.sh # Deploy code to /tmp │ │ ├── backup-data.sh # Backup persistent data │ │ ├── checkpoint-code.sh # Checkpoint /tmp to ~ │ │ ├── restore.sh # Restore from backup + Git │ │ └── health-check.sh # Detect /tmp code loss │ └── local/ │ └── dev.sh # Local development setup └── docs/ ├── deployment/ │ ├── local.md │ ├── stackcp.md # Complete StackCP guide │ └── vps.md # Future: standard VPS └── architecture/ └── ...
### StackCP Server Layout (Deployed)
/tmp/navidocs/ # Application code (volatile) ├── server/ │ ├── index.js │ ├── node_modules/ # Native modules compiled here │ │ └── better-sqlite3/ │ ├── config/ │ ├── db/ │ ├── routes/ │ ├── services/ │ └── workers/ └── client/ └── dist/
~/navidocs/ # Application data (persistent) ├── db/ │ └── navidocs.db # SQLite database ├── uploads/ │ └── *.pdf # Uploaded documents ├── logs/ │ ├── server.log │ └── worker.log ├── .env # Configuration (secrets) ├── google-credentials.json # Google Cloud Vision credentials └── code-checkpoint/ # Daily snapshot of /tmp code └── (copy of /tmp/navidocs)
~/backups/navidocs/ # Data backups ├── data-20251019-030000.tar.gz └── ...
~/public_html/ # Web root (frontend) ├── index.html └── assets/
~/bin/ # Helper scripts ├── backup-navidocs-data ├── checkpoint-navidocs-code ├── check-navidocs-code └── restore-navidocs
~/stackcp-setup.sh # Environment helpers (source this)
---
## Revised Deployment Steps (StackCP-Specific)
### Step 1: Prerequisites (One-Time)
```bash
# 1. Sign up for external services
- Redis Cloud: https://redis.com/try-free/
- Google Cloud: Enable Vision API
# 2. SSH into StackCP
ssh your-username@ssh-node-gb.lhr.stackcp.net
# 3. Verify Node.js and Meilisearch
/tmp/node --version # Should be v20.19.5
curl http://127.0.0.1:7700/health # Should return "available"
# 4. Install helper scripts
cd ~
git clone <navidocs-repo> /tmp/navidocs-setup
cp /tmp/navidocs-setup/scripts/stackcp/setup.sh ~/stackcp-setup.sh
source ~/stackcp-setup.sh
Step 2: Initial Deployment
# 1. Deploy code to /tmp
cd /tmp
git clone <navidocs-repo> navidocs
cd navidocs/server
/tmp/npm install --production
# 2. Build frontend
cd /tmp/navidocs/client
/tmp/npm install
/tmp/npm run build
# 3. Create data directories
mkdir -p ~/navidocs/{db,uploads,logs,code-checkpoint}
mkdir -p ~/backups/navidocs
# 4. Configure environment
cp /tmp/navidocs/.env.stackcp.example ~/navidocs/.env
# Edit ~/navidocs/.env with:
# - Redis Cloud credentials
# - Google Vision credentials path
# - StackCP-specific paths
# Symlink .env to code location
ln -s ~/navidocs/.env /tmp/navidocs/server/.env
# 5. Upload Google credentials
# (From local machine)
scp google-credentials.json stackcp:~/navidocs/
# 6. Initialize database
cd /tmp/navidocs/server
/tmp/node db/init.js
Step 3: Start Services
# Option A: StackCP Node.js Manager (Recommended)
# 1. Log into StackCP control panel
# 2. Advanced → Node.js Applications → Add Application
# - Path: /tmp/navidocs/server
# - Startup: index.js
# - Node: 20.x
# - Port: 3001
# 3. Start application
# 4. Configure reverse proxy: yoursite.com → http://127.0.0.1:3001
# Option B: Manual (for testing)
source ~/stackcp-setup.sh
navidocs-start
Step 4: Operational Setup
# 1. Install backup scripts
cp /tmp/navidocs/scripts/stackcp/* ~/bin/
chmod +x ~/bin/backup-navidocs-data
chmod +x ~/bin/checkpoint-navidocs-code
chmod +x ~/bin/check-navidocs-code
chmod +x ~/bin/restore-navidocs
# 2. Set up cron jobs
crontab -e
# Add:
0 3 * * * ~/bin/backup-navidocs-data
0 4 * * * ~/bin/checkpoint-navidocs-code
0 */6 * * * ~/bin/check-navidocs-code
# 3. Test backup/restore
~/bin/backup-navidocs-data
~/bin/restore-navidocs
Step 5: Updates and Maintenance
# Update code (from Git)
cd /tmp/navidocs
git pull
cd server && /tmp/npm install
cd ../client && /tmp/npm run build
cp -r dist/* ~/public_html/
# Restart services
navidocs-stop
navidocs-start
# Or use StackCP Manager:
# Advanced → Node.js Applications → NaviDocs → Restart
Path Configuration Strategy
Central Configuration File
server/config/paths.js:
import path from 'path';
import { fileURLToPath } from 'url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
// Detect environment
const isStackCP = process.env.STACKCP_DEPLOYMENT === 'true';
const homeDir = process.env.HOME || '/home/sites/7a/c/cb8112d0d1';
// Base paths
export const CODE_PATH = isStackCP
? '/tmp/navidocs'
: path.join(__dirname, '../..');
export const DATA_PATH = isStackCP
? path.join(homeDir, 'navidocs')
: path.join(__dirname, '../..');
// Derived paths
export const DATABASE_PATH = process.env.DATABASE_PATH ||
path.join(DATA_PATH, 'db', 'navidocs.db');
export const UPLOAD_DIR = process.env.UPLOAD_DIR ||
path.join(DATA_PATH, 'uploads');
export const LOG_DIR = process.env.LOG_DIR ||
path.join(DATA_PATH, 'logs');
export const CONFIG_DIR = process.env.CONFIG_DIR ||
path.join(DATA_PATH, 'config');
// Validate paths exist (create if needed)
import fs from 'fs';
[DATA_PATH, path.dirname(DATABASE_PATH), UPLOAD_DIR, LOG_DIR, CONFIG_DIR]
.forEach(dir => {
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir, { recursive: true });
}
});
Environment Variables (.env.stackcp.example)
# StackCP Deployment Configuration
STACKCP_DEPLOYMENT=true
NODE_ENV=production
PORT=3001
# Paths (StackCP-specific)
# Code is in /tmp/navidocs (executable)
# Data is in ~/navidocs (persistent)
HOME=/home/sites/7a/c/cb8112d0d1
DATA_PATH=${HOME}/navidocs
DATABASE_PATH=${DATA_PATH}/db/navidocs.db
UPLOAD_DIR=${DATA_PATH}/uploads
LOG_DIR=${DATA_PATH}/logs
CONFIG_DIR=${DATA_PATH}/config
# External Services
REDIS_HOST=redis-xxxxx.c1.us-east-1-2.ec2.redis.cloud
REDIS_PORT=12345
REDIS_PASSWORD=your-password-here
# Meilisearch (already running on StackCP)
MEILISEARCH_HOST=http://127.0.0.1:7700
MEILISEARCH_MASTER_KEY=find-on-server
MEILISEARCH_INDEX_NAME=navidocs-pages
# OCR - Google Cloud Vision API
PREFERRED_OCR_ENGINE=google-vision
GOOGLE_APPLICATION_CREDENTIALS=${CONFIG_DIR}/google-credentials.json
# Security
JWT_SECRET=generate-with-openssl-rand-base64-32
JWT_EXPIRES_IN=7d
Operational Concerns
1. /tmp Persistence Risk
Problem: /tmp may be cleared on reboot or system maintenance
Mitigation:
- Daily checkpoint:
rsync /tmp/navidocs/ → ~/navidocs/code-checkpoint/ - Health check: Detect missing
/tmp/navidocsand restore from checkpoint - Automated recovery: Script that detects loss and re-deploys from Git
- Monitoring: Alert if
/tmp/navidocsdisappears
Implementation:
#!/bin/bash
# ~/bin/check-navidocs-code (runs every 6 hours)
CODE_DIR=/tmp/navidocs/server/index.js
CHECKPOINT_DIR=~/navidocs/code-checkpoint
if [ ! -f "$CODE_DIR" ]; then
logger "NaviDocs code missing from /tmp, attempting recovery"
# Try checkpoint first (faster)
if [ -d "$CHECKPOINT_DIR" ]; then
cp -r $CHECKPOINT_DIR /tmp/navidocs
logger "NaviDocs code restored from checkpoint"
navidocs-start
exit 0
fi
# Checkpoint missing, deploy from Git
cd /tmp
git clone <repo-url> navidocs
cd navidocs/server
/tmp/npm install --production
logger "NaviDocs code deployed from Git"
navidocs-start
fi
2. Node Module Recompilation
Problem: Native modules must be recompiled on each deployment
Mitigation:
- Cache node_modules in checkpoint: Include in daily checkpoint
- Verify after restore: Run health check that validates better-sqlite3 works
- Document recompilation time: ~2 minutes for full npm install
Implementation:
#!/bin/bash
# ~/bin/checkpoint-navidocs-code
rsync -av --delete /tmp/navidocs/ ~/navidocs/code-checkpoint/
# Verify native modules work
cd ~/navidocs/code-checkpoint/server
/tmp/node -e "require('better-sqlite3'); console.log('Native modules OK');"
3. StackCP Node.js Manager Integration
Problem: StackCP's GUI-based manager may conflict with manual process management
Mitigation:
- Choose one approach: Either StackCP Manager OR manual (nohup/screen)
- Document clearly: QUICKSTART must specify which to use
- Provide both: Helper functions for manual, instructions for GUI
Recommendation: Use StackCP Manager for API server, manual for OCR worker
4. External Service Dependencies
Problem: Redis Cloud and Google Vision introduce network dependencies
Mitigation:
- Graceful degradation: Queue fails → log error but don't crash
- Health checks: Monitor external service connectivity
- Fallback options: Document SQLite-based queue as fallback
- Rate limiting: Respect free tier limits (1K Google Vision pages/month)
Revised Mission Statement
Original (ARCHITECTURE-SUMMARY.md):
"A future-proof, multi-vertical document management platform for boat owners, marinas, and property managers."
Revised for StackCP Reality:
"A future-proof, multi-vertical document management platform for boat owners, marinas, and property managers, architected for constrained shared hosting environments with split executable/data directories."
Key Additions to Mission:
- Deployment flexibility: Design for StackCP constraints while maintaining portability
- Path abstraction: All file operations use configurable paths
- External service integration: Cloud-first approach (Redis Cloud, Google Vision)
- Operational resilience: Handle
/tmpvolatility with checkpoints and auto-recovery - Developer experience: Provide environment-specific tooling (helpers, wrappers)
Sections Requiring Complete Rewrite
1. QUICKSTART.md
Reason: Assumes standard environment, all commands are wrong for StackCP Action: Split into QUICKSTART-LOCAL.md and QUICKSTART-STACKCP.md
2. docs/roadmap/2-week-launch-plan.md
Reason: Deployment steps assume local services, PM2, systemd Action: Rewrite deployment phase with StackCP-specific steps
3. server/.env.example
Reason: Uses relative paths, assumes local Redis/Tesseract Action: Create .env.stackcp.example with absolute paths, cloud services
4. Any deployment scripts in scripts/
Reason: Assumes standard environment Action: Create scripts/stackcp/ directory with StackCP-specific scripts
5. docs/DEPLOYMENT_STACKCP.md (existing)
Reason: Outdated, assumes VPS-like environment Action: Complete rewrite based on verified StackCP constraints
Sections Requiring Minor Revisions
1. README.md
Changes:
- Add "Deployment Options" section (Local, StackCP, VPS)
- Link to deployment-specific guides
- Update prerequisites with "OR" options (local Redis OR Redis Cloud)
2. ARCHITECTURE-SUMMARY.md
Changes:
- Add "Deployment Constraints" section
- Update "Technology Stack" with StackCP variants
- Add "Path Configuration Strategy"
3. server/package.json
Changes:
- Add scripts for StackCP:
"stackcp:install": "/tmp/npm install" - Document that native modules need executable directory
4. docs/architecture/hardened-production-guide.md
Changes:
- Add StackCP-specific security considerations
- Document StackCP Node.js Manager security model
- Note that
/tmpcode is readable by other StackCP users (security risk)
Testing Strategy Additions
1. StackCP Environment Tests
# Verify deployment environment
test-stackcp-env() {
# Check /tmp is executable
touch /tmp/test-exec && chmod +x /tmp/test-exec
/tmp/test-exec && echo "OK: /tmp is executable" || echo "FAIL: /tmp not executable"
# Check home is noexec
touch ~/test-exec && chmod +x ~/test-exec
~/test-exec && echo "FAIL: Home should be noexec" || echo "OK: Home is noexec"
# Check npm wrapper works
/tmp/npm --version || echo "FAIL: npm wrapper broken"
# Check Meilisearch reachable
curl -s http://127.0.0.1:7700/health | grep available || echo "FAIL: Meilisearch not running"
}
2. Path Configuration Tests
// server/config/paths.test.js
import { DATABASE_PATH, UPLOAD_DIR, LOG_DIR } from './paths.js';
import fs from 'fs';
describe('Path Configuration', () => {
test('DATABASE_PATH is in data directory (not /tmp)', () => {
expect(DATABASE_PATH).not.toContain('/tmp/');
expect(DATABASE_PATH).toContain('/navidocs/db/');
});
test('All data directories exist', () => {
expect(fs.existsSync(path.dirname(DATABASE_PATH))).toBe(true);
expect(fs.existsSync(UPLOAD_DIR)).toBe(true);
expect(fs.existsSync(LOG_DIR)).toBe(true);
});
});
3. Checkpoint/Restore Tests
# Test checkpoint and restore procedure
test-checkpoint-restore() {
echo "Testing checkpoint/restore..."
# 1. Backup current /tmp/navidocs
mv /tmp/navidocs /tmp/navidocs-backup
# 2. Restore from checkpoint
~/bin/check-navidocs-code
# 3. Verify code is restored
[ -f /tmp/navidocs/server/index.js ] || { echo "FAIL: Code not restored"; exit 1; }
# 4. Verify native modules work
/tmp/node -e "require('/tmp/navidocs/server/node_modules/better-sqlite3')" || {
echo "FAIL: Native modules broken after restore"
exit 1
}
# 5. Cleanup
rm -rf /tmp/navidocs
mv /tmp/navidocs-backup /tmp/navidocs
echo "OK: Checkpoint/restore works"
}
Recommended Implementation Priority
Phase 1: Critical Path Fixes (Week 1)
- Create
server/config/paths.js(centralized path configuration) - Update all code to use path configuration (database, uploads, logs)
- Create
.env.stackcp.examplewith correct paths - Write StackCP deployment scripts (
scripts/stackcp/) - Test deployment on StackCP server
Phase 2: Operational Resilience (Week 2)
- Write checkpoint script (
checkpoint-navidocs-code) - Write health check script (
check-navidocs-code) - Write backup/restore scripts (
backup-navidocs-data,restore-navidocs) - Test recovery scenarios (/tmp loss, data corruption)
- Set up cron jobs
Phase 3: Documentation (Week 2)
- Rewrite QUICKSTART-STACKCP.md
- Update ARCHITECTURE-SUMMARY.md
- Revise 2-week-launch-plan.md for StackCP
- Create deployment runbook
- Document troubleshooting procedures
Phase 4: Developer Experience (Week 3)
- Create
stackcp-setup.shhelper script - Create npm wrapper (
/tmp/npm) - Add management functions (navidocs-start, stop, status)
- Test on fresh StackCP account
- Gather feedback from beta deployment
Risks and Mitigations
| Risk | Impact | Probability | Mitigation |
|---|---|---|---|
/tmp cleared during production |
CRITICAL | Medium | Daily checkpoint + auto-recovery script |
| Native module compilation fails | HIGH | Low | Document known-good Node version, keep checkpoint |
| Redis Cloud free tier exhausted | MEDIUM | Low | Monitor usage, implement SQLite queue fallback |
| Google Vision free tier exhausted | MEDIUM | Medium | Rate limit uploads, warn users, offer paid tier |
| StackCP restricts long-running processes | HIGH | Low | Use StackCP Manager, contact support if issues |
/tmp code readable by other users |
MEDIUM | High | Document security risk, recommend VPS for sensitive data |
| Data path misconfiguration | CRITICAL | Medium | Add startup validation, fail fast if paths wrong |
| Checkpoint script fails silently | HIGH | Medium | Add monitoring, alert on checkpoint failures |
Conclusion
The existing NaviDocs development roadmap and documentation requires substantial revisions to accommodate StackCP's unique constraints. The discovery that only /tmp has executable permissions fundamentally changes:
- Repository structure: Code vs data must be split
- Deployment process: Custom npm wrappers, StackCP Manager
- Backup strategy: Code from Git, data from backups
- Operational resilience: Checkpoint scripts, auto-recovery
- Developer workflow: Different commands for local vs StackCP
Estimated effort to revise:
- Code changes: 2-3 days (path configuration, testing)
- Documentation: 1-2 days (rewrites)
- Operational scripts: 1 day (backup, checkpoint, health checks)
- Testing: 1 day (deployment verification)
Total: 5-7 days of focused work to make NaviDocs production-ready for StackCP.
Recommendation: Treat this as a StackCP-specific deployment variant, not the primary development environment. Maintain standard local development workflow, add StackCP deployment as a supported option with its own documentation and tooling.
Next Steps for Debate:
- Review this analysis with Security team (
.tmpcode readability risk) - Review with OCI team (container deployment alternative)
- Decide: Proceed with StackCP adaptations OR migrate to VPS
- If proceeding: Implement Phase 1 (Critical Path Fixes) first
- Test on StackCP, validate assumptions, iterate
Document Status: Ready for technical review Prepared by: Tech Lead Date: 2025-10-19 Version: 1.0