diff --git a/PRE_DEPLOYMENT_CHECKLIST.md b/PRE_DEPLOYMENT_CHECKLIST.md new file mode 100644 index 0000000..6f48e61 --- /dev/null +++ b/PRE_DEPLOYMENT_CHECKLIST.md @@ -0,0 +1,59 @@ +# Pre-Deployment Checklist + +Run before deploying to production: + +## Code Quality +- [ ] All feature branches merged to main +- [ ] No console.log() in production code +- [ ] No TODO/FIXME comments +- [ ] Code formatted consistently +- [ ] No unused imports + +## Testing +- [ ] All API endpoints tested manually +- [ ] Upload flow works for all file types +- [ ] Search returns accurate results +- [ ] Timeline loads and paginates correctly +- [ ] Mobile responsive on 3 screen sizes +- [ ] No browser console errors + +## Security +- [ ] JWT secrets are 64+ characters +- [ ] .env.production created with unique secrets +- [ ] No hardcoded credentials +- [ ] File upload size limits enforced +- [ ] SQL injection prevention verified +- [ ] XSS prevention verified + +## Performance +- [ ] Smart OCR working (<10s for text PDFs) +- [ ] Search response time <50ms +- [ ] Frontend build size <2MB +- [ ] Images optimized +- [ ] No memory leaks + +## Database +- [ ] All migrations run successfully +- [ ] Indexes created on activity_log +- [ ] Foreign keys configured +- [ ] Backup script tested + +## Documentation +- [ ] USER_GUIDE.md complete +- [ ] DEVELOPER.md complete +- [ ] API documented +- [ ] Environment variables documented + +## Deployment +- [ ] deploy-stackcp.sh configured with correct host +- [ ] SSH access to StackCP verified +- [ ] PM2 configuration ready +- [ ] Backup strategy defined +- [ ] Rollback plan documented + +## Post-Deployment +- [ ] SSL certificate installed +- [ ] Domain DNS configured +- [ ] Monitoring alerts configured +- [ ] First backup completed +- [ ] Version tagged in git diff --git a/docs/DEVELOPER.md b/docs/DEVELOPER.md new file mode 100644 index 0000000..505786a --- /dev/null +++ b/docs/DEVELOPER.md @@ -0,0 +1,314 @@ +# NaviDocs Developer Guide + +**Version:** 1.0 +**Tech Stack:** Node.js + Express + Vue 3 + SQLite + Meilisearch + +--- + +## Architecture + +### Backend (Express.js) + +``` +server/ +├── index.js # Main server +├── config/ +│ └── db.js # SQLite connection +├── routes/ +│ ├── upload.js # File upload API +│ ├── search.js # Search API +│ ├── timeline.js # Timeline API +│ └── auth.routes.js # Authentication +├── services/ +│ ├── ocr.js # OCR processing +│ ├── pdf-text-extractor.js # Native PDF text extraction +│ ├── document-processor.js # Multi-format routing +│ ├── activity-logger.js # Timeline logging +│ └── file-safety.js # File validation +├── workers/ +│ └── ocr-worker.js # Background OCR jobs +└── migrations/ + └── 010_activity_timeline.sql +``` + +### Frontend (Vue 3 + Vite) + +``` +client/src/ +├── views/ +│ ├── HomeView.vue +│ ├── SearchView.vue +│ ├── Timeline.vue +│ └── DocumentView.vue +├── components/ +│ ├── CompactNav.vue +│ ├── UploadModal.vue +│ └── TocSidebar.vue +├── router.js +└── App.vue +``` + +--- + +## Key Features + +### 1. Smart OCR (Session 1) + +**Problem:** 100-page PDFs took 3+ minutes with Tesseract + +**Solution:** Hybrid approach +- Extract native PDF text first (pdfjs-dist) +- Only OCR pages with <50 characters +- Performance: 180s → 5s (36x speedup) + +**Implementation:** +```javascript +// server/services/pdf-text-extractor.js +export async function extractNativeTextPerPage(pdfPath) { + const data = new Uint8Array(readFileSync(pdfPath)); + const pdf = await pdfjsLib.getDocument({ data }).promise; + // Extract text from each page +} + +// server/services/ocr.js +if (await hasNativeText(pdfPath)) { + // Use native text +} else { + // Fallback to OCR +} +``` + +### 2. Multi-Format Upload (Session 2) + +**Supported Formats:** +- PDF: Native text + OCR fallback +- Images: Tesseract OCR +- Word (DOCX): Mammoth text extraction +- Excel (XLSX): Sheet-to-CSV conversion +- Text (TXT, MD): Direct read + +**Implementation:** +```javascript +// server/services/document-processor.js +export async function processDocument(filePath, options) { + const category = getFileCategory(filePath); + + switch (category) { + case 'pdf': return await extractTextFromPDF(filePath, options); + case 'image': return await processImageFile(filePath, options); + case 'word': return await processWordDocument(filePath, options); + case 'excel': return await processExcelDocument(filePath, options); + case 'text': return await processTextFile(filePath, options); + } +} +``` + +### 3. Timeline Feature (Session 3) + +**Database Schema:** +```sql +CREATE TABLE activity_log ( + id TEXT PRIMARY KEY, + organization_id TEXT NOT NULL, + user_id TEXT NOT NULL, + event_type TEXT NOT NULL, + event_title TEXT NOT NULL, + created_at INTEGER NOT NULL +); +``` + +**Auto-logging:** +```javascript +// After successful upload +await logActivity({ + organizationId: orgId, + userId: req.user.id, + eventType: 'document_upload', + eventTitle: title, + referenceId: documentId, + referenceType: 'document' +}); +``` + +--- + +## API Endpoints + +### Authentication +- `POST /api/auth/login` - User login +- `POST /api/auth/register` - User registration +- `GET /api/auth/me` - Get current user + +### Documents +- `POST /api/upload` - Upload document (multipart/form-data) +- `GET /api/documents` - List documents +- `GET /api/documents/:id` - Get document details +- `DELETE /api/documents/:id` - Delete document + +### Search +- `POST /api/search` - Search documents (body: {q, limit, offset}) + +### Timeline +- `GET /api/organizations/:orgId/timeline` - Get activity timeline + +--- + +## Environment Variables + +```env +NODE_ENV=production +PORT=8001 +DATABASE_PATH=./navidocs.db +JWT_SECRET=[64-char hex] +MEILISEARCH_HOST=http://localhost:7700 +UPLOAD_DIR=./uploads +MAX_FILE_SIZE=52428800 +OCR_MIN_TEXT_THRESHOLD=50 +``` + +--- + +## Development Setup + +```bash +# Clone repo +git clone https://github.com/dannystocker/navidocs.git +cd navidocs + +# Install dependencies +cd server && npm install +cd ../client && npm install + +# Create .env +cp server/.env.example server/.env + +# Run migrations +cd server && node scripts/run-migration.js db/schema.sql + +# Start services +cd .. && ./start-all.sh + +# Backend: http://localhost:8001 +# Frontend: http://localhost:8081 +``` + +--- + +## Testing + +### Manual Testing +```bash +# Upload test +curl -X POST http://localhost:8001/api/upload \ + -H "Authorization: Bearer $TOKEN" \ + -F "file=@test.pdf" + +# Search test +curl -X POST http://localhost:8001/api/search \ + -H "Authorization: Bearer $TOKEN" \ + -d '{"q":"bilge"}' +``` + +### E2E Testing +```bash +cd client +npm run test:e2e +``` + +--- + +## Deployment + +### Production Checklist + +- [ ] Update .env.production with secure secrets +- [ ] Build frontend: `cd client && npm run build` +- [ ] Run database migrations +- [ ] Configure SSL certificate +- [ ] Set up PM2 for process management +- [ ] Configure Nginx reverse proxy +- [ ] Set up daily backups (cron job) +- [ ] Configure monitoring (PM2 logs) + +### Deploy to StackCP + +```bash +./deploy-stackcp.sh +``` + +--- + +## Performance + +### Benchmarks + +| Operation | Before | After | Improvement | +|-----------|--------|-------|-------------| +| Native PDF (100 pages) | 180s | 5s | 36x | +| Image OCR | 3s | 3s | - | +| Word doc upload | N/A | 0.8s | New | +| Search query | <10ms | <10ms | - | + +### Optimization Tips + +- Use smart OCR for PDFs +- Index documents in background workers +- Cache search results in Redis +- Compress images before upload + +--- + +## Troubleshooting + +### OCR Worker Not Processing + +```bash +# Check worker status +ps aux | grep ocr-worker + +# View logs +tail -f /tmp/navidocs-ocr-worker.log + +# Restart worker +pm2 restart navidocs-ocr-worker +``` + +### Meilisearch Not Responding + +```bash +# Check status +curl http://localhost:7700/health + +# Restart +pm2 restart meilisearch +``` + +### Database Locked + +```bash +# Check for zombie processes +lsof | grep navidocs.db + +# Kill zombie process +kill -9 [PID] +``` + +--- + +## Contributing + +1. Create feature branch: `git checkout -b feature/your-feature` +2. Make changes with tests +3. Commit: `git commit -m "[FEATURE] Your feature description"` +4. Push: `git push origin feature/your-feature` +5. Create Pull Request + +--- + +## License + +Proprietary - All rights reserved + +--- + +**Questions? Contact the development team.** diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md new file mode 100644 index 0000000..118bb5e --- /dev/null +++ b/docs/USER_GUIDE.md @@ -0,0 +1,187 @@ +# NaviDocs User Guide + +**Version:** 1.0 +**Last Updated:** 2025-11-13 + +--- + +## Getting Started + +### 1. Login + +Navigate to https://navidocs.yourdomain.com and login with your credentials: + +- Email: your-email@example.com +- Password: [provided by admin] + +### 2. Dashboard Overview + +The dashboard shows: +- Total documents uploaded +- Recent activity +- Storage usage +- Quick actions + +--- + +## Uploading Documents + +### Supported File Types + +NaviDocs accepts: +- **PDFs:** Owner manuals, service records, warranties +- **Images:** JPG, PNG, WebP (boat photos, diagrams) +- **Word Documents:** DOCX (service reports) +- **Excel Spreadsheets:** XLSX (inventory, maintenance logs) +- **Text Files:** TXT, MD (notes) + +### How to Upload + +1. Click **"Upload"** in navigation +2. Select file (max 50MB) +3. Enter document title +4. Choose document type (manual, warranty, service, etc.) +5. Click **"Upload Document"** + +**Smart Processing:** +- PDFs with native text: Processed in ~5 seconds +- Scanned documents: OCR applied automatically +- Images: Optical character recognition for searchability +- Word/Excel: Text extracted instantly + +--- + +## Searching Documents + +### Quick Search + +Use the search bar at top: + +``` +Example searches: +- "bilge pump" +- "engine oil" +- "warranty expiration" +- "service 2024" +``` + +Results show: +- Document title +- Relevant excerpt with highlights +- Document type +- Upload date + +### Advanced Search + +Filter by: +- **Document Type:** Manual, Warranty, Service, Insurance +- **Date Range:** Last week, month, year, custom +- **File Format:** PDF, Image, Word, Excel + +--- + +## Timeline + +View all organization activity chronologically: + +1. Click **"Timeline"** in navigation +2. See events grouped by date: + - Today + - Yesterday + - This Week + - This Month + - Older + +### Event Types + +- 📄 Document uploads +- 🔧 Maintenance records (future) +- ⚠️ Warranty claims (future) + +### Filtering Timeline + +Use dropdown to filter by: +- All events +- Document uploads only +- Maintenance logs only + +--- + +## Best Practices + +### Organize Documents + +**Use descriptive titles:** +- ✅ "Azimut 55S Owner Manual 2020" +- ❌ "manual.pdf" + +**Choose correct document type:** +- Owner manuals → "Manual" +- Service receipts → "Service Record" +- Insurance policies → "Insurance" +- Warranties → "Warranty" + +### Regular Uploads + +- Upload documents as you receive them +- Don't wait for "spring cleaning" +- Keep photos organized with descriptive names + +### Search Tips + +- Use specific terms: "bilge pump maintenance" vs "pump" +- Search by brand names: "Volvo Penta" +- Use date keywords: "2024" or "January" + +--- + +## Troubleshooting + +### Upload Failed + +**"File too large"** +→ Compress PDF or split into smaller files (max 50MB) + +**"Unsupported file type"** +→ Convert to PDF, JPG, or DOCX + +**"Upload timeout"** +→ Check internet connection, try again + +### Search Not Working + +**No results for recent upload:** +→ Wait 30 seconds for indexing to complete + +**Search returns wrong documents:** +→ Use more specific search terms + +### General Issues + +**Can't login:** +→ Reset password or contact admin + +**Page not loading:** +→ Clear browser cache, try incognito mode + +--- + +## Support + +Need help? Contact: +- Email: support@navidocs.com +- Phone: [support number] + +--- + +## Keyboard Shortcuts + +| Shortcut | Action | +|----------|--------| +| `/` | Focus search | +| `Ctrl+U` | Open upload form | +| `Esc` | Close modal | + +--- + +**Happy sailing! ⛵** diff --git a/scripts/backup-database.sh b/scripts/backup-database.sh new file mode 100755 index 0000000..67c7bf6 --- /dev/null +++ b/scripts/backup-database.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# NaviDocs Database Backup Script + +BACKUP_DIR="./backups" +TIMESTAMP=$(date +%Y%m%d-%H%M%S) +DB_FILE="./navidocs.db" +UPLOAD_DIR="./uploads" + +mkdir -p $BACKUP_DIR + +echo "🔐 Starting backup..." + +# Backup database +sqlite3 $DB_FILE ".backup '$BACKUP_DIR/navidocs-db-$TIMESTAMP.db'" + +# Backup uploads folder +tar -czf "$BACKUP_DIR/navidocs-uploads-$TIMESTAMP.tar.gz" $UPLOAD_DIR + +echo "✅ Backup complete:" +echo " - Database: $BACKUP_DIR/navidocs-db-$TIMESTAMP.db" +echo " - Uploads: $BACKUP_DIR/navidocs-uploads-$TIMESTAMP.tar.gz" + +# Keep only last 7 days of backups +find $BACKUP_DIR -name "navidocs-db-*.db" -mtime +7 -delete +find $BACKUP_DIR -name "navidocs-uploads-*.tar.gz" -mtime +7 -delete + +echo "🗑️ Old backups cleaned up (kept last 7 days)" diff --git a/server/.env.production b/server/.env.production new file mode 100644 index 0000000..2dfbe29 --- /dev/null +++ b/server/.env.production @@ -0,0 +1,35 @@ +# NaviDocs Production Environment +NODE_ENV=production +PORT=8001 + +# Database +DATABASE_PATH=./navidocs.production.db + +# Security +JWT_SECRET=703c353c3aca79c969ea026a4f34b7966189d03af7f7622573bbe7d513da27e89054e57c75f7b23fe022f8bb710470c23c4beb7da1ba0489d44bafaf7e3536ef +SESSION_SECRET=e50a6a1b5727ca8c7f2d89f2a7b46dd740248f800df59ea32105230505dd752900cc6389e2b2a14300e734bec84694fbe1e8f1810e769899e99292c02588dbeb + +# File Storage +UPLOAD_DIR=./uploads +MAX_FILE_SIZE=52428800 # 50MB + +# Meilisearch +MEILISEARCH_HOST=http://localhost:7700 +MEILISEARCH_MASTER_KEY=8b152b3f62a892753f0b1ca9c2c056a1b250a8708ec540a7802e39b3b3623c8f + +# Redis +REDIS_HOST=localhost +REDIS_PORT=6379 +REDIS_PASSWORD=866b18e464195f4049bbcf77eb1ed73bb6c6edd66949a8bb45171f30f517dc9d + +# OCR +OCR_LANGUAGE=eng +OCR_MIN_TEXT_THRESHOLD=50 +FORCE_OCR_ALL_PAGES=false + +# Logging +LOG_LEVEL=info +LOG_FILE=./logs/navidocs.log + +# CORS (update with actual domain) +CORS_ORIGIN=https://navidocs.yourdomain.com