This commit adds comprehensive image extraction and OCR functionality to the OCR worker:
Features:
- Created image-extractor.js worker module with extractImagesFromPage() function
- Uses pdftoppm (with ImageMagick fallback) to convert PDF pages to high-res images
- Images saved to /uploads/{documentId}/images/page-{N}-img-{M}.png
- Returns image metadata: id, path, position, width, height
OCR Worker Integration:
- Imports image-extractor module and extractTextFromImage from OCR service
- After processing page text, extracts images from each page
- Runs Tesseract OCR on extracted images
- Stores image data in document_images table with extracted text and confidence
- Indexes images in Meilisearch with type='image' for searchability
- Updates document.imageCount and sets imagesExtracted flag
Database:
- Uses existing document_images table from migration 004
- Stores image metadata, OCR text, and confidence scores
Dependencies:
- Added pdf-img-convert and sharp packages
- Uses system tools (pdftoppm/ImageMagick) for reliable PDF conversion
Testing:
- Created test-image-extraction.js to verify image extraction
- Created test-full-pipeline.js to test end-to-end extraction + OCR
- Successfully tested with 05-versions-space.pdf test document
Error Handling:
- Graceful degradation if image extraction fails
- Continues OCR processing even if images cannot be extracted
- Comprehensive logging for debugging
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Import dotenv in worker to load .env configuration
- Specify explicit path to server/.env file
- Update Meilisearch config to use changeme123 as default key
- Add debug logging to Meilisearch client initialization
- Add meilisearch-data/ to .gitignore
OCR pipeline is fully functional with 85% confidence:
- PDF upload ✅
- Queue processing ✅
- PDF to image conversion ✅
- Tesseract OCR ✅
- Database storage ✅
Remaining issue: Meilisearch authentication needs to be resolved
to enable search indexing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace Tesseract.js with local tesseract CLI due to CDN 404 issues
- Fix queue name mismatch (ocr-processing vs ocr-jobs)
- Local tesseract uses pre-installed training data
- Faster and more reliable than downloading from CDN
\ud83e\udd16 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
