navidocs/REDIS_KNOWLEDGE_BASE_USAGE.md

Redis Knowledge Base - Quick Reference

Status: LIVE Location: localhost:6379 Total Keys: 2,438 Memory: 1.15 GB Schema: navidocs:{branch}:{file_path}

---

One-Line Commands

Get File Content

# Extract and display a file
redis-cli GET "navidocs:navidocs-cloud-coordination:package.json" | \
  python3 -c "import json,sys; d=json.load(sys.stdin); print(d['content'])"

List Files in a Branch

# Show all files from a branch
redis-cli KEYS "navidocs:navidocs-cloud-coordination:*" | wc -l

# First 5 files
redis-cli KEYS "navidocs:navidocs-cloud-coordination:*" | head -5

Search by Extension

# All Markdown files
redis-cli KEYS "navidocs:*:*.md"

# All Python files
redis-cli KEYS "navidocs:*:*.py"

# All JSON configs
redis-cli KEYS "navidocs:*:*.json"

Get Metadata

# Display file author and commit date
redis-cli GET "navidocs:navidocs-cloud-coordination:SESSION_RESUME_AGGRESSIVE_2025-11-13.md" | \
  python3 -c "import json,sys; d=json.load(sys.stdin); print(f\"Author: {d['author']}\nCommit: {d['last_commit']}\nSize: {d['size_bytes']} bytes\")"

---

Python API

Initialize Connection

import redis
import json

r = redis.Redis(host='localhost', port=6379, decode_responses=True)
print(f"Connected: {r.ping()}")

Retrieve File

def get_file(branch, filepath):
    key = f"navidocs:{branch}:{filepath}"
    data = json.loads(r.get(key))
    return {
        'content': data['content'],
        'author': data['author'],
        'last_commit': data['last_commit'],
        'size': data['size_bytes']
    }

# Usage
file_data = get_file('navidocs-cloud-coordination', 'package.json')
print(file_data['content'])

List Branch Files

def list_branch_files(branch, pattern="*"):
    prefix = f"navidocs:{branch}:{pattern}"
    keys = r.keys(prefix)
    files = [k.replace(f"navidocs:{branch}:", "") for k in keys]
    return sorted(files)

# Usage
files = list_branch_files('navidocs-cloud-coordination', '*.md')
print(f"Found {len(files)} markdown files")

Search for Files

def search_files(pattern):
    keys = r.keys(f"navidocs:*:{pattern}")
    results = {}
    for key in keys:
        branch, filepath = key.replace('navidocs:', '').split(':', 1)
        if branch not in results:
            results[branch] = []
        results[branch].append(filepath)
    return results

# Usage - find all PDFs
pdfs = search_files('*.pdf')
for branch, files in pdfs.items():
    print(f"{branch}: {len(files)} PDFs")

Iterate All Files

def iterate_all_files(branch=None):
    pattern = f"navidocs:{branch}:*" if branch else "navidocs:*:*"
    cursor = 0
    while True:
        cursor, keys = r.scan(cursor, match=pattern, count=100)
        for key in keys:
            data = json.loads(r.get(key))
            yield {
                'key': key,
                'filepath': key.split(':', 2)[2] if ':' in key else key,
                'author': data['author'],
                'commit': data['last_commit'],
                'size': data['size_bytes']
            }
        if cursor == 0:
            break

# Usage - process all files
for file_info in iterate_all_files('navidocs-cloud-coordination'):
    if file_info['size'] > 100000:  # > 100KB
        print(f"Large file: {file_info['filepath']}")

Get Branch Statistics

def branch_stats(branch):
    pattern = f"navidocs:{branch}:*"
    keys = r.keys(pattern)

    total_size = 0
    file_types = {}

    for key in keys:
        data = json.loads(r.get(key))
        total_size += data['size_bytes']

        filepath = key.split(':', 2)[2]
        ext = filepath.split('.')[-1] if '.' in filepath else 'no-ext'
        file_types[ext] = file_types.get(ext, 0) + 1

    return {
        'files': len(keys),
        'total_size_mb': total_size / (1024 * 1024),
        'file_types': file_types
    }

# Usage
stats = branch_stats('navidocs-cloud-coordination')
print(f"Files: {stats['files']}")
print(f"Size: {stats['total_size_mb']:.1f} MB")
print(f"Types: {stats['file_types']}")

---

Branches Available

Processed (3 branches)

1. navidocs-cloud-coordination (831 files)

   • Base: navidocs:navidocs-cloud-coordination:

2. claude/navidocs-cloud-coordination-011CV53By5dfJaBfbPXZu9XY (803 files)

   • Base: navidocs:claude/navidocs-cloud-coordination-011CV53By5dfJaBfbPXZu9XY:

3. claude/session-2-completion-docs-011CV53B2oMH6VqjaePrFZgb (804 files)

   • Base: navidocs:claude/session-2-completion-docs-011CV53B2oMH6VqjaePrFZgb:

Not Processed (20 branches)

These branches couldn't be checked out (see REDIS_INGESTION_COMPLETE.md)

---

Example Use Cases

1. Find All Configuration Files

patterns = ['*.json', '*.yaml', '*.yml', '*.env', '*.config']
for pattern in patterns:
    keys = r.keys(f"navidocs:*:{pattern}")
    print(f"{pattern}: {len(keys)} files")

2. Extract README Files

readmes = r.keys("navidocs:*:**/README.md")
for key in readmes:
    data = json.loads(r.get(key))
    print(f"\n=== {key} ===")
    print(data['content'][:500])

3. Find Recent Changes

from datetime import datetime, timedelta

recent = datetime.now() - timedelta(days=7)

for file_info in iterate_all_files():
    commit_date = datetime.fromisoformat(file_info['commit'])
    if commit_date > recent:
        print(f"Updated: {file_info['filepath']} by {file_info['author']}")

4. Identify Large Files

large_files = []

for file_info in iterate_all_files():
    if file_info['size'] > 1_000_000:  # > 1MB
        large_files.append((file_info['filepath'], file_info['size']))

for filepath, size in sorted(large_files, key=lambda x: x[1], reverse=True)[:10]:
    print(f"{filepath}: {size / (1024*1024):.1f} MB")

5. Decode Base64 PDFs

import base64

def extract_pdf(branch, pdf_path):
    key = f"navidocs:{branch}:{pdf_path}"
    data = json.loads(r.get(key))

    if data['is_binary']:
        pdf_bytes = base64.b64decode(data['content'])
        return pdf_bytes
    else:
        return None

# Usage
pdf_data = extract_pdf('navidocs-cloud-coordination', 'uploads/somefile.pdf')
if pdf_data:
    with open('output.pdf', 'wb') as f:
        f.write(pdf_data)

---

Maintenance

Check Health

# Ping server
redis-cli ping
# Output: PONG

# Memory stats
redis-cli INFO memory | grep -E "used_memory|peak_memory|fragmentation"

# Check navidocs keys
redis-cli KEYS "navidocs:*" | wc -l
# Output: 2438

Monitor Commands

# Watch real-time commands
redis-cli MONITOR

# Find slowest commands
redis-cli SLOWLOG GET 10

# Clear slow log
redis-cli SLOWLOG RESET

Backup

# Trigger snapshot
redis-cli BGSAVE

# Check backup
ls -lh /var/lib/redis/dump.rdb

# AOF rewrite (if enabled)
redis-cli BGREWRITEAOF

---

Integration Examples

Flask API Wrapper

from flask import Flask, jsonify
import redis
import json

app = Flask(__name__)
r = redis.Redis(host='localhost', port=6379, decode_responses=True)

@app.route('/api/navidocs/<branch>/<path:filepath>')
def get_file(branch, filepath):
    key = f"navidocs:{branch}:{filepath}"
    data = r.get(key)

    if not data:
        return {'error': 'File not found'}, 404

    parsed = json.loads(data)
    return {
        'filepath': filepath,
        'branch': branch,
        'author': parsed['author'],
        'last_commit': parsed['last_commit'],
        'content': parsed['content'][:1000],  # First 1000 chars
        'size_bytes': parsed['size_bytes']
    }

@app.route('/api/navidocs/<branch>/files')
def list_files(branch):
    pattern = f"navidocs:{branch}:*"
    keys = r.keys(pattern)
    files = [k.replace(f"navidocs:{branch}:", "") for k in keys]
    return {'branch': branch, 'files': sorted(files)[:100]}

Automation Script

#!/bin/bash
# Sync Redis knowledge base hourly

while true; do
    echo "Checking for updates..."
    cd /home/setup/navidocs

    # Fetch latest
    git fetch origin

    # Re-ingest if changes detected
    if git status --porcelain | grep -q .; then
        echo "Changes detected, re-ingesting..."
        python3 redis_ingest.py
    fi

    # Wait 1 hour
    sleep 3600
done

---

Troubleshooting

Connection Issues

# Test connection
redis-cli ping

# Check if running
ps aux | grep redis-server

# Restart if needed
redis-server /etc/redis/redis.conf

Data Inconsistencies

# Count keys
redis-cli DBSIZE

# Verify index
redis-cli SCARD navidocs:index

# Should match (2,756 for all keys, 2,438 for index)

Large Memory Usage

# Find biggest keys
redis-cli --bigkeys

# Profile memory
redis-cli --mem-stats

# Consider compression or archival

---

Performance Tips

1. Use Pipelines for multiple operations:

   pipe = r.pipeline()
   for key in keys:
       pipe.get(key)
   results = pipe.execute()
   

2. Batch Scanning to avoid blocking:

   cursor, keys = r.scan(cursor, match=pattern, count=1000)
   

3. Cache Frequently Accessed files in application memory

4. Use KEYS Sparingly - prefers SCAN for large datasets

5. Monitor Slow Queries:

   redis-cli SLOWLOG GET 10
   redis-cli CONFIG SET slowlog-log-slower-than 10000
   

---

Last Updated: 2025-11-27 Ready for Production: YES