Cesar Petrescu

Lab · AI

  • Production
  • Platform

Sparkboarding

Turn enterprise documents into searchable knowledge with citations and compliance

Multi-tenant knowledge platform that ingests enterprise documents (including scanned PDFs), lets teams chat with citations, generate compliance quizzes, and explore relationships in a knowledge graph. GPU microservices handle heavy OCR and conversion with fallback so the main API stays responsive.

  • FastAPI
  • React 18
  • TypeScript
  • PostgreSQL
  • Redis Stack
  • Neo4j
  • Keycloak
  • MinIO
  • LiteLLM

Open the project

How it works

From an enterprise document to a cited answer

The ingestion and retrieval paths preserve tenant scope while selecting OCR, vector, graph, or multi-agent processing based on the document and question.

  1. Ingest
  2. Process
  3. Index
  4. Answer

Step 1 of 4 · FastAPI · Keycloak · MinIO

A document enters a personal, team, or organization scope

The API validates the Keycloak tenant context, records catalog state, and stores the original file in MinIO before expensive processing begins.

What moves forward A durable source document with an enforced retrieval scope.

Step 2 of 4 · Marker · OCR · GPU workers

Specialized services convert text, layout, and scans

Marker converts PDF or DOCX content into structured Markdown; GPU OCR handles image-heavy scans. Slow or unavailable processors can fall back without freezing the primary API.

What moves forward Normalized text, layout, tables, and breadcrumb metadata.

Step 3 of 4 · Redis HNSW · Neo4j

Chunks become vector and graph knowledge

The platform enriches chunks, creates embeddings, writes coarse and fine HNSW indexes in Redis, and projects document relationships and semantic connections into Neo4j.

What moves forward Tenant-scoped vector candidates and traversable relationships.

Step 4 of 4 · LiteLLM · SSE · GraphRAG

The selected agent retrieves, verifies, and cites

Quick, Balanced, Deep Research, or Graphy chooses the appropriate retrieval path. Results stream as phases, tokens, sources, and diagnostics; Graphy permits sanitized read-only Cypher only.

What moves forward A cited answer, quiz, report, or graph view with source traceability.

Surfaces

Documents · Upload, scope, and index
Markdown/OCR ingestion, catalog telemetry, embedding controls, and chunk rewrite diagnostics.
Chat · Citations + live diagnostics
Multi-agent sessions streamed over SSE with phases, sources, and router telemetry.
Compliance · Quizzes + reports
Generate quizzes from selected docs, score attempts, and export org-wide results.
Knowledge surfaces · Graph + Canvas
Neo4j visualization with semantic connection jobs, plus a canvas to link chats and docs.
Governance · Multi-tenant RBAC
Keycloak OIDC (PKCE), org/team scopes, admin tooling, and audit views.
Reach · Telegram bot
Out-of-band chat + quizzes, plus notifications.

Assistant modes

Quick Responder

Fast answers with citations for day-to-day questions.

Redis dual-index retrieval (coarse + fine) with semantic cache + recency boost; streams tokens + sources over SSE.

Balanced Researcher

More accurate answers when the question is ambiguous or needs verification.

Hybrid retrieval + cross-encoder reranking; selects the best chunks before generating a cited response.

Deep Research

End-to-end investigations that compare options and produce structured outputs.

Multi-step plan: iterates retrieval, refines queries, and synthesizes results; can delegate to other agents.

Graphy

Relationship questions: what connects to what, and where a concept is referenced.

GraphRAG: generates sanitized read-only Cypher, traverses Neo4j, enriches with chunk text; vector fallback if the graph is sparse.

Greg

Long-term assistant: remembers decisions, tracks todos, and pushes reminders/notifications.

Memory/docs/todos sidecar with delegation to other agents; feeds updates to the UI and Telegram workflows.

Document pipeline

  1. Upload (personal / team / org scope)
  2. Store raw + derived files (MinIO)
  3. Convert to Markdown (Marker) + OCR when needed
  4. Chunk + enrich metadata (breadcrumbs, section paths)
  5. Embeddings + dual-index (Redis HNSW)
  6. Graph projection + semantic connections (Neo4j)
  7. Chat/retrieval selects the best path per agent

Integrations

Identity & tenancy

  • Keycloak OIDC (PKCE) with org-admin and superadmin surfaces.
  • Tenant context is enforced server-side across documents, chat, and graph queries.

Document processors (microservices)

  • Text-first conversion service (Marker) for PDF/DOCX to Markdown with layout + tables.
  • Dedicated OCR service for image/scanned PDFs (GPU-enabled).
  • Automatic fallback to local processing when a processor is slow/unavailable.

Retrieval + GraphRAG

  • Redis Stack HNSW vector search with a dual-index strategy (coarse + fine) and semantic cache.
  • Neo4j graph projection for chunk/document relationships and semantic connections.
  • Graphy agent runs sanitized read-only Cypher (no writes, no multi-statement).

Channels

  • Web app (React + TypeScript)
  • Telegram bot (long-poller for chat/quizzes/notifications)
  • Optional voice transcription integration

Production notes

Streaming UX
Chat streams phases, tokens, sources, and agent telemetry to the UI over SSE.
Tenant isolation
Graph queries and document retrieval enforce organization scope; no cross-tenant leakage.
Graph query safety
Graphy only allows read-only Cypher (no CREATE/MERGE/DELETE/SET, no multi-statement).
Resilient ingestion
Document processing is isolated into microservices; the API can fall back to local conversion.