VectorPipe
Ingestion pipelines for restless builders
GraphQL360 begat Twirrl; Twirrl is morphing into VectorPipe—a rapid-to-market way to upload anything and push it into vector stores without bespoke glue.
VectorPipe is a unified SvelteKit+tRPC app born from the Twirrl GraphQL stack. No more juggling Render + Vercel deployments.
Projects own their ingestion pipelines—upload PDFs, podcasts, YouTube channels, or web archives and send them through Pinecone/Chroma embeddings.
Users bring their own OpenAI/Pinecone keys. Docs land in S3 during beta, with a "fair use" note so experiments stay cheap.
UI lineage
Twirrl screens → VectorPipe console
The UI mocks below evolved from capture workflows into ingestion pipelines. Same philosophy: keep the work visible, one project at a time.
Twirrl capture selector reused for pipeline project creation.
Project workspace concept that now stores ingestion status and pipeline runs.
List view mock showing how multiple ingestion projects are tracked.
Earlier computer-vision tooling feeding into today's document alignment + QA ideas.
Build log
From walking notes to action items
May 2024 voice memos set the tone: consolidate deployments, drop GraphQL baggage, and expose the ingestion pipeline as fast as possible.
- • Strip remaining scene/image GraphQL types and replace with resource entities.
- • Add project-level pipeline configuration and API key vault.
- • Switch to tRPC for typed endpoints; keep Prisma + Clerk auth wiring.
- • Ship a basic Pinecone query console so people can sanity-check embeddings.
Why it matters
Clients keep asking for repeatable ingestion + retrieval setups. VectorPipe packages the lessons from Lighten, Zolidar, and Twirrl so every LLM residency starts with working plumbing.
Back to Labs