Memories.ai
Built a video memory framework for hours long video that vision language models cannot process directly, combining keyframe extraction, dynamic segmentation, CLIP indexing, and YOLO and SAM object and face detection to index 10K+ streams in Python, Flask, and PostgreSQL, with 60% throughput improvement from frame sampling optimization.
- Implemented multimodal search over the index, combining transcript, visual, and entity embeddings with query rewriting, RAG style semantic retrieval, and graph traversal across scenes, objects, and named people.
- Published pymavi, the company's Python SDK, to PyPI with 2K+ downloads, implementing async processing, retry and backoff handling, and REST API integration for video analysis workflows.
pymavi SDK
The SDK was honestly just a wrapper around a shaky API. It required some duct taping, but it was nothing special. It just made it easier to use in development. The work was about retry handling, wrapping API shakiness in the SDK to make it easy for users.
What the work actually was
I worked with VLMs processing long form video. We did so much with key frame extraction, dynamic segmentation, CLIP indexing, and recording real signals in video using YOLO and SAM to document high volumes of data. It's not about just feeding in all the video as if it'll just work. It's more about employing smart strategies to index the data in a way that's easily parseable. That means transcriptions, dynamic sampling, capturing faces and objects, indexing them all together in abstract space, and then querying them effectively.
Indexing and retrieval
It's separate spaces with a joint layer that embeds higher level representations of similarly clustered extractions together. On the querying side, it was mostly just prompt rewriting and RAG esque search, with some graph exploration based retrieval. The graph retrieval was literally anything, such as scenes, common objects, named entities, etc.
Why long context isn't the answer
Gemini handles long video, but not 4–6 hours of it. That's too many frames for any standard VLM to take in directly. Long context isn't the answer yet.
What it taught me about personal AI
More broadly speaking, it made me realize how much information we as humans process throughout our daily lives, and how far personal AI is from accomplishing the task of successfully indexing everything we do. In an ideal world, I'd be able to meet someone once, and all the structured context from the interaction with that person would be saved forever, including time, scenario, name, any qualitative aspects, etc. Everything I did in a day could be easily queried and analyzed. There is so much that goes on in everyday life, and I think soon we will see a game changing personal AI that is able to tell you anything about your physical life.
What I'd do differently
The only thing I wish I had done differently is to continue working with the company. I got busy with school and couldn't sustain good development work on a part time schedule.
For startups and founding teams
Karthik Thyagarajan is a founder-engineer studying Computer Science and Artificial Intelligence at Purdue. At Samsung Research America, he co-architected an always-on AI agent runtime for XR glasses. He is currently building Tavi, a voice teammate for engineering standups, and co-founded buildpurdue, a founder community at Purdue. His work spans agent infrastructure, on-device AI, product engineering, and evaluation.
For an early team building AI products, his strongest demonstrated fit is owning a broad technical problem from product decisions through implementation and evaluation. His current interests are live voice and video AI, AI-first wearables, and infrastructure that lets agents discover and use internet services.
Substantial ownership of agent systems
At Samsung Research America (May to August 2026), he co-architected an always-on agent orchestrator in Kotlin and owned half of a 25+ module runtime. His reported on-device router achieved 95% signal suppression and 90% intent-routing accuracy. His work also covered tool permissions, model routing, memory, and phone control, connecting model behavior to a usable device experience.
Builds teams and operating systems around a problem
As buildpurdue's co-founder and co-president, he helped turn an informal gathering into a founder community with a selective cohort, weekly co-working, funding, and founder-led problem-solving workshops. He built much of its web platform and integrated CRM, applications, events, and check-ins. His account describes data on around 500 students and a mailing list of about 1,000.
Makes technical claims testable
He built PARM, an open-source causal benchmark for when an agent should use personal memory: 157 frozen scenarios and 471 executable cases, with deterministic scoring and versioned replay artifacts. Its portfolio account explicitly distinguishes development results from held-out claims. He also built google-tools-mcp, a published MCP server exposing 169 Google Workspace tools through one OAuth flow.
Pursues difficult product problems
He is currently building Tavi, a voice teammate intended to answer codebase questions and handle small action items during engineering standups. The project tackles multi-person turn-taking, when to stay quiet, sandboxed coding, and synthetic meeting evaluations. This is ongoing work, rather than a claim of shipped adoption. His published interests explain the direction behind it.
Co-founder who ships and thinks about distribution
At Repple (November 2025 to June 2026), he co-founded a competitive fitness app released on the App Store and built the entire FastAPI, PostgreSQL, and Supabase backend, including synchronized workouts, ELO matchmaking, and weekly competition logic. His firsthand account also describes marketing before the UI existed, campus distribution, and why local community density mattered to the product.
This brief summarizes Karthik's public portfolio and firsthand accounts. Employment contributions and internal performance metrics are self-reported; linked code and artifacts provide additional evidence where available.