— Portfolio
Hi, my name is Vishnu,
I am a |
MEng in AI for Product Innovation from Duke University. I'm obsessed with end-to-end Computer Vision, Product Development, and Generative AI—and I love diving unreasonably deep into hard problems around vision, GenAI, and LLMs until they turn into shippable systems.
A bit about me
I am extremely passionate about all aspects of Computer Vision, Product Development and Generative AI. I embrace working on research and development in those topics to improve lives and experiences.
The atmosphere in which I work is essential, and I find it difficult to collaborate with individuals who aren't inspired. I'm driven to grow as a developer and learn from others.
I enjoy spending time with my friends, training myself in the arts of Mixed Martial Arts, going to the gym, and at the beach where I Surf and Scuba dive.
My Experience
Human Archive (YC W26)
MLE Intern
- ●Architected a spatiotemporal pre-filtering pipeline on frozen V-JEPA 2 embeddings to gate egocentric headcam windows before Gemini inference, reducing annotation costs by 80%.
- ●Designed and trained a 578K-parameter attentive probe on V-JEPA-2 ViT-g (1B parameters, frozen) embeddings, achieving 0.85 F1 in detecting factory worker action cycles across 9 factories.
- ●Integrated multimodal industrial sensing streams including head mounted egocentric video, hand mounted object view cameras, and IMU based pressure glove sensors to curate and open source a large scale embodied factory dataset.
TRUST Lab Duke DeepTech
Lead AI Developer
- ●Developed voice-based liaison agents as part of a research project in collaboration with OpenAI to explore how voice-based, conversational LLM agents can function as 'research translators' in interdisciplinary collaborations.
- ●Orchestrated low-latency (800ms) text-to-speech agents using OpenAI Whisper, leveraging an orchestrator-worker workflow and agent memory for real-time information retrieval.
JPMorgan Chase & Co.
AIML Associate Intern
- ●Led end-to-end automation of Suspicious Activity Report (SAR) narrative generation using AWS Bedrock, designing structured chain-of-thought prompting and context engineering workflows.
- ●Reduced production time by 90% and lowered operational costs by $50K by optimizing prompt structure, retrieval context, and output standardization for compliance review.
- ●Built a SHAP-based explainability framework to interpret fraud and AML model predictions, improving transparency and auditability for risk and compliance teams.
ProAxion
Student Machine Learning Engineer
- ●Built an industrial IoT chatbot enabling natural-language queries against machine health and maintenance data.
- ●Integrated with ProAxion's sensor platform for real-time equipment status and predictive maintenance insights.
Deakin University
Research Intern
- ●Developed a pipeline capable of on-edge video text detection using the Google Vision API, and wrote the C# code for the wrapper capable of running on a Microsoft Hololens headset.
- ●Collaborated with Dr. William Raffe to deploy it as a scalable implementation.
Sentics GmbH
Computer Vision Engineer
- ●Engineered an algorithm that accurately estimated the base point of an object using pose keypoint data from TRTPose and 2D-3D correspondence, resulting in a 100% improvement in object location estimation accuracy.
- ●Conducted extensive research and experimentation with various object and keypoint tracking methods to evaluate performance trade-offs.
Miniscule Technologies
Cloud AIOps Engineer
- ●Performed extensive research on evaluating major cloud service providers and their readiness for industrial 5G use cases.
- ●Deployed an on-edge custom face detection model through Amazon Rekognition trained on employee data stored on Amazon S3, achieving an accuracy of 88% on the Hikvision AcuSense camera module.
Education
Duke University
MEng in AI for Product Innovation
GPA: 3.72 / 4.0
Vellore Institute of Technology
Integrated MTech in Computer Science
Specialization in Data Science
Bala Vidya Mandir Senior Secondary School
Technologies
Languages
Frameworks
Platforms
Filter by skill
Things I've Developed
project — 00
SpeakWhenSpoken2
A Voice AI agent built for group conversations — not 1-on-1 (those are solved). Handles overlapping speakers, fragmented context, backchannels misread as turn endings, and silence that belongs to someone thinking rather than open floor.
Built on Voice Activity Projection, Streaming Sortformer diarization, and Mercury 2 (1000+ tok/s diffusion LLM). Runs on a single rented 3090 for under $10 in compute.
Tap to view details
project — 01
EgoCut
Cheap spatiotemporal filtering for egocentric factory video — so Gemini only annotates what matters. Reduces annotation costs by filtering out 80%+ of unusable footage before it reaches expensive long-context LLM calls.
Uses Meta's V-JEPA 2 (1B params, frozen) as a backbone with a 578K-parameter attentive probe trained on Gemini-calibrated labels across 9 factories, achieving 0.85 F1 on action cycle detection.
Tap to view details
project — 02
PicoChat
Implementing inference optimizations for Large Language Models using Karpathy's nanochat as the base. Built a working distillation pipeline with MQA, multi-token prediction heads, speculative decoding, structured pruning, and INT8 quantization.
Trained a 375M parameter student model from a 2B teacher on a single A100 for under $10. Achieved 70.7% model compression (1.2GB to 363MB) via INT8 quantization while keeping the full inference pipeline production-ready.
Tap to view details
project — 03
PersonaPlex Voice Diarization
A speaker diarization system that identifies and segments different speakers in audio recordings, enabling per-speaker analysis and transcription.
Combines state-of-the-art voice activity detection with embedding-based clustering to accurately separate overlapping speakers in real-world audio.
Tap to view details
project — 04
Duke Agentic Chatbot
A dual-implementation agentic chatbot for Duke University: one built from scratch with Flask, LangGraph, and Gemini, and another using Google Cloud Conversational Agents.
Features a structured agentic workflow -- Planning, Tool Execution, Thinking, and Evaluation -- querying Duke Events API, Google Custom Search, and program-specific data sources.
Tap to view details
project — 05
RAG Evaluation Framework
A comprehensive evaluation framework for Retrieval-Augmented Generation systems, benchmarking retrieval quality, generation faithfulness, and end-to-end performance.
Uses RAGAS metrics across different embedding models and chunk strategies, providing actionable insights for RAG pipeline optimization.
Tap to view details
project — 06
HyperExplainer
A Chrome extension that uses AI to explain hyperparameters in ML code. Hover over any parameter to see an interactive explanation with visualizations.
Powered by OpenAI and D3.js, it helps practitioners understand tuning implications in context without leaving their browser.
Tap to view details
project — 07
ProAxion Industrial Chatbot
A capstone project building an industrial IoT chatbot for ProAxion, enabling natural-language queries against machine health and maintenance data.
Integrates with ProAxion's sensor platform to provide real-time equipment status, predictive maintenance insights, and historical trend analysis through a conversational interface.
Tap to view details
project — 08
Blind.Ai
An app to make the lives of visually impaired people a little more ordinary, powered by Flutter, Python, Twilio, and Flask.
Offers an intuitive UI facilitating key features: SOS emergency alerts, real-time object detection, currency recognition, and text-to-speech OCR -- all accessible with minimal effort.
Tap to view details
project — 09
LLMTalk
A Streamlit-based application that lets you chat with your audio files, powered by LangChain, ChromaDB, and OpenAI.
Chat with any YouTube video by inputting the URL -- the pipeline extracts the transcript and feeds it into GPT-4 for context-aware Q&A.
Tap to view details
Other Projects
Multimodal RAG Chatbot
A retrieval-augmented generation chatbot that handles both text and image queries, embedding multimodal documents into a vector store for context-aware answers.
Tap to view details
GNN Watch Market Analysis
Graph Neural Network-based analysis of the luxury watch market, modeling brand-model-feature relationships as a graph to uncover pricing patterns and market dynamics.
Tap to view details
ImproViz
A real-time voice-to-visualization tool that converts spoken descriptions into interactive data charts and diagrams on the fly.
Tap to view details
Anemia Cell Detection
A computer vision pipeline that segments and classifies red blood cells from microscope images to detect anemia subtypes using deep learning.
Tap to view details
XR Dyslexia-Friendly Text Detector
An XR-based application that detects text in the environment and converts it to a dyslexia-friendly font (OpenDyslexic) using Google Cloud Vision API and Unity.
Tap to view details
Research & Notebooks
Indian Sign Language Detection
Published at AII2023 Dubai -- a MediaPipe + Keras pipeline that recognizes ISL alphabets from video and converts them to speech in real time.
Tamil Character Recognition
Published at ICCUBEA-2023 -- CatBoost classifier with Optuna tuning for Tamil handwritten character recognition using shape-based features.
Adversarial Patches with FGSM
Explores adversarial robustness by generating FGSM-based adversarial patches to fool image classifiers, and evaluates defense strategies.
Flow Matching Adversarial Attacks
Studies adversarial attacks on flow-matching generative models, analyzing how perturbations to the learned vector field degrade generation quality.
GAM for Customer Churn
Applies Generalized Additive Models to customer churn prediction, offering interpretable smooth feature effects compared to black-box alternatives.
Dimensionality Reduction for XAI in LLMs
Visualizes high-dimensional LLM embeddings using t-SNE, UMAP, and PCA to understand how language models represent semantic structure.
SHAP & Partial Dependence Plots
Model-agnostic explainability using SHAP values and PDP to attribute feature importance and visualize marginal feature effects.
LIME Explainability
Local Interpretable Model-agnostic Explanations applied to image and tabular classifiers, highlighting which input regions drive predictions.
iModels: Interpretable ML
Benchmarks inherently interpretable models -- rule lists, decision sets, and optimal trees -- against black-box alternatives on real-world datasets.
Writing
What Others Say
Vishnu was part of a three person Capstone Project ProAxion worked with to explore AI, vision, and engagement applications that would simplify our user interface such that maintenance personnel could take action of Machine Health Diagnostics we provided.
What I appreciated most about Vishnu's contribution was that he implemented advanced technology with a clear grasp that many maintenance users don't have advanced IT skills and need intuitive tools they can apply with urgency when the plant is down.
Vishnu always came prepared to demonstrate working examples of his work to engage and capture input from team members, was on time and present in the moment during our work, and showed a high level of commitment/accountability.
