2025 — 2026
YouTube Transcript Intelligence Studio
Searches YouTube, pulls transcripts through a two-source fallback, translates everything into English, and makes the whole library searchable with FAISS.
The problem
A lot of what people actually say about a product ends up on YouTube, and video is miserable to research. Working through a shortlist on one ingredient trend means sitting through most of each video to find the part that matters. The hard part isn't the analysis though, it's getting the text at all. Transcripts go missing often enough that one source leaves holes, and when they do exist they're frequently not in English.
Running system
One question answered across the whole transcript corpus. Worth reading the third result: the model points out that it's a song rather than an explainer. That's grounding doing its job instead of pattern-matching the title. 
100 videos fetched, 74 dropped on duration, 16 for missing transcripts. The header shows the Apify fallback wired up and ready. 
Korean, Japanese and Hindi videos sitting alongside the English ones, all translated before embedding so a single vector space covers the whole library.
Architecture
- 01
Search YouTube by keyword, date range and sort order through the Data API, pulling views, likes, comments and duration for each video.
- 02
Fetch transcripts with youtube-transcript-api, falling back to an Apify actor when the primary source comes back empty.
- 03
Translate anything non-English into English before embedding, so the whole corpus shares one vector space.
- 04
Chunk at 900 tokens, embed with OpenAI, index in FAISS, and pull the top 20 chunks per question.
Decisions that mattered
Normalize language before embedding, not after
Embeddings aren't reliably aligned across languages, so a mixed-language index fragments without telling you. An English question pulls back only the English part of the corpus, and the answer still reads fine. That's the bug I was most worried about here, because nothing errors and nothing looks broken. Translating everything to English at ingestion gives one coherent vector space, and it moves the failure to ingestion time where I can actually see it.
Two transcript sources, not one
Transcript availability isn't something you can rely on. Auto-captions get disabled, region-locked, or were never generated in the first place. With a single fetcher, coverage comes down to luck, and the gaps aren't evenly spread: the videos worth watching are often the ones with captions switched off. Falling back to an Apify actor turns a miss into a slower hit.
900-token chunks with top-20 retrieval
Speech is much less dense than writing. A presenter spends a minute making a point an author would make in one sentence. Chunk sizes tuned for prose cut through the middle of a thought and leave the conclusion stranded in the next chunk. 900 tokens is enough to hold a whole argument, and pulling 20 of them gives the model spread across several videos, so it can notice a pattern instead of echoing one clip.
Collection and indexing kept separate
The expensive parts are API quota, transcript fetching and translation. Doing those once into a saved, deduplicated dataset makes the FAISS index cheap to throw away and rebuild, which keeps chunk size and retrieval depth easy to experiment with.
Stack
- Retrieval
- LangChainFAISSOpenAI Embeddings
- Ingestion
- YouTube Data API v3youtube-transcript-apiApify
- Interface
- StreamlitPandas