Create AI-Ready Vector Datasets for LLMs with Bright Data, Gemini & Pinecone
Who this is for? This workflow enables automated, scalable collection of high-quality, AI-ready data from websites using Bright Data’s Web Unlocker, with a focus on preparing that data for LLM training. Leveraging LLM Chains and AI agents, the system formats and extracts key information, then stores the structured embeddings in a Pinecone vector database. This workflow is tailored for: ML Engineers & Researchers building or fine-tuning domain-specific LLMs. AI Startups needing clean, structured content for product training. Data Teams preparing knowledge bases for enterprise-grade AI apps. LLM-as-a-Service Providers sourcing dynamic web content across niches. What problem is this workflow solving? Training a large language model (LLM) requires vast amounts of clean, relevant, and structured data. Manual collection is slow, error-prone, and lacks scalability. This workflow: Automatically extracts web data from specified URLs. Bypasses anti-bot measures using Bright Data’s Web Unlocker. Formats, cleans, and transforms raw content using LLM agents. Stores semantically searchable vectors in Pinecone. Makes datasets AI-ready for fine-tuning, RAG, or domain-specific training.
Tags
Related workflows
See all AI Automation→Draft personalized cold emails from Google Sheets using Gemini and HubSpot
Quick overview This workflow batch-processes leads from Google Sheets, researches each person and company using Tavily plus Google Gemini, drafts a personalized cold email, then creates/updates the contact in HubSpot with a research note and a saved email draft before marking the lead as Done in the sheet. How it works Runs manually and sets a batch size that controls how many leads to process per run. Reads leads from Google Sheets, skips rows already marked as Done, and normalizes the selected batch into clean lead fields. Processes the batch one lead at a time and uses Tavily search (last 60 days) with Google Gemini to extract the most recent public activity, pain points, and a personalization hook. Cleans and parses the research JSON so it can be reused reliably downstream. Uses Google Gemini to draft a 90–120 word cold outreach email (subject and body) that incorporates the research and offers a free “Automated LinkedIn Post Creator” n8n workflow. Creates or reuses the HubSpot contact, logs a HubSpot note with the research and drafted email, and saves the email as a HubSpot engagement in DRAFT status. Updates the matching row in Google Sheets (by Email) to set Status to “Done
AI: Summarize podcast episode and enhance using Wikipedia
The workflow automates the process of creating a summarized and enriched podcast digest, which is then sent via email. Note that to use this template, you need to be on n8n version 1.19.4 or later.
Answer Slack knowledge base questions with Notion, Pinecone, and OpenAI
Quick overview This workflow syncs an internal Notion knowledge base to Pinecone on a daily schedule, then runs a Slack bot that answers questions in a specific channel using Pinecone retrieval and an OpenAI chat model, replying in a thread with citations back to the source Notion pages. How it works Runs on a daily schedule to sync your Notion knowledge base into a Pinecone vector index. Fetches all pages from a specified Notion database and then pulls each page’s full block content (including nested blocks). Combines each page’s blocks into a single document and attaches metadata such as the Notion title, URL, and last edited time. Splits each document into overlapping text chunks, generates OpenAI embeddings, and inserts them into Pinecone while clearing and rebuilding the target namespace. Triggers when a new message is posted in the configured Slack channel and ignores bot/system messages to prevent reply loops. Uses an OpenAI-powered agent that always queries Pinecone for relevant chunks (including source metadata) and generates an answer in the user’s language. Posts the response back to Slack as a threaded reply, including cited Notion sources. Setup Create a Notion interna