Quick Overview This workflow starts a ScraperAPI crawler job from an incoming webhook request and stores each crawled page returned via a callback webhook into an n8n Data Table. How it works Receives a POST request on a webhook with crawl inputs like startUrl, optional includeRegexp, and optional maxDepth. Builds ScraperAPI crawler parameters, including defaults (includeRegexp scoped to the start URL, maxDepth 1, country code, and render setting) and a callback URL for results. Creates a ScraperAPI crawler job and returns the job details in the webhook response. Receives per-page crawl results from ScraperAPI on a second webhook callback endpoint. Filters out job-summary payloads and failed page fetches, then extracts each page’s URL, markdown content, status code, and scrape timestamp. Upserts the parsed page results into an n8n Data Table named crawl_results keyed by URL. Setup Create and connect your ScraperAPI credentials in n8n so the workflow can create crawler jobs. Run the manual storage setup once (or ensure it exists) to create the crawl_results Data Table with the required columns.
Tags
Related workflows
See all General Automation→Sync Shopify customers with Cegid Y2 on create or update
Quick overview This workflow receives Shopify customer create/update webhooks and syncs the customer record to CEGID Y2 by creating the customer when missing or updating the existing customer when CEGID reports a duplicate. How it works Receives a POST webhook from Shopify when a customer is created or updated. Builds a CEGID Y2 customer payload with identifiers, tax/currency details, and the default store/workspace settings. Formats the customer address and chooses whether to send individual or company contact details based on whether a company name is present. Sends the assembled payload to CEGID Y2 to create the customer. If CEGID returns a “customer already exists” error, extracts the existing CEGID customer ID from the error message and sends a PATCH request to update that customer in CEGID. If the create request fails for any other reason, stops the workflow and surfaces the CEGID error. Setup In Shopify, create a customer webhook for create/update events and point it to this workflow’s webhook URL, matching the allowed shop domain. Add CEGID Y2 HTTP Basic Auth credentials and ensure the API user has permission to create and update customers. Update the CEGID API base URL, wo
Solve image CAPTCHAs via webhook using CaptchaSonic
Quick overview This workflow exposes a POST webhook that accepts base64 CAPTCHA images and uses the CaptchaSonic community node to solve multiple CAPTCHA types (OCR, reCAPTCHA v2, AWS WAF, TikTok, Binance, and more), returning a standardized JSON response. How it works Receives a POST request on the captchasonic-solve webhook containing a JSON body with a type and required fields like image (base64) and sometimes question. Validates the request payload, checks required fields for the requested CAPTCHA type, and routes the request to the matching solver. Sends the image challenge to CaptchaSonic using the appropriate recognition operation for the selected type. Normalizes the CaptchaSonic result into a consistent JSON structure with success, type, solution, and a solvedAt timestamp. Returns the formatted solution to the original webhook caller, or responds with HTTP 400 and an error message if validation fails. Setup Self-host n8n and install the n8n-nodes-captchasonic community node. Create and select a CaptchaSonic API credential (API key from https://my.captchasonic.com) for the CaptchaSonic nodes. Activate the workflow, copy the production webhook URL for captchasonic-solve, and
Proxy OpenAI-style chat completions to Gemini with async webhooks
Quick overview This workflow exposes an OpenAI-compatible /v1/chat/completions endpoint in n8n that forwards requests (including optional file attachments) to a configurable LLM HTTP provider, supporting both synchronous replies and asynchronous processing via a callback URL. How it works Receives a POST request on /v1/chat/completions via a webhook, accepting JSON or multipart form-data with optional binary file uploads. Converts any uploaded files into OpenAI-style message content (images as image_url data URLs and other files as text entries) and merges them into the last user message. Prepares a job ID, extracts optional callback and provider override settings (URL and headers), and keeps the original request body for later use. If a callback URL is provided, immediately returns HTTP 202 with the job ID, then sends the request (without callback/provider fields) to the configured LLM provider endpoint. Formats the provider response (or error) into a completion payload with a status code and posts the result to the callback URL with job metadata. If no callback URL is provided, sends the request to the LLM provider synchronously and returns the provider response (or error) to the