Trigger a web scraper from Zapier and get the data back
How to trigger a web scraper from Zapier and get the data back: install the Chrome extension, click Build from description, tell the agent what you want, and follow its prompts. A working bot is built and tested in minutes.
Here is how the automation works. To trigger a web scraper from Zapier, add a Webhooks by Zapier POST action that sends the URLs to https://lar.axiom.ai/api/v3/trigger. The cloud run takes them in a Receive data from another app step, scrapes the page, and a Trigger webhook step posts the data to a second Zap with a Catch Raw Hook trigger, which sends it on. Any Zapier trigger can start it: a new row in a sheet, a form submission, a new email, a schedule.
Build the Zapier scraper from a description
Describe it and click the button: the agent builds and tests the scraper in your Chrome extension; the two Zaps are set up in Zapier, in the questions below. Webhooks need a plan that includes them (pricing) and your API key (Authentication).
- Take the URL Zapier sends when a row is added
- Open it and scrape the title and price
- Post the title, price and row number to Zapier
You need a Zapier account on a paid plan, where Webhooks by Zapier is, and the Google Sheet your team pastes product URLs into. Make the second Zap's Catch Raw Hook first so you can give the agent its URL, or leave the Trigger webhook step's Endpoint empty and paste it in later. Name the automation something you can type exactly, such as Scrape product pages.
Build the Zapier scraper step by step
The steps the agent builds, as the Builder shows them: each step by its name, with the fields to set under it. Add them yourself in this order in the step-by-step builder, or open the agent's result there to adjust it.
- 1.0
Receive data from another appTest data: a row with the URL and the sheet's row number, in the shape the Zap sends
- 2.0
Loop through data- repeats for every row in
[webhook-data]
- 2.1
Go to pageURL: the first column of[webhook-data]
- 2.2
Get data from bot's current pageSelect: the title and price on the page, one column each
- repeats for every row in
- 3.0
Trigger webhookEndpoint: the second Zap's Catch Raw Hook URLPayload:[scrape-data]
Common questions about the Zapier scraper
How do I start the scraper run from a Zap with Webhooks by Zapier?
With a Webhooks by Zapier POST action after the Zap's trigger:
- In Zapier, click + Create and pick the trigger that suits the job, for example Google Sheets: New Spreadsheet Row.
- Add an action, search for Webhooks by Zapier, and set the action event to POST.
- Set URL to
https://lar.axiom.ai/api/v3/trigger. - Set Payload Type to Json.
- In Data, add three rows: name
keywith your API key, namenamewith the automation's exact name, and namedatawith the table of URLs. For one URL from the trigger, the value is[["followed by the trigger's URL field, then"]]. To send the row number back as well, make it[["+ the URL field +", "+ the Row ID field +"]]. - Leave Wrap Request In Array as No and Unflatten as Yes, then click Continue.
Data takes a name in the left box and a value in the right box. The data value is a JSON table typed into the right box with the trigger's field inserted where the URL goes. Zapier passes it through as JSON because Payload Type is Json, and sends this:
{
"key": "your-api-key-here",
"name": "Scrape product pages",
"data": [["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html", "2"]]
}
Click Test step. The trigger answers straight away:
| Response | What it means |
|---|---|
"OPEN LINK IN BROWSER" with a link | The run started. Open the link to watch it live. |
"status": "queued" | No cloud slot is free yet. The run starts within a few minutes. |
Task not found | The name row does not match the automation's name exactly, including case and spaces. |
Invalid key | The key row is wrong. Copy the key again from the extension's settings. |
How do I catch the scraped data in a second Zap?
With a Catch Raw Hook as the second Zap's trigger:
- In Zapier, click + Create and add a trigger. Search for Webhooks by Zapier and set the trigger event to Catch Raw Hook.
- Copy the URL under Your webhook URL.
- Paste it into the Trigger webhook step's
Endpointand save the automation. - Run the automation once from the builder, then click Test trigger in the Zap. The scraped rows arrive as the raw body.
- Add a Formatter by Zapier step: action event Text, Transform set to Split Text, Input set to Raw Body, Separator a single speech mark, Segment Index set to All (as separate fields). Each scraped value is then a field you can send to any app.
- End with Google Sheets: Update Spreadsheet Row, using the row number that came back to put the title and price next to the URL.
Then turn both Zaps on and fire the first Zap's trigger, for example by adding a row to the sheet: the run starts, scrapes, and the second Zap receives the data. The automation's run reports show each run, and notifications can tell you when one fails.
Can I send several URLs to the web scraper in one Zap?
Yes. The data value is a table, so [["url-1"], ["url-2"]] sends two rows and one run scrapes both. A Looping by Zapier step that posts once per URL starts one run per URL instead, which costs more runtime.
How do I stop Zapier rows without a URL starting a run?
Add a Filter by Zapier step between the trigger and the POST so only rows with a URL start a run. Each call is one run and counts against your runtime.
Can I send different sites to different scrapers from Zapier?
Yes. A Paths by Zapier step can send Amazon URLs to one automation and eBay URLs to another, each with its own scrape and its own name.
Is my axiom.ai API key visible in Zapier?
Yes. Zapier shows the request body in task history, so use a dedicated key where you can, and rotate it from the extension's settings if it is exposed.
Can I trigger a web scraper from Zapier without code?
Yes. The scraper is built from the three lines above with no code, and Webhooks by Zapier, Formatter and the Google Sheets actions are standard Zapier steps; the only text you type is the data table in the POST action.
Can the scraper run on my own machine instead of the cloud?
Yes. For a page that needs your own browser's login, see trigger a desktop run from Zapier.
Can I trigger a web scraper from Make or n8n instead of Zapier?
Yes. See trigger a web scraper from Make and trigger a web scraper from n8n. The plain request is in trigger a web scraper from a webhook.
Where do I get help?
Get in touch with support, or ask in the axiom.ai community.
Related guides
Trigger a web scraper from Make and get the data back
- Take the competitor URLs Make sends each morning
- Open each one and scrape the name and price
- Post the whole list of prices to my Make webhook
Trigger a web scraper from n8n and get the data back
- Take the URL someone submitted in my n8n form
- Open it and scrape the title and price
- Post what was scraped to my n8n webhook
Send scraped data to Zapier Tables and run an automation from a table
- Go to the listing page I give you
- Scrape the title and price of every item
- Post what was scraped to my Zapier webhook