Scrape links from a website into a Google Sheet
How to scrape links from a website into a Google Sheet: install the Chrome extension, click Build from description, tell the agent what you want, and follow its prompts. A working bot is built and tested in minutes.
Here is how the automation works. To scrape links from a website, point the Get a list of links to pages from a URL step at the page, select one link, and write the list to a Google Sheet with Write data to a Google Sheet. It works on listing pages, paginated results, scrolling pages and feeds.
Build the link scraper from a description
Describe it and click the button: the agent builds and tests the link scraper in your Chrome extension.
- Go to the listing page I give you
- Scrape every product link on it
- Do every page of results
- Add the links to my Google Sheet
You need a Google Sheet to write to, with a named tab such as Links. Type sheet.new in Chrome to make one.
The video shows the agent building the link scraper from the lines above, in a few minutes.
Build the link scraper step by step
The steps the agent builds, as the Builder shows them: each step by its name, with the fields to set under it. Add them yourself in this order in the step-by-step builder, or open the agent's result there to adjust it.
- 1.0
Get a list of links to pages from a URLURL: the listing pageSelect: one product linkFind pager: the Next buttonMax results: 10 for a test run
- 2.0
Write data to a Google SheetSpreadsheet: your sheetSheet name: LinksDATA:[scraped-link-data]Write options: Add to existing data
Common questions about scraping links from a website
Why does the link scraper find the wrong links, or none?
Reselect them. In Get a list of links to pages from a URL, pick a different example link. If the page's links have no stable pattern, paste a custom CSS selector instead.
Why is the link scraper slow?
Turn off retries and lower Max results. Open Configure scraper on the step and set No. of retries to 0. A Max results far above the number of links on the page also slows it down, because axiom.ai keeps looking for links that are not there.
Why are no links written to the Google Sheet?
The step is pointed at the wrong data or tab. In Write data to a Google Sheet, check that DATA is [scraped-link-data] and that the tab in Sheet name exists.
How do I stop each run overwriting the links in the Google Sheet?
Set Write options on the sheet step to Add to existing data.
Does the link scraper work on paginated results?
Yes. Point Find pager on Get a list of links to pages from a URL at the page's Next button and it turns the pages until it has Max results links.
Can I scrape links from a website without code?
Yes. The whole automation is two no-code steps in the Chrome extension: one that collects the links, one that writes them to the sheet.
Can I scrape the links on a schedule?
Yes. Save the automation and set a schedule if the page changes often; with Add to existing data each run adds the new links below the last.
How do I scrape the pages behind the links?
Build a second automation that reads this sheet and loops through the links: scrape data from every page in a list. Keeping the two apart makes each one easier to rerun and fix.
Can I save the links to a CSV file instead of a Google Sheet?
Yes. Swap the sheet step for Export to CSV: scrape links from a website into a CSV file walks through it.
Where do I get help?
Get in touch with support, or ask in the axiom.ai community.
Related guides
Scrape links from a website into a CSV file
- Go to the listing page I give you
- Scrape every product link on it
- Do every page of results
- Save the links as a CSV file
Scrape Instagram users from a hashtag into a Google Sheet
- Open the Instagram hashtag page in my Google Sheet
- Collect the links to every post on it
- Open each post and get the poster's profile link
- Write the post and profile links to my sheet
Scrape Instagram post links for every search term in a Google Sheet
- Read the search terms from my Terms tab
- Search Instagram for each term
- Collect every post link on the results
- Write them to my Posts tab and delete the term