How to extract links and write to a Google Sheet

Learn how to build an automation that allows you to scrape links from a webpage and store them using our Google Sheets integration. Get started by adding steps from the Builder and selecting the links using the Selector Tool. Follow the design pattern below to get started:

Design pattern: Extract links web scraper

Want to get started fast? Try the pre-existing template.

# Building the automation

There are two steps used within the automation:

# Setup

Create a blank Google Sheet - this is where we will write the scrapped links. Keep the name of this sheet in mind for later use.

Create a new Axiom.ai automation by opening the extension and clicking "+ New automation". Use the Step Finder to add new steps to your automation.

# Set up the scraper

The "Get a list of links to pages from a URL" step will perform the scraping of links from the page. Configure as follows:

  1. URL: the URL of the page containing the links.
  2. Select: click "Select" to open the Selector Tool. Select the links that you wish to scrap. Learn more about the Selector Tool.
  3. Find pager: (optional) select the "next" button on the page, if applicable.
  4. Max results: the maximum number of items the step will scrape.

This will output a data token called scraped-link-data that will be used later in the automation.

Now that we have the scraped-link-data returned from the "Get a list of links to pages from a URL" step, we can use the "Write data to a Google Sheet" step to write this to your Google Sheet that was created above. Configure as follows:

  1. Spreadsheet: select the spreadsheet to write to.
  2. Sheet name: (optional) select the sheet to write to.
  3. DATA: select the scraped-link-data token.
  4. Write options: select "Add to existing data" to append to the end of your sheet, select "Clear data before writing" to clear the sheet and write the new data.

# Wrapping up

Easily create an automation that extracts links from a website and stores them in a Google Sheet using our Google Sheet integration. There are several use cases for an automation such as this:

  • Getting a list of links to posts on a social media site.
  • Getting a list of links to pages within a list of pages from a webpage.
  • Getting a list of links of images in a gallery.

This can be combined with the Loop through data step to then loop through this list of links and visit each of these links to perform actions, helpful when working through a list of items.

Tip

💡 To test your automation, set the max results option. Run your automation and confirm the output is as expected before performing a larger run.

Prev: Web scraping guides
Next: Scrape data from a list of links