Scrape data from a list of links

Learn how to build an automation that allows you to loop through a list of links from a spreadsheet using our Google Sheets integration and then extract data from each page. Get started by adding steps from the [Builder](/docs/no-code-tool/the-builder/builder and selecting the links using the Selector Tool. ::

Tip: We recommend using websites that have an identical structure to ensure that the Selector Tool does not throw any errors, for example, a list of LinkedIn profiles.

To get started quickly, see our template.

Building the automation

There are multiple steps used within the automation:

Setup

Prepare your Google Sheets spreadsheet. Add a single link per row.

Col A
https://example.com
https://example.com
https://example.com

Create a new axiom.ai automation by opening the extension and clicking "+ New automation". Use the Step Finder to add new steps to your automation.

Start from blank

  • 1.0 Read data from a Google Sheet
    • Spreadsheet: Select the spreadsheet containing your list of links.
    • Sheet name: (Optional) Select the sheet within the spreadsheet.
    • First cell: (Optional) Start cell, e.g. "A1".
    • Last cell: (Optional) End cell, e.g. "A21".
  • 2.0 Loop through data
    • Data: Click Insert data, select google-sheet-data.
    • 2.1 Go to page
      • Enter URL: Click Insert data, select the column in google-sheet-data with the links.
      • Tip: Click "Clear all" to ensure only one column is selected.
    • 2.2 Get data from bot's current page
      • Select: Use the selector tool to select the data to scrape from the page.
    • 2.3 Write data to a Google Sheet
      • Spreadsheet: Select the spreadsheet to write to.
      • Sheet name: (Optional) Select the sheet.
      • DATA: Select scraped-data.
      • Write options: Choose "Add to existing data" to append, or "Clear data before writing" to overwrite.
    • 2.4 Delete rows from a Google Sheet (optional)
      • Spreadsheet: Select the spreadsheet.
      • Sheet name: (Optional) Select the sheet.
      • First row to delete: Set to 1.
      • Last row to delete: Set to 1.

Wrapping up

This design pattern is best suited to web pages that share an identical structure, an ecommerce product listing or social media profile page, for example. Combine with the How to extract links and write to a Google Sheet guide to quickly get started.

On this page