Pricing
Templates
Support
AI
Roadmap
Install Chrome Extension

How to scrape listing pages with pager using a keypress scroll

Advanced • 8 minutes
Install Chrome Extension

This scraper is a workaround for scraping listing pages that have a pager. Use this template if the page needs to scroll faster or results are missing/inconsistent.

Please do remember the video! if you get stuck.

To install this web scraper Axiom click 'Install template'. If you are a new user you will be required to create an Axiom.ai account before you can edit the template.

# Configure a blank Google Sheet to scrape the data into

  1. Create a new Google Sheet. You can do this in your Chrome browser by entering the shortcut 'sheet.new', presuming you already have an account.
  2. Name your sheet something like 'Keypress Scraper’.
  3. Set up a tab called 'Data'.

# Configure the first 'Interact with a page's interface' step

This interact step loads the site.

  1. Enter URL - add the URl of the lisiting page.

# Set up the second 'Interact with a page's interface' step

This interact step puts the cursor into the scroller setting up the scroll. Its also the step we jump back to.

Axiom.ai enter (skip) to keep page loaded
  1. Enter URL - add (skip) this stops the page from reloading.
  2. Click Elemnet - Do not select a button or click event. Select the are which scrolls to place the cursor in the right area for the next step (See video to make sense).
  3. Press Key(s): Down - Click record and press the down arrow several times.

Do watch the video if this is not clear.

# Configure the third 'Interact with a page's interface' step

This interact step does the scraping.

Axiom.ai scrape Sales Navigator
  1. Enter URL - add (skip) this stops the page from reloading.
  2. SELECT - Select the data you wish to scrape, Its as easy as point and click, this video shows you how.
  3. Find pager (optional) - Do nothing here!!!
  4. Max Results - Set this match the amount of sectors used in step 2.

# Set up the fouth 'Interact with a page's interface' step

This interact steps clicks the pager.

  1. Select - select the next button.
Axiom.ai select pager for scraping

# Jump to another step

  1. Jump to step - Set to number 2 to loop the bot.
  2. Set to the amount of pagers use wish to click.

# Configure the 'Write Data to a Google Sheet' step

  1. Spreadsheet - In the field called 'Spreadsheet', you can search for the Google Sheet you created. Once found, click to select.
  2. Sheet name - Choose the tab you created.
  3. DATA - Select the Scrape data.
  4. Clear data before writing | Add to existing data - Set this option to 'Add to existing data'.

# Test run

We always recommed doing a test run - click run then check the output from the scraper. In this case in the sheet.

Try the following if your bot is not working. We would also recommend you watch the video to troubleshoot.

  • The links come out as text - Set the selector type to link not text.
  • Selectors fail and the bot stops, try a new selection.
  • The pager is not working - Re-select the next button or try custom selector.
  • No output in the sheet - In 'Write Data to a Google Sheet' check data step is connected.
  • Content does not paginate - Try a custom selector on the next button or the 'Use text element' if the button says 'next' for example.
  • Bot does not loop - Check the Jump step that controls the loop.

# More web scraping templates