Monitor a website for new products, this bot is a simple and effective way to achieve your goal. You will need to use this bot on a listing or search page.
The bot scrapes a list of content, such as titles or prices comparing them against a previous scrape to monitor for changes.
This template is simple and adaptable to different websites, from products to stock listings. A Google Sheet is required to store data for comparison.
To use this template, you will first need to scrape a list of data into your sheet to compare against. We suggest you adpat this template.
# Advantages to using this web monitoring design pattern
- Adaptable to different types of website
- Monitoring can be scheduled at regular intervals
# Install the Monitor bot template
On this page click 'Install template'. If you are a new user you will be required to create an Axiom.ai account before you can edit the template.
# Prepare a Google Sheet
A Google sheet is required with a list of data to compare against.
# Scrape a list of content to compare against
We suggest you adpat this template.
# Get started open the bot for editing
On the Axiom dashboard, hover over the text 'Monitor web page and send email' and click.
# Set up the 'Get data from a webpage' step
This step scrapes the data you wish to compare.
- URL - Insert the link of the page you wish to monitor.
- SELECT - Select the content you wish to monitor, Its as easy as point and click, this video shows you how.
- Find pager (optional) - Select the next button if the page has a pager. If it scrolls then do nothing.
- Max Results - When testing set to 10-20 results max.
# Set up the 'Read data from Google Sheet' step
This step stores the data for comparison on each run.
- Spreadsheet - In the field called 'Spreadsheet', you can search for the Google Sheet you created. Once found, click to select.
- Sheet name - Choose the sheet that contains the comparision data.
- First cell - No changed required.
- Last cell - No changed required.
- You should now see a preview of the data.
# Remove results which do not contain certain words
This steps filters out the old content leaving the new.
- Data - Click 'Insert data and select 'scape-data'.
- Words (comma separated) - lick 'Insert data and select 'google-sheet-data'.
- Word matching mode - leave set to 'Any'.
- Match on word boundary - leave set to unticked.
- Columns (optional) - leave on defualt.
# Continue only if a condition is met
The conditional step stops the bot from sending a message if no new content is not found.
- Data to check - Click insert data and then choose '[matching-word-data]'
- Words to check for (comma separated) - Leave set to (a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,0,1,2,3,4,5,6,7,8,9)
- Word matching mode - Leave set on 'Any'
- Match on word boundary - Leave unticked.
- Fail if condition not met - Tick this box.
- Reverse condition - Leave unticked.
# Set up the 'Write Data to a Google Sheet' step
Writes new data to the sheet.
- Spreadsheet - In the field called 'Spreadsheet', you can search for the Google Sheet you created. Once found, click to select.
- Sheet name - Choose the tab you created with the existing content to check against.
- DATA - Select the '[word-matching-data]'.
- Clear data before writing | Add to existing data - Set this option to 'Add to existing data'.
# Send an email
- Email addresses - Enter the email addresses one per line.
- Email subject - Add subject to message.
- Email body - Add message you can click 'insert data' and choose [matching-word-data] to sed the new content.
# Test run
We always recommed doing a test run - click run then check the output from the scraper. In this case in the sheet.
# Troubleshooting the monitoring Axiom
As mentioned at the top of the page, you will need to scrape data to compare against before running this bot.
- All results write to the Google Sheet, check the 'Write' Step is set to '[word-matching-data]'.
- No message sent - perhaps no change has been detected.