How to monitor a web page and trigger webhook

Monitor a website for new products then send a notification via a webhook, this bot is a simple and effective way to achieve your goal. You will need to use this bot on a listing or search page.

The bot scrapes a list of content, such as titles or prices comparing them against a previous scrape to monitor for changes.

This template is simple and adaptable to different websites, from products to stock listings. A Google Sheet is required to store data for comparison.

To use this template, you will first need to scrape a list of data into your sheet to compare against. We suggest you adpat this template.

Advantages to using this web monitoring design pattern

  1. Adaptable to different types of website
  2. Monitoring can be scheduled at regular intervals

Install the Monitor bot template

On this page click 'Install template'. If you are a new user you will be required to create an axiom.ai account before you can edit the template.

Prepare a Google Sheet

A Google sheet is required with a list of data to compare against.

We suggest you adpat this template.

Get started open the bot for editing

  • 1.0 Get data from bot's current page
    • URL: Insert the link of the page you wish to monitor.
    • Select: Point and click to select the content to monitor.
    • Find pager (optional): Select the “Next” button if the page has pagination. If the page scrolls, leave this blank.
    • Max Results: For testing, set to 10–20.
  • 2.0 Read data from a Google Sheet
    • Spreadsheet: Search for and select the Google Sheet you created.
    • Sheet name: Choose the sheet containing the comparison data.
    • First cell: Leave as default.
    • Last cell: Leave as default.
    • A preview of the data should now appear.
  • 3.0 Remove results which do not contain certain words
    • Data: Click Insert data, select scrape-data.
    • Words (comma separated): Click Insert data, select google-sheet-data.
    • Word matching mode: Set to "Any".
    • Match on word boundary: Leave unticked.
    • Columns (optional): Leave default.
  • 4.0 If condition: Continue only if new content is found
    • Data to check: Click Insert data, select [matching-word-data].
    • Words to check: Leave as (a,b,c,...,z,0–9).
    • Word matching mode: "Any".
    • Match on word boundary: Leave unticked.
    • Fail if condition not met: Tick this box.
    • Reverse condition: Leave unticked.
  • 5.0 Write data to a Google Sheet
    • Spreadsheet: Search for and select the Google Sheet you created.
    • Sheet name: Choose the sheet you’re appending to.
    • DATA: Select [word-matching-data].
    • Write options: Set to "Add to existing data".
  • 6.0 Trigger a webhook
    • Endpoint: Insert the webhook endpoint.
    • Payload: Enter your payload in JSON format.

Webhook trigger example:

{
  "name": "test webhook",
  "type": 1,
  "channel_id": "199737254929760256",
  "token": "3d89bb7572e0fb30d8128367b3b1b44fecd1726de135cbe28a41f8b2f777c372ba2939e72279b94526ff5d1bd4358d65cf11",
  "avatar": null,
  "guild_id": "199737254929760256",
  "id": "223704706495545344",
  "application_id": null,
  "user": {
    "username": "test",
    "discriminator": "7479",
    "id": "190320984123768832",
    "avatar": "b004ec1740a63ca06ae2e14c5cee11f3",
    "public_flags": 131328
  }

Test run

We always recommend doing a test run - You should test the filter and conditional. Do the first scrape, then delete some rows to test the full bot. You can also test if the webhook is triggered.

Troubleshooting the monitoring Axiom

As mentioned at the top of the page, you will need to scrape data to compare against before running this bot.

  • All results write to the Google Sheet, check the 'Write' Step is set to 'word-matching-data'.
  • No message sent - Check your JSON
On this page