Speeding up slow browser bot execution
Unfortunately, bots with faster execution can lead to more errors.
Axiom prioritises reducing errors - this means bots can sometimes be slow!
Here's some tips to speed up your bots:
# Set a maximum number of results
If you know you need to scrape 1 result per page (or 5, or 10 etc.) set 'Max results' to the number of results you need to scrape.
This means axiom will will stop once it has scraped 1 result, then move to the next iteration of the loop.
# Use the Keyboard
Using Keyboard shortcuts and 'Tabbing' between inputs can be faster than selecting 'Enter Text' to enter data.
# Jump ahead with conditional logic
This may only apply to some scenarios.
Let's look at the example of entering data into multi-step form with 5 steps.
If at step 2, it is evident steps 3 and 4 are not required, you can jump ahead to Step 5 using conditional logic.
You could also end the automation early. See:
# Reduce the wait time on scrapers
If the scraper can find no more content matching your selectors, and has found fewer results than the specified max results setting, it will attempt to retry the scrape to make sure it hasn't missed anything.
Each retry takes a few seconds to complete. Retries are not attempted if the scraper has found 100% of the requested results, which is why setting a lower Max Results often results in good speed improvements.
You can control this behaviour using the "Missing result wait" optional parameter in the "Get data from a webpage" step. Setting a lower number here will reduce the number of retries.
Only use this if you are certain that the page contents are fast loading - as you reduce this number, the scraper becomes more likely to miss results.
# Disable page change monitoring
By default Axiom waits for a page to finish making changes. This is to make sure that page contents which load in dynamically have fully loaded.
Sometimes, this can slow down an automation. This is most common when there are animated elements, and Axiom must take a while to determine whether the animation is related to page loads or not.
In these cases, you can turn off the page monitoring to skip it. Note you may need to add back manual waits in these cases, in case Axiom starts jumping ahead.
You can turn this on with the following steps:
- Clicking the drop-down menu at the top right of the axiom builder
- Selecting "Settings"
- Clicking the "Disable page monitoring" tab
- Tick the checkbox for "Disable page change monitoring"