[{"data":1,"prerenderedAt":12176},["ShallowReactive",2],{"author-karl-jones":3,"content-doc-\u002Fauthors\u002Fkarl-jones-inline":3,"blog-list-{\"where\":[{\"author\":\"Karl Jones\"}],\"sort\":[{\"date\":-1}]}":29},{"id":4,"title":5,"bio":6,"body":7,"date":18,"description":13,"extension":19,"image":20,"linkedin":21,"meta":22,"name":23,"navigation":24,"order":18,"path":25,"seo":26,"stem":27,"__hash__":28},"authors\u002Fauthors\u002Fkarl-jones.md","Technical writer","Karl is a technical writer with axiom.ai with a computer science background and 10+ years of customer support experience. In his spare time he enjoys continuing his technical education, reading, gaming, and working on development side projects.",{"type":8,"value":9,"toc":14},"minimark",[10],[11,12,13],"p",{},".",{"title":15,"searchDepth":16,"depth":16,"links":17},"",3,[],null,"md","\u002Fauthors\u002Fkarl-jones.jpeg","https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fkjoones\u002F",{},"Karl Jones",true,"\u002Fauthors\u002Fkarl-jones",{"title":5,"description":13},"authors\u002Fkarl-jones","l5DQMsmGpWJWTJV_72QRoKmeXxNdiXuVfJYx7194UFI",[30,198,365,2698,2848,3337,3825,4053,5161,5630,6224,6430,6603,7380,7530,8088,8428,9160,9519,9743,10076,10328,10431,10982,11139,11382,11799],{"id":31,"title":32,"author":23,"body":33,"date":179,"description":180,"draft":181,"extension":19,"meta":182,"navigation":24,"path":194,"seo":195,"stem":196,"__hash__":197},"blog\u002Fblog\u002Fpuppeteer-web-scraper.md","Can a no coder build a web scraper with Puppeteer?",{"type":8,"value":34,"toc":170},[35,38,42,47,56,59,66,70,77,80,86,90,97,105,111,115,122,133,137,143,147,154,157,163,167],[11,36,37],{},"While axiom.ai automations can be triggered manually, there are methods of automatically triggering these, or even just manually triggering them from a third-party service. This allows for your automations to be included in a larger workflow, or include other services in your current workflow. In this article, we are going to discuss triggering your automations from third-party services such as IFTTT, Make, Zapier, Power Automate and Supabase - we have additional guides for all of these services if you wish to take a deeper dive, check out the sections below for more details.",[39,40],"hero-media",{"alt":41},"5 Methods of Triggering an Automation via Third-party Services",[43,44,46],"h2",{"id":45},"ifttt","IFTTT",[11,48,49,55],{},[50,51,46],"a",{"href":52,"rel":53},"https:\u002F\u002Fifttt.com",[54],"nofollow"," (If This Then That) is a web-based service that automates tasks by connecting apps, devices, and services through simple conditional statements called \"applets.\" These applets can include triggers for your automations, including sending information over that can be used later in your automations. Your applets can also be triggered directly from your automations. You may find that you already have an established library of IFTTT applets that could benefit from integrating with axiom.ai.",[11,57,58],{},"Personally, I found this helpful with managing my homes energy usage. As a member of Octopus Energy, my energy prices change throughout the day depending on how much \"clean\" energy is being produced on the grid here in the UK. I make use of the \"Agile Octopus\" integration within IFTTT to trigger an axiom.ai automation anytime the price of energy drops below 10p\u002FkWh so I can be notified and turn on all of my power hungry appliances!",[11,60,61,62],{},"Guide: ",[50,63,65],{"href":64},"\u002Fguides\u002Fifttt","How to trigger axiom.ai automations using IFTTT",[43,67,69],{"id":68},"make","Make",[11,71,72,76],{},[50,73,69],{"href":74,"rel":75},"https:\u002F\u002Fmake.com",[54]," (formerly Integromat) is a visual platform for building and automating workflows by connecting apps, services, and devices through customizable integrations and logic-based scenarios. These workflows offer powerful options when it comes to allowing logic to be introduced to your workflows, similar to the logic steps that are offered within axiom.ai.",[11,78,79],{},"We have seen some pretty inventive automations created to make use of Make workflows come through our Support team, including the automation of Telegram messages or even the automation of their Etsy store.",[11,81,61,82],{},[50,83,85],{"href":84},"\u002Fguides\u002Fpost-data-to-make","How to trigger and post data to Make with an axiom.ai automation",[43,87,89],{"id":88},"power-automate","Power Automate",[11,91,92,96],{},[50,93,89],{"href":94,"rel":95},"https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fpower-platform\u002Fproducts\u002Fpower-automate",[54]," is a Microsoft platform that enables users to create automated workflows between apps and services to streamline tasks, integrate systems, and improve productivity. It can be useful for a wide variety of tasks, from simple social media automation to reporting on system stats and system monitoring. Something to note when it comes to Power Automate is that there are two versions: the desktop version that can be used to access system resources, and the online version that is more of an automation platform. It would be best to review the platforms to determine which one suits your needs best. The online version is cross platform, which has it's benefits, but has more limited functionality.",[11,98,99,100,104],{},"An interesting use case for this may be to report the status of a system that has been designated a server - for example, if you'd like to trigger an automation to run when a server hits a certain memory threshold, you could do this by setting up a Power Automate workflow on that server that then triggers your axiom.ai automation. You could then have your automation send an email, or send a message along to Slack - check out our ",[50,101,103],{"href":102},"\u002Fguides\u002Fslack","Slack"," guide for details on how to integrate Slack.",[11,106,61,107],{},[50,108,110],{"href":109},"\u002Fguides\u002Fpower-automate","How to trigger axiom.ai automations using Power Automate",[43,112,114],{"id":113},"zapier","Zapier",[11,116,117,121],{},[50,118,114],{"href":119,"rel":120},"https:\u002F\u002Fzapier.com",[54]," is an automation platform that connects apps and services, enabling users to create workflows called \"Zaps\" that perform tasks automatically based on triggers and actions. Zapier has a massive library of over 1,000 integrations that can be integrated into your Zaps - and guess what one of those integrations is? You guessed right (I hope) - axiom.ai! You can search for axiom.ai right in your Zapier library and get started pretty quickly.",[11,123,124,125,130,131,104],{},"Internally, we use this integration quite a bit. To help manage our ",[50,126,129],{"href":127,"rel":128},"https:\u002F\u002Fwww.reddit.com\u002Fr\u002Faxiom_ai",[54],"Reddit community"," we have a Zap that detects when new Reddit posts are made within our community. When a new post is made in the community it will trigger the Zap, that will then trigger the axiom.ai automation which sends the data onto a Slack workflow which drops a message in our \"#reddit-new-posts\" channel, ready for our Support team to review! Check out our ",[50,132,103],{"href":102},[134,135],"img",{"src":136},"\u002Fblog\u002F5-methods-of-triggering-via-third-party-zapier-flowchart.png",[11,138,61,139],{},[50,140,142],{"href":141},"\u002Fguides\u002Fzapier","How to trigger Zapier Zaps using axiom.ai",[43,144,146],{"id":145},"supabase","Supabase",[11,148,149,153],{},[50,150,146],{"href":151,"rel":152},"https:\u002F\u002Fsupabase.com",[54]," is an open-source backend-as-a-service platform that provides developers with tools like a Postgres database, authentication, and real-time APIs to build scalable applications. As a backend-as-a-service platform it helps you store data at scale - combined with your axiom.ai automations, it can really help you store the data that you are working with. For example, if you wish to store large amounts of data that you have scraped, Supabase can be useful for storing that data. This can be a great alternative to Google Sheets as it provides a better platform to manage data at scale, plus your data can be exported to be used in your favourite spreadsheet software, if required.",[11,155,156],{},"As a Firebase alternative, we've seen instances where this data has been fed into a Supabase database to then be used to power other website through their Postgres database offerings. As a developer myself with plenty of experience with Firebase, Supabase offers a great alternative that when combined with your axiom.ai automations could offer quite a lot of opportunities.",[11,158,61,159],{},[50,160,162],{"href":161},"\u002Fguides\u002Fsupabase","How to automate Supabase with axiom.ai",[43,164,166],{"id":165},"wrapping-up","Wrapping up",[11,168,169],{},"All of the services above offer various advantages when used with axiom.ai to power up your automations. Even if you are not passing data to your axiom.ai automations, the simple act of triggering them from a third-party service can give you the ability add additional features to your workflows - from something as simple as notifications, to triggering completely separate workflows using logic steps within any of the apps above.",{"title":15,"searchDepth":16,"depth":16,"links":171},[172,174,175,176,177,178],{"id":45,"depth":173,"text":46},2,{"id":68,"depth":173,"text":69},{"id":88,"depth":173,"text":89},{"id":113,"depth":173,"text":114},{"id":145,"depth":173,"text":146},{"id":165,"depth":173,"text":166},"2025-09-18","Learn five methods of triggering an axiom.ai automation using a third-party service",false,{"read":183,"type":184,"tool":185,"category":187,"tags":189,"location":191,"featuredimg":192,"landingimg":193,"summary":180,"video":18,"metaTitle":41},"4 min read","no-code",[186],"no-code runner",[188],"Web scraping",[46,68,190,113,145],"power automate","London","\u002Fblog\u002F5-triggers.webp","\u002Fblog\u002F5-triggers-sq.webp","\u002Fblog\u002Fpuppeteer-web-scraper",{"title":32,"description":180},"blog\u002Fpuppeteer-web-scraper","ywLTmlrHDX_8v7rAdKB2m0TtTX89jWN50Yofb5LSLV0",{"id":199,"title":200,"author":23,"body":201,"date":349,"description":350,"draft":181,"extension":19,"meta":351,"navigation":24,"path":361,"seo":362,"stem":363,"__hash__":364},"blog\u002Fblog\u002Fmastering-the-multi-selector-tool.md","Mastering the Multi-selector Tool",{"type":8,"value":202,"toc":340},[203,218,222,225,228,232,235,238,241,245,248,267,270,274,277,291,294,298,301,305,308,311,323,332,334,337],[11,204,205,206,211,212,217],{},"The ",[50,207,210],{"href":208,"rel":209},"https:\u002F\u002Faxiom.ai\u002F",[54],"axiom.ai"," Selector tool allows you to select elements from a page to make use of within your automation. For example, when using the ",[50,213,216],{"href":214,"rel":215},"https:\u002F\u002Faxiom.ai\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fenter-text",[54],"Enter text"," step this will allow you to select the text field that you want to enter text into. There are two versions of this tool depending on which step you are using it in. We are primarily discussing the Multi-Selector tool within this article, but it’s still important to know about the Single-Selector tool.",[43,219,221],{"id":220},"single-selector-tool","Single-Selector tool",[11,223,224],{},"The Single-Selector tool will allow you to select a single target within the step that you have initiated the tool from. For example, this tool will open when selecting your target for the “Enter text” step, or the “Click element” step - steps where only a single target is used.",[11,226,227],{},"This tool will allow you to also use advanced features such as custom selectors, or using the text of an element as a selector. Setting custom selectors allows you to target more complex elements that the standard selector tool may not recognise.",[43,229,231],{"id":230},"multi-selector-tool","Multi-Selector tool",[11,233,234],{},"The Multi-Selector tool operates similar to the Single-Selector tool, but has a lot more functionality. This tool allows you to define multiple patterns of selectors to allow the step to identify a wider range of data from the page and multiple different sets of data. We’re going to dive into the features of this tool, and then we are going to give you some tips on really mastering the tool. This tool mostly appears within steps that are designed to scrape data from a webpage.",[11,236,237],{},"This tool will appear when you are using steps such as the “Get data from bot’s current page” step where you may wish to scrape multiple pieces of data from a webpage.",[11,239,240],{},"There are two key concepts to understand about the tool before making use of it: data types and columns. These two concepts make up the structure of the tool and understanding them ensures that you have the ability to select the elements you need to extract the data that fits your requirements.",[43,242,244],{"id":243},"data-types","Data types",[11,246,247],{},"Selecting the correct data type is one of the first things that you will need to do when creating a new column of data. These data types will determine what data will be targeted by the automation. The data types available within the tool are:",[249,250,251,255,258,261,264],"ul",{},[252,253,254],"li",{},"Text, the text of the element.",[252,256,257],{},"HTML, the HTML of the element.",[252,259,260],{},"Link, the URL behind the element.",[252,262,263],{},"Image, the image within the element.",[252,265,266],{},"Custom selector, a custom CSS selector.",[11,268,269],{},"Most data types are pretty self explanatory, the custom selector option being the outlier here - this allows you to write custom CSS selectors for more complex HTML elements that you may wish to use within your automation.",[43,271,273],{"id":272},"columns","Columns",[11,275,276],{},"One of the best features of the Multi-Selector tool is to be able to select multiple pieces of data. Within the tool itself, you will see the “Add column +” button once you have a data type set for the column before it. This allows you to collect multiple columns of data and output it for use within future steps of your automation.",[11,278,279,280,285,286,13],{},"For example, you would use separate columns if you were scraping a news website and wanted to grab the headline, the sub-heading and the news story itself. When output, this would include the information in separate columns ready to use or write to another service such as ",[50,281,284],{"href":282,"rel":283},"https:\u002F\u002Faxiom.ai\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002Fgoogle-sheets",[54],"Google Sheets"," or ",[50,287,290],{"href":288,"rel":289},"https:\u002F\u002Faxiom.ai\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002Fexcel",[54],"Microsoft Excel",[11,292,293],{},"You can mix and match your data types across different columns - Column A may be set to extract text from a page, where Column B may be set to extract a link from the page. This can be useful for extracting lists of links from a page.",[43,295,297],{"id":296},"selecting-elements","Selecting elements",[11,299,300],{},"Understanding how to select elements with the Multi-Selector tool is very important - you can think of them as patterns. When you have the column selected, click on the element within the page that you want to select. Select another element that matches the first and the tool will automatically match any other elements on the page that match the same pattern.",[43,302,304],{"id":303},"using-the-tool","Using the tool",[11,306,307],{},"To get started with the tool, it’s important to understand the data that you are looking to extract. Let’s use a Google Search page as an example.",[11,309,310],{},"There are three pieces of information that we want to get from each result that is on this page: the result title, the result description and the URL to the result. Consider each of these pieces of information as a column of data within the Multi-Selector tool. We can create three columns that can be set up as follows:",[312,313,314,317,320],"ol",{},[252,315,316],{},"Column A: Set to “text”, select the result titles",[252,318,319],{},"Column B: Set to “text”, select the result description",[252,321,322],{},"Column C: Set to “link”, select the result titles - this title contains the link to the result page",[11,324,325,326,331],{},"Once the automation is run this will output a list of results containing the data items that we are looking for - you can write these to a Google Sheet using the ",[50,327,330],{"href":328,"rel":329},"https:\u002F\u002Faxiom.ai\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fwrite-data-to-a-google-sheet-step",[54],"Write data to a Google Sheet"," step.",[43,333,166],{"id":165},[11,335,336],{},"Understanding the basic features of the Multi-Selector tool allows you to take advantage of the tool in order to be sure that the elements that you are targeting with your automation are the correct targets.",[11,338,339],{},"By setting different data types within the columns that you have created, this allows you to target different data from elements depending on your needs. Setting the column to “text” will allow you to grab the text from the element, but setting this to “link” will allow you to get a URL from the same element.",{"title":15,"searchDepth":16,"depth":16,"links":341},[342,343,344,345,346,347,348],{"id":220,"depth":173,"text":221},{"id":230,"depth":173,"text":231},{"id":243,"depth":173,"text":244},{"id":272,"depth":173,"text":273},{"id":296,"depth":173,"text":297},{"id":303,"depth":173,"text":304},{"id":165,"depth":173,"text":166},"2025-09-10","The axiom.ai Selector tool allows you to select elements from a page to make use of within your automation. For example, when using the Enter text step this will allow you to select the text field that you want to enter text into. There are two versions of this tool depending on which step you are using it in. We are primarily discussing the Multi-Selector tool within this article, but it’s still important to know about the Single-Selector tool.",{"read":352,"type":184,"tool":353,"category":355,"tags":356,"location":191,"featuredimg":359,"landingimg":360},"5 min read",[354],"No-code tool",[188],[357,358],"css selectors","selector tool","\u002Fblog\u002Fcustom-selectors-post.webp","\u002Fblog\u002Fgroup-selectors-box.webp","\u002Fblog\u002Fmastering-the-multi-selector-tool",{"title":200,"description":350},"blog\u002Fmastering-the-multi-selector-tool","sqWjwBaI4bVZRL-SY0Nor8UOhIG_sL_-z65QrtfnE28",{"id":366,"title":367,"author":23,"body":368,"date":2679,"description":2680,"draft":181,"extension":19,"meta":2681,"navigation":24,"path":2694,"seo":2695,"stem":2696,"__hash__":2697},"blog\u002Fblog\u002Fwriting-tests-with-llms.md","Writing Tests With LLMs",{"type":8,"value":369,"toc":2669},[370,373,377,380,769,773,776,787,790,796,799,803,806,814,817,822,833,1128,1132,1138,1499,1510,1512,1515,1518,1522,1525,2665],[11,371,372],{},"Large-language models (LLMs) are fantastic at writing code, this means that you can take full advantage of them when you are building out your projects. LLMs such as ChatGPT, Gemini, Claude or Le Chat are great at helping you write code - not to mention Cursor or Github Copilot that are specifically designed to help you with code. Whether you are following a standard project timeline or a test-driven development framework, writing tests is essential to ensure that your scripts work as you expect them to.",[43,374,376],{"id":375},"writing-the-script","Writing the script",[11,378,379],{},"We're not going to be looking at writing scripts in this article, but we are going to introduce a script that we are going to use to ask an LLM to help us write tests for. We're going to stick with JavaScript for this article, and we are going to be using Gemini, but this will work for all LLMs. We're going to use a snippet that uses Puppeteer to fills a string into the Google search bar, this will connect to an external browser, go to Google.com, and then fill in the text before closing the browser session.",[381,382,386],"pre",{"className":383,"code":384,"language":385,"meta":15,"style":15},"language-js shiki shiki-themes github-light-default github-dark-default","import puppeteer from 'puppeteer-core'\n\nconst start = async () => {\n  console.log('start')\n  const browser = await puppeteer.connect({\n    browserWSEndpoint: '\u003CENDPOINT>',\n  })\n  console.log('browser connected')\n  try {\n    const page = await browser.newPage()\n    await page.goto('https:\u002F\u002Fgoogle.com\u002F')\n\n    \u002F\u002F Locate the search bar on the page\n    await page.locator('aria\u002FSearch').fill('Testing axiom.ai')\n\n    await new Promise((resolve) => {\n      setTimeout(() => {\n        resolve()\n      }, 5000)\n    })\n\n    await page.close()\n    await browser.close()\n    console.log('browser closed')\n  } catch (e) {\n    alert(e.message)\n    await browser.close()\n  }\n}\n\nstart()\n","js",[387,388,389,409,414,438,456,480,492,498,512,520,542,561,566,573,601,606,631,644,652,663,669,674,686,697,712,724,733,744,750,756,761],"code",{"__ignoreMap":15},[390,391,394,398,402,405],"span",{"class":392,"line":393},"line",1,[390,395,397],{"class":396},"sjeE4","import",[390,399,401],{"class":400},"s4rv2"," puppeteer ",[390,403,404],{"class":396},"from",[390,406,408],{"class":407},"sSVrQ"," 'puppeteer-core'\n",[390,410,411],{"class":392,"line":173},[390,412,413],{"emptyLinePlaceholder":24},"\n",[390,415,416,419,423,426,429,432,435],{"class":392,"line":16},[390,417,418],{"class":396},"const",[390,420,422],{"class":421},"sbjLL"," start",[390,424,425],{"class":396}," =",[390,427,428],{"class":396}," async",[390,430,431],{"class":400}," () ",[390,433,434],{"class":396},"=>",[390,436,437],{"class":400}," {\n",[390,439,441,444,447,450,453],{"class":392,"line":440},4,[390,442,443],{"class":400},"  console.",[390,445,446],{"class":421},"log",[390,448,449],{"class":400},"(",[390,451,452],{"class":407},"'start'",[390,454,455],{"class":400},")\n",[390,457,459,462,466,468,471,474,477],{"class":392,"line":458},5,[390,460,461],{"class":396},"  const",[390,463,465],{"class":464},"sHrmB"," browser",[390,467,425],{"class":396},[390,469,470],{"class":396}," await",[390,472,473],{"class":400}," puppeteer.",[390,475,476],{"class":421},"connect",[390,478,479],{"class":400},"({\n",[390,481,483,486,489],{"class":392,"line":482},6,[390,484,485],{"class":400},"    browserWSEndpoint: ",[390,487,488],{"class":407},"'\u003CENDPOINT>'",[390,490,491],{"class":400},",\n",[390,493,495],{"class":392,"line":494},7,[390,496,497],{"class":400},"  })\n",[390,499,501,503,505,507,510],{"class":392,"line":500},8,[390,502,443],{"class":400},[390,504,446],{"class":421},[390,506,449],{"class":400},[390,508,509],{"class":407},"'browser connected'",[390,511,455],{"class":400},[390,513,515,518],{"class":392,"line":514},9,[390,516,517],{"class":396},"  try",[390,519,437],{"class":400},[390,521,523,526,529,531,533,536,539],{"class":392,"line":522},10,[390,524,525],{"class":396},"    const",[390,527,528],{"class":464}," page",[390,530,425],{"class":396},[390,532,470],{"class":396},[390,534,535],{"class":400}," browser.",[390,537,538],{"class":421},"newPage",[390,540,541],{"class":400},"()\n",[390,543,545,548,551,554,556,559],{"class":392,"line":544},11,[390,546,547],{"class":396},"    await",[390,549,550],{"class":400}," page.",[390,552,553],{"class":421},"goto",[390,555,449],{"class":400},[390,557,558],{"class":407},"'https:\u002F\u002Fgoogle.com\u002F'",[390,560,455],{"class":400},[390,562,564],{"class":392,"line":563},12,[390,565,413],{"emptyLinePlaceholder":24},[390,567,569],{"class":392,"line":568},13,[390,570,572],{"class":571},"sU953","    \u002F\u002F Locate the search bar on the page\n",[390,574,576,578,580,583,585,588,591,594,596,599],{"class":392,"line":575},14,[390,577,547],{"class":396},[390,579,550],{"class":400},[390,581,582],{"class":421},"locator",[390,584,449],{"class":400},[390,586,587],{"class":407},"'aria\u002FSearch'",[390,589,590],{"class":400},").",[390,592,593],{"class":421},"fill",[390,595,449],{"class":400},[390,597,598],{"class":407},"'Testing axiom.ai'",[390,600,455],{"class":400},[390,602,604],{"class":392,"line":603},15,[390,605,413],{"emptyLinePlaceholder":24},[390,607,609,611,614,617,620,624,627,629],{"class":392,"line":608},16,[390,610,547],{"class":396},[390,612,613],{"class":396}," new",[390,615,616],{"class":464}," Promise",[390,618,619],{"class":400},"((",[390,621,623],{"class":622},"sTDnQ","resolve",[390,625,626],{"class":400},") ",[390,628,434],{"class":396},[390,630,437],{"class":400},[390,632,634,637,640,642],{"class":392,"line":633},17,[390,635,636],{"class":421},"      setTimeout",[390,638,639],{"class":400},"(() ",[390,641,434],{"class":396},[390,643,437],{"class":400},[390,645,647,650],{"class":392,"line":646},18,[390,648,649],{"class":421},"        resolve",[390,651,541],{"class":400},[390,653,655,658,661],{"class":392,"line":654},19,[390,656,657],{"class":400},"      }, ",[390,659,660],{"class":464},"5000",[390,662,455],{"class":400},[390,664,666],{"class":392,"line":665},20,[390,667,668],{"class":400},"    })\n",[390,670,672],{"class":392,"line":671},21,[390,673,413],{"emptyLinePlaceholder":24},[390,675,677,679,681,684],{"class":392,"line":676},22,[390,678,547],{"class":396},[390,680,550],{"class":400},[390,682,683],{"class":421},"close",[390,685,541],{"class":400},[390,687,689,691,693,695],{"class":392,"line":688},23,[390,690,547],{"class":396},[390,692,535],{"class":400},[390,694,683],{"class":421},[390,696,541],{"class":400},[390,698,700,703,705,707,710],{"class":392,"line":699},24,[390,701,702],{"class":400},"    console.",[390,704,446],{"class":421},[390,706,449],{"class":400},[390,708,709],{"class":407},"'browser closed'",[390,711,455],{"class":400},[390,713,715,718,721],{"class":392,"line":714},25,[390,716,717],{"class":400},"  } ",[390,719,720],{"class":396},"catch",[390,722,723],{"class":400}," (e) {\n",[390,725,727,730],{"class":392,"line":726},26,[390,728,729],{"class":421},"    alert",[390,731,732],{"class":400},"(e.message)\n",[390,734,736,738,740,742],{"class":392,"line":735},27,[390,737,547],{"class":396},[390,739,535],{"class":400},[390,741,683],{"class":421},[390,743,541],{"class":400},[390,745,747],{"class":392,"line":746},28,[390,748,749],{"class":400},"  }\n",[390,751,753],{"class":392,"line":752},29,[390,754,755],{"class":400},"}\n",[390,757,759],{"class":392,"line":758},30,[390,760,413],{"emptyLinePlaceholder":24},[390,762,764,767],{"class":392,"line":763},31,[390,765,766],{"class":421},"start",[390,768,541],{"class":400},[43,770,772],{"id":771},"prompting-the-llm","Prompting the LLM",[11,774,775],{},"There are a few things that the LLM will assume when you give it code to write tests from - for example, if you copy in JavaScript code, it's going to assume that you are going to want JavaScript output. However, being specific even where it may seem redundant is recommended to increase your chances of getting a right answer from the beginning. Recommended things to mention in your prompt:",[249,777,778,781,784],{},[252,779,780],{},"The programming language you are using",[252,782,783],{},"The testing framework that you want to use",[252,785,786],{},"Any area of the code that you want it to specifically concentrate on",[11,788,789],{},"We used the following prompt when prompting the LLM to generate the tests:",[791,792,793],"blockquote",{},[11,794,795],{},"Generate Jest unit tests in JavaScript using the following script:",[11,797,798],{},"Followed by the code that we included above.",[43,800,802],{"id":801},"the-result","The result",[11,804,805],{},"Gemini provided us with two tests that can be run to test the code, including the set up before and after the tests run. These tests include:",[249,807,808,811],{},[252,809,810],{},"should successfully navigate and fill the search bar",[252,812,813],{},"should handle connection errors gracefully",[11,815,816],{},"We're going to break down the code into smaller segments, but the full code can be found at the end of the article",[818,819,821],"h3",{"id":820},"the-setup","The setup",[11,823,824,825,828,829,832],{},"Step one within the tests is to set up the ",[387,826,827],{},"beforeEach"," and ",[387,830,831],{},"afterEach"," functions within your tests - these functions are called before and after each test is run within the script. These steps can be used to set up mock objects to test your code against. IN this instance we need to set up a mock page, browser, puppeteer instance and timers.",[381,834,836],{"className":383,"code":835,"language":385,"meta":15,"style":15},"let mockPage\nlet mockBrowser\nlet mockPuppeteer\nlet mockLocator\n\nbeforeEach(() => {\n  \u002F\u002F Mock the locator first\n  mockLocator = {\n    fill: jest.fn().mockResolvedValue(),\n  }\n\n  \u002F\u002F Mock the page object and its methods\n  mockPage = {\n    goto: jest.fn().mockResolvedValue(),\n    locator: jest.fn().mockResolvedValue(mockLocator), \u002F\u002F Return the mock locator\n    close: jest.fn().mockResolvedValue(),\n  }\n\n  \u002F\u002F Mock the browser object and its methods\n  mockBrowser = {\n    newPage: jest.fn().mockResolvedValue(mockPage),\n    close: jest.fn().mockResolvedValue(),\n  }\n\n  \u002F\u002F Mock the Puppeteer module itself\n  mockPuppeteer = {\n    connect: jest.fn().mockResolvedValue(mockBrowser),\n  }\n\n  \u002F\u002F Mock the global setTimeout to prevent the test from waiting 5 seconds.\n  jest.useFakeTimers()\n})\n\nafterEach(() => {\n  \u002F\u002F Restore the timers after each test\n  jest.useRealTimers()\n})\n",[387,837,838,846,853,860,867,871,881,886,896,913,917,921,926,935,948,965,978,982,986,991,1000,1014,1026,1030,1034,1039,1048,1062,1066,1070,1075,1085,1091,1096,1107,1113,1123],{"__ignoreMap":15},[390,839,840,843],{"class":392,"line":393},[390,841,842],{"class":396},"let",[390,844,845],{"class":400}," mockPage\n",[390,847,848,850],{"class":392,"line":173},[390,849,842],{"class":396},[390,851,852],{"class":400}," mockBrowser\n",[390,854,855,857],{"class":392,"line":16},[390,856,842],{"class":396},[390,858,859],{"class":400}," mockPuppeteer\n",[390,861,862,864],{"class":392,"line":440},[390,863,842],{"class":396},[390,865,866],{"class":400}," mockLocator\n",[390,868,869],{"class":392,"line":458},[390,870,413],{"emptyLinePlaceholder":24},[390,872,873,875,877,879],{"class":392,"line":482},[390,874,827],{"class":421},[390,876,639],{"class":400},[390,878,434],{"class":396},[390,880,437],{"class":400},[390,882,883],{"class":392,"line":494},[390,884,885],{"class":571},"  \u002F\u002F Mock the locator first\n",[390,887,888,891,894],{"class":392,"line":500},[390,889,890],{"class":400},"  mockLocator ",[390,892,893],{"class":396},"=",[390,895,437],{"class":400},[390,897,898,901,904,907,910],{"class":392,"line":514},[390,899,900],{"class":400},"    fill: jest.",[390,902,903],{"class":421},"fn",[390,905,906],{"class":400},"().",[390,908,909],{"class":421},"mockResolvedValue",[390,911,912],{"class":400},"(),\n",[390,914,915],{"class":392,"line":522},[390,916,749],{"class":400},[390,918,919],{"class":392,"line":544},[390,920,413],{"emptyLinePlaceholder":24},[390,922,923],{"class":392,"line":563},[390,924,925],{"class":571},"  \u002F\u002F Mock the page object and its methods\n",[390,927,928,931,933],{"class":392,"line":568},[390,929,930],{"class":400},"  mockPage ",[390,932,893],{"class":396},[390,934,437],{"class":400},[390,936,937,940,942,944,946],{"class":392,"line":575},[390,938,939],{"class":400},"    goto: jest.",[390,941,903],{"class":421},[390,943,906],{"class":400},[390,945,909],{"class":421},[390,947,912],{"class":400},[390,949,950,953,955,957,959,962],{"class":392,"line":603},[390,951,952],{"class":400},"    locator: jest.",[390,954,903],{"class":421},[390,956,906],{"class":400},[390,958,909],{"class":421},[390,960,961],{"class":400},"(mockLocator), ",[390,963,964],{"class":571},"\u002F\u002F Return the mock locator\n",[390,966,967,970,972,974,976],{"class":392,"line":608},[390,968,969],{"class":400},"    close: jest.",[390,971,903],{"class":421},[390,973,906],{"class":400},[390,975,909],{"class":421},[390,977,912],{"class":400},[390,979,980],{"class":392,"line":633},[390,981,749],{"class":400},[390,983,984],{"class":392,"line":646},[390,985,413],{"emptyLinePlaceholder":24},[390,987,988],{"class":392,"line":654},[390,989,990],{"class":571},"  \u002F\u002F Mock the browser object and its methods\n",[390,992,993,996,998],{"class":392,"line":665},[390,994,995],{"class":400},"  mockBrowser ",[390,997,893],{"class":396},[390,999,437],{"class":400},[390,1001,1002,1005,1007,1009,1011],{"class":392,"line":671},[390,1003,1004],{"class":400},"    newPage: jest.",[390,1006,903],{"class":421},[390,1008,906],{"class":400},[390,1010,909],{"class":421},[390,1012,1013],{"class":400},"(mockPage),\n",[390,1015,1016,1018,1020,1022,1024],{"class":392,"line":676},[390,1017,969],{"class":400},[390,1019,903],{"class":421},[390,1021,906],{"class":400},[390,1023,909],{"class":421},[390,1025,912],{"class":400},[390,1027,1028],{"class":392,"line":688},[390,1029,749],{"class":400},[390,1031,1032],{"class":392,"line":699},[390,1033,413],{"emptyLinePlaceholder":24},[390,1035,1036],{"class":392,"line":714},[390,1037,1038],{"class":571},"  \u002F\u002F Mock the Puppeteer module itself\n",[390,1040,1041,1044,1046],{"class":392,"line":726},[390,1042,1043],{"class":400},"  mockPuppeteer ",[390,1045,893],{"class":396},[390,1047,437],{"class":400},[390,1049,1050,1053,1055,1057,1059],{"class":392,"line":735},[390,1051,1052],{"class":400},"    connect: jest.",[390,1054,903],{"class":421},[390,1056,906],{"class":400},[390,1058,909],{"class":421},[390,1060,1061],{"class":400},"(mockBrowser),\n",[390,1063,1064],{"class":392,"line":746},[390,1065,749],{"class":400},[390,1067,1068],{"class":392,"line":752},[390,1069,413],{"emptyLinePlaceholder":24},[390,1071,1072],{"class":392,"line":758},[390,1073,1074],{"class":571},"  \u002F\u002F Mock the global setTimeout to prevent the test from waiting 5 seconds.\n",[390,1076,1077,1080,1083],{"class":392,"line":763},[390,1078,1079],{"class":400},"  jest.",[390,1081,1082],{"class":421},"useFakeTimers",[390,1084,541],{"class":400},[390,1086,1088],{"class":392,"line":1087},32,[390,1089,1090],{"class":400},"})\n",[390,1092,1094],{"class":392,"line":1093},33,[390,1095,413],{"emptyLinePlaceholder":24},[390,1097,1099,1101,1103,1105],{"class":392,"line":1098},34,[390,1100,831],{"class":421},[390,1102,639],{"class":400},[390,1104,434],{"class":396},[390,1106,437],{"class":400},[390,1108,1110],{"class":392,"line":1109},35,[390,1111,1112],{"class":571},"  \u002F\u002F Restore the timers after each test\n",[390,1114,1116,1118,1121],{"class":392,"line":1115},36,[390,1117,1079],{"class":400},[390,1119,1120],{"class":421},"useRealTimers",[390,1122,541],{"class":400},[390,1124,1126],{"class":392,"line":1125},37,[390,1127,1090],{"class":400},[43,1129,1131],{"id":1130},"the-tests","The tests",[11,1133,1134,1135,1137],{},"We're going to look at one of the two tests generated, the one that tests the main functionality of the script that we are testing - whether or not it fills in the Google search bar. In this case we are testing to confirm that the ",[387,1136,593],{}," command has been called successfully:",[381,1139,1141],{"className":383,"code":1140,"language":385,"meta":15,"style":15},"test('should successfully navigate and fill the search bar', async () => {\n  \u002F\u002F Wrap the logic in a mock for the global alert function if needed, but since\n  \u002F\u002F the original `alert()` is not standard for Node.js, we should change it to `throw`.\n  try {\n    \u002F\u002F Run the main automation function with our mocked puppeteer.\n    await runAutomation(mockPuppeteer)\n\n    \u002F\u002F Fast-forward time to skip the setTimeout call.\n    jest.advanceTimersByTime(5000)\n\n    \u002F\u002F Assertions to check if the correct methods were called.\n    \u002F\u002F Check that the script connected to the browser.\n    expect(mockPuppeteer.connect).toHaveBeenCalledTimes(1)\n    expect(mockPuppeteer.connect).toHaveBeenCalledWith(\n      expect.objectContaining({\n        browserWSEndpoint: expect.any(String), \u002F\u002F We don't need to check the full string\n      })\n    )\n\n    \u002F\u002F Check that a new page was opened.\n    expect(mockBrowser.newPage).toHaveBeenCalledTimes(1)\n\n    \u002F\u002F Check that it navigated to the correct URL.\n    expect(mockPage.goto).toHaveBeenCalledTimes(1)\n    expect(mockPage.goto).toHaveBeenCalledWith('https:\u002F\u002Fgoogle.com\u002F')\n\n    \u002F\u002F Check that it located the search bar.\n    expect(mockPage.locator).toHaveBeenCalledWith('aria\u002FSearch')\n\n    \u002F\u002F Check that it filled the search bar with the correct text.\n    expect(mockLocator.fill).toHaveBeenCalledTimes(1)\n    expect(mockLocator.fill).toHaveBeenCalledWith('Testing axiom.ai')\n\n    \u002F\u002F Check that the page and browser were closed correctly.\n    expect(mockPage.close).toHaveBeenCalledTimes(1)\n    expect(mockBrowser.close).toHaveBeenCalledTimes(1)\n  } catch (e) {\n    \u002F\u002F If an error is thrown, the test will fail.\n    \u002F\u002F This is how we test the happy path.\n    throw e\n  }\n})\n",[387,1142,1143,1165,1170,1175,1181,1186,1196,1200,1205,1219,1223,1228,1233,1251,1263,1273,1287,1292,1297,1301,1306,1321,1325,1330,1345,1359,1363,1368,1383,1387,1392,1407,1421,1425,1430,1445,1460,1468,1474,1480,1489,1494],{"__ignoreMap":15},[390,1144,1145,1148,1150,1153,1156,1159,1161,1163],{"class":392,"line":393},[390,1146,1147],{"class":421},"test",[390,1149,449],{"class":400},[390,1151,1152],{"class":407},"'should successfully navigate and fill the search bar'",[390,1154,1155],{"class":400},", ",[390,1157,1158],{"class":396},"async",[390,1160,431],{"class":400},[390,1162,434],{"class":396},[390,1164,437],{"class":400},[390,1166,1167],{"class":392,"line":173},[390,1168,1169],{"class":571},"  \u002F\u002F Wrap the logic in a mock for the global alert function if needed, but since\n",[390,1171,1172],{"class":392,"line":16},[390,1173,1174],{"class":571},"  \u002F\u002F the original `alert()` is not standard for Node.js, we should change it to `throw`.\n",[390,1176,1177,1179],{"class":392,"line":440},[390,1178,517],{"class":396},[390,1180,437],{"class":400},[390,1182,1183],{"class":392,"line":458},[390,1184,1185],{"class":571},"    \u002F\u002F Run the main automation function with our mocked puppeteer.\n",[390,1187,1188,1190,1193],{"class":392,"line":482},[390,1189,547],{"class":396},[390,1191,1192],{"class":421}," runAutomation",[390,1194,1195],{"class":400},"(mockPuppeteer)\n",[390,1197,1198],{"class":392,"line":494},[390,1199,413],{"emptyLinePlaceholder":24},[390,1201,1202],{"class":392,"line":500},[390,1203,1204],{"class":571},"    \u002F\u002F Fast-forward time to skip the setTimeout call.\n",[390,1206,1207,1210,1213,1215,1217],{"class":392,"line":514},[390,1208,1209],{"class":400},"    jest.",[390,1211,1212],{"class":421},"advanceTimersByTime",[390,1214,449],{"class":400},[390,1216,660],{"class":464},[390,1218,455],{"class":400},[390,1220,1221],{"class":392,"line":522},[390,1222,413],{"emptyLinePlaceholder":24},[390,1224,1225],{"class":392,"line":544},[390,1226,1227],{"class":571},"    \u002F\u002F Assertions to check if the correct methods were called.\n",[390,1229,1230],{"class":392,"line":563},[390,1231,1232],{"class":571},"    \u002F\u002F Check that the script connected to the browser.\n",[390,1234,1235,1238,1241,1244,1246,1249],{"class":392,"line":568},[390,1236,1237],{"class":421},"    expect",[390,1239,1240],{"class":400},"(mockPuppeteer.connect).",[390,1242,1243],{"class":421},"toHaveBeenCalledTimes",[390,1245,449],{"class":400},[390,1247,1248],{"class":464},"1",[390,1250,455],{"class":400},[390,1252,1253,1255,1257,1260],{"class":392,"line":575},[390,1254,1237],{"class":421},[390,1256,1240],{"class":400},[390,1258,1259],{"class":421},"toHaveBeenCalledWith",[390,1261,1262],{"class":400},"(\n",[390,1264,1265,1268,1271],{"class":392,"line":603},[390,1266,1267],{"class":400},"      expect.",[390,1269,1270],{"class":421},"objectContaining",[390,1272,479],{"class":400},[390,1274,1275,1278,1281,1284],{"class":392,"line":608},[390,1276,1277],{"class":400},"        browserWSEndpoint: expect.",[390,1279,1280],{"class":421},"any",[390,1282,1283],{"class":400},"(String), ",[390,1285,1286],{"class":571},"\u002F\u002F We don't need to check the full string\n",[390,1288,1289],{"class":392,"line":633},[390,1290,1291],{"class":400},"      })\n",[390,1293,1294],{"class":392,"line":646},[390,1295,1296],{"class":400},"    )\n",[390,1298,1299],{"class":392,"line":654},[390,1300,413],{"emptyLinePlaceholder":24},[390,1302,1303],{"class":392,"line":665},[390,1304,1305],{"class":571},"    \u002F\u002F Check that a new page was opened.\n",[390,1307,1308,1310,1313,1315,1317,1319],{"class":392,"line":671},[390,1309,1237],{"class":421},[390,1311,1312],{"class":400},"(mockBrowser.newPage).",[390,1314,1243],{"class":421},[390,1316,449],{"class":400},[390,1318,1248],{"class":464},[390,1320,455],{"class":400},[390,1322,1323],{"class":392,"line":676},[390,1324,413],{"emptyLinePlaceholder":24},[390,1326,1327],{"class":392,"line":688},[390,1328,1329],{"class":571},"    \u002F\u002F Check that it navigated to the correct URL.\n",[390,1331,1332,1334,1337,1339,1341,1343],{"class":392,"line":699},[390,1333,1237],{"class":421},[390,1335,1336],{"class":400},"(mockPage.goto).",[390,1338,1243],{"class":421},[390,1340,449],{"class":400},[390,1342,1248],{"class":464},[390,1344,455],{"class":400},[390,1346,1347,1349,1351,1353,1355,1357],{"class":392,"line":714},[390,1348,1237],{"class":421},[390,1350,1336],{"class":400},[390,1352,1259],{"class":421},[390,1354,449],{"class":400},[390,1356,558],{"class":407},[390,1358,455],{"class":400},[390,1360,1361],{"class":392,"line":726},[390,1362,413],{"emptyLinePlaceholder":24},[390,1364,1365],{"class":392,"line":735},[390,1366,1367],{"class":571},"    \u002F\u002F Check that it located the search bar.\n",[390,1369,1370,1372,1375,1377,1379,1381],{"class":392,"line":746},[390,1371,1237],{"class":421},[390,1373,1374],{"class":400},"(mockPage.locator).",[390,1376,1259],{"class":421},[390,1378,449],{"class":400},[390,1380,587],{"class":407},[390,1382,455],{"class":400},[390,1384,1385],{"class":392,"line":752},[390,1386,413],{"emptyLinePlaceholder":24},[390,1388,1389],{"class":392,"line":758},[390,1390,1391],{"class":571},"    \u002F\u002F Check that it filled the search bar with the correct text.\n",[390,1393,1394,1396,1399,1401,1403,1405],{"class":392,"line":763},[390,1395,1237],{"class":421},[390,1397,1398],{"class":400},"(mockLocator.fill).",[390,1400,1243],{"class":421},[390,1402,449],{"class":400},[390,1404,1248],{"class":464},[390,1406,455],{"class":400},[390,1408,1409,1411,1413,1415,1417,1419],{"class":392,"line":1087},[390,1410,1237],{"class":421},[390,1412,1398],{"class":400},[390,1414,1259],{"class":421},[390,1416,449],{"class":400},[390,1418,598],{"class":407},[390,1420,455],{"class":400},[390,1422,1423],{"class":392,"line":1093},[390,1424,413],{"emptyLinePlaceholder":24},[390,1426,1427],{"class":392,"line":1098},[390,1428,1429],{"class":571},"    \u002F\u002F Check that the page and browser were closed correctly.\n",[390,1431,1432,1434,1437,1439,1441,1443],{"class":392,"line":1109},[390,1433,1237],{"class":421},[390,1435,1436],{"class":400},"(mockPage.close).",[390,1438,1243],{"class":421},[390,1440,449],{"class":400},[390,1442,1248],{"class":464},[390,1444,455],{"class":400},[390,1446,1447,1449,1452,1454,1456,1458],{"class":392,"line":1115},[390,1448,1237],{"class":421},[390,1450,1451],{"class":400},"(mockBrowser.close).",[390,1453,1243],{"class":421},[390,1455,449],{"class":400},[390,1457,1248],{"class":464},[390,1459,455],{"class":400},[390,1461,1462,1464,1466],{"class":392,"line":1125},[390,1463,717],{"class":400},[390,1465,720],{"class":396},[390,1467,723],{"class":400},[390,1469,1471],{"class":392,"line":1470},38,[390,1472,1473],{"class":571},"    \u002F\u002F If an error is thrown, the test will fail.\n",[390,1475,1477],{"class":392,"line":1476},39,[390,1478,1479],{"class":571},"    \u002F\u002F This is how we test the happy path.\n",[390,1481,1483,1486],{"class":392,"line":1482},40,[390,1484,1485],{"class":396},"    throw",[390,1487,1488],{"class":400}," e\n",[390,1490,1492],{"class":392,"line":1491},41,[390,1493,749],{"class":400},[390,1495,1497],{"class":392,"line":1496},42,[390,1498,1090],{"class":400},[11,1500,1501,1502,1505,1506,1509],{},"As you can see from the script above, this essentially mimics the actions of the original script - using the mock objects to fill in the gaps of the puppeteer instance that does not exist within the scripts. We can see that the script is testing to confirm that the specific functions that we called in the original script are also being called successfully within the test scripts, such as ",[387,1503,1504],{},"mockPage.locator"," to locate the searchbar, or ",[387,1507,1508],{},"mockLocator.fill"," to fill the text into the search bar once it's been found.",[43,1511,166],{"id":165},[11,1513,1514],{},"While the tests and the script above are very basic, the idea remains, allowing LLMs to take a review your code and then output testing scripts for them can speed up development time rapidly. We do still recommend that you have an understanding of the tests that are being output by these LLMs as they can make mistakes and assumptions about the code that you should be able to highlight or manually modify yourself. Even if you decide against using the code that has been generated, this can be used as inspiration when it comes to writing your own code manually.",[11,1516,1517],{},"For junior developers, this can be an extremely helpful learning exercise to understand the structure of the tests within your language of choice, meaning that you can develop your skills to be able to include tests within your projects.",[43,1519,1521],{"id":1520},"full-code","Full code",[11,1523,1524],{},"The following is the full code provided by Gemini, this has not extensively been tested so we would recommend testing before blindly coping this into your code.",[381,1526,1528],{"className":383,"code":1527,"language":385,"meta":15,"style":15},"const start = async () => {\n  console.log('start')\n  const browser = await puppeteer.connect({\n    browserWSEndpoint: '\u003CENDPOINT>',\n  })\n  console.log('browser connected')\n  try {\n    const page = await browser.newPage()\n    await page.goto('https:\u002F\u002Fgoogle.com\u002F')\n\n    \u002F\u002F Locate the search bar on the page\n    await page.locator('aria\u002FSearch').fill('Testing axiom.ai')\n\n    await new Promise((resolve) => {\n      setTimeout(() => {\n        resolve()\n      }, 5000)\n    })\n\n    await page.close()\n    await browser.close()\n    console.log('browser closed')\n  } catch (e) {\n    alert(e.message)\n    await browser.close()\n  }\n}\n\n\u002F\u002F We will use Jest to run these tests.\n\u002F\u002F This is the beginning of our test suite.\ndescribe('Puppeteer Automation Script', () => {\n  \u002F\u002F Before each test, we'll set up our mocks.\n  let mockPage\n  let mockBrowser\n  let mockPuppeteer\n  let mockLocator\n\n  beforeEach(() => {\n    \u002F\u002F Mock the locator first\n    mockLocator = {\n      fill: jest.fn().mockResolvedValue(),\n    }\n\n    \u002F\u002F Mock the page object and its methods\n    mockPage = {\n      goto: jest.fn().mockResolvedValue(),\n      locator: jest.fn().mockResolvedValue(mockLocator), \u002F\u002F Return the mock locator\n      close: jest.fn().mockResolvedValue(),\n    }\n\n    \u002F\u002F Mock the browser object and its methods\n    mockBrowser = {\n      newPage: jest.fn().mockResolvedValue(mockPage),\n      close: jest.fn().mockResolvedValue(),\n    }\n\n    \u002F\u002F Mock the Puppeteer module itself\n    mockPuppeteer = {\n      connect: jest.fn().mockResolvedValue(mockBrowser),\n    }\n\n    \u002F\u002F Mock the global setTimeout to prevent the test from waiting 5 seconds.\n    jest.useFakeTimers()\n  })\n\n  afterEach(() => {\n    \u002F\u002F Restore the timers after each test\n    jest.useRealTimers()\n  })\n\n  test('should successfully navigate and fill the search bar', async () => {\n    \u002F\u002F Wrap the logic in a mock for the global alert function if needed, but since\n    \u002F\u002F the original `alert()` is not standard for Node.js, we should change it to `throw`.\n    try {\n      \u002F\u002F Run the main automation function with our mocked puppeteer.\n      await runAutomation(mockPuppeteer)\n\n      \u002F\u002F Fast-forward time to skip the setTimeout call.\n      jest.advanceTimersByTime(5000)\n\n      \u002F\u002F Assertions to check if the correct methods were called.\n      \u002F\u002F Check that the script connected to the browser.\n      expect(mockPuppeteer.connect).toHaveBeenCalledTimes(1)\n      expect(mockPuppeteer.connect).toHaveBeenCalledWith(\n        expect.objectContaining({\n          browserWSEndpoint: expect.any(String), \u002F\u002F We don't need to check the full string\n        })\n      )\n\n      \u002F\u002F Check that a new page was opened.\n      expect(mockBrowser.newPage).toHaveBeenCalledTimes(1)\n\n      \u002F\u002F Check that it navigated to the correct URL.\n      expect(mockPage.goto).toHaveBeenCalledTimes(1)\n      expect(mockPage.goto).toHaveBeenCalledWith('https:\u002F\u002Fgoogle.com\u002F')\n\n      \u002F\u002F Check that it located the search bar.\n      expect(mockPage.locator).toHaveBeenCalledWith('aria\u002FSearch')\n\n      \u002F\u002F Check that it filled the search bar with the correct text.\n      expect(mockLocator.fill).toHaveBeenCalledTimes(1)\n      expect(mockLocator.fill).toHaveBeenCalledWith('Testing axiom.ai')\n\n      \u002F\u002F Check that the page and browser were closed correctly.\n      expect(mockPage.close).toHaveBeenCalledTimes(1)\n      expect(mockBrowser.close).toHaveBeenCalledTimes(1)\n    } catch (e) {\n      \u002F\u002F If an error is thrown, the test will fail.\n      \u002F\u002F This is how we test the happy path.\n      throw e\n    }\n  })\n\n  test('should handle connection errors gracefully', async () => {\n    \u002F\u002F Mock a connection error.\n    mockPuppeteer.connect.mockRejectedValue(new Error('Connection Failed'))\n\n    \u002F\u002F Use a spy to see if browser.close was called.\n    const mockBrowserCloseSpy = jest.spyOn(mockBrowser, 'close')\n\n    \u002F\u002F Expect the function to throw an error, since we've replaced the alert().\n    await expect(runAutomation(mockPuppeteer)).rejects.toThrow(\n      'Connection Failed'\n    )\n\n    \u002F\u002F Even on failure, the browser should attempt to close.\n    expect(mockBrowserCloseSpy).toHaveBeenCalledTimes(1)\n  })\n})\n",[387,1529,1530,1546,1558,1574,1582,1586,1598,1604,1620,1634,1638,1642,1664,1668,1686,1696,1702,1710,1714,1718,1728,1738,1750,1758,1764,1774,1778,1782,1786,1791,1796,1813,1818,1825,1831,1837,1843,1847,1858,1863,1872,1885,1890,1895,1901,1911,1925,1941,1955,1960,1965,1971,1981,1995,2008,2013,2018,2024,2034,2048,2053,2058,2064,2073,2078,2083,2095,2101,2110,2115,2120,2140,2146,2152,2160,2166,2176,2181,2187,2201,2206,2212,2218,2234,2245,2255,2267,2273,2279,2284,2290,2305,2310,2316,2331,2346,2351,2357,2372,2377,2383,2398,2413,2418,2424,2439,2454,2464,2470,2476,2484,2489,2494,2499,2519,2525,2550,2555,2561,2585,2590,2596,2617,2623,2628,2633,2639,2655,2660],{"__ignoreMap":15},[390,1531,1532,1534,1536,1538,1540,1542,1544],{"class":392,"line":393},[390,1533,418],{"class":396},[390,1535,422],{"class":421},[390,1537,425],{"class":396},[390,1539,428],{"class":396},[390,1541,431],{"class":400},[390,1543,434],{"class":396},[390,1545,437],{"class":400},[390,1547,1548,1550,1552,1554,1556],{"class":392,"line":173},[390,1549,443],{"class":400},[390,1551,446],{"class":421},[390,1553,449],{"class":400},[390,1555,452],{"class":407},[390,1557,455],{"class":400},[390,1559,1560,1562,1564,1566,1568,1570,1572],{"class":392,"line":16},[390,1561,461],{"class":396},[390,1563,465],{"class":464},[390,1565,425],{"class":396},[390,1567,470],{"class":396},[390,1569,473],{"class":400},[390,1571,476],{"class":421},[390,1573,479],{"class":400},[390,1575,1576,1578,1580],{"class":392,"line":440},[390,1577,485],{"class":400},[390,1579,488],{"class":407},[390,1581,491],{"class":400},[390,1583,1584],{"class":392,"line":458},[390,1585,497],{"class":400},[390,1587,1588,1590,1592,1594,1596],{"class":392,"line":482},[390,1589,443],{"class":400},[390,1591,446],{"class":421},[390,1593,449],{"class":400},[390,1595,509],{"class":407},[390,1597,455],{"class":400},[390,1599,1600,1602],{"class":392,"line":494},[390,1601,517],{"class":396},[390,1603,437],{"class":400},[390,1605,1606,1608,1610,1612,1614,1616,1618],{"class":392,"line":500},[390,1607,525],{"class":396},[390,1609,528],{"class":464},[390,1611,425],{"class":396},[390,1613,470],{"class":396},[390,1615,535],{"class":400},[390,1617,538],{"class":421},[390,1619,541],{"class":400},[390,1621,1622,1624,1626,1628,1630,1632],{"class":392,"line":514},[390,1623,547],{"class":396},[390,1625,550],{"class":400},[390,1627,553],{"class":421},[390,1629,449],{"class":400},[390,1631,558],{"class":407},[390,1633,455],{"class":400},[390,1635,1636],{"class":392,"line":522},[390,1637,413],{"emptyLinePlaceholder":24},[390,1639,1640],{"class":392,"line":544},[390,1641,572],{"class":571},[390,1643,1644,1646,1648,1650,1652,1654,1656,1658,1660,1662],{"class":392,"line":563},[390,1645,547],{"class":396},[390,1647,550],{"class":400},[390,1649,582],{"class":421},[390,1651,449],{"class":400},[390,1653,587],{"class":407},[390,1655,590],{"class":400},[390,1657,593],{"class":421},[390,1659,449],{"class":400},[390,1661,598],{"class":407},[390,1663,455],{"class":400},[390,1665,1666],{"class":392,"line":568},[390,1667,413],{"emptyLinePlaceholder":24},[390,1669,1670,1672,1674,1676,1678,1680,1682,1684],{"class":392,"line":575},[390,1671,547],{"class":396},[390,1673,613],{"class":396},[390,1675,616],{"class":464},[390,1677,619],{"class":400},[390,1679,623],{"class":622},[390,1681,626],{"class":400},[390,1683,434],{"class":396},[390,1685,437],{"class":400},[390,1687,1688,1690,1692,1694],{"class":392,"line":603},[390,1689,636],{"class":421},[390,1691,639],{"class":400},[390,1693,434],{"class":396},[390,1695,437],{"class":400},[390,1697,1698,1700],{"class":392,"line":608},[390,1699,649],{"class":421},[390,1701,541],{"class":400},[390,1703,1704,1706,1708],{"class":392,"line":633},[390,1705,657],{"class":400},[390,1707,660],{"class":464},[390,1709,455],{"class":400},[390,1711,1712],{"class":392,"line":646},[390,1713,668],{"class":400},[390,1715,1716],{"class":392,"line":654},[390,1717,413],{"emptyLinePlaceholder":24},[390,1719,1720,1722,1724,1726],{"class":392,"line":665},[390,1721,547],{"class":396},[390,1723,550],{"class":400},[390,1725,683],{"class":421},[390,1727,541],{"class":400},[390,1729,1730,1732,1734,1736],{"class":392,"line":671},[390,1731,547],{"class":396},[390,1733,535],{"class":400},[390,1735,683],{"class":421},[390,1737,541],{"class":400},[390,1739,1740,1742,1744,1746,1748],{"class":392,"line":676},[390,1741,702],{"class":400},[390,1743,446],{"class":421},[390,1745,449],{"class":400},[390,1747,709],{"class":407},[390,1749,455],{"class":400},[390,1751,1752,1754,1756],{"class":392,"line":688},[390,1753,717],{"class":400},[390,1755,720],{"class":396},[390,1757,723],{"class":400},[390,1759,1760,1762],{"class":392,"line":699},[390,1761,729],{"class":421},[390,1763,732],{"class":400},[390,1765,1766,1768,1770,1772],{"class":392,"line":714},[390,1767,547],{"class":396},[390,1769,535],{"class":400},[390,1771,683],{"class":421},[390,1773,541],{"class":400},[390,1775,1776],{"class":392,"line":726},[390,1777,749],{"class":400},[390,1779,1780],{"class":392,"line":735},[390,1781,755],{"class":400},[390,1783,1784],{"class":392,"line":746},[390,1785,413],{"emptyLinePlaceholder":24},[390,1787,1788],{"class":392,"line":752},[390,1789,1790],{"class":571},"\u002F\u002F We will use Jest to run these tests.\n",[390,1792,1793],{"class":392,"line":758},[390,1794,1795],{"class":571},"\u002F\u002F This is the beginning of our test suite.\n",[390,1797,1798,1801,1803,1806,1809,1811],{"class":392,"line":763},[390,1799,1800],{"class":421},"describe",[390,1802,449],{"class":400},[390,1804,1805],{"class":407},"'Puppeteer Automation Script'",[390,1807,1808],{"class":400},", () ",[390,1810,434],{"class":396},[390,1812,437],{"class":400},[390,1814,1815],{"class":392,"line":1087},[390,1816,1817],{"class":571},"  \u002F\u002F Before each test, we'll set up our mocks.\n",[390,1819,1820,1823],{"class":392,"line":1093},[390,1821,1822],{"class":396},"  let",[390,1824,845],{"class":400},[390,1826,1827,1829],{"class":392,"line":1098},[390,1828,1822],{"class":396},[390,1830,852],{"class":400},[390,1832,1833,1835],{"class":392,"line":1109},[390,1834,1822],{"class":396},[390,1836,859],{"class":400},[390,1838,1839,1841],{"class":392,"line":1115},[390,1840,1822],{"class":396},[390,1842,866],{"class":400},[390,1844,1845],{"class":392,"line":1125},[390,1846,413],{"emptyLinePlaceholder":24},[390,1848,1849,1852,1854,1856],{"class":392,"line":1470},[390,1850,1851],{"class":421},"  beforeEach",[390,1853,639],{"class":400},[390,1855,434],{"class":396},[390,1857,437],{"class":400},[390,1859,1860],{"class":392,"line":1476},[390,1861,1862],{"class":571},"    \u002F\u002F Mock the locator first\n",[390,1864,1865,1868,1870],{"class":392,"line":1482},[390,1866,1867],{"class":400},"    mockLocator ",[390,1869,893],{"class":396},[390,1871,437],{"class":400},[390,1873,1874,1877,1879,1881,1883],{"class":392,"line":1491},[390,1875,1876],{"class":400},"      fill: jest.",[390,1878,903],{"class":421},[390,1880,906],{"class":400},[390,1882,909],{"class":421},[390,1884,912],{"class":400},[390,1886,1887],{"class":392,"line":1496},[390,1888,1889],{"class":400},"    }\n",[390,1891,1893],{"class":392,"line":1892},43,[390,1894,413],{"emptyLinePlaceholder":24},[390,1896,1898],{"class":392,"line":1897},44,[390,1899,1900],{"class":571},"    \u002F\u002F Mock the page object and its methods\n",[390,1902,1904,1907,1909],{"class":392,"line":1903},45,[390,1905,1906],{"class":400},"    mockPage ",[390,1908,893],{"class":396},[390,1910,437],{"class":400},[390,1912,1914,1917,1919,1921,1923],{"class":392,"line":1913},46,[390,1915,1916],{"class":400},"      goto: jest.",[390,1918,903],{"class":421},[390,1920,906],{"class":400},[390,1922,909],{"class":421},[390,1924,912],{"class":400},[390,1926,1928,1931,1933,1935,1937,1939],{"class":392,"line":1927},47,[390,1929,1930],{"class":400},"      locator: jest.",[390,1932,903],{"class":421},[390,1934,906],{"class":400},[390,1936,909],{"class":421},[390,1938,961],{"class":400},[390,1940,964],{"class":571},[390,1942,1944,1947,1949,1951,1953],{"class":392,"line":1943},48,[390,1945,1946],{"class":400},"      close: jest.",[390,1948,903],{"class":421},[390,1950,906],{"class":400},[390,1952,909],{"class":421},[390,1954,912],{"class":400},[390,1956,1958],{"class":392,"line":1957},49,[390,1959,1889],{"class":400},[390,1961,1963],{"class":392,"line":1962},50,[390,1964,413],{"emptyLinePlaceholder":24},[390,1966,1968],{"class":392,"line":1967},51,[390,1969,1970],{"class":571},"    \u002F\u002F Mock the browser object and its methods\n",[390,1972,1974,1977,1979],{"class":392,"line":1973},52,[390,1975,1976],{"class":400},"    mockBrowser ",[390,1978,893],{"class":396},[390,1980,437],{"class":400},[390,1982,1984,1987,1989,1991,1993],{"class":392,"line":1983},53,[390,1985,1986],{"class":400},"      newPage: jest.",[390,1988,903],{"class":421},[390,1990,906],{"class":400},[390,1992,909],{"class":421},[390,1994,1013],{"class":400},[390,1996,1998,2000,2002,2004,2006],{"class":392,"line":1997},54,[390,1999,1946],{"class":400},[390,2001,903],{"class":421},[390,2003,906],{"class":400},[390,2005,909],{"class":421},[390,2007,912],{"class":400},[390,2009,2011],{"class":392,"line":2010},55,[390,2012,1889],{"class":400},[390,2014,2016],{"class":392,"line":2015},56,[390,2017,413],{"emptyLinePlaceholder":24},[390,2019,2021],{"class":392,"line":2020},57,[390,2022,2023],{"class":571},"    \u002F\u002F Mock the Puppeteer module itself\n",[390,2025,2027,2030,2032],{"class":392,"line":2026},58,[390,2028,2029],{"class":400},"    mockPuppeteer ",[390,2031,893],{"class":396},[390,2033,437],{"class":400},[390,2035,2037,2040,2042,2044,2046],{"class":392,"line":2036},59,[390,2038,2039],{"class":400},"      connect: jest.",[390,2041,903],{"class":421},[390,2043,906],{"class":400},[390,2045,909],{"class":421},[390,2047,1061],{"class":400},[390,2049,2051],{"class":392,"line":2050},60,[390,2052,1889],{"class":400},[390,2054,2056],{"class":392,"line":2055},61,[390,2057,413],{"emptyLinePlaceholder":24},[390,2059,2061],{"class":392,"line":2060},62,[390,2062,2063],{"class":571},"    \u002F\u002F Mock the global setTimeout to prevent the test from waiting 5 seconds.\n",[390,2065,2067,2069,2071],{"class":392,"line":2066},63,[390,2068,1209],{"class":400},[390,2070,1082],{"class":421},[390,2072,541],{"class":400},[390,2074,2076],{"class":392,"line":2075},64,[390,2077,497],{"class":400},[390,2079,2081],{"class":392,"line":2080},65,[390,2082,413],{"emptyLinePlaceholder":24},[390,2084,2086,2089,2091,2093],{"class":392,"line":2085},66,[390,2087,2088],{"class":421},"  afterEach",[390,2090,639],{"class":400},[390,2092,434],{"class":396},[390,2094,437],{"class":400},[390,2096,2098],{"class":392,"line":2097},67,[390,2099,2100],{"class":571},"    \u002F\u002F Restore the timers after each test\n",[390,2102,2104,2106,2108],{"class":392,"line":2103},68,[390,2105,1209],{"class":400},[390,2107,1120],{"class":421},[390,2109,541],{"class":400},[390,2111,2113],{"class":392,"line":2112},69,[390,2114,497],{"class":400},[390,2116,2118],{"class":392,"line":2117},70,[390,2119,413],{"emptyLinePlaceholder":24},[390,2121,2123,2126,2128,2130,2132,2134,2136,2138],{"class":392,"line":2122},71,[390,2124,2125],{"class":421},"  test",[390,2127,449],{"class":400},[390,2129,1152],{"class":407},[390,2131,1155],{"class":400},[390,2133,1158],{"class":396},[390,2135,431],{"class":400},[390,2137,434],{"class":396},[390,2139,437],{"class":400},[390,2141,2143],{"class":392,"line":2142},72,[390,2144,2145],{"class":571},"    \u002F\u002F Wrap the logic in a mock for the global alert function if needed, but since\n",[390,2147,2149],{"class":392,"line":2148},73,[390,2150,2151],{"class":571},"    \u002F\u002F the original `alert()` is not standard for Node.js, we should change it to `throw`.\n",[390,2153,2155,2158],{"class":392,"line":2154},74,[390,2156,2157],{"class":396},"    try",[390,2159,437],{"class":400},[390,2161,2163],{"class":392,"line":2162},75,[390,2164,2165],{"class":571},"      \u002F\u002F Run the main automation function with our mocked puppeteer.\n",[390,2167,2169,2172,2174],{"class":392,"line":2168},76,[390,2170,2171],{"class":396},"      await",[390,2173,1192],{"class":421},[390,2175,1195],{"class":400},[390,2177,2179],{"class":392,"line":2178},77,[390,2180,413],{"emptyLinePlaceholder":24},[390,2182,2184],{"class":392,"line":2183},78,[390,2185,2186],{"class":571},"      \u002F\u002F Fast-forward time to skip the setTimeout call.\n",[390,2188,2190,2193,2195,2197,2199],{"class":392,"line":2189},79,[390,2191,2192],{"class":400},"      jest.",[390,2194,1212],{"class":421},[390,2196,449],{"class":400},[390,2198,660],{"class":464},[390,2200,455],{"class":400},[390,2202,2204],{"class":392,"line":2203},80,[390,2205,413],{"emptyLinePlaceholder":24},[390,2207,2209],{"class":392,"line":2208},81,[390,2210,2211],{"class":571},"      \u002F\u002F Assertions to check if the correct methods were called.\n",[390,2213,2215],{"class":392,"line":2214},82,[390,2216,2217],{"class":571},"      \u002F\u002F Check that the script connected to the browser.\n",[390,2219,2221,2224,2226,2228,2230,2232],{"class":392,"line":2220},83,[390,2222,2223],{"class":421},"      expect",[390,2225,1240],{"class":400},[390,2227,1243],{"class":421},[390,2229,449],{"class":400},[390,2231,1248],{"class":464},[390,2233,455],{"class":400},[390,2235,2237,2239,2241,2243],{"class":392,"line":2236},84,[390,2238,2223],{"class":421},[390,2240,1240],{"class":400},[390,2242,1259],{"class":421},[390,2244,1262],{"class":400},[390,2246,2248,2251,2253],{"class":392,"line":2247},85,[390,2249,2250],{"class":400},"        expect.",[390,2252,1270],{"class":421},[390,2254,479],{"class":400},[390,2256,2258,2261,2263,2265],{"class":392,"line":2257},86,[390,2259,2260],{"class":400},"          browserWSEndpoint: expect.",[390,2262,1280],{"class":421},[390,2264,1283],{"class":400},[390,2266,1286],{"class":571},[390,2268,2270],{"class":392,"line":2269},87,[390,2271,2272],{"class":400},"        })\n",[390,2274,2276],{"class":392,"line":2275},88,[390,2277,2278],{"class":400},"      )\n",[390,2280,2282],{"class":392,"line":2281},89,[390,2283,413],{"emptyLinePlaceholder":24},[390,2285,2287],{"class":392,"line":2286},90,[390,2288,2289],{"class":571},"      \u002F\u002F Check that a new page was opened.\n",[390,2291,2293,2295,2297,2299,2301,2303],{"class":392,"line":2292},91,[390,2294,2223],{"class":421},[390,2296,1312],{"class":400},[390,2298,1243],{"class":421},[390,2300,449],{"class":400},[390,2302,1248],{"class":464},[390,2304,455],{"class":400},[390,2306,2308],{"class":392,"line":2307},92,[390,2309,413],{"emptyLinePlaceholder":24},[390,2311,2313],{"class":392,"line":2312},93,[390,2314,2315],{"class":571},"      \u002F\u002F Check that it navigated to the correct URL.\n",[390,2317,2319,2321,2323,2325,2327,2329],{"class":392,"line":2318},94,[390,2320,2223],{"class":421},[390,2322,1336],{"class":400},[390,2324,1243],{"class":421},[390,2326,449],{"class":400},[390,2328,1248],{"class":464},[390,2330,455],{"class":400},[390,2332,2334,2336,2338,2340,2342,2344],{"class":392,"line":2333},95,[390,2335,2223],{"class":421},[390,2337,1336],{"class":400},[390,2339,1259],{"class":421},[390,2341,449],{"class":400},[390,2343,558],{"class":407},[390,2345,455],{"class":400},[390,2347,2349],{"class":392,"line":2348},96,[390,2350,413],{"emptyLinePlaceholder":24},[390,2352,2354],{"class":392,"line":2353},97,[390,2355,2356],{"class":571},"      \u002F\u002F Check that it located the search bar.\n",[390,2358,2360,2362,2364,2366,2368,2370],{"class":392,"line":2359},98,[390,2361,2223],{"class":421},[390,2363,1374],{"class":400},[390,2365,1259],{"class":421},[390,2367,449],{"class":400},[390,2369,587],{"class":407},[390,2371,455],{"class":400},[390,2373,2375],{"class":392,"line":2374},99,[390,2376,413],{"emptyLinePlaceholder":24},[390,2378,2380],{"class":392,"line":2379},100,[390,2381,2382],{"class":571},"      \u002F\u002F Check that it filled the search bar with the correct text.\n",[390,2384,2386,2388,2390,2392,2394,2396],{"class":392,"line":2385},101,[390,2387,2223],{"class":421},[390,2389,1398],{"class":400},[390,2391,1243],{"class":421},[390,2393,449],{"class":400},[390,2395,1248],{"class":464},[390,2397,455],{"class":400},[390,2399,2401,2403,2405,2407,2409,2411],{"class":392,"line":2400},102,[390,2402,2223],{"class":421},[390,2404,1398],{"class":400},[390,2406,1259],{"class":421},[390,2408,449],{"class":400},[390,2410,598],{"class":407},[390,2412,455],{"class":400},[390,2414,2416],{"class":392,"line":2415},103,[390,2417,413],{"emptyLinePlaceholder":24},[390,2419,2421],{"class":392,"line":2420},104,[390,2422,2423],{"class":571},"      \u002F\u002F Check that the page and browser were closed correctly.\n",[390,2425,2427,2429,2431,2433,2435,2437],{"class":392,"line":2426},105,[390,2428,2223],{"class":421},[390,2430,1436],{"class":400},[390,2432,1243],{"class":421},[390,2434,449],{"class":400},[390,2436,1248],{"class":464},[390,2438,455],{"class":400},[390,2440,2442,2444,2446,2448,2450,2452],{"class":392,"line":2441},106,[390,2443,2223],{"class":421},[390,2445,1451],{"class":400},[390,2447,1243],{"class":421},[390,2449,449],{"class":400},[390,2451,1248],{"class":464},[390,2453,455],{"class":400},[390,2455,2457,2460,2462],{"class":392,"line":2456},107,[390,2458,2459],{"class":400},"    } ",[390,2461,720],{"class":396},[390,2463,723],{"class":400},[390,2465,2467],{"class":392,"line":2466},108,[390,2468,2469],{"class":571},"      \u002F\u002F If an error is thrown, the test will fail.\n",[390,2471,2473],{"class":392,"line":2472},109,[390,2474,2475],{"class":571},"      \u002F\u002F This is how we test the happy path.\n",[390,2477,2479,2482],{"class":392,"line":2478},110,[390,2480,2481],{"class":396},"      throw",[390,2483,1488],{"class":400},[390,2485,2487],{"class":392,"line":2486},111,[390,2488,1889],{"class":400},[390,2490,2492],{"class":392,"line":2491},112,[390,2493,497],{"class":400},[390,2495,2497],{"class":392,"line":2496},113,[390,2498,413],{"emptyLinePlaceholder":24},[390,2500,2502,2504,2506,2509,2511,2513,2515,2517],{"class":392,"line":2501},114,[390,2503,2125],{"class":421},[390,2505,449],{"class":400},[390,2507,2508],{"class":407},"'should handle connection errors gracefully'",[390,2510,1155],{"class":400},[390,2512,1158],{"class":396},[390,2514,431],{"class":400},[390,2516,434],{"class":396},[390,2518,437],{"class":400},[390,2520,2522],{"class":392,"line":2521},115,[390,2523,2524],{"class":571},"    \u002F\u002F Mock a connection error.\n",[390,2526,2528,2531,2534,2536,2539,2542,2544,2547],{"class":392,"line":2527},116,[390,2529,2530],{"class":400},"    mockPuppeteer.connect.",[390,2532,2533],{"class":421},"mockRejectedValue",[390,2535,449],{"class":400},[390,2537,2538],{"class":396},"new",[390,2540,2541],{"class":421}," Error",[390,2543,449],{"class":400},[390,2545,2546],{"class":407},"'Connection Failed'",[390,2548,2549],{"class":400},"))\n",[390,2551,2553],{"class":392,"line":2552},117,[390,2554,413],{"emptyLinePlaceholder":24},[390,2556,2558],{"class":392,"line":2557},118,[390,2559,2560],{"class":571},"    \u002F\u002F Use a spy to see if browser.close was called.\n",[390,2562,2564,2566,2569,2571,2574,2577,2580,2583],{"class":392,"line":2563},119,[390,2565,525],{"class":396},[390,2567,2568],{"class":464}," mockBrowserCloseSpy",[390,2570,425],{"class":396},[390,2572,2573],{"class":400}," jest.",[390,2575,2576],{"class":421},"spyOn",[390,2578,2579],{"class":400},"(mockBrowser, ",[390,2581,2582],{"class":407},"'close'",[390,2584,455],{"class":400},[390,2586,2588],{"class":392,"line":2587},120,[390,2589,413],{"emptyLinePlaceholder":24},[390,2591,2593],{"class":392,"line":2592},121,[390,2594,2595],{"class":571},"    \u002F\u002F Expect the function to throw an error, since we've replaced the alert().\n",[390,2597,2599,2601,2604,2606,2609,2612,2615],{"class":392,"line":2598},122,[390,2600,547],{"class":396},[390,2602,2603],{"class":421}," expect",[390,2605,449],{"class":400},[390,2607,2608],{"class":421},"runAutomation",[390,2610,2611],{"class":400},"(mockPuppeteer)).rejects.",[390,2613,2614],{"class":421},"toThrow",[390,2616,1262],{"class":400},[390,2618,2620],{"class":392,"line":2619},123,[390,2621,2622],{"class":407},"      'Connection Failed'\n",[390,2624,2626],{"class":392,"line":2625},124,[390,2627,1296],{"class":400},[390,2629,2631],{"class":392,"line":2630},125,[390,2632,413],{"emptyLinePlaceholder":24},[390,2634,2636],{"class":392,"line":2635},126,[390,2637,2638],{"class":571},"    \u002F\u002F Even on failure, the browser should attempt to close.\n",[390,2640,2642,2644,2647,2649,2651,2653],{"class":392,"line":2641},127,[390,2643,1237],{"class":421},[390,2645,2646],{"class":400},"(mockBrowserCloseSpy).",[390,2648,1243],{"class":421},[390,2650,449],{"class":400},[390,2652,1248],{"class":464},[390,2654,455],{"class":400},[390,2656,2658],{"class":392,"line":2657},128,[390,2659,497],{"class":400},[390,2661,2663],{"class":392,"line":2662},129,[390,2664,1090],{"class":400},[2666,2667,2668],"style",{},"html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}html pre.shiki code .sTDnQ, html code.shiki .sTDnQ{--shiki-default:#953800;--shiki-dark:#FFA657}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":2670},[2671,2672,2673,2676,2677,2678],{"id":375,"depth":173,"text":376},{"id":771,"depth":173,"text":772},{"id":801,"depth":173,"text":802,"children":2674},[2675],{"id":820,"depth":16,"text":821},{"id":1130,"depth":173,"text":1131},{"id":165,"depth":173,"text":166},{"id":1520,"depth":173,"text":1521},"2025-08-27","Learn more how to take advantage of large-language models in order to build tests for your automation scripts.",{"metaTitle":367,"read":2682,"summary":2680,"type":387,"tool":2683,"category":2685,"tags":2687,"location":191,"featuredimg":2691,"landingimg":2692,"canonicalUrl":2693},"8 min read",[2684],"Code Dashboard",[2686],"Browser automation",[2688,2689,2690],"code dashboard","testing","javascript","\u002Fblog\u002Fcode-text-post.webp","\u002Fblog\u002Fcode-text-sq.webp","https:\u002F\u002Faxiom.ai\u002Fblog\u002Fwriting-tests-with-llms","\u002Fblog\u002Fwriting-tests-with-llms",{"title":367,"description":2680},"blog\u002Fwriting-tests-with-llms","JBkG002hsOQPN4JvfAmU5m4hVsIvR-aN0hbrmRw5HJY",{"id":2699,"title":2700,"author":23,"body":2701,"date":2835,"description":2705,"draft":181,"extension":19,"meta":2836,"navigation":24,"path":2844,"seo":2845,"stem":2846,"__hash__":2847},"blog\u002Fblog\u002Fwhat-are-anti-detect-browsers-and-why-use-them.md","What Are Anti-Detect Browsers and Why You Should Use Them",{"type":8,"value":2702,"toc":2827},[2703,2706,2709,2713,2716,2719,2722,2726,2729,2732,2753,2756,2760,2763,2766,2769,2772,2776,2779,2782,2790,2794,2797,2811,2814,2817,2820,2824],[11,2704,2705],{},"In today's interconnected digital landscape, every online interaction leaves a trace. From IP addresses and user-agent strings to canvas rendering and WebGL capabilities, websites are increasingly adept at compiling a unique \"digital fingerprint\" for each user. While this practice often serves legitimate purposes like security and personalisation, it also presents significant challenges for developers and organisations engaged in activities that require maintaining privacy, managing multiple accounts, or performing large-scale data collection without triggering sophisticated anti-bot systems.",[11,2707,2708],{},"Anti-detect browsers came in to save the day. These browsers are specialised tools that are engineered to meticulously control and manipulate the data points that make up a browsers digital identity. For developers, understanding the underlying mechanisms of anti-detect browsers isn't just about bypassing restrictions; it's about gaining a deeper insight into browser forensics, advanced fingerprinting techniques, and the evolving cat-and-mouse game between online security measures and privacy-preserving technologies.",[43,2710,2712],{"id":2711},"what-are-digital-fingerprints","What are digital fingerprints",[11,2714,2715],{},"Browser fingerprinting is a method that websites use to identify and track users online by collecting unique information about their web browser and device. This information, such as browser type, operating system, screen resolution, installed fonts, User-Agent, IP address, and much more, are combined to create a \"fingerprint\" that can uniquely identify a user across different websites and browsing sessions.",[11,2717,2718],{},"This fingerprint is then used for a variety of activities by the trackers, such as targeted advertising, website personalisation or as an alternative to cookies - as users can decline permission to set cookies. Fingerprints can also be used for fraud detection and additional security for the website.",[11,2720,2721],{},"Fingerprints are generally difficult to work with as users can change them, or can clear them completely. Anti-detect browser take advantage of this weakness to allow you to spoof your fingerprint.",[43,2723,2725],{"id":2724},"how-anti-detect-browser-work","How anti-detect browser work",[11,2727,2728],{},"Anti-detect browsers take fingerprinting into consideration while you're using them. They use techniques such as changing a value within a fingerprint but not enough to make it look like a fake fingerprint, they keep it as close to the original to make it as genuine as possible to trick websites into thinking it is real. They allow you to create profiles that have consistent fingerprints to ensure that websites know no different!",[11,2730,2731],{},"They spoof your fingerprint using various methods, including:",[249,2733,2734,2737,2740,2747,2750],{},[252,2735,2736],{},"Manipulating your User-Agent string.",[252,2738,2739],{},"Using proxies to change your IP address.",[252,2741,2742,2743,2746],{},"Intercepting and modifying the ",[387,2744,2745],{},"navigator"," object properties to change your hardware details.",[252,2748,2749],{},"Overriding timezone and geo-location information.",[252,2751,2752],{},"Managing cookies and local storage for profile-specific storage and isolations.",[11,2754,2755],{},"All of these factors may be used to spoof a fingerprint, but remember, these will always stay consistent within the profile that's created by the anti-detect browser. Consistency is key and any deviation may give the trackers a hint that it may be spoofed.",[43,2757,2759],{"id":2758},"why-use-an-anti-detect-browser","Why use an anti-detect browser",[11,2761,2762],{},"Anti-detect browsers are quite popular with developers as a method of automated testing, web scraping and research.",[11,2764,2765],{},"From a web scraping, data collection and automated testing point of view, anti-detect browsers allow developers to bypass advanced anti-bot measures such as Cloudflare, while avoiding IP bans. This is important in an automated scenario as it's important that these processes do not get stuck.",[11,2767,2768],{},"For social media managers, anti-detect browsers can still offer some great solutions - specifically around handling multiple social media accounts and remaining logged into them. Each profile, with it's unique fingerprint, can stay logged into multiple social media accounts and can protect the accounts from being blocked due to perceived activity on the accounts. Social media platforms generally do not like automation!",[11,2770,2771],{},"Similar to automated testing, anti-detect browser can be useful in cybersecurity research and penetration testing for applications. As multiple profiles can be used, this gives the testers an opportunity of emulating various user profiles to test various vulnerabilities and to bypass detection from anti-bot detection systems.",[43,2773,2775],{"id":2774},"challenges-and-limitations","Challenges and limitations",[11,2777,2778],{},"Anti-bot detection methods are ever evolving - this is a constant race against the next method that is developed. Cloudflare are one of the major players in this space, even if you don't run bots you will have likely came across one of their screens to confirm if you are a human. The good news is that this is a two sided fight and many anti-detection browsers are fighting back by developing techniques to circumvent these blocking methods.",[11,2780,2781],{},"Cost and performance overhead is something else to consider. Most anti-detection browsers offer a small number of profiles that you can set up for free to test out their product, but this is often not enough for large scale operations where you may need to opt for their subscription which will add extra costs to your project. On top of this, there is additional overhead in setting up profiles, isolating these profiles and continuously spoofing them.",[11,2783,2784,2785,2789],{},"There are always risks that the anti-detect browser itself may be detected which could put your IP at risk of being banned, or your social media accounts being restricted, if this is what you are using the browser for. The chances of this happening are often lower than when you are not using an anti-detect browser but it's ",[2786,2787,2788],"em",{},"never"," zero.",[43,2791,2793],{"id":2792},"choosing-an-anti-detect-browser","Choosing an anti-detect browser",[11,2795,2796],{},"The anti-detect browser that you will need to use will very much depend on your project's needs. There are some key features that we recommend keeping an eye out for:",[249,2798,2799,2802,2805,2808],{},[252,2800,2801],{},"Fingerprint management",[252,2803,2804],{},"Proxy integrations",[252,2806,2807],{},"Pricing",[252,2809,2810],{},"Community support\u002Fdocumentation",[11,2812,2813],{},"If you are looking for web automation, you're going to want to look for an API that can integrate with an automation framework such as Puppeteer or Playwright to allow for the browser to be controlled by automation.",[11,2815,2816],{},"Documentation and community support are an important feature, especially if this is a paid product, support that can be there to help if you run into issues, and documentation that you can refer to to get started quickly and help you solve problems.",[11,2818,2819],{},"Test! Testing the browser itself is important. Make sure to take full advantage of any free plan that's on offer to try a slimmed down version of your project to ensure that it does meet your needs. Using sites like amiunique.org and browserleaks.com can help you check that the anti-detect features have been set up correctly and are working effectively.",[43,2821,2823],{"id":2822},"conclusion","Conclusion",[11,2825,2826],{},"Anti-detect browsers can be extremely helpful when navigating the web in instances where you need multiple digital fingerprints. From automated testing, cybersecurity research and social media account management, they can be used for a variety of tasks that require delicate handling of fingerprints to avoid being detected by websites that could potentially ban your IP address and cause more trouble.",{"title":15,"searchDepth":16,"depth":16,"links":2828},[2829,2830,2831,2832,2833,2834],{"id":2711,"depth":173,"text":2712},{"id":2724,"depth":173,"text":2725},{"id":2758,"depth":173,"text":2759},{"id":2774,"depth":173,"text":2775},{"id":2792,"depth":173,"text":2793},{"id":2822,"depth":173,"text":2823},"2025-07-30",{"read":352,"type":387,"category":2837,"tags":2838,"location":191,"featuredimg":2841,"landingimg":2842,"summary":2843,"video":18},[188],[2839,2840],"automation","anti-detect browsers","\u002Fblog\u002Fanti-detection-browser.webp","\u002Fblog\u002Fanti-detection-browser-sq.webp","Learn about digital fingerprinting, anti-detect browsers and when to take advantage of them in your workflow.","\u002Fblog\u002Fwhat-are-anti-detect-browsers-and-why-use-them",{"title":2700,"description":2705},"blog\u002Fwhat-are-anti-detect-browsers-and-why-use-them","Vcg_es5QYOF2KSZAhtRWQ9XEfAcPRxA5pzHQeq6adM8",{"id":2849,"title":2850,"author":23,"body":2851,"date":3324,"description":15,"draft":181,"extension":19,"meta":3325,"navigation":24,"path":3333,"seo":3334,"stem":3335,"__hash__":3336},"blog\u002Fblog\u002Fmastering-xpath-selectors.md","Mastering XPath Selectors",{"type":8,"value":2852,"toc":3309},[2853,2857,2860,2863,2867,2870,2873,2876,2879,2883,2886,2907,2910,2946,2953,2979,2993,2997,3005,3060,3071,3075,3081,3108,3111,3170,3174,3177,3181,3197,3261,3265,3268,3272,3275,3279,3282,3286,3289,3291,3294,3297,3306],[43,2854,2856],{"id":2855},"introduction","Introduction",[11,2858,2859],{},"You can think of XPath selectors as more powerful CSS selectors, addressing the limitations of standard CSS selectors for complex document navigation. As the web continues to move more towards frameworks that use automatically generated IDs and classes within their code, it has become more difficult to create appropriate CSS selectors that gets the content that you want without a selector that takes 2-3 business days just to read.",[11,2861,2862],{},"This is where xpath selectors come in. XPath (XML Path Language) is a powerful query language that can be used for navigating through nodes and selecting them based on the query. While XML is in the name, this works with HTML where it is often treated as XML (XHTML) or pared into a DOM (Document Object Model) tree.",[43,2864,2866],{"id":2865},"benefits-of-xpath","Benefits of XPath",[11,2868,2869],{},"XPath selectors allows you to identify and extract specific elements from a webpage document with far more flexibility than standard CSS selectors allow for. For example, xpath allows you to query elements within a webpage based on an attribute, which can be helpful to identify specific elements such as inputs based on an attribute like placeholder. This added flexibility allows you to locate elements based on a wider range of criteria.",[11,2871,2872],{},"On top of the flexibility that XPath offers, it can also be more effective for identifying content on dynamic pages that may change on each load or may not have consistent IDs or classes. In addition to this, it's also possible to partially match selectors to help locate dynamic content.",[11,2874,2875],{},"XPath is a W3C standard, which means that it's well-defined and widely adopted. This means that there is plenty of support out there for the standard and support through various programming languages - for example, this is built into Python libraries such as Scrapy, BeautifulSoup and web automation frameworks such as Selenium, Puppeteer and Playright, to name a few.",[11,2877,2878],{},"One of the biggest benefits that people love, including ourselves, is the ability to target an element based on the text that is contained within that element. This can be super helpful when trying to target an element such as a button, or a link, within a page, like a \"Buy now\" button.",[43,2880,2882],{"id":2881},"xpath-fundamentals","XPath Fundamentals",[11,2884,2885],{},"It can sometimes be beneficial to think about xpath queries as file paths as they allow you to specify not just where an element will be in the structure of the page, but also its attributes, content and it's relationship to other elements.",[11,2887,2888,2889,1155,2892,1155,2895,2898,2899,2902,2903,2906],{},"We start off with the standard elements that are available within standard CSS selectors, including ",[387,2890,2891],{},"document",[387,2893,2894],{},"\u003Cdiv>",[387,2896,2897],{},"\u003Cp>"," and all of the usual characters that you'll find in HTML. Attribute nodes are also available, such as the ",[387,2900,2901],{},"href"," attribute in the ",[387,2904,2905],{},"\u003Ca>"," tag.",[11,2908,2909],{},"When constructing an XPath selector there's a specific format that you are required to use:",[249,2911,2912,2929,2943],{},[252,2913,2914,2917,2918,2921,2922,2925,2926],{},[387,2915,2916],{},"\u002F"," to start at the root of the document, for example, ",[387,2919,2920],{},"\u002Fhtml\u002Fbody\u002Fdiv",". Or ",[387,2923,2924],{},"\u002F\u002F"," to target anywhere in the document, ",[387,2927,2928],{},"\u002F\u002Fdiv[@class='my-element']",[252,2930,2931,2932,2934,2935,2938,2939,2942],{},"The name of the node, for example, ",[387,2933,2920],{}," which selects all the ",[387,2936,2937],{},"div"," elements within the ",[387,2940,2941],{},"body"," node",[252,2944,2945],{},"Any other element that you wish to target",[11,2947,2948,2949,2952],{},"There are multiple filters that are available within your XPath selector, we are not going to get into them in this article but it's good to be aware of what can be done. We already seen one above with the ",[387,2950,2951],{},"@class"," filter, but you can also filter by:",[249,2954,2955,2964,2970],{},[252,2956,2957,2960,2961],{},[387,2958,2959],{},"@attribute",", for example, ",[387,2962,2963],{},"@src",[252,2965,2966,2967],{},"By index, if you are targetting a list, for example, ",[387,2968,2969],{},"\u002F\u002Fli[position() \u003C 3]",[252,2971,2972,2973,2960,2976],{},"By text content using ",[387,2974,2975],{},"text()",[387,2977,2978],{},"\u002F\u002Fh2[text()='Add to cart']",[11,2980,2981,2982,1155,2985,2988,2989,2992],{},"You can combine these filters using the ",[387,2983,2984],{},"and",[387,2986,2987],{},"or",", and ",[387,2990,2991],{},"not()"," logic operators.",[43,2994,2996],{"id":2995},"using-xpath-with-javascript","Using XPath with JavaScript",[11,2998,205,2999,3004],{},[50,3000,3003],{"href":3001,"rel":3002},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FAPI\u002FDocument\u002Fevaluate",[54],"document.evaluate()"," function can be used to bring your xpath queries into your JavaScript code. For example, if you wanted to get all of the add to cart buttons on the page, the following code would work:",[381,3006,3008],{"className":383,"code":3007,"language":385,"meta":15,"style":15},"const buttons = document.evaluate(\n    \"\u002F\u002Fbutton[text()='Add to cart']\",\n    null,\n    XPathResult.ANY_TYPE,\n    null\n)\n",[387,3009,3010,3027,3034,3041,3051,3056],{"__ignoreMap":15},[390,3011,3012,3014,3017,3019,3022,3025],{"class":392,"line":393},[390,3013,418],{"class":396},[390,3015,3016],{"class":464}," buttons",[390,3018,425],{"class":396},[390,3020,3021],{"class":400}," document.",[390,3023,3024],{"class":421},"evaluate",[390,3026,1262],{"class":400},[390,3028,3029,3032],{"class":392,"line":173},[390,3030,3031],{"class":407},"    \"\u002F\u002Fbutton[text()='Add to cart']\"",[390,3033,491],{"class":400},[390,3035,3036,3039],{"class":392,"line":16},[390,3037,3038],{"class":464},"    null",[390,3040,491],{"class":400},[390,3042,3043,3046,3049],{"class":392,"line":440},[390,3044,3045],{"class":400},"    XPathResult.",[390,3047,3048],{"class":464},"ANY_TYPE",[390,3050,491],{"class":400},[390,3052,3053],{"class":392,"line":458},[390,3054,3055],{"class":464},"    null\n",[390,3057,3058],{"class":392,"line":482},[390,3059,455],{"class":400},[11,3061,3062,3063,3066,3067,3070],{},"You can see that the second argument provided is in the format that was provided in the ",[50,3064,2882],{"href":3065},"#xpath-fundamentals"," section, nothing about the formatting changes when you bring it into JavaScript. To iterate through these elements you can use ",[387,3068,3069],{},"buttons.iterateNext()"," to access each element individually.",[43,3072,3074],{"id":3073},"combining-xpath-with-puppeteer","Combining XPath with Puppeteer",[11,3076,3077,3078,3080],{},"If you are using a library such as Puppeteer to run your web automations it also supports xpath. This takes advantage of the ",[387,3079,3003],{}," function that is native to JavaScript, but has some slight differences on how it's written, for example, the query to select the add to cart buttons would look like:",[381,3082,3084],{"className":383,"code":3083,"language":385,"meta":15,"style":15},"const buttons = await page.waitForSelector('::-p-xpath(\u002F\u002Fbutton[text()=\"Add to cart\"])')\n",[387,3085,3086],{"__ignoreMap":15},[390,3087,3088,3090,3092,3094,3096,3098,3101,3103,3106],{"class":392,"line":393},[390,3089,418],{"class":396},[390,3091,3016],{"class":464},[390,3093,425],{"class":396},[390,3095,470],{"class":396},[390,3097,550],{"class":400},[390,3099,3100],{"class":421},"waitForSelector",[390,3102,449],{"class":400},[390,3104,3105],{"class":407},"'::-p-xpath(\u002F\u002Fbutton[text()=\"Add to cart\"])'",[390,3107,455],{"class":400},[11,3109,3110],{},"It contains a bit more boilerplate code to get the job done, but works the same - you may wish to skip this and just go straight for the standard implementation. You can then go ahead and use Puppeteers built in options to work with the elements, such as clicking on the button:",[381,3112,3114],{"className":383,"code":3113,"language":385,"meta":15,"style":15},"for (var i = 0; i \u003C buttons.length; i++) {\n    buttons[i].click()\n}\n",[387,3115,3116,3156,3166],{"__ignoreMap":15},[390,3117,3118,3121,3124,3127,3130,3132,3135,3138,3141,3144,3147,3150,3153],{"class":392,"line":393},[390,3119,3120],{"class":396},"for",[390,3122,3123],{"class":400}," (",[390,3125,3126],{"class":396},"var",[390,3128,3129],{"class":400}," i ",[390,3131,893],{"class":396},[390,3133,3134],{"class":464}," 0",[390,3136,3137],{"class":400},"; i ",[390,3139,3140],{"class":396},"\u003C",[390,3142,3143],{"class":400}," buttons.",[390,3145,3146],{"class":464},"length",[390,3148,3149],{"class":400},"; i",[390,3151,3152],{"class":396},"++",[390,3154,3155],{"class":400},") {\n",[390,3157,3158,3161,3164],{"class":392,"line":173},[390,3159,3160],{"class":400},"    buttons[i].",[390,3162,3163],{"class":421},"click",[390,3165,541],{"class":400},[390,3167,3168],{"class":392,"line":16},[390,3169,755],{"class":400},[43,3171,3173],{"id":3172},"using-xpath-with-axiomai","Using XPath with axiom.ai",[11,3175,3176],{},"While axiom.ai does not currently support the use of xpath selectors as custom selectors, it does offer the ability to use custom JavaScript, and access the Puppeteer library to run your custom scripts. This includes the ability to run your xpath queries in order to extract data from websites manually. You can even return data from the 'Write Javascript' step.",[43,3178,3180],{"id":3179},"using-xpath-with-python","Using XPath with Python",[11,3182,3183,3184,3189,3190,828,3193,3196],{},"Interacting with your HTML content with Python and xpath queries is another good choice. This can be done in a similar method that we highlight in our ",[50,3185,3188],{"href":3186,"rel":3187},"https:\u002F\u002Faxiom.ai\u002Fblog\u002Fweb-scraping-with-python",[54],"Web Scraping With Python Tutorial"," - using BeautifulSoup. The only different in this method is making use of ",[387,3191,3192],{},"lxml",[387,3194,3195],{},"requests",", let's review the code:",[381,3198,3202],{"className":3199,"code":3200,"language":3201,"meta":15,"style":15},"language-py shiki shiki-themes github-light-default github-dark-default","import requests \nfrom lxml import etree\nfrom bs4 import BeautifulSoup\n\nhtml = requests.get(\"https\u002F\u002Faxiom.ai\")\nbs = BeautifulSoup(html.text, \"html.parser\")\ndom = etree.HTML(str(bs))\n\nhero_title = dom.xpath('\u002F\u002Fh1')[0].text\nprint(hero_title)\n\n# Output as of 22 July 2025: Browser Automation. Quickly, without code.\n","py",[387,3203,3204,3209,3214,3219,3223,3228,3233,3238,3242,3247,3252,3256],{"__ignoreMap":15},[390,3205,3206],{"class":392,"line":393},[390,3207,3208],{},"import requests \n",[390,3210,3211],{"class":392,"line":173},[390,3212,3213],{},"from lxml import etree\n",[390,3215,3216],{"class":392,"line":16},[390,3217,3218],{},"from bs4 import BeautifulSoup\n",[390,3220,3221],{"class":392,"line":440},[390,3222,413],{"emptyLinePlaceholder":24},[390,3224,3225],{"class":392,"line":458},[390,3226,3227],{},"html = requests.get(\"https\u002F\u002Faxiom.ai\")\n",[390,3229,3230],{"class":392,"line":482},[390,3231,3232],{},"bs = BeautifulSoup(html.text, \"html.parser\")\n",[390,3234,3235],{"class":392,"line":494},[390,3236,3237],{},"dom = etree.HTML(str(bs))\n",[390,3239,3240],{"class":392,"line":500},[390,3241,413],{"emptyLinePlaceholder":24},[390,3243,3244],{"class":392,"line":514},[390,3245,3246],{},"hero_title = dom.xpath('\u002F\u002Fh1')[0].text\n",[390,3248,3249],{"class":392,"line":522},[390,3250,3251],{},"print(hero_title)\n",[390,3253,3254],{"class":392,"line":544},[390,3255,413],{"emptyLinePlaceholder":24},[390,3257,3258],{"class":392,"line":563},[390,3259,3260],{},"# Output as of 22 July 2025: Browser Automation. Quickly, without code.\n",[43,3262,3264],{"id":3263},"use-cases","Use cases",[11,3266,3267],{},"There are a lot of potential use cases for using xpath selectors with your content, this ranges from page interactions, through data extraction and automated testing.",[818,3269,3271],{"id":3270},"automating-add-to-cart","Automating add to cart",[11,3273,3274],{},"Using our example above, it's possible to automate the full process of purchasing through an online store. Using the text filter on an xpath selector allows you to directly target the add to cart button that is required in order for you to add an item to the cart - you may run into issues with the checkout requiring some manual intervention but this should be able to automate most of the process.",[818,3276,3278],{"id":3277},"automated-testing","Automated testing",[11,3280,3281],{},"Xpath queries can be used to automatically test web pages - for example, it can be used to click through user journeys in order to ensure that key features of your application work as expected, such as the ability to add new products to the card. When combined with a library such as Puppeteer can be a super power for your QA team.",[818,3283,3285],{"id":3284},"data-extraction","Data extraction",[11,3287,3288],{},"The most obvious use case for this is data extraction - the process where you automatically extract data from a website in order to save the data for research or analytics. Using xpath queries will allow you to have more flexibility over the elements that you are targetting - especially if you are attempting to extract data from multiple websites that do not have a consistent layout.",[43,3290,2823],{"id":2822},[11,3292,3293],{},"Xpath queries can be used as a great companion to regular CSS selectors and offer an additional layer of flexibility that standard selectors just can't offer. This allows you to create selectors that can easily handle dynamic content, and can handle a great change of structure of web pages that you are looking to interact with. XPath is supported by a wide variety of languages, we have included JavaScript and Python in this example, but the options are pretty limitless, you may just need to search for a specific library that suits your needs.",[3295,3296],"hr",{},[11,3298,3299,3300,3305],{},"We would love to hear what you do with this information, we would love to hear over in our ",[50,3301,3304],{"href":3302,"rel":3303},"https:\u002F\u002Freddit.com\u002Fr\u002Faxiom_ai",[54],"community",". Got a suggestion on how we can improve this article? Let us know!",[2666,3307,3308],{},"html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":3310},[3311,3312,3313,3314,3315,3316,3317,3318,3323],{"id":2855,"depth":173,"text":2856},{"id":2865,"depth":173,"text":2866},{"id":2881,"depth":173,"text":2882},{"id":2995,"depth":173,"text":2996},{"id":3073,"depth":173,"text":3074},{"id":3172,"depth":173,"text":3173},{"id":3179,"depth":173,"text":3180},{"id":3263,"depth":173,"text":3264,"children":3319},[3320,3321,3322],{"id":3270,"depth":16,"text":3271},{"id":3277,"depth":16,"text":3278},{"id":3284,"depth":16,"text":3285},{"id":2822,"depth":173,"text":2823},"2025-07-23",{"read":352,"type":184,"tool":3326,"category":3327,"tags":3328,"location":191,"featuredimg":3329,"landingimg":3330,"layout":3331,"summary":3332,"video":18},[354],[2686],[357,358],"\u002Fblog\u002Fxpath-post.webp","\u002Fblog\u002Fx-path-sq.webp","Guide","Learn all about XPath selectors, how to use them and what they can do to power up your data extraction process.","\u002Fblog\u002Fmastering-xpath-selectors",{"title":2850,"description":15},"blog\u002Fmastering-xpath-selectors","pdKGjmFZNd_mXl-N4sR8JNhlHHBGlNkjX7Sz5AIBRoM",{"id":3338,"title":3188,"author":23,"body":3339,"date":3812,"description":3813,"draft":181,"extension":19,"meta":3814,"navigation":24,"path":3821,"seo":3822,"stem":3823,"__hash__":3824},"blog\u002Fblog\u002Fweb-scraping-with-python.md",{"type":8,"value":3340,"toc":3801},[3341,3344,3347,3351,3354,3357,3368,3375,3379,3390,3423,3426,3461,3465,3473,3484,3542,3549,3587,3594,3598,3601,3660,3663,3725,3729,3732,3736,3739,3744,3747,3752,3756,3759,3764,3787,3790,3792,3798],[11,3342,3343],{},"There are many code and no-code methods of interacting with the web automatically - whether that's to extract data from websites, perform tasks automatically, or any other interaction that you can think of. From extracting prices from a website, automatically filling in websites, or keeping track of social media trends automatically, web scraping can help you with the data side of things.",[11,3345,3346],{},"We are going to be looking at Python in this tutorial and how we can take advantage of it to scrape data from a website. We recommend that you have a basic understanding of Python before attempting to implement this as this tutorial will get into the code side of things.",[43,3348,3350],{"id":3349},"the-process","The process",[11,3352,3353],{},"The process of web scraping can be split into a few different stages.",[11,3355,3356],{},"First, we need to identify the data that we are looking for from the page. This might be something like a price, or a description of a product, or really anything that is present on the page that we are going to retrieve. We are going to need to understand some of the underlying HTML that is used to identify these elements on the page.",[11,3358,3359],{},[2786,3360,3361,3362,3367],{},"Tip: check out our article ",[50,3363,3366],{"href":3364,"rel":3365},"https:\u002F\u002Faxiom.ai\u002Fblog\u002Fbest-custom-css-selectors-for-web-scraping",[54],"The Best Custom CSS Selectors to Use for Automations"," to help you identify good CSS selectors to use.",[11,3369,3370,3371,3374],{},"Remember, it's always wise to respect a websites ",[2786,3372,3373],{},"robots.txt"," file in respect to what they allow for scraping and what they don't - this is a common method that sites use to protect their content. You should also consider the legal ramifications on scraping the data, for example, if you are scraping copyrighted content that you intend to use for commerical purposes this may be a legal no-no. We can't offer advice on legal issues but we would recommend reaching out to a legal professional if you are unsure.",[43,3376,3378],{"id":3377},"retrieving-website-data","Retrieving website data",[11,3380,3381,3382,3385,3386,3389],{},"We are going to want to start by retrieving the data from the website. We are going to use the ",[387,3383,3384],{},"urllib3"," library that is installable using the following command: ",[387,3387,3388],{},"pip install urllib3",". Once this has been installed we can make use of this to make HTTP requests that we can use in order to retrieve data from the website. This library will keep things simple when making these requests. Let's look at an example:",[381,3391,3393],{"className":3199,"code":3392,"language":3201,"meta":15,"style":15},"import urllib3\n\npool_manager = urllib3.PoolManager()\nweb_data = pool_manager.request('GET', 'http:\u002F\u002Fwww.google.com')\n\nprint(web_data.data)\n",[387,3394,3395,3400,3404,3409,3414,3418],{"__ignoreMap":15},[390,3396,3397],{"class":392,"line":393},[390,3398,3399],{},"import urllib3\n",[390,3401,3402],{"class":392,"line":173},[390,3403,413],{"emptyLinePlaceholder":24},[390,3405,3406],{"class":392,"line":16},[390,3407,3408],{},"pool_manager = urllib3.PoolManager()\n",[390,3410,3411],{"class":392,"line":440},[390,3412,3413],{},"web_data = pool_manager.request('GET', 'http:\u002F\u002Fwww.google.com')\n",[390,3415,3416],{"class":392,"line":458},[390,3417,413],{"emptyLinePlaceholder":24},[390,3419,3420],{"class":392,"line":482},[390,3421,3422],{},"print(web_data.data)\n",[11,3424,3425],{},"We're just printing out the data for now but later in the tutorial we will make use of this data and extract from it. If we want to take this one step further and take advantage of proxies, which is recommended for large scale scraping, we can do the following:",[381,3427,3429],{"className":3199,"code":3428,"language":3201,"meta":15,"style":15},"import urllib3\n\nuser_agent_header = urllib3.make_headers(user_agent=\"\u003CUSER_AGENT>\")\npool_manager = urllib3.ProxyManager('\u003CPROXY_IP>', headers=user_agent_header)\nweb_data = pool_manager.request('GET', 'http:\u002F\u002Fwww.google.com')\n\nprint(web_data.data)\n",[387,3430,3431,3435,3439,3444,3449,3453,3457],{"__ignoreMap":15},[390,3432,3433],{"class":392,"line":393},[390,3434,3399],{},[390,3436,3437],{"class":392,"line":173},[390,3438,413],{"emptyLinePlaceholder":24},[390,3440,3441],{"class":392,"line":16},[390,3442,3443],{},"user_agent_header = urllib3.make_headers(user_agent=\"\u003CUSER_AGENT>\")\n",[390,3445,3446],{"class":392,"line":440},[390,3447,3448],{},"pool_manager = urllib3.ProxyManager('\u003CPROXY_IP>', headers=user_agent_header)\n",[390,3450,3451],{"class":392,"line":458},[390,3452,3413],{},[390,3454,3455],{"class":392,"line":482},[390,3456,413],{"emptyLinePlaceholder":24},[390,3458,3459],{"class":392,"line":494},[390,3460,3422],{},[43,3462,3464],{"id":3463},"extracting-website-data","Extracting website data",[11,3466,3467,3468,3472],{},"Now that we have the data from the website we can work with it to extract the data that we are looking for. For the time being this data includes everything from the website, which we can't really do much with for the time being - unless that's your goal, if so, you can stop here. There are a couple of methods that we can use in order to extract data from the website data, including using ",[50,3469,3471],{"href":3470},"#regular-expressions-regex","Regular Expressions"," or additional libraries such as BeautifulSoup. To keep things simple, we are going to make use of BeautifulSoup as it offers an easier to understand method.",[11,3474,3475,3476,3479,3480,3483],{},"To get started, we will need to install ",[387,3477,3478],{},"BeautifulSoup"," using the following command: ",[387,3481,3482],{},"pip install beautifulsoup4",". We are going to take advantage of the snippet above that we used to retrieve the data to build out a script that looks like the following:",[381,3485,3487],{"className":3199,"code":3486,"language":3201,"meta":15,"style":15},"import urllib3\nfrom bs4 import BeautifulSoup\n\npool_manager = urllib3.PoolManager()\nweb_data = pool_manager.request('GET', 'http:\u002F\u002Fwww.google.com')\n\nsoup = BeautifulSoup(web_data.data, 'html.parser')\n\nfor link_tag in soup.find_all('a'):\n  href = link_tag.get('href')\n  if href:\n    print(href)\n",[387,3488,3489,3493,3497,3501,3505,3509,3513,3518,3522,3527,3532,3537],{"__ignoreMap":15},[390,3490,3491],{"class":392,"line":393},[390,3492,3399],{},[390,3494,3495],{"class":392,"line":173},[390,3496,3218],{},[390,3498,3499],{"class":392,"line":16},[390,3500,413],{"emptyLinePlaceholder":24},[390,3502,3503],{"class":392,"line":440},[390,3504,3408],{},[390,3506,3507],{"class":392,"line":458},[390,3508,3413],{},[390,3510,3511],{"class":392,"line":482},[390,3512,413],{"emptyLinePlaceholder":24},[390,3514,3515],{"class":392,"line":494},[390,3516,3517],{},"soup = BeautifulSoup(web_data.data, 'html.parser')\n",[390,3519,3520],{"class":392,"line":500},[390,3521,413],{"emptyLinePlaceholder":24},[390,3523,3524],{"class":392,"line":514},[390,3525,3526],{},"for link_tag in soup.find_all('a'):\n",[390,3528,3529],{"class":392,"line":522},[390,3530,3531],{},"  href = link_tag.get('href')\n",[390,3533,3534],{"class":392,"line":544},[390,3535,3536],{},"  if href:\n",[390,3538,3539],{"class":392,"line":563},[390,3540,3541],{},"    print(href)\n",[11,3543,3544,3545,3548],{},"In the sample above we are just retrieving the links from the page and printing them, but this can be changed to more complex selectors. We can take advantage of the ",[387,3546,3547],{},"select"," function available in BeautifulSoup, for example:",[381,3550,3552],{"className":3199,"code":3551,"language":3201,"meta":15,"style":15},"# ... including the code above\n\n# Return elements based on classname\nclassname_text = soup.select('a.my-favourite-class')\n\n# Return based on attribute\nattribute_text = soup.select('[data-id=\"1234\"]')\n",[387,3553,3554,3559,3563,3568,3573,3577,3582],{"__ignoreMap":15},[390,3555,3556],{"class":392,"line":393},[390,3557,3558],{},"# ... including the code above\n",[390,3560,3561],{"class":392,"line":173},[390,3562,413],{"emptyLinePlaceholder":24},[390,3564,3565],{"class":392,"line":16},[390,3566,3567],{},"# Return elements based on classname\n",[390,3569,3570],{"class":392,"line":440},[390,3571,3572],{},"classname_text = soup.select('a.my-favourite-class')\n",[390,3574,3575],{"class":392,"line":458},[390,3576,413],{"emptyLinePlaceholder":24},[390,3578,3579],{"class":392,"line":482},[390,3580,3581],{},"# Return based on attribute\n",[390,3583,3584],{"class":392,"line":494},[390,3585,3586],{},"attribute_text = soup.select('[data-id=\"1234\"]')\n",[11,3588,3589,3590,3593],{},"You can use this code for all CSS selectors that you can use in the ",[387,3591,3592],{},"document.querySelector"," function available in JavaScript.",[43,3595,3597],{"id":3596},"real-world-example","Real world example",[11,3599,3600],{},"Now that we have some of the basics down it's time to look at a real world example. Let's say that we want to go to rte.ie and download the current news headlines that they are displaying on their homepage. To do this we will first need to retrieve the website data itself, and then we want to extract the headlines from the data. Our code would look something like the following:",[381,3602,3604],{"className":3199,"code":3603,"language":3201,"meta":15,"style":15},"import urllib3\nfrom bs4 import BeautifulSoup\n\npool_manager = urllib3.PoolManager()\nwebsite_data = pool_manager.request('GET', 'http:\u002F\u002Fwww.rte.ie')\n\nsoup = BeautifulSoup(website_data.data, 'html.parser')\n\nfor article_titles in soup.select('.article-title span'):\n    title = article_titles.get('title')\n    if title:\n        print(title)\n",[387,3605,3606,3610,3614,3618,3622,3627,3631,3636,3640,3645,3650,3655],{"__ignoreMap":15},[390,3607,3608],{"class":392,"line":393},[390,3609,3399],{},[390,3611,3612],{"class":392,"line":173},[390,3613,3218],{},[390,3615,3616],{"class":392,"line":16},[390,3617,413],{"emptyLinePlaceholder":24},[390,3619,3620],{"class":392,"line":440},[390,3621,3408],{},[390,3623,3624],{"class":392,"line":458},[390,3625,3626],{},"website_data = pool_manager.request('GET', 'http:\u002F\u002Fwww.rte.ie')\n",[390,3628,3629],{"class":392,"line":482},[390,3630,413],{"emptyLinePlaceholder":24},[390,3632,3633],{"class":392,"line":494},[390,3634,3635],{},"soup = BeautifulSoup(website_data.data, 'html.parser')\n",[390,3637,3638],{"class":392,"line":500},[390,3639,413],{"emptyLinePlaceholder":24},[390,3641,3642],{"class":392,"line":514},[390,3643,3644],{},"for article_titles in soup.select('.article-title span'):\n",[390,3646,3647],{"class":392,"line":522},[390,3648,3649],{},"    title = article_titles.get('title')\n",[390,3651,3652],{"class":392,"line":544},[390,3653,3654],{},"    if title:\n",[390,3656,3657],{"class":392,"line":563},[390,3658,3659],{},"        print(title)\n",[11,3661,3662],{},"Once this has been run, we will end up with a list of headlines being output in the console, as of the time of writing, this is a subset of the results that were retrieved:",[381,3664,3668],{"className":3665,"code":3666,"language":3667,"meta":15,"style":15},"language-txt shiki shiki-themes github-light-default github-dark-default","New documentary highlights inspiring stories from Irish educators\n'I have to have hope' says Tuam relative amid excavation\nEU says could target €72bn of goods if tariff talks fail\nOver half of Wallace allegations substantiated - report\nFive young siblings rescued from sea by off-duty nurses\nStatus Yellow thunderstorm warning for 14 counties\nReliance on Clifford doesn't bode well for Kerry\nWhy we made it - The Last Irish Missionaries\nBryan Dobson on the joy of retirement and his advice to Joe Duffy\nLocal students win national robotics championship\n...\n","txt",[387,3669,3670,3675,3680,3685,3690,3695,3700,3705,3710,3715,3720],{"__ignoreMap":15},[390,3671,3672],{"class":392,"line":393},[390,3673,3674],{},"New documentary highlights inspiring stories from Irish educators\n",[390,3676,3677],{"class":392,"line":173},[390,3678,3679],{},"'I have to have hope' says Tuam relative amid excavation\n",[390,3681,3682],{"class":392,"line":16},[390,3683,3684],{},"EU says could target €72bn of goods if tariff talks fail\n",[390,3686,3687],{"class":392,"line":440},[390,3688,3689],{},"Over half of Wallace allegations substantiated - report\n",[390,3691,3692],{"class":392,"line":458},[390,3693,3694],{},"Five young siblings rescued from sea by off-duty nurses\n",[390,3696,3697],{"class":392,"line":482},[390,3698,3699],{},"Status Yellow thunderstorm warning for 14 counties\n",[390,3701,3702],{"class":392,"line":494},[390,3703,3704],{},"Reliance on Clifford doesn't bode well for Kerry\n",[390,3706,3707],{"class":392,"line":500},[390,3708,3709],{},"Why we made it - The Last Irish Missionaries\n",[390,3711,3712],{"class":392,"line":514},[390,3713,3714],{},"Bryan Dobson on the joy of retirement and his advice to Joe Duffy\n",[390,3716,3717],{"class":392,"line":522},[390,3718,3719],{},"Local students win national robotics championship\n",[390,3721,3722],{"class":392,"line":544},[390,3723,3724],{},"...\n",[43,3726,3728],{"id":3727},"helpful-tips","Helpful tips",[11,3730,3731],{},"We've compiled some helpful tips that you can use within your scripts to help you get the most out of it.",[818,3733,3735],{"id":3734},"regular-expressions-regex","Regular Expressions (RegEx)",[11,3737,3738],{},"Regular Expressions, commonly known as RegEx, are patterns that can be used to match data without knowing the exact data. RegEx is commonly used to match data based on the characters it includes, or the format of the text. A common example of this is using RegEx to check if a string of text is an email, checking for things like the @ symbol and that it looks like an email. These RegExs are not very human-readable, but they are readable by the programming language that you are working with, let's look at a basic one for email:",[11,3740,3741],{},[387,3742,3743],{},"email_regex = r\"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$\"",[11,3745,3746],{},"This looks like a mess but does check for various important pieces of an email address, including: the @ symbol, the format of the domain and that there are enough characters in it to ensure that it is a valid email. It won't catch everything, but should catch most common emails.",[11,3748,3749],{},[2786,3750,3751],{},"Tip: LLMs are fantastic for this type of work, try: \"Can you create a RegEx for checking valid email addresses in Python?\"",[818,3753,3755],{"id":3754},"ssl-error","SSL error",[11,3757,3758],{},"When running this code for the first time, you may experience the following error from urllib3:",[11,3760,3761],{},[387,3762,3763],{},"urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='\u003CSITE>', port=443): Max retries exceeded with url: \u002F (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1028)')))",[11,3765,3766,3767,3770,3771,3774,3775,3778,3779,3782,3783,3786],{},"This most often occurs with macOS ans can be resolved by first installing ",[387,3768,3769],{},"certifi"," using the ",[387,3772,3773],{},"pip install --upgrade certifi"," command. Once this has been done, head into your Python installation, often located at ",[387,3776,3777],{},"Applications\u002FPython 3.x\u002F"," and run the ",[387,3780,3781],{},"Install Certificates.command"," file (you may not see the ",[387,3784,3785],{},".command"," part but that's okay).",[11,3788,3789],{},"Try running your code once more and this should be resolved.",[43,3791,166],{"id":165},[11,3793,3794,3795,13],{},"Web scraping with Python can be pretty straight forward and requires little experience with Python - depending on how in-depth you want to get with your script, of course. We introduced one method of retrieving data from a website and then extracting data from it using BeautifulSoup and urllib3. We're excited to hear about what you do with this - let us know over in our ",[50,3796,3304],{"href":3302,"rel":3797},[54],[2666,3799,3800],{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":3802},[3803,3804,3805,3806,3807,3811],{"id":3349,"depth":173,"text":3350},{"id":3377,"depth":173,"text":3378},{"id":3463,"depth":173,"text":3464},{"id":3596,"depth":173,"text":3597},{"id":3727,"depth":173,"text":3728,"children":3808},[3809,3810],{"id":3734,"depth":16,"text":3735},{"id":3754,"depth":16,"text":3755},{"id":165,"depth":173,"text":166},"2025-07-15","Learn how to scrape the web with your python scripts to give you full control over your scraping process.",{"read":352,"type":387,"tool":3815,"category":3816,"tags":3817,"location":191,"featuredimg":3819,"landingimg":3820,"summary":3813,"video":18},[2684],[188],[2839,3818],"updates","\u002Fblog\u002Fpython-scraping-post.webp","\u002Fblog\u002Fpython-scraping.webp","\u002Fblog\u002Fweb-scraping-with-python",{"title":3188,"description":3813},"blog\u002Fweb-scraping-with-python","p1LRwoGMd3SROC6AvJjkhzpMmI9Oqfs14jdlu-P8Nd0",{"id":3826,"title":3827,"author":23,"body":3828,"date":4038,"description":4039,"draft":181,"extension":19,"meta":4040,"navigation":24,"path":4049,"seo":4050,"stem":4051,"__hash__":4052},"blog\u002Fblog\u002Fweb-scraping-without-getting-blocked.md","Web Scraping Without Getting Blocked",{"type":8,"value":3829,"toc":4026},[3830,3832,3835,3843,3857,3860,3868,3871,3875,3880,3883,3886,3889,3893,3898,3901,3904,3908,3913,3922,3926,3929,3933,3938,3941,3944,3951,3955,3960,3963,3967,3974,3977,3980,4007,4014,4016,4019],[43,3831,2856],{"id":2855},[11,3833,3834],{},"Web scraping, also known as web crawling, is the act of scraping data from a website by downloading and parsing the HTML to extract data. Search engines often do this to be able to surface search results. Often it's easier to find an API that can provide this data to you - this is because it already comes in a machine readable format that is much easier to store and manipulate.",[11,3836,3837,3838,3842],{},"We've seen numerous use cases where ",[50,3839,210],{"href":3840,"rel":3841},"https:\u002F\u002Faxiom.ai",[54]," has helped individuals and businesses automate the scraping of web pages to help them with:",[249,3844,3845,3848,3851,3854],{},[252,3846,3847],{},"Competitor monitoring",[252,3849,3850],{},"Price monitoring",[252,3852,3853],{},"Lead generation",[252,3855,3856],{},"Research dataset collection",[11,3858,3859],{},"Scraping data from the web can be a powerful tool for generating datasets to be later used in wider applications and there are various tools out there that can help you achieve your goals. However, the downside of web scraping (with any of these tools) is sites generally don't like bots scraping their content and will put active blockers in your path, such as:",[249,3861,3862,3865],{},[252,3863,3864],{},"CAPTCHA - where you may need to solve a puzzle to confirm you're human.",[252,3866,3867],{},"CloudFlare verification - a verification tool that will only let you enter a site once it's confirmed you're human (or can convince them you are!).",[11,3869,3870],{},"Let's dive into some things that you can do to unblock yourself.",[43,3872,3874],{"id":3873},"proxies","Proxies",[791,3876,3877],{},[11,3878,3879],{},"A proxy is a server that acts as an intermediary for requests between clients and servers.",[11,3881,3882],{},"Using a proxy allows you to determine various factors on how your activity is perceived by the server that you are requesting resources from - this includes when you're just visiting a site. When making a large number of requests to a specific server, it's possible that they recognise the traffic and block the IP that you are accessing the site from. Using proxies allows you to get around this as it routes your traffic through an IP address that you can change if it gets blocked - this can allow your connection to appear as if it's coming through multiple sources, lessening the risk of being recognised.",[11,3884,3885],{},"Residential proxies are IP addresses that are generally provided to residential connections by ISPs, which means that they are less likely to trigger alarms as they are considered a safer connection - just like you may be accessing this article from! Stealth proxies are also available and are designed to be difficult to recognise as a proxy. Data center proxies are also available, but may be at a higher risk of identification - however, these may be cheaper to purchase. Choosing the right proxy type will depend on the amount of traffic and the level of security\u002Fspoofing that you require for your project.",[11,3887,3888],{},"If you have a large scale project, consider rotating your proxies. This practice allows you to have a single script that can rotate between various proxies. This dramatically reduces the risk of your proxies being blocked as if done right this will distribute the traffic between any proxy that you have access to within your script. This can also help you recover if one of the proxies is blocked.",[43,3890,3892],{"id":3891},"headless","Headless",[791,3894,3895],{},[11,3896,3897],{},"A headless browser is a lightweight browser that lacks a user interface, primarily used for automated testing and web scraping.",[11,3899,3900],{},"While headless browsers lack a user interface, their ability to be controlled programmatically is their real superpower. They can render webpages, interact with them and execute JavaScript as it would be when run on a non-headless browser. Combined with a library like Puppeteer or Selenium this allows you to write scripts to control a headless browser - whether this is for testing or for web scraping. These libraries allow for user interactions, page navigations, handling cookies and other complex tasks like executing their own JavaScript scripts on pages.",[11,3902,3903],{},"The one downside of using a headless browser for testing and web scraping is the lack of user interface - this makes debugging more difficult and we would recommend that you add robust error handling within your scripts to avoid being in the dark when things do go wrong while testing.",[43,3905,3907],{"id":3906},"captcha-solving","CAPTCHA Solving",[791,3909,3910],{},[11,3911,3912],{},"CAPTCHA is a tool that website administrators use to avoid spam or protect login\u002Fregistration forms. There are many types of CAPTCHA that present puzzles to the website visitor.",[11,3914,3915,3916,3921],{},"We all know the dreaded \"I'm not a robot\" prompt that appears from time to time to check if you're human - CAPTCHA will stop your scripts in their tracks unless you account for it during your script runs. There are various services available that can help you solve CAPTCHAs on webpages when operating your browser programmatically - ",[50,3917,3920],{"href":3918,"rel":3919},"https:\u002F\u002F2captcha.com",[54],"2Captcha",", for example. It's worth noting that these services do not always have the ability to solve more complex CAPTCHA, or even custom ones that are created by the organisation who owns the site that you are looking to automate.",[43,3923,3925],{"id":3924},"storing-cookies-and-local-storage","Storing cookies and local storage",[11,3927,3928],{},"Sometimes, the best way to get around CAPTCHA prompts is to avoid what causes them to appear. CAPTCHA will often appear in login\u002Fregistration forms and can prevent you from continuing with the script execution. Storing cookies and local storage means that you can carry over an authentication session into your scripts meaning that you can skip the login process completely and avoid the CAPTCHA showing. This has the benefit of speeding up your scripts.",[43,3930,3932],{"id":3931},"apis","APIs",[791,3934,3935],{},[11,3936,3937],{},"An API (Application Programming Interface) is a set of rules and protocols that allows different software applications to communicate and exchange data with each other.",[11,3939,3940],{},"APIs are used all around us - websites, apps, and operating systems use them for a wide range of applications. Let's say you open the weather app on your phone, the app will reach out to the services API to retrieve the most up-to-date weather data to show it to you.",[11,3942,3943],{},"Using APIs to access the data that you are looking for tends to be a better path when you are looking to extract data from a website - however, finding an API that has the data that you need can be difficult as they are not always officially publicly available. Using an API provides an already formatted dataset that can be used within your script easier than scraping the data from a webpage.",[11,3945,3946,3947,13],{},"As an example of this, you can check out the following website and compare scraping it via a script versus retrieving it using the API: ",[50,3948,3949],{"href":3949,"rel":3950},"https:\u002F\u002Fjsonplaceholder.typicode.com\u002Fposts",[54],[43,3952,3954],{"id":3953},"rate-limits","Rate limits",[791,3956,3957],{},[11,3958,3959],{},"A rate limit is a control mechanism that defines the frequency or number of requests a user or client can make to a server or API within a specific period of time.",[11,3961,3962],{},"Rate limits apply to most APIs in order to protect them from attacks that may render the service unavailable to other users. We use it here at axiom.ai to protect our API. Understanding an APIs rate limit, or a services rate limit, is important to prevent you hitting the ceiling when visiting the website. Rate limits are often based on a per minute basis - for example, for axiom.ai this is 100 requests per minute.",[43,3964,3966],{"id":3965},"bypass-bot-detection","Bypass Bot Detection",[11,3968,3969,3970,3973],{},"This is an axiom.ai specific option - the ability to automatically bypass the Cloudflare verification screen. Cloudflare aims to protect sites from DDoS attacks and ensure that they remain available. This may be thought about as a more advanced CAPTCHA that can't be worked around with standard methods. When your script hits this roadblock it will not continue to run and will likely run into an error, or get stuck running. axiom.ai offers a ",[50,3971,3966],{"href":3972},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsettings\u002Fchrome\u002Fbypass-bot-detection"," option that allows your scripts to get around these roadblocks.",[43,3975,210],{"id":3976},"axiomai",[11,3978,3979],{},"While most roadblocks above can be solved using libraries and services that are available to add into your web scraping scripts, axiom.ai has a lot of these features built into it that can help you get going quickly. We have documentation on getting around all of these roadblocks:",[249,3981,3982,3987,3992,3997,4001],{},[252,3983,3984],{},[50,3985,3874],{"href":3986},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsettings\u002Frun-options\u002Fproxy",[252,3988,3989],{},[50,3990,3892],{"href":3991},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsettings\u002Frun-options\u002Fheadless",[252,3993,3994],{},[50,3995,3907],{"href":3996},"\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002F2captcha",[252,3998,3999],{},[50,4000,3966],{"href":3972},[252,4002,4003],{},[50,4004,4006],{"href":4005},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsettings\u002Frun-options\u002Fstore-cookies","Store cookies and local storage",[11,4008,4009,4010,4013],{},"axiom.ai does not directly support APIs, but they can be used within the ",[50,4011,4012],{"href":15},"Write JavaScript"," step that is available.",[43,4015,166],{"id":165},[11,4017,4018],{},"Encountering these roadblocks can seem like the end of the path when you first encounter them, but that's not the case! Getting around these roadblocks is actually quite easy if you know some of the tools that are available. From using proxies to using CAPTCHA solving services, there are method of getting around these bumps in the road.",[11,4020,4021,4022],{},"Have any other methods that you like to use in your scripts? ",[50,4023,4025],{"href":3302,"rel":4024},[54],"Let us know!",{"title":15,"searchDepth":16,"depth":16,"links":4027},[4028,4029,4030,4031,4032,4033,4034,4035,4036,4037],{"id":2855,"depth":173,"text":2856},{"id":3873,"depth":173,"text":3874},{"id":3891,"depth":173,"text":3892},{"id":3906,"depth":173,"text":3907},{"id":3924,"depth":173,"text":3925},{"id":3931,"depth":173,"text":3932},{"id":3953,"depth":173,"text":3954},{"id":3965,"depth":173,"text":3966},{"id":3976,"depth":173,"text":210},{"id":165,"depth":173,"text":166},"2025-06-03","Learn how to scrape the web without fear of getting blocked.",{"read":352,"type":184,"tool":4041,"category":4042,"tags":4043,"featuredimg":4047,"landingimg":4048,"summary":4039,"video":18,"metaTitle":3827},[354,2684],[188],[3873,3891,4044,4045,4046],"captcha","cookies","rate limits","\u002Fblocked-post.webp","\u002Fblocked-sq.webp","\u002Fblog\u002Fweb-scraping-without-getting-blocked",{"title":3827,"description":4039},"blog\u002Fweb-scraping-without-getting-blocked","oL0s9YWyABpOFChbQuUAG05w0mOlgTBbywMd5PAVIGw",{"id":4054,"title":4055,"author":23,"body":4056,"date":5148,"description":5149,"draft":181,"extension":19,"meta":5150,"navigation":24,"path":5157,"seo":5158,"stem":5159,"__hash__":5160},"blog\u002Fblog\u002Fscraping-web-with-js.md","Scraping Web Content with JavaScript",{"type":8,"value":4057,"toc":5129},[4058,4061,4069,4072,4076,4079,4090,4094,4097,4100,4111,4121,4125,4131,4214,4221,4224,4373,4377,4380,4384,4400,4403,4512,4516,4525,4528,4626,4630,4637,4640,4647,4773,4783,4787,4790,4842,4846,4856,4859,4954,4957,5090,5094,5097,5101,5104,5115,5118,5121,5123,5126],[11,4059,4060],{},"Scraping web content with JavaScript opens up a world of possibilities. This is aimed at developers who are comfortable with the basics of JavaScript and understand how to run scripts. We recommend knowing the basics of:",[249,4062,4063,4066],{},[252,4064,4065],{},"JavaScript and Node.js",[252,4067,4068],{},"DevTools",[11,4070,4071],{},"If you don't have experience with these, don't worry! We are going to attempt to make this article as accessibly as possible to all experience levels.",[43,4073,4075],{"id":4074},"introduction-to-javascript-nodejs","Introduction to JavaScript & Node.js",[11,4077,4078],{},"JavaScript is a scripting language that powers most interactive features on a website that you use - some sites are even built purely in JavaScript using frameworks like React! When you click a button on a website, they will often use JavaScript to perform the expected action, such as submitting a form, or calling another resource.",[11,4080,4081,4082,4086,13],{},"Node.js provides a JavaScript runtime that is based on Chrome's V8 JavaScript engine that allows you to execute your JavaScript libraries outside of a browser context. This means that the scripts that you write no longer need to be included in a web page for them to run and perform actions. This is most useful for allowing JavaScript scripts to run on a server. You can learn more about it at ",[50,4083],{"href":4084,"target":4085},"https:\u002F\u002Fnodejs.org","_blank",[50,4087,4088],{"href":4088,"rel":4089},"https:\u002F\u002Fnodejs.org\u002F",[54],[43,4091,4093],{"id":4092},"querying-a-website","Querying a website",[11,4095,4096],{},"HTTP clients can be used to send requests to a server and then receive a response from the server. When you load a webpage, resources are retrieved from the server of the site that you are viewing.",[11,4098,4099],{},"Ultimately, there are some considerations that you should take into account when choosing the method to use to query the data from a website, such as:",[249,4101,4102,4105,4108],{},[252,4103,4104],{},"Your current environment - you may find that the project that you are working on already has a library installed, or a preferred method of handling HTTP requests.",[252,4106,4107],{},"Features - certain libraries such as Axios and Got offer additional features on top of the standard Fetch API implementation.",[252,4109,4110],{},"Bundle size - any time we add new libraries to a project it can increase the bundle size. If you are concerned about bundle size, it may be best to stick with the Fetch API as a built in option.",[11,4112,4113,4114,285,4117,4120],{},"Where possible, we would recommend attempting to find an API for the content that you are looking to scrape rather than scraping the content directly from the site. For example, rather than scraping a subreddit for the content that you are looking for, consider using their ",[387,4115,4116],{},".json",[387,4118,4119],{},".csv"," endpoints that can give you the data in a machine-readable format.",[43,4122,4124],{"id":4123},"querying-with-pure-javascript","Querying with pure JavaScript",[11,4126,205,4127,4130],{},[387,4128,4129],{},"fetch"," function that is built into JavaScript offers a method of extracting web data from a website, provided it's in a specific format, such as JSON. This function returns a promise and should be dealt as such, for example to retrieve the data on axiom.ai's subreddit:",[381,4132,4134],{"className":383,"code":4133,"language":385,"meta":15,"style":15},"const fetch_data = async () => {\n    const res = await fetch(\"https:\u002F\u002Fwww.reddit.com\u002Fr\u002Faxiom_ai.json\");\n\n    const data = await res.json();\n}\n\nfetch_data();\n",[387,4135,4136,4153,4175,4179,4199,4203,4207],{"__ignoreMap":15},[390,4137,4138,4140,4143,4145,4147,4149,4151],{"class":392,"line":393},[390,4139,418],{"class":396},[390,4141,4142],{"class":421}," fetch_data",[390,4144,425],{"class":396},[390,4146,428],{"class":396},[390,4148,431],{"class":400},[390,4150,434],{"class":396},[390,4152,437],{"class":400},[390,4154,4155,4157,4160,4162,4164,4167,4169,4172],{"class":392,"line":173},[390,4156,525],{"class":396},[390,4158,4159],{"class":464}," res",[390,4161,425],{"class":396},[390,4163,470],{"class":396},[390,4165,4166],{"class":421}," fetch",[390,4168,449],{"class":400},[390,4170,4171],{"class":407},"\"https:\u002F\u002Fwww.reddit.com\u002Fr\u002Faxiom_ai.json\"",[390,4173,4174],{"class":400},");\n",[390,4176,4177],{"class":392,"line":16},[390,4178,413],{"emptyLinePlaceholder":24},[390,4180,4181,4183,4186,4188,4190,4193,4196],{"class":392,"line":440},[390,4182,525],{"class":396},[390,4184,4185],{"class":464}," data",[390,4187,425],{"class":396},[390,4189,470],{"class":396},[390,4191,4192],{"class":400}," res.",[390,4194,4195],{"class":421},"json",[390,4197,4198],{"class":400},"();\n",[390,4200,4201],{"class":392,"line":458},[390,4202,755],{"class":400},[390,4204,4205],{"class":392,"line":482},[390,4206,413],{"emptyLinePlaceholder":24},[390,4208,4209,4212],{"class":392,"line":494},[390,4210,4211],{"class":421},"fetch_data",[390,4213,4198],{"class":400},[11,4215,4216,4217,4220],{},"As this returns a Promise, this will wait for the response from the server before continuing. You can then use the data stored in the ",[387,4218,4219],{},"data"," variable as required.",[11,4222,4223],{},"Node.js also has a built in option to handle HTTP clients to retrieve data from website. This is one of the simplest methods of getting started as it does not require the installation of additional libraries or dependencies.",[381,4225,4227],{"className":383,"code":4226,"language":385,"meta":15,"style":15},"const http = require('http');\n\nconst req = https.request('http:\u002F\u002Fexample.com', res => {\n    const data = [];\n\n    res.on('data', _ => data.push(_));\n    res.on('end', () => console.log(data.join()));\n})\n\nreq.end();\n",[387,4228,4229,4248,4252,4282,4293,4297,4326,4355,4359,4363],{"__ignoreMap":15},[390,4230,4231,4233,4236,4238,4241,4243,4246],{"class":392,"line":393},[390,4232,418],{"class":396},[390,4234,4235],{"class":464}," http",[390,4237,425],{"class":396},[390,4239,4240],{"class":421}," require",[390,4242,449],{"class":400},[390,4244,4245],{"class":407},"'http'",[390,4247,4174],{"class":400},[390,4249,4250],{"class":392,"line":173},[390,4251,413],{"emptyLinePlaceholder":24},[390,4253,4254,4256,4259,4261,4264,4267,4269,4272,4274,4277,4280],{"class":392,"line":16},[390,4255,418],{"class":396},[390,4257,4258],{"class":464}," req",[390,4260,425],{"class":396},[390,4262,4263],{"class":400}," https.",[390,4265,4266],{"class":421},"request",[390,4268,449],{"class":400},[390,4270,4271],{"class":407},"'http:\u002F\u002Fexample.com'",[390,4273,1155],{"class":400},[390,4275,4276],{"class":622},"res",[390,4278,4279],{"class":396}," =>",[390,4281,437],{"class":400},[390,4283,4284,4286,4288,4290],{"class":392,"line":440},[390,4285,525],{"class":396},[390,4287,4185],{"class":464},[390,4289,425],{"class":396},[390,4291,4292],{"class":400}," [];\n",[390,4294,4295],{"class":392,"line":458},[390,4296,413],{"emptyLinePlaceholder":24},[390,4298,4299,4302,4305,4307,4310,4312,4315,4317,4320,4323],{"class":392,"line":482},[390,4300,4301],{"class":400},"    res.",[390,4303,4304],{"class":421},"on",[390,4306,449],{"class":400},[390,4308,4309],{"class":407},"'data'",[390,4311,1155],{"class":400},[390,4313,4314],{"class":622},"_",[390,4316,4279],{"class":396},[390,4318,4319],{"class":400}," data.",[390,4321,4322],{"class":421},"push",[390,4324,4325],{"class":400},"(_));\n",[390,4327,4328,4330,4332,4334,4337,4339,4341,4344,4346,4349,4352],{"class":392,"line":494},[390,4329,4301],{"class":400},[390,4331,4304],{"class":421},[390,4333,449],{"class":400},[390,4335,4336],{"class":407},"'end'",[390,4338,1808],{"class":400},[390,4340,434],{"class":396},[390,4342,4343],{"class":400}," console.",[390,4345,446],{"class":421},[390,4347,4348],{"class":400},"(data.",[390,4350,4351],{"class":421},"join",[390,4353,4354],{"class":400},"()));\n",[390,4356,4357],{"class":392,"line":500},[390,4358,1090],{"class":400},[390,4360,4361],{"class":392,"line":514},[390,4362,413],{"emptyLinePlaceholder":24},[390,4364,4365,4368,4371],{"class":392,"line":522},[390,4366,4367],{"class":400},"req.",[390,4369,4370],{"class":421},"end",[390,4372,4198],{"class":400},[43,4374,4376],{"id":4375},"querying-using-javascript-libraries","Querying using JavaScript libraries",[11,4378,4379],{},"There are various libraries available that can help speed up your development. Let's look into some of them.",[818,4381,4383],{"id":4382},"axios","Axios",[11,4385,4386,4389,4390,4393,4394,4396,4397,13],{},[50,4387,4383],{"href":4388,"target":4085},"https:\u002F\u002Faxios-http.com\u002Fdocs\u002Fintro"," is a Nodejs library that extends the functionality of the native ",[387,4391,4392],{},"http"," module that comes built in. It's quite similar to ",[387,4395,4129],{}," in that it is also promise based. This will need to be installed prior to use by calling: ",[387,4398,4399],{},"npm install axios",[11,4401,4402],{},"To use Axios to retrieve data you can use the following snippet:",[381,4404,4406],{"className":383,"code":4405,"language":385,"meta":15,"style":15},"const axios = require('axios');\n\naxios.get('https:\u002F\u002Fapi.example.com')\n  .then(response => {\n    console.log(response.status);\n    console.log(response.data);\n  })\n  .catch(error => {\n    console.error(error);\n  })\n",[387,4407,4408,4426,4430,4445,4462,4471,4480,4484,4499,4508],{"__ignoreMap":15},[390,4409,4410,4412,4415,4417,4419,4421,4424],{"class":392,"line":393},[390,4411,418],{"class":396},[390,4413,4414],{"class":464}," axios",[390,4416,425],{"class":396},[390,4418,4240],{"class":421},[390,4420,449],{"class":400},[390,4422,4423],{"class":407},"'axios'",[390,4425,4174],{"class":400},[390,4427,4428],{"class":392,"line":173},[390,4429,413],{"emptyLinePlaceholder":24},[390,4431,4432,4435,4438,4440,4443],{"class":392,"line":16},[390,4433,4434],{"class":400},"axios.",[390,4436,4437],{"class":421},"get",[390,4439,449],{"class":400},[390,4441,4442],{"class":407},"'https:\u002F\u002Fapi.example.com'",[390,4444,455],{"class":400},[390,4446,4447,4450,4453,4455,4458,4460],{"class":392,"line":440},[390,4448,4449],{"class":400},"  .",[390,4451,4452],{"class":421},"then",[390,4454,449],{"class":400},[390,4456,4457],{"class":622},"response",[390,4459,4279],{"class":396},[390,4461,437],{"class":400},[390,4463,4464,4466,4468],{"class":392,"line":458},[390,4465,702],{"class":400},[390,4467,446],{"class":421},[390,4469,4470],{"class":400},"(response.status);\n",[390,4472,4473,4475,4477],{"class":392,"line":482},[390,4474,702],{"class":400},[390,4476,446],{"class":421},[390,4478,4479],{"class":400},"(response.data);\n",[390,4481,4482],{"class":392,"line":494},[390,4483,497],{"class":400},[390,4485,4486,4488,4490,4492,4495,4497],{"class":392,"line":500},[390,4487,4449],{"class":400},[390,4489,720],{"class":421},[390,4491,449],{"class":400},[390,4493,4494],{"class":622},"error",[390,4496,4279],{"class":396},[390,4498,437],{"class":400},[390,4500,4501,4503,4505],{"class":392,"line":514},[390,4502,702],{"class":400},[390,4504,4494],{"class":421},[390,4506,4507],{"class":400},"(error);\n",[390,4509,4510],{"class":392,"line":522},[390,4511,497],{"class":400},[818,4513,4515],{"id":4514},"got","Got",[11,4517,4518,4521,4522,13],{},[50,4519,4515],{"href":4520,"target":4085},"https:\u002F\u002Fwww.npmjs.com\u002Fpackage\u002Fgot"," is a feature-rich HTTP request library that is specifically designed to be used with Nodejs. Under the hood it uses the fetch API to send your requests but also offers some advanced functionality that you might like to take advantage of. This will need to be installed prior to use by calling: ",[387,4523,4524],{},"npm install got",[11,4526,4527],{},"To use Got to retrieve data you can use the following snippet:",[381,4529,4531],{"className":383,"code":4530,"language":385,"meta":15,"style":15},"const got = require('got');\n\ngot('https:\u002F\u002Fapi.example.com')\n  .json()\n  .then(data => {\n    console.log(data);\n  })\n  .catch(error => {\n    console.error(error);\n  })\n",[387,4532,4533,4551,4555,4565,4573,4587,4596,4600,4614,4622],{"__ignoreMap":15},[390,4534,4535,4537,4540,4542,4544,4546,4549],{"class":392,"line":393},[390,4536,418],{"class":396},[390,4538,4539],{"class":464}," got",[390,4541,425],{"class":396},[390,4543,4240],{"class":421},[390,4545,449],{"class":400},[390,4547,4548],{"class":407},"'got'",[390,4550,4174],{"class":400},[390,4552,4553],{"class":392,"line":173},[390,4554,413],{"emptyLinePlaceholder":24},[390,4556,4557,4559,4561,4563],{"class":392,"line":16},[390,4558,4514],{"class":421},[390,4560,449],{"class":400},[390,4562,4442],{"class":407},[390,4564,455],{"class":400},[390,4566,4567,4569,4571],{"class":392,"line":440},[390,4568,4449],{"class":400},[390,4570,4195],{"class":421},[390,4572,541],{"class":400},[390,4574,4575,4577,4579,4581,4583,4585],{"class":392,"line":458},[390,4576,4449],{"class":400},[390,4578,4452],{"class":421},[390,4580,449],{"class":400},[390,4582,4219],{"class":622},[390,4584,4279],{"class":396},[390,4586,437],{"class":400},[390,4588,4589,4591,4593],{"class":392,"line":482},[390,4590,702],{"class":400},[390,4592,446],{"class":421},[390,4594,4595],{"class":400},"(data);\n",[390,4597,4598],{"class":392,"line":494},[390,4599,497],{"class":400},[390,4601,4602,4604,4606,4608,4610,4612],{"class":392,"line":500},[390,4603,4449],{"class":400},[390,4605,720],{"class":421},[390,4607,449],{"class":400},[390,4609,4494],{"class":622},[390,4611,4279],{"class":396},[390,4613,437],{"class":400},[390,4615,4616,4618,4620],{"class":392,"line":514},[390,4617,702],{"class":400},[390,4619,4494],{"class":421},[390,4621,4507],{"class":400},[390,4623,4624],{"class":392,"line":522},[390,4625,497],{"class":400},[43,4627,4629],{"id":4628},"data-extraction-with-javascript","Data Extraction with JavaScript",[11,4631,4632,4633,4636],{},"Once you have queried the site for that data that you wish to use within your script, you'll need to figure out how you want to extract that data to be able to actually ",[2786,4634,4635],{},"use"," the data.",[818,4638,4639],{"id":4195},"JSON",[11,4641,4642,4643,4646],{},"If the data that you have retrieved from the website is in JSON format we have good news for you - JavaScript can handle this natively using the ",[387,4644,4645],{},"JSON.parse()"," function. When combined with the Fetch API this can be quite simple to do, for example:",[381,4648,4650],{"className":383,"code":4649,"language":385,"meta":15,"style":15},"fetch('https:\u002F\u002Fapi.example.com')\n  .then(response => {\n    return response.text();\n  })\n  .then(data => {\n    try {\n      const parsedData = JSON.parse(data);\n      console.log(\"Parsed data:\", parsedData);\n    } catch (error) {\n      console.error(error);\n    }\n  })\n",[387,4651,4652,4662,4676,4689,4693,4707,4713,4733,4748,4757,4765,4769],{"__ignoreMap":15},[390,4653,4654,4656,4658,4660],{"class":392,"line":393},[390,4655,4129],{"class":421},[390,4657,449],{"class":400},[390,4659,4442],{"class":407},[390,4661,455],{"class":400},[390,4663,4664,4666,4668,4670,4672,4674],{"class":392,"line":173},[390,4665,4449],{"class":400},[390,4667,4452],{"class":421},[390,4669,449],{"class":400},[390,4671,4457],{"class":622},[390,4673,4279],{"class":396},[390,4675,437],{"class":400},[390,4677,4678,4681,4684,4687],{"class":392,"line":16},[390,4679,4680],{"class":396},"    return",[390,4682,4683],{"class":400}," response.",[390,4685,4686],{"class":421},"text",[390,4688,4198],{"class":400},[390,4690,4691],{"class":392,"line":440},[390,4692,497],{"class":400},[390,4694,4695,4697,4699,4701,4703,4705],{"class":392,"line":458},[390,4696,4449],{"class":400},[390,4698,4452],{"class":421},[390,4700,449],{"class":400},[390,4702,4219],{"class":622},[390,4704,4279],{"class":396},[390,4706,437],{"class":400},[390,4708,4709,4711],{"class":392,"line":482},[390,4710,2157],{"class":396},[390,4712,437],{"class":400},[390,4714,4715,4718,4721,4723,4726,4728,4731],{"class":392,"line":494},[390,4716,4717],{"class":396},"      const",[390,4719,4720],{"class":464}," parsedData",[390,4722,425],{"class":396},[390,4724,4725],{"class":464}," JSON",[390,4727,13],{"class":400},[390,4729,4730],{"class":421},"parse",[390,4732,4595],{"class":400},[390,4734,4735,4738,4740,4742,4745],{"class":392,"line":500},[390,4736,4737],{"class":400},"      console.",[390,4739,446],{"class":421},[390,4741,449],{"class":400},[390,4743,4744],{"class":407},"\"Parsed data:\"",[390,4746,4747],{"class":400},", parsedData);\n",[390,4749,4750,4752,4754],{"class":392,"line":514},[390,4751,2459],{"class":400},[390,4753,720],{"class":396},[390,4755,4756],{"class":400}," (error) {\n",[390,4758,4759,4761,4763],{"class":392,"line":522},[390,4760,4737],{"class":400},[390,4762,4494],{"class":421},[390,4764,4507],{"class":400},[390,4766,4767],{"class":392,"line":544},[390,4768,1889],{"class":400},[390,4770,4771],{"class":392,"line":563},[390,4772,497],{"class":400},[11,4774,4775,4776,4779,4780],{},"This will allow you to access the data stored within the ",[387,4777,4778],{},"parsedData"," variable. ",[2786,4781,4782],{},"Note, we've left out some error handling for brevity.",[818,4784,4786],{"id":4785},"html-regular-expressions","HTML - Regular expressions",[11,4788,4789],{},"If the data that you are retrieving is HTML content then the built in method of handling this would be to use regular expressions (RegEx) to parse the data. This is a bit cumbersome as RegEx can be quite a complex function to construct. We won't be diving into the ins and outs of RegEx during this article, but below is a short snippet on how to extract H1 titles from a string of HTML.",[381,4791,4793],{"className":383,"code":4792,"language":385,"meta":15,"style":15},"const results = htmlString.match(\u002F\u003Ch1>(.+)\u003C\\\u002Fh1>\u002F);\nconsole.log(results);\n",[387,4794,4795,4832],{"__ignoreMap":15},[390,4796,4797,4799,4802,4804,4807,4810,4812,4815,4817,4820,4823,4827,4830],{"class":392,"line":393},[390,4798,418],{"class":396},[390,4800,4801],{"class":464}," results",[390,4803,425],{"class":396},[390,4805,4806],{"class":400}," htmlString.",[390,4808,4809],{"class":421},"match",[390,4811,449],{"class":400},[390,4813,4814],{"class":407},"\u002F\u003Ch1>(",[390,4816,13],{"class":464},[390,4818,4819],{"class":396},"+",[390,4821,4822],{"class":407},")\u003C",[390,4824,4826],{"class":4825},"sa8KN","\\\u002F",[390,4828,4829],{"class":407},"h1>\u002F",[390,4831,4174],{"class":400},[390,4833,4834,4837,4839],{"class":392,"line":173},[390,4835,4836],{"class":400},"console.",[390,4838,446],{"class":421},[390,4840,4841],{"class":400},"(results);\n",[818,4843,4845],{"id":4844},"html-jsdom","HTML - jsdom",[11,4847,4848,4852,4853,13],{},[50,4849,4851],{"href":4850,"target":4085},"https:\u002F\u002Fwww.npmjs.com\u002Fpackage\u002Fjsdom","jsdom"," replicates jQuery's functionality in the browser DOM. This would work best if you are retrieving HTML data from a website, such as scraping the whole site. This library can also parse JavaScript within the HTML, allowing you to interact with basic JavaScript - though it should be noted that this may not work as expected with JS files that are loaded into the page, which most sites will do these days, this is disabled by default. To get started, you'll need to install jsdom in your project by running: ",[387,4854,4855],{},"npm install jsdom",[11,4857,4858],{},"Let's look at an example:",[381,4860,4862],{"className":383,"code":4861,"language":385,"meta":15,"style":15},"const jsdom = require('jsdom');\nconst { JSDOM } = jsdom;\n\nconst dom = new JSDOM(`\u003C!DOCTYPE html>\u003Ch1>Hello, World!\u003C\u002Fh1>\u003C\u002Fhtml>`);\n\n\u002F\u002F Print the content of the H1 element.\nconsole.log(dom.window.document.querySelector('h1').textContent);\n",[387,4863,4864,4882,4900,4904,4925,4929,4934],{"__ignoreMap":15},[390,4865,4866,4868,4871,4873,4875,4877,4880],{"class":392,"line":393},[390,4867,418],{"class":396},[390,4869,4870],{"class":464}," jsdom",[390,4872,425],{"class":396},[390,4874,4240],{"class":421},[390,4876,449],{"class":400},[390,4878,4879],{"class":407},"'jsdom'",[390,4881,4174],{"class":400},[390,4883,4884,4886,4889,4892,4895,4897],{"class":392,"line":173},[390,4885,418],{"class":396},[390,4887,4888],{"class":400}," { ",[390,4890,4891],{"class":464},"JSDOM",[390,4893,4894],{"class":400}," } ",[390,4896,893],{"class":396},[390,4898,4899],{"class":400}," jsdom;\n",[390,4901,4902],{"class":392,"line":16},[390,4903,413],{"emptyLinePlaceholder":24},[390,4905,4906,4908,4911,4913,4915,4918,4920,4923],{"class":392,"line":440},[390,4907,418],{"class":396},[390,4909,4910],{"class":464}," dom",[390,4912,425],{"class":396},[390,4914,613],{"class":396},[390,4916,4917],{"class":421}," JSDOM",[390,4919,449],{"class":400},[390,4921,4922],{"class":407},"`\u003C!DOCTYPE html>\u003Ch1>Hello, World!\u003C\u002Fh1>\u003C\u002Fhtml>`",[390,4924,4174],{"class":400},[390,4926,4927],{"class":392,"line":458},[390,4928,413],{"emptyLinePlaceholder":24},[390,4930,4931],{"class":392,"line":482},[390,4932,4933],{"class":571},"\u002F\u002F Print the content of the H1 element.\n",[390,4935,4936,4938,4940,4943,4946,4948,4951],{"class":392,"line":494},[390,4937,4836],{"class":400},[390,4939,446],{"class":421},[390,4941,4942],{"class":400},"(dom.window.document.",[390,4944,4945],{"class":421},"querySelector",[390,4947,449],{"class":400},[390,4949,4950],{"class":407},"'h1'",[390,4952,4953],{"class":400},").textContent);\n",[11,4955,4956],{},"This can be used for larger websites, with the results stored in an array, for example:",[381,4958,4960],{"className":383,"code":4959,"language":385,"meta":15,"style":15},"const jsdom = require('jsdom');\nconst { JSDOM } = jsdom;\n\n\u002F\u002F Let's imagine we have a bunch of HTML in here like the example above.\nconst dom = new JSDOM(`...`);\n\nconst results = dom.window.document.querySelector('h1.title');\n\nfor (var i = 0; i \u003C results.length; i++) {\n  console.log(results[i]);\n}\n",[387,4961,4962,4978,4992,4996,5001,5020,5024,5044,5048,5077,5086],{"__ignoreMap":15},[390,4963,4964,4966,4968,4970,4972,4974,4976],{"class":392,"line":393},[390,4965,418],{"class":396},[390,4967,4870],{"class":464},[390,4969,425],{"class":396},[390,4971,4240],{"class":421},[390,4973,449],{"class":400},[390,4975,4879],{"class":407},[390,4977,4174],{"class":400},[390,4979,4980,4982,4984,4986,4988,4990],{"class":392,"line":173},[390,4981,418],{"class":396},[390,4983,4888],{"class":400},[390,4985,4891],{"class":464},[390,4987,4894],{"class":400},[390,4989,893],{"class":396},[390,4991,4899],{"class":400},[390,4993,4994],{"class":392,"line":16},[390,4995,413],{"emptyLinePlaceholder":24},[390,4997,4998],{"class":392,"line":440},[390,4999,5000],{"class":571},"\u002F\u002F Let's imagine we have a bunch of HTML in here like the example above.\n",[390,5002,5003,5005,5007,5009,5011,5013,5015,5018],{"class":392,"line":458},[390,5004,418],{"class":396},[390,5006,4910],{"class":464},[390,5008,425],{"class":396},[390,5010,613],{"class":396},[390,5012,4917],{"class":421},[390,5014,449],{"class":400},[390,5016,5017],{"class":407},"`...`",[390,5019,4174],{"class":400},[390,5021,5022],{"class":392,"line":482},[390,5023,413],{"emptyLinePlaceholder":24},[390,5025,5026,5028,5030,5032,5035,5037,5039,5042],{"class":392,"line":494},[390,5027,418],{"class":396},[390,5029,4801],{"class":464},[390,5031,425],{"class":396},[390,5033,5034],{"class":400}," dom.window.document.",[390,5036,4945],{"class":421},[390,5038,449],{"class":400},[390,5040,5041],{"class":407},"'h1.title'",[390,5043,4174],{"class":400},[390,5045,5046],{"class":392,"line":500},[390,5047,413],{"emptyLinePlaceholder":24},[390,5049,5050,5052,5054,5056,5058,5060,5062,5064,5066,5069,5071,5073,5075],{"class":392,"line":514},[390,5051,3120],{"class":396},[390,5053,3123],{"class":400},[390,5055,3126],{"class":396},[390,5057,3129],{"class":400},[390,5059,893],{"class":396},[390,5061,3134],{"class":464},[390,5063,3137],{"class":400},[390,5065,3140],{"class":396},[390,5067,5068],{"class":400}," results.",[390,5070,3146],{"class":464},[390,5072,3149],{"class":400},[390,5074,3152],{"class":396},[390,5076,3155],{"class":400},[390,5078,5079,5081,5083],{"class":392,"line":522},[390,5080,443],{"class":400},[390,5082,446],{"class":421},[390,5084,5085],{"class":400},"(results[i]);\n",[390,5087,5088],{"class":392,"line":544},[390,5089,755],{"class":400},[43,5091,5093],{"id":5092},"using-these-methods-within-axiomai","Using these methods within axiom.ai",[11,5095,5096],{},"The methods above can be used within axiom.ai to extend it's existing functionality. Let's dive into how.",[818,5098,5100],{"id":5099},"no-code-extension","No-code extension",[11,5102,5103],{},"As standard, the axiom.ai extension offers a range of scraping steps that can be used in order to scrape data from a website. These require very little set up and gives you control over the selection of elements on the page without the need to add code.",[11,5105,5106,5107,5110,5111,5114],{},"In addition to this, it's possible to use ",[2786,5108,5109],{},"some"," of the methods above within the ",[50,5112,5113],{"href":15},"Write Javascript"," step within your automations. It's important to note that the importing of libraries is not supported so it would only be possible to use features that make use of the Fetch API.",[818,5116,5117],{"id":387},"Code",[11,5119,5120],{},"As part of our future updates to axiom.ai, it will be possible to build your automations with code and thus would enable all of the methods above within your automations, including the ability to install your own libraries to run your scripts.",[43,5122,166],{"id":165},[11,5124,5125],{},"We covered a lot in this article that has been centred around scraping web content with JavaScript, including some tips on some libraries that you can use in order to parse the data that you have extracted. The above article is by no means exhaustive so you may find other solutions that fit your project better than those above - but hopefully we have pointed you in the right direction!",[2666,5127,5128],{},"html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sTDnQ, html code.shiki .sTDnQ{--shiki-default:#953800;--shiki-dark:#FFA657}html pre.shiki code .sa8KN, html code.shiki .sa8KN{--shiki-default:#116329;--shiki-default-font-weight:bold;--shiki-dark:#7EE787;--shiki-dark-font-weight:bold}html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}",{"title":15,"searchDepth":16,"depth":16,"links":5130},[5131,5132,5133,5134,5138,5143,5147],{"id":4074,"depth":173,"text":4075},{"id":4092,"depth":173,"text":4093},{"id":4123,"depth":173,"text":4124},{"id":4375,"depth":173,"text":4376,"children":5135},[5136,5137],{"id":4382,"depth":16,"text":4383},{"id":4514,"depth":16,"text":4515},{"id":4628,"depth":173,"text":4629,"children":5139},[5140,5141,5142],{"id":4195,"depth":16,"text":4639},{"id":4785,"depth":16,"text":4786},{"id":4844,"depth":16,"text":4845},{"id":5092,"depth":173,"text":5093,"children":5144},[5145,5146],{"id":5099,"depth":16,"text":5100},{"id":387,"depth":16,"text":5117},{"id":165,"depth":173,"text":166},"2025-05-20","Learn how to get started with Javascript web scraping using pure JavaScript, Axios, Got, JSDOM and the Fetch API",{"read":352,"type":387,"tool":5151,"category":5152,"tags":5153,"featuredimg":5155,"landingimg":5156,"summary":5149,"video":18,"metaTitle":4055},[2684],[188],[2690,4129,5154],"require","\u002Fjavascript-web-scraping-post.webp","\u002Fjavascript-web-scraping-sq.webp","\u002Fblog\u002Fscraping-web-with-js",{"title":4055,"description":5149},"blog\u002Fscraping-web-with-js","UQbFsYyZB_nMbmAQbTKkTF-FPheFDi-QstcjaUdrm0w",{"id":5162,"title":5163,"author":23,"body":5164,"date":5615,"description":5616,"draft":181,"extension":19,"meta":5617,"navigation":24,"path":5626,"seo":5627,"stem":5628,"__hash__":5629},"blog\u002Fblog\u002Fwebhooks-explained-build-reactive-automations-today.md","Webhooks Explained - Build Reactive Automations Today",{"type":8,"value":5165,"toc":5606},[5166,5170,5173,5180,5186,5192,5198,5204,5210,5213,5217,5220,5223,5227,5245,5448,5465,5469,5477,5528,5538,5542,5545,5551,5562,5566,5569,5582,5585,5592,5594,5597,5600,5603],[43,5167,5169],{"id":5168},"what-are-webhooks","What are webhooks?",[11,5171,5172],{},"Webhooks are a method for applications to send real-time (or near enough real-time) information to each other automatically. They enable one-way communication that allows for the sending of information between applications. They have some key characteristics that differentiate them from standard APIs, let's explore them.",[11,5174,5175,5179],{},[5176,5177,5178],"strong",{},"Event-driven:"," webhooks are event-driven, meaning that they operate on the principle of \"events.\" When a specific event occurs in one application, it can trigger a webhook that sends information to another application.",[11,5181,5182,5185],{},[5176,5183,5184],{},"Real-time:"," the process of sending information over a webhook happens in real-time, or very close to it, allowing for instant updates and seamless communication between applications.",[11,5187,5188,5191],{},[5176,5189,5190],{},"HTTP-based:"," webhooks typically use HTTP requests, most commonly POST requests, to transmit data.",[11,5193,5194,5197],{},[5176,5195,5196],{},"URL-based destination:"," a webhook requires a designated URL where the payload is sent - this acts as the endpoint for receiving the webhook's message. This allows for multiple endpoints to be created to handle different payloads within the same domain.",[11,5199,5200,5203],{},[5176,5201,5202],{},"Payload:"," webhooks often carry a payload, often formatted in JSON or XML, containing information about the triggering event - this is fully configurable and can be dynamically changed for each request.",[11,5205,5206,5209],{},[5176,5207,5208],{},"One-way communication:"," webhooks are primarily designed for one way communication, from the triggering application to the destination application. The triggering application will often receive a HTTP status code to inform the status of the data sent but will never receive a payload in return.",[11,5211,5212],{},"It has some other benefits, including being simple to integrate with other applications, and reducing server load as the application does not need to constantly 'poll' for changes - that  the usual methods of communication often use. Using webhooks introduces certain disadvantages, notably: security risks from improper implementation, debugging challenges, reliance on external application reliability, and heightened complexity in advanced workflows.",[43,5214,5216],{"id":5215},"why-use-webhooks","Why use webhooks?",[11,5218,5219],{},"Webhooks offer an efficient solution for initiating automated event-driven notifications, particularly when two-way communication is unnecessary. For example, when a user action takes place in an application, a webhook can automatically send a message to a notification system (or messaging platform), which then delivers an alert to the intended recipient. The real strength of webhooks is their ease of configuration. You can dynamically modify the notification recipient through the webhook settings, allowing you to use the same notification system for a variety of users without needing to hardcode specific recipients. Think of webhooks as a dynamic, one-way pipeline, seamlessly connecting applications and delivering event-triggered information.",[11,5221,5222],{},"For developers, webhooks offer broad compatibility, and are readily integrated into applications across diverse programming environments, which will be demonstrated with Javascript and Python examples later in this article. This makes it a good choice when it comes to implementing communications in their applications.",[43,5224,5226],{"id":5225},"implementing-webhooks-in-javascript","Implementing webhooks in JavaScript",[11,5228,5229,5230,5234,5235,5239,5240,590],{},"While there are various libraries available that can help with the implementation of webhooks, such as ",[50,5231,4382],{"href":5232,"rel":5233},"https:\u002F\u002Fwww.npmjs.com\u002Fpackage\u002Faxios",[54],", we are going to stick with the basics - we believe that you should know the basics before diving into libraries as it helps with troubleshooting when issues do occur. To do this in JavaScript, we are going to use the ",[50,5236,4129],{"href":5237,"rel":5238},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FAPI\u002FFetch_API",[54]," function. You can check out a code snippet below (",[50,5241,5244],{"href":5242,"rel":5243},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FAPI\u002FFetch_API\u002FUsing_Fetch",[54],"source",[381,5246,5248],{"className":383,"code":5247,"language":385,"meta":15,"style":15},"async function getData() {\n  try {\n    const response = await fetch('https:\u002F\u002Fexample.org\u002Fwebhook', {\n      method: 'POST',\n      headers: [ 'Content-Type': 'application\u002Fjson' ],\n      body: JSON.stringify({ username: 'example' }),\n      \u002F\u002F ...\n    })\n\n    if (!response.ok) {\n      throw new Error(`Response status: ${response.status}`)\n    }\n\n    const json = await response.json()\n    console.log(json)\n  } catch (error) {\n    console.error(error.message)\n  }\n}\n",[387,5249,5250,5263,5269,5290,5300,5317,5338,5343,5347,5351,5364,5389,5393,5397,5414,5423,5431,5440,5444],{"__ignoreMap":15},[390,5251,5252,5254,5257,5260],{"class":392,"line":393},[390,5253,1158],{"class":396},[390,5255,5256],{"class":396}," function",[390,5258,5259],{"class":421}," getData",[390,5261,5262],{"class":400},"() {\n",[390,5264,5265,5267],{"class":392,"line":173},[390,5266,517],{"class":396},[390,5268,437],{"class":400},[390,5270,5271,5273,5276,5278,5280,5282,5284,5287],{"class":392,"line":16},[390,5272,525],{"class":396},[390,5274,5275],{"class":464}," response",[390,5277,425],{"class":396},[390,5279,470],{"class":396},[390,5281,4166],{"class":421},[390,5283,449],{"class":400},[390,5285,5286],{"class":407},"'https:\u002F\u002Fexample.org\u002Fwebhook'",[390,5288,5289],{"class":400},", {\n",[390,5291,5292,5295,5298],{"class":392,"line":440},[390,5293,5294],{"class":400},"      method: ",[390,5296,5297],{"class":407},"'POST'",[390,5299,491],{"class":400},[390,5301,5302,5305,5308,5311,5314],{"class":392,"line":458},[390,5303,5304],{"class":400},"      headers: [ ",[390,5306,5307],{"class":407},"'Content-Type'",[390,5309,5310],{"class":400},": ",[390,5312,5313],{"class":407},"'application\u002Fjson'",[390,5315,5316],{"class":400}," ],\n",[390,5318,5319,5322,5324,5326,5329,5332,5335],{"class":392,"line":482},[390,5320,5321],{"class":400},"      body: ",[390,5323,4639],{"class":464},[390,5325,13],{"class":400},[390,5327,5328],{"class":421},"stringify",[390,5330,5331],{"class":400},"({ username: ",[390,5333,5334],{"class":407},"'example'",[390,5336,5337],{"class":400}," }),\n",[390,5339,5340],{"class":392,"line":494},[390,5341,5342],{"class":571},"      \u002F\u002F ...\n",[390,5344,5345],{"class":392,"line":500},[390,5346,668],{"class":400},[390,5348,5349],{"class":392,"line":514},[390,5350,413],{"emptyLinePlaceholder":24},[390,5352,5353,5356,5358,5361],{"class":392,"line":522},[390,5354,5355],{"class":396},"    if",[390,5357,3123],{"class":400},[390,5359,5360],{"class":396},"!",[390,5362,5363],{"class":400},"response.ok) {\n",[390,5365,5366,5368,5370,5372,5374,5377,5379,5381,5384,5387],{"class":392,"line":544},[390,5367,2481],{"class":396},[390,5369,613],{"class":396},[390,5371,2541],{"class":421},[390,5373,449],{"class":400},[390,5375,5376],{"class":407},"`Response status: ${",[390,5378,4457],{"class":400},[390,5380,13],{"class":407},[390,5382,5383],{"class":400},"status",[390,5385,5386],{"class":407},"}`",[390,5388,455],{"class":400},[390,5390,5391],{"class":392,"line":563},[390,5392,1889],{"class":400},[390,5394,5395],{"class":392,"line":568},[390,5396,413],{"emptyLinePlaceholder":24},[390,5398,5399,5401,5404,5406,5408,5410,5412],{"class":392,"line":575},[390,5400,525],{"class":396},[390,5402,5403],{"class":464}," json",[390,5405,425],{"class":396},[390,5407,470],{"class":396},[390,5409,4683],{"class":400},[390,5411,4195],{"class":421},[390,5413,541],{"class":400},[390,5415,5416,5418,5420],{"class":392,"line":603},[390,5417,702],{"class":400},[390,5419,446],{"class":421},[390,5421,5422],{"class":400},"(json)\n",[390,5424,5425,5427,5429],{"class":392,"line":608},[390,5426,717],{"class":400},[390,5428,720],{"class":396},[390,5430,4756],{"class":400},[390,5432,5433,5435,5437],{"class":392,"line":633},[390,5434,702],{"class":400},[390,5436,4494],{"class":421},[390,5438,5439],{"class":400},"(error.message)\n",[390,5441,5442],{"class":392,"line":646},[390,5443,749],{"class":400},[390,5445,5446],{"class":392,"line":654},[390,5447,755],{"class":400},[11,5449,5450,5451,1155,5455,1155,5457,828,5461,13],{},"You can check out more specific examples in our guides for ",[50,5452,5454],{"href":5453},"\u002Fguides\u002Fbluesky","Bluesky",[50,5456,146],{"href":161},[50,5458,5460],{"href":5459},"\u002Fguides\u002Fbaserow","Baserow",[50,5462,5464],{"href":5463},"\u002Fguides\u002Fpost-data-to-airtable","Airtable",[43,5466,5468],{"id":5467},"implementing-webhooks-in-python","Implementing webhooks in Python",[11,5470,5471,5472,5476],{},"Python is a very popular language for data science, and as such, you may find the need to trigger a webhook from your Python scripts to another application (did someone say ",[50,5473,5475],{"href":5474},"\u002Fdocs\u002Fdeveloper-hub\u002Fapi","axiom.ai's API","?). This can be done relatively simple within Python and has code that's simpler than JavaScript, see below:",[381,5478,5480],{"className":3199,"code":5479,"language":3201,"meta":15,"style":15},"import requests\n\nwebhook_url = 'https:\u002F\u002Fwww.example.com\u002Fwebhook'\n\ndata = {\n  'event': 'button_triggered',\n  'user_id': 12345\n}\n\nrequests.post(webhook_url, json=data)\n",[387,5481,5482,5487,5491,5496,5500,5505,5510,5515,5519,5523],{"__ignoreMap":15},[390,5483,5484],{"class":392,"line":393},[390,5485,5486],{},"import requests\n",[390,5488,5489],{"class":392,"line":173},[390,5490,413],{"emptyLinePlaceholder":24},[390,5492,5493],{"class":392,"line":16},[390,5494,5495],{},"webhook_url = 'https:\u002F\u002Fwww.example.com\u002Fwebhook'\n",[390,5497,5498],{"class":392,"line":440},[390,5499,413],{"emptyLinePlaceholder":24},[390,5501,5502],{"class":392,"line":458},[390,5503,5504],{},"data = {\n",[390,5506,5507],{"class":392,"line":482},[390,5508,5509],{},"  'event': 'button_triggered',\n",[390,5511,5512],{"class":392,"line":494},[390,5513,5514],{},"  'user_id': 12345\n",[390,5516,5517],{"class":392,"line":500},[390,5518,755],{},[390,5520,5521],{"class":392,"line":514},[390,5522,413],{"emptyLinePlaceholder":24},[390,5524,5525],{"class":392,"line":522},[390,5526,5527],{},"requests.post(webhook_url, json=data)\n",[11,5529,5530,5531,5533,5534,5537],{},"As yuo can see, the ",[387,5532,4219],{}," variable can be modified to use any data that you wish to include in your payload. This data will then be sent to the webhook defined in the ",[387,5535,5536],{},"webhook_url"," variable.",[43,5539,5541],{"id":5540},"testing-your-webhooks","Testing your webhooks",[11,5543,5544],{},"There are various tools available for testing webhooks to ensure that your implementation is working as expected, let's look at a couple of these tools:",[11,5546,5547,5550],{},[5176,5548,5549],{},"Postman:"," this tool allows you to send HTTP requests and observe their responses, this can be useful in ensuring that you are sending your payload to the correct endpoint and observe the status that you receive back. This can enable you to debug your POST requests and troubleshoot any issues that you are experiencing. It's free to use so well worth checking out.",[11,5552,5553,5556,5557,13],{},[5176,5554,5555],{},"Webhook.site:"," another free tool - Website.site allows you to test triggering of your webhooks. You can use this within your source application to observe what payload the destination application is receiving when you send the payload. This can be helpful for debugging the payload that you are sending. Learn more: ",[50,5558,5561],{"href":5559,"rel":5560},"https:\u002F\u002Fwebhook.site",[54],"https:\u002F\u002Fwebhook.site\u002F",[43,5563,5565],{"id":5564},"using-webhooks-in-axiomai","Using webhooks in axiom.ai",[11,5567,5568],{},"There are a few methods of sending data to a webhook in your automation using axiom.ai, such as using:",[249,5570,5571,5577],{},[252,5572,205,5573,331],{},[50,5574,5576],{"href":5575},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Ftrigger-webhook","Trigger webhook",[252,5578,205,5579,331],{},[50,5580,5113],{"href":5581},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fwrite-javascript",[11,5583,5584],{},"Both options allow you full control over the endpoint and the payload that contains your data - plus, both have the ability to use data tokens to automatically include data from your automations. This allows you to create a payload that contains information from your automation, such as scrape data, alerts and other notifications. The only benefit of using the JavaScript method is having more control over the HTTP status response that the webhook would receive - this does not come through the \"Trigger webhook\" step.",[11,5586,5587,5588,13],{},"You might also be interested in how you can use webhooks to send notifications from your automation: ",[50,5589,5591],{"href":5590},"\u002Fblog\u002Fenhance-your-automations-with-notifications","Enhance Your Automations with Notifications",[43,5593,166],{"id":165},[11,5595,5596],{},"Webhooks offer a powerful and efficient way to connect applications, enabling real-time data exchange and automated workflows. Their event-driven nature, combined with simplicity and flexibility, makes them a valuable tool for developers. By understanding the core concepts – event triggers, HTTP-based communication, and payload delivery – you can effectively leverage webhooks to streamline processes and create seamless integrations.",[11,5598,5599],{},"While we've explored basic implementations in JavaScript and Python, remember that the true potential of webhooks lies in their adaptability. From simple notifications to complex data pipelines, they provide a configurable bridge between services. However, it's crucial to be mindful of potential challenges, such as security risks and debugging complexities.",[11,5601,5602],{},"Whether you're looking to enhance your automations with real-time alerts or build sophisticated data-driven applications, webhooks offer a versatile solution. And with tools like axiom.ai, you can further simplify the process by easily incorporating webhooks into your automated workflows. As you continue to explore the world of webhooks, you'll discover their ability to unlock new possibilities and drive innovation in your projects.",[2666,5604,5605],{},"html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":5607},[5608,5609,5610,5611,5612,5613,5614],{"id":5168,"depth":173,"text":5169},{"id":5215,"depth":173,"text":5216},{"id":5225,"depth":173,"text":5226},{"id":5467,"depth":173,"text":5468},{"id":5540,"depth":173,"text":5541},{"id":5564,"depth":173,"text":5565},{"id":165,"depth":173,"text":166},"2025-05-13","Unlock seamless app integrations with webhooks. Explore how these powerful tools enable real-time communication, automate data flow, and connect your favourite applications effortlessly.",{"read":352,"type":387,"tool":5618,"category":5619,"tags":5620,"featuredimg":5624,"landingimg":5625,"summary":5616,"video":18,"metaTitle":5163},[354,2684],[2686],[5621,5622,5623],"webhooks","payload","post","\u002Fwebhook-post.webp","\u002Fwebhook-sq.webp","\u002Fblog\u002Fwebhooks-explained-build-reactive-automations-today",{"title":5163,"description":5616},"blog\u002Fwebhooks-explained-build-reactive-automations-today","KTnmnYjbrr3hswgbzEiiNsDrc_zTke1ccLxF8JkpeBk",{"id":5631,"title":5632,"author":23,"body":5633,"date":6208,"description":6209,"draft":181,"extension":19,"meta":6210,"navigation":24,"path":6220,"seo":6221,"stem":6222,"__hash__":6223},"blog\u002Fblog\u002Fcreating-a-no-code-auto-clicker.md","Creating a No-code Auto Clicker in Minutes",{"type":8,"value":5634,"toc":6195},[5635,5638,5642,5645,5649,5656,5669,5673,5679,5687,5692,5696,5704,5716,5719,5723,5733,5739,5798,5801,5855,5858,5907,5911,5914,5918,5921,5929,5932,6082,6087,6091,6094,6099,6107,6110,6113,6118,6121,6181,6183,6186,6192],[11,5636,5637],{},"Creating an auto clicker with axiom.ai can allow you to use the tool to automatically click a button, or element, multiple times. There are many reasons why you may wish to automate this - the primary reason being to avoid a repetitive strain injury! Automatic clickers can help solve a wide range of use cases with browser automation.",[43,5639,5641],{"id":5640},"what-is-an-auto-clicker-bot","What is an auto clicker bot?",[11,5643,5644],{},"Exactly what it says on the tin, an auto clicker bot is a piece of software that is designed to automatically perform a click action - in this article we are going to be concentrating on auto clicking within a browser environment. They are useful for applications where you may need to click multiple times where you may not wish to do this manually.",[43,5646,5648],{"id":5647},"how-to-create-an-auto-clicker-bot","How to create an auto clicker bot",[11,5650,5651,5652,5655],{},"To get started, we are going to be utilising ",[50,5653,210],{"href":3840,"rel":5654},[54]," to create a bot that can perform the clicks for us. If you haven't already signed up, you can do so to take advantage of the free trial that's on offer - from our testing for this article, we believe you should be able to create this auto clicker bot within the trial period.",[11,5657,5658,5659,5663,5664,5668],{},"Create a new automation, or navigate to the automation you want to add this functionality to. Ensure that there is a ",[50,5660,5662],{"href":5661},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fgo-to-page","Go to page"," step added to your automation to ensure that there are elements loaded to be able to click them. We will be using the ",[50,5665,5667],{"href":5666},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fclick-element","Click element"," step to perform the action on the webpage once it has loaded.",[818,5670,5672],{"id":5671},"using-the-jump-method","Using the 'Jump' method",[11,5674,5675,5676,5678],{},"The first step after the \"Go to page\" step that you will want to add is the ",[50,5677,5667],{"href":5666}," step, this step will allow you to click the element on the page itself. Add the \"Click element\" step and follow the instructions in the step to select the element that you wish to click.",[11,5680,5681,5682,5686],{},"Once the step has been set up as you require, add a ",[50,5683,5685],{"href":5684},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fjump-step","Jump to another step"," step to your automation, you'll want to set this up to \"jump back\" to the \"Click element\" step that you previously created. You'll be able to configure the maximum number of jumps that this step will perform - consider the impact on runtime this may have if set to a larger umber.",[11,5688,5689],{},[2786,5690,5691],{},"Note, if you change the order of steps in the future, you will need to update the \"Jump to another step\" step with the new step number.",[818,5693,5695],{"id":5694},"using-the-loop-method","Using the 'Loop' method",[11,5697,5698,5699,5703],{},"axiom.ai offers a ",[50,5700,5702],{"href":5701},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Floop","Loop through data"," step that is normally used to loop through rows of data within a data token - this allows you access to the individual rows of data. However, this can also be used to loop and perform repetitive tasks as a little \"hack\".",[11,5705,5706,5707,285,5711,5715],{},"To get started, create a Google Sheet and add content to the number of iterations that you would like to iterate through. For example, set rows A1-A10 with the numbers 1-10. The content in this case does not matter once the number of rows with content matches the number of iterations that you'd like to perform. Read this data into your automation using a ",[50,5708,5710],{"href":5709},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fread-data-from-a-google-sheet-step","Read data from a Google Sheet",[50,5712,5714],{"href":5713},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fread-data-from-excel","Read data from Excel"," step and setting the data token into the \"Loop through data\" step.",[11,5717,5718],{},"Inside of the \"Loop through data\" step you can then add the \"Click element\" step that will perform the click action.",[818,5720,5722],{"id":5721},"using-the-code-method","Using the 'Code' method",[11,5724,5725,5726,5728,5729,5732],{},"What! Code in a no-code article? I know, but hear me out. Using the built-in tools with axiom.ai it's quite easy to use a small about of code to achieve big things, like an auto clicker. This is an alternative option that would use the ",[50,5727,5113],{"href":5581}," step to write code that can be used to repeatedly click the button or element. You'll need to know the selector for the element for this method - see our article ",[50,5730,3366],{"href":5731},"\u002Fblog\u002Fbest-custom-css-selectors-for-web-scraping"," for some tips and tricks.",[11,5734,5735,5736,5738],{},"To get started, create a ",[387,5737,3120],{}," loop within your code, setting it tot he desired number of iterations, for example:",[381,5740,5742],{"className":383,"code":5741,"language":385,"meta":15,"style":15},"const iterations = 10;\n\nfor (var i = 0 ; i \u003C iterations ; i++) {\n    \u002F\u002F Click code\n}\n",[387,5743,5744,5759,5763,5789,5794],{"__ignoreMap":15},[390,5745,5746,5748,5751,5753,5756],{"class":392,"line":393},[390,5747,418],{"class":396},[390,5749,5750],{"class":464}," iterations",[390,5752,425],{"class":396},[390,5754,5755],{"class":464}," 10",[390,5757,5758],{"class":400},";\n",[390,5760,5761],{"class":392,"line":173},[390,5762,413],{"emptyLinePlaceholder":24},[390,5764,5765,5767,5769,5771,5773,5775,5777,5780,5782,5785,5787],{"class":392,"line":16},[390,5766,3120],{"class":396},[390,5768,3123],{"class":400},[390,5770,3126],{"class":396},[390,5772,3129],{"class":400},[390,5774,893],{"class":396},[390,5776,3134],{"class":464},[390,5778,5779],{"class":400}," ; i ",[390,5781,3140],{"class":396},[390,5783,5784],{"class":400}," iterations ; i",[390,5786,3152],{"class":396},[390,5788,3155],{"class":400},[390,5790,5791],{"class":392,"line":440},[390,5792,5793],{"class":571},"    \u002F\u002F Click code\n",[390,5795,5796],{"class":392,"line":458},[390,5797,755],{"class":400},[11,5799,5800],{},"From here, there are two options that you can take to click the element. Both are written in Javascript, however, one takes advantage of the Puppeteer library. The second option will require that you run your automation locally using the desktop application.",[381,5802,5804],{"className":383,"code":5803,"language":385,"meta":15,"style":15},"\u002F\u002F Pure JS\ndocument.getElementById(\"#id\").click();\n\n\u002F\u002F Puppeteer\nawait page.click('\u003Cselector>');\n",[387,5805,5806,5811,5830,5834,5839],{"__ignoreMap":15},[390,5807,5808],{"class":392,"line":393},[390,5809,5810],{"class":571},"\u002F\u002F Pure JS\n",[390,5812,5813,5816,5819,5821,5824,5826,5828],{"class":392,"line":173},[390,5814,5815],{"class":400},"document.",[390,5817,5818],{"class":421},"getElementById",[390,5820,449],{"class":400},[390,5822,5823],{"class":407},"\"#id\"",[390,5825,590],{"class":400},[390,5827,3163],{"class":421},[390,5829,4198],{"class":400},[390,5831,5832],{"class":392,"line":16},[390,5833,413],{"emptyLinePlaceholder":24},[390,5835,5836],{"class":392,"line":440},[390,5837,5838],{"class":571},"\u002F\u002F Puppeteer\n",[390,5840,5841,5844,5846,5848,5850,5853],{"class":392,"line":458},[390,5842,5843],{"class":396},"await",[390,5845,550],{"class":400},[390,5847,3163],{"class":421},[390,5849,449],{"class":400},[390,5851,5852],{"class":407},"'\u003Cselector>'",[390,5854,4174],{"class":400},[11,5856,5857],{},"Using Puppeteer has a few advantages, such as being able to click an element based on text or XPath selectors, for example:",[381,5859,5861],{"className":383,"code":5860,"language":385,"meta":15,"style":15},"\u002F\u002F Click based on text\nawait page.click(\"a:has-text('Buy now')\");\n\n\u002F\u002F Click based on XPath\nawait page.click('xpath=\u002F\u002Finput[@id=\"search-input\"]');\n",[387,5862,5863,5868,5883,5887,5892],{"__ignoreMap":15},[390,5864,5865],{"class":392,"line":393},[390,5866,5867],{"class":571},"\u002F\u002F Click based on text\n",[390,5869,5870,5872,5874,5876,5878,5881],{"class":392,"line":173},[390,5871,5843],{"class":396},[390,5873,550],{"class":400},[390,5875,3163],{"class":421},[390,5877,449],{"class":400},[390,5879,5880],{"class":407},"\"a:has-text('Buy now')\"",[390,5882,4174],{"class":400},[390,5884,5885],{"class":392,"line":16},[390,5886,413],{"emptyLinePlaceholder":24},[390,5888,5889],{"class":392,"line":440},[390,5890,5891],{"class":571},"\u002F\u002F Click based on XPath\n",[390,5893,5894,5896,5898,5900,5902,5905],{"class":392,"line":458},[390,5895,5843],{"class":396},[390,5897,550],{"class":400},[390,5899,3163],{"class":421},[390,5901,449],{"class":400},[390,5903,5904],{"class":407},"'xpath=\u002F\u002Finput[@id=\"search-input\"]'",[390,5906,4174],{"class":400},[43,5908,5910],{"id":5909},"additional-configuration","Additional configuration",[11,5912,5913],{},"These auto clickers on their own are great, however, there may be times when you need to make changes to them in order for them to meet your requirements.",[818,5915,5917],{"id":5916},"adding-delays","Adding delays",[11,5919,5920],{},"Delays can be added to all of the above methods, this can be useful if you are waiting for something else to happen, or if you are looking to avoid a site detecting your bot.",[11,5922,5923,5924,5928],{},"For the no-code options, simply add a ",[50,5925,5927],{"href":5926},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fwait","Wait"," step after your “Click element\" step. Configure this to your desired delay and this will cause a delay between clicks.",[11,5930,5931],{},"The coded option is a bit more involved and does require modification to the code, we’ve included a full example below:",[381,5933,5935],{"className":383,"code":5934,"language":385,"meta":15,"style":15},"var i = 1;          \u002F\u002F rather than 0\nconst delay = 3000;     \u002F\u002F milliseconds, change as required\nconst iterations = 10;\n\nfunction clickLoop() {\n    setTimeout(function() {\n        page.click('#example');\n        i++;\n\n        if (i \u003C iterations) {\n            clickLoop();\n        }\n    }, delay)\n};\n\nclickLoop();\n",[387,5936,5937,5954,5972,5984,5988,5998,6009,6023,6032,6036,6049,6056,6061,6066,6071,6075],{"__ignoreMap":15},[390,5938,5939,5941,5943,5945,5948,5951],{"class":392,"line":393},[390,5940,3126],{"class":396},[390,5942,3129],{"class":400},[390,5944,893],{"class":396},[390,5946,5947],{"class":464}," 1",[390,5949,5950],{"class":400},";          ",[390,5952,5953],{"class":571},"\u002F\u002F rather than 0\n",[390,5955,5956,5958,5961,5963,5966,5969],{"class":392,"line":173},[390,5957,418],{"class":396},[390,5959,5960],{"class":464}," delay",[390,5962,425],{"class":396},[390,5964,5965],{"class":464}," 3000",[390,5967,5968],{"class":400},";     ",[390,5970,5971],{"class":571},"\u002F\u002F milliseconds, change as required\n",[390,5973,5974,5976,5978,5980,5982],{"class":392,"line":16},[390,5975,418],{"class":396},[390,5977,5750],{"class":464},[390,5979,425],{"class":396},[390,5981,5755],{"class":464},[390,5983,5758],{"class":400},[390,5985,5986],{"class":392,"line":440},[390,5987,413],{"emptyLinePlaceholder":24},[390,5989,5990,5993,5996],{"class":392,"line":458},[390,5991,5992],{"class":396},"function",[390,5994,5995],{"class":421}," clickLoop",[390,5997,5262],{"class":400},[390,5999,6000,6003,6005,6007],{"class":392,"line":482},[390,6001,6002],{"class":421},"    setTimeout",[390,6004,449],{"class":400},[390,6006,5992],{"class":396},[390,6008,5262],{"class":400},[390,6010,6011,6014,6016,6018,6021],{"class":392,"line":494},[390,6012,6013],{"class":400},"        page.",[390,6015,3163],{"class":421},[390,6017,449],{"class":400},[390,6019,6020],{"class":407},"'#example'",[390,6022,4174],{"class":400},[390,6024,6025,6028,6030],{"class":392,"line":500},[390,6026,6027],{"class":400},"        i",[390,6029,3152],{"class":396},[390,6031,5758],{"class":400},[390,6033,6034],{"class":392,"line":514},[390,6035,413],{"emptyLinePlaceholder":24},[390,6037,6038,6041,6044,6046],{"class":392,"line":522},[390,6039,6040],{"class":396},"        if",[390,6042,6043],{"class":400}," (i ",[390,6045,3140],{"class":396},[390,6047,6048],{"class":400}," iterations) {\n",[390,6050,6051,6054],{"class":392,"line":544},[390,6052,6053],{"class":421},"            clickLoop",[390,6055,4198],{"class":400},[390,6057,6058],{"class":392,"line":563},[390,6059,6060],{"class":400},"        }\n",[390,6062,6063],{"class":392,"line":568},[390,6064,6065],{"class":400},"    }, delay)\n",[390,6067,6068],{"class":392,"line":575},[390,6069,6070],{"class":400},"};\n",[390,6072,6073],{"class":392,"line":603},[390,6074,413],{"emptyLinePlaceholder":24},[390,6076,6077,6080],{"class":392,"line":608},[390,6078,6079],{"class":421},"clickLoop",[390,6081,4198],{"class":400},[11,6083,6084],{},[2786,6085,6086],{},"Note, you may need to also add a “Wait” step after the “Write Javascript” step that equals the total time your loop will take. For the example above, we would add a wait of 31000 (delay * iterations + 1000 (to account for any loading)).",[818,6088,6090],{"id":6089},"exiting-a-loop","Exiting a loop",[11,6092,6093],{},"If you are using your automation to monitor a page for changes, there may not always be a need to complete the entire loop. These are various methods of exiting a loop early in this instance.",[11,6095,6096],{},[5176,6097,6098],{},"No-code options",[11,6100,6101,6102,6106],{},"Both options for the no-code methods will make use of ",[50,6103,6105],{"href":6104},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002F#control-flow","Conditional steps"," to determine when to leave the loop. We won’t be reviewing these steps in this article but we would recommend checking out the documentation on them.",[11,6108,6109],{},"When using the “jump” method, we’ll want to add a “Conditional step”, such as an “If\u002Felse” step, to your automation before the “Jump to another step” step. Configure this conditional step as you require, and then add another “Jump to another step” inside the conditional step - you’ll want to set this to “jump” to the step after the original “Jump to another step”.",[11,6111,6112],{},"For the “loop” method, as of the time of writing, we are developing an “End loop” step that can be inserted into a conditional step that can be used to exit your loop.",[11,6114,6115],{},[5176,6116,6117],{},"Code options",[11,6119,6120],{},"For the “code” method of looping, you can use an if statement within your loop, adding the break keyword will allow you to break out of the loop, for example:",[381,6122,6124],{"className":383,"code":6123,"language":385,"meta":15,"style":15},"for (var i = 0; i \u003C 10; i++) {\n    if (i === 10){\n        break;\n    }\n}\n",[387,6125,6126,6152,6166,6173,6177],{"__ignoreMap":15},[390,6127,6128,6130,6132,6134,6136,6138,6140,6142,6144,6146,6148,6150],{"class":392,"line":393},[390,6129,3120],{"class":396},[390,6131,3123],{"class":400},[390,6133,3126],{"class":396},[390,6135,3129],{"class":400},[390,6137,893],{"class":396},[390,6139,3134],{"class":464},[390,6141,3137],{"class":400},[390,6143,3140],{"class":396},[390,6145,5755],{"class":464},[390,6147,3149],{"class":400},[390,6149,3152],{"class":396},[390,6151,3155],{"class":400},[390,6153,6154,6156,6158,6161,6163],{"class":392,"line":173},[390,6155,5355],{"class":396},[390,6157,6043],{"class":400},[390,6159,6160],{"class":396},"===",[390,6162,5755],{"class":464},[390,6164,6165],{"class":400},"){\n",[390,6167,6168,6171],{"class":392,"line":16},[390,6169,6170],{"class":396},"        break",[390,6172,5758],{"class":400},[390,6174,6175],{"class":392,"line":440},[390,6176,1889],{"class":400},[390,6178,6179],{"class":392,"line":458},[390,6180,755],{"class":400},[43,6182,166],{"id":165},[11,6184,6185],{},"Performing repetitive tasks by hand can be time consuming, which is exactly where an auto clicker bot comes in. This can be used for a wide range of applications, such as automatically refreshing an element on a page to check if there are updates to the page that you are automating. The ease of development of these no-code auto clicker bots means that there is less friction when creating these, and the code option means that you can expand the functionality to ensure that it meets your requirements.",[11,6187,6188,6189,13],{},"We look forward to hearing from you on how you have used axiom.ai to create an auto clicker bot, feel free to join us over in our ",[50,6190,3304],{"href":3302,"rel":6191},[54],[2666,6193,6194],{},"html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}",{"title":15,"searchDepth":16,"depth":16,"links":6196},[6197,6198,6203,6207],{"id":5640,"depth":173,"text":5641},{"id":5647,"depth":173,"text":5648,"children":6199},[6200,6201,6202],{"id":5671,"depth":16,"text":5672},{"id":5694,"depth":16,"text":5695},{"id":5721,"depth":16,"text":5722},{"id":5909,"depth":173,"text":5910,"children":6204},[6205,6206],{"id":5916,"depth":16,"text":5917},{"id":6089,"depth":16,"text":6090},{"id":165,"depth":173,"text":166},"2025-05-08","Learn how to build a no-code auto clicker in minutes with axiom.ai—automate repetitive browser clicks, reduce strain, and streamline your workflow without writing a single line of code.",{"read":6211,"type":184,"tool":6212,"category":6213,"tags":6215,"featuredimg":6217,"landingimg":6218,"summary":6219,"video":18,"metaTitle":5632},"6 min read",[354],[6214],"Workflow automation",[5667,6216],"Clicker","\u002Fauto-clicker-post.webp","\u002Fauto-clicker-sq.webp","Learn more about how to get started with axiom.ai to create an auto clicker.","\u002Fblog\u002Fcreating-a-no-code-auto-clicker",{"title":5632,"description":6209},"blog\u002Fcreating-a-no-code-auto-clicker","RQ9_TWJTPwIIxq2wbymKdIqIxOaOGIP7tpkYJnXafv8",{"id":6225,"title":6226,"author":23,"body":6227,"date":6415,"description":6416,"draft":181,"extension":19,"meta":6417,"navigation":24,"path":6426,"seo":6427,"stem":6428,"__hash__":6429},"blog\u002Fblog\u002Fmaking-your-automations-mobile-with-apple-shortcuts.md","Making Your Automations Mobile with Apple Shortcuts",{"type":8,"value":6228,"toc":6401},[6229,6232,6235,6238,6241,6245,6248,6252,6255,6258,6262,6265,6291,6294,6299,6303,6311,6314,6319,6323,6326,6329,6343,6346,6354,6356,6359,6363,6366,6370,6380,6384,6387,6389,6392,6395],[11,6230,6231],{},"Apple Shortcuts is an application that comes pre-installed on most Apple devices and can provide a powerful solution for performing automations on your devices. It provides access to various system events that can be used to start a Shortcut, we will explore this later in this article.",[11,6233,6234],{},"One of the benefits of using Apple Shortcuts is that the shortcuts that you create will be available on any device that supports Shortcuts, and that you have signed into your iCloud account on. For instance, you can create a shortcut on your Mac that can then be run on your iPhone, or iPad, and vice versa.",[11,6236,6237],{},"Your Shortcuts live and run on your Apple device - which can be great from a privacy and security point of view as it means that none of the data that you are processing with it leaves your device. This is similar to when you run an axiom.ai automation on your computer, no information leaves your device unless you explicitly instruct it to.",[11,6239,6240],{},"As Shortcuts runs on your device, it can tie into the applications on your device - most applications provide steps to Shortcuts that can be used. For example, if you wanted to read data from the Apple Health application, you could do so with a Shortcut and then share this information to another application or service. Or, a Shortcut can be used to write data to another app, such as creating a new calendar event when the Shortcut is run.",[43,6242,6244],{"id":6243},"triggering-a-shortcut","Triggering a Shortcut",[11,6246,6247],{},"There are multiple methods that can be used to trigger your Shortcut that are provided by the Apple Shortcuts application, let’s explore them.",[818,6249,6251],{"id":6250},"manually","Manually",[11,6253,6254],{},"Shortcuts can be triggered manually by either navigating to the Shortcuts application and selecting the one you wish to trigger, adding it to your Control Centre, or even adding it to your homescreen as a widget. This can make triggering super convenient and available right at your fingertips. These can be triggered by Siri as a hands-free option by saying, \"Hey Siri,\" followed by the Shortcut name.",[11,6256,6257],{},"Some additional options are available to manually trigger a shortcut on macOS devices, such as adding the Shortcut to your Share Sheet, Quick Actions, or Menu Bar for easy access.",[818,6259,6261],{"id":6260},"automatically","Automatically",[11,6263,6264],{},"The other option is to have this automatically triggered. Apple provide a wide range of triggers that can be set up to automatically trigger your Shortcuts, such as:",[249,6266,6267,6270,6273,6276,6279,6282,6285,6288],{},[252,6268,6269],{},"Time of day",[252,6271,6272],{},"When your alarm stops",[252,6274,6275],{},"When you reach or leave a location",[252,6277,6278],{},"When you receive an email or message",[252,6280,6281],{},"When a device setting has been changed, such as connecting to a Wi-Fi network",[252,6283,6284],{},"When an application is opened",[252,6286,6287],{},"When your battery reaches a certain percentage",[252,6289,6290],{},"When a focus mode has been activated\u002Fdeactivated",[11,6292,6293],{},"These can be very convenient when requiring actions to be performed at certain times, or under certain conditions. For example, a personal favourite of mine is when I enable the “Sleep” focus mode on my iPhone, this will prompt me to turn off all of the lights in my home.",[11,6295,6296],{},[2786,6297,6298],{},"Note: as of time of writing, automatic triggers are currently only available on iOS and iPadOS devices.",[818,6300,6302],{"id":6301},"via-axiomai","Via axiom.ai",[11,6304,6305,6306,6310],{},"Even though axiom.ai does not officially integrate with Apple Shortcuts, there is a clever workaround that can be used to trigger your Shortcuts from an axiom.ai automation. To do this, create an automation in axiom.ai as normal. Once you are ready to trigger your Shortcut, add a ",[50,6307,6309],{"href":6308},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fsend-an-email","Send an email"," step to your automation and set this to send an email to an email address that you have set up on your Apple device.",[11,6312,6313],{},"Next, create a new Shortcut that triggers when an email has been received. We would recommend setting the “Subject Contains” field to match the subject that you have added in your axiom.ai automation. Click “Run” in axiom.ai to test your automation and confirm that the Shortcut has been triggered - you’ll also see the email in your Mail inbox to confirm receipt.",[11,6315,6316],{},[2786,6317,6318],{},"Note: your axiom.ai automation will continue to run after the email has been sent, it does not have oversight over the Shortcut on your device.",[43,6320,6322],{"id":6321},"triggering-an-axiomai-automation","Triggering an axiom.ai automation",[11,6324,6325],{},"While axiom.ai does not officially integrate with Shortcuts, as we do not provide an iOS application, you can still trigger your automations from Shortcuts. This means that you can automatically trigger your axiom.ai automations as part of your workflow.",[11,6327,6328],{},"To get started, create your Shortcut as normal, when you are ready to trigger your axiom.ai automation, add a “Get Contents of URL” step to your Shortcut. Change the method to “POST” and add two key\u002Fvalue pairs in the “headers” section of this step, as follows:",[249,6330,6331,6337],{},[252,6332,6333,6334],{},"key: ",[387,6335,6336],{},"\u003Cyour api key>",[252,6338,6339,6340],{},"name: ",[387,6341,6342],{},"\u003Cyour automation name>",[11,6344,6345],{},"Once you run your Shortcut, you will see your axiom.ai automation run a few moments later.",[11,6347,6348,6349,6353],{},"You can learn more about this in our ",[50,6350,6352],{"href":6351},"\u002Fguides\u002Fapple-shortcuts","How to use Apple Shortcuts to trigger an axiom.ai automation"," guide.",[43,6355,3264],{"id":3263},[11,6357,6358],{},"There are a bunch of use cases that can be used to take advantage of this combination of tools, such as:",[818,6360,6362],{"id":6361},"social-media-automation","Social media automation",[11,6364,6365],{},"Shortcuts have the ability to interact with most social media applications that are installed on your device. Automatically posting to your social media account via your Apple device can be even more useful when you trigger your Shortcut via axiom.ai. This can be a useful alternative to automating social media when using multiple accounts without relying on sessions or steps to log in and out of multiple accounts.",[818,6367,6369],{"id":6368},"workout-tracking","Workout tracking",[11,6371,6372,6373,6377,6378,13],{},"Your iPhone has the ability to trigger a Shortcut when a workout has been started or ended on your Apple Watch - this can be used to share information with your axiom.ai automation using the “Health” steps provided, this could be used to log this data in a ",[50,6374,6376],{"href":6375},"\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002Fgoogle-sheets","Google Sheet"," for further analysis, or other services such as ",[50,6379,146],{"href":161},[818,6381,6383],{"id":6382},"credit-card-transactions","Credit card transactions",[11,6385,6386],{},"Transactions can be used to automate the triggering of Shortcuts - these will be triggered when an Apple Pay linked card has been used to make a purchase. You can configure this based on what type of card you would like notifications about, and if you only want certain types of transactions to trigger the Shortcut. This can be useful to log Apple Pay transactions for later analysis on spending.",[43,6388,166],{"id":165},[11,6390,6391],{},"Combining the power of Apple Shortcuts with the power of axiom.ai allows you to create a chain of automation that can be triggered from your Apple devices, or can trigger events to occur on your Apple devices.",[11,6393,6394],{},"With Apple Shortcuts' ability to integrate with system applications, and downloaded applications, on your device, this gives you an opportunity to expand the functionality offered to your automations. From automatically triggering an axiom.ai automation based on a change in device settings via Shortcuts, to triggering a Shortcut based on criteria set within your axiom.ai automation, the possibilities are endless.",[11,6396,6397,6398,13],{},"We’re excited to hear about what you build with Apple Shortcuts and axiom.ai, let us know over in our ",[50,6399,3304],{"href":3302,"rel":6400},[54],{"title":15,"searchDepth":16,"depth":16,"links":6402},[6403,6408,6409,6414],{"id":6243,"depth":173,"text":6244,"children":6404},[6405,6406,6407],{"id":6250,"depth":16,"text":6251},{"id":6260,"depth":16,"text":6261},{"id":6301,"depth":16,"text":6302},{"id":6321,"depth":173,"text":6322},{"id":3263,"depth":173,"text":3264,"children":6410},[6411,6412,6413],{"id":6361,"depth":16,"text":6362},{"id":6368,"depth":16,"text":6369},{"id":6382,"depth":16,"text":6383},{"id":165,"depth":173,"text":166},"2025-05-06","Learn how to harness Apple Shortcuts to make your axiom.ai automations more mobile, trigger them from anywhere with a shortcut.",{"read":2682,"type":184,"tool":6418,"category":6419,"tags":6420,"location":191,"featuredimg":6424,"landingimg":6425,"summary":6416},[354],[6214],[6421,6422,6423],"shortcuts","apple","trigger","\u002Fappleshortucts-post.webp","\u002Fappleshortucts-sq.webp","\u002Fblog\u002Fmaking-your-automations-mobile-with-apple-shortcuts",{"title":6226,"description":6416},"blog\u002Fmaking-your-automations-mobile-with-apple-shortcuts","yxI3RShCKqKOPeYdP-2MKxqyz12XGCYBk6smt9XH768",{"id":6431,"title":6432,"author":23,"body":6433,"date":6589,"description":6590,"draft":181,"extension":19,"meta":6591,"navigation":24,"path":6599,"seo":6600,"stem":6601,"__hash__":6602},"blog\u002Fblog\u002Fcombining-the-power-of-apis-with-browser-automation.md","Combining the Power of APIs with Browser Automation",{"type":8,"value":6434,"toc":6579},[6435,6438,6441,6444,6447,6451,6454,6457,6460,6464,6467,6474,6477,6485,6488,6492,6503,6506,6510,6513,6530,6534,6542,6546,6564,6567,6569,6572],[11,6436,6437],{},"Before we dive into how combining the power of APIs with browser automation can upgrade your workflows, let’s dive into what an API is, and what browser automation is.",[11,6439,6440],{},"An API (Application Programming Interface) is used to specify how applications interact with each other - it’s essentially a contract that outlines the requests, responses, and data formats that different applications can use to communicate. For example, when you use your axiom.ai account, we use our API to interact with our database to retrieve your automations and account information.",[11,6442,6443],{},"Browser automation is the process of using software tools to control and interact with web browsers automatically. Instead of manually navigating websites, filling out forms, clicking buttons, and extracting data, these tasks are performed by scripts or specialised software. They’re great at simulating user interactions with the browser.",[11,6445,6446],{},"When you combine these two to create workflows, you can unlock the ability to create dynamic workflows that can be used for a wider range of tasks. For example, if you use an API to trigger and send data to an automation that carries out a browser automation task, you can have this automation behave differently depending on what data is sent.",[43,6448,6450],{"id":6449},"more-about-apis","More about APIs",[11,6452,6453],{},"You likely use tens of APIs a day and don’t even realise it - they power everything from weather apps, messaging services, social media, smart home products, and so much more. Some services even rely on multiple APIs - when you hit “Run” on an axiom.ai automation, our API is triggered to run the automation and it may then use the Google Sheets API to interact with sheets, if you have the steps in your automation.",[11,6455,6456],{},"They can be used for a wide range of applications, such as retrieving or storing data, triggering tasks, or sending analytics data for the application running the code that interacts with the API.",[11,6458,6459],{},"In all of these cases, the API has laid out a contract that the calling application is required to adhere to to be able to interact with the API. This contract often lays out the HTTP method that is required to be used, as well as the attributes that the API expects, such as an API key, or parameters.",[43,6461,6463],{"id":6462},"the-axiomai-api","The axiom.ai API",[11,6465,6466],{},"The axiom.ai API can be used for a variety of tasks with your automations, the most common use of this would be to trigger your automations and send information to them. Triggering your automations via the API allows you to send dynamic data to your automations that allow for more dynamic use cases. The axiom.ai API allows you to include your automations as part of a larger workflow within your organisation, or personal projects.",[11,6468,6469,6470,6473],{},"We’d recommend checking out our ",[50,6471,6472],{"href":5474},"API"," documentation for more details on getting started.",[11,6475,6476],{},"Currently, the API supports the following functions:",[249,6478,6479,6482],{},[252,6480,6481],{},"Trigger an automation",[252,6483,6484],{},"Check status",[11,6486,6487],{},"A future update to the API is planned and will introduce more features for your automations.",[43,6489,6491],{"id":6490},"combining-an-api-with-browser-automation","Combining an API with browser automation",[11,6493,6494,6495,828,6497,6499,6500,13],{},"A great example of building a dynamic automation comes in the form of a web form filling automation - dynamic data can be sent to your automation over the API, triggering the automation and navigating to and filling in the web form that you are looking to automate. As part of a larger system, you may have a system that has the ability to trigger a webhook when a change occurs which would allow you to fully automate the process. ",[50,6496,5464],{"href":5463},[50,6498,146],{"href":161}," are examples of services that can do this. A library is also available for automatically triggering your automations from your website, this is not affiliated with axiom.ai but is available on ",[390,6501,6502],{},"npm",[11,6504,6505],{},"Let’s explore some possible use cases.",[818,6507,6509],{"id":6508},"cicd","CI\u002FCD",[11,6511,6512],{},"Continuous integration and\u002For continuous deployment is a set of practices used by development and devops teams to automate the software development lifecycle, aiming for faster and more reliable software releases. Teams will create a “pipeline” that is a set of automated tasks that are required to be performed within the lifecycle of an application - for example, building the project from the source repository.",[11,6514,6515,6516,6519,6520,6522,6523,285,6525,6529],{},"As part of the pipeline, you may wish to perform browser automation, or interact with APIs that may not be supported by the build system that you are utilising. One alternative for this would be to use the axiom.ai API to trigger an automation that can perform additional tasks - for example, triggering a ",[50,6517,6518],{"href":141},"Zapier Zap",", notifying your team via ",[50,6521,103],{"href":102},", creating a new entry in ",[50,6524,146],{"href":161},[50,6526,6528],{"href":6527},"\u002Fguides\u002Ftadabase","Tadabase"," and much more.",[818,6531,6533],{"id":6532},"data-storage","Data storage",[11,6535,6536,6537,285,6539,6541],{},"Building out applications that can handle the storage of data in services such as ",[50,6538,146],{"href":161},[50,6540,6528],{"href":6527}," can be difficult when you are building browser automation. Combining the API and browser automation means you can store any data that you need to perform your automation, or any data that is produced during the automation - for example, data that you have scraped from the page.",[818,6543,6545],{"id":6544},"social-media","Social media",[11,6547,6548,6549,6551,6552,285,6554,6558,6559,6563],{},"Most social media platforms offer APIs that allow you to retrieve insights from your profiles, or automatically post. For example, ",[50,6550,5454],{"href":5453}," provide a very easy to use API that can be used to post to your profile - this can be very useful for taking data from a spreadsheet, such as ",[50,6553,284],{"href":6375},[50,6555,6557],{"href":6556},"\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002Fexcel","Excel"," and pushing this to your account, combine this with axiom.ai ",[50,6560,6562],{"href":6561},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsettings\u002Frun-options\u002Fschedule","scheduling"," and you can have a fully automated social media presence that you can create a backlog for.",[11,6565,6566],{},"Certain CRMs offer an API, this could be utilised within your automations to draw on data that you have about a person before contacting them. For example, you could have a link to their LinkedIn and their name, you could then input this into your browser automation to automatically send them a message like, “Hey Karl, I see you’re still with axiom.ai. How are you finding it there? Do you need any automation solutions?”.",[43,6568,166],{"id":165},[11,6570,6571],{},"Combining APIs and browser automation gives you a great opportunity to add both into larger workflows - leveraging the power of APIs to perform more complex tasks within the context of a browser. This provides you with the opportunity to add additional functionality to your browser automation, or to other tools that you already use. It can be useful for CI\u002FCD, managing data storage and automating social media accounts to level up your marketing or scheduling.",[11,6573,6574,6575,6578],{},"We would love to hear how you use APIs and browser automation (even if it’s not with axiom.ai!) - feel free to hop into our ",[50,6576,3304],{"href":3302,"rel":6577},[54]," and share your ideas!",{"title":15,"searchDepth":16,"depth":16,"links":6580},[6581,6582,6583,6588],{"id":6449,"depth":173,"text":6450},{"id":6462,"depth":173,"text":6463},{"id":6490,"depth":173,"text":6491,"children":6584},[6585,6586,6587],{"id":6508,"depth":16,"text":6509},{"id":6532,"depth":16,"text":6533},{"id":6544,"depth":16,"text":6545},{"id":165,"depth":173,"text":166},"2025-05-02","Learn how to leverage the power of APIs in your browser automations to level up your data storage, social media and CI\u002FCD pipelines",{"read":352,"type":387,"tool":6592,"category":6593,"tags":6594,"featuredimg":6597,"landingimg":6598,"summary":6590,"video":18,"metaTitle":6432},[354],[2686],[2690,6595,6596],"api","Axiom API","\u002Fapi-browser-post.webp","\u002Fapi-browser-sq.webp","\u002Fblog\u002Fcombining-the-power-of-apis-with-browser-automation",{"title":6432,"description":6590},"blog\u002Fcombining-the-power-of-apis-with-browser-automation","Aa5VxfOOAONSucX3D5kNDaulAz6RPZQ_rcGP-xk7t3M",{"id":6604,"title":6605,"author":23,"body":6606,"date":7366,"description":7367,"draft":181,"extension":19,"meta":7368,"navigation":24,"path":7376,"seo":7377,"stem":7378,"__hash__":7379},"blog\u002Fblog\u002Fhow-axiom-ai-helper-was-created.md","How axiom.ai Helper was created",{"type":8,"value":6607,"toc":7352},[6608,6628,6631,6645,6652,6662,6666,6675,6680,6689,6693,6696,6704,6715,6719,6726,6729,6774,6781,6785,6790,6838,6841,6890,6938,6941,6998,7002,7007,7037,7039,7087,7130,7132,7186,7190,7196,7201,7208,7212,7215,7219,7222,7268,7274,7279,7285,7306,7310,7313,7324,7327,7329,7332,7335,7341,7349],[11,6609,6610,6611,6613,6614,1155,6616,1155,6618,1155,6622,828,6624,13],{},"The axiom.ai ",[50,6612,6472],{"href":5474}," can be super powerful when used to trigger your automations, pass data to them, and check the status of the automations. The API has a wide variety of users, such as being used to trigger your automations from third-party services such as ",[50,6615,114],{"href":141},[50,6617,69],{"href":84},[50,6619,6621],{"href":6620},"\u002Fguides\u002Fcurl","cURL",[50,6623,89],{"href":109},[50,6625,6627],{"href":6626},"\u002Fguides","much more",[11,6629,6630],{},"When considering different methods of using the API an idea came up regarding the use of axiom.ai as 'middleware' - a piece of software that can be used to connect two other services. The initial idea of this was using axiom.ai to capture form data from a website and pass it onto your automation to do one of the following operations:",[312,6632,6633,6636,6639,6642],{},[252,6634,6635],{},"Send an email notification containing the form data.",[252,6637,6638],{},"Logging the form data in a Google Sheet.",[252,6640,6641],{},"Performing operations on the data.",[252,6643,6644],{},"Pass the data onto another service.",[11,6646,6647,6648,6651],{},"Or all of the above - all of these options could be added into a single automation, meaning that with a single function call to the ",[387,6649,6650],{},"axiom-ai-helper"," package you could have the data propagate throughout your system.",[791,6653,6654],{},[11,6655,6656,6657,13],{},"💡 Note, this package was created by the author, an employee of axiom.ai, but is not an official product of axiom.ai. Issues, questions and feature requests should be directed to the author through the repository: ",[50,6658,6661],{"href":6659,"rel":6660},"https:\u002F\u002Fwww.npmjs.com\u002Fpackage\u002Faxiom-ai-helper",[54],"npmjs.com",[43,6663,6665],{"id":6664},"installing-the-package","Installing the package",[11,6667,6668,6670,6671,6674],{},[387,6669,6650],{}," is available to download on ",[50,6672,6661],{"href":6659,"rel":6673},[54],". To install, navigate to your project and install the package using the following command:",[11,6676,6677],{},[387,6678,6679],{},"npm install axiom-ai-helper",[11,6681,6682,6683,828,6685,6688],{},"You can then import the required functions individually, at time of writing, the ",[387,6684,6423],{},[387,6686,6687],{},"checkStatus"," functions are available - there are plans to continue development if new API features become available in the future.",[43,6690,6692],{"id":6691},"usage","Usage",[11,6694,6695],{},"I wanted to make this package as easy as possible to use, and as such, only included the standard requirements of the axiom.ai API. You'll need the following to be able to use the package, all can be acquired from axiom.ai:",[312,6697,6698,6701],{},[252,6699,6700],{},"An account",[252,6702,6703],{},"An API key",[11,6705,6706,6707,6711,6712],{},"You'll also need to have an automation to use with your code - for triggering automations, you'll need to start your automation with a ",[50,6708,6710],{"href":6709},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Freceive-data-from-another-app","Receive data from another app"," step. ",[2786,6713,6714],{},"Tip: ensure that you have the 'test data' filled out, it'll help with testing and debugging, I promise!",[43,6716,6718],{"id":6717},"functions","Functions",[11,6720,6721,6722,13],{},"The package is built using JavaScript and is based on simple HTTP requests, this code is open source and is available on Github: ",[50,6723,6650],{"href":6724,"rel":6725},"https:\u002F\u002Fgithub.com\u002FKarljoones\u002Faxiom-ai-helper",[54],[11,6727,6728],{},"The functions directly relate to the options that are available within the axiom.ai API.",[6730,6731,6732,6748],"table",{},[6733,6734,6735],"thead",{},[6736,6737,6738,6742,6745],"tr",{},[6739,6740,6741],"th",{},"Function",[6739,6743,6744],{},"Signature",[6739,6746,6747],{},"Endpoint",[6749,6750,6751,6763],"tbody",{},[6736,6752,6753,6757,6760],{},[6754,6755,6756],"td",{},"Trigger",[6754,6758,6759],{},"trigger(name, data, key)",[6754,6761,6762],{},"\u002Ftrigger",[6736,6764,6765,6768,6771],{},[6754,6766,6767],{},"Status",[6754,6769,6770],{},"checkStatus(name, key)",[6754,6772,6773],{},"\u002Fcheck-status",[11,6775,6776,6777,6780],{},"I'd recommend storing the API key in an ",[387,6778,6779],{},".env"," file to safeguard it and ensure that it's not uploaded to a public repository.",[818,6782,6784],{"id":6783},"triggering-your-automations","Triggering your automations",[11,6786,205,6787,6789],{},[387,6788,6423],{}," function can be used to trigger an automation that you have previously build in axiom.ai, this accepts 3 parameters:",[6730,6791,6792,6805],{},[6733,6793,6794],{},[6736,6795,6796,6799,6802],{},[6739,6797,6798],{},"Name",[6739,6800,6801],{},"Details",[6739,6803,6804],{},"Optional",[6749,6806,6807,6818,6828],{},[6736,6808,6809,6812,6815],{},[6754,6810,6811],{},"name",[6754,6813,6814],{},"Name of the automation",[6754,6816,6817],{},"No",[6736,6819,6820,6823,6826],{},[6754,6821,6822],{},"key",[6754,6824,6825],{},"API key",[6754,6827,6817],{},[6736,6829,6830,6832,6835],{},[6754,6831,4219],{},[6754,6833,6834],{},"Data to send to the automation, formatted as an array of arrays",[6754,6836,6837],{},"Yes",[11,6839,6840],{},"This will return an object containing 3 values that have been returned from the API, see below:",[381,6842,6845],{"className":6843,"code":6844,"language":4195,"meta":15,"style":15},"language-json shiki shiki-themes github-light-default github-dark-default","{\n    status: \"\",\n    link: \"\",\n    message: \"\"\n}\n",[387,6846,6847,6852,6865,6876,6886],{"__ignoreMap":15},[390,6848,6849],{"class":392,"line":393},[390,6850,6851],{"class":400},"{\n",[390,6853,6854,6858,6860,6863],{"class":392,"line":173},[390,6855,6857],{"class":6856},"sZcZs","    status",[390,6859,5310],{"class":400},[390,6861,6862],{"class":407},"\"\"",[390,6864,491],{"class":400},[390,6866,6867,6870,6872,6874],{"class":392,"line":16},[390,6868,6869],{"class":6856},"    link",[390,6871,5310],{"class":400},[390,6873,6862],{"class":407},[390,6875,491],{"class":400},[390,6877,6878,6881,6883],{"class":392,"line":440},[390,6879,6880],{"class":6856},"    message",[390,6882,5310],{"class":400},[390,6884,6885],{"class":407},"\"\"\n",[390,6887,6888],{"class":392,"line":458},[390,6889,755],{"class":400},[6730,6891,6892,6904],{},[6733,6893,6894],{},[6736,6895,6896,6899,6901],{},[6739,6897,6898],{},"Key",[6739,6900,6801],{},[6739,6902,6903],{},"Values",[6749,6905,6906,6916,6927],{},[6736,6907,6908,6910,6913],{},[6754,6909,5383],{},[6754,6911,6912],{},"The status of your automation",[6754,6914,6915],{},"\"success\"\"error\"",[6736,6917,6918,6921,6924],{},[6754,6919,6920],{},"link",[6754,6922,6923],{},"A link to the running automation",[6754,6925,6926],{},"\"{link}\"\u002Fnull",[6736,6928,6929,6932,6935],{},[6754,6930,6931],{},"message",[6754,6933,6934],{},"Any error messages",[6754,6936,6937],{},"\"{error}\"\u002F{\"Automation successfully triggered\"}",[11,6939,6940],{},"An example of running this is included below:",[381,6942,6944],{"className":383,"code":6943,"language":385,"meta":15,"style":15},"import { trigger } from 'axiom-ai-helper';\n\n...\n\nconst result = await trigger(name, key, data);\nconsole.log(result.message);\n",[387,6945,6946,6960,6964,6968,6972,6989],{"__ignoreMap":15},[390,6947,6948,6950,6953,6955,6958],{"class":392,"line":393},[390,6949,397],{"class":396},[390,6951,6952],{"class":400}," { trigger } ",[390,6954,404],{"class":396},[390,6956,6957],{"class":407}," 'axiom-ai-helper'",[390,6959,5758],{"class":400},[390,6961,6962],{"class":392,"line":173},[390,6963,413],{"emptyLinePlaceholder":24},[390,6965,6966],{"class":392,"line":16},[390,6967,3724],{"class":396},[390,6969,6970],{"class":392,"line":440},[390,6971,413],{"emptyLinePlaceholder":24},[390,6973,6974,6976,6979,6981,6983,6986],{"class":392,"line":458},[390,6975,418],{"class":396},[390,6977,6978],{"class":464}," result",[390,6980,425],{"class":396},[390,6982,470],{"class":396},[390,6984,6985],{"class":421}," trigger",[390,6987,6988],{"class":400},"(name, key, data);\n",[390,6990,6991,6993,6995],{"class":392,"line":482},[390,6992,4836],{"class":400},[390,6994,446],{"class":421},[390,6996,6997],{"class":400},"(result.message);\n",[818,6999,7001],{"id":7000},"checking-the-status-of-your-automations","Checking the status of your automations",[11,7003,205,7004,7006],{},[387,7005,6687],{}," function can be used to check the status of an automation that you have previously run in axiom.ai, this accepts 2 parameters:",[6730,7008,7009,7019],{},[6733,7010,7011],{},[6736,7012,7013,7015,7017],{},[6739,7014,6798],{},[6739,7016,6801],{},[6739,7018,6804],{},[6749,7020,7021,7029],{},[6736,7022,7023,7025,7027],{},[6754,7024,6811],{},[6754,7026,6814],{},[6754,7028,6817],{},[6736,7030,7031,7033,7035],{},[6754,7032,6822],{},[6754,7034,6825],{},[6754,7036,6817],{},[11,7038,6840],{},[381,7040,7042],{"className":6843,"code":7041,"language":4195,"meta":15,"style":15},"{\n    status: \"\",\n    message: \"\",\n    data: {\"google-sheet-data\": [[],[]]}\n}\n",[387,7043,7044,7048,7058,7068,7083],{"__ignoreMap":15},[390,7045,7046],{"class":392,"line":393},[390,7047,6851],{"class":400},[390,7049,7050,7052,7054,7056],{"class":392,"line":173},[390,7051,6857],{"class":6856},[390,7053,5310],{"class":400},[390,7055,6862],{"class":407},[390,7057,491],{"class":400},[390,7059,7060,7062,7064,7066],{"class":392,"line":16},[390,7061,6880],{"class":6856},[390,7063,5310],{"class":400},[390,7065,6862],{"class":407},[390,7067,491],{"class":400},[390,7069,7070,7073,7076,7080],{"class":392,"line":440},[390,7071,7072],{"class":6856},"    data",[390,7074,7075],{"class":400},": {",[390,7077,7079],{"class":7078},"sjgCt","\"google-sheet-data\"",[390,7081,7082],{"class":400},": [[],[]]}\n",[390,7084,7085],{"class":392,"line":458},[390,7086,755],{"class":400},[6730,7088,7089,7099],{},[6733,7090,7091],{},[6736,7092,7093,7095,7097],{},[6739,7094,6898],{},[6739,7096,6801],{},[6739,7098,6903],{},[6749,7100,7101,7109,7117],{},[6736,7102,7103,7105,7107],{},[6754,7104,5383],{},[6754,7106,6912],{},[6754,7108,6915],{},[6736,7110,7111,7113,7115],{},[6754,7112,6931],{},[6754,7114,6934],{},[6754,7116,6937],{},[6736,7118,7119,7121,7124],{},[6754,7120,4219],{},[6754,7122,7123],{},"Data written to a Google Sheet",[6754,7125,7126,7127,7129],{},"{\"google-sheet-data: [[],",[390,7128],{},"]\"}\u002Fnull",[11,7131,6940],{},[381,7133,7135],{"className":383,"code":7134,"language":385,"meta":15,"style":15},"import { checkStatus } from 'axiom-ai-helper';\n\n...\n\nconst result = await checkStatus(name, key);\nconsole.log(result.message);\n",[387,7136,7137,7150,7154,7158,7162,7178],{"__ignoreMap":15},[390,7138,7139,7141,7144,7146,7148],{"class":392,"line":393},[390,7140,397],{"class":396},[390,7142,7143],{"class":400}," { checkStatus } ",[390,7145,404],{"class":396},[390,7147,6957],{"class":407},[390,7149,5758],{"class":400},[390,7151,7152],{"class":392,"line":173},[390,7153,413],{"emptyLinePlaceholder":24},[390,7155,7156],{"class":392,"line":16},[390,7157,3724],{"class":396},[390,7159,7160],{"class":392,"line":440},[390,7161,413],{"emptyLinePlaceholder":24},[390,7163,7164,7166,7168,7170,7172,7175],{"class":392,"line":458},[390,7165,418],{"class":396},[390,7167,6978],{"class":464},[390,7169,425],{"class":396},[390,7171,470],{"class":396},[390,7173,7174],{"class":421}," checkStatus",[390,7176,7177],{"class":400},"(name, key);\n",[390,7179,7180,7182,7184],{"class":392,"line":482},[390,7181,4836],{"class":400},[390,7183,446],{"class":421},[390,7185,6997],{"class":400},[43,7187,7189],{"id":7188},"using-the-data-in-axiomai","Using the data in axiom.ai",[11,7191,7192,7193,7195],{},"To trigger an automation, you'll need to have an automation created. Once this has been created, set a ",[50,7194,6710],{"href":6709}," step as the first step, this will allow the automation to be able to receive the webhook that we are sending. In this step, ensure that you have the \"Test data\" filled in with the same format that you expect to receive the data, this will help with testing. For example:",[11,7197,7198],{},[387,7199,7200],{},"name, email",[11,7202,7203,7204,7207],{},"Once this has been done, you can then access the data that you send to the automation using the ",[387,7205,7206],{},"webhook-data"," data token.",[43,7209,7211],{"id":7210},"potential-use-cases","Potential use cases",[11,7213,7214],{},"There are various use cases that you could make use of while using the axiom.ai API, such as:",[818,7216,7218],{"id":7217},"contact-forms","Contact forms",[11,7220,7221],{},"Using the package to send along the data from a webform can allow you to build completely custom contact or feedback forms. You could use the following code to send along the data:",[381,7223,7225],{"className":383,"code":7224,"language":385,"meta":15,"style":15},"const result = await trigger(\"Contact form\", \"KEY\", [[\"Karl Jones\", \"example@example.com\", \"Hi, I'm looking to enquire about a recent order.\"]]);\n",[387,7226,7227],{"__ignoreMap":15},[390,7228,7229,7231,7233,7235,7237,7239,7241,7244,7246,7249,7252,7255,7257,7260,7262,7265],{"class":392,"line":393},[390,7230,418],{"class":396},[390,7232,6978],{"class":464},[390,7234,425],{"class":396},[390,7236,470],{"class":396},[390,7238,6985],{"class":421},[390,7240,449],{"class":400},[390,7242,7243],{"class":407},"\"Contact form\"",[390,7245,1155],{"class":400},[390,7247,7248],{"class":407},"\"KEY\"",[390,7250,7251],{"class":400},", [[",[390,7253,7254],{"class":407},"\"Karl Jones\"",[390,7256,1155],{"class":400},[390,7258,7259],{"class":407},"\"example@example.com\"",[390,7261,1155],{"class":400},[390,7263,7264],{"class":407},"\"Hi, I'm looking to enquire about a recent order.\"",[390,7266,7267],{"class":400},"]]);\n",[11,7269,7270,7271,7273],{},"Jump into the axiom.ai extension and create a new automation called \"Contact form\". As the first step, add a ",[50,7272,6710],{"href":15}," step, with the following test data set, this will help with creating your automation:",[11,7275,7276],{},[387,7277,7278],{},"name, email, message",[11,7280,7281,7282,7284],{},"You can use this data by accessing the ",[387,7283,7206],{}," data token. You could then:",[249,7286,7287,7293,7299],{},[252,7288,7289,7290,7292],{},"Use a ",[50,7291,6309],{"href":6308}," step to email this data to your team.",[252,7294,7295,7296,331],{},"Store using the ",[50,7297,330],{"href":7298},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fwrite-data-to-a-google-sheet-step",[252,7300,7301,7302,331],{},"Automatically craft a response using the ",[50,7303,7305],{"href":7304},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fgenerate-text-with-chatgpt","Generate text with ChatGPT",[818,7307,7309],{"id":7308},"automating-actions-or-track-errors-in-your-web-app","Automating actions or track errors in your web app",[11,7311,7312],{},"Whether it's a standard web app or a mobile web app, you can use the API to automation actions or track errors within your web app. This can be helpful for various use cases, such as:",[249,7314,7315,7318,7321],{},[252,7316,7317],{},"Tracking newsletter sign ups.",[252,7319,7320],{},"Logging errors into a Google Sheet.",[252,7322,7323],{},"Logging attributes about users, such as browser, or language.",[11,7325,7326],{},"Other tools do offer these features, however, this may be worth a look before committing to a larger, and often more expensive, option.",[43,7328,166],{"id":165},[11,7330,7331],{},"Implementing the API into your projects is relatively simple, and if you have the skills or time you likely won't need to use this library - this is purely designed to be a time saver to ensure that you are meeting the requirements of the API without having to write the code yourself.",[11,7333,7334],{},"Future plans for this library will ensure that this will be kept updated with the latest developments in the axiom.ai API.",[11,7336,7337,7338],{},"You can get started with axiom-ai-helper over on npm: ",[50,7339,6659],{"href":6659,"rel":7340},[54],[791,7342,7343],{},[11,7344,7345,7346,13],{},"💡 Reminder, this package was created by the author, an employee of axiom.ai, but is not an official product of axiom.ai. Issues, questions and feature requests should be directed to the author through the repository: ",[50,7347,6661],{"href":6659,"rel":7348},[54],[2666,7350,7351],{},"html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sZcZs, html code.shiki .sZcZs{--shiki-default:#82071E;--shiki-default-font-style:italic;--shiki-dark:#FFA198;--shiki-dark-font-style:italic}html pre.shiki code .sjgCt, html code.shiki .sjgCt{--shiki-default:#116329;--shiki-dark:#7EE787}",{"title":15,"searchDepth":16,"depth":16,"links":7353},[7354,7355,7356,7360,7361,7365],{"id":6664,"depth":173,"text":6665},{"id":6691,"depth":173,"text":6692},{"id":6717,"depth":173,"text":6718,"children":7357},[7358,7359],{"id":6783,"depth":16,"text":6784},{"id":7000,"depth":16,"text":7001},{"id":7188,"depth":173,"text":7189},{"id":7210,"depth":173,"text":7211,"children":7362},[7363,7364],{"id":7217,"depth":16,"text":7218},{"id":7308,"depth":16,"text":7309},{"id":165,"depth":173,"text":166},"2025-04-29","Learn more about how the axiom-ai-helper package was created, why it was created and how it can help you get started quickly.",{"read":352,"type":184,"tool":7369,"category":7370,"tags":7371,"featuredimg":7374,"landingimg":7375,"video":18},[354],[2686],[7372,5992,6423,7373],"helper","notifications","\u002Fnpm-axiom-post.webp","\u002Fnpm-axiom-sq.webp","\u002Fblog\u002Fhow-axiom-ai-helper-was-created",{"title":6605,"description":7367},"blog\u002Fhow-axiom-ai-helper-was-created","7j4OYXsuI5iyGXCOldqZ54YwWE3Y5feQ46VUckcJAYs",{"id":7381,"title":7382,"author":23,"body":7383,"date":7515,"description":7516,"draft":181,"extension":19,"meta":7517,"navigation":24,"path":7526,"seo":7527,"stem":7528,"__hash__":7529},"blog\u002Fblog\u002Fgetting-to-know-the-trycatch-step.md","Getting to Know the Try\u002FCatch Step - Tips and Tricks",{"type":8,"value":7384,"toc":7504},[7385,7388,7395,7398,7400,7403,7406,7409,7413,7416,7419,7422,7426,7429,7439,7442,7445,7456,7463,7466,7470,7483,7487,7490,7493,7499,7501],[11,7386,7387],{},"A critical concept to understand development or automation building is how to handle errors within your workflows in a way that does not cause the workflow to stop. The idea of a Try\u002FCatch has existed in programming for a long time and this is something that we brought into axiom.ai to help you manage your automations.",[11,7389,205,7390,7394],{},[50,7391,7393],{"href":7392},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Ftry-catch","Try\u002FCatch"," step is a step within axiom.ai that can be used within your automations to gracefully catch any errors that you may encounter. Let’s look at an example of when this may be applicable:",[11,7396,7397],{},"You have an automation that visits a website and clicks a button - however, this button does not always appear on the page. Set up without the “Try\u002Fcatch” step, this would cause an error within your automation and the automation will stop. Instead, if you wrap the “Click element” step within a “Try\u002Fcatch” step you can add additional steps inside the “catch” portion that will be run when the error occurs - you could try to click an additional button or simply just send an email letting you know that the error occurred. The steps after the “Try\u002Fcatch” step will still run in this case.",[43,7399,6692],{"id":6691},[11,7401,7402],{},"To get started, add a “Try\u002Fcatch” step to your automation. You’ll notice that this step has two sections, a “try” and a “catch” section. You can use the “+ Add substep” button to add steps into these sections.",[11,7404,7405],{},"There are a couple of ways that you can implement this step into your automation - either proactively or reactively. Proactively adding this to your automation allows you to ensure that you are more likely to have a successful run the first time the automation is run, if you believe that there may be issues or if you are aware that the site is likely to change quite regularly. Adding these reactively would mean that you add these to your automation based on errors that you have experienced during the running of your automation.",[11,7407,7408],{},"If you are familiar with development, the “Try\u002Fcatch” step works exactly like it would when you are writing code yourself so you can likely understand the concept and best practices.",[818,7410,7412],{"id":7411},"try","Try",[11,7414,7415],{},"The “try” portion of this step can be used to add steps that you believe may cause an error within your automation. The steps will run as normal in the flow of your automation and have access to the data tokens from the rest of the automation. You can imagine these running as if the “Try\u002Fcatch” step was not present.",[818,7417,7418],{"id":720},"Catch",[11,7420,7421],{},"The “catch” portion of this still can be used to add steps that are run if an error occurs within the steps included in the “try” section. These steps will not be run during normal operation of the automation and will only run when an error occurs with the steps in the “try” section, if no error is encountered, these steps will never run within your automation. We don't recommend adding steps that you need to run every time your automation runs as this may cause additional errors in your automation outside the \"Try\u002Fcatch\" step if the steps aren't run every run.",[43,7423,7425],{"id":7424},"handling-errors","Handling errors",[11,7427,7428],{},"When an error occurs within your automation, there are a few methods that you can use to handle it. We are going to concentrate on the “catch” portion of the “Try\u002Fcatch” step for this blog. There are many things that you can do once your automation has encountered an error.",[791,7430,7431],{},[11,7432,7433,7434,7438],{},"💡 To learn more about error handling, you can check out our ",[50,7435,7437],{"href":7436},"\u002Fdocs\u002Fno-code-tool\u002Ftroubleshooting\u002Ferror-handling","Error Handling"," documentation where you will find more advice on handling errors.",[43,7440,7441],{"id":7373},"Notifications",[11,7443,7444],{},"If you have your automation scheduled, you’re going to want to know if the automation runs into any errors that occur so you can jump in and make any changes that are required. There are a few steps that you can make use of to alert you or another system, such as:",[249,7446,7447,7451],{},[252,7448,7449],{},[50,7450,6309],{"href":6308},[252,7452,7453],{},[50,7454,7455],{"href":5575},"Trigger a webhook",[11,7457,7458,7459,7462],{},"Personally, the “Trigger a webhook” can be quite useful for alerting other systems that there has been an issue, for example, you could have this trigger a ",[50,7460,7461],{"href":102},"Slack workflow"," webhook that can alert your team, if you have multiple people who require the results of the automation.",[11,7464,7465],{},"Simply adding these steps and configuring them in the “catch” portion of the “Try\u002Fcatch” step will be sufficient to catch any errors.",[43,7467,7469],{"id":7468},"ending-the-run","Ending the run",[11,7471,7472,7473,7477,7478,7482],{},"There may be situations where you wish to end the run in the event of the error rather than continue with the other steps in your workflow. For example, if you want the automation to finish if a button is not present on a page. To do this, you can use the ",[50,7474,7476],{"href":7475},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fend-run","End run"," step within the “catch” portion of the automation - once your automation reaches this step it will stop the automation run. This will be paired with a ",[50,7479,7481],{"href":7480},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsettings\u002Frun-options\u002Fnotifications","notification"," to alert you to this.",[43,7484,7486],{"id":7485},"triggering-additional-steps","Triggering additional steps",[11,7488,7489],{},"It’s important to remember that when an error occurs, it is caught by the “catch” portion of the “Try\u002Fcatch” step, once the steps inside the “catch” portion are completed the rest of the steps within the automation will run.",[11,7491,7492],{},"However, there may be instances where you wish to trigger additional steps when an error occurs with a certain step. For example, if you have a button being clicked inside the “try” portion, and if this is not found an error is caught by the “catch” portion, you may want to attempt to click another button - worth noting though that any errors inside the “catch” portion will not be caught and will cause your automation to fail, consider nesting “Try\u002Fcatch” steps if you are worried about this.",[11,7494,7495,7496,7498],{},"You may also want to skip certain steps within your workflow, you can use the ",[50,7497,5685],{"href":5684}," step in the “catch” section to navigate between your steps in your workflow to create a more dynamic workflow.",[43,7500,166],{"id":165},[11,7502,7503],{},"Handling errors gracefully can help ensure that your automations can dynamically adapt to errors that it may encounter when working with websites that may be changing. The “Try\u002Fcatch” step allows you to decide the actions that are performed in the event of an error so that your automation can continue to run and achieve your goals. This is a great tool for error handling within your automations to prevent them from stopping when they encounter an error, if required.",{"title":15,"searchDepth":16,"depth":16,"links":7505},[7506,7510,7511,7512,7513,7514],{"id":6691,"depth":173,"text":6692,"children":7507},[7508,7509],{"id":7411,"depth":16,"text":7412},{"id":720,"depth":16,"text":7418},{"id":7424,"depth":173,"text":7425},{"id":7373,"depth":173,"text":7441},{"id":7468,"depth":173,"text":7469},{"id":7485,"depth":173,"text":7486},{"id":165,"depth":173,"text":166},"2025-04-22","Learn how you can utilise the try\u002Fcatch step to handle errors, end runs, or send notifications during your bot runs.",{"read":352,"type":184,"tool":7518,"category":7519,"tags":7520,"featuredimg":7524,"landingimg":7525,"summary":7516,"video":18,"metaTitle":7382},[354],[2686],[7521,7522,7523,7373],"try \u002F catch","logic","errors","\u002Ftry-catch-post.webp","\u002Ftry-catch-sq.webp","\u002Fblog\u002Fgetting-to-know-the-trycatch-step",{"title":7382,"description":7516},"blog\u002Fgetting-to-know-the-trycatch-step","vOgDNq2PjmeLRo-RAP3nE4MrmDHJ87MyMUDtTa0XgaM",{"id":7531,"title":7532,"author":23,"body":7533,"date":8072,"description":8073,"draft":181,"extension":19,"meta":8074,"navigation":24,"path":8084,"seo":8085,"stem":8086,"__hash__":8087},"blog\u002Fblog\u002Fcreating-a-reddit-notification-bot.md","Creating a Reddit Notification Bot",{"type":8,"value":7534,"toc":8062},[7535,7538,7541,7543,7546,7550,7553,7557,7560,7563,7567,7570,7597,7600,7604,7607,7615,7618,7622,7625,7629,7632,7643,7646,8015,8018,8021,8028,8032,8035,8038,8042,8045,8048,8051,8053,8056,8059],[11,7536,7537],{},"We had a problem here at axiom.ai, we wanted to be sure that our teams were notified of new posts within our community so that we could review them, help where we can, or simply collect feedback from all of you.",[11,7539,7540],{},"Originally, we had a Zapier workflow that would detect new posts in the subreddit, and then trigger an axiom.ai automation that would then trigger a Slack workflow. Unfortunately, due to changes with Zapier, this stopped working as expected which meant that we missed a few posts.",[39,7542],{},[11,7544,7545],{},"Let’s look into how we solved this issue and simplified this process purely with axiom.ai and Google Sheets.",[43,7547,7549],{"id":7548},"the-problem-to-solve","The problem to solve",[11,7551,7552],{},"We will start by defining the problem that we are looking to solve with axiom.ai. Our team wanted to be notified when new posts go live into our community. This would ensure that we can get back to those who need help, and collect any feedback that has been posted there so we can feed it back into our development process.",[43,7554,7556],{"id":7555},"solving-the-problem","Solving the problem",[11,7558,7559],{},"Enter axiom.ai and Google Sheets. To solve this problem we are going to want an automation that can automatically get the data from the subreddit, and then check if there is new data on the subreddit. If there is new data, we then want the automation to let the team know.",[11,7561,7562],{},"Note: this will only track the newest ‘new’ post.",[818,7564,7566],{"id":7565},"setting-up-your-google-sheet","Setting up your Google Sheet",[11,7568,7569],{},"Before getting started, we would recommend setting up a Google Sheet that will be used to keep track of the most recent post. We created a sheet with three columns and we will be continuing this guide with the assumption that you have done the same. You can see the sample below of how we have it set up.",[6730,7571,7572,7585],{},[6733,7573,7574],{},[6736,7575,7576,7579,7582],{},[6739,7577,7578],{},"Subreddit",[6739,7580,7581],{},"Latest Post",[6739,7583,7584],{},"Date of last check",[6749,7586,7587],{},[6736,7588,7589,7592,7595],{},[6754,7590,7591],{},"r\u002Faxiom_ai",[6754,7593,7594],{},"No current post",[6754,7596],{},[11,7598,7599],{},"The first step in your automation should be a Read data from a Google Sheet that reads in the data that you are using, we set the “First cell” input to “A2”, and the “Last cell” input to “B2” - at this stage we only need the subreddit name, and the latest post URL.",[818,7601,7603],{"id":7602},"getting-the-subreddit-data","Getting the subreddit data",[11,7605,7606],{},"The first step that we need to do is to be able to get the data from the subreddit. It would be possible to directly scrape this from the page but we have found that a more reliable method is to do this using the RSS feed that is provided for each subreddit. In our case, the URL we use is the following:",[791,7608,7609],{},[11,7610,7611],{},[50,7612,7613],{"href":7613,"rel":7614},"https:\u002F\u002Fwww.reddit.com\u002Fr\u002Faxiom_ai\u002F.rss",[54],[11,7616,7617],{},"You can replace the subreddit name with the subreddit that you’d like to scrape. We are going to use the subreddit name that we have read in from the Google Sheet to make the automation more dynamic. You’ll need to set up your Go to page step to do the following:",[134,7619],{"src":7620,"alt":7621},"\u002Fblog\u002Fcreating-a-reddit-notification-bot-go-to-page.png","A dialog box titled 'Go to page: reddit.com' is displayed. It contains a field labelled 'Enter URL' with the pre-filled URL 'https:\u002F\u002Fwww.reddit.com\u002F[google-sheet-data?all&0]\u002F.rss'. Below this field is the text 'You can also plugin data from other steps to provide a URL. Enter (skip) to skip this step.' Two toggle switches are present at the bottom: 'Do not share localstorage' is toggled off, and 'Open in new tab' is also toggled off. Above the URL field are two buttons: 'Open url in new tab' and 'Get current URL'.",[11,7623,7624],{},"You can also skip the step of adding in the data token and insert the full URL manually. To the human eye, this looks like a jumble of information, we will look at how we can extract the data next.",[818,7626,7628],{"id":7627},"deciding-if-there-is-new-content","Deciding if there is new content",[11,7630,7631],{},"To determine if there are new posts, we will first need to read the content of the page, and then compare it to the URL of the latest post that we have stored in our Google Sheet. To do this we will need to make use of custom code within the Write Javascript step. To get started, add the following steps to your automation:",[7633,7634,7635],"guide",{},[312,7636,7637,7640],{},[252,7638,7639],{},"A Try\u002Fcatch step - to catch any errors",[252,7641,7642],{},"Inside of the “try” section of this step, add a Write Javascript step.",[11,7644,7645],{},"Once this has been finished, you can insert the code below into the Write Javascript step:",[381,7647,7649],{"className":383,"code":7648,"language":385,"meta":15,"style":15},"\u002F\u002F Parse the text that we have retrieved from the 'pre' tag containing the RSS data.\nconst parseRss = (rssText) => {\n  const parser = new DOMParser()\n  return parser.parseFromString(rssText, 'text\u002Fxml')\n}\n\n\u002F\u002F Extract the individual items from the data into rows to be used in your automation.\nconst extractData = (xmlDoc) => {\n  \u002F\u002F Grab the individual 'entry' items\n  const items = xmlDoc.getElementsByTagName('entry')\n  var entries = []\n\n  for (let i = 0; i \u003C items.length; i++) {\n    const author = items[i]\n      .getElementsByTagName('author')[0]\n      .getElementsByTagName('name')[0].textContent\n    const link = items[i].getElementsByTagName('link')[0].getAttribute('href')\n    const title = items[i].getElementsByTagName('title')[0].textContent\n\n    entries.push([author, link, title])\n  }\n  return entries\n}\n\nconst preBody = document.getElementsByTagName('pre')[0].innerText\nconst parsedRss = parseRss(preBody)\n\nreturn extractData(parsedRss)\n",[387,7650,7651,7656,7676,7692,7711,7715,7719,7724,7744,7749,7771,7784,7788,7818,7830,7851,7869,7905,7929,7933,7943,7947,7954,7958,7962,7987,8001,8005],{"__ignoreMap":15},[390,7652,7653],{"class":392,"line":393},[390,7654,7655],{"class":571},"\u002F\u002F Parse the text that we have retrieved from the 'pre' tag containing the RSS data.\n",[390,7657,7658,7660,7663,7665,7667,7670,7672,7674],{"class":392,"line":173},[390,7659,418],{"class":396},[390,7661,7662],{"class":421}," parseRss",[390,7664,425],{"class":396},[390,7666,3123],{"class":400},[390,7668,7669],{"class":622},"rssText",[390,7671,626],{"class":400},[390,7673,434],{"class":396},[390,7675,437],{"class":400},[390,7677,7678,7680,7683,7685,7687,7690],{"class":392,"line":16},[390,7679,461],{"class":396},[390,7681,7682],{"class":464}," parser",[390,7684,425],{"class":396},[390,7686,613],{"class":396},[390,7688,7689],{"class":421}," DOMParser",[390,7691,541],{"class":400},[390,7693,7694,7697,7700,7703,7706,7709],{"class":392,"line":440},[390,7695,7696],{"class":396},"  return",[390,7698,7699],{"class":400}," parser.",[390,7701,7702],{"class":421},"parseFromString",[390,7704,7705],{"class":400},"(rssText, ",[390,7707,7708],{"class":407},"'text\u002Fxml'",[390,7710,455],{"class":400},[390,7712,7713],{"class":392,"line":458},[390,7714,755],{"class":400},[390,7716,7717],{"class":392,"line":482},[390,7718,413],{"emptyLinePlaceholder":24},[390,7720,7721],{"class":392,"line":494},[390,7722,7723],{"class":571},"\u002F\u002F Extract the individual items from the data into rows to be used in your automation.\n",[390,7725,7726,7728,7731,7733,7735,7738,7740,7742],{"class":392,"line":500},[390,7727,418],{"class":396},[390,7729,7730],{"class":421}," extractData",[390,7732,425],{"class":396},[390,7734,3123],{"class":400},[390,7736,7737],{"class":622},"xmlDoc",[390,7739,626],{"class":400},[390,7741,434],{"class":396},[390,7743,437],{"class":400},[390,7745,7746],{"class":392,"line":514},[390,7747,7748],{"class":571},"  \u002F\u002F Grab the individual 'entry' items\n",[390,7750,7751,7753,7756,7758,7761,7764,7766,7769],{"class":392,"line":522},[390,7752,461],{"class":396},[390,7754,7755],{"class":464}," items",[390,7757,425],{"class":396},[390,7759,7760],{"class":400}," xmlDoc.",[390,7762,7763],{"class":421},"getElementsByTagName",[390,7765,449],{"class":400},[390,7767,7768],{"class":407},"'entry'",[390,7770,455],{"class":400},[390,7772,7773,7776,7779,7781],{"class":392,"line":544},[390,7774,7775],{"class":396},"  var",[390,7777,7778],{"class":400}," entries ",[390,7780,893],{"class":396},[390,7782,7783],{"class":400}," []\n",[390,7785,7786],{"class":392,"line":563},[390,7787,413],{"emptyLinePlaceholder":24},[390,7789,7790,7793,7795,7797,7799,7801,7803,7805,7807,7810,7812,7814,7816],{"class":392,"line":568},[390,7791,7792],{"class":396},"  for",[390,7794,3123],{"class":400},[390,7796,842],{"class":396},[390,7798,3129],{"class":400},[390,7800,893],{"class":396},[390,7802,3134],{"class":464},[390,7804,3137],{"class":400},[390,7806,3140],{"class":396},[390,7808,7809],{"class":400}," items.",[390,7811,3146],{"class":464},[390,7813,3149],{"class":400},[390,7815,3152],{"class":396},[390,7817,3155],{"class":400},[390,7819,7820,7822,7825,7827],{"class":392,"line":575},[390,7821,525],{"class":396},[390,7823,7824],{"class":464}," author",[390,7826,425],{"class":396},[390,7828,7829],{"class":400}," items[i]\n",[390,7831,7832,7835,7837,7839,7842,7845,7848],{"class":392,"line":603},[390,7833,7834],{"class":400},"      .",[390,7836,7763],{"class":421},[390,7838,449],{"class":400},[390,7840,7841],{"class":407},"'author'",[390,7843,7844],{"class":400},")[",[390,7846,7847],{"class":464},"0",[390,7849,7850],{"class":400},"]\n",[390,7852,7853,7855,7857,7859,7862,7864,7866],{"class":392,"line":608},[390,7854,7834],{"class":400},[390,7856,7763],{"class":421},[390,7858,449],{"class":400},[390,7860,7861],{"class":407},"'name'",[390,7863,7844],{"class":400},[390,7865,7847],{"class":464},[390,7867,7868],{"class":400},"].textContent\n",[390,7870,7871,7873,7876,7878,7881,7883,7885,7888,7890,7892,7895,7898,7900,7903],{"class":392,"line":633},[390,7872,525],{"class":396},[390,7874,7875],{"class":464}," link",[390,7877,425],{"class":396},[390,7879,7880],{"class":400}," items[i].",[390,7882,7763],{"class":421},[390,7884,449],{"class":400},[390,7886,7887],{"class":407},"'link'",[390,7889,7844],{"class":400},[390,7891,7847],{"class":464},[390,7893,7894],{"class":400},"].",[390,7896,7897],{"class":421},"getAttribute",[390,7899,449],{"class":400},[390,7901,7902],{"class":407},"'href'",[390,7904,455],{"class":400},[390,7906,7907,7909,7912,7914,7916,7918,7920,7923,7925,7927],{"class":392,"line":646},[390,7908,525],{"class":396},[390,7910,7911],{"class":464}," title",[390,7913,425],{"class":396},[390,7915,7880],{"class":400},[390,7917,7763],{"class":421},[390,7919,449],{"class":400},[390,7921,7922],{"class":407},"'title'",[390,7924,7844],{"class":400},[390,7926,7847],{"class":464},[390,7928,7868],{"class":400},[390,7930,7931],{"class":392,"line":654},[390,7932,413],{"emptyLinePlaceholder":24},[390,7934,7935,7938,7940],{"class":392,"line":665},[390,7936,7937],{"class":400},"    entries.",[390,7939,4322],{"class":421},[390,7941,7942],{"class":400},"([author, link, title])\n",[390,7944,7945],{"class":392,"line":671},[390,7946,749],{"class":400},[390,7948,7949,7951],{"class":392,"line":676},[390,7950,7696],{"class":396},[390,7952,7953],{"class":400}," entries\n",[390,7955,7956],{"class":392,"line":688},[390,7957,755],{"class":400},[390,7959,7960],{"class":392,"line":699},[390,7961,413],{"emptyLinePlaceholder":24},[390,7963,7964,7966,7969,7971,7973,7975,7977,7980,7982,7984],{"class":392,"line":714},[390,7965,418],{"class":396},[390,7967,7968],{"class":464}," preBody",[390,7970,425],{"class":396},[390,7972,3021],{"class":400},[390,7974,7763],{"class":421},[390,7976,449],{"class":400},[390,7978,7979],{"class":407},"'pre'",[390,7981,7844],{"class":400},[390,7983,7847],{"class":464},[390,7985,7986],{"class":400},"].innerText\n",[390,7988,7989,7991,7994,7996,7998],{"class":392,"line":726},[390,7990,418],{"class":396},[390,7992,7993],{"class":464}," parsedRss",[390,7995,425],{"class":396},[390,7997,7662],{"class":421},[390,7999,8000],{"class":400},"(preBody)\n",[390,8002,8003],{"class":392,"line":735},[390,8004,413],{"emptyLinePlaceholder":24},[390,8006,8007,8010,8012],{"class":392,"line":746},[390,8008,8009],{"class":396},"return",[390,8011,7730],{"class":421},[390,8013,8014],{"class":400},"(parsedRss)\n",[11,8016,8017],{},"This code will return the author, link and title of the latest post that has been created on the subreddit. This will be stored in the code-data data token that can then be used in later steps of your automation. This code can be extended to take more values out of the code of the page - feel free to copy it and play around with it for yourself!",[11,8019,8020],{},"Once we have this data, we will need to decide if the post is a new post, or if it’s an existing post. To do this, we will want to check the URL of the latest post against the URL of the post that you have stored in your Google Sheet.",[11,8022,8023,8024,8027],{},"To get started, add an “If condition is true, run steps” step to your automation. The data to check will be the URL that has been stored in your Google Sheet - this should be in Column B of your data that was imported in the “Read data from a Google Sheet” step. We will want to compare this to Column B from the ",[387,8025,8026],{},"code-data"," data token that was exported from the “Write JavaScript” step. It’s important to note that we want the steps inside of the “If condition” step to run if these do not match - enable the “Reverse condition” option.",[134,8029],{"src":8030,"alt":8031},"\u002Fblog\u002Fcreating-a-reddit-notification-bot-if-condition.png","A dialog box titled 'If condition' is displayed. The 'Data to check:' field shows '[google-sheet-data?all&1]'. The 'Condition to check:' is set to 'When any of '[code-data?&1]'' is present'. Below this, there are tabs for 'Words', 'Numbers', and 'JS == true', with 'Words' selected. A field contains the inserted data '[code-data?&1]' with a close button. Below this is the instruction: 'Enter either a list of any number of words to check for, separated by commas, or data containing a list of words, one in each row. Leave blank to match anything.' 1  The 'Apply rule when match' section has buttons for 'Any word' (selected) and 'All words'. A checkbox for 'Match only complete word' is unchecked, with explanatory text below it. Finally, a 'Reverse condition:' dropdown is set to 'Run steps if condition is false'.",[11,8033,8034],{},"Now, any steps that you include within the “If condition” step will run when the URL that is stored within the Google Sheet does not match the URL of the latest post.",[11,8036,8037],{},"You can add steps to notify your team of any new posts, this could be done using a Send an Email step, the Trigger a webhook step, or simply just logging this into a Google Sheet. We decided to go ahead and send this onto a Slack channel using our guide on How to automate Slack with axiom.ai.",[818,8039,8041],{"id":8040},"logging-the-status-of-the-automation","Logging the status of the automation",[11,8043,8044],{},"As previously mentioned, the post URL will need to be logged each time a new post has been picked up by the automation. Now that we have confirmation that a new post has been created, we will want to replace the URL of the last post that the team was notified about and replace this with the URL of the latest post. These steps will need to remain inside the “If condition” step that we created above.",[11,8046,8047],{},"First, we will want to replace the URL within the Google Sheet. To do this, create a new “Write data to a Google Sheet” step. Configure this to use the same sheet that you used previously, and set the data to write to be the code-data data token, using Column B, when prompted. The step should be set to “Add to existing data” and “B2” should be entered in the cell.",[11,8049,8050],{},"Next, we are going to log the last time that the automation ran and successfully detected a new post - first, add a “Date and time” step and configure this with the time format that you wish to use. Then, create a new “Write data to a Google Sheet” step. Configure this to use the same sheet that you used previously, and set the data to write to be the date-and-time data token. The step should be set to “Add to existing data” and “C2” should be entered in the cell.",[43,8052,166],{"id":165},[11,8054,8055],{},"In this article, we explored the problem of automating Reddit notifications using axiom.ai and Google Sheets. We walked through the steps involved in setting up the automation, from reading data from a Google Sheet to getting the subreddit data and comparing it to the latest stored post. We also demonstrated how to use custom code to parse the RSS feed and extract the individual items from the data.",[11,8057,8058],{},"By leveraging the power of axiom.ai and Google Sheets, we were able to create a reliable and efficient automation that helps our team stay informed about new posts in our community. This solution allows us to be more responsive to our users and collect valuable feedback that we can use to improve our products and services.",[2666,8060,8061],{},"html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sTDnQ, html code.shiki .sTDnQ{--shiki-default:#953800;--shiki-dark:#FFA657}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":8063},[8064,8065,8071],{"id":7548,"depth":173,"text":7549},{"id":7555,"depth":173,"text":7556,"children":8066},[8067,8068,8069,8070],{"id":7565,"depth":16,"text":7566},{"id":7602,"depth":16,"text":7603},{"id":7627,"depth":16,"text":7628},{"id":8040,"depth":16,"text":8041},{"id":165,"depth":173,"text":166},"2025-04-17","Discover how to create a bot that can check for new posts within a subreddit and send a notification when there is one",{"read":183,"type":387,"tool":8075,"category":8076,"tags":8077,"featuredimg":8082,"landingimg":8083,"summary":8073,"video":18,"metaTitle":7532},[354],[6214],[8078,8079,8080,8081],"subreddit","subreddit data","google sheet","rss feed","\u002Fblog\u002Freddit-notification-post.webp","\u002Fblog\u002Freddit-sq-notifications.webp","\u002Fblog\u002Fcreating-a-reddit-notification-bot",{"title":7532,"description":8073},"blog\u002Fcreating-a-reddit-notification-bot","2KFL0_hj7YsT0oLFhUk_6VypgRur5JFbgkyh_bsbZVU",{"id":8089,"title":8090,"author":23,"body":8091,"date":8413,"description":8414,"draft":181,"extension":19,"meta":8415,"navigation":24,"path":8424,"seo":8425,"stem":8426,"__hash__":8427},"blog\u002Fblog\u002Fbeyond-the-basics-javascript-and-axiom.md","Beyond the Basics - JavaScript and axiom.ai",{"type":8,"value":8092,"toc":8402},[8093,8099,8115,8119,8125,8130,8138,8198,8205,8208,8236,8240,8248,8251,8259,8263,8272,8280,8283,8287,8290,8294,8297,8300,8303,8307,8314,8323,8326,8335,8338,8372,8385,8388,8391,8394,8397,8400],[11,8094,8095,8096,8098],{},"axiom.ai automations are already very powerful on their own, add JavaScript to them? You can extend it's functionalities tenfold, you're only limited by your development skills and imagination. The ",[50,8097,5113],{"href":5581}," step that we provide allows you to add your scripts right into your automation and have them executed on the page that your bot is working on. This can be useful for adding custom logic, data processing and accessing Puppeteer functions within your runs.",[11,8100,8101,8102,828,8106,8110,8111,8114],{},"Did you know that we also provide some snippets? Our growing library of ",[50,8103,8105],{"href":8104},"\u002Fdocs\u002Fdeveloper-hub\u002Fsnippets\u002Fjavascript","JavaScript",[50,8107,8109],{"href":8108},"\u002Fdocs\u002Fdeveloper-hub\u002Fsnippets\u002Fpuppeteer","Puppeteer"," snippets can help solve some standard use cases within your automations. Our ",[50,8112,3304],{"href":127,"rel":8113},[54]," can also be a great place to get some help, if you are unsure.",[43,8116,8118],{"id":8117},"using-javascript-within-an-automation","Using JavaScript within an automation",[11,8120,8121,8122,8124],{},"To get started, add a ",[50,8123,5113],{"href":5581}," step to your automation. This step allows you to write a JavaScript script in your automation. The script that you write will only be executed when the automation run reaches the step that you have inserted the \"Write Javascript\" step at.",[791,8126,8127],{},[11,8128,8129],{},"💡 Your script will be executed in the context of a function, this is important to note to understand the syntax and features that can be used within this step.",[11,8131,8132,8133,8137],{},"While axiom.ai does provide a library of ",[50,8134,8136],{"href":8135},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002F#manipulate-data","data manipulation"," steps that can be used to manipulate any data that you pass into your automation, or gather through your automations actions, there are times when you may want a more than the built in features provide. An example of this is adding a constant value to any empty cells of scrape data - when scrape data is written to a Google Sheet, these blanks will cause the data to shift as a result of their API. To get around this, a simple script can be used:",[381,8139,8141],{"className":383,"code":8140,"language":385,"meta":15,"style":15},"var data = [scrape - data]\nconst placeholder = '-'\n\nfor (var i = 0; i \u003C data.length; i++) {\n  for (var j = 0; j \u003C data[i].length; j++) {\n    if (data[i][j] == '') {\n      data[i][j] = placeholder\n    }\n  }\n}\n\nreturn data\n",[387,8142,8143,8148,8153,8157,8162,8167,8172,8177,8181,8185,8189,8193],{"__ignoreMap":15},[390,8144,8145],{"class":392,"line":393},[390,8146,8147],{},"var data = [scrape - data]\n",[390,8149,8150],{"class":392,"line":173},[390,8151,8152],{},"const placeholder = '-'\n",[390,8154,8155],{"class":392,"line":16},[390,8156,413],{"emptyLinePlaceholder":24},[390,8158,8159],{"class":392,"line":440},[390,8160,8161],{},"for (var i = 0; i \u003C data.length; i++) {\n",[390,8163,8164],{"class":392,"line":458},[390,8165,8166],{},"  for (var j = 0; j \u003C data[i].length; j++) {\n",[390,8168,8169],{"class":392,"line":482},[390,8170,8171],{},"    if (data[i][j] == '') {\n",[390,8173,8174],{"class":392,"line":494},[390,8175,8176],{},"      data[i][j] = placeholder\n",[390,8178,8179],{"class":392,"line":500},[390,8180,1889],{},[390,8182,8183],{"class":392,"line":514},[390,8184,749],{},[390,8186,8187],{"class":392,"line":522},[390,8188,755],{},[390,8190,8191],{"class":392,"line":544},[390,8192,413],{"emptyLinePlaceholder":24},[390,8194,8195],{"class":392,"line":563},[390,8196,8197],{},"return data\n",[11,8199,8200,8201,13],{},"Source: ",[50,8202,8204],{"href":8203},"\u002Fdocs\u002Fdeveloper-hub\u002Fsnippets\u002Fjavascript\u002Fdata#adding-a-constant-to-scrape-data","JavaScript Snippets",[11,8206,8207],{},"Functions can be declared within your scripts, allowing you to structure your code neatly, for example:",[381,8209,8211],{"className":383,"code":8210,"language":385,"meta":15,"style":15},"const add = (a, b) => {\n  return a + b\n}\n\nconst sum = add(1, 2) \u002F\u002F 3\n",[387,8212,8213,8218,8223,8227,8231],{"__ignoreMap":15},[390,8214,8215],{"class":392,"line":393},[390,8216,8217],{},"const add = (a, b) => {\n",[390,8219,8220],{"class":392,"line":173},[390,8221,8222],{},"  return a + b\n",[390,8224,8225],{"class":392,"line":16},[390,8226,755],{},[390,8228,8229],{"class":392,"line":440},[390,8230,413],{"emptyLinePlaceholder":24},[390,8232,8233],{"class":392,"line":458},[390,8234,8235],{},"const sum = add(1, 2) \u002F\u002F 3\n",[818,8237,8239],{"id":8238},"javascript-limitations","JavaScript limitations",[11,8241,8242,8243,8247],{},"Your scripts are encapsulated and will not interact with each other. For example, if you have a script that declares the variable ",[390,8244,8246],{"style":8245},"font-family: monospace;","example",", you will not be able to access this within another script.",[11,8249,8250],{},"As the script that you insert into this step is executed as a function, you will not be able to import any additional libraries or modules into your script - doing so will actually cause issues with your automation and may prevent it from running.",[11,8252,8253,8254,8258],{},"If you are using JavaScript to access an API, it's important to remember that you may run into cross-origin resources sharing (CORS) issues. You may attempt ",[50,8255,8257],{"href":8256},"#running-in-app","running in-app"," option, but we are unable to guarantee that this will resolve the problem.",[43,8260,8262],{"id":8261},"using-puppeteer-within-an-automation","Using Puppeteer within an automation",[11,8264,8265,8266,8271],{},"In the same way that you can use JavaScript within your automations, you can also access certain Puppeteer functions. This allows you to make completely custom automations that can interact with pages and perform actions using custom code - essentially using axiom.ai as a wrapper for your custom scripts. Generally, you are able to use any function that is available in the ",[50,8267,8270],{"href":8268,"rel":8269},"https:\u002F\u002Fpptr.dev\u002Fapi\u002Fpuppeteer.page",[54],"Page"," class. This means that you can access functions such as:",[249,8273,8274,8277],{},[252,8275,8276],{},"click()",[252,8278,8279],{},"reload()",[11,8281,8282],{},"And many other functions that can be found in the Puppeteer documentation. This can be helpful if you need to access certain functions that are not natively available through axiom.ai, such as automating the scrolling on a page, or performing actions on a subset of elements on a page, for example, only buttons that contain a certain string of text rather than all buttons on the page.",[818,8284,8286],{"id":8285},"puppeteer-limitations","Puppeteer limitations",[11,8288,8289],{},"Puppeteer scripts can only be run locally and can not be run in the cloud, this means that you will need to have the desktop application installed and active to run your scripts - you'll need to select \"Run in app\" within the step. This does limit the functionality of this integration, but JavaScript can usually be a stand-in in the event that you need to run this in the cloud.",[43,8291,8293],{"id":8292},"running-in-app","Running in-app",[11,8295,8296],{},"The \"Run in app\" option within the \"Write Javascript\" step can be used to run code in the context of the desktop application, rather than the context of the browser. If you are running your automation locally, you will need to have the desktop application installed and active. There are some benefits to using it with your other scripts, such as:",[11,8298,8299],{},"Ensuring your code does not interfere with the site's code - for example, triggering event listeners, conflicting function or variable names.\nAccessing the Puppeteer library (required to use the \"Run in app\" option).",[11,8301,8302],{},"The downside of running code in the desktop application is the inability to access elements, functions or variables from the page that your automation is running on. You will not have the ability to see the console output from code within the \"Write Javascript\" step while using this option.",[43,8304,8306],{"id":8305},"using-data-tokens","Using data tokens",[11,8308,8309,8310,8313],{},"Data tokens contain information that has been output from steps - for example, the ",[387,8311,8312],{},"[scrape-data]"," data token that is output from the \"Get data from bot's current page\" step contains all of the data that the step has retrieved from the page. These can be used within your custom scripts by clicking the \"Insert data\" option and this will insert the data token into your script. For example:",[381,8315,8317],{"className":383,"code":8316,"language":385,"meta":15,"style":15},"let data = [scrape - data]\n",[387,8318,8319],{"__ignoreMap":15},[390,8320,8321],{"class":392,"line":393},[390,8322,8316],{},[11,8324,8325],{},"For data tokens that just contain a single value, you can insert it like so, replacing the data token with the one from your automation:",[381,8327,8329],{"className":383,"code":8328,"language":385,"meta":15,"style":15},"let data = '[data-token]'\n",[387,8330,8331],{"__ignoreMap":15},[390,8332,8333],{"class":392,"line":393},[390,8334,8328],{},[11,8336,8337],{},"From this point you have an array of arrays that you can loop through, manipulate or use for logic conditions throughout your script. For example:",[381,8339,8341],{"className":383,"code":8340,"language":385,"meta":15,"style":15},"let data = [scrape - data]\n\nfor (var i = 0; i \u003C data.length; i++) {\n  for (var j = 0; j \u003C data[i].length; j++) {\n    console.log(data[i][j])\n  }\n}\n",[387,8342,8343,8347,8351,8355,8359,8364,8368],{"__ignoreMap":15},[390,8344,8345],{"class":392,"line":393},[390,8346,8316],{},[390,8348,8349],{"class":392,"line":173},[390,8350,413],{"emptyLinePlaceholder":24},[390,8352,8353],{"class":392,"line":16},[390,8354,8161],{},[390,8356,8357],{"class":392,"line":440},[390,8358,8166],{},[390,8360,8361],{"class":392,"line":458},[390,8362,8363],{},"    console.log(data[i][j])\n",[390,8365,8366],{"class":392,"line":482},[390,8367,749],{},[390,8369,8370],{"class":392,"line":494},[390,8371,755],{},[11,8373,8374,8375,8377,8378,285,8380,8382,8383,13],{},"This can be helpful when creating custom integrations with services that make use of the data from your automations, such as posting to ",[50,8376,5454],{"href":5453},", sending data to ",[50,8379,146],{"href":161},[50,8381,6528],{"href":6527},", or triggering other automations such as from ",[50,8384,114],{"href":141},[43,8386,8387],{"id":165},"Wrapping Up",[11,8389,8390],{},"By incorporating JavaScript and Puppeteer into your axiom.ai automations, you unlock a realm of possibilities far beyond standard automation capabilities. Whether you're refining scraped data with custom logic, interacting with web pages using Puppeteer's powerful functions, or building intricate integrations with external services, these tools empower you to tailor your automations to your precise needs.",[11,8392,8393],{},"Remember, while JavaScript offers flexibility for data manipulation and custom logic, Puppeteer enables direct interaction with web pages, albeit with the limitation of local execution. Leverage our growing library of snippets and community support to navigate common use cases and overcome any challenges you encounter.",[11,8395,8396],{},"Keep in mind the limitations of encapsulated scripts and CORS issues when working with APIs, and always explore the \"Run in app\" option when necessary. By effectively utilizing data tokens, you can seamlessly integrate data from various steps within your automation into your custom scripts, creating dynamic and responsive workflows.",[11,8398,8399],{},"With axiom.ai's JavaScript and Puppeteer integration, you're not just automating tasks; you're crafting bespoke solutions that enhance your productivity and streamline your workflows. We encourage you to experiment, explore, and push the boundaries of what's possible with automation.",[2666,8401,3800],{},{"title":15,"searchDepth":16,"depth":16,"links":8403},[8404,8407,8410,8411,8412],{"id":8117,"depth":173,"text":8118,"children":8405},[8406],{"id":8238,"depth":16,"text":8239},{"id":8261,"depth":173,"text":8262,"children":8408},[8409],{"id":8285,"depth":16,"text":8286},{"id":8292,"depth":173,"text":8293},{"id":8305,"depth":173,"text":8306},{"id":165,"depth":173,"text":8387},"2025-04-15","Take your axiom.ai skills to the next level. Learn how to use JavaScript for custom logic, data processing, and advanced automation, moving beyond standard workflows.",{"read":8416,"type":387,"tool":8417,"category":8418,"tags":8419,"featuredimg":8422,"landingimg":8423,"summary":8414,"video":18,"metaTitle":8090},"7 min read",[354],[2686],[2690,8109,8420,8421],"Data tokens","Write JavaScript step","\u002Faxiom-js.webp","\u002Faxiom-js-sq.webp","\u002Fblog\u002Fbeyond-the-basics-javascript-and-axiom",{"title":8090,"description":8414},"blog\u002Fbeyond-the-basics-javascript-and-axiom","EMvO8aMExjKIhKX3lJM5KesN9ztznzzwWexyAdRMZsw",{"id":8429,"title":8430,"author":23,"body":8431,"date":9144,"description":9145,"draft":181,"extension":19,"meta":9146,"navigation":24,"path":9156,"seo":9157,"stem":9158,"__hash__":9159},"blog\u002Fblog\u002Funderstanding-and-working-with-the-shadow-dom.md","Understanding and Working with the Shadow DOM",{"type":8,"value":8432,"toc":9132},[8433,8437,8440,8444,8447,8453,8459,8463,8466,8470,8473,8502,8508,8599,8606,8610,8618,8849,8861,8864,8883,8891,8895,8898,8903,8909,8920,8931,8979,8982,8986,8989,9092,9095,9099,9102,9118,9120,9123,9126,9129],[43,8434,8436],{"id":8435},"what-is-the-shadow-dom","What is the Shadow DOM?",[11,8438,8439],{},"As web applications grow in complexity, managing styles and preventing conflicts between different components becomes increasingly challenging. Enter the Shadow DOM, a key technology that empowers developers to create encapsulated and reusable web components. By providing a way to isolate HTML, CSS, and JavaScript, the Shadow DOM enables the construction of modular and self-contained elements that enhance code organization and maintainability. This article will guide you through the core concepts of the Shadow DOM, demonstrating its practical use and highlighting its role in modern web development.",[43,8441,8443],{"id":8442},"why-use-shadow-dom","Why use Shadow DOM?",[11,8445,8446],{},"Shadow DOM can be a very powerful feature of web development that allows for elements to be created separately from the main document of the website. You can think of these as \"sub documents\". There are a few benefits to using this:",[11,8448,8449,8452],{},[5176,8450,8451],{},"Encapsulation:"," as the code within the Shadow DOM are self-contained, this means that the code inside of them is self-contained. This can be useful to avoid the resources from the main document interacting with the code within the Shadow DOM, which could have unexpected side-effects.",[11,8454,8455,8458],{},[5176,8456,8457],{},"Style scoping:"," as a result of encapsulation, you can include styling within the Shadow DOM that will only have an effect on the elements included within. This can give you the ability to customise certain elements, without impacting others.",[43,8460,8462],{"id":8461},"working-with-the-shadow-dom","Working with the Shadow DOM",[11,8464,8465],{},"Working with the Shadow DOM requires the use of JavaScript and they can be interacted with programmatically.",[818,8467,8469],{"id":8468},"creating-a-shadow-dom","Creating a Shadow DOM",[11,8471,8472],{},"To get started, you'll first need to create a Shadow DOM on your page, consider the following HTML - this will demonstrate where we are wanting to place the code. We will be attaching the Shadow DOM to this element using JavaScript.",[381,8474,8478],{"className":8475,"code":8476,"language":8477,"meta":15,"style":15},"language-html shiki shiki-themes github-light-default github-dark-default","\u003Cdiv id=\"host\">\u003C\u002Fdiv>\n","html",[387,8479,8480],{"__ignoreMap":15},[390,8481,8482,8484,8486,8489,8491,8494,8497,8499],{"class":392,"line":393},[390,8483,3140],{"class":400},[390,8485,2937],{"class":7078},[390,8487,8488],{"class":464}," id",[390,8490,893],{"class":400},[390,8492,8493],{"class":407},"\"host\"",[390,8495,8496],{"class":400},">\u003C\u002F",[390,8498,2937],{"class":7078},[390,8500,8501],{"class":400},">\n",[11,8503,8504,8505,8507],{},"Now, we can come in with JavaScript to create our Shadow DOM, we are going to use the ",[387,8506,2894],{}," with the ID of \"host\" as our placeholder:",[381,8509,8511],{"className":383,"code":8510,"language":385,"meta":15,"style":15},"const host = document.querySelector('#host')\nconst shadow = host.attachShadow({ mode: 'open' })\nconst span = document.createElement('span')\nspan.textContent = \"I'm in the shadow DOM\"\nshadow.appendChild(span)\n",[387,8512,8513,8533,8557,8578,8588],{"__ignoreMap":15},[390,8514,8515,8517,8520,8522,8524,8526,8528,8531],{"class":392,"line":393},[390,8516,418],{"class":396},[390,8518,8519],{"class":464}," host",[390,8521,425],{"class":396},[390,8523,3021],{"class":400},[390,8525,4945],{"class":421},[390,8527,449],{"class":400},[390,8529,8530],{"class":407},"'#host'",[390,8532,455],{"class":400},[390,8534,8535,8537,8540,8542,8545,8548,8551,8554],{"class":392,"line":173},[390,8536,418],{"class":396},[390,8538,8539],{"class":464}," shadow",[390,8541,425],{"class":396},[390,8543,8544],{"class":400}," host.",[390,8546,8547],{"class":421},"attachShadow",[390,8549,8550],{"class":400},"({ mode: ",[390,8552,8553],{"class":407},"'open'",[390,8555,8556],{"class":400}," })\n",[390,8558,8559,8561,8564,8566,8568,8571,8573,8576],{"class":392,"line":16},[390,8560,418],{"class":396},[390,8562,8563],{"class":464}," span",[390,8565,425],{"class":396},[390,8567,3021],{"class":400},[390,8569,8570],{"class":421},"createElement",[390,8572,449],{"class":400},[390,8574,8575],{"class":407},"'span'",[390,8577,455],{"class":400},[390,8579,8580,8583,8585],{"class":392,"line":440},[390,8581,8582],{"class":400},"span.textContent ",[390,8584,893],{"class":396},[390,8586,8587],{"class":407}," \"I'm in the shadow DOM\"\n",[390,8589,8590,8593,8596],{"class":392,"line":458},[390,8591,8592],{"class":400},"shadow.",[390,8594,8595],{"class":421},"appendChild",[390,8597,8598],{"class":400},"(span)\n",[11,8600,8601,8602,8605],{},"From a user perspective, they will not notice any different in the ",[387,8603,8604],{},"\u003Cspan>"," that has been created on the page, but under the hood this implementation will allow you to fully take advantage of the benefits that Shadow DOM provides.",[818,8607,8609],{"id":8608},"encapsulation","Encapsulation",[11,8611,8612,8613,8617],{},"Having the ability to ensure that CSS and JavaScript on the page does not impact your custom element can be a powerful tool to ensuring that the elements inside your Shadow DOM will operate as expected. Let's look at the example below (",[50,8614,5244],{"href":8615,"rel":8616},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FAPI\u002FWeb_components\u002FUsing_shadow_DOM#encapsulation_from_javascript",[54],"):",[381,8619,8621],{"className":383,"code":8620,"language":385,"meta":15,"style":15},"const host = document.querySelector('#host')\nconst shadow = host.attachShadow({ mode: 'open' })\nconst span = document.createElement('span')\nspan.textContent = \"I'm in the shadow DOM\"\nshadow.appendChild(span)\n\nconst upper = document.querySelector('button#upper')\nupper.addEventListener('click', () => {\n  const spans = Array.from(document.querySelectorAll('span'))\n  for (const span of spans) {\n    span.textContent = span.textContent.toUpperCase()\n  }\n})\n\nconst reload = document.querySelector('#reload')\nreload.addEventListener('click', () => document.location.reload())\n",[387,8622,8623,8641,8659,8677,8685,8693,8697,8717,8736,8762,8778,8793,8797,8801,8805,8825],{"__ignoreMap":15},[390,8624,8625,8627,8629,8631,8633,8635,8637,8639],{"class":392,"line":393},[390,8626,418],{"class":396},[390,8628,8519],{"class":464},[390,8630,425],{"class":396},[390,8632,3021],{"class":400},[390,8634,4945],{"class":421},[390,8636,449],{"class":400},[390,8638,8530],{"class":407},[390,8640,455],{"class":400},[390,8642,8643,8645,8647,8649,8651,8653,8655,8657],{"class":392,"line":173},[390,8644,418],{"class":396},[390,8646,8539],{"class":464},[390,8648,425],{"class":396},[390,8650,8544],{"class":400},[390,8652,8547],{"class":421},[390,8654,8550],{"class":400},[390,8656,8553],{"class":407},[390,8658,8556],{"class":400},[390,8660,8661,8663,8665,8667,8669,8671,8673,8675],{"class":392,"line":16},[390,8662,418],{"class":396},[390,8664,8563],{"class":464},[390,8666,425],{"class":396},[390,8668,3021],{"class":400},[390,8670,8570],{"class":421},[390,8672,449],{"class":400},[390,8674,8575],{"class":407},[390,8676,455],{"class":400},[390,8678,8679,8681,8683],{"class":392,"line":440},[390,8680,8582],{"class":400},[390,8682,893],{"class":396},[390,8684,8587],{"class":407},[390,8686,8687,8689,8691],{"class":392,"line":458},[390,8688,8592],{"class":400},[390,8690,8595],{"class":421},[390,8692,8598],{"class":400},[390,8694,8695],{"class":392,"line":482},[390,8696,413],{"emptyLinePlaceholder":24},[390,8698,8699,8701,8704,8706,8708,8710,8712,8715],{"class":392,"line":494},[390,8700,418],{"class":396},[390,8702,8703],{"class":464}," upper",[390,8705,425],{"class":396},[390,8707,3021],{"class":400},[390,8709,4945],{"class":421},[390,8711,449],{"class":400},[390,8713,8714],{"class":407},"'button#upper'",[390,8716,455],{"class":400},[390,8718,8719,8722,8725,8727,8730,8732,8734],{"class":392,"line":500},[390,8720,8721],{"class":400},"upper.",[390,8723,8724],{"class":421},"addEventListener",[390,8726,449],{"class":400},[390,8728,8729],{"class":407},"'click'",[390,8731,1808],{"class":400},[390,8733,434],{"class":396},[390,8735,437],{"class":400},[390,8737,8738,8740,8743,8745,8748,8750,8753,8756,8758,8760],{"class":392,"line":514},[390,8739,461],{"class":396},[390,8741,8742],{"class":464}," spans",[390,8744,425],{"class":396},[390,8746,8747],{"class":400}," Array.",[390,8749,404],{"class":421},[390,8751,8752],{"class":400},"(document.",[390,8754,8755],{"class":421},"querySelectorAll",[390,8757,449],{"class":400},[390,8759,8575],{"class":407},[390,8761,2549],{"class":400},[390,8763,8764,8766,8768,8770,8772,8775],{"class":392,"line":522},[390,8765,7792],{"class":396},[390,8767,3123],{"class":400},[390,8769,418],{"class":396},[390,8771,8563],{"class":464},[390,8773,8774],{"class":396}," of",[390,8776,8777],{"class":400}," spans) {\n",[390,8779,8780,8783,8785,8788,8791],{"class":392,"line":544},[390,8781,8782],{"class":400},"    span.textContent ",[390,8784,893],{"class":396},[390,8786,8787],{"class":400}," span.textContent.",[390,8789,8790],{"class":421},"toUpperCase",[390,8792,541],{"class":400},[390,8794,8795],{"class":392,"line":563},[390,8796,749],{"class":400},[390,8798,8799],{"class":392,"line":568},[390,8800,1090],{"class":400},[390,8802,8803],{"class":392,"line":575},[390,8804,413],{"emptyLinePlaceholder":24},[390,8806,8807,8809,8812,8814,8816,8818,8820,8823],{"class":392,"line":603},[390,8808,418],{"class":396},[390,8810,8811],{"class":464}," reload",[390,8813,425],{"class":396},[390,8815,3021],{"class":400},[390,8817,4945],{"class":421},[390,8819,449],{"class":400},[390,8821,8822],{"class":407},"'#reload'",[390,8824,455],{"class":400},[390,8826,8827,8830,8832,8834,8836,8838,8840,8843,8846],{"class":392,"line":608},[390,8828,8829],{"class":400},"reload.",[390,8831,8724],{"class":421},[390,8833,449],{"class":400},[390,8835,8729],{"class":407},[390,8837,1808],{"class":400},[390,8839,434],{"class":396},[390,8841,8842],{"class":400}," document.location.",[390,8844,8845],{"class":421},"reload",[390,8847,8848],{"class":400},"())\n",[11,8850,8851,8852,8854,8855,8857,8858,8860],{},"You'll note that there is a ",[387,8853,8604],{}," created inside a newly created Shadow DOM, this will appear on the page as normal alongside any other ",[387,8856,8604],{}," elements on the page. An event listener has been added to a button that will cause any ",[387,8859,8604],{}," in the document to have it's text content changed to uppercase - however, as the Shadow DOM is encapsulated, this will not effect the content inside of the Shadow DOM.",[11,8862,8863],{},"This concept also applies to CSS styles that have been applied to the main document - they will not impact your content that has been placed inside of a Shadow DOM. In order to modify the CSS inside of the Shadow DOM, you would need to add a bit more code to it, such as:",[249,8865,8866,8873],{},[252,8867,8868,8869,8872],{},"Creating a ",[387,8870,8871],{},"CSSStyleSheet"," object programmatically.",[252,8874,8875,8876,8879,8880,2906],{},"Including a ",[387,8877,8878],{},"\u003Cstyle>"," tag inside of your ",[387,8881,8882],{},"\u003Ctemplate>",[11,8884,8885,8886,13],{},"You can read more about both of these methods here: ",[50,8887,8890],{"href":8888,"rel":8889},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FAPI\u002FWeb_components\u002FUsing_shadow_DOM#applying_styles_inside_the_shadow_dom",[54],"developer.mozilla.org",[43,8892,8894],{"id":8893},"custom-elements","Custom Elements",[11,8896,8897],{},"When combined with custom elements, Shadow DOMs create a useful framework for developing reusable elements that can level up your website, as well as make them more maintainable. Custom elements share a lot of the same characteristics of the Shadow DOM, let's look at them.",[11,8899,8900,8902],{},[5176,8901,8451],{}," as custom elements have the ability to encapsulate the HTML, CSS, and JavaScript that has been used to create them. This allows for the creation of self-contained elements that are generally more maintainable. Furthermore, this means that it is a lot more difficult for the component to be accidentally modified by other elements that are being manipulated in the main document.",[11,8904,8905,8908],{},[5176,8906,8907],{},"Reusability:"," as they are self-contained, custom elements can be rapidly reused within a webpage. This means that if you have a webpage with multiple versions of a similar component, you may be able to create a single custom element that can be used to handle the different states with differing information relevant to the document.",[11,8910,8911,8913,8914,8916,8917,8919],{},[5176,8912,8457],{}," styles can be applied to the specific custom element, rather than the document as a whole. This means that if you wish for the ",[387,8915,8604],{}," element to have a specific style, you can add this into your custom element and it would not effect the rest of the ",[387,8918,8604],{}," elements in the main document.",[11,8921,8922,8923,8926,8927,8930],{},"Custom elements can be created to extend the features of HTML elements, customising them to the web developers requirements. Behaviours can be defined by the web developer. They can either extend a specific HTML element, such as ",[387,8924,8925],{},"HTMLImageElement",", or just generally extend ",[387,8928,8929],{},"HTMLElement"," to create an autonomous custom element. They are developed using JavaScript. For example,",[381,8932,8934],{"className":383,"code":8933,"language":385,"meta":15,"style":15},"class CustomParagraph extends HTMLParagraphElement {\n  constructor() {\n    super()\n  }\n  \u002F\u002F Element functionality written in here\n}\n",[387,8935,8936,8952,8959,8966,8970,8975],{"__ignoreMap":15},[390,8937,8938,8941,8944,8947,8950],{"class":392,"line":393},[390,8939,8940],{"class":396},"class",[390,8942,8943],{"class":622}," CustomParagraph",[390,8945,8946],{"class":396}," extends",[390,8948,8949],{"class":464}," HTMLParagraphElement",[390,8951,437],{"class":400},[390,8953,8954,8957],{"class":392,"line":173},[390,8955,8956],{"class":396},"  constructor",[390,8958,5262],{"class":400},[390,8960,8961,8964],{"class":392,"line":16},[390,8962,8963],{"class":464},"    super",[390,8965,541],{"class":400},[390,8967,8968],{"class":392,"line":440},[390,8969,749],{"class":400},[390,8971,8972],{"class":392,"line":458},[390,8973,8974],{"class":571},"  \u002F\u002F Element functionality written in here\n",[390,8976,8977],{"class":392,"line":482},[390,8978,755],{"class":400},[11,8980,8981],{},"Custom elements and Shadow DOM go hand-in-hand - without it, custom elements would be fragile and could break with other changes to the main document of the website. For example, selectors within the custom element would be prone to changes using CSS or JS within the main document, breaking it's functionality and rendering it inoperable.",[43,8983,8985],{"id":8984},"shadow-dom-and-scraping","Shadow DOM and scraping",[11,8987,8988],{},"The encapsulated nature of Shadow DOM makes it difficult for web scraping. Standard selectors, designed for the main document, cannot penetrate the shadow tree, requiring specialized methods to access its content. Let's look at some code that can be used to access the shadow tree.",[381,8990,8992],{"className":383,"code":8991,"language":385,"meta":15,"style":15},"const shadowHost = document.querySelector('\u003CSELECTOR>')\n\nif (shadowHost) {\n  const shadowRoot = shadowHost.shadowRoot\n\n  \u002F\u002F Locate the element inside the shadow root that you wish to interact with.\n  const innerElm = shadowRoot.querySelector('\u003CSELECTOR>')\n\n  if (innerElm) {\n    \u002F\u002F Example: innerElm.click();\n  }\n}\n",[387,8993,8994,9014,9018,9026,9038,9042,9047,9067,9071,9079,9084,9088],{"__ignoreMap":15},[390,8995,8996,8998,9001,9003,9005,9007,9009,9012],{"class":392,"line":393},[390,8997,418],{"class":396},[390,8999,9000],{"class":464}," shadowHost",[390,9002,425],{"class":396},[390,9004,3021],{"class":400},[390,9006,4945],{"class":421},[390,9008,449],{"class":400},[390,9010,9011],{"class":407},"'\u003CSELECTOR>'",[390,9013,455],{"class":400},[390,9015,9016],{"class":392,"line":173},[390,9017,413],{"emptyLinePlaceholder":24},[390,9019,9020,9023],{"class":392,"line":16},[390,9021,9022],{"class":396},"if",[390,9024,9025],{"class":400}," (shadowHost) {\n",[390,9027,9028,9030,9033,9035],{"class":392,"line":440},[390,9029,461],{"class":396},[390,9031,9032],{"class":464}," shadowRoot",[390,9034,425],{"class":396},[390,9036,9037],{"class":400}," shadowHost.shadowRoot\n",[390,9039,9040],{"class":392,"line":458},[390,9041,413],{"emptyLinePlaceholder":24},[390,9043,9044],{"class":392,"line":482},[390,9045,9046],{"class":571},"  \u002F\u002F Locate the element inside the shadow root that you wish to interact with.\n",[390,9048,9049,9051,9054,9056,9059,9061,9063,9065],{"class":392,"line":494},[390,9050,461],{"class":396},[390,9052,9053],{"class":464}," innerElm",[390,9055,425],{"class":396},[390,9057,9058],{"class":400}," shadowRoot.",[390,9060,4945],{"class":421},[390,9062,449],{"class":400},[390,9064,9011],{"class":407},[390,9066,455],{"class":400},[390,9068,9069],{"class":392,"line":500},[390,9070,413],{"emptyLinePlaceholder":24},[390,9072,9073,9076],{"class":392,"line":514},[390,9074,9075],{"class":396},"  if",[390,9077,9078],{"class":400}," (innerElm) {\n",[390,9080,9081],{"class":392,"line":522},[390,9082,9083],{"class":571},"    \u002F\u002F Example: innerElm.click();\n",[390,9085,9086],{"class":392,"line":544},[390,9087,749],{"class":400},[390,9089,9090],{"class":392,"line":563},[390,9091,755],{"class":400},[11,9093,9094],{},"While we acknowledge that the method above is more technical than some users may wish, this is currently the best option for interacting with Shadow DOM in your axiom.ai automations.",[43,9096,9098],{"id":9097},"shadow-dom-and-page-interactions","Shadow DOM and page interactions",[11,9100,9101],{},"For the same reasons mentioned in the scraping section above, programmatically interacting with a page that makes use of Shadow DOM can be difficult for most automation tools. For example, if we ask a script to click a button that has the ID \"button\", it will fail to find this in the main document of the page where it would be looking - due to the additional requirements, it will often not check the shadow tree for this element as this would slow down any script. There are a few methods of getting around this, such as:",[249,9103,9104,9112,9115],{},[252,9105,9106,9107,9111],{},"Using the keyboard to navigate on the page - see the ",[50,9108,9110],{"href":9109},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fpress-key","Press Key(s)"," step in axiom.ai.",[252,9113,9114],{},"Using code, see the example above.",[252,9116,9117],{},"Clicking on the page using coordinates - this isn't really recommended as elements may move depending on the size of the window that the website is open on.",[43,9119,166],{"id":165},[11,9121,9122],{},"The Shadow DOM is a vital tool for building modern, robust web components. It provides crucial encapsulation, isolating styles and DOM structure to prevent conflicts and enhance maintainability. We explored how to create and manipulate shadow trees with JavaScript, showcasing its benefits for style scoping and code isolation.",[11,9124,9125],{},"We also discussed the powerful combination of Shadow DOM and custom elements, highlighting their role in creating reusable and modular components. Finally, we addressed the challenges of web scraping within Shadow DOM, offering a practical approach to accessing encapsulated elements.",[11,9127,9128],{},"By mastering the Shadow DOM, you gain the ability to create cleaner, more maintainable, and scalable web applications. It's an essential skill for any developer aiming to build high-quality web experiences.",[2666,9130,9131],{},"html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sjgCt, html code.shiki .sjgCt{--shiki-default:#116329;--shiki-dark:#7EE787}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .sTDnQ, html code.shiki .sTDnQ{--shiki-default:#953800;--shiki-dark:#FFA657}html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}",{"title":15,"searchDepth":16,"depth":16,"links":9133},[9134,9135,9136,9140,9141,9142,9143],{"id":8435,"depth":173,"text":8436},{"id":8442,"depth":173,"text":8443},{"id":8461,"depth":173,"text":8462,"children":9137},[9138,9139],{"id":8468,"depth":16,"text":8469},{"id":8608,"depth":16,"text":8609},{"id":8893,"depth":173,"text":8894},{"id":8984,"depth":173,"text":8985},{"id":9097,"depth":173,"text":9098},{"id":165,"depth":173,"text":166},"2025-04-10","Learn how the Shadow DOM enhances web development by providing encapsulation for HTML, CSS, and JavaScript. Build reusable, modular web components and prevent style conflicts.",{"read":352,"type":387,"tool":9147,"category":9148,"tags":9149,"featuredimg":9154,"landingimg":9155},[354],[2686],[9150,9151,9152,9153],"shadow dom","scoping","host","dom","\u002Fshadow-dom-post.webp","\u002Fshadow-dom-sq.webp","\u002Fblog\u002Funderstanding-and-working-with-the-shadow-dom",{"title":8430,"description":9145},"blog\u002Funderstanding-and-working-with-the-shadow-dom","PuZ6eNLzRVWeFw92EXLJ38gWrLAN9ky8enRNIElrHk4",{"id":9161,"title":9162,"author":23,"body":9163,"date":9505,"description":9506,"draft":181,"extension":19,"meta":9507,"navigation":24,"path":9515,"seo":9516,"stem":9517,"__hash__":9518},"blog\u002Fblog\u002Fhow-axiom-protects-your-data.md","How axiom.ai Protects Your Data",{"type":8,"value":9164,"toc":9488},[9165,9168,9171,9173,9180,9184,9191,9306,9309,9315,9319,9322,9325,9329,9332,9336,9342,9346,9349,9353,9361,9381,9385,9388,9399,9402,9405,9409,9416,9421,9425,9428,9432,9438,9442,9451,9458,9462,9465,9474,9476,9479,9485],[11,9166,9167],{},"Protecting your data is a high priority for us here at axiom.ai. We store the code to run your automations, not the data that you process with them. Your data is your own and is never stored, sold, or used to train AI models.",[11,9169,9170],{},"Let's dive into how your data has been handled within your automations, what is stored and what is not stored as well as how to ensure that your use of axiom.ai in your projects complies with any regulations that your organisation may be required to adhere to.",[39,9172],{},[11,9174,9175,9176,13],{},"For more details on specifics, see our ",[50,9177,9179],{"href":9178},"\u002Fprivacy-policy","privacy policy",[43,9181,9183],{"id":9182},"how-step-data-is-stored","How step data is stored",[11,9185,9186,9187,9190],{},"When you create an automation, we store the data for the steps that make up your automation. This includes any of the settings that you have enabled within your automation. For example, if you have an \"Enter text\" step in your automation that is set to input the text \"Hello, World!\", we are going to store the text and any of the other options that you have enabled. In the exported automation, this will appear in the ",[387,9188,9189],{},"params"," of the step as follows:",[381,9192,9194],{"className":6843,"code":9193,"language":4195,"meta":15,"style":15},"{\n  \"collapsible\": 0,\n  \"configurable\": true,\n  \"default_value\": \"\",\n  \"description\": [\"Input the text to enter.\"],\n  \"help\": [],\n  \"image\": \"h-text-input\",\n  \"name\": \"Text\",\n  \"type\": \"long_text_required\",\n  \"value\": \"Hello, World!\"\n}\n",[387,9195,9196,9200,9211,9223,9234,9248,9256,9268,9280,9292,9302],{"__ignoreMap":15},[390,9197,9198],{"class":392,"line":393},[390,9199,6851],{"class":400},[390,9201,9202,9205,9207,9209],{"class":392,"line":173},[390,9203,9204],{"class":7078},"  \"collapsible\"",[390,9206,5310],{"class":400},[390,9208,7847],{"class":464},[390,9210,491],{"class":400},[390,9212,9213,9216,9218,9221],{"class":392,"line":16},[390,9214,9215],{"class":7078},"  \"configurable\"",[390,9217,5310],{"class":400},[390,9219,9220],{"class":464},"true",[390,9222,491],{"class":400},[390,9224,9225,9228,9230,9232],{"class":392,"line":440},[390,9226,9227],{"class":7078},"  \"default_value\"",[390,9229,5310],{"class":400},[390,9231,6862],{"class":407},[390,9233,491],{"class":400},[390,9235,9236,9239,9242,9245],{"class":392,"line":458},[390,9237,9238],{"class":7078},"  \"description\"",[390,9240,9241],{"class":400},": [",[390,9243,9244],{"class":407},"\"Input the text to enter.\"",[390,9246,9247],{"class":400},"],\n",[390,9249,9250,9253],{"class":392,"line":482},[390,9251,9252],{"class":7078},"  \"help\"",[390,9254,9255],{"class":400},": [],\n",[390,9257,9258,9261,9263,9266],{"class":392,"line":494},[390,9259,9260],{"class":7078},"  \"image\"",[390,9262,5310],{"class":400},[390,9264,9265],{"class":407},"\"h-text-input\"",[390,9267,491],{"class":400},[390,9269,9270,9273,9275,9278],{"class":392,"line":500},[390,9271,9272],{"class":7078},"  \"name\"",[390,9274,5310],{"class":400},[390,9276,9277],{"class":407},"\"Text\"",[390,9279,491],{"class":400},[390,9281,9282,9285,9287,9290],{"class":392,"line":514},[390,9283,9284],{"class":7078},"  \"type\"",[390,9286,5310],{"class":400},[390,9288,9289],{"class":407},"\"long_text_required\"",[390,9291,491],{"class":400},[390,9293,9294,9297,9299],{"class":392,"line":522},[390,9295,9296],{"class":7078},"  \"value\"",[390,9298,5310],{"class":400},[390,9300,9301],{"class":407},"\"Hello, World!\"\n",[390,9303,9304],{"class":392,"line":544},[390,9305,755],{"class":400},[11,9307,9308],{},"If you have used a data token within the step, you will see this here, too. This data, along with all other steps, are stored within our cloud services ensure availability when you need them - your data is encrypted at rest and in transit.",[11,9310,9311,9312,9314],{},"If you are concerned about how your data is being stored within your automations, we recommend making use of data tokens and an external service to pass data into your automation. Going back to the example of the \"Enter text\" step, you could use a ",[50,9313,5710],{"href":5709}," step to read in the data that you wish to enter in the textfield, when exporting your automation, you would only see the data token that you have input into the step.",[818,9316,9318],{"id":9317},"exporting-your-automations","Exporting your automations",[11,9320,9321],{},"When contacting our support team it is quite common for us to ask for a copy of your automation so that we can assess it for any issues that you are experiencing. For additional security purposes, we limit internal access to your automations. When requesting a copy of your automation we are aware that there are times when you may be hesitant to share due to the sensitivity of the data that you are working with - unless shared with us, we will not have access to the data that you are working with apart from the data that you have stored in steps. We will not have access to the Google or Excel sheets that you are using.",[11,9323,9324],{},"If your automation contains credentials, we would recommend removing these - we generally recommend storing these elsewhere regardless of whether or not you are sharing your automation. There are instances where we may need additional access but we will discuss this with your directly in the event of contacting our team. If there are credentials present in the automation when shared, our team will always request permission before running the automation - but would always rather a test account be set up for the team where possible.",[43,9326,9328],{"id":9327},"how-processed-data-is-stored","How processed data is stored",[11,9330,9331],{},"Understanding how data is stored when being processed by an automation is key to ensuring that you have met the information security requirements that you may have for your project.",[818,9333,9335],{"id":9334},"running-your-automations-locally","Running your automations locally",[11,9337,9338,9339,9341],{},"When you make use of the desktop application to run your automations locally the data that you process with your automation will ",[5176,9340,2788],{}," leave your local computer, or server - unless you have steps specifically to send this data. This can be massively beneficial when you are working with sensitive data that is not shareable outside of your organisation, for example, when working with PII or medical data. If required, the axiom.ai desktop application can be installed on a server to be used.",[818,9343,9345],{"id":9344},"running-in-the-cloud","Running in the cloud",[11,9347,9348],{},"When running in the cloud, we only store the data that is processed by your automation for the amount of time that your automation runs. Once your run has finished, no matter what the state of that finish is, the data will be erased from the \"pod\" that your automation has run in. Your automation is only able to access data that has been created during the run.",[818,9350,9352],{"id":9351},"things-to-note","Things to note",[11,9354,9355,9356,9360],{},"It's important to note that ",[50,9357,9359],{"href":9358},"\u002Fdocs\u002Fno-code-tool\u002Ftroubleshooting\u002Fhow-to-debug#run-reports","Run Reports"," will be stored for each run and this data is stored within axiom.ai's servers. In the event of your automation running into an error, it's possible that some data could be included in these error messages. For example, if a \"Click element\" step is unable to find the selector that you have selected, this selector may be included in the error message.",[11,9362,9363,9364,9368,9369,285,9371,9373,9374,9377,9378,9380],{},"Similarly, if you make use of the ",[50,9365,9367],{"href":9366},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fadd-error-metadata","Add error metadata"," step and include your data, or data tokens, in this field, then this will be included in the Run Reports. An alternative to this may be to use the ",[50,9370,6309],{"href":6308},[50,9372,5576],{"href":5575}," to catch errors and report them to your own systems rather than let the automation run into errors. See our documentation on ",[50,9375,9376],{"href":7436},"handling errors"," for more tips. You could also make use of the ",[50,9379,6472],{"href":5474}," to pass in credentials, if you are on an appropriate axiom.ai plan.",[818,9382,9384],{"id":9383},"third-party-services","Third-party services",[11,9386,9387],{},"We provide a few steps that allow you to directly interact with third-party services, such as:",[249,9389,9390,9393,9395,9397],{},[252,9391,9392],{},"OpenAI (ChatGPT)",[252,9394,114],{},[252,9396,284],{},[252,9398,290],{},[11,9400,9401],{},"When you use any of these steps to interact with these services, the data that you set to be shared with this service will be shared to provide the service. For example, the prompt that you input into the ChatGPT steps will be shared, along with your API key that has been set within the step or in the settings. You should refer to their privacy policies for more details on how your data is managed within those services.",[11,9403,9404],{},"We do not share any usage data with these service providers, only the data required to perform the action, and the data that you have input.",[43,9406,9408],{"id":9407},"connecting-your-accounts","Connecting your accounts",[11,9410,9411,9412,9415],{},"To allow you to make use of certain steps, you have the ability to connect certain accounts to your axiom.ai account. For example, you can connect your Google or Microsoft (work or school) account to be able to use their respective steps. When you use this feature to connect to your account, you will be prompted to use their native login flow to connect your account - when you do this, we do ",[5176,9413,9414],{},"not"," store your login credentials. These services provide us with an access token that can be used to interact with these services.",[11,9417,9418],{},[2786,9419,9420],{},"Note, as part of their security features, these services can revoke these access tokens at any point for reasons that they do not publish.",[43,9422,9424],{"id":9423},"storing-api-keys","Storing API keys",[11,9426,9427],{},"Some services that we allow you to interact with within your automations require an API key to be used, these can be stored within the automation itself but we would recommend storing these in our \"External API keys\" section within your account. These keys are encrypted at rest and in transit within our systems, ensuring that they remain private and only for your use. When used within a step, this will not be included in the export of your automation.",[43,9429,9431],{"id":9430},"storing-cookies","Storing cookies",[11,9433,205,9434,9437],{},[50,9435,9436],{"href":4005},"Store cookies"," feature allows you to sync your local cookies with your automation to carry them over to your cloud browser session, this can be massively beneficial to carry over login sessions. These cookies are encrypted and will not be included in the export of the automation.",[43,9439,9441],{"id":9440},"regulatory-compliance","Regulatory compliance",[11,9443,9444,9445,9450],{},"axiom.ai always strives to protect your data in line with information security standards and regulations, and has a ",[50,9446,9449],{"href":9447,"rel":9448},"https:\u002F\u002Fappdefensealliance.dev\u002Fcasa",[54],"Cloud App Security Assessment (CASA)"," Tier II certification. However, if your organisation requires you to adhere to specific standards and regulations while using third-party services, there are still methods of using axiom.ai within these limits.",[11,9452,9453,9454,9457],{},"As discussed above in ",[50,9455,9335],{"href":9456},"#running-your-automations-locally",", your data never leaves your device or network when being process on a locally run automation. Only the data stored in the steps themselves will be stored by axiom.ai. This means that you can maintain compliance with local regulations such as HIPAA (US), GDPR (EU), DPA (UK), PIPEDA (CAN) and the Fair Information Processing Principles (US).",[43,9459,9461],{"id":9460},"tips","Tips",[11,9463,9464],{},"If you are concerned with storing login credentials within your automations, consider storing them in Google Sheets and then importing them into your automation using the \"Read data from a Google Sheet\" step or the \"Read data from an Excel\" step. This would mean that they are imported each time that the automation runs rather than being stored in the automation.",[11,9466,9467,9468,9470,9471,9473],{},"Alternatively, you can use the ",[50,9469,9436],{"href":4005}," option within your automation to store the authentication token from sites that you are already logged into ony our desktop. For most sites, this will mean that you do not need to log into the site when your automation runs. When running locally, your automation will automatically use your local cookies. We recommend setting up ",[50,9472,7373],{"href":7480}," when using this method as your automation may run into errors if the site revokes the authentication cookie at any time.",[43,9475,166],{"id":165},[11,9477,9478],{},"Understanding how axiom.ai handles your data is key to ensuring security and compliance with your organization's regulations. While we store step configuration data to keep your automations available, processed data is only stored temporarily when running in the cloud and never leaves your device when running locally. By leveraging data tokens, external storage, and local execution, you can take full control over how sensitive data is handled within your workflows.",[11,9480,9481,9482,9484],{},"If you have any concerns about data storage or security, we encourage you to review our ",[50,9483,9179],{"href":9178}," or reach out to our support team for further guidance.",[2666,9486,9487],{},"html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sjgCt, html code.shiki .sjgCt{--shiki-default:#116329;--shiki-dark:#7EE787}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":9489},[9490,9493,9499,9500,9501,9502,9503,9504],{"id":9182,"depth":173,"text":9183,"children":9491},[9492],{"id":9317,"depth":16,"text":9318},{"id":9327,"depth":173,"text":9328,"children":9494},[9495,9496,9497,9498],{"id":9334,"depth":16,"text":9335},{"id":9344,"depth":16,"text":9345},{"id":9351,"depth":16,"text":9352},{"id":9383,"depth":16,"text":9384},{"id":9407,"depth":173,"text":9408},{"id":9423,"depth":173,"text":9424},{"id":9430,"depth":173,"text":9431},{"id":9440,"depth":173,"text":9441},{"id":9460,"depth":173,"text":9461},{"id":165,"depth":173,"text":166},"2025-03-27","Learn how axiom.ai handles your data within automations, including what is stored, how processed data is managed, and best practices for security and compliance. Discover how to protect sensitive information while automating workflows with axiom.ai.",{"read":2682,"type":184,"tool":9508,"category":9509,"tags":9510,"featuredimg":9513,"landingimg":9514,"summary":9506,"video":18,"metaTitle":9162},[354],[2686],[9511,9512],"security","privacy","\u002Fblog\u002Fsafe-data-post.webp","\u002Fblog\u002Fsafe-data-sq.webp","\u002Fblog\u002Fhow-axiom-protects-your-data",{"title":9162,"description":9506},"blog\u002Fhow-axiom-protects-your-data","IKdx3nv4pfY6Uyv1lQg4fu8hXCuk5kglNbPwDN2BTxY",{"id":9520,"title":9521,"author":23,"body":9522,"date":9727,"description":9728,"draft":181,"extension":19,"meta":9729,"navigation":24,"path":9739,"seo":9740,"stem":9741,"__hash__":9742},"blog\u002Fblog\u002Fcost-effective-ai.md","Cost-Effective AI - Smart Strategies for Maximising Value",{"type":8,"value":9523,"toc":9717},[9524,9527,9532,9534,9538,9541,9558,9561,9565,9568,9571,9576,9589,9593,9596,9599,9604,9607,9612,9615,9619,9622,9626,9629,9640,9643,9654,9663,9674,9678,9687,9705,9707,9714],[11,9525,9526],{},"With the use of AI becoming ubiquitous in personal and work lives, understanding how to take advantage of it without it costing an arm and a leg can be beneficial to ensuring that you are maximising the value that you are getting out of it. Using AI can save yourself, or your organisation, a lot of time and effort but used inefficiently can lead to value loss.",[791,9528,9529],{},[11,9530,9531],{},"💡 Note, we are going to use the term \"AI\" in this article to refer generally to LLMs, GenAI and other models.",[39,9533],{},[43,9535,9537],{"id":9536},"how-ai-can-help-your-organisation","How AI can help your organisation",[11,9539,9540],{},"AI can be used to navigate around tasks that usually would require a large number of man-hours to either complete, or develop a system to complete the task. This can reduce it down to a few minutes with the help of AI. For example:",[249,9542,9543,9546,9549,9552,9555],{},[252,9544,9545],{},"Speeding up code development.",[252,9547,9548],{},"Automating customer service.",[252,9550,9551],{},"Data analytics.",[252,9553,9554],{},"Generating marketing copy.",[252,9556,9557],{},"Product innovation.",[11,9559,9560],{},"All of the use cases above can be used with the currently available tools, such as ChatGPT, Gemini, Copilot and Le Chat.",[43,9562,9564],{"id":9563},"understanding-the-cost-of-ai","Understanding the cost of AI",[11,9566,9567],{},"When using commercial products (such as ChatGPT, Gemini, Copilot and Le Chat) to power your AI tasks it's important to understand how (most) products charge for the use of their services. If you are using the standard web or app interface to interact with the AI, your usage is often restricted based on the subscription tier that you have subscribed to.",[11,9569,9570],{},"If you are using an API to access the AI, you'll be charged based on \"tokens\". \"Tokens\" are used to measure the data that you are sending to the API and are used to charge users. OpenAI states on their website:",[791,9572,9573],{},[11,9574,9575],{},"You can think of tokens as pieces of words, where 1,000 tokens is about 750 words.",[11,9577,9578,9579,9582,9583,9588],{},"As of the time of writing, they charge $2.50\u002F1m input tokens. Your input would include your prompt, along with all of the data that you are sending over. For example, the sentence, ",[2786,9580,9581],{},"\"axiom.ai is great!\""," is about 5 tokens - this can be language dependent, too. You can learn more from OpenAi on their definition of a token in their ",[50,9584,9587],{"href":9585,"rel":9586},"https:\u002F\u002Fhelp.openai.com\u002Fen\u002Farticles\u002F4936856-what-are-tokens-and-how-to-count-them",[54],"What are tokens and how to count them"," documentation.",[43,9590,9592],{"id":9591},"prompt-engineering","Prompt engineering",[11,9594,9595],{},"Prompt engineering is the practice of designing and optimising inputs (prompts) to guide AI models to generate desired outputs efficiently and effectively. It's a critical skill for working with LLMs and other generative AI systems. Learning how to craft prompts that not only get the answer that you need, but does so without wasting resources.",[11,9597,9598],{},"Consider the following prompts, both used in ChatGPT using an axiom.ai automation export as an attachment:",[11,9600,9601],{},[2786,9602,9603],{},"\"Can you describe this file?\"",[11,9605,9606],{},"vs.",[11,9608,9609],{},[2786,9610,9611],{},"\"Can you count the number of steps in the automation, returning the value as a number only.\"",[11,9613,9614],{},"The first prompt will give us a full rundown of the file, including information on the keys that are contained within the JSON, the steps and details about them. It'll also give you a description of what the automation does. While this would be helpful in certain situations, you may find that you only need a subset of this data - the second step would give a much shorter response, responding simply with \"4\". Simply defining the output that you wish can be an effective way of ensuring that the prompt is more efficient.",[43,9616,9618],{"id":9617},"alternatives","Alternatives",[11,9620,9621],{},"While using the commerical off the shelf products to fulfil your AI needs, there are alternatives if you are looking to maximise the value of AI to your team, lowering costs but sustaining the helpfulness of it.",[818,9623,9625],{"id":9624},"local-processing","Local processing",[11,9627,9628],{},"If you are concerned about maximising value for your team, you may need to reconsider which parts of a project really need AI. For example, if you are processing a large amount of data and then performing analysis on it using AI, you may wish to break down the project into smaller parts, such as:",[249,9630,9631,9634,9637],{},[252,9632,9633],{},"Data collection",[252,9635,9636],{},"Data cleaning",[252,9638,9639],{},"Data analysis",[11,9641,9642],{},"By doing this, you can then determine which parts are more cost efficient to be done by AI and which may be more cost efficient to be done by the team. An approach such as this may work:",[249,9644,9645,9648,9651],{},[252,9646,9647],{},"Data collection: this could be handled by browser automation, such as axiom.ai, or simply data that has been acquired from a data broker.",[252,9649,9650],{},"Data cleaning: for larger projects, writing a Python script may be more cost efficient than AI, this also gives you much more control over the data that you are going to be inputting in the analysis step.",[252,9652,9653],{},"Data analysis: this is where AI excels, you've already cleaned your data so it will be easy to feed into the AI to gain accurate insights.",[11,9655,9656,9657,9662],{},"Often, processing data or manipulating data through a local script can be more beneficial than doing it through AI as the scripts are often not overly complex, libraries such as ",[50,9658,9661],{"href":9659,"rel":9660},"https:\u002F\u002Fpandas.pydata.org",[54],"pandas"," can help make this a simple task within Python, alternatives are available for R. If you do not have access to engineers, it may be worth a once off conversation with an AI to develop a script that could be used for this task.",[11,9664,9665,9666,9669,9670,9673],{},"Tools like ",[50,9667,210],{"href":3840,"rel":9668},[54]," can also help in manipulating data and preparing it for processing by an AI, see our ",[50,9671,9672],{"href":8135},"Manipulate Data"," steps to learn more.",[818,9675,9677],{"id":9676},"local-llm","Local LLM",[11,9679,9680,9681,9686],{},"Running AI\u002FLLMs locally can be another alternative for your organisation - models, such as ",[50,9682,9685],{"href":9683,"rel":9684},"https:\u002F\u002Fai.meta.com\u002Fblog\u002Fllama-3-2-connect-2024-vision-edge-mobile-devices",[54],"Llama 3.2",", can be run locally on your PC to allow access without having to go through the Internet. This has security benefits as your data does not leave your device, however, it's also worth noting that these do not get updated with live data as it does not have access to the Internet.",[11,9688,9689,9690,1155,9695,828,9700,13],{},"Using an AI locally completely cuts the costs of accessing an commercial AI - replacing these with the cost of hosting the AI. Your IT department may be able to set up a server running this to use within your organisation, or you may run this on a single workstation. There are many existing tools out there that can be used to enable this, such as ",[50,9691,9694],{"href":9692,"rel":9693},"https:\u002F\u002Fjs.langchain.com\u002Fv0.1\u002Fdocs\u002Fuse_cases\u002Fquestion_answering\u002Flocal_retrieval_qa",[54],"Langchain",[50,9696,9699],{"href":9697,"rel":9698},"https:\u002F\u002Follama.com",[54],"Ollama",[50,9701,9704],{"href":9702,"rel":9703},"https:\u002F\u002Flmstudio.ai",[54],"LM Studio",[43,9706,166],{"id":165},[11,9708,9709,9710,9713],{},"Breaking down your projects and considering what parts ",[2786,9711,9712],{},"actually"," need to be done using AI is one of the most efficient ways of maximising value with you and your teams use of AI. Weigh the cost of doing a task manually (or using a script) against the cost of doing it with AI - you may be surprised to find out the differences! Running a local LLM is also definitely an option - and a lot more accessible than most people realise, if you have hardware that is powerful enough to run it.",[11,9715,9716],{},"We are still a long way from AI replacing most jobs, but it is still worth noting that there are areas where AI excels - specifically around data analysis. Even internally, we will use this to get insights on data or help with research on blog articles such as this one, though we rarely rely on the generative nature of these tools.",{"title":15,"searchDepth":16,"depth":16,"links":9718},[9719,9720,9721,9722,9726],{"id":9536,"depth":173,"text":9537},{"id":9563,"depth":173,"text":9564},{"id":9591,"depth":173,"text":9592},{"id":9617,"depth":173,"text":9618,"children":9723},[9724,9725],{"id":9624,"depth":16,"text":9625},{"id":9676,"depth":16,"text":9677},{"id":165,"depth":173,"text":166},"2025-03-20","Discover how to leverage AI efficiently and maximise its value while keeping costs under control",{"read":183,"type":184,"tool":9730,"category":9731,"tags":9732,"featuredimg":9736,"landingimg":9737,"summary":9728,"video":18,"metaTitle":9738},[354],[2686],[9733,9734,9592,9735],"local LLM","LLM","ai","\u002Fblog\u002Fai-cost-effective.webp","\u002Fblog\u002Fai-cost-effective-sq.webp","Cost-Effective AI - Smart Strategies for Maximizing Value","\u002Fblog\u002Fcost-effective-ai",{"title":9521,"description":9728},"blog\u002Fcost-effective-ai","dTQFDKYn7PwWBgYLkmGsU-1bm5BeDiC_Nz1TUaCONLU",{"id":9744,"title":9745,"author":23,"body":9746,"date":10063,"description":10064,"draft":181,"extension":19,"meta":10065,"navigation":24,"path":10072,"seo":10073,"stem":10074,"__hash__":10075},"blog\u002Fblog\u002Fis-ai-suitable-for-large-scale-scraping.md","Is AI Suitable for Large Scale Scraping?",{"type":8,"value":9747,"toc":10052},[9748,9751,9754,9758,9761,9764,9775,9778,9781,9785,9792,9796,9799,9803,9806,9809,9817,9895,9899,9914,9923,9933,9937,9949,9952,9965,9969,9975,9984,9988,9991,9994,9998,10005,10010,10020,10022,10025,10039,10042,10049],[11,9749,9750],{},"With the hype around AI continuing to grow as new technologies are flooding onto the market, it can be tempting to assume that AI is the solution to most problems that you are hoping to solve, including web scraping. There are factors that should be considered when using AI to solve problems, and these should be used in your decision making process.",[39,9752],{"alt":9753},"Is ai suitable for large scale scraping",[43,9755,9757],{"id":9756},"what-is-ai","What is AI?",[11,9759,9760],{},"Artificial intelligence is a blanket term that we have given to any application that can perform tasks that typically require human intelligence - for example, text generation and code development. A key benefit of this for web scraping is the ability to perform analysis on large amounts of data and recognise patterns in the data. In its current state, people often refer to large-language models and generative AI under the umbrella term \"AI\". ChatGPT, Gemini and Copilot are some of the more popular LLMs on the market at the moment.",[11,9762,9763],{},"These AI models are trained using a massive amount of data, including books, web pages, news articles, social media posts and code snippets. Though most developers of these models are quite tight-lipped about how much data is required for training, we can assume that the actual amount if into the tens of terabytes.",[11,9765,9766,9767,828,9771,9774],{},"Currently, axiom.ai offers two steps that allow you to integrate generative AI (GenAI) into your automations - the ",[50,9768,9770],{"href":9769},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fextract-data-with-chatgpt","Extract data with ChatGPT",[50,9772,9773],{"href":7304},"Generate Text with ChatGPT"," steps. This can help you analyse a large set of data that has been scraped from a web page, or help you generate text that you can use with your Interact steps. We plan on expanding this functionality in future updates.",[11,9776,9777],{},"The largest benefit that people are seeing with GenAI is the ability to generate content and code. This can help speed up content or code creation, but should be used sparingly to avoid over-reliance on it. With content creation, it's important to note that the model may not have all of the required context to assist you - for example, currently asking it to create the steps required to create an automation within axiom.ai will give you about 75% of the right answer, however, it will \"create\" new steps that are not part of the tool and may cause additional confusion with users. Being more precise with your prompts can mitigate some of the issues.",[11,9779,9780],{},"When it comes to using it for code generation, again we see a large number of benefits from GenAI - however, there are common pitfalls that catch newer developers. While it can speed up development, we are seeing a large number of developers adding code to their codebase that they may not fully understand. GenAI often will explain the code that it's written, but it's still critical that the developer understands the code that they are copying into a project - this is just like when a developer blindly copies code from StackOverflow! Blindly copying code into your project may get the job done, however, this will decrease the maintainability of your project. Instead, use it as a learning tool - ask the model to explain the code to you so you can learn how to implement it yourself, or at a minimum, so you can modify or add additional comments to the code.",[134,9782],{"src":9783,"alt":9784},"\u002Fblog\u002Fis-ai-suitable-for-large-scale-scraping-code-generation.png","The image contains a screenshot of a conversation where a user asks for a JavaScript function to filter items from an array. The response includes a code snippet defining a filterArray function that uses the .filter() method. The example usage demonstrates filtering even numbers from an array. Below the code, there is an explanation of how the function works, highlighting the use of a callback function for filtering logic. The response concludes with an offer to modify the function if needed.",[11,9786,9787,9788,9791],{},"Both content and code generation can provide benefits when it comes to large scale scraping, including being able to extract ",[50,9789,9790],{"href":5731},"custom CSS selectors"," to use to ensure that you are targeting the correct elements on the page for efficiency. Let's dive into some other considerations.",[43,9793,9795],{"id":9794},"cost","Cost",[11,9797,9798],{},"Using AI as an individual user tends to not be too expensive, however, if you decide to use this for large scale operations, such as scraping, you should consider the cost of using these services. Most services, such as OpenAI (ChatGPT) will offer an API that can be used and charged at a pay-as-you-go rate. \"Tokens\" are used to measure the data that you are sending to the API and are used to charge users. OpenAI states on their website:",[791,9800,9801],{},[11,9802,9575],{},[11,9804,9805],{},"As of the time of writing, they charge $2.50\u002F1m input tokens. Your input would include your prompt, along with all of the data that you are sending over.",[11,9807,9808],{},"Where this becomes expensive is operations like large scale scraping - if you decide to use AI for data manipulation, you may end up sending a lot of data tokens to the service when you are sending the data that you have scraped. For example, if you scrape the \"contact us\" page of a list of URLs and want to extract a list of email addresses you may end up sending a large amount of data inserted inside of your prompt. Recently, this would be considered the easiest and most reliable solution - that's not always the case.",[11,9810,9811,9812,9816],{},"Instead of sending all of the data in a prompt and using a large number of input tokens, consider doing the data manipulation using a short JavaScript script locally or exporting the data to a service such as Google Sheets. Check out our ",[50,9813,9815],{"href":9814},"\u002Fdocs\u002Fdeveloper-hub\u002Fsnippets\u002Fjavascript\u002Fdata#manipulating-data-with-regex","Manipulating data with regex"," snippets to learn more about doing this locally, or in your axiom.ai automation. For our example above, we could use the \"Extracting data\" snippet:",[381,9818,9820],{"className":383,"code":9819,"language":385,"meta":15,"style":15},"\u002F\u002F Let's use data scraped from a website, this contains the string \"115 Records\".\nconst data = '[scrape-data?all&0]';\n\n\u002F\u002F We just want the number of records here, let's extract it with regex.\nconst records = data.match(\u002F\\+d\u002F)[0];\n\n\u002F\u002F Return this to be included in the 'code-data' data token.\nreturn records;\n",[387,9821,9822,9827,9840,9844,9849,9879,9883,9888],{"__ignoreMap":15},[390,9823,9824],{"class":392,"line":393},[390,9825,9826],{"class":571},"\u002F\u002F Let's use data scraped from a website, this contains the string \"115 Records\".\n",[390,9828,9829,9831,9833,9835,9838],{"class":392,"line":173},[390,9830,418],{"class":396},[390,9832,4185],{"class":464},[390,9834,425],{"class":396},[390,9836,9837],{"class":407}," '[scrape-data?all&0]'",[390,9839,5758],{"class":400},[390,9841,9842],{"class":392,"line":16},[390,9843,413],{"emptyLinePlaceholder":24},[390,9845,9846],{"class":392,"line":440},[390,9847,9848],{"class":571},"\u002F\u002F We just want the number of records here, let's extract it with regex.\n",[390,9850,9851,9853,9856,9858,9860,9862,9864,9866,9869,9872,9874,9876],{"class":392,"line":458},[390,9852,418],{"class":396},[390,9854,9855],{"class":464}," records",[390,9857,425],{"class":396},[390,9859,4319],{"class":400},[390,9861,4809],{"class":421},[390,9863,449],{"class":400},[390,9865,2916],{"class":407},[390,9867,9868],{"class":4825},"\\+",[390,9870,9871],{"class":407},"d\u002F",[390,9873,7844],{"class":400},[390,9875,7847],{"class":464},[390,9877,9878],{"class":400},"];\n",[390,9880,9881],{"class":392,"line":482},[390,9882,413],{"emptyLinePlaceholder":24},[390,9884,9885],{"class":392,"line":494},[390,9886,9887],{"class":571},"\u002F\u002F Return this to be included in the 'code-data' data token.\n",[390,9889,9890,9892],{"class":392,"line":500},[390,9891,8009],{"class":396},[390,9893,9894],{"class":400}," records;\n",[43,9896,9898],{"id":9897},"environment","Environment",[11,9900,9901,9902,9907,9908,9913],{},"At its core, it is important to remember that AI models require a lot of resources to train and to query. When we ask AI a question, or ask it to perform a task, this will require computing resources in the data centres that the service are running these models on. In 2018, OpenAI researchers Dario Amodei and Danny Hernandez published an article stating that the computing requirements of training AI has been increasing at a 3.4 month doubling time (",[50,9903,9906],{"href":9904,"rel":9905},"https:\u002F\u002Fopenai.com\u002Findex\u002Fai-and-compute\u002F",[54],"OpenAI",", 2018) - this is massive when compared to ",[50,9909,9912],{"href":9910,"rel":9911},"https:\u002F\u002Fwww.intel.com\u002Fcontent\u002Fwww\u002Fus\u002Fen\u002Fnewsroom\u002Fresources\u002Fmoores-law.html",[54],"Moore's Law"," which has a 2-year doubling period.",[11,9915,9916,9917,9922],{},"Increased computing requirements is expected to continue as these models continue to get trained on an increasing number of parameters, and as \"agents\" start to be become more generally available. It's expected that emissions from the ICT sector as a whole \"will reach 14% of the global emissions, with the majority of those emissions coming from the ICT infrastructure, particularly data centres and communication networks\" (",[50,9918,9921],{"href":9919,"rel":9920},"https:\u002F\u002Fearth.org\u002Fthe-green-dilemma-can-ai-fulfil-its-potential-without-harming-the-environment",[54],"Earth.org",", 2023). This may be mitigated in the future with more efficient code, however, current trends would suggest that this is not going to be solved in the near future.",[11,9924,9925,9926,9930,9931,13],{},"When considering using AI to power up your scraping workflows, you should consider this in addition to your organisations current carbon footprint as using AI in your workflows may significantly increase the carbon footprint of the task you are attempting to automate. An alternative may be to automate these tasks locally, such as running operations on your local PC using axiom.ai, ",[50,9927,9929],{"href":9928},"\u002Fguides\u002Fautomator","Automator",", or ",[50,9932,89],{"href":109},[43,9934,9936],{"id":9935},"technical-challenges","Technical Challenges",[11,9938,9939,9940,9942,9943,9948],{},"Even if you are not a bot wandering through the internet you will have run into a CAPTCHA, or a bot-detection algorithm (such as Cloudflare) that are there to confirm that you are a human. While these are pretty simple to bypass as a human, they are effective at preventing bots from entering the site that they are protecting. This can stop your bot in it's tracks and prevent it from scraping content - on a large scale, this may cause a lot of lost time as it's not always obvious when the bot is stuck at one of these checkpoints. Features like ",[50,9941,3966],{"href":3972}," in axiom.ai, ",[50,9944,9947],{"href":9945,"rel":9946},"https:\u002F\u002Fgithub.com\u002Fberstend\u002Fpuppeteer-extra\u002Ftree\u002Fmaster\u002Fpackages\u002Fpuppeteer-extra-plugin-stealth#readme",[54],"puppeteer-extra-plugin-stealth",", or another Puppeteer plugin can help you get around these limitations.",[11,9950,9951],{},"For large scale scraping, its also recommended to employ efficient proxy management - routing all traffic through a single IP address may lead to the IP address being blacklisted by certain websites and prevent your ability to access them without rotating your proxies. If you are in a large organisation, it may be possible to do this using your current infrastructure, if not, there are plenty of services that offer the ability to purchase residential IPs for a fee, depending on how much traffic you anticipate to be running through them.",[11,9953,9954,9955,828,9959,9964],{},"With frameworks like React, Vue and Angular becoming popular it means that web content can load into a web page dynamically using JavaScript rather than the static pages we are used to. This means that there needs to be mechanisms implemented that can wait for pages to finish loading before continuing with the interaction. Headless browsers such as ",[50,9956,8109],{"href":9957,"rel":9958},"https:\u002F\u002Fpptr.dev",[54],[50,9960,9963],{"href":9961,"rel":9962},"https:\u002F\u002Fplaywright.dev",[54],"Playwright"," can help you with this.",[818,9966,9968],{"id":9967},"page-interactions","Page interactions",[11,9970,9971,9972,9974],{},"More and more commonly, you will come across websites that do not initially reveal all of the data that is present on the page when it loads. The increased popularity of Javascript libraries such as React.js means that data can by dynamically added and removed from pages depending on user interactions with the pages - think of dropdowns, accordions or any other elements that require you to \"click\" to reveal information. Most AI agents, or web crawlers will only scrape data that is currently visible on a page and miss out on any data that is hidden on the page, or requires interaction for the page to be dynamically added. You can get around this by using a script, or tool like ",[50,9973,210],{"href":2916},", to interact with the page prior to scraping to ensure that the data is present on the page, but this can add significant overhead when working on a large scale.",[11,9976,9977,9978,9983],{},"The same concept applies to page loading - when a page is initially loaded, it may decide to ",[50,9979,9982],{"href":9980,"rel":9981},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FPerformance\u002FGuides\u002FLazy_loading",[54],"lazy load"," content to speed up the initial load of the page. This feature is popular with resource heavy files, such as images and video, where they are not loaded until the user scrolls to the part of the page that contains the resource. This can trip up some AI agents and web crawlers for the reasons that were mentioned above - they are not yet present on the page. You can get around this by scrolling the page prior to scraping data from the page, which can, again, add significant overhead when working on a large scale.",[43,9985,9987],{"id":9986},"legal-ethical-concerns","Legal & Ethical Concerns",[11,9989,9990],{},"When building a large-scale scraper, it's essential to consider legal and ethical implications. Always review the terms of service of the websites you're scraping, as violating them could lead to account suspensions or even legal consequences if the target service takes action. In most cases, terms of service explicitly prohibit scraping.",[11,9992,9993],{},"Privacy laws should also be top of mind for larger operations - GDPR (EU) and CCPA (California) restrict automated data collection, especially for personal or sensitive data, while intellectual property laws and policies. Scraping and republishing certain types of data (e.g., news articles, research papers) can raise copyright concerns.",[43,9995,9997],{"id":9996},"recommendation-learn-prompt-engineering","Recommendation: Learn Prompt Engineering",[11,9999,10000,10004],{},[50,10001,9592],{"href":10002,"rel":10003},"https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fprompt-engineering",[54]," is an emerging field within AI, it's defined as:",[791,10006,10007],{},[11,10008,10009],{},"The process of structuring or crafting an instruction in order to produce the best possible output from a generative artificial intelligence (AI) model.",[11,10011,10012,10013,10016,10017,10019],{},"To get the most out of GenAI, you really need to be specific with your instructions. Prompt engineering on the surface seems simple, however, remember that the prompt makes up the input tokens we discussed in the ",[50,10014,9794],{"href":10015},"#cost"," section of this article - you need to ensure that your prompt is as specific ",[2786,10018,2984],{}," as concise as possible to reduce the number of input tokens that your prompt will consume.",[43,10021,166],{"id":165},[11,10023,10024],{},"When used correctly AI has the power to provide efficiency boosts to your organisation when it comes to large-scale scrapers and general operations. It can help speed up development activities, such as writing code or content generation. However, there are certain things that you should take into consideration before committing to using AI for your large-scale scrapers, such as:",[249,10026,10027,10030,10033,10036],{},[252,10028,10029],{},"The cost using AI - specifically surrounding API costs.",[252,10031,10032],{},"The environmental impacts of training and using AI.",[252,10034,10035],{},"The technical challenges - bot blocking, proxies, and dynamically loading content.",[252,10037,10038],{},"The legal & ethical concerns surrounding scraping.",[11,10040,10041],{},"Finally, we would recommend learning prompt engineering to make sure that you are creating prompts that get you the results that meet your requirements, but also reduce the number of input tokens in order to reduce the API costs.",[11,10043,10044,10045,10048],{},"Have some thoughts you'd like to share? Drop a post over on our ",[50,10046,129],{"href":127,"rel":10047},[54],", we would love to hear from you!",[2666,10050,10051],{},"html pre.shiki code .sU953, html code.shiki .sU953{--shiki-default:#6E7781;--shiki-dark:#8B949E}html pre.shiki code .sjeE4, html code.shiki .sjeE4{--shiki-default:#CF222E;--shiki-dark:#FF7B72}html pre.shiki code .sHrmB, html code.shiki .sHrmB{--shiki-default:#0550AE;--shiki-dark:#79C0FF}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sbjLL, html code.shiki .sbjLL{--shiki-default:#8250DF;--shiki-dark:#D2A8FF}html pre.shiki code .sa8KN, html code.shiki .sa8KN{--shiki-default:#116329;--shiki-default-font-weight:bold;--shiki-dark:#7EE787;--shiki-dark-font-weight:bold}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":10053},[10054,10055,10056,10057,10060,10061,10062],{"id":9756,"depth":173,"text":9757},{"id":9794,"depth":173,"text":9795},{"id":9897,"depth":173,"text":9898},{"id":9935,"depth":173,"text":9936,"children":10058},[10059],{"id":9967,"depth":16,"text":9968},{"id":9986,"depth":173,"text":9987},{"id":9996,"depth":173,"text":9997},{"id":165,"depth":173,"text":166},"2025-03-06","Is AI Suitable for Large Scale Scraping? Read more about the considerations that you should be thinking about before committing to using AI.",{"read":2682,"type":184,"tool":10066,"category":10067,"tags":10068,"location":191,"featuredimg":10070,"landingimg":10071,"summary":10064,"video":18},[354],[188],[2839,10069,9735],"data extraction","\u002Fblog\u002Fweb-scraping-at-scale-post-box.webp","\u002Fblog\u002Fweb-scraping-at-scale-sq.webp","\u002Fblog\u002Fis-ai-suitable-for-large-scale-scraping",{"title":9745,"description":10064},"blog\u002Fis-ai-suitable-for-large-scale-scraping","i2HFiqzppHNYaxCZ1OMwGVZBkaQARSlOMyFp7GEsZ78",{"id":10077,"title":10078,"author":23,"body":10079,"date":10316,"description":10317,"draft":181,"extension":19,"meta":10318,"navigation":24,"path":5590,"seo":10325,"stem":10326,"__hash__":10327},"blog\u002Fblog\u002Fenhance-your-automations-with-notifications.md","Enhancing Your Automations with Notifications",{"type":8,"value":10080,"toc":10308},[10081,10084,10087,10091,10094,10097,10101,10104,10107,10113,10117,10120,10131,10143,10147,10153,10163,10167,10170,10205,10210,10234,10239,10245,10252,10256,10266,10276,10280,10283,10287,10289,10292,10298,10305],[11,10082,10083],{},"When running your automations unattended, it's important to know that your automation is running as expected - and when it's not running as expected, you need to know as soon as possible so you can intervene and resolve any issues that your automation is experiencing. axiom.ai offers a few options to receive notifications from your automations in the event of success, warnings, or failures.",[39,10085],{"alt":10086},"How to use notifications when building bots with axiom.ai",[43,10088,10090],{"id":10089},"why-use-notifications","Why use notifications",[11,10092,10093],{},"Receiving notifications from your automation helps you track runs and ensure everything is working as expected, whether running in the cloud or on your desktop. If an automation fails, instant notifications let you quickly jump into axiom.ai, resolve any issues, and get back on track. Notifications can be configured for each of your automations, meaning that you can have additional notifications set up for your more mission critical automations.",[11,10095,10096],{},"Notifications are also a great way of being notified that your automation has completed successfully - if you're using an automation to gather data, this may be a signal to your team that the data is now ready for use. Or it could be used to trigger a third-party service using webhooks, more on this later in the article.",[43,10098,10100],{"id":10099},"setting-up-notifications","Setting up notifications",[11,10102,10103],{},"To set up notifications, open your automation, go to Settings, and select \"Set up notifications.\" You can enable notifications based on the automation’s final status: success, success with warnings, or failure. If you're running large-scale operations in axiom.ai, we recommend enabling failure notifications only to avoid overwhelming your team with emails. For more mission-critical automations, you may wish to enable the \"Notify only when a run has warnings\" options within the notification settings.",[11,10105,10106],{},"When you enable any notification type, you will then be shown the \"How to notify\" section, this is where you will configure the notifications. Let's dive into this next!",[11,10108,10109,10110,550],{},"You can learn more about setting up notifications in our ",[50,10111,10112],{"href":7480},"notification tutorial",[43,10114,10116],{"id":10115},"email-notifications","Email notifications",[11,10118,10119],{},"Notifications can be set to notify you via email. This field accepts a list of email addresses that can be entered by adding a single email address per line, for example:",[11,10121,10122,10126,10129],{},[50,10123,10125],{"href":10124},"mailto:example@organisation.com","example@organisation.com",[10127,10128],"br",{},[50,10130,10125],{"href":10124},[11,10132,10133,10134,10137,10138,10142],{},"The email notification will include the error that your automation has encountered - this is the same error message that you would see within your ",[50,10135,10136],{"href":9358},"Run report"," or within the builder. See ",[50,10139,10141],{"href":10140},"\u002Fdocs\u002Fno-code-tool\u002Ftroubleshooting\u002Ferrors","common errors"," for more details on specific errors.",[134,10144],{"src":10145,"alt":10146},"\u002Fblog\u002Fenhance-your-automations-with-notifications-email-notification-standard.png","axiom.ai email notification about a task completion issue. The email states that an automation encountered an error in step 2 ('Click element') because the element could not be found during the run. It advises the user to reselect the element using the selector tool or use a custom selector. The email also provides axiom.ai's support email (support@axiom.ai) for assistance. The email is signed by 'The Axiom team.'",[11,10148,10149,10150,10152],{},"Email notifications can also be added into your automations as \"checkpoints\" - using the ",[50,10151,6309],{"href":6308}," step within your notification allows you to receive notifications throughout the running of your automation. For example, you might want to track the data flow through your automation runs but do not have the time to view the automation as it runs - adding \"Send an email\" steps that have custom bodies, including the data tokens that are being generated, can give you insight as to what's happening within your automation.",[791,10154,10155],{},[11,10156,10157,10158,10162],{},"💡 See ",[50,10159,10161],{"href":10160},"#adding-more-error-data-to-your-notifications","adding more error data to your notifications"," to learn more about powering up your email notifications.",[43,10164,10166],{"id":10165},"webhook-notifications","Webhook notifications",[11,10168,10169],{},"Notifications can be set to notify you via webhook. This field will accept a single webhook that a request will be sent to in the event of a notification. This can be very useful for triggering another service when your automation has finished it's run. The webhook will be sent in the following format:",[381,10171,10173],{"className":6843,"code":10172,"language":4195,"meta":15,"style":15},"{\n    \"status\": \"\u003CSTATUS>\",\n    \"log\": \"\u003CMESSAGE>\"\n}\n",[387,10174,10175,10179,10191,10201],{"__ignoreMap":15},[390,10176,10177],{"class":392,"line":393},[390,10178,6851],{"class":400},[390,10180,10181,10184,10186,10189],{"class":392,"line":173},[390,10182,10183],{"class":7078},"    \"status\"",[390,10185,5310],{"class":400},[390,10187,10188],{"class":407},"\"\u003CSTATUS>\"",[390,10190,491],{"class":400},[390,10192,10193,10196,10198],{"class":392,"line":16},[390,10194,10195],{"class":7078},"    \"log\"",[390,10197,5310],{"class":400},[390,10199,10200],{"class":407},"\"\u003CMESSAGE>\"\n",[390,10202,10203],{"class":392,"line":440},[390,10204,755],{"class":400},[11,10206,205,10207,10209],{},[387,10208,446],{}," key will include \"Axiom run completed successfully\" if the run has completed as expected, however, in the event of a failure it will include the error message that prevented the automation from finishing successfully. This can be useful for debugging your automation.",[11,10211,10212,10213,10216,10217,10216,10219,10223,10224,9930,10227,10229,10230,10233],{},"An example of using this to trigger a third party service may be to use this webhook notification to trigger a ",[50,10214,10215],{"href":102},"Slack Workflow",", a ",[50,10218,6518],{"href":141},[50,10220,10222],{"href":10221},"\u002Fguides\u002Fdiscord","Discord notification",", an ",[50,10225,10226],{"href":64},"IFTTT automation",[50,10228,69],{"href":84},". Check out our ",[50,10231,10232],{"href":6626},"API guides"," for some other services that we have written about, but you can use any service that can accept a webhook, including your own!",[791,10235,10236],{},[11,10237,10238],{},"💡 Use webhook notifications to trigger a third-party service to trigger a new automation run in the event of failure",[11,10240,10241,10242,10244],{},"Similar to email notifications, webhooks can also be used throughout your automation to send data throughout the lifecycle of the automation run. The ",[50,10243,5576],{"href":5575}," step can be used to trigger a webhook with a custom payload, and has the ability to insert data tokens into your automations. We find this helpful for having a log of how the automation runs are going - being table to track the data throughout the automation run as it changes.",[791,10246,10247],{},[11,10248,10157,10249,10251],{},[50,10250,10161],{"href":10160}," to learn more about powering up your webhook notifications.",[43,10253,10255],{"id":10254},"adding-more-error-data-to-your-notifications","Adding more error data to your notifications",[11,10257,10258,10259,10261,10262,10265],{},"In the event of an automation failure notification, this will always include the error message that has stopped the automation from completing successfully - this is to help you debug your automation. In addition to this error, there may be instances where you want to add your own metadata to this. To do this, you'll need to add an ",[50,10260,9367],{"href":9366}," step ",[2786,10263,10264],{},"before"," the step that you believe may cause an error, such as a click element that you expect may fail on occasion.",[11,10267,10268,10269,10271,10272,10275],{},"You may use this to add additional context to your error messages, for example, let's say that we have a custom selector stored in a custom data token that we want to use within a ",[50,10270,5667],{"href":5666}," step. We would add the \"Add error metadata\" step before the click element and add the message: \"Attempting click with selector: ",[390,10273,10274],{},"custom-data",".\"",[134,10277],{"src":10278,"alt":10279},"\u002Fblog\u002Fenhance-your-automations-with-notifications-add-error-metadata-setup.png","A screenshot of the axiom.ai interface displaying an automation setup. It includes three steps: 'Enter custom data,' 'Add error metadata,' and 'Click element.' In the 'Add error metadata' step, there is a custom error message being configured. The message field contains the text 'Attempting click with selector:' followed by a token labelled '[custom-data].' A description above the message field explains that users can build custom error messages and insert tokens to display runtime values.",[11,10281,10282],{},"When an error is triggered, this will include the metadata within the body of the email alongside the error message that the offending step has produced:",[134,10284],{"src":10285,"alt":10286},"\u002Fblog\u002Fenhance-your-automations-with-notifications-email-notifications-metadata.png","axiom.ai email notification about a task completion issue. The email states that an automation encountered an error in step 4 ('Click element') because the element could not be found during the run. It advises the user to reselect the element using the selector tool or use a custom selector. The email includes error metadata, showing the attempted click with the selector '.my-custom-selector'. It also provides axiom.ai's support email (support@axiom.ai) for assistance. The email is signed by 'The Axiom team.'",[43,10288,166],{"id":165},[11,10290,10291],{},"Notifications can provide a very helpful health check for your automations - they give you peace of mind that you don't need to keep checking in to ensure that everything is happening as it should be while you go about your day. From mission-critical automations to automations for personal use, it's vital that you know when there is an error - or when an automation is finished.",[11,10293,10294,10295,10297],{},"Here at axiom.ai we find it helpful to be notified when an automation has finished - using a Slack Workflow (",[50,10296,7633],{"href":102},") we can get notified when a specific automation has finished running. This sends a notification to our team so we know that the data that it has processed is ready to act on.",[11,10299,10300,10301,13],{},"Have you built something creative using the notification system? We would love to hear about it over in our ",[50,10302,3304],{"href":10303,"rel":10304},"https:\u002F\u002Freddit.com\u002Faxiom_ai",[54],[2666,10306,10307],{},"html pre.shiki code .s4rv2, html code.shiki .s4rv2{--shiki-default:#1F2328;--shiki-dark:#E6EDF3}html pre.shiki code .sjgCt, html code.shiki .sjgCt{--shiki-default:#116329;--shiki-dark:#7EE787}html pre.shiki code .sSVrQ, html code.shiki .sSVrQ{--shiki-default:#0A3069;--shiki-dark:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":15,"searchDepth":16,"depth":16,"links":10309},[10310,10311,10312,10313,10314,10315],{"id":10089,"depth":173,"text":10090},{"id":10099,"depth":173,"text":10100},{"id":10115,"depth":173,"text":10116},{"id":10165,"depth":173,"text":10166},{"id":10254,"depth":173,"text":10255},{"id":165,"depth":173,"text":166},"2025-03-04","Enhance your automations with run status notifications, automated email notifications, and webhooks",{"read":352,"type":184,"tool":10319,"category":10320,"tags":10321,"location":191,"featuredimg":10323,"landingimg":10324,"summary":10317,"video":18,"metaTitle":10078},[354],[2686],[7373,10322,6595],"email","\u002Fblog\u002Fnotifications-post-box.webp","\u002Fblog\u002Fnotifications-sq.webp",{"title":10078,"description":10317},"blog\u002Fenhance-your-automations-with-notifications","Ol-nuHaP0p-0G5Z9nsWghNx-OVVZlsBW9dN7jFLLfaY",{"id":10329,"title":41,"author":23,"body":10330,"date":10422,"description":180,"draft":181,"extension":19,"meta":10423,"navigation":24,"path":10427,"seo":10428,"stem":10429,"__hash__":10430},"blog\u002Fblog\u002F5-methods-of-triggering-via-third-party.md",{"type":8,"value":10331,"toc":10414},[10332,10334,10336,10338,10343,10345,10349,10351,10356,10358,10362,10364,10369,10373,10377,10379,10384,10391,10393,10397,10399,10404,10406,10410,10412],[11,10333,37],{},[39,10335],{"alt":41},[43,10337,46],{"id":45},[11,10339,10340,55],{},[50,10341,46],{"href":52,"rel":10342},[54],[11,10344,58],{},[11,10346,61,10347],{},[50,10348,65],{"href":64},[43,10350,69],{"id":68},[11,10352,10353,76],{},[50,10354,69],{"href":74,"rel":10355},[54],[11,10357,79],{},[11,10359,61,10360],{},[50,10361,85],{"href":84},[43,10363,89],{"id":88},[11,10365,10366,96],{},[50,10367,89],{"href":94,"rel":10368},[54],[11,10370,99,10371,104],{},[50,10372,103],{"href":102},[11,10374,61,10375],{},[50,10376,110],{"href":109},[43,10378,114],{"id":113},[11,10380,10381,121],{},[50,10382,114],{"href":119,"rel":10383},[54],[11,10385,124,10386,130,10389,104],{},[50,10387,129],{"href":127,"rel":10388},[54],[50,10390,103],{"href":102},[134,10392],{"src":136},[11,10394,61,10395],{},[50,10396,142],{"href":141},[43,10398,146],{"id":145},[11,10400,10401,153],{},[50,10402,146],{"href":151,"rel":10403},[54],[11,10405,156],{},[11,10407,61,10408],{},[50,10409,162],{"href":161},[43,10411,166],{"id":165},[11,10413,169],{},{"title":15,"searchDepth":16,"depth":16,"links":10415},[10416,10417,10418,10419,10420,10421],{"id":45,"depth":173,"text":46},{"id":68,"depth":173,"text":69},{"id":88,"depth":173,"text":89},{"id":113,"depth":173,"text":114},{"id":145,"depth":173,"text":146},{"id":165,"depth":173,"text":166},"2025-02-27",{"read":183,"type":184,"tool":10424,"category":10425,"tags":10426,"location":191,"featuredimg":192,"landingimg":193,"summary":180,"video":18,"metaTitle":41},[186],[2686],[46,68,190,113,145,6595,6423],"\u002Fblog\u002F5-methods-of-triggering-via-third-party",{"title":41,"description":180},"blog\u002F5-methods-of-triggering-via-third-party","AyUJjHLKRVsyeQD5DBjSeZbsS1Cn8cXDPRCxgp65GA4",{"id":10432,"title":3366,"author":23,"body":10433,"date":10971,"description":10972,"draft":181,"extension":19,"meta":10973,"navigation":24,"path":5731,"seo":10979,"stem":10980,"__hash__":10981},"blog\u002Fblog\u002Fbest-custom-css-selectors-for-web-scraping.md",{"type":8,"value":10434,"toc":10958},[10435,10443,10446,10448,10451,10454,10458,10461,10515,10518,10525,10529,10532,10574,10638,10644,10648,10651,10667,10715,10726,10773,10776,10780,10789,10792,10799,10803,10818,10832,10836,10839,10842,10868,10873,10876,10905,10908,10910,10913,10917,10920,10936,10940,10944,10956],[11,10436,10437,10438,10442],{},"For the majority of web scraping applications, the ",[50,10439,10441],{"href":10440},"\u002Fdocs\u002Fno-code-tool\u002Fthe-builder\u002Fselector-tool","Selector Tool"," within axiom.ai will suit your needs to select elements on a page that you want to extract. However, there are times when you may need more options to select elements, such as extending the functionality that we offer, or accessing elements on the page that may hidden behind an overlay.",[39,10444],{"alt":10445},"The best css selectors to choose when scraping the web",[43,10447,2856],{"id":2855},[11,10449,10450],{},"CSS (Cascading Style Sheets) are used to control the appearance and layout of HTML elements on a webpage. It allows the developers of the website to apply styles such as colors, fonts, spacing, and positioning, ensuring a consistent and visually appealing design. Most modern websites will have multiple stylesheets that are used to construct the style of a site - some larger websites may have tens of these files!",[11,10452,10453],{},"In order to target the HTML elements on the page, CSS uses a variety of selectors to target the element based on a few methods: using the element tag, an ID, a class (or multiple) or an attribute. You can use a mixture of all of these to create a CSS selector, we will dive into CSS specificity next as an important concept to taking advantage of combining selectors.",[43,10455,10457],{"id":10456},"css-specificity","CSS Specificity",[11,10459,10460],{},"CSS specificity should be kept in mind when writing CSS selectors, this is a set of rules which determine which styles are applied to an element when combining multiple selectors that are targetting the same element. It assigns weight to selectors based on their type, prioritising more specific rules over less specific one - hence the name! The hierarchy works as follows, listed from highest specificity to lowest:",[312,10462,10463,10472,10480,10500],{},[252,10464,10465,10468,10469],{},[5176,10466,10467],{},"Inline styles"," such as ",[387,10470,10471],{},"style=\"color: blue;\"",[252,10473,10474,10468,10477],{},[5176,10475,10476],{},"ID selectors",[387,10478,10479],{},"#list-item",[252,10481,10482,1155,10485,2988,10488,10468,10491,1155,10494,1155,10497],{},[5176,10483,10484],{},"Class selectors",[5176,10486,10487],{},"attributes",[5176,10489,10490],{},"pseudo-classes",[387,10492,10493],{},".list-item",[387,10495,10496],{},"[type=\"checkbox\"]",[387,10498,10499],{},":hover",[252,10501,10502,828,10505,10468,10508,1155,10510,1155,10512],{},[5176,10503,10504],{},"Element selectors",[5176,10506,10507],{},"pseudo-elements",[387,10509,2937],{},[387,10511,11],{},[387,10513,10514],{},"::before",[11,10516,10517],{},"When multiple rules conflict, the browser will use the rule with the highest specificity. If specificity is equal, the last declared rule will prevail.",[11,10519,10520,10521,10524],{},"Learning how to take advantage of specificity gives you the advantage of being able to avoid using the dreaded ",[387,10522,10523],{},"!important"," attribute which can have unexpected side effects when used where it's not needed.",[818,10526,10528],{"id":10527},"knowledge-check-css-specificity","Knowledge check: CSS specificity",[11,10530,10531],{},"In the following code, what color will the text be when the CSS is applied?",[381,10533,10535],{"className":8475,"code":10534,"language":8477,"meta":15,"style":15},"\u003Cp id=\"text-1\" class=\"my-text\">\n    Hello, World!\n\u003C\u002Fp>\n",[387,10536,10537,10560,10565],{"__ignoreMap":15},[390,10538,10539,10541,10543,10545,10547,10550,10553,10555,10558],{"class":392,"line":393},[390,10540,3140],{"class":400},[390,10542,11],{"class":7078},[390,10544,8488],{"class":464},[390,10546,893],{"class":400},[390,10548,10549],{"class":407},"\"text-1\"",[390,10551,10552],{"class":464}," class",[390,10554,893],{"class":400},[390,10556,10557],{"class":407},"\"my-text\"",[390,10559,8501],{"class":400},[390,10561,10562],{"class":392,"line":173},[390,10563,10564],{"class":400},"    Hello, World!\n",[390,10566,10567,10570,10572],{"class":392,"line":16},[390,10568,10569],{"class":400},"\u003C\u002F",[390,10571,11],{"class":7078},[390,10573,8501],{"class":400},[381,10575,10579],{"className":10576,"code":10577,"language":10578,"meta":15,"style":15},"language-css shiki shiki-themes github-light-default github-dark-default",".my-text { color: red; }\n\n#text-1 { color: green; }\n\np { color: blue; }\n","css",[387,10580,10581,10599,10603,10619,10623],{"__ignoreMap":15},[390,10582,10583,10586,10588,10591,10593,10596],{"class":392,"line":393},[390,10584,10585],{"class":464},".my-text",[390,10587,4888],{"class":400},[390,10589,10590],{"class":464},"color",[390,10592,5310],{"class":400},[390,10594,10595],{"class":464},"red",[390,10597,10598],{"class":400},"; }\n",[390,10600,10601],{"class":392,"line":173},[390,10602,413],{"emptyLinePlaceholder":24},[390,10604,10605,10608,10610,10612,10614,10617],{"class":392,"line":16},[390,10606,10607],{"class":464},"#text-1",[390,10609,4888],{"class":400},[390,10611,10590],{"class":464},[390,10613,5310],{"class":400},[390,10615,10616],{"class":464},"green",[390,10618,10598],{"class":400},[390,10620,10621],{"class":392,"line":440},[390,10622,413],{"emptyLinePlaceholder":24},[390,10624,10625,10627,10629,10631,10633,10636],{"class":392,"line":458},[390,10626,11],{"class":7078},[390,10628,4888],{"class":400},[390,10630,10590],{"class":464},[390,10632,5310],{"class":400},[390,10634,10635],{"class":464},"blue",[390,10637,10598],{"class":400},[11,10639,10640],{},[50,10641,10643],{"href":10642},"#knowledge-check-answers","Jump to answer",[43,10645,10647],{"id":10646},"combinators","Combinators",[11,10649,10650],{},"Combinators are CSS elements that can be used to relate two CSS rules. These can be useful in addition to CSS specificity to ensure that you are targetting the right elements.",[11,10652,10653,10658,10659,10662,10663,10666],{},[50,10654,10657],{"href":10655,"rel":10656},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FCSS\u002FChild_combinator",[54],"Child combinators"," use the ",[387,10660,10661],{},">"," to match elements that are matched by the second selector that are direct children of the elements matched by the first. For example, ",[387,10664,10665],{},"div > span"," will match the following:",[381,10668,10670],{"className":8475,"code":10669,"language":8477,"meta":15,"style":15},"\u003Cdiv>\n    \u003Cspan>This is matched as a direct child of the div element.\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003Cspan>This is not matched as it is not a direct child of a div element.\u003C\u002Fspan>\n",[387,10671,10672,10680,10694,10702],{"__ignoreMap":15},[390,10673,10674,10676,10678],{"class":392,"line":393},[390,10675,3140],{"class":400},[390,10677,2937],{"class":7078},[390,10679,8501],{"class":400},[390,10681,10682,10685,10687,10690,10692],{"class":392,"line":173},[390,10683,10684],{"class":400},"    \u003C",[390,10686,390],{"class":7078},[390,10688,10689],{"class":400},">This is matched as a direct child of the div element.\u003C\u002F",[390,10691,2937],{"class":7078},[390,10693,8501],{"class":400},[390,10695,10696,10698,10700],{"class":392,"line":16},[390,10697,10569],{"class":400},[390,10699,2937],{"class":7078},[390,10701,8501],{"class":400},[390,10703,10704,10706,10708,10711,10713],{"class":392,"line":440},[390,10705,3140],{"class":400},[390,10707,390],{"class":7078},[390,10709,10710],{"class":400},">This is not matched as it is not a direct child of a div element.\u003C\u002F",[390,10712,390],{"class":7078},[390,10714,8501],{"class":400},[11,10716,10717,10722,10723,10666],{},[50,10718,10721],{"href":10719,"rel":10720},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FCSS\u002FDescendant_combinator",[54],"Descendant combinators"," use a single space character to combine two selectors such that elements matched by the second selector are selected if they have an ancestor element matching the first. For example, ",[387,10724,10725],{},"div span",[381,10727,10729],{"className":8475,"code":10728,"language":8477,"meta":15,"style":15},"\u003Cdiv>\n    \u003Cspan>This is matched as a child of the div element.\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003Cspan>This is not matched as it is not a child of a div element.\u003C\u002Fspan>\n",[387,10730,10731,10739,10752,10760],{"__ignoreMap":15},[390,10732,10733,10735,10737],{"class":392,"line":393},[390,10734,3140],{"class":400},[390,10736,2937],{"class":7078},[390,10738,8501],{"class":400},[390,10740,10741,10743,10745,10748,10750],{"class":392,"line":173},[390,10742,10684],{"class":400},[390,10744,390],{"class":7078},[390,10746,10747],{"class":400},">This is matched as a child of the div element.\u003C\u002F",[390,10749,2937],{"class":7078},[390,10751,8501],{"class":400},[390,10753,10754,10756,10758],{"class":392,"line":16},[390,10755,10569],{"class":400},[390,10757,2937],{"class":7078},[390,10759,8501],{"class":400},[390,10761,10762,10764,10766,10769,10771],{"class":392,"line":440},[390,10763,3140],{"class":400},[390,10765,390],{"class":7078},[390,10767,10768],{"class":400},">This is not matched as it is not a child of a div element.\u003C\u002F",[390,10770,390],{"class":7078},[390,10772,8501],{"class":400},[11,10774,10775],{},"There are other combinators that can be used to enhance your custom selectors, but they should suffice to get you started.",[43,10777,10779],{"id":10778},"identifying-selectors-to-use","Identifying selectors to use",[11,10781,10782,10783,10788],{},"In order to use selectors, you'll first need to investigate which selectors will work for your use case. Generally, assuming you're not the developer of the site, ",[50,10784,10787],{"href":10785,"rel":10786},"https:\u002F\u002Fdeveloper.chrome.com\u002Fdocs\u002Fdevtools",[54],"Chrome Devtools"," is the best tool to use to inspect the underlying code of the page. To open Devtools, right-click on the page and click \"Inspect\" in the context menu that appears. It should automatically drop you in the \"Elements\" tab, but if it doesn't, navigate there.",[11,10790,10791],{},"We recommend having some web development knowledge from this point, however, you do not need to fully understand the code that is displayed in order to extract a CSS selector provided you have read the sections above.",[11,10793,10794,10795,10798],{},"To jump to the code behind an element, right-click on the element on the page and click \"Inspect\" again within the context menu that appears. Provided it does not have an overlay, the window should jump to the section of the code that contains the element where you will be able to extract a selector. For a ",[2786,10796,10797],{},"very"," specific selector, you can right-click on the element in the code, click \"copy\" and then \"copy selector\" - we wouldn't really recommend this method as some sites use frameworks that mean that the ID may change frequently and this would break your automation if the site even has a small update.",[43,10800,10802],{"id":10801},"using-css-in-your-steps","Using CSS in your steps",[11,10804,10805,10806,828,10810,10814,10815,10817],{},"Quite a few axiom.ai steps allow for custom CSS selectors to be used to target elements on the page as your automation runs, primarily the ",[50,10807,10809],{"href":10808},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002F#scrape","Scrape",[50,10811,10813],{"href":10812},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002F#interact","Interact"," steps. For example, to use a custom CSS selector within the ",[50,10816,5667],{"href":5666}," step, perform the following steps:",[312,10819,10820,10823,10826,10829],{},[252,10821,10822],{},"Search for and add a \"Click element\" step to your automation.",[252,10824,10825],{},"Click \"Select\".",[252,10827,10828],{},"Click \"Custom\".",[252,10830,10831],{},"Enter your custom selector.",[43,10833,10835],{"id":10834},"dynamically-setting-a-custom-css-selector","Dynamically setting a custom CSS selector",[11,10837,10838],{},"While explicitly setting the custom CSS selector within your steps is useful for consistently interacting with an element on a page, there may be times when you need to dynamically set this using a data token. To get started, click \"Set selector from data\" within the custom CSS selector input within the step, this will prompt you to select a data token from your automation.",[11,10840,10841],{},"There may even be times where you need to loop through a list of custom selectors. In order to loop through a list of CSS selectors, you will first need to create a list of selectors. This can be useful when you may need to click through a list of items. For example:",[6730,10843,10844,10851],{},[6733,10845,10846],{},[6736,10847,10848],{},[6739,10849,10850],{},"Selector",[6749,10852,10853,10858,10863],{},[6736,10854,10855],{},[6754,10856,10857],{},"list-item-1",[6736,10859,10860],{},[6754,10861,10862],{},"list-item-2",[6736,10864,10865],{},[6754,10866,10867],{},"list-item-3",[11,10869,10870],{},[2786,10871,10872],{},"Tip: this may be stored in a Google Sheet for ease.",[11,10874,10875],{},"Once you have your list of custom CSS selectors, you can use the following pattern to loop through your list of selectors:",[312,10877,10878,10884,10893,10902],{},[252,10879,10880,10881,10883],{},"Add a ",[50,10882,5710],{"href":5709}," step, select the sheet that contains your list of selectors.",[252,10885,10880,10886,10888,10889,10892],{},[50,10887,5702],{"href":5701}," step, setting the ",[387,10890,10891],{},"google-sheet-data"," data token as the data to loop through.",[252,10894,10880,10895,285,10897,10888,10899,10901],{},[50,10896,10809],{"href":10808},[50,10898,10813],{"href":10812},[387,10900,10891],{}," token inside the \"Loop through data\" step as the custom CSS selector.",[252,10903,10904],{},"Add any other steps inside the loop.",[11,10906,10907],{},"This can be useful in situations where you have a list of items that you wish to click, perform actions, and then continue on to click on the next item of the list. This can be used to perform multiple actions on lists, for checkboxes, for example.",[43,10909,166],{"id":165},[11,10911,10912],{},"Learning how to take advantage of custom CSS selectors can be a powerful skill to have in your toolbox - whether this is for use within your axiom.ai automations, or generally in any web development adjacent field. This can enhance your automations to ensure consistent results on pages that may change frequently, or are not suitable to the standard features of the Selector Tool.",[43,10914,10916],{"id":10915},"further-reading","Further reading",[11,10918,10919],{},"If you're interested in learning more about CSS and CSS selectors, check out the resources below:",[249,10921,10922,10929],{},[252,10923,10924],{},[50,10925,10928],{"href":10926,"rel":10927},"https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FCSS\u002FReference",[54],"Mozilla Developer Docs",[252,10930,10931],{},[50,10932,10935],{"href":10933,"rel":10934},"https:\u002F\u002Fwww.w3schools.com\u002Fcss\u002Fcss_intro.asp",[54],"W3Schools",[43,10937,10939],{"id":10938},"knowledge-check-answers","Knowledge check answers",[11,10941,10942],{},[5176,10943,10457],{},[11,10945,10946,10947,10949,10950,10952,10953,10955],{},"As we have a CSS selector using the ID (",[387,10948,10607],{},") this will be the highest specificity, followed by ",[387,10951,10585],{}," and finally the ",[387,10954,11],{}," selector.",[2666,10957,9487],{},{"title":15,"searchDepth":16,"depth":16,"links":10959},[10960,10961,10964,10965,10966,10967,10968,10969,10970],{"id":2855,"depth":173,"text":2856},{"id":10456,"depth":173,"text":10457,"children":10962},[10963],{"id":10527,"depth":16,"text":10528},{"id":10646,"depth":173,"text":10647},{"id":10778,"depth":173,"text":10779},{"id":10801,"depth":173,"text":10802},{"id":10834,"depth":173,"text":10835},{"id":165,"depth":173,"text":166},{"id":10915,"depth":173,"text":10916},{"id":10938,"depth":173,"text":10939},"2025-02-06","Learn more about the best custom CSS selectors that can be used to enhance your web scrapers with axiom.ai",{"read":352,"type":184,"tool":10974,"category":10975,"tags":10976,"location":191,"featuredimg":10977,"landingimg":10978,"summary":10972,"video":18,"metaTitle":3366},[358,354],[188],[10578,357,10487],"\u002Fblog\u002Fcss-selector-large.webp","\u002Fblog\u002Fcss-selector-sq.webp",{"title":3366,"description":10972},"blog\u002Fbest-custom-css-selectors-for-web-scraping","d-06FRP2U99ZVt9B9ixSmuho8fnm1I6EV2unW3EKjy8",{"id":10983,"title":10984,"author":23,"body":10985,"date":11125,"description":11126,"draft":181,"extension":19,"meta":11127,"navigation":24,"path":11135,"seo":11136,"stem":11137,"__hash__":11138},"blog\u002Fblog\u002Fbot-master-2025.md","The search for the next Bot Master is on!",{"type":8,"value":10986,"toc":11118},[10987,10990,10993,10997,11000,11004,11007,11011,11014,11021,11024,11028,11031,11034,11054,11058,11061,11075,11080,11084,11087,11089,11091,11099,11106],[11,10988,10989],{},"We are searching for a new Bot Master who wants to demonstrate their automation building skills to the community!",[39,10991],{"alt":10992},"Become the Axiom bot master 2025",[43,10994,10996],{"id":10995},"what-is-this-thing","What is this thing?",[11,10998,10999],{},"We are looking for experienced bot builders to show off their talents and get featured on our blog. Have you got a business that runs part of their workflow with axiom.ai, or just an individual who really enjoys automating their life? We want to hear from you! We have seen some really neat things built with axiom.ai, including businesses that have been created using axiom.ai as a backend. This doesn't have to be anything new, it might just be something you use in your everyday life!",[43,11001,11003],{"id":11002},"when-does-it-happen","When does it happen?",[11,11005,11006],{},"Submissions will open on the 16th January 2025 and remain open until 28th February 2025 to give you plenty of opportunity to build something special. We are aiming to get in contact with winners around mid-March.",[43,11008,11010],{"id":11009},"what-can-we-win","What can we win?",[11,11012,11013],{},"We're giving one person the chance to win a $50 Amazon voucher, along with the opportunity to work with our team to develop a blog article for our website to showcase your work and talent, including a link back to your website, or blog. This blog article will be shared with our social media accounts, including on our Reddit community.",[11,11015,11016,11017,11020],{},"Five runner up submissions will each receive a $10 axiom.ai credit",[11018,11019,1248],"sup",{}," applied to their account.",[11,11022,11023],{},"Results will be announced two weeks after submissions close to give our judges a chance to review all submissions. Winners will be contacted by email using the email provided during submission - this email should be the same as your axiom.ai account.",[43,11025,11027],{"id":11026},"are-there-any-criteria","Are there any criteria?",[11,11029,11030],{},"Yes, there are a few ground rules that need to be followed, along with some requirements.",[11,11032,11033],{},"Requirements:",[249,11035,11036,11039,11045,11051],{},[252,11037,11038],{},"You must have an active subscription with axiom.ai, free users are not eligible.",[252,11040,11041,11042],{},"Automation must be a business process automation",[11018,11043,11044],{},"2",[252,11046,11047,11048,13],{},"axiom.ai must have the ability to run your automation or you must be able to provide a screen recording",[11018,11049,11050],{},"3",[252,11052,11053],{},"The automation must be capable of a \"successful\" run within a reasonable timeframe.",[43,11055,11057],{"id":11056},"how-do-we-submit","How do we submit?",[11,11059,11060],{},"To submit your automation, you'll need to fill out the form below, we'll ask you for the following information:",[312,11062,11063,11066,11069,11072],{},[252,11064,11065],{},"Your axiom.ai email - just to confirm your account, we won't be using this for marketing.",[252,11067,11068],{},"Details about your automation - we want to know how it works, what it does and why it stands out.",[252,11070,11071],{},"Any requirements to run the automation.",[252,11073,11074],{},"The automation",[2937,11076],{"className":11077,"dataTfLive":11079},[11078],"submission-button","01JHQVSR4RW6P0CDM6SSQXYNAV",[11081,11082],"script",{"src":11083},"\u002F\u002Fembed.typeform.com\u002Fnext\u002Fembed.js",[2666,11085,11086],{},".submission-button button { margin: 0 auto !important; display: block !important;}",[10127,11088],{},[3295,11090],{},[11,11092,11093,11095,11096],{},[11018,11094,1248],{}," ",[2786,11097,11098],{},"This does not have a monetary value and can not be exchanged for cash or a voucher.",[11,11100,11101,11095,11103],{},[11018,11102,11044],{},[2786,11104,11105],{},"Automation must not access illegal content, gambling sites or private content.",[11,11107,11108,11095,11110],{},[11018,11109,11050],{},[2786,11111,11112,11113,11117],{},"If we need to run the automation, and it accessed services behind a paywall or subscription, we may need you to provide us with login credentials. ",[50,11114,11116],{"href":11115},"\u002Fcustomer-support","Contact us"," for more details.",{"title":15,"searchDepth":16,"depth":16,"links":11119},[11120,11121,11122,11123,11124],{"id":10995,"depth":173,"text":10996},{"id":11002,"depth":173,"text":11003},{"id":11009,"depth":173,"text":11010},{"id":11026,"depth":173,"text":11027},{"id":11056,"depth":173,"text":11057},"2025-01-21","We're announcing a new competition to showcase the best builders in our community",{"read":8416,"type":184,"tool":11128,"category":11129,"tags":11130,"location":191,"featuredimg":11133,"landingimg":11134,"summary":11126,"video":18,"metaTitle":10984},[354],[2686],[11131,11132],"bot building","competition","\u002Fblog\u002Fbot-master-landing.webp","\u002Fblog\u002Fbot-master-sq.webp","\u002Fblog\u002Fbot-master-2025",{"title":10984,"description":11126},"blog\u002Fbot-master-2025","U8ngARWMaQTjioCVTcDe578LSGvVij5N9IxX5xC1P7I",{"id":11140,"title":11141,"author":23,"body":11142,"date":11367,"description":11368,"draft":181,"extension":19,"meta":11369,"navigation":24,"path":11378,"seo":11379,"stem":11380,"__hash__":11381},"blog\u002Fblog\u002Fwhats-coming-2025.md","What's coming in 2025",{"type":8,"value":11143,"toc":11352},[11144,11151,11154,11158,11171,11179,11183,11186,11192,11196,11204,11211,11215,11218,11223,11226,11231,11235,11238,11241,11254,11258,11261,11264,11267,11270,11273,11276,11281,11286,11290,11293,11295,11298,11302,11305,11308,11312,11315,11323,11327,11334,11340,11342,11345],[11,11145,11146,11147,13],{},"Hi, everyone! 2024 was an exciting year here at axiom.ai and we are very proud of what we have released and what we have planned for 2025. We are going to begin with a 2024 recap, but if you are just interested in our future plans, skip ahead to ",[50,11148,11150],{"href":11149},"#whats-next","What's next",[39,11152],{"alt":11153},"New fetaures for axiom.ai in 2025",[43,11155,11157],{"id":11156},"what-we-were-up-to-last-year","What we were up to last year",[11,11159,11160,11161,11165,11166,11170],{},"We concentrated on implementing user requested features and improving the overall quality of the extension, and our documentation. We are delighted to have launched our improved ",[50,11162,11164],{"href":11163},"\u002Fdocs","documentation",", our ",[50,11167,11169],{"href":127,"rel":11168},[54],"Reddit"," community and general improvements throughout the tool!",[11,11172,11173,11174,11178],{},"This recap is some of our best new features and improvements, but the full list of features and improvements is much more than would fit in this post! Review our ",[50,11175,11177],{"href":11176},"\u002Fdocs\u002Fno-code-tool\u002Frelease-notes","Release Notes"," for a full list of changes.",[43,11180,11182],{"id":11181},"bot-detection-bypass","Bot detection bypass",[11,11184,11185],{},"We have all been there - you make the perfect automation and then you get hit with a Cloudflare verification screen that blocks progress once you hit the \"Run\" button. We wanted our users to get around this so we launched the Bypass Bot Detection option within your automation settings - this will automatically handle the \"verify you are human\" prompt that sites protected by Cloudflare display. This feature has been loved by our users and allows your automations to be run on sites that employ Cloudflare to prevent bot activity.",[11,11187,11188,11189],{},"Learn more: ",[50,11190,11191],{"href":3972},"Bot blocking",[43,11193,11195],{"id":11194},"snippets-and-templates","Snippets and templates",[11,11197,11198,11199,11203],{},"axiom.ai 4.0 brought along the introduction of ",[50,11200,11202],{"href":11201},"\u002Fdocs\u002Fno-code-tool\u002Fget-started#get-started-from-a-snippet","Snippets"," which allows you to get started even faster with common design patterns, such as scraping data and writing it to a Google Sheet - a common feature that we know people love to use axiom.ai for. You'll find Snippets in the Step Finder.",[11,11205,11206,11207,11210],{},"How can we go one step further, what would really help people get started? Guided Templates. We launched our ",[50,11208,11209],{"href":6626},"Guided Templates"," that allow you to to quickly build an automation for some popular services, such as Instagram, Facebook, LinkedIn and Gmail. These templates will walk you through the entire set up process and teach you how the automation works - you can also build off these templates to build additional features into your automations.",[43,11212,11214],{"id":11213},"local-storage-cookies-and-proxies","Local storage, cookies and proxies",[11,11216,11217],{},"To make it easier to allow the automation to run on websites as if you were running it yourself, we added the ability to store your local storage and cookies within your automations. This can be super helpful when it comes to bringing your authentication sessions into your automations. Our users find this very helpful when managing social media accounts, but this can also be useful for managing general settings within a website.",[11,11219,11188,11220],{},[50,11221,11222],{"href":4005},"Learn how to store cookies and local storage",[11,11224,11225],{},"We launched proxy support in 4.2, allowing you to bring along your own proxy and run your automations through it rather than our IP addresses. This was a very popular feature and allowed users to do even more with their automations, like getting around geo-restrictions and organisational VPNs. We added the option to add basic authentication in 4.3, which allowed for the use of more secure proxies.",[11,11227,11188,11228],{},[50,11229,11230],{"href":3986},"Learn how to use a proxy",[43,11232,11234],{"id":11233},"step-management","Step management",[11,11236,11237],{},"Improving step management was on our agenda for 2024 and we made some changes and improvements to the overall experience that we are happy to have released. We made a bunch of updates to the Step Finder to allow you to more easily find and add steps to your automations.",[11,11239,11240],{},"We received feedback regarding moving, copying and removing steps from within the Builder itself. We made some changes to the Builder to make this options more accessible and easier to use.",[11,11242,11243,11244,1155,11246,1155,11250,11253],{},"Nested steps launched with axiom.ai 4.0, and has become an invaluable feature of the Builder. When combined with other steps launched this year, including the ",[50,11245,5702],{"href":5701},[50,11247,11249],{"href":11248},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fif-else-condition","If\u002Felse",[50,11251,11252],{"href":7392},"Try\u002Fcatch"," steps, nesting can be really useful for managing the structure of your automation.",[43,11255,11257],{"id":11256},"other-improvements","Other improvements",[11,11259,11260],{},"Here are some of the other improvements we made, this list is not a full list of improvements, just a small summary!",[11,11262,11263],{},"Power users! We hear you - we launched Keyboard Shortcuts just for you.",[11,11265,11266],{},"AI aficionados! We launched something for you, too - we launched GPT 4 support (spoiler: we are updating this in 2025).",[11,11268,11269],{},"We added the ability to set your maximum runtime, send run notifications to multiple email addresses and improvements to the instructions of most steps to make it even easier.",[43,11271,11150],{"id":11272},"whats-next",[11,11274,11275],{},"We have lots of exciting updates planned for 2025, some we can share, some you'll have to wait and see for yourself! Our main focus is providing a framework that you can use to scale up your automations. Axiom Cloud 2.0 will bring more concurrency, more customisation, and more control. Let's dive into some of the features we expect to bring to Axiom Cloud 2.0.",[11,11277,11278],{},[2786,11279,11280],{},"Note: not all features may launch when Axiom Cloud 2.0 launches, features may change or be removed from our roadmap. Certain features may be tied to certain subscription tiers. Launch date is to be decided at a later date.",[791,11282,11283],{},[11,11284,11285],{},"Want to test out or learn more about the next iteration of axiom.ai? Check out our Coming soon page for more details.",[43,11287,11289],{"id":11288},"concurrency","Concurrency",[11,11291,11292],{},"We know that concurrently can sometimes be a limiting factor when it comes to running your automations, so we want to remove that barrier and allow for the scaling up of your operations with axiom.ai. We are aiming to offer a 10X improvement over the current number of bots that you can run concurrently in-browser, and even more when your automation is run headless. Concurrency for desktop runs will also be included for users with powerful computers, or when running on a server.",[43,11294,5117],{"id":387},[11,11296,11297],{},"We love no-code and how simply it is to build really effective automations by dropping in steps. However, there are times when you may need to go beyond no-code - we plan on enabling the transition from no-code to code to give you more flexibility, customisation and control over your automations. Axiom already has great features for running your JavaScript in your automations, we hope to enable the next step. We'll continue to support Puppeteer and JavaScript, but hope to offer support for other frameworks and libraries - we will have more to share at a later date!",[43,11299,11301],{"id":11300},"proxies-and-bot-detection-bypass","Proxies and bot detection bypass",[11,11303,11304],{},"We launched proxy support in 2024 and promised more - we plan on bringing native proxy support to Axiom Cloud 2.0, allowing subscribers to quickly get started with a proxy rather than needing to set up a third-party proxy. We also plan on releasing proxy rotation options within your automation.",[11,11306,11307],{},"Our bot detection bypass was a big hit last year. The main feedback we received when we launched was: \"Why doesn't it work on the cloud?\". Good question - we hope to fix that this year with the launch of bot detection bypass for the cloud.",[43,11309,11311],{"id":11310},"data-management","Data management",[11,11313,11314],{},"Google Sheets is a fantastic tool that we integrate with to allow you to read and store your data, but we are aware that there are some limitations to this - including Google's habit of revoking the access token when you use it too often. Google Sheets will remain a great option for most automations but we recognise that there is sometimes a need for a more scalable solution. While we can not go into the details just yet, we plan on integrating with additional database services to give you more options and scalability when considering where to store your data. This will allow you to truly manage your data at scale when running your automations without the limitations of Google Sheets.",[11,11316,11317,11318,13],{},"Do you have a tool that you use to store your data that you'd love to see axiom.ai work with? ",[50,11319,11322],{"href":11320,"rel":11321},"https:\u002F\u002Fwww.reddit.com\u002Fr\u002Faxiom_ai\u002Fcomments\u002F1fptsst\u002Ffeedback_megathread",[54],"Let us know",[43,11324,11326],{"id":11325},"community-and-content","Community and content",[11,11328,11329,11330,11333],{},"We believe our ",[50,11331,11169],{"href":127,"rel":11332},[54]," community has already grown into a great resource hub for support and sharing ideas - we hope to continue to expand this community to bring our users together to share ideas and help one another. We've already seen over 100 users join since the community was created! Come join us over there, we look forward to hearing from you.",[11,11335,11336,11337,13],{},"There are so many different ways that you can use axiom.ai, including with third-party tools to extend the functionality of your automations. We hope to develop new content that'll help you get started quicker with implementing new tools into your workflow. You can see our current guides that we have worked on in our ",[50,11338,11339],{"href":6626},"guides",[43,11341,166],{"id":165},[11,11343,11344],{},"We are delighted with the warm reception that we received when we launched our new features and improvements and hope to continue releasing updates that delight you all. We wouldn't be able to do this without your feedback and ideas - thank you! We are very excited about this year and hope to have you along for the ride!",[11,11346,11347,11348,13],{},"If you have any feedback on either of the above points, drop a comment in our ",[50,11349,11351],{"href":11320,"rel":11350},[54],"Feedback Megathread",{"title":15,"searchDepth":16,"depth":16,"links":11353},[11354,11355,11356,11357,11358,11359,11360,11361,11362,11363,11364,11365,11366],{"id":11156,"depth":173,"text":11157},{"id":11181,"depth":173,"text":11182},{"id":11194,"depth":173,"text":11195},{"id":11213,"depth":173,"text":11214},{"id":11233,"depth":173,"text":11234},{"id":11256,"depth":173,"text":11257},{"id":11272,"depth":173,"text":11150},{"id":11288,"depth":173,"text":11289},{"id":387,"depth":173,"text":5117},{"id":11300,"depth":173,"text":11301},{"id":11310,"depth":173,"text":11311},{"id":11325,"depth":173,"text":11326},{"id":165,"depth":173,"text":166},"2025-01-16","Learn more about what we have planned for axiom.ai in 2025",{"read":2682,"type":184,"tool":11370,"category":11372,"tags":11373,"location":191,"featuredimg":11376,"landingimg":11377,"summary":11368,"video":18,"metaTitle":11141},[354,11371,2684],"Selector tool",[2686],[2538,9735,387,11374,11375],"cloud","desktop","\u002Fblog\u002Fnew-for-25.webp","\u002Fblog\u002Fnew-for-25-sq.webp","\u002Fblog\u002Fwhats-coming-2025",{"title":11141,"description":11368},"blog\u002Fwhats-coming-2025","eYx0rHMTxUA7PrLVYaeojr3SufrrTMP2okXFB7FKEEQ",{"id":11383,"title":11384,"author":23,"body":11385,"date":11783,"description":11784,"draft":181,"extension":19,"meta":11785,"navigation":24,"path":11795,"seo":11796,"stem":11797,"__hash__":11798},"blog\u002Fblog\u002F5-methods-of-triggering-an-automation.md","5 Methods of Triggering an Automation",{"type":8,"value":11386,"toc":11774},[11387,11390,11393,11422,11425,11428,11431,11439,11442,11450,11452,11458,11461,11464,11471,11479,11481,11487,11494,11497,11512,11516,11527,11531,11534,11545,11551,11574,11577,11589,11596,11602,11604,11611,11614,11632,11637,11640,11667,11670,11676,11682,11686,11693,11707,11714,11720,11726,11730,11737,11745,11752,11756,11759],[39,11388],{"alt":11389},"Trigger axiom bots using APIs",[11,11391,11392],{},"Now that you have created your automation, it's time to learn about the different methods of triggering the automation. There are quite a few different methods of triggering your automations, which one you choose really depends on your use case and what works best for you and\u002For your team. We will review a few different methods of triggering your automations in this article:",[249,11394,11395,11401,11407,11412,11417],{},[252,11396,11397],{},[50,11398,11400],{"href":11399},"#manual","Manual",[252,11402,11403],{},[50,11404,11406],{"href":11405},"#scheduling","Scheduling",[252,11408,11409],{},[50,11410,114],{"href":11411},"#zapier",[252,11413,11414],{},[50,11415,46],{"href":11416},"#ifttt",[252,11418,11419],{},[50,11420,6472],{"href":11421},"#api-and-other-services",[11,11423,11424],{},"Each trigger method has it's own advantages and disadvantages and ultimately the one you choose will be determined by your needs. Let's dive into these methods and give a bit more detail to help you decide what works best for you.",[43,11426,11400],{"id":11427},"manual",[11,11429,11430],{},"Manually triggering your automations is one of the easiest ways to trigger your automation and requires no set up to get started. There are two methods of triggering your automation manually:",[312,11432,11433,11436],{},[252,11434,11435],{},"From the Builder, hit \"Run\".",[252,11437,11438],{},"From the Dashboard, hit \"Run\" on the automation you wish to run.",[11,11440,11441],{},"Manually triggering your automations has a lot of benefits, such as being able to test your automations and watch the run to diagnose any issues. This allows you to simplify a complex process into a single button click that can be run whenever you need.",[11,11443,11444],{},[2786,11445,11446],{},[50,11447,11449],{"href":11448},"\u002Fdocs\u002Fno-code-tool\u002Fhow-it-works\u002Fschedule-and-run\u002Frun","Documentation",[43,11451,11406],{"id":6562},[11,11453,11454,11455,11117],{},"Scheduling your automations increases the autonomy of your automations - say that fast 10 times in a row! This feature allows for the automatic triggering of your automations on a schedule that fits your needs. Availability and the frequency that you can set your schedule will depend on your current subscription plan, see ",[50,11456,2807],{"href":11457},"\u002Fpricing",[11,11459,11460],{},"Your scheduled automation can be set to start on a specific date and will continue to run at the frequency that you have set until you manually cancel the schedule. For example, if you set it to start on the 1st January at 1am with a frequency of 1 day, it'll first run on 1st of January at 1am, and then run every 24 hours after that.",[11,11462,11463],{},"To increase the autonomy of your automations, we run your scheduled automations on the cloud - this means that you don't need to have your computer turned on for the automation to run. This can be super helpful for doing behind the scenes tasks, like getting the previous days sales numbers so you can hit the ground running when you get into the office. We appreciate that there are times when you may wish to run these locally - you can change this at any time, but your computer will need to be turned on and your browser open for your automation to run successfully.",[11,11465,11466],{},[2786,11467,11468],{},[50,11469,11449],{"href":11470},"\u002Fdocs\u002Fno-code-tool\u002Fhow-it-works\u002Fschedule-and-run\u002Fautomation",[11,11472,11473],{},[2786,11474,11475,11476,11117],{},"This method of triggering automations uses paid features, see ",[50,11477,11478],{"href":11457},"pricing",[43,11480,114],{"id":113},[11,11482,11483,11486],{},[50,11484,114],{"href":119,"rel":11485},[54]," can be used to trigger your automations from a larger workflow. This allows you to access their library of over 7,000 integrations that can be used to pass information into an axiom.ai automation. Need to trigger an axiom.ai automation everytime a new lead comes into Salesforce? You can. Want to make a log of uploads to a YouTube channel? You can. Need to pull in order details from Amazon Seller Central into an axiom.ai automation? You guessed it - you can!",[11,11488,11489,11490,11493],{},"To get started, select your trigger from the list of apps available in Zapier and set up your Zapier flow like normal. For the example below, we are going to create a workflow that triggers an automation when there is a new post in our ",[50,11491,129],{"href":3302,"rel":11492},[54]," using an RSS feed. This automation will log the new post in a Google Sheet and trigger a Slack Workflow that'll notify our team of new posts.",[11,11495,11496],{},"Let's jump over to Zapier and do the following:",[312,11498,11499,11502,11509],{},[252,11500,11501],{},"Create a new \"Zap\".",[252,11503,11504,11505,11508],{},"Add the \"RSS\" app as the first step, set the \"Trigger event\" to \"New Item in Feed\" and configure as instructed - we are using ",[50,11506,7613],{"href":7613,"rel":11507},[54]," RSS feed.",[252,11510,11511],{},"(Optional) Add a \"Filter\" step, and configure this to check the author name for anyone you may not want to be notified of posts from - in our case, we don't need notifications when we share new content!",[134,11513],{"src":11514,"alt":11515},"\u002Fblog\u002Freddit-filter-conditions.png","reddit step configuration showing \u002Fu\u002Fkarl_axiom filtered out",[312,11517,11518,11521,11524],{"start":440},[252,11519,11520],{},"Add another step and search for \"axiom.ai\", set the \"Action event\" to \"Run an Axiom\".",[252,11522,11523],{},"Configure as instructed, setting the automation name to match the name of your automation within the extension and setting the output from the \"RSS\" step into your column inputs using the \"+\" on the right of each \"Input column\".",[252,11525,11526],{},"Click \"Publish\" when you are ready.",[134,11528],{"src":11529,"alt":11530},"\u002Fblog\u002Freddit-run-an-axiom-rss.png","reddit step configuration showing run an axiom configuration",[11,11532,11533],{},"Now, let's jump over to your axiom.ai automation.",[312,11535,11536,11539],{},[252,11537,11538],{},"Create a new automation.",[252,11540,11541,11542,11544],{},"Add the ",[50,11543,6710],{"href":6709}," step, configure the \"Test data\" input with a sample of the data you expect to receive. For our example, we used the following to replicate the data we are expecting: \"title, link, description, author\".",[11,11546,11547,11548,11550],{},"From this point, you can add any steps that you'd like to use within your automation to get to your end goal. Use the \"Insert data\" option to access the data that was received by the \"Receive data from another app\" step. For multiple rows, you will need to use a ",[50,11549,5702],{"href":5701}," step to loop through each row of data to use the data within other steps. We have a few more things to add to complete our Reddit notifier.",[312,11552,11553,11562,11571],{"start":16},[252,11554,10880,11555,11557,11558,11561],{},[50,11556,330],{"href":7298}," step, configure as instructed and click \"Insert data\" to select the ",[387,11559,11560],{},"[webhook-data]"," from the dropdown - this will push the data received by the automation into your Google Sheet.",[252,11563,11564,11565,11567,11568,13],{},"We want to send this along to our customer support team to ensure that our users get the support they need, add a ",[50,11566,5576],{"href":5575}," step to send your data to a Slack Workflow. For more details on setting up the Slack Workflow, see our guide on ",[50,11569,11570],{"href":102},"How to trigger a Slack Workflow with axiom.ai",[252,11572,11573],{},"Click \"Save\" when you're ready.",[11,11575,11576],{},"Now, head over to your Zap and hit \"Run\" to test, or create a new post on the subreddit.",[11,11578,11579,11580,11583,11584,11588],{},"Keep an eye out in Zapier for any errors that may have occurred and follow their instructions to resolve the issues. If the issue has occurred within axiom.ai, you'll find this error noted in the ",[50,11581,10136],{"href":11582},"\u002Fdocs\u002Fno-code-tool\u002Fthe-builder\u002Fdashboard#run-reports"," of the automation. See our ",[50,11585,11587],{"href":11586},"\u002Fdocs\u002Fno-code-tool\u002Ftroubleshooting\u002Fhow-to-debug","How to debug"," guide for helpful tips on debugging your automations.",[11,11590,11591],{},[2786,11592,11593],{},[50,11594,11449],{"href":11595},"\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002Fzapier",[11,11597,11598],{},[2786,11599,11475,11600,11117],{},[50,11601,11478],{"href":11457},[43,11603,46],{"id":45},[11,11605,11606,11607,11610],{},"Similar to Zapier, ",[50,11608,46],{"href":52,"rel":11609},[54]," unlocks a massive library of over 900 services that you can use to automate workflows and trigger your axiom.ai automations. Where IFTTT differs is in the services that it connects to - they offer various connections to social media platforms, such as Facebook, LINE, and Instagram, as well as IoT services like Hue, Honeywell, and Govee. You are only limited by your imagination when you combine axiom.ai and IFTTT.",[11,11612,11613],{},"Here are some ideas on how to use IFTTT and axiom.ai to get you started,",[249,11615,11616,11621,11626],{},[252,11617,11618,11619,331],{},"Keeping a log of app updates using the IFTTT Apple App Store integration and axiom.ai's ",[50,11620,330],{"href":7298},[252,11622,11623,11624,13],{},"Trigger an automation using the Alexa integration and axiom.ai's ",[50,11625,6472],{"href":5474},[252,11627,11628,11629,13],{},"Re-post and log your LinkedIn posts using the LinkedIn integration and axiom.ai's ",[50,11630,11631],{"href":6626},"templates",[11,11633,11634,11635,11117],{},"While axiom.ai does not have an official integration with IFTTT, triggering your automations is simple. See our guide on ",[50,11636,65],{"href":64},[11,11638,11639],{},"To get started, create your IFTTT workflow as normal - selecting a service you'd like to trigger your axiom.ai automation from for the \"If this\" portion of your workflow. In order to trigger your axiom.ai automation, you'll need to add a \"Webhooks\" action to your IFTTT flow. You'll need to configure this action as per the instructions below:",[312,11641,11642,11648,11654,11660],{},[252,11643,11644,11645,13],{},"Set the \"URL\" to our current endpoint: ",[387,11646,11647],{},"https:\u002F\u002Flar.axiom.ai\u002Fapi\u002Fv3\u002Ftrigger",[252,11649,11650,11651,13],{},"Set the \"Method\" to ",[387,11652,11653],{},"POST",[252,11655,11656,11657,13],{},"Set the \"Content type\" to ",[387,11658,11659],{},"application.json",[252,11661,11662,11663,9588],{},"Configure the \"Request body\" as per our ",[50,11664,11666],{"href":11665},"\u002Fdocs\u002Fdeveloper-hub\u002Fapi\u002Frequests","API | Triggering an automation",[11,11668,11669],{},"Adding data from your trigger into the \"Request body\" field will allow you to send information from your service of choice for use within your axiom.ai automation.",[11,11671,11672],{},[2786,11673,11674],{},[50,11675,11449],{"href":64},[11,11677,11678],{},[2786,11679,11475,11680,11117],{},[50,11681,11478],{"href":11457},[43,11683,11685],{"id":11684},"api-and-other-services","API and other services",[11,11687,11688,11689,11692],{},"In addition to our official Zapier and Make integrations, we offer the ability to trigger your axiom.ai automations from nearly any third-party service that offers the ability to send outbound webhooks. Using the ",[50,11690,11691],{"href":6626},"axiom.ai API",", you can send data from a third-party service to be used within your automations. Common uses cases of this include:",[249,11694,11695,11698,11701],{},[252,11696,11697],{},"Connecting a service that we have yet to build an integration for.",[252,11699,11700],{},"Connecting to internal services that your organisation utilises.",[252,11702,11703,11704,11706],{},"Writing custom scripts that trigger your automations, batch files using ",[50,11705,6621],{"href":6620},", for example.",[11,11708,11709,11710,11713],{},"Want to request a new integration? Add your thoughts to our ",[50,11711,11351],{"href":11320,"rel":11712},[54]," over on Reddit.",[11,11715,11716],{},[2786,11717,11718],{},[50,11719,11449],{"href":5474},[11,11721,11722],{},[2786,11723,11475,11724,11117],{},[50,11725,11478],{"href":11457},[43,11727,11729],{"id":11728},"final-thoughts","Final thoughts",[11,11731,11732,11733,331],{},"While each method does have it's advantages and disadvantages, there are some things that you should be aware of when using an integration, the API or a schedule - as automations that are triggered using these methods are run in the cloud by default, this means that you are unable to use steps that require the desktop application, such as the ",[50,11734,11736],{"href":11735},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fdownload-file-step","Download a file",[11,11738,11739,11740,11744],{},"We have a guide available for running your automations that are triggered by an integration or the API on your local computer, see ",[50,11741,11743],{"href":11742},"\u002Fguides\u002Ftrigger-desktop","How to trigger a desktop run using webhooks"," for more details. Your computer will need to be turned on and your browser will need to be open for the automations to run. This is ideal for running your axiom.ai instance on a server.",[11,11746,11747,11748,11751],{},"When running your automations unattended, we recommend setting up run notifications to be alerted if your automation has an error - this will ensure that you can step in and fix the issues with your automation. Notifications can be sent via email or they can trigger a webhook to be consumed by another service. See ",[50,11749,11750],{"href":7480},"Learn how to set up notifications"," to learn more.",[43,11753,11755],{"id":11754},"additional-reading","Additional reading",[11,11757,11758],{},"Want to dive into more content? Check out our guides below.",[249,11760,11761,11768],{},[252,11762,11763,11764],{},"Integrations: ",[50,11765,11767],{"href":11766},"\u002Fdocs\u002Fno-code-tool\u002Fintegrations","\u002Fdocs\u002Fno-code-tool\u002Fintegrations\u002F",[252,11769,11770,11771],{},"API\u002Fthird-party service guides: ",[50,11772,11773],{"href":6626},"\u002Fguides\u002F",{"title":15,"searchDepth":16,"depth":16,"links":11775},[11776,11777,11778,11779,11780,11781,11782],{"id":11427,"depth":173,"text":11400},{"id":6562,"depth":173,"text":11406},{"id":113,"depth":173,"text":114},{"id":45,"depth":173,"text":46},{"id":11684,"depth":173,"text":11685},{"id":11728,"depth":173,"text":11729},{"id":11754,"depth":173,"text":11755},"2024-11-22","Learn five methods of triggering an axiom.ai automation. From manually triggering your run, setting up a schedule or triggering it from a third-party service.",{"read":8416,"type":184,"tool":11786,"category":11787,"tags":11788,"location":191,"featuredimg":192,"landingimg":193,"summary":11784,"video":18,"metaTitle":11384,"related":11790},[186],[2686],[6595,6423,11789,46,113,145],"schedule",[11791,11793,11794],{"path":5590,"format":11792},"blog",{"path":10072,"format":11792},{"path":10427,"format":11792},"\u002Fblog\u002F5-methods-of-triggering-an-automation",{"title":11384,"description":11784},"blog\u002F5-methods-of-triggering-an-automation","dRHuvwgcMKsn6KFDFUWmm3_gKZN406z93egDQhMtpGg",{"id":11800,"title":11801,"author":23,"body":11802,"date":12161,"description":12162,"draft":181,"extension":19,"meta":12163,"navigation":24,"path":12172,"seo":12173,"stem":12174,"__hash__":12175},"blog\u002Fblog\u002Fdebugging-like-a-pro.md","Debugging your browser automations like a pro",{"type":8,"value":11803,"toc":12145},[11804,11807,11810,11814,11817,11825,11829,11832,11836,11842,11846,11849,11853,11864,11868,11878,11882,11888,11891,11895,11899,11921,11936,11942,11992,12001,12005,12008,12012,12015,12019,12030,12037,12041,12056,12060,12064,12074,12078,12081,12088,12094,12100,12106,12110,12129,12131,12133,12143],[11,11805,11806],{},"As you begin to explore the world of browser automation you are likely to run into errors and issues with your automations - this is completely normal and is part of the learning process. Understanding the errors and issues that you are facing is key to being able to get back on track when something goes wrong.",[39,11808],{"alt":11809},"Learn to debug your browser automations",[43,11811,11813],{"id":11812},"what-is-debugging","What is debugging?",[11,11815,11816],{},"Debugging is the process of identifying, analyzing, and fixing errors or issues within a software program, or automation, to ensure it runs smoothly and functions as intended. It involves detecting where the code or automation fails or behaves unexpectedly and making the necessary corrections to improve performance and reliability.",[11,11818,11819,11820,11824],{},"In the context of automations, we want to understand which step has caused the issue, and why. For example, you may have a ",[50,11821,11823],{"href":11822},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fget-data-from-website","get data from bot's current page"," step that has returned the error \"your chosen selectors have failed to find any content on page \u002F element not found\" - this tells us which step to review, and what went wrong, in this case, that the selected element was not found when the automation has run.",[43,11826,11828],{"id":11827},"what-is-error-handling","What is error handling?",[11,11830,11831],{},"Error handling is the process of anticipating, detecting, and responding to errors in a software program, or automation, to ensure it runs smoothly and can recover gracefully from unexpected issues. We want to be sure that if your automation runs into an error that it can be handled gracefully and prevent the automation from failure.",[43,11833,11835],{"id":11834},"why-is-debugging-and-error-handling-important","Why is debugging and error handling important?",[11,11837,11838,11839,11841],{},"Debugging and error handling is an underrated but very important part of building software, and building out your automations. While debugging is more concerned with getting back on track once an error has occurred, error handling can be used to gracefully handle errors ",[2786,11840,10264],{}," they occur within your workflow. Knowing what to do when an error occurs can save a lot of time and effort compared to a trial-and-error method that looks a lot like finding a needle in a haystack.",[43,11843,11845],{"id":11844},"proactive-debugging-methods","Proactive debugging methods",[11,11847,11848],{},"There are steps that you can take before you run into errors in order to proactively prevent your automation from failing during it's runs. Investing time to implement preventative methods early in your development process can save time and effort in the longrun. Some of these actions may include catching errors during he run, adding additional data to your errors or logging debug information with JavaScript.",[43,11850,11852],{"id":11851},"optional-actions","Optional actions",[11,11854,11855,11856,11859,11860,331],{},"There are a couple of steps that allow you to set their action to \"optional\". This can be used if you are working with an element on the page that is sometimes present, sometimes not. This can be useful to dismiss cookie popups, for example, as these may not always appear in every run. This will prevent your automation from failing if the button can not be found on the page when the automation is run. This option can be toggled within the ",[50,11857,11858],{"href":5666},"click element"," step, and soon the ",[50,11861,11863],{"href":11862},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fenter-text","enter text",[43,11865,11867],{"id":11866},"catching-errors-during-runs","Catching errors during runs",[11,11869,205,11870,11873,11874,11877],{},[50,11871,11872],{"href":7392},"try\u002Fcatch"," step can be used to wrap steps that may cause an issue within your automation. Any steps included in the \"try\" section of this step will run as normal, however, if an error occurs this will trigger the steps that have been included in the \"catch\" section and your automation will not fail. There are various use cases for this, including running an additional set of steps if an error occurs because an interaction step cannot find an element on the page, catching an error from a ",[50,11875,11876],{"href":5581},"write JavaScript"," step, and catching errors from file handling steps to name a few.",[43,11879,11881],{"id":11880},"adding-error-metadata","Adding error metadata",[11,11883,205,11884,11887],{},[50,11885,11886],{"href":9366},"add error metadata"," step can be used to add additional context to your errors. When placed before an step that could potentially run into an issue this gives you the opportunity to add a custom message to your errors. This can be helpful in automations that may have multiple versions of the same steps that carry out similar functions.",[11,11889,11890],{},"This step may be used to output the value of a variable token that you are passing into a step - in the event of an error in the step that you are passing data to, the data will be output within the error message. Being able to view the data that you are passing into a step can be really helpful in understanding why the step may have had an error.",[134,11892],{"src":11893,"alt":11894},"\u002Fblog\u002Ferror-add-error-metadata.jpg","automation add error metadata step",[43,11896,11898],{"id":11897},"debugging-with-javascript","Debugging with Javascript",[11,11900,11901,11902,11905,11906,11908,11909,11912,11913,11916,11917,11920],{},"When running your automations locally, you have access to the ",[387,11903,11904],{},"console.log()"," function, and all variants of it, through the ",[50,11907,11876],{"href":5581}," step. This can be used to output the tokens from your automation as the automation runs - you can use the ",[5176,11910,11911],{},"insert data"," option to insert your tokens into your JavaScript code. When your automation opens in a new Chromium window, right-click on the page, click ",[5176,11914,11915],{},"inspect"," and navigate to the ",[5176,11918,11919],{},"console"," tab.",[381,11922,11924],{"className":383,"code":11923,"language":385,"meta":15,"style":15},"\u002F\u002F Output scrape data\nconsole.log([scrape-data])\n",[387,11925,11926,11931],{"__ignoreMap":15},[390,11927,11928],{"class":392,"line":393},[390,11929,11930],{},"\u002F\u002F Output scrape data\n",[390,11932,11933],{"class":392,"line":173},[390,11934,11935],{},"console.log([scrape-data])\n",[11,11937,11938,11939,11941],{},"When combined with JavaScript code, this can be an even more powerful tool for debugging. From manipulating data, requesting information from APIs or working with the node.js filesystem API, you can use ",[387,11940,11904],{}," to keep track of the information that you are working with to help you debug your automations. For example, if you are adding a constant to the last column within a dataset of data that has been scraped from a page, you may wish to view the data before and after it's been manipulated and before it's passed onto the next step. See how we implemented this below:",[381,11943,11945],{"className":383,"code":11944,"language":385,"meta":15,"style":15},"var scrapeData = [scrape-data];\nvar data = 'example';\n\nconsole.log(scrapeData);\n\nscrapeData.forEach((item) => item.push(data));\n\nconsole.log(scrapeData);\n\nreturn scrapeData;\n",[387,11946,11947,11952,11957,11961,11966,11970,11975,11979,11983,11987],{"__ignoreMap":15},[390,11948,11949],{"class":392,"line":393},[390,11950,11951],{},"var scrapeData = [scrape-data];\n",[390,11953,11954],{"class":392,"line":173},[390,11955,11956],{},"var data = 'example';\n",[390,11958,11959],{"class":392,"line":16},[390,11960,413],{"emptyLinePlaceholder":24},[390,11962,11963],{"class":392,"line":440},[390,11964,11965],{},"console.log(scrapeData);\n",[390,11967,11968],{"class":392,"line":458},[390,11969,413],{"emptyLinePlaceholder":24},[390,11971,11972],{"class":392,"line":482},[390,11973,11974],{},"scrapeData.forEach((item) => item.push(data));\n",[390,11976,11977],{"class":392,"line":494},[390,11978,413],{"emptyLinePlaceholder":24},[390,11980,11981],{"class":392,"line":500},[390,11982,11965],{},[390,11984,11985],{"class":392,"line":514},[390,11986,413],{"emptyLinePlaceholder":24},[390,11988,11989],{"class":392,"line":522},[390,11990,11991],{},"return scrapeData;\n",[11,11993,11994],{},[2786,11995,11996,11997,12000],{},"Tip: you may require a ",[50,11998,11999],{"href":5926},"wait"," step at the end of your automation to ensure that you have time to review the console.",[43,12002,12004],{"id":12003},"reactive-debugging-methods","Reactive debugging methods",[11,12006,12007],{},"Sometimes we come across errors that we are not anticipating during automation runs and we will need to debug on-the-fly in order to get the automation running as we want it to. These will often cause the automation to fail, but can be used as a learning experience. There are various methods that can be used, including reviewing error messages as they occur or reviewing run reports that have been generated by your runs.",[43,12009,12011],{"id":12010},"in-builder-error-messages-and-debugger","In-builder error messages and debugger",[11,12013,12014],{},"If you have the builder open when you are running your automation, you will see errors popup at the top of the builder detailing what went wrong.",[134,12016],{"src":12017,"alt":12018},"\u002Fblog\u002Ferror-report.jpg","automation error report",[11,12020,12021,12022,12025,12026,12029],{},"You'll also find the debugger within the builder, there are a couple of methods of opening this: in the overflow menu on the top-right of the builder, click ",[5176,12023,12024],{},"debugger",", or click ",[5176,12027,12028],{},"debug"," on any error messages that appear in the builder. The side panel that appears will give you more contact on the error that you have within your automation - including the step number that encountered the issue.",[11,12031,12032,12033,12036],{},"For longer automations, you can toggle on the ",[5176,12034,12035],{},"only show errors"," option to only show the steps with errors - there is often only one as the automation will stop when it encounters an error, but you may find some additional warnings that may help diagnose the issue.",[43,12038,12040],{"id":12039},"run-reports","Run reports",[11,12042,12043,12044,12047,12048,12051,12052,12055],{},"Every automation run produces a ",[50,12045,12046],{"href":9358},"run report"," that can be found within the ",[5176,12049,12050],{},"run reports"," section of your dashboard in the axiom.ai Chrome extension. When an automation fails due to an error, or succeeds with warnings, these will show within your run report. From the run reports section of the dashboard, hit ",[5176,12053,12054],{},"full report"," to learn more about the errors or warnings.",[134,12057],{"src":12058,"alt":12059},"\u002Fblog\u002Ferror-run-report.jpg","automation run report showing error",[43,12061,12063],{"id":12062},"displaying-variable-tokens-in-a-popup","Displaying variable tokens in a popup",[11,12065,205,12066,12070,12071,12073],{},[50,12067,12069],{"href":12068},"\u002Fdocs\u002Fno-code-tool\u002Freference\u002Fsteps\u002Fdisplay-a-message","display a message"," step is a small but powerful step for debugging, it allows you to output data as a popup within the builder so that you can see see the contents of a variable token. You can add a token to this step by clicking ",[5176,12072,11911],{}," and selecting the variable token that you would like to display. This will display as a table when you insert a variable token and can be helpful when trying to visualize the data that you have within your automation, scrape data is a good example of data that could be displayed this way but any token may be used.",[43,12075,12077],{"id":12076},"understanding-the-structure-of-errors","Understanding the structure of errors",[11,12079,12080],{},"Understanding the structure of errors is an important skill to have, let's dive into the format of an axiom.ai error message and break it down into it's constitute parts.",[381,12082,12086],{"className":12083,"code":12085,"language":4686},[12084],"language-text","Error in step 4 - \"Enter text\": Couldn't find the element during run. Please reselect this element in the selector tool or use a custom selector.\n",[387,12087,12085],{"__ignoreMap":15},[11,12089,12090,12093],{},[5176,12091,12092],{},"Error in step 4"," - there has been an error in step 4 of your automation, this give us a good start on where to look for any issues.",[11,12095,12096,12099],{},[5176,12097,12098],{},"\"Enter text\": Couldn't find the element during run"," - the error message is telling us what went wrong, in this instance, that the element that has been selected has not been found on the page. This may be due to the page not loading correctly, or changing since the automation was set up.",[11,12101,12102,12105],{},[5176,12103,12104],{},"Please reselect this element in the selector tool or use a custom selector"," - this is advice on how to resolve the error that has occurred - for this particular error we may need to revisit step 4 and re-select the elements on the page.",[43,12107,12109],{"id":12108},"finding-information-on-common-errors","Finding information on common errors",[11,12111,12112,12113,12116,12117,12120,12121,12124,12125,12128],{},"You can read up more on our common errors in our comprehensive ",[50,12114,12115],{"href":10140},"list of errors",", through our ",[50,12118,3304],{"href":127,"rel":12119},[54]," for free users, or by contacting ",[50,12122,12123],{"href":11115},"customer support"," if you are a paid subscriber. You can also find our full documentation on ",[50,12126,12127],{"href":11586},"how to debug"," in our help center.",[10127,12130],{},[3295,12132],{},[11,12134,12135,12137],{},[10127,12136],{},[2786,12138,12139,12140,13],{},"Have you any tips and tricks that you have learned while using axiom.ai that you think would help other users? We would love to hear from you over in our ",[50,12141,3304],{"href":3302,"rel":12142},[54],[2666,12144,3800],{},{"title":15,"searchDepth":16,"depth":16,"links":12146},[12147,12148,12149,12150,12151,12152,12153,12154,12155,12156,12157,12158,12159,12160],{"id":11812,"depth":173,"text":11813},{"id":11827,"depth":173,"text":11828},{"id":11834,"depth":173,"text":11835},{"id":11844,"depth":173,"text":11845},{"id":11851,"depth":173,"text":11852},{"id":11866,"depth":173,"text":11867},{"id":11880,"depth":173,"text":11881},{"id":11897,"depth":173,"text":11898},{"id":12003,"depth":173,"text":12004},{"id":12010,"depth":173,"text":12011},{"id":12039,"depth":173,"text":12040},{"id":12062,"depth":173,"text":12063},{"id":12076,"depth":173,"text":12077},{"id":12108,"depth":173,"text":12109},"2024-11-07","Learn how to debug your axiom.ai automations like a pro by understanding error messages and using error handling steps to fix errors and issues within your automation.",{"metaTitle":11801,"read":6211,"type":184,"tool":12164,"category":12165,"tags":12166,"location":191,"featuredimg":12170,"landingimg":12171,"summary":12162,"video":18},[354],[2686],[7523,12167,12168,10141,12169],"error handling","try catch","debugging","\u002Fblog\u002Fdebugging-postbox.jpg","\u002Fblog\u002Fdebugging-sq.jpg","\u002Fblog\u002Fdebugging-like-a-pro",{"title":11801,"description":12162},"blog\u002Fdebugging-like-a-pro","42NCq1gVjvmeJEf3VZhJlGtQWZlyPKb-cpvpNYhw4Cc",1782914872311]