Scrape data from Instagram

What I mean by scraping Instagram

Scraping Instagram means a bot reads public information off the website into a sheet. A profile's name, bio, and counts, the captions and dates on public posts, the results under a hashtag or a search. Point it at a profile or a list of them and it gathers the same fields across each, so research that meant scrolling becomes rows you can sort.

Public data, within Instagram's limits

Instagram is one of the strictest sites to automate. It rate-limits actions and watches for anything that looks like a bot, so scraping too fast gets you blocked. The numbers move, but the rule does not, go gently or get stopped.

So this is small, slow scraping of public data. Public profiles, public posts, hashtag results, into a sheet, at a human pace with real pauses. Private accounts and anything behind a follow are off the table, that is not public, and it is not yours to take. Read what people chose to show the world, slowly, and stop the moment Instagram pushes back.

Who this is for

This is for the researcher or marketer gathering public Instagram data for their own work. Mapping creators in a niche, tracking a hashtag, pulling public follower counts for analysis. Public data, low volume. If you want to harvest at scale or reach private data, this is not the tool. No-coders and coders both, since you can build it without code and drop into code when you want.

How I'd approach it

Make a list of the profiles or hashtags you care about. Point the bot at the first, read the public fields you want, and write them to a sheet, then loop the list slowly with pauses between each. Take only what is public, watch for any block, and back off if you see one. Public fields, a steady pace, stop on a block.

I would lay out the first draft with Build with description.

Scrape Instagram from a description

Describe what you want in plain words in the Chrome extension and it builds the steps for you. Give it a few short lines, check what it made, and run it. Explore no-code.

Scrape Instagram in code

Build with code. If you would rather script it yourself, this is the path. Explore code

import { chromium } from "playwright";

const browser = await chromium.connectOverCDP(
  `wss://cdp-lb.axiom.ai/?token=${process.env.AXIOM_API_KEY}`
);

try {
  const context = browser.contexts()[0];
  const page = context.pages()[0] ?? await context.newPage();

  // Public profiles, from your sheet
  const profiles = ["https://www.instagram.com/some_public_profile/"];
  const data = [];

  for (const url of profiles) {
    await page.goto(url);
    await page.waitForSelector("header");

    data.push({
      name: await page.locator("header h2").innerText(),
      meta: await page.locator("header li").allInnerTexts(), // public counts
    });

    await page.waitForTimeout(5000); // a human-paced pause
  }

  console.log(data); // write these to your sheet
} finally {
  await browser.close();
}

+

Build with a Claude skill

Build no-code or code bots with a skill.

Add the Claude skill and describe the public Instagram data you want. It builds the scraper for you, no-code or code, into a sheet, slowly.

Install the skillLearn more

What can you scrape?

Public data, small and slow. A couple of lines to stay on the right side of.

What I'd watch out for

Instagram punishes scraping that overdoes it, so read this one carefully. Here is what I would watch for.

Stay well under the limits

Instagram blocks fast. Run few requests, space them out, and stop at the first sign of a block. A block is the warning, a ban is what comes next.

Public only

Read public profiles and posts, never private accounts or data behind a follow. Respect what people chose to make public.

Use your own session, slowly

Sign in as yourself, store the cookies, and ease into it. Bursts of activity are exactly what Instagram flags.

Pick fields reliably

Instagram changes its markup, so pick what you read with the selector tool and recheck after a big update.