Web scraping has always been… messy, especially if you don’t know HTML too much.

h1’s tags body’s…

Messy.

I’ve written dozens of scripts, some of which might even get me mad!

As a data scientist, web scraping is like my best friend, whom I’ve not spoken to for a while.

He was always there when I needed him, but never easy to deal with.

Enter Claude’s MCP: Planning, Not Prompting

Reference

Over the last period, I have been reading news about Claude’s MCPs.

Yesterday, I sat and read everything about it and was amazed!

MCP(Model Context Protocol) — a way for Claude to connect with tools and data sources in a clean, standard way.

It’s like giving Claude super-powers. Here you can find options.

Reference

But let’s stop chit-chat.

Let’s start!

Getting Started: How to Set Up Claude MCP in Your Workflow

Photo by Jake Hills on Unsplash

These are pretty straightforward steps, so there is no need to add comments inside. Here are the steps to follow.

Steps-to-Follow

• Install Claude Desktop here.
• Sign in to FireCrawl here
• Settings/Developer/Edit config file
• Visit here and copy this code block.

"mcp-server-firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "api-key-here"
      }
    }

• Paste this into the claude_desktop_config.json.

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-filesystem",
        "/Users/[PCNAME]/Desktop",
        "/Users/[PCNAME]/Downloads"
      ]
    },
    "mcp-server-firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "api-key-here"
      }
    }
  }
}

• Quit Claude Desktop and open it again.

The Test: My Claude Agent Scraped the Web So I Didn’t Have To

Photo by Kelly Sikkema on Unsplash

I always wonder:
What makes an article really take off on Medium?

Is it the headline? The format? The timing?
Maybe all of it. Perhaps something else entirely.

So I decided to find out — but with help.

Instead of manually crawling through dozens of posts under the AI tag on Medium, I handed the task to my Claude-powered agent.

Here is my prompt;

Visit here: https://medium.com/tag/ai/recommended , 
pull 15 recommended articles, extract their titles, authors, claps, and 
publish dates. 
Then build a dashboard to show the results.

It will ask you to click on allow.

That’s it.

Claude took the plan, triggered Firecrawl, parsed the HTML, and structured the data beautifully.

No broken CSS selectors.
No headaches with pagination.
No outdated BeautifulSoup scripts from Stack Overflow.

Dashboard Claude Generates

Suddenly, I wasn’t debugging. I was exploring.
I wasn’t scraping. I was strategizing. And that shift? That’s where the real value lives.

Final Thoughts

This wasn’t just a test.

It was a part of the future about how far AI has come.

Here, Claude did not just answer me.

It reasoned, delegated, and adapted. It gave me back something valuable over time:

Clarity.

Now, instead of debugging the code, I can focus on what really matters —

Finding insights, telling stories.

Now I am like a boss who has his team.

If you want to follow AI news, use our assistants and catch the future, visit our platform, where you will have them all.

Here are the free resources.

Here is the ChatGPT cheat sheet.

Here is the Prompt Techniques cheat sheet.

Here is my NumPy cheat sheet.

Here is the source code of the “How to be a Billionaire” data project.

Here is the source code of the “Classification Task with 6 Different Algorithms using Python” data project.

Here is the source code of the “Decision Tree in Energy Efficiency Analysis” data project.

Here is the source code of the “DataDrivenInvestor 2022 Articles Analysis” data project.

“Machine learning is the last invention that humanity will ever need to make.” Nick Bostrom