Firecrawl: Як отримати «очі» для вашого AI та заробити на це

This episode is the clearest explanation of firecrawl on the internet and how you can use it to build a real business that makes you real money. Firecrawl feels like giving your AI eyes. Right now, AI is smart, but it's blind. It can't see the internet. It can't go to a website. It can't grab data. So, Firecrawl fixes that. Once you see it in action, it changes how you think about building products, how you think about collecting data, and how you think about what's possible with AI. In this episode, I break down what firecrawl actually is, how it plays into your AI stack, and walk you through a bunch of startup ideas that you can make money from it. I use firecrawl with ideabser.com, and I reached out to them to ask them to sponsor this video. They said yes so that more people can see this, get the sauce, and build and make money with it. If firecrawl has been on your radar and you just want a clear explanation of what it is and how you can use it as a founder, then this episode is for you. And if you've never heard of it, honestly, that's even better because what I'm about to show you is going to change how you think about what you can build with AI and where the next 12 months of building is going. Let's get into it. It's time. >> By the end of the episode, you're going to understand why AI is blind, why it needs hands and eyes, why firecrawl is that, and why the people that understand how to use firecrawl are going to be able to create SAS apps and software that are super super valuable to people. I'm talking the most valuable software products are going to be using this data scraping tool at the backbone because it makes their AI 10 times smarter. But in order to understand this, we need to take a step back. The problem is AI is blind. If you listen to this channel, you know that, you know, the more context you give to a claude, the more context you give to a chat GPT, the better output you're going to get. So, we know that AI models need web data. It needs top tier data to actually go and provide really good outputs. Why does this matter now? Well, it matters because you know if you think about the first era of AI, that was the chatbot era. Chat GPT just came out 2022. It answers questions. It was cool but pretty limited. Then we entered the co-pilot era. Uh you know cursor GitHub co-pilot. It was faster but you still needed to drive. It was you the human being that was doing it. We've now entered this AI agent era. AI is doing the work for you. Things like cloud code. It browses, it researches, it builds, but it still needs the data. And Firecrawl is how you're going to get that data. This is often called the computer use era. We now have AI agents that can see and control computers. In the past, it was human beings, right? we bought mouses and keyboards and we had human beings actually going and clicking and doing things right that's you know going to be the minority as weird as it is to say that you have tools like perplexity computer open AI had operator came out about a year ago uh AI browses the web for you you know GPT 5.4 four beats humans at computer tasks. You know, code has its computer use API, screenshots and clicks. It's got full desktop control. Manis was the one who uh was one of the first to do that. You have browser use, which is an open-source um you know, so you all these computer uses, all these AI agents that are going and doing things, well, what do they all need? Well, they need clean web data. And that's firecrawl. And the reason I got interested in firecrawl is because I built ideas browser.com. And ide.com is a place where you have trends and the best startup ideas on the planet. And I needed the data. I needed the trend data. And we built on top of firecrawl to actually go and get some of that data. Now we have the number one startup ideas and trends product on the planet. And it's all because in largely part that we're, you know, using tools like firecrawl to actually go and and get that data. What most people don't get about this whole era that we're in is they think that AI is just chat bots that answer questions. They think web scrapers are illegal and shady. They think you need to code everything yourself. They think data is free and easy to get. And they they think that, you know, web scraping is a, you know, it's a thing for developers. But what it actually is happening is AI agents are doing work autonomously. You know, web data is critical AI infrastructure. Literally critical. One API uh call replaces thousands of lines and clean structured data is the new oil. By the end of this episode, I think you're going to agree by that. So the people that understand how important uh the clean data is and how important you can use the clean data and wrap it around a brain, an LLM, and wrap that around a piece of software. Um those are the people that are going to be able to create the most valuable startups in the next 12 months. And I think that the people understand that have a 12-month head start. And that's why I wanted to make this episode. Traditional scraping versus new scrapers like firecraw. Let's just talk about that so we can understand what the difference is. The old way of scraping was you wrote a custom scraper per site. You managed proxies in browsers. You handled anti-bot detection. You had to parse messy HTML manually. The scripts would break when site changes. This happened all the time. Basically, it was a massive headache. Now, you just do one API call. You get clean data back in seconds. It could work on any site. uh or I think like 99% or 98% some some high 90% of sites and the AI handles layout changes. So the way I think about uh my agent stack um is that every builder if you're listening this you're probably going to need five different layers. You're going to need an a agent harness. So that's going to be something like a cloud code cursor codeex or idea browser pro. uh you're going to need something that basically, you know, is handling all the different agents uh in one place. Then you're going to need something like a search layer. So something that's going to go and search different things like Perplexity has a good MCP. Exa uh as well. Then you're going to need a web data layer and that's what we're talking about today in this episode. So you're going to use Firecrawl for scraping, browsing, and extraction. um firecrow basically the web data layer your agents need to see the internet you're going to need need to be able to see the internet to see the data in order to provide value back in the form of a startup and software you're going to need an ops brain so I did recently did an episode I encourage you to listen to it if you haven't already around obsidian and cloud code so you're going to you know I don't care if you use notion I don't care if you use Apple notes but you're going to need some brain for, you know, storing your meeting notes, storing your context. Um, and you can use something like notion or obsidian. And then you're going to have to have some outbound and audience uh stack as well, something like an instantly and Apollo. And you know, if people are interested, I can spend more time and do a whole separate episode on some of these tools. But today we're going to be talking about the firecrawl uh the firecrolled web data layer. So what is it? What is firecrawl? What is the clearest way to understand it? You put in a website, goes through the firecrawl a API and you get back a clean markdown, a structured JSON, some screenshots, and you can feed that to any ai model. That's it. It's simple as that. We don't need to overthink about it. Think it the way I think about it is firecrawl has six superpowers. You can scrape. So, you can go and scrape one page to a clean markdown. So something like um scrape one blog post from you know gregisenberg.com uh blogpost you can crawl an entire site automatically. So what do I mean by that? I mean you can go and say give it cnn.com and it's going to go and crawl all of the different articles on CNN.com and you'll get that data back. You can map all URLs on a domain instantly. So that's super helpful. There's so much metadata and context into mapping and URLs. Maybe, you know, think about a URL. Maybe there's a date in it. There's a title in it. Having that map is going to be helpful in some capacity to to you depending on what you're trying to do. You can go and search. You can use Google and you can put, you know, the full content in one call. Super super valuable. It has uh an agent that you can describe data and it goes and finds it. you know, tell it, I want the 50, you know, highest rated uh Cuban restaurants in South Florida, and it's going to give it back to you. Going to give you the most clear data on it as well. And then it's got a browser. So, AI controls a real browser. Um, super super helpful. And it's three lines of code. You can screenshot this or I'll put it in the description uh for how to sign up, but basically it gives you a clean markdown of the entire website for any AI model in three lines of code. This is what excites me about it. So I believe that this is the AWS moment for web data. What do I mean by that? In 2006, if you wanted to build a web app, what did you have to do? Well, you had to go out and buy servers. Spend thousands of dollars buying servers. You had to go and manage racks and cables. Things would break all the time. All the time. All the time. And then one day AWS said one API call and you can use our servers in the cloud. Now in if you want AI to use web data what do you have to what you have to do? You had to build scrapers, manage proxies, manage browsers, deal with security. Firecrawl says one a API call and we got you. This is a big deal because the companies that built were built that were built on top of AWS, some of them became trillion dollar companies, some of them became billiond dollarar companies and a lot became million-dollar companies. Uh, of course, a lot failed, but the point is it be, you know, people didn't have to deal with the headaches of servers. So, they got to focus on building incredible product and they they were those products were able to scale. some of the, you know, the biggest companies of the last 10 years came because of AWS. So, what gets built on the web data layer? I'm going to give you some ideas on some, you know, not billion dollar ideas, but some multi-million dollar, you know, 1 to10 to $25 million a year, $50 million a year businesses uh that you can start by understanding, you know, what the web data layer is. And I think a lot of people are sleeping on how how big of a movement this is. So let's go into how uh how it works. So um you know here you are right you're the builder um you've got this AI agent and the AI agent is going to go and talk to your brain. So you you can use GPT, you can use code, you can use Gemini, you've got a nervous system. I that's the way at least I think about it, which is your MCP protocol. And now you have your eyes and hands. Your eyes and hands is firecrawl. Now firecall can go out to the internet and it's going to get back clean data and you're going to use that data to wrap it around products and services you sell. So this is the big idea, right? You've got brain, you've got nervous system, and you've now got eyes and hands. Um, of course, you can go and do it yourself scraping. You can use, you know, playright or selenium. You're going to just, the bottom line is it's just going to be a lot of work. I'm trying to do the simplest thing possible. So, the reason I like firecrawl, um, is it's one API call, proxies are built in, anti-bot built in, the AI extracts the data for you. It's just less headaches than actually going in doing it yourself. And you and you've got the browser sandbox which is really cool. So the browser sandbox it's a secure way to fill out to have uh fire crawl fill out forms click buttons and links handle login and off navigate pagenation you can watch live as your AI browses stay logged in across sessions. It's really crazy, right? So the, you know, think about it in a world where you can go and have, you have these hands and eyes out there on the internet, you know, what are the big ideas that you can build? And we're going to be talking, we're going to be talking about that soon. Um, so you know, the way the agent endpoint works is you type in a prompt, the firewall agent searches the web, it clicks through pages, it extracts data, and it returns the JSON. So um if you think about the AI infrastructure stack, I think about it like layers of the internet, you've got applications, you've got uh like chat GBT, Perplexity, a SAS product, you've got AI agents, um you've got protocols, you've got web data, and you've got the internet. So I believe that people are sleeping on the web data layer. Um, and if you understand how to get, you know, great data out of, you know, tools like Firecrawl and Exa, you can build, you know, the picks and shovels of the AI gold rush. So, let's just talk about what an agent prompt if you prompt fire call like what can you actually get back? So you can say find all of Y Combinator's you know winter 24 dev tool companies and their founders and emails and what you get back is a structured list of 50 plus companies with names and contact info. You can say compare pricing tiers across Stripen, Square and PayPal and you get sidebyside pricing table with all features and costs. You can say get all running shoes from Nike under $150 with ratings and you get back full product catalog with specs and prices. And you could say find 50 AI research papers from 2024 with citations. You get the academic data set with authors and institution and institutions. So super super um powerful stuff. Now let's talk about a few ideas that you can use to go and build uh build you know using firecrawl. So price the first idea is around price monitoring. So, there's tools like uh Precinct and Visual Ping, which you'll pay, you know, $200 to $1,000 a month. You basically get get an e-commerce focused price monitoring software. There's a self-s served dashboard. It tracks any product, but why don't you just use Firecrawl? You can build this probably in a weekend and you can build a sneaker resale prices only. So, auto alerts on StockX, on goat, on eBay. and eBay, you can charge $50 to run or sell for $500 a month. So, basically, pick a niche. Um, you know, could be sneakers, it could be, you know, collect, you know, different collectibles, it could be whatever, and use that as u, you know, I'm just using sneaker re resale as an example, right? It could be any any niche that you understand better than someone else. um and setting alerts and you know and then just charging people to people to use it. Number two, SEO SEO gapfinder. So HFS and SEM Rush like you know I think SEM Rush just sold for like $1.9 billion or something. Hrefs probably does hundreds of millions a year in revenue. They charge hundreds of dollars a month. Uh it requires SEO expertise. It's got these complex dashboards. It's pretty general purpose. What if you use Firecrawl to create uh you know SEO audits for dentist only? So Firecrawl reads competitor sites plus GMBB listings. You know you get a one-click report. So you rank for 12, they rank for 47. And then you sell the reports for maybe it's $500 or $200 a month. So again, take a big idea that's already generating hundreds of millions of dollars. you recreate it very quickly with a very niche spoke uh focus and again these are just example niches but it could be you know Canadian dentist if you even want to go more niche think about uh Indeed Zilla Well these are massive uh horizontal platforms they've got billions in fund funding that's generic search for everyone they use mostly do ad supported models so what if you did a fire crawl version maybe you just do remote AI and ML jobs only. Firecrawl monitors 500 company career pages daily. So, it's going and grabbing that data. The AI filters and ranks by fit score. And then you can charge for premium alerts for $29 a month. Indeed has 300 million listings. Nobody wants 300 million. They want 50 that matter. Again, this is why Fire Crawl is really good at getting the top stuff. AI re research reports. So yes, there's big companies like Consensus or Tavali, but these are general purpose research, academic or broad. The user does the prompting and there's no vertical expertise. What if you did like a niche crypto token due diligence reports? So you have firecrawl read papers in Twitter and other places. It autogenerates a risk score and summary and then you can sell that to VCs, private equity or different funds for, you know, a,000 to 500 $5,000 a month. A VC will pay $5,000 for a report that saves them from a bad 500k bet all day long. So, uh, again, picking a niche, getting the best possible data. Uh, a couple more ideas. um an agent in the box. So you know you have Harvey Aai uh it's got you know now hundreds I think of millions in funding. It's got an enterprise sales cycle horizontal agent platform. It takes months to cut customize. What if you did like a real estate comp report agent? So you use firecrawl to pull listings, tax records and permits and the agent generates comp reports in like 30 seconds and then you sell that to retailers for $300 a month. So, don't raise any money. You go and do this $300 a month. Um, you know, could could work review intelligence. So, yes, there's there's companies like Brand 24 and App Follow. Uh, they charge few hundred bucks a month. They basically monitor social and reviews broadly. Their dashboards for marketing teams, generic sentiment analysis. But what if you did an Amazon FBA seller review tracker? So, Firecrawl monitors competitor review daily. The AI spots trends, right? Complaints about battery life up to 40% and you sell that to Amazon sellers for $99 a month. And something like this could also, by the way, get acquired by like a Shopify or an Amazon. Amazon sellers will gladly pay $99 a month to find product gaps before uh competitors do. So, these are just a few ideas to get your creative juices flowing around how to use firecrawl to scrape ideas. Scrape ideas. Go niche and and you can compete on price. You can compete on nicheness. I don't know if that's a word, but we're we're going with it. and uh and and and just create like I said, you know, clean structured data uh using AI to actually build and vibe code a lot of these products um and start, you know, start selling them to to these niches that are looking for um that that they're looking for this stuff and they they they want, you know, the truth is the vertical the reason why like vertical software is such a big business. Why is uh Constellation Software, you know, almost a $75 billion company or whatever? They have hundreds of vertical software companies because people like buying very specific products. So, there's always going to be room for these horizontal ideas. There's always going to be room for the SEM rushes and the Indeeds and the LinkedIn, stuff like that. But if you can carve out a little niche that could do 1 million a year to 10 million to 20 million to 30 million, there's opportunity there. Um, you know, incumbents are charging hundreds of dollars a month for generic tools. Your your version charges, you know, 20, $50, $70 for a tool that does one thing perfectly for one customer. So another idea would be to build a legion lead genen business. So a client gives you 50 company names. What if you uh grabbed a fire crawl agent that founders and emails? It returns a structured JSON with all data. You deliver enriched the enriched CSV and you just charge I don't know $500, $200, $100 per batch. Your cost is like $2 in Firecrawl credits. Firecrawl actually have here like there's a bunch of free, you know, there's a free tier. The agent run gives you five free per day. Um, and then you know to scrape cost one, credit, a crawl cost one. Um, but the point is like you know if you can figure out a way to you know get 95% margin, 98% margin, 99% margin. Um, you're happy uh clients happy uh because you know hopefully they're closing on some of these deals, right? So there's something here around uh you know using some of the data charging per output and creating high margin businesses. This is the framework for how I would think about how you can build and make money with firecrawl this week. So the first step is going to be picking a niche. So what data do people in this industry actually pay for? The second step is going to be building the scraper. So use fire crawl agent, maybe a simple Python script and nflow or just use cloud code to go and build that for you. Step three is going to package it. So CSV or dashboard or Slack alert or API. And step four is going to be about selling the output, right? Not just the tool. You're going to be selling the data. So you can charge maybe $500 to $5,000 per month per client. And then you're going to automate it. How do you schedule it and and let it run while you sleep? Compounding clients and that sort of thing. So, I think that a lot of people are going to be starting to do this. They're going to be picking niches. They're going to be building scrapers. They're going to be packaging it. They're going to be selling the output and they're going to automate it. It's a flywheel that I think is just just getting started. So, just a few more ideas uh for you. You can do something like real estate pricing data. You can do SAS competitor monitoring. You can do job aggregation. You can do patent and legal filings. You can do influencer contact databases. You can do government contact alerts. You can do e-commerce price tracking tracking. You can do academic research data sets. And then you can, and this is what I suggest you do is just do more niche versions of this, right? So, real estate pricing, go more niche. SAS competitor monitoring, go more niche. This is just ideas to get your creative juices flow flowing. So, how I actually heard I want to end with this, but how I actually heard about um Firecrawl was, you know, a year ago, uh, I tweeted this actually. I saw that they had posted a job saying they were hiring a Firecrawl example creator, but they only wanted to hire an AI agent. So, they said, "Please only apply if you're an AI agent. We're seeking an AI agent capable of autonomously researching trending tech and models and then using the information to create, test, and refine highquality example applications. These sample apps will live in our example repository showcasing the full potential of firecrawl in real world scenarios. Your work will guide and inspire developers helping them quickly adopt fireside firecrawl alongside modern tools and approaches. So, uh, if Firecrawl is hiring AI agents as employees, it got me thinking that this is probably where the world is is going. So, for example, hiring a content creator agent, writes blog posts autonomously, watches metrics and improves, maybe that's a $5,000 per month salary. A customer support agent, handles tickets in two minutes, knows when to escalate, maybe that's a $5,000 per month salary. a junior develop developer agent, triage GitHub issues, writes docs and code. That's a $5,000 per month salary. So that's a million to million dollar total budget. Uh 50 applications in the in the first week. So my startup idea was, you know, how do you build AI agents that companies like Firecrawl want to hire? Yes, it looks super weird right now that Firecrawl is hiring a AI agent. Um, and I, you know, feels like a little bit of a joke, but I think that, uh, it got me thinking that using tools like Firecrawl and building products and agents around it, uh, you know, I could see a world where this becomes more and more popular, right? Um, and I think that there's an opportunity to think about it as, you know, from a framework perspective is how can you use tools like fire firecrawl to build AI agents and build products that would, you know, that that companies would want to hire. Um, so I just thought, by the way, I just thought that that was, you know, just wanted to end with that. So overall, this is my breakdown for why I think there's a tremendous opportunity in uh in in the web data layer and using firecrawl for scraping, why I think there's a lot of ideas around it. Um and uh yeah, you know, hope this got your creative juices flowing. Um, it's certainly something that I'm exploring in real time building products uh with with Firecrawl because uh it's valuable. It's it's super valuable in um in getting the right data and uh and it's just working. So, hope this has been helpful. Please comment if like what you want to see next from me, what do you want me to teach you. Um, I'm just sharing things that I'm learning in real time and hopeful that it it it's helping you along your journey. So, thank you so much for if you made it to the end, thank you so much for being here. I'm rooting for you for whatever it is you're building and I can't wait to see you on the next episode.