Close Menu
Best in TechnologyBest in Technology
  • News
  • Phones
  • Laptops
  • Gadgets
  • Gaming
  • AI
  • Tips
  • More
    • Web Stories
    • Global
    • Press Release

Subscribe to Updates

Get the latest tech news and updates directly to your inbox.

What's On

iPhone 17 to Feature Slightly Larger Display Than iPhone 16, Tipster Claims

28 June 2025

Monster Hunter Wilds Title Update 2 Features Returning Beasts, New Cosmetic Options, And More

28 June 2025

Pragmata Gameplay Breakdown Details Its Blend Of Hacking And Gunplay

28 June 2025
Facebook X (Twitter) Instagram
Just In
  • iPhone 17 to Feature Slightly Larger Display Than iPhone 16, Tipster Claims
  • Monster Hunter Wilds Title Update 2 Features Returning Beasts, New Cosmetic Options, And More
  • Pragmata Gameplay Breakdown Details Its Blend Of Hacking And Gunplay
  • The Era of ‘Woke’ Brand Activism Is Over
  • Super Mario Strikers Is The Next GameCube Title Coming To Nintendo Switch Online + Expansion Pack
  • The Obsessive Fans Playing God on Love Island—and Living for the Crash-Outs
  • How Gearbox Designs “Billions of Guns” in Borderlands 4
  • The 28 Best Outdoor Deals From the REI Fourth of July Sale
Facebook X (Twitter) Instagram Pinterest Vimeo
Best in TechnologyBest in Technology
  • News
  • Phones
  • Laptops
  • Gadgets
  • Gaming
  • AI
  • Tips
  • More
    • Web Stories
    • Global
    • Press Release
Subscribe
Best in TechnologyBest in Technology
Home » Good Search Borrows, Great Search … Steals?
News

Good Search Borrows, Great Search … Steals?

News RoomBy News Room20 June 20242 Mins Read
Share Facebook Twitter Pinterest LinkedIn Tumblr Reddit Telegram Email
Share
Facebook Twitter LinkedIn Pinterest Email

Web crawling—the act of indexing information across the internet—has been around for decades. It has primarily been used by search engines like Google and nonprofits like Internet Archive and Common Crawl to catalog the contents of the open internet and make it searchable. Until recently, the practice of web crawling has rarely been seen as controversial, as websites depended on the process as a way for people to find their content. But now crawling tech has been subsumed by the great AI-ening of everything, and is being used by companies like Google and Perplexity AI to absorb whole articles that are fed into their summarizing machines.

This week on Gadget Lab, WIRED senior writer Kate Knibbs joins the show to talk about web crawling and the controversy over Common Crawl. Then we talk with Forbes’ chief content officer and editor Randall Lane about how Perplexity.AI repurposed a Forbes article and presented it as its own story, without first asking permission or properly citing the source.

Show Notes

Read Kate’s story about how publishers are going after Common Crawl over AI training data. Read Randall’s story about how Preplexity.AI copied the work of two Forbes reporters.

Recommendations

Randall recommends his new horse racing league, the National Thoroughbred League. Kate recommends the book Victim by Andrew Boryga. Lauren recommends the show Hacks on Max.

Randall Lane can be found on social media @RandallLane. Kate Knibbs is @Knibbs. Lauren Goode is @LaurenGoode. Michael Calore is @snackfight. Bling the main hotline at @GadgetLab. The show is produced by Boone Ashworth (@booneashworth). Our theme music is by Solar Keys.

How to Listen

You can always listen to this week’s podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here’s how:

If you’re on an iPhone or iPad, open the app called Podcasts, or just tap this link. You can also download an app like Overcast or Pocket Casts, and search for Gadget Lab. If you use Android, you can find us in the Google Podcasts app just by tapping here. We’re on Spotify too. And in case you really need it, here’s the RSS feed.

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
Previous ArticleVivo Y58 5G With 6,000mAh Battery, Snapdragon 4 Gen 2 SoC Launched in India: Price, Availability
Next Article How to fix a red light error on a motherboard

Related Articles

News

The Era of ‘Woke’ Brand Activism Is Over

28 June 2025
News

The Obsessive Fans Playing God on Love Island—and Living for the Crash-Outs

28 June 2025
News

The 28 Best Outdoor Deals From the REI Fourth of July Sale

28 June 2025
News

Persona 5: The Phantom X Brings The Series To Your Phone—and It’s Shockingly Good

27 June 2025
News

OpenAI’s Unreleased AGI Paper Could Complicate Microsoft Negotiations

27 June 2025
News

Substack Is Having a Moment—Again. But Time Is Running Out

27 June 2025
Demo
Top Articles

ChatGPT o1 vs. o1-mini vs. 4o: Which should you use?

15 December 202499 Views

Costco partners with Electric Era to bring back EV charging in the U.S.

28 October 202495 Views

Oppo Reno 14, Reno 14 Pro India Launch Timeline and Colourways Leaked

27 May 202581 Views

Subscribe to Updates

Get the latest tech news and updates directly to your inbox.

Latest News
News

The Obsessive Fans Playing God on Love Island—and Living for the Crash-Outs

News Room28 June 2025
Gaming

How Gearbox Designs “Billions of Guns” in Borderlands 4

News Room28 June 2025
News

The 28 Best Outdoor Deals From the REI Fourth of July Sale

News Room28 June 2025
Most Popular

The Spectacular Burnout of a Solar Panel Salesman

13 January 2025124 Views

ChatGPT o1 vs. o1-mini vs. 4o: Which should you use?

15 December 202499 Views

Costco partners with Electric Era to bring back EV charging in the U.S.

28 October 202495 Views
Our Picks

The Era of ‘Woke’ Brand Activism Is Over

28 June 2025

Super Mario Strikers Is The Next GameCube Title Coming To Nintendo Switch Online + Expansion Pack

28 June 2025

The Obsessive Fans Playing God on Love Island—and Living for the Crash-Outs

28 June 2025

Subscribe to Updates

Get the latest tech news and updates directly to your inbox.

Facebook X (Twitter) Instagram Pinterest
  • Privacy Policy
  • Terms of use
  • Advertise
  • Contact Us
© 2025 Best in Technology. All Rights Reserved.

Type above and press Enter to search. Press Esc to cancel.