Comment on Sites scramble to block ChatGPT web crawler after instructions emerge

Sites scramble to block ChatGPT web crawler after instructions emerge

Enlarge (credit: Getty Images) Without announcement, OpenAI recently added details about its web crawler, GPTBot, to its online documentation site. GPTBot is the name of the user agent that the company uses to retrieve webpages to train the AI models behind ChatGPT, such as GPT-4. Earlier this week, some sites quickly announced their intention to block GPTBot's access to their content. In the new documentation, OpenAI says that webpages crawled with GPTBot "may potentially be used to improve future models," and that allowing GPTBot to access your site "can help AI models become more accurate and improve their general capabilities and safety." OpenAI claims it has implemented filters ensuring that sources behind paywalls, those collecting personally identifiable information, or any content violating OpenAI's policies will not be accessed by GPTBot.Read 12 remaining paragraphs | Comments

 

Comment On This Story

Welcome to Wopular!

Welcome to Wopular

Wopular is an online newspaper rack, giving you a summary view of the top headlines from the top news sites.

Senh Duong (Founder)
Wopular, MWB, RottenTomatoes

Subscribe to Wopular's RSS Fan Wopular on Facebook Follow Wopular on Twitter Follow Wopular on Google Plus

MoviesWithButter : Our Sister Site

More Business News