OpenAI admits to using web crawlers to collect data for training its AI models. The company assures that these crawlers strictly adhere to paywall rules and do not collect personally identifiable information. However, public trust in the company remains strained. Data, computing power, and algorithms are considered the core elements of generative AI, with data scarcity being a concern for companies like OpenAI.
The use of web crawlers for data collection has long been controversial, and suspicions have arisen about OpenAI secretly collecting online data from individuals. While OpenAI has implemented measures to limit the crawler's access and entered into agreements to purchase AI training data, public trust in the company continues to wane.