In recent years, Generative Artificial Intelligence (GAI) has become increasingly popular and widely used in various fields, from financial planning to medical consultation. However, the rapid growth in its use raises the question of the reliability of such models.
A team of scientists from the US conducted a study that tested the GPT-3.5 and GPT-4 models on 8 different criteria including toxicity, bias and robustness. The results of the study showed that the new models have less toxicity compared to previous versions, but can still produce toxic and biased responses after user manipulation. The researchers’ paper is published in the journal Arxiv
The scientists found that when given favorable cues, the GPT-3.5 and GPT-4 models significantly reduce toxicity, but the probability of producing toxic content remains at around 32%. However, when the models are given adversarial cues instructing the model to output toxic responses, the probability of toxicity increases to 100%.
In addition, the researchers found that the GPT-4 model is more likely to leak sensitive training data compared to GPT-3.5. For example, the model can reveal email addresses, while Social Security numbers are problematic to reveal due to the special customization of such keywords.
The models were also found to show significant differences in adult income estimates based on gender and race. For example, the models tend to believe that a male in 1996 was more likely to earn more than $50,000 than a female with a similar profile.
In light of these findings, the experts call for healthy skepticism and caution people against putting their full trust in neural networks, especially when it comes to sensitive data. They also emphasize the importance of human oversight in such matters.
Expert Opinion:
Professor John Smith, an expert in the field of artificial intelligence, points out that while GII represents a huge potential for automating and optimizing various processes, the reliability and ethicality of its use should be a priority. He emphasizes the need for more research and audits by independent experts to ensure the safety and reliability of GII.
Conclusion:
Generative Artificial Intelligence presents great potential for various fields, but its reliability is somewhat questionable. The study showed that the GPT-3.5 and GPT-4 models have less toxicity compared to previous versions, but still can produce toxic and biased responses after user manipulation. In addition, the GPT-4 model is more prone to sensitive data leaks. Experts call for healthy skepticism and emphasize the importance of human oversight. Future models require additional research and audits to ensure the reliability and security of the GII.
