Source URL: https://www.wired.com/story/nist-humane-intelligence-generative-ai-red-team-contest/
Source: Wired
Title: The US Government Wants You—Yes, You—to Hunt Down Generative AI Flaws
Feedly Summary: AI ethics nonprofit Humane Intelligence and the US National Institute of Standards and Technology are launching a series of contests to get more people probing for problems in generative AI systems.
AI Summary and Description: Yes
**Summary:** The text discusses a significant red-teaming initiative at the 2023 Defcon conference, where AI tech companies and organizations are collaborating to evaluate the security and ethical implications of generative AI systems. This initiative aims to democratize the evaluation process, inviting a wide range of participants, including everyday users and experts, to identify potential weaknesses and biases in AI models, ultimately enhancing accountability and transparency in AI technology.
**Detailed Description:**
The text highlights an important movement within the AI community toward greater security and ethical considerations. The initiative, supported by the US government and organized by Humane Intelligence, is part of a larger effort to assess the integrity of generative AI technologies through collaborative evaluation. Key points include:
– **Red-Teaming Initiative:**
– At Defcon 2023, the collaboration emphasized the need to scrutinize generative AI models.
– Inviting both developers and the general public to participate democratizes the evaluation process.
– **NIST Involvement:**
– The National Institute of Standards and Technology (NIST) is facilitating a nationwide challenge known as ARIA (Assessing Risks and Impacts of AI).
– Participants who qualify will engage in both red-teaming (attacking the AI systems) and blue-teaming (defending against attacks) at the Conference on Applied Machine Learning in Information Security.
– **Objectives and Benefits:**
– The initiative aims to create rigorous testing frameworks for assessing the security, resilience, and ethical standards of AI systems.
– As per Theo Skeadas, the goal is to enable users to evaluate if AI models meet their specific needs.
– **Development of Evaluation Frameworks:**
– The event will use NIST’s AI risk management framework (AI 600-1) for measuring outcomes and behaviors of AI systems in response to attacks.
– It emphasizes the need for structured user feedback to understand real-world AI applications.
– **Future Collaborations:**
– Humane Intelligence plans to announce more collaborations with various organizations to promote transparency in AI algorithms.
– Concepts like “bias bounty challenges” are introduced to incentivize individuals to identify and report issues within AI models.
– **Inclusivity and Diversity in Participation:**
– The initiative recognizes the importance of including a diverse range of voices, particularly from underrepresented communities, in the evaluation and testing of AI systems.
This ongoing effort signifies a robust step towards ensuring AI technologies are not just secure but also fair and accountable, resonating deeply with professionals in AI security, compliance, and ethical governance.