August 20, 2024
.

News from NIST: Dioptra, AI Risk Management Framework (AI RMF) Generative AI Profile, and How PII Identification and Redaction can Support Suggested Best Practices

Acting on its obligations flowing from a 2023 Executive Order, the US Department of Commerce’s National Institute of Standards and Technology (NIST) has recently released two new tools to aid companies developing Generative AI models (GenAI) do so responsibly and securely.

Kathrin Gardhouse
Private

Acting on its obligations flowing from a 2023 Executive Order, the US Department of Commerce’s National Institute of Standards and Technology (NIST) has recently released two new tools to aid companies developing Generative AI models (GenAI) do so responsibly and securely.

Dioptra

The first tool is geared towards the GenAI system developers themselves, instead of governance professionals. Citing from the GitHub repository:

Dioptra is a software test platform for assessing the trustworthy characteristics of artificial intelligence (AI). Trustworthy AI is: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair - with harmful bias managed. Dioptra supports the Measure function of the NIST AI Risk Management Framework by providing functionality to assess, analyze, and track identified AI risks.

Dioptra is designed to serve a variety of use cases across different stages of AI model development, evaluation, and deployment. For model testing, it offers comprehensive assessment capabilities throughout the development lifecycle for first-party developers. Second-party users can leverage Dioptra to evaluate AI models during acquisition processes or within controlled lab environments. Third-party auditors and compliance professionals can utilize the platform to conduct thorough assessments as part of their regulatory or quality assurance activities.

In the research domain, Dioptra aids trustworthy AI researchers by providing a robust system for tracking experiments, ensuring reproducibility and facilitating collaboration. For evaluations and challenges, it serves as a common platform, offering standardized resources and environments for participants to compete fairly and effectively.

Lastly, Dioptra supports red-teaming activities by providing a controlled environment where models and resources can be exposed to security experts. This allows for the identification of vulnerabilities and the improvement of model robustness in a safe and managed setting. Overall, Dioptra's versatility makes it a valuable tool for a wide range of stakeholders in the AI ecosystem, from developers and researchers to auditors and security professionals.

Dioptra is designed with several key properties that enhance its functionality and user experience. At its core, Dioptra emphasizes reproducibility by automatically creating snapshots of resources, ensuring that experiments can be accurately reproduced and validated. This is complemented by its traceability feature, which maintains a comprehensive history of experiments and their inputs, allowing for detailed analysis and auditing.

The platform's extensibility is achieved through a plugin system that supports the expansion of functionality and seamless integration of existing Python packages. Interoperability between these plugins is facilitated by a robust type system, promoting smooth interaction between different components.Dioptra's modular architecture allows users to compose new experiments from pre-existing components using simple YAML files, enhancing flexibility and ease of use. Security is prioritized with user authentication, and access controls are in development to further strengthen data protection.

Users benefit from an intuitive web interface that provides interactive access to Dioptra's features. Furthermore, the platform is designed for shareability and reusability, supporting multi-tenant deployment. This enables users to share and reuse components efficiently, fostering collaboration and knowledge exchange within the AI research and development community.

AI RMF Generative AI Profile

The second tool is the AI RMF Generative AI Profile, an expansion on the AI Risk Management Framework NIST published in January 2023 that addresses GenAI risks and mitigation strategies. The AI Profile lists 12 risk categories and almost 200 recommended actions that should be taken to mitigate these risks. These actions focus on governance mechanisms like establishing and implementing policies, oversight and incident reporting mechanisms, and engaging diversely composed teams and representative populations throughout the AI system lifecycle.

In this table we are listing the Suggested Actions that Private AI’s solutions can support, including a brief explanation of Private AI’s relevant capabilities.

The Redaction Capability can remove direct identifiers such as names, emails, numerical identifiers like SSN and driver’s license number, as well as indirect identifiers, e.g., data of birth, age, or physical attribute, thereby enhancing the privacy of the dataset and minimizing risk for individuals. 

MEASURE 2.6: The AI system is evaluated regularly for safety risks – as identified in the MAP function. The AI system to be deployed is demonstrated to be safe, its residual negative risk does not exceed the risk tolerance, and it can fail safely, particularly if made to operate beyond its knowledge limits. Safety metrics reflect system reliability and robustness, real-time monitoring, and response times for AI system failures.

MS-2.6-002

Assess existence or levels of harmful bias, intellectual property infringement, data privacy violations, obscenity, extremism, violence, or CBRN information in system training data.

The Detection and Reporting Capability can support the bias assessment requirement by identifying entities in the data with which bias is often associated, such as gender, sexual orientation, age, nationality, race, disability, and income. The identification of PII can aid the inquiry into whether training data infringes upon privacy obligations.

MEASURE 2.7: AI system security and resilience – as identified in the MAP function – are evaluated and documented.

MS-2.7-001

Apply established security measures to: Assess likelihood and magnitude of vulnerabilities and threats such as backdoors, compromised dependencies, data breaches, eavesdropping, man-in-the-middle attacks, reverse engineering, autonomous agents, model theft or exposure of model weights, AI inference, bypass, extraction, and other baseline security concerns

Pseudonymization is a well-established security technique that helps minimize the impact of data breaches, and which can be achieved with support of the Redaction Capability.

MEASURE 2.10: Privacy risk of the AI system – as identified in the MAP function – is examined and documented.

MS-2.10-001

Conduct AI red-teaming to assess issues such as: Outputting of training data samples, and subsequent reverse engineering, model extraction, and

membership inference risks; Revealing biometric, confidential, copyrighted,

licensed, patented, personal, proprietary, sensitive, or trade-marked information; Tracking or revealing location information of users or members of training datasets.

Red-teaming efforts can be supported using the Detection and Reporting Capability which can give quick and reliable insights into whether output contains PII or confidential information.

MANAGE 2.2: Mechanisms are in place and applied to sustain the value of deployed AI systems.

MG-2.2-009

Consider opportunities to responsibly use synthetic data and other privacy enhancing techniques in GAI development, where appropriate and applicable, match the statistical properties of real-world data without disclosing personally identifiable information or contributing to homogenization.

The Redaction Capability can replace PII with synthetic PII to retain data utility while preserving privacy. 

MANAGE 3.1: AI risks and benefits from third-party resources are regularly monitored, and risk controls are applied and documented.

MG-3.1-002

Test GAI system value chain risks (e.g., data poisoning, malware, other software and hardware vulnerabilities; labor practices; data privacy and localization compliance; geopolitical alignment).

When procuring any data from a third party, it is advisable to use the Detection and Reporting Capability to identify whether any PII or sensitive information is contained in the data, which helps with the risk assessment of these third-party resources.

MANAGE 4.3: Incidents and errors are communicated to relevant AI Actors, including affected communities. Processes for tracking, responding to, and recovering from incidents and errors are followed and documented.

MG-4.3-003

Report GAI incidents in compliance with legal and regulatory requirements (e.g., HIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022) autonomous vehicle crash reporting requirements.

GAI incident reporting can be aided by the Detection and Reporting Capability insofar as the incident involved a breach of PII. This Capability can provide accurate information on whose and what type of data was affected.

Conclusion

The release of Dioptra and the AI RMF Generative AI Profile by NIST marks a significant step forward in promoting responsible and secure development of Generative AI systems. These tools provide developers, researchers, and compliance professionals with valuable resources to assess, manage, and mitigate risks associated with AI technologies. As the field of AI continues to evolve rapidly, the importance of such frameworks and platforms cannot be overstated. By leveraging these tools alongside privacy-enhancing technologies like those offered by Private AI, organizations can better navigate the complex landscape of AI development and deployment.

Related Articles

No items found.