Data Privacy and AI Ethics: Living in the Future

Sameer Ahirrao -

Data privacy is one of the primary concerns of the information age. Who has access to your personal information, and what are they doing with it? Data is not managed merely by humans, however--AI has an ever-increasing role in data processing. As the use and capabilities of AI develop, ethical questions around AI and data management become more pressing. Some of the main ethical dilemmas brought up by AI data processing are connected to automated decision-making? Automated decision-making brings many opportunities for increased organizational efficiency, but limited human involvement can lead to misuse or mishandling of personal data. Improper AI management impairs society by harming those that AI discriminates against and creating public mistrust that limits the potential benefits of AI.

As described in the US National Artificial Intelligence Initiative Act of 2020 (NAIIA), AI data systems use data inputs to (1) perceive real and virtual environments, (2) abstract those perceptions into models through automated analysis, and (3) use inference from those models to develop options for action. For example, predictive algorithms on social media observe users' activity, then offer them content (and advertising) that is relevant to their interests.

AI data processing is a revolutionary technique in fields that must analyse large volumes of data, such as finance, healthcare, education, and marketing. Without it, such data would have to be processed with human labor, taking a significant amount of time and money. However, the lack of human involvement is also a major point of concern. The process of how an AI system comes to a decision is often unclear to ordinary users. Users will likely not understand how the process works, how it affects them, or even that they are being analyzed by AI at all.

AI can produced biased or incorrect conclusions. Biases of the AI's creators or the data used to train the AI can be reflected in its output. For example, many facial recognition algorithms have higher error rates when used to identify women and racial minorities, likely due to being trained mostly on images of white men. The AI can also draw mistaken conclusions--finding patterns and drawing conclusions from correlations that are only coincidences without deeper meaning.

Bias can have serious consequences. AI decision-making is used in a number of serious processes, such as resume analysis, insurance approval, and predictions of future criminal activity. AI systems trained on biased input data create biased outcomes, preserving the human biases that objective computer analysis is meant to eliminate.

Ethical Codes and Restrictions on AI

Many governments and organizations are starting understand the limitations and complications of AI data processing, in addition to its benefits. As a result, agencies and organizations are developing AI ethics codes to correct for these flaws and increase transparency and oversight of AI systems.

For example, in 2020 the US Department of Defense adopted its own set of AI ethics principles. The DoD's AI principles are a good example of the concept, having been developed with input from national AI experts in government, academia, and the private sector. In the DoD's ethical code, use of AI must be:

  1. Responsible: DoD personnel will exercise judgment and care over the Department's use and development of AI.
  2. Equitable: The Department will actively attempt to minimize bias in AI capabilities.
  3. Traceable: AI will be developed in such a way that relevant personnel understand the technology, design procedure, and operations of the Department's AI.
  4. Reliable: AI will have specific, well-defined uses, and will be tested on the safety, security, and effectiveness in those uses throughout its life-cycle.
  5. Governable: AI will be designed and made to fulfill their intended functions, while being able to detect and avoid unintended consequences and to deactivate systems that have unintended behavior.

AI use in data management can also benefit from this ethical code. Since AI used in data management is often handling personal data and can make very impactful decisions (like whether or not to approve someone for insurance), it is important to ensure that the AI's conclusions are unbiased and that the people overseeing its processes understand how it works. Knowing how an AI's decision-making works (as opposed to the "black box" of unknown processes) makes it easier to repair errors, correct for bias, or determine how much weight to give the AI's recommendations. While these five principles are an illustrative model of AI ethics, the DoD has struggled to develop internal rules to actually implement them. Without clear rules or detailed internal guidance to clarify implementation, having a set of ethical principles is of limited effectiveness.

Singapore's Model AI Governance Framework

As an example of a complete ethical AI framework, Singapore's Model AI Governance Framework has both a set of broad AI ethical principles as well as detailed implementation guidance. The two primary principles for responsible AI are that (1) decisions made by AI should be explainable, transparent, and fair; and (2) AI solutions should be human-centric, protecting people's safety and interests. The simplified set of general ethics is augment by guidance from Singapore's Personal Data Protection Commission (PDPC). The PDPC Framework sets out four areas that organizations must consider in their AI implementation:

  1. Internal Governance: Organizations using AI need to be structured in a way that allows more effective oversight over their AI operations. For example, personnel responsible for overseeing AI processes should have clear guidance in their roles and duties, as well as implementing AI-specific risk management controls.
  2. Level of Human Involvement: Determine the appropriate level of human involvement based on the severity and probability of harm. If the severity and probability are low (like content recommendation), AI can act without human involvement. If there is a moderate risk (like GPS navigation), the user should have a supervisory role to take over when the AI encounters problems. If there is a high risk and/or severe harm (like medical diagnosis), AI should only provide recommendations and all decision-making authority should rest with the user.
  3. Operations Management: Implementation of the AI process must be reasonably controlled. Organizations must take measures to mitigate bias in both the data and the AI model. The AI's decision-making processes must be explainable, traceable, reliable, and easily able to be audited and assessed for errors.
  4. Stakeholder Interaction and Communication: An organization's AI policies must be available and known to users, in a clear and easy-to-understand way. Users should be allowed to provide feedback on the effectiveness of AI data management, if possible.

Singapore's AI governance framework is accompanied by two volumes of use cases from government, financial, health, and tech organizations. The real-world examples of effectively implementing and benefiting from accountable AI use, as well as the detailed official guidance documents, make Singapore's Model AI Governance Framework an international model for effective AI ethical policy.

The EU AI Act

The EU, home of the flagship data privacy law GDPR, is preparing its own AI framework. Unlike others, however, the EU AI Act will be a fully-enforceable law and not an internal ethical code or guidance from an agency. First unveiled in 2021, the AI Act could become the new standard for AI regulation.

The AI Act is based on the EU's ethics guidelines for trustworthy AI, written by High-Level Expert Group on AI and the European AI Alliance. The bill draft covers a wide array of software and automated tools, and imposes four levels of regulation based on the risk the present to the public. These range from total bans to no regulation at all.

The Commission deems certain AI systems to be an unacceptable risk to the public by their very nature, and would ban these systems outright. There are four banned technologies: social scoring, dark-pattern AI, manipulation, and real-time biometric ID systems. The ban on social scoring means that public authorities cannot use AI to calculate people's trustworthiness based on their behavior. The ban on dark patterns prohibits subliminal techniques to manipulate people's behavior; for example the Act would forbid sounds at inaudible frequencies to force truck drivers to stay awake longer. Manipulation in this context means that AI systems may be used not exploit people's age or disabilities to alter their behavior. Real-time biometric ID--i.e., facial recognition--is limited for law enforcement. It may only be used with judicial approval (1) as part of a specific investigation, (2) to prevent a specific, substantial, and imminent threat to life, or (3) to identify and locate a suspect of a serious crime like terrorism or human trafficking. Private entities may use real-time facial recognition, but will be subject to the restrictions on high-risk AI.

High-risk AI includes AI that could pose a risk to human well-being if misused or poorly implemented. This includes AI used in educational systems (like exam scoring), employment (like automated resume sorting), biometric ID, critical infrastructure like water and power, emergency services, or immigration and law enforcement. AI in one of these categories must comply with five requirements before they can be implemented:

  1. Data Governance: The data used to train and test the AI must be relevant to its purpose, representative of reality, error-free, and complete. Bias and data shortcomings must be accounted for.
  2. Transparency: Developers must disclose certain information about the system--the AI's capabilities, limitations, intended use, and information necessary for maintenance.
  3. Human Oversight: Humans must be a part of the implementation--checking for bias or dysfunction, and shutting the system down if it poses a risk to human safety or rights.
  4. Accuracy, Robustness, and Cybersecurity: High-risk AI systems must be accurate, robust, and secure in proportion to their purpose. AI developers will have to provide accuracy metrics to users and develop plans to ensure system robustness and security.
  5. Traceability and Auditability: AI developers must provide documentation on a very long list of criteria to prove they are in compliance with the above requirements.

Limited-risk AI under the EU AI Act are those that do not pose a major risk to human safety, but can still be abused to affect humans. This category includes deepfakes, AIs designed to converse with humans, and AI-powered emotion-recognition systems. The main issue with these is transparency--how does a user know if they are interacting with an AI designed to emulate a human? The Act would grant EU residents a right to know if they are looking at/hearing a deepfake, talking to a chatbot, or being subject to AI emotion recognition.

The vast majority of AI pose a minimal risk to human well-being, and any type of AI system not specifically mentioned in the Act falls into this category. The AI Act will not regulate these AIs, but it does strongly encourage developing codes of conduct to voluntarily apply good AI ethics to these systems.

Conclusion

AI has revolutionized data processing, allowing for dramatically increased efficiency in analysis and decision-making. However, irresponsible or poorly-managed AI use can lead to worse outcomes. AI decision-making can harm its subjects if the AI makes major decisions with little human oversight. If not corrected for, the human biases AI was meant to eliminate can become baked in to the process. Organizations may attempt to implement AI decision-making into their process without clearly establishing who is responsible for the AI or understands how it analyzes data.

A comprehensive code of AI ethics can ensure that organizations and governments treat their data subjects with dignity while still reaping the benefits of AI data processing. However, the general guidelines of a code of ethics must be followed up with clear rules for implementation in order to have any meaningful impact. Some codes, like Singapore's Model Framework, are already in active use and have seen meaningful results, while others, like the U.S. Department of Defense's code, lack a practical means of application. As more organizations integrate AI into their decision-making--and as it becomes a bigger part of everyday life--AI ethics codes with clear, practical guidance like Singapore's Framework and actual enforcement power like the EU's AI Act will become more important than ever to protect the rights of data subjects.

About Ardent Privacy

Ardent Privacy is an "Enterprise Data Privacy Technology" solutions provider based in the Maryland/DC region of the United States and Pune, India. Ardent harnesses the power of AI to aid companies with data discovery and automated compliance with RBI Security Guidelines, GDPR (EU), CCPA/CPRA (California), and other global regulations by taking a data-driven approach. Ardent Privacy's solution utilizes machine learning and artificial intelligence to identify, inventory, map, minimize, and securely delete data in enterprises to reduce legal and financial liability.

For more information visit https://ardentprivacy.ai/ and for more resources here.

Ardent Privacy articles should not be considered legal advice on data privacy regulations or any other specific facts or circumstances.