Smart board level questions to ask about AI.
Artificial intelligence, or “AI,” raises legal and ethical issues beyond those generally found in investments in technology. Due to the rapid growth in this area, the lack of standards for evaluation and oversight and the risks associated with AI use, AI projects would particularly benefit from board inquiry and oversight.
Board members should ask the following questions as their company evaluates its use of AI.
Will AI be replacing human judgment?
As board members well know, our legal system relies fundamentally on human judgment in the areas of greatest importance. No board would simply turn over the question of whether a buyout offer is in the best interests of shareholders to an AI system, for example. Each board needs to inquire about whether sufficient consideration has been given to the potential uses of AI, particularly for businesses where legal compliance, fairness and adapting to new situations are important.
AI-based decisions must satisfy the laws and regulations that apply to your business. Of particular concern that AI-based decisions may discriminate because they rely on data that reflects a discriminatory past or looks only at correlation instead of causal factors. Companies that use AI tools in hiring, for example, need to ensure that these tools do not discriminate against certain protected classes of applicants or employees. In regulated areas like insurance, AI tools used for underwriting decisions will have to follow recently-issued requirements from the New York Department of Financial Services on the use of “unconventional sources or types of external data” to address the risk of unlawful discrimination and a lack of data transparency.
Companies can mitigate these AI risks by utilizing oversight, risk management and controls to meet legal compliance and ethical objectives. Data scientists who understand the AI tools and the context of the data and who implement controls designed to eliminate bias, inaccuracies and coincidence can reduce the chance of these unintended consequences.
In addition, AI systems will need to produce output that is transparent, auditable and that can be explained — sometimes called “Explainable AI.” For the AI hiring tool example above, a company will need to be able to demonstrate that favorable hiring qualification scores of applicants are based on legitimate criteria, and not, on machine-determined prohibited factors such as race or gender identification.
What are the concerns around the data used in AI?
Data is the fuel for AI. AI systems rely on statistical analysis and deliver the best results with large volumes of accurate, well-coded data. Companies using “machine learning” systems need a “data supply chain” to deliver a continued flow of current, accurate data.
Data use must comply with the privacy, data security, export control and other laws that apply to the data. For example, Europe now has tough data protection laws that prohibit the use of individual data for automated processing to evaluate any behavior, preferences or location absent the explicit consent of the individual, and yet, automated processing of individual data to determine preferences is the hallmark of many AI tools. In addition, the data use must comply with any contractual requirements to data suppliers.These are often not well understood. To guard against these data pitfalls, board members should inquire as to the level of legal and regulatory diligence that has been done on the uses of data to fuel AI systems.
How will the company protect what it builds?
Patent, copyright, trade secret and other intellectual property (IP) laws were written to protect human creativity. IP laws in the United States do not square nicely with AI. Not only may your company not own AI that you pay to create, there may be no way to fully protect it under our IP laws.
Contractual and trade-secret protections are key elements of capturing and preserving value in the an investment in AI. These protections, to be effective, must be implemented before the AI effort begins.
How will AI be implemented from a contractual, marketing and operational perspective?
IP protection may not be the only area where AI changes your business model. There may be effects on (and objections from) contracting parties, customers and employees. Recognize that your internal and external stakeholders have great (and possibly unrealistic) hopes for the benefits and, perhaps, also have considerable fears.
AI should be a cross-functional effort, including review and oversight by people focused on risk and potential harm. As a board member, you should inquire about the types of controls that are in place to avoid damage to relationships, brand, employees and communities.
How will evolving laws affect the AI initiative?
There is an evolving understanding of how legal concepts such as reasonable care and agency will be applied to traditionally human processes now implemented by AI. There are also new laws related to AI, including “automated profiling,” some of which carry substantial potential penalties. This analysis requires sophistication both in computing technologies and in the applicable laws generally, and you should probe for whether this level of analysis has been done.
How does this fit with general risk management?
The AI risk management framework should fit into the company's broader risk management framework and include standards for building, using and validating that AI models do not contain the problems discussed above. Company policies should require that new uses of AI undergo risk management review, and ultimately board review where appropriate. While it is vital to involve technical and security functions, we recommend that the board actively oversee whether the level of risk is appropriate for the company and whether the interests of internal and external stakeholders have been properly considered.
Rebecca Eisner and Brad Peterson are partners in the Technology Transactions practice of Mayer Brown LLP. They regularly advise companies on complex digital transformation, data, software and managed services matters, and are recognized by Chambers, Legal 500 and others as top lawyers in their fields.