Malaysia: Public Consultation on the AI Governance Bill

The PCP provides that the Bill intends to regulate:

1. An "AI system", being the actual operative system, (including stand-alone, embedded, integrated, modular, remote, or service-based), through which AI functionality is manifested in practice.
   It is a system that incorporates, applies, or is enabled by artificial intelligence capability. This can take many forms including a standalone system, a module embedded within another system, an application powered by one or more AI models, a remotely hosted service, an application or service through or integrated with an Application Programming Interface (API), or another digital system capable of generating outputs such as predictions, recommendations, content or decisions.
2. The "AI lifecycle" of the "AI system" (i.e., the overall journey of an AI system, from early development and deployment to market, until the system is withdrawn).

In short, the Bill does appear to cast a significantly broad and deep net of AI products and services.

The Bill will also draw a distinction between two categories of regulated parties:

1. Developers: A "Developer" is any person or organisation that materially shapes what the AI system is capable of doing, how it is intended to function, how well it performs, what limits are built into it, or what risks it may create. This includes the party that trains the model, the party that adapts it for a specific use case, the party that integrates it into a wider system, or the party that later modifies it after deployment.
2. Deployers: A "Deployer" is the person or organisation that causes the AI system to operate in the real world or domain of deployment. The Deployer may not have created the system's underlying capability, but it decides whether, where, how and under what conditions that capability is used.

Parties would have varying degrees of accountability over the AI system at different stages of the AI lifecycle based on the degree of control over the AI system. This means that an organisation may be regulated as a Developer, a Deployer, or both, depending on its role.

The PCP proposes that the Bill will regulate the AI lifecycle of AI systems which are:

1. Placed on the market or put into service within Malaysia
2. Designed, developed, or used in Malaysia
3. Used by a Deployer established in Malaysia, regardless of where the system is physically hosted.

This appears intentionally broad and may capture both domestic and foreign organisations whose AI systems touch Malaysia in any meaningful way.

The Bill proposes the establishment of a Central AI Authority acting as an institutional anchor for the AI governance framework in Malaysia. Its role is to oversee and operationalise a clear set of national "baseline" principles and standards, and strengthen the overall ecosystem by addressing areas of uneven capacity between differing sectors.

The Central AI Authority will carry out three core functions:

1. An AI Safety function by maintaining a risk framework, supervising assessments, supporting testing, developing incident reporting mechanisms, and engaging in international technical cooperation.
2. An Investigation and Enforcement function, undertaking or coordinating technical fact-finding when AI incidents occur, determining what happened, identifying the systems and actors involved, and producing findings to support corrective actions.
3. An AI Enablement function, including capacity-building functions for the public sector, by developing guidance, templates, training, and practical support for public authorities, regulators and regulated organisations, and operating or overseeing AI sandboxes.

The Central AI Authority may then work together with existing sectoral leads ("Sectoral Lead") (who in turn, may be delegated specific powers under the Bill) to support implementation of the baseline framework in areas where the Sectoral Lead has sufficient legal authority, technical expertise, and governance capacity (e.g., by translating the national framework into sectoral instruments for a specific sectoral or domain context).

Pertinently, the PCP suggests that the Bill is intending for Developers and Deployers to take an active posture in respect of risks. In short, they must have "due regard" (i.e., they must actively apply the principles in a way that is proportionate to the system's nature, context, and potential impact) to five principles in the development/deployment of AI systems.

The five proposed principles are:

1. Principle 1 — Human Dignity: AI systems must not be used in a manner that removes meaningful human agency or places important decisions entirely beyond human review, and AI governance must uphold human dignity by ensuring that individuals are not reduced to mere data points.
2. Principle 2 — Transparency and Explainability: AI systems with material effects should be developed or deployed in a manner that enables appropriate transparency and explainability, having regard to the context, risk, intended use and affected stakeholders.
3. Principle 3 — Accountability: Responsibility for AI must remain attributable to identifiable persons and must not be displaced onto the AI system itself, supported by a clear and traceable chain of accountability.
4. Principle 4 — Safety and Security: AI systems should be developed and deployed with proactive measures to anticipate, assess, and address risks before they materialise, with the level of precaution proportionate to the foreseeability and severity of potential harm.
5. Principle 5 — Data Governance: Data used in relation to AI systems must be governed and managed responsibly, lawfully and securely, recognising that the quality, integrity, provenance and security of data directly affect the reliability, fairness and trustworthiness of AI systems.

In assessing the risks (i.e., the possibility that an AI system may cause defined harms across its lifecycle), the PCP sets out four categories of harms as a base threshold i.e.,:

1. Death
2. Bodily injury
3. Unlawful deprivation of fundamental liberty anchored to the Federal Constitution
4. Contravention of any written law.

Risk is also further classified across three tiers:

 
The PCP also recognises that risk is not a static label but is evaluated by assessing: likelihood (the probability of a harm pathway occurring); severity and scale (the intensity of the impact and how many individuals are affected); and duration and reversibility (how long the harm lasts and whether it can be undone).

Further, the PCP is proposing that parties with control and responsibility over the AI system (particularly Developers and Deployers) bear the obligation to submit AI incident reports.

An AI incident may be understood as an event, failure, weakness, misuse, unexpected effect or material circumstance arising from an AI system that causes or may cause AI harm. Incidents may not be limited only to events that have already caused harm — near misses can also be considered as they can provide early warning of system weaknesses, inadequate controls or increasing risk.

The Bill is also proposing the introduction of a "sandbox" as a controlled environment to enable the testing of an AI system in a safe production environment. The "AI Sandbox" is envisioned to be a structured, supervised space in which Malaysian Developers, startups and enterprises can experiment with cutting-edge AI applications without being immediately subject to the full weight of regulatory requirements.

This may be particularly beneficial for small and medium-sized enterprises that may otherwise lack the resources to navigate complex regulatory requirements.