Can AI model capabilities be stolen?

TL;DR

The dispute around Claude’s capabilities shows that AI protection is no longer only about securing code, data, and infrastructure. Companies also need to protect model behavior. If someone systematically queries a model through an API, they may try to reproduce its reasoning patterns, decision logic, and specialized capabilities. For businesses, this creates a new risk: an advantage built on data, processes, and expert knowledge can leak through a legitimate access channel if usage rules, monitoring, and contractual controls are weak.

---

Introduction

The dispute between Anthropic and Alibaba points to a problem that will appear more often: the value of an AI model does not sit only in its code, training data, or compute infrastructure. Increasingly, the real value lies in the model’s capabilities themselves — the way it responds, reasons, solves tasks, writes code, analyzes data, plans actions, and handles complex business scenarios.

At the center of the issue are allegations of large-scale, unauthorized use of access to Claude in order to extract its capabilities. The alleged mechanism involved creating multiple accounts, generating a very high volume of prompts, and using the model’s outputs to train or improve another system. In practice, this relates to a technique known as model distillation, where one model is trained based on the behavior of another.

Model distillation is not inherently illegal or suspicious. In many companies, it is a normal optimization technique: a larger, more expensive model helps train a smaller, cheaper, and faster system. The problem starts when this happens without the model owner’s permission, outside the service terms, at scale, and with the intention of copying a technological advantage.

For business leaders, this is not just a story about a conflict between two technology companies. It is a signal that AI is becoming an asset that must be protected differently from traditional software. If a company deploys its own models, builds AI agents, creates specialized decision systems, or exposes model capabilities through an API, it needs to think about a new category of risk: not only data theft, but also the unauthorized replication of system capabilities.

Example

Nadia works as operations director at a company that has spent the last two years building an internal AI assistant for customer service. The system is not a simple chatbot. It understands the structure of offers, analyzes complaints, suggests decisions aligned with company policy, and identifies situations that should be escalated to a human.

The company does not sell the model as a standalone product. It gives selected partners access through a portal and API to speed up service for shared customers. After a few months, however, the technical team notices an unusual pattern: several partner accounts are sending thousands of similar prompts every day. The questions are slightly modified, but they concern the same scenarios: complaints, deviations from procedure, pricing exceptions, and refund decisions.

At first, this looks like intensive testing. Only later do the analysts realize that the prompts form a systematic grid of cases. Someone is checking how the model behaves across hundreds of variants of the same situation. They are not asking in order to serve a customer. They are asking in order to map the model’s operating logic.

A few weeks later, a competing service appears on the market and handles complaints in a very similar way. There is no proof that code was copied. There is no leaked database either. Yet Nadia’s company loses something real: an advantage built through thousands of hours of expert work, historical data, testing, and process refinement.

This scenario shows why disputes over AI model capabilities matter. In the traditional software world, it was easier to point to what had been stolen: a file, a database, a piece of code, technical documentation. In the AI world, value can be reconstructed by observing how a system behaves. This makes the boundary between normal use, testing, reverse engineering, and unauthorized extraction much more complicated.

---

What the problem is

In the traditional view of technology ownership, a company mainly protects source code, system architecture, documentation, customer data, and infrastructure secrets. With advanced AI models, that picture is incomplete. A model may be technically protected, while still exposing its value through its outputs.

Every prompt sent to a model is a small experiment. A user submits a problem, receives an answer, and can evaluate how the system reacts. With a small number of prompts, this is a normal interaction. With millions of prompts, it can become a way to reconstruct patterns: how the model reasons, where it is precise, how it writes code, how it classifies risk, and what decisions it prefers in ambiguous situations.

The key distinction is between access to a service and the right to copy its capabilities. The fact that a user can interact with a model through an interface or API does not automatically mean they can use its outputs as training material for a competing system. For many companies, this distinction will become critical, especially as they start exposing their own models to customers, partners, or external integrators.

The problem can be divided into several layers that often appear together in practice:

• The technical layer concerns the way model outputs are collected. If someone uses many accounts, automates prompts, and hides the origin of traffic, this no longer looks like normal use of a service. The company must be able to distinguish real demand from systematic collection of model behavior.

• The contractual layer concerns the terms of service. Many organizations still lack precise clauses that clearly separate permitted operational use from prohibited training of competing models. Without such clauses, enforcement becomes harder.

• The business layer concerns loss of advantage. If a model is the result of investment in data, experts, processes, and testing, its capabilities are an asset. Unauthorized replication can lower the barrier to entry for competitors.

• The reputational layer concerns trust among customers and partners. If a company cannot control who uses its models and how, the market may start questioning its ability to protect technology, data, and processes.

The most difficult part is that there may be no traditional breach. The system may work exactly as designed: accepting prompts and returning answers. The risk arises because a normal access channel can be used at abnormal scale and for a different purpose than declared.

---

Why it matters for companies

For companies outside Big Tech, this topic may seem distant. In reality, it is highly practical. More organizations are building their own AI layers on top of commercial models, open-source models, or systems trained on industry-specific data. The value of these systems often does not come from being the largest. It comes from being adapted to a specific business process.

In manufacturing, this could be a model supporting maintenance planning. In banking, a system analyzing transaction risk. In logistics, an agent optimizing operational decisions. In customer service, an assistant that understands policies, exceptions, and dependencies between products. Each of these systems may become a target not because it contains a spectacular algorithm, but because it contains practical operational knowledge.

The impact on the organization should be assessed across several areas, because the consequences do not stop at the IT department:

• Operations and processes lose uniqueness if competitors can replicate the way decisions are made. An AI model often encodes the company’s best procedures in practice. If someone reproduces its behavior, they can move faster toward a similar level of service, planning, or analysis quality.

• Finance feels the pressure when the time needed to catch up with the leader becomes shorter. A company that spent years and significant resources developing a model may see a competitor achieve a comparable effect at a lower cost. This changes the economics of AI investment, because part of the return on innovation can be captured by other players.

• Product teams lose control over what actually constitutes advantage. If the advantage comes from model behavior rather than a visible product feature, it is harder to protect, measure, and communicate. A product may be copied not through its interface, but through systematic probing of its outputs.

• Legal teams need to keep up with technology that does not fit old categories. Traditional license agreements, API terms, and confidentiality clauses may not be enough if they do not clearly define whether model outputs may be used to train other systems.

• Management must treat AI as a strategic asset, not only as a productivity tool. If a model affects margin, service quality, sales, or risk decisions, its protection becomes part of managing company value.

This shifts the AI conversation from “how do we deploy the tool?” to “how do we protect the organization’s ability to operate better than competitors?” That is why disputes over model distillation will matter even for companies that do not build frontier models themselves.

---

A new dimension of intellectual property

Intellectual property in AI is difficult because a model is not a simple equivalent of a software program. Code can be protected, data can be covered by agreements, and a brand can be registered. But model behavior is more fluid. It is a combination of architecture, data, fine-tuning, system instructions, safeguards, testing, and feedback from users.

In practice, a company may not know which exact element creates its advantage. Sometimes it is not the base model itself, but the deployment decisions around it: how the system prompt was designed, which examples were shown to the model, how response validation works, when the system escalates to a human, what data enters the context, and how quality is measured.

This is why AI protection must cover not only “hard” assets, but also patterns of system behavior. That requires a different way of thinking about product boundaries. If a company exposes an API, it should assume that every user may try not only to use the service, but also to study how it behaves.

This new dimension of intellectual property has several practical consequences:

• Terms of service must be more precise than before. A generic clause prohibiting abuse may not be enough. It is worth clearly defining whether model outputs can be used for training, fine-tuning, benchmarking, or validating other models.

• Partner agreements should separate operational use from development use. A partner may need API access to serve customers, but that does not necessarily mean they should have the right to generate training data at scale. These two use cases require separate rules.

• Monitoring should detect intent, not only volume. The number of prompts alone is not always enough. Important signals include repetition, systematic prompt variation, unusual activity hours, many accounts with similar behavior, and prompt sequences that resemble capability mapping.

• Protection must be designed at the product stage, not only after an incident. If a company starts thinking about limits, logs, and contractual clauses only after detecting abuse, it is usually too late to fully control the situation.

In the AI world, intellectual property is no longer only a document managed by the legal department. It becomes a set of product, technical, contractual, and operational decisions.

---

Operational and technology risks

The biggest risk is that organizations will treat AI models like ordinary SaaS applications. In a traditional application, an API exposes functions: retrieve a record, add an order, check a status, generate a report. In an AI model, an API exposes not only a function, but also part of the system’s cognitive capability. A user can ask questions, test boundaries, and collect outputs in a way that reveals more and more over time.

This changes the approach to security. It is not enough to secure logins, API keys, and infrastructure. Companies must monitor whether usage patterns match the intended business purpose. A model can be abused without any technical bug in the system.

The most important risk areas include:

• Abuse of accounts and user identities. If one organization or group creates many accounts, it can bypass limits and distribute traffic so that it looks natural. This means that limiting prompts per account may not be sufficient.

• Mass generation of training data. Prompts can be designed to obtain input-output examples. Such a dataset can later be used to train another model, even if the attacker never sees the original model’s code or weights.

• Mapping specialized model capabilities. The most valuable outputs are often in areas where the model has been carefully refined: cybersecurity, legal analysis, coding, healthcare, finance, complaint handling, or credit decisions. The advantage there does not come from general fluency, but from the quality of judgments.

• Bypassing safeguards through prompt variants. If someone systematically changes the wording of prompts, they can test when the model reveals more than it should. This applies both to operational information and to the behavior of guardrails.

• Contamination of product metrics. Mass traffic generated for extraction may look like usage growth. If a company does not distinguish traffic quality, it may misread product adoption, infrastructure costs, and customer needs.

In practice, the AI team should work closely with security, legal, product, and finance. A siloed approach is not enough, because the risk is technical, contractual, and economic at the same time.

---

What organizations should do

Companies that build or expose AI systems should treat this category of risk as part of their operating architecture. This is not about panic or closing every model off from the world. It is about consciously deciding who can use the model, for what purpose, at what scale, and under what conditions.

The first step is to identify where AI creates real advantage in the organization. Not every chatbot requires the same level of protection. A simple FAQ assistant should be treated differently from a system that contains pricing decision logic, exception handling, expert knowledge, or automation of a process with direct revenue impact.

A practical approach should include several actions:

• Define which model capabilities are strategic. The company should know whether it is protecting general functionality, industry knowledge, decision-making logic, data integration, or a unique operating process. Without this definition, it is difficult to choose the right safeguards.

• Introduce clear rules for using model outputs. Terms of service, API agreements, and partner contracts should specify whether users may use outputs to train, fine-tune, or evaluate other AI systems. Without this clause, an unnecessary grey area appears.

• Monitor patterns of use, not only technical limits. The team should analyze repetitive prompts, sequences that test model boundaries, similarities between accounts, and unusual output generation rates. This makes it easier to detect systematic extraction attempts.

• Segment access to models and functions. Not every user should have access to the full capability set. Partners, pilot customers, internal users, and integration teams may require different access levels, limits, and usage rules.

• Build an abuse response procedure. The organization should know when it blocks an account, when it restricts access, when it contacts a partner, when it launches legal analysis, and when it treats the event as a security incident. Without such a procedure, the response will be delayed and chaotic.

• Review infrastructure costs and anomalies. Large-scale extraction of model capabilities generates costs for the provider. Financial monitoring may be one of the first signals that the product is being used differently from intended.

• Test the model for susceptibility to behavior replication. It is worth checking which outputs are most valuable, which scenarios reveal the most logic, and whether the model produces overly stable, easy-to-collect patterns.

The most important point is not to treat AI protection as a one-off project. Models, users, and abuse methods change from month to month. AI security should therefore become an operating process, just like vulnerability management, access control, or fraud monitoring.

---

Summary

The Anthropic and Alibaba dispute shows that the AI market is entering a phase in which technological advantage can be attacked not only through theft of code, data, or talent. It can also be replicated through large-scale observation of model behavior. This is a harder problem, because the boundary between using a service and copying its capabilities is not always obvious at first glance.

For companies, the key conclusion is simple: an AI model exposed through an API is not just a technical function. It is an access channel to an asset that may contain operational knowledge, decision logic, expert experience, and process advantage. If that channel is not monitored, described contractually, and managed as a product, it can become a source of value leakage.

Not every organization needs safeguards at the level of the largest AI labs. Every organization should, however, answer a few questions: which capabilities of our AI systems are truly valuable, who has access to them, whether outputs may be used to train other models, how we detect unusual usage patterns, and what we do when someone tries to systematically map model behavior.

Intellectual property in AI does not end with files, repositories, and training data. Increasingly, it includes the system’s ability to operate in a specific business context. Companies that understand this earlier will be better prepared not only for legal disputes, but above all for the real protection of their operational advantage.

---