KAPOW

KAPOW (Knowledge on Automata Platformized and Operationalized as a Warehouse) is a bot detection system in the Data Platform.

What is bot detection and why do we need it?

A large (and increasing) fraction of Wikipedia traffic is automated. A 2025 Diff post describes global trends that account for this, and ways in which the Wikimedia Foundation is responding to them.

Bot detection is the process of classifying traffic to our sites, such that automated traffic is separated from direct human traffic to the best of our ability, and categorized to inform future decisions.

This includes clients that identify themselves as automated, for example via the User-Agent header, and also clients that go through great lengths to appear to be human and avoid detection.

A rich bot detection system provides input to many critical domains, broadly under:

• Analytics, so we can make better product decisions, plan fundraising campaigns and understand usage of our projects.
• Infrastructure defense, so we can improve our ability to block excessive traffic that could make our projects less reliable.

What is KAPOW?

KAPOW is the collective name for:

• A bot detection model incorporating various signals for detecting automated agents such as web scrapers.
• Automation (data pipelines) that apply this model to incoming traffic, generating tables for further analysis and integration with downstream systems.

We intend to centralize bot detection logic using KAPOW so that it can be used across a variety of use cases, including public pageview counts on Wikistats and for blocking excessive or harmful traffic to our servers.

KAPOW visibility

Given its use in infrastructure protection and adversarial nature of some bots, we consider the details of KAPOW to be private, including its code, detailed documentation and output data.

Phabricator tickets are also restricted, although we strive to keep them public where possible. For example, high level milestones for integrating KAPOW into other systems will generally be public.

How we improve KAPOW

KAPOW is designed to be easy to extend with new signals, and adjust to unexpected sources of automated traffic. We learn a great deal about bots when we don’t detect them automatically, and this leads to improvement to KAPOW.

We have a response protocol for when automated traffic slips by KAPOW, which ends with a report published to Data Platform/Data Lake/Data_Issues including the estimated impact in terms of pageviews, and an annotation to Wikistats.

While the public reports are relatively sparse, each helps us identify and prioritize work in KAPOW to make it more effective next time a similar kind of bot appears.

See also

• For the old documentation, see Obsolete:BotDetection
• phab:T425661 - [Epic] KAPOW: The next generation of bot detection in the Data Platform.