# Reporting and statistics - rehearsal script

Draft v1 · Ian Ibbotson, CTO, Alert-Hub.org · CAP training session, The CAP Workshop and training 2026 · 5 October 2026.

19 slides. Prepared delivery: 26:00, including the screen-reading pauses indicated below. Questions: 4:00. Read spoken paragraphs; timing, stage cues and references are not spoken. Pause on each capture so the audience can locate the feature before hearing the interpretation. Figures are observations on 5 October, not permanent service statistics.

## 1. Reporting and statistics

**00:00-00:45 · 0:45.**

In this training session I want to look at the CAP ecosystem as a whole. Until now, we have not assembled the alerts collected by alert-hub.org in a form suited to analysis across publishers. These reporting tools let us examine patterns across the wider emergency alerting problem space.

I'll use actual screens and observations to show what we can learn. Individual feeds provide concrete examples, but the aim is broader insight. We also need to understand the collection's coverage and what each count measures.

## 2. Evidence for ecosystem analysis

**00:45-01:30 · 0:45.**

We will work through four views. First, the mechanics and reliability of collection, using a source page. Second, vocabulary use, using the OET report. Third, keyword exploration, where we can compare words and phrases across alert fields and senders. Finally, monthly reports, which give us a common period reference.

There is a question running through the whole session: what does this count actually measure? It could be collection attempts, code matches, text matches, or records attributed to a feed. Keeping those units separate makes the screens much easier to interpret.

## 3. The feed is the collection boundary

**01:30-03:00 · 1:30. Pause 10 seconds for the source URL and schedule.**

Our ecosystem view depends on which alerts reach the collection and where observations may be missing. Mexico's Servicio Meteorologico Nacional gives us one example of that collection boundary. The diagnostics identify the upstream feed and reflector route, whether collection is enabled, when another attempt is due, and whether backoff is active.

The basic mechanism is to check the feed, discover alert links, retrieve the referenced documents and process their contents. The feed tells the collector what to look for. It does not necessarily describe the publisher's complete historical output.

That is why source context matters before we compare counts. A gap in our archive could mean there was nothing new to collect, or it could mean we could not reach something. A successful feed response is useful evidence, but it does not tell us that every linked alert was available or that every field was suitable for downstream consumers.

We can keep the conversation concrete by starting with this boundary: which feed did we observe, and what happened when we tried to collect it?

Source: https://console.alerting-apps.net/cap-aggregator/sources/mx-smn-es

## 4. Guidance Signals explain the evidence

**03:00-05:00 · 2:00. Pause 15 seconds on the pills, then point to the hover excerpt.**

The Guidance Signals section connects observed characteristics with implementation advice. Each pill identifies a clause, and hovering gives the reason for the signal plus the evidence behind it.

Here the unknown-value signal refers to severity and certainty. The hover text identifies 65 matches on the unknown-severity tag and 172 on unknown-certainty. The pill shows 237 because those evidence counts are added. Some alerts may be in both sets, so we should not call that 237 distinct affected alerts.

The other visible signals concern event text length and geocodes without polygons. They provide a starting point for a conversation with the feed owner about how consumers handle the messages.

These are guidance signals. The linked advice is an implementation guidance document, labelled as a draft, rather than the CAP schema or a national profile. A signal is not automatically a failed conformance test. It asks us to examine a characteristic and its consequences.

The text beneath the signals gives the evidence for the unknown-value flag. The full guidance wording is available in the accompanying source register.

Sources: source page; sources/guidance-hover.txt; https://preparecenter.org/wp-content/sites/default/files/cap-enabled-alerting_0.pdf

## 5. Latest Errors and Health Buckets

**05:00-07:00 · 2:00. Pause 15 seconds for timestamps and bar legend.**

These two sections describe the collection process at different time scales. On the left, Latest Errors says what failed, when it was last seen and how often that summary has occurred. In this example there are DNS resolution and timeout errors.

On the right, Health Buckets shows the most recent 24 hours. The bar height represents attempts. Green and red divide successful from failed attempts. The hours are in UTC.

An older error can remain listed while the recent buckets are healthy. The API evidence for this draft showed 468 successful attempts and no failures in the rolling health summary. That figure will change, but the useful distinction stays the same: the presence of an error summary does not by itself mean there is a current outage.

There is a second distinction. Successful checks do not equal newly published alerts. The collector may repeatedly receive the same feed without discovering a new document. We need publication volume to answer that question.

When somebody asks why an archive count fell, these screens help us investigate the collection side of the explanation. They cannot, on their own, establish what the publisher intended to issue.

Source: Mexico SMN source page; source API aggregate captured 5 October 2026.

## 6. Longer operational context

**07:00-08:00 · 1:00. Pause 10 seconds for the windows and categories.**

Feed Health Telemetry is one additional screen I would include in this introduction. It offers longer windows and separates network errors from semantic issues and advisories.

The benefit is context. A brief network failure, repeated malformed content and a guidance concern are different investigation tasks. A single score can help direct attention, but the categories and timing explain what to examine next.

The nearby reporting and history links let us relate operational evidence to publication volume, then inspect examples. The current reporting month is still in progress. We should not compare it with a completed month as though they cover equal amounts of time.

With that collection context established, we can move to a different question: which vocabulary terms appear in the archive?

Source: Mexico SMN source page, Feed Health Telemetry.

## 7. OET vocabulary evidence

**08:00-10:00 · 2:00. Pause 15 seconds for version, period and evidence categories.**

Shared vocabularies give us a way to describe hazards across the ecosystem. This OET report lets us examine how the term list relates to the alerts we have collected. It uses version ID two in the console. There are 224 terms, and all 224 completed in the selected saved report. There are no missing or failed computations in this snapshot.

Fifty-four terms have explicit event-code value evidence. One hundred and thirty-three have evidence for their preferred-label phrase. Forty-five have both. Eighty-two have neither. The first two categories overlap, so we should not add them as separate groups.

The period control needs an explanation. In this implementation it selects a saved snapshot. Although it says September 2026, the underlying term counts cover the archive as it stood when the snapshot was generated. These are not September-only usage counts.

There is also an important distinction between the two kinds of evidence. Writing an ordinary phrase such as air quality does not establish adoption of its OET code. A code-value match is more specific, but this report says qualified code matching is unsupported. We cannot establish the complete code and vocabulary-scheme pairing from that metric.

Those qualifications do not make the screen unhelpful. They tell us which question each column can answer.

Source: OET report; ControlledTermSalienceArchiveQueryService.countEventCodeValue/countPreferredLabelPhrase; saved aggregate in sources/evidence.json.

## 8. Explicit code matches are concentrated

**10:00-11:30 · 1:30. Pause 10 seconds on the first three rows.**

The strongest rows show a concentration worth discussing. Air quality, OET-004, has 220,956 explicit code matches in the saved snapshot. Thunderstorm has 30,524, and wind shear has 22,995.

Air quality alone accounts for about 77 percent of the sum of all per-term code counts. The first three rows account for about 96 percent. These percentages concern a sum across terms; an alert can carry several codes, so they are not shares of distinct alerts.

The contrast with phrase evidence is also useful. Wind appears very frequently in text, but much less frequently as its OET code. Wind shear shows the opposite relationship. That is a clue about publishing conventions, rather than an explanation we can assume from the totals alone.

A large number of code matches is therefore only part of the adoption question. We also need to know how many senders contribute and how they use the code.

Source: OET saved snapshot, first three term counts; calculation notes in sources/SOURCES.md.

## 9. A constructive vocabulary review

**11:30-12:30 · 1:00. Pause 10 seconds on the least-evidenced view.**

The least-evidenced view gives us another route into term review. Eighty-two terms have no evidence under the two searches in this completed snapshot.

That is not a reason to delete them automatically. Some hazards are rare. Some publishing communities are underrepresented in the collection. Preferred English labels may miss another language or a local expression.

The constructive question is how term development can be informed by observed publishing practice. We can look for common needs, useful synonyms and distinctions that publishers find hard to apply. Frequency is one input alongside expert judgement and the need to represent uncommon hazards.

The practical next step is a review with publishers and vocabulary maintainers, using messages as evidence. The report gives that discussion a firmer starting point.

Source: OET report, least-evidenced order.

## 10. A usage graph needs sender context

**12:30-14:00 · 1:30. Pause 15 seconds on the graph and sender.**

The usage graph link takes us from a vocabulary row into the keyword explorer. This example searches for the quoted code OET-004 across the selected fields.

The live query returned 222,145 matching alerts. In this result, every match was attributed to one sender value: the AirNow address shown beneath the graph. That is a strong reminder that a high message count can reflect one high-volume contribution.

It does not prove that only one publisher uses OET anywhere. It describes this query in this collection. But it changes how we should talk about the headline number: message frequency is not the same thing as breadth of adoption.

The live count also differs from the vocabulary snapshot. The query fields and observation times differ. If we wanted a numerical reconciliation, we would first align those conditions.

The graph is a useful way to ask when code matches appear, then follow the result into sender context and individual examples.

Source: https://console.alerting-apps.net/cap-aggregator/archive-keyword-explorer?q=%22OET-004%22

## 11. Related words and quoted phrases

**14:00-15:30 · 1:30. Pause 15 seconds for the query controls.**

An ecosystem view needs to account for differences in the language publishers use. Here we search for hurricane, cyclone, typhoon and the quoted phrase tropical storm. These expressions are related, but they are not interchangeable in every region or every meteorological context. The tool helps us examine their use without assuming a single mapping in advance.

The controls let us choose the fields and a period. Quotes keep tropical storm together as a phrase. We can search broadly through descriptions, or ask a narrower question about event labels.

I have bounded this comparison to September 2026. The unrestricted query includes a suspect future bucket. Publishers' alternate calendars are an active data-quality concern, and those records need interpretation against the original dates. We should not describe that bucket as a forecast.

For today's terminology comparison, a named month makes the question clearer. It also avoids confusing a changing all-history query with the saved vocabulary snapshot we just examined.

Sources: original storm query; bounded query with dateFrom=2026-09 and dateTo=2026-09; alternate-calendar concern identified by the project maintainer.

## 12. Union counts avoid double counting

**15:30-17:00 · 1:30. Pause 15 seconds to compare the four term counts.**

For September, the four-term query matches 1,353 unique alerts. Tropical storm appears in 1,013; hurricane in 437; cyclone in 231; and typhoon in 183.

If we add the four term counts, we get 1,864. That is larger than the union because a message can contain more than one expression. The union answers how many alerts match any of the terms.

The difference is 511 additional term memberships. We cannot call it 511 overlapping alerts, because one alert could contribute to three or four columns.

This matters whenever we compare synonyms or related language. Separate columns help us understand wording. The union gives us the combined record count. Both are useful, as long as we do not give them the same interpretation.

Across several months, the charts could help investigate changes in wording. We would still need to check whether the contributing feeds changed over the same period.

Source: September bounded keyword query, per-term counts and union.

## 13. Where the words occur

**17:00-18:30 · 1:30. Pause 15 seconds on the Event and Description rows.**

The field prevalence matrix shows where the matches occur. In this query, 1,334 alerts match in Description, while 977 match in Event. Headline has 913 and Instruction has 35. The rows overlap.

Description gives broader coverage, but it can include background explanation. An event label is a more focused statement of the warning's subject. Neither query is universally better; they answer different questions.

The Event Code row is zero for these English words. That does not mean the messages lack event codes. An event code may be a numeric or alphanumeric identifier rather than the word hurricane.

The matrix is particularly useful when designing a mapping or evaluating a search rule. We can see whether a match depends on incidental description text, then follow a cell into the corresponding archive records to inspect what it found.

Source: September query, fieldKeywordMatrix and fieldBreakdown.

## 14. Sender context changes the interpretation

**18:30-20:00 · 1:30. Pause 10 seconds for the top sender.**

The top sender contributes 947 of the 1,353 matching alerts, about 70 percent. That means this result is strongly influenced by one sender's publishing practice.

Sender breakdowns let us ask whether a term is spread across publishers or concentrated in one contribution. The sender-period matrix can help investigate whether that composition changes over time.

The tool also offers tag matrices and sample alerts. Those are useful follow-up views when we need to understand the characteristics of the matches. A sample is a small set of examples, not necessarily a representative sample of the whole result.

For a fair comparison, we would keep the period and selected fields constant, inspect messages, and account for language. Four English expressions do not cover every way these hazards are described. Updates, repeated warnings and different publisher mandates also affect the counts.

We now have a route from a broad language question to concrete messages. The monthly report supplies a shared period reference for taking those questions further.

Source: September query, topSenders and senderPeriodMatrix.

## 15. The September monthly report

**20:00-22:00 · 2:00. Pause 15 seconds for the period and final two rows.**

To investigate change across the ecosystem, we need a common period reference that includes different feeds' contributions. The September report provides that view of our collection. It covers October 2025 through September 2026, has public Markdown and JSON downloads, and was finalised on 3 October.

September contains 152,471 records in the monthly total, compared with 267,062 in August. That is a decrease of about 43 percent in the observed total. It is not a measure of how many hazards occurred worldwide.

The source breakdown gives us a way to investigate. One air-quality feed falls from 45,356 records to 3,885. That accounts arithmetically for about 37 percent of the net decrease. It is an investigation target, not an established explanation. Collection, publisher output and processing could all matter.

The report lists 189 sources, with 152 contributing a non-zero count in September. Those are two different measures of participation.

The value of this screen is a common reference: the period, the creation time and the counts are visible together. Downloading a copy also preserves the particular report used in a comparison.

Source: https://console.alerting-apps.net/cap-aggregator/reporting/monthly/2026-09 ; finalised report JSON.

## 16. The source index connects totals to feeds

**22:00-23:15 · 1:15. Pause 10 seconds for the filtered row.**

Filtering the source index brings us back to Mexico SMN. This is a useful bridge between the overall report and the operational source page we started with.

The row's Total is 782 across the rolling report window. It is not a September-only number. The source's month histogram records 154 in September. The same caution applies to the other window-based columns.

Recent average is calculated from the first non-zero month in the window, including zero months after that. That helps avoid treating the months before a newly observed feed began contributing as though they were all established quiet months.

Selecting View details opens the source's report tables. We can investigate its contribution within the same finalised period rather than switching immediately to a changing live view.

Source: September monthly report, source index and Mexico SMN month histogram.

## 17. Monthly source detail

**23:15-24:45 · 1:30. Pause 15 seconds for the average calculation and publication table.**

The detail screen shows the full-window total, both averages and the calculation. Below that, the publication-frequency table preserves the month-by-month contribution. Further tables describe tags, severity, message types and event types.

This helps explain a total rather than merely display it. We can ask whether the feed started partway through the period, whether there are gaps, and how updates contribute to the message count.

There is another counting detail here. Tags are counted against information elements, so tag totals can exceed the number of alerts. Multilingual blocks and overlapping categories need care when interpreting these breakdowns.

A monthly report is useful for a repeatable comparison, but its creation time still matters. Reports can be regenerated. Retain the downloaded evidence used for an analysis.

The JSON download is also a practical route for further reporting. We can work with bounded aggregates rather than requiring a copy of the database or downloading the entire archive.

Source: September monthly report, Mexico SMN source detail; API response metadata.

## 18. A shared method for CAP ecosystem analysis

**24:45-25:45 · 1:00.**

These four views give us a practical method. We can establish the collection context, compare vocabulary evidence, explore wording in its fields and sender context, and use a monthly report as the shared period reference.

The useful habit is to name the question, period and counting unit, then follow surprising results back to examples. Operational health, code use and hazard incidence are different subjects, even when their charts look similar.

For vocabulary work, I would suggest a discussion with publishers and maintainers informed by these observed messages. For dataset work, calendar interpretation and collection coverage remain active concerns.

The aim is to gain insight across the CAP ecosystem, using our collection to identify shared patterns and differences in publishing practice. We can follow each result back to the messages and collection conditions behind it, and identify where we need better evidence.

## 19. Contact and resources

**25:45-26:00 · 0:15. Questions to 30:00.**

The companion article contains the screenshots, definitions and calculation notes, with the PowerPoint and PDF available to download. My contact details and the CAP community link are here. We have about four minutes for questions.

Sources: https://blog.alert-hub.org/posts/cap-ecosystem/2026-10-05-cap-reporting-statistics/ ; https://t.me/CAP_community