Metadata requests no longer tracked in PyPI download counts

Metadata requests no longer tracked in PyPI download counts
Metadata requests no longer tracked in PyPI download counts

On 2026-08-24 I shipped a change to the configuration that emits PyPI’s download logs so that only requests for actual distribution artifacts are counted. A request has to end in .whl, .tar.gz, or .zip to produce a download record.

The counts are more accurate now, and existing systems based on PyPI’s BigQuery dataset, like pypistats.org will display a different shape going forward.

If you maintain something that reads these numbers from the public dataset, the discontinuity at 2026-08-24 is expected and permanent.

This effort would not be possible without the continued support from Alpha-Omega.

Background

The logs are generated by Fastly’s Edge, shipped to an AWS S3 bucket, parsed and anonymized by our linehaul functions in Google Cloud, and published to a Google-managed public BigQuery dataset.

Other objects

Everything PyPI serves for a release lives under the same /packages/<xx>/<yy>/<hash>/ prefix. Matching on that prefix alone counted a fetch of any part of a release as a download of the distribution itself.

  • PEP 658 .metadata sidecars. (estimated at 40%) Installers may fetch these to read a wheel’s metadata without downloading the wheel. Every metadata fetch was being logged as a distribution download.
  • .asc GPG signatures. PyPI stopped accepting these in 2023, but the ones uploaded before then are still served, and no longer counted.
  • .egg uploads were separately deprecated in 2023. All of these remain downloadable, and none of them are counted.
  • Formats frozen for upload since 2016 under PEP 527, such as .exe, .msi, and .rpm, are about 0.2% of the files on PyPI, and are no longer counted.

The correction

Daily downloads across all of PyPI, 60 days Daily download quantity across all of PyPI, 60-day window ending 2026-08-31, from pypistats.org.

Before the change, daily totals cycled between roughly 4.5 billion on weekends and a little over 7 billion midweek. The curve steps down after 2026-08-24. The last few days of that window were still settling when the chart was captured, so read the shape of the change rather than the final data point.

Impact on existing data

If you compare download counts across 2026-08-24, the numbers will not line up. The counts after that date are lower and more accurate. Nothing was lost from the historical record – the older rows in BigQuery are unchanged, they were just measuring something broader than “someone downloaded this package” – they measured “someone downloaded a given file from PyPI”. Most consumers of this dataset today do not filter specific to filename extensions, leading to inflated download counts for a given package.

However, if you wanted to query the dataset yourself, you can write your own SQL queries to be able to distinguish a single project’s distribution file downloads from others with a BigQuery statement that looks a bit like this:

SELECT  DATE(timestamp) AS download_date,  COUNT(*) AS all_objects,  COUNTIF(REGEXP_CONTAINS(file.filename, r'.(whl|tar.gz|zip)$')) AS distributions_only,  COUNT(*) - COUNTIF(REGEXP_CONTAINS(file.filename, r'.(whl|tar.gz|zip)$')) AS others,  ROUND(100 * (COUNT(*) - COUNTIF(REGEXP_CONTAINS(file.filename, r'.(whl|tar.gz|zip)$'))) / COUNT(*), 1) AS others_pct FROM `bigquery-public-data.pypi.file_downloads` WHERE project = 'urllib3'  AND timestamp >= TIMESTAMP('2026-08-18')  AND timestamp < TIMESTAMP('2026-08-26') GROUP BY download_date ORDER BY download_date 

Results:

download_date all_objects distributions_only others others_pct
2026-08-18 77477790 47268787 30209003 39.0
2026-08-19 74742551 45808710 28933841 38.7
2026-08-20 73495894 45011207 28484687 38.8
2026-08-21 69652309 42467783 27184526 39.0
2026-08-22 49665300 30080582 19584718 39.4
2026-08-23 50965479 30344490 20620989 40.5
2026-08-24 64241532 44012651 20228881 31.5
2026-08-25 46619401 46619401 0 0.0

For this particular query and time range, ~39% of the download counts are not the actual distribution files.

Downloads are not a popularity metric

Download counts are tricky to get right, and should not be used as a proxy for criticality or popularity. Volumes are often driven by misconfiguration of proxies or caches, CI systems running wild, or any variety of problems that occur with software. There’s even folks out there who drive these numbers up artificially.

This change removes one source of potential confusion and misinterpretation, but does not mean download counts are an accurate measure of how many people use a package.

Future work: range requests

An HTTP response code 206 Partial Content counts exactly the same as a 200 OK right now. Reading two bytes out of a wheel to inspect its ZIP central directory is recorded as a download of the whole wheel, alongside a client that pulled all 30 MB.

We want to record the request type and the bytes actually transferred, so that analysis can distinguish bytes served from downloads counted. That work is not scheduled yet. Follow https://github.com/pypi/linehaul-cloud-function/issues/232 and https://github.com/pypi/linehaul-cloud-function/issues/252 for that.

Leave a Reply

Your email address will not be published. Required fields are marked *