Data Collection Methodology

Overview

Collection runs in three main stages:

1. Discovery — candidate hostnames are gathered from public data, Certificate Transparency logs, and user submissions, then fed into the detector.
2. DNS classification — each name is looked up to decide whether it is IPv6-only.
3. HTTP checking — IPv6-only names are probed over HTTP and HTTPS; response codes and page titles are recorded.

Only entries marked active in the database appear on the public website. Names that fail the IPv6-only test are kept out of the list (or marked inactive on recheck). Domain owners may also opt out; opted-out names are excluded even when they would otherwise qualify.

1. Discovering hostnames

Candidate names come from publicly available sources and voluntary submissions, not from crawling the live web for content. Typical sources include:

• Certificate Transparency (CT) logs — publicly logged TLS certificate issuances are monitored for hostnames and domains that later prove to be IPv6-only.
• User-submitted hostnames — names sent in by users or operators who want a site considered for the list.
• BIND zone files — delegated domains are extracted from NS (or other) records.
• Registry CSV exports — for example AFNIC-style lists with a domain name in the first column.
• Plain domain lists — one fully qualified domain name per line.
• DNS walk lists — files of deep hostnames (e.g. host.dept.university.ac.th) are expanded to include each parent domain up to the public suffix, so zone cuts and delegations are also tested.

This approach casts a wide net across known names. It does not attempt to enumerate every possible hostname on the Internet.

2. DNS classification

The ipv6_detector service receives hostnames and performs DNS lookups, preferring IPv6-capable resolvers. For each name:

AAAA (IPv6) lookup

• At least one valid IPv6 address must be returned.
• Loopback (::1), link-local, unique-local, documentation, and other non-global addresses are ignored.
• If every AAAA address is loopback, the name is treated as having no usable IPv6.

A (IPv4) lookup

• If any publicly routable IPv4 address is returned, the name is classified as dual-stack and is not IPv6-only.
• Private, reserved, loopback, link-local, and multicast IPv4 addresses are ignored (e.g. RFC 1918 space). A name with only such addresses and valid global IPv6 still counts as IPv6-only.
• “No such host” for A records is expected for true IPv6-only names.

Result

Names that have usable global IPv6 and no public IPv4 are stored in the database with active=1, along with their IPv6 address(es). Others are not promoted to the public list. Lookups are repeated periodically in recheck mode so entries that gain or lose IPv4 are updated.

3. HTTP and HTTPS probing

The http_checker service tests names over IPv6. For each hostname:

• Connections are made over IPv6 only (not via IPv4).
• Both http:// and https:// are tried.
• The HTTP status code is recorded for each protocol. A code of 0 means the connection failed or timed out.
• Page titles are extracted when a response body is available.
• 401 Unauthorized is treated as a working service; the title is set from the WWW-Authenticate header.
• For HTTPS, certificate problems may still allow the HTTP status (e.g. a redirect) to be captured using a relaxed TLS check, while noting that the certificate is invalid.
• Requests identify themselves with the User-Agent IPv6-Only Site Survey and a link back to this project.

To avoid hammering sites, hosts already checked within the last 24 hours are skipped. Domains on the opt-out blacklist (and their subdomains) are not checked at all. New hostnames encountered only by the HTTP checker are inserted as inactive so they are tracked for cooldown purposes but do not appear on the public list until the DNS pipeline confirms they are IPv6-only.

A site may be IPv6-only but show neither HTTP nor HTTPS working — it can still appear on the list if it resolves correctly; it may simply use IPv6 for non-web services.

4. Opt-out / blacklist

Domain owners can request that their domain be excluded from the survey. Opted-out domains are recorded in a blacklist and are not HTTP-checked. Matching is by domain suffix on label boundaries: if example.com is opted out, then www.example.com and other subdomains under that name are excluded as well.

Blacklisted domains stay off the published list even if they resolve as IPv6-only. Operators who want to be removed should contact the address in the footer.

5. What appears on the website

The public list at v6sites_optimized.php shows database rows where:

• active = 1 (passed the IPv6-only DNS test)
• the name is not covered by an opt-out / blacklist entry
• Name, IPv6 address(es), last update time, HTTP/HTTPS status codes, and title (when known)

Statistics and CSV export use the same active set. Result pages may be served from a short-lived file cache when the database is under load or temporarily unavailable.

6. Limitations

• Not exhaustive — only names from the chosen sources (including CT logs and submissions) are tested. Many IPv6-only sites will never appear.
• Opt-outs are respected — a domain that has opted out will not appear, even if it has IPv6-only sites.
• DNS-dependent — classification reflects what resolvers return at check time. CDN, geo, or split-horizon DNS can affect results.
• Dual-stack sites are excluded — a site with both public IPv4 and IPv6 is not listed, even if it is reachable over IPv6.
• Stale data — rechecks run on a schedule, not continuously; a site can change between passes.
• HTTP status is informational — a zero or error status does not remove a site from the IPv6-only list if DNS still qualifies it.