TODO.txt

split everything into
1) fetch to zone2rr
2) parse from zone2rr

more intelligent ARPA scans? (record scanned ranges/depths and NXDOMAINs somehow (store unwalked ranges?), use AXFR/NSEC walk if possible instead of brute forcing)

collect CNAMEs in all net_* functions?

populate maybe_zone (domain lists?)
mark NXDOMAINs on ENTs (parent of valid name is NXDOMAIN; only check for parents on validated existing names and don't trust NXDOMAINs to recurse?)

scan SPF records for unreg domains?
+ other stuff relying on DNS

for name in {id,version,hostname}.bind; do echo $name; dig +short -c CH -t TXT $name @${ns_to_scan}; done

figure out better method for checking for unregistered domains (fails due to NXDOMAIN on ENT on non-spec-following DNS auth servers)

flag more stuff as registered/valid_tried based on e.g working queries

avoid registration check for parent_name insertions (check from_parent in nsRRF)

mark registrations in net_ip (NS name)/net_mx/net_ns (zone name)

add name_parent during zone file parsing + axfr?

https://medium.com/nlnetlabs/the-peculiar-case-of-nsec-processing-using-expanded-wildcard-records-ae8285f236be
https://datatracker.ietf.org/doc/html/rfc7129#section-5.3

split PSL/TLD/whatever into a special/ dir?
"maybe-zone" flag set to true for entries from domain lists, with some queries unsetting it after ascertaining whether or not it is a zone apex?
PSL too

what to do with blatantly incorrect NSEC ranges? e.g [www.example.com. .. dkim._domainkey.example.com.)
example offenders: vian.ee, laurella.ee, (both use nameservers ns{1,2}.timeweb.ru, ns{3,4}.timeweb.org)

add (more) indexes to DB?

script with prewritten SQL expressions to e.g find domains with unregistered NS-es, CNAMEs in incorrect places, AXFRable NSes etc.

move python/fetch_all.py here as well? (easier to parallelize)

profile

TLDs using root zone?

CNAMEs on apex
	subdomain takeover options caused by dangling CNAMEs
dns dumpster? sublist3r?
check Hardenize for RIA domains + domains project

enumerate ip6.arpa nameserver and other .arpa enumeration?

avoid retries on queries on e.g NXDOMAIN?

fetch zone_walk_res to RRs

TCP-responsive and UDP-responsive? only AXFR (TCP) needs direct access?

don't bother with axfr with e.g cloudflare, aws, zone.ee, dnspod etc. (mark as unresponsive? seperate flag?)

[NO-RISK]
add more data to tables to e.g prevent zone refetches
more data sources
	just domains from domains project
	just domains from old/outdated zone files (kp/ru/vn/by)
add checks for active domains due to potentially outdated sources?

domain availability check via Gandi API? https://api.gandi.net/docs/domains/
filter out invalid domains (e.g IP addresses or monstrosities like ns1.185.194.124.200.); use PSL for this?

[MEDIUM-RISK]
figure out if unresponsive DNS server IPs are on "available" IPs?