Domain and DNS Management System

A FastAPI application that allows users to upload CSV files containing domain and DNS records, storing them in a database and displaying them via an interactive UI and API interface.

Features

• File upload system with detailed upload history, including ability to delete uploads
• Main SLD view with filtering by upload and time
• Comprehensive DNS Records view with multi-faceted filtering
• TinyDB database storage with precise timestamps for data persistence
• Deduplicated DNS records (by domain, class, and type) and domains (by FQDN) showing most recent entries
• Option to view only records from a specific upload
• Clean, simplified interface showing only the latest records
• Server-side filtering by upload, record type, class, TLD, and SLD (no JavaScript needed)
• Provides comprehensive RESTful API endpoints for programmatic access
• Responsive design that works on desktop and mobile

Containerized Deployment

The application can be easily deployed in a secure rootless container using Podman or Docker.

For detailed container instructions, see CONTAINER_INSTRUCTIONS.md.

Quick start:

# Make the run script executable
chmod +x run_container.sh

# Run the container (automatically uses podman if available, falls back to docker)
./run_container.sh

Setup with Virtual Environment (Development)

1. Create a virtual environment:

   python -m venv venv
   

2. Activate the virtual environment:

   • On Windows:

     venv\Scripts\activate
     

   • On macOS/Linux:

     source venv/bin/activate
     

3. Install dependencies:

   pip install -r requirements.txt
   

4. Run the application:

   python main.py
   

5. Access the application:

   • Web interface: http://localhost:8000
   • API endpoints:
     • http://localhost:8000/api/slds
     • http://localhost:8000/api/slds/{sld}

6. Deactivate the virtual environment when finished:

   deactivate
   

CSV File Format

The application accepts CSV files with the following format:

domain.example.com,3600,IN,A,192.168.1.1
domain.example.com,3600,IN,MX,10 mail.example.com.
subdomain.example.com,3600,IN,CNAME,domain.example.com.

Where columns are:

1. Domain name (fully qualified domain name)
2. TTL (Time To Live) in seconds
3. Record Class (usually IN for Internet)
4. Record Type (A, AAAA, MX, CNAME, TXT, etc.)
5. Record Data (IP address, hostname, or other data depending on record type)

Domain Base Name Detection

The application includes functionality to identify base domains from fully qualified domain names, including handling of multi-part TLDs like ".co.uk" or ".com.au".

Multi-Part TLD List

The application uses a hardcoded list of common multi-part TLDs to correctly extract base domains (e.g., "example.co.uk" from "mail.example.co.uk").

This list can be found in main.py as MULTI_PART_TLDS.

Updating the TLD List

To ensure accurate domain parsing, you should periodically update the multi-part TLD list. The best sources for this information are:

1. Public Suffix List (PSL): The most comprehensive and authoritative source

   • Website: https://publicsuffix.org/list/
   • GitHub: https://github.com/publicsuffix/list
   • This list is maintained by Mozilla and used by browsers and DNS applications

2. IANA's TLD Database: The official registry of top-level domains

   • Website: https://www.iana.org/domains/root/db

3. Commercial Domain Registrars: Often provide lists of available TLDs

   • Examples: GoDaddy, Namecheap, etc.

For the most accurate and comprehensive implementation, consider implementing a parser for the Public Suffix List or using a library that maintains this list (e.g., publicsuffix2 for Python).

API Endpoints

• /api/uploads - Get all uploads
• /api/domains - Get all domains
• /api/base-domains - Get only unique base domains (e.g., example.com, example.co.uk) with simplified response format
• /api/domains/{domain} - Get domains by name
• /api/dns - Get all DNS records
• /api/dns/types - Get unique values for filters

Query Parameters

You can filter the API results using the following query parameters:

• upload_id - Filter by specific upload
• record_type - Filter by DNS record type
• record_class - Filter by DNS record class
• domain - Search by domain name
• base_domains_only - Only show base domains (e.g., example.com not mail.example.com)
• deduplicate - For DNS records, control whether to show all records or deduplicate

Examples:

• /api/domains?base_domains_only=true - Show only base domains
• /api/base-domains - Get a simplified list of unique base domains
• /api/dns?record_type=A&domain=example.com&deduplicate=false - Show all A records for example.com without deduplication

Response Format Examples

1. Base Domains Endpoint (/api/base-domains):

[
  {
    "domain": "example.com",
    "timestamp": "2025-04-08T12:00:00"
  },
  {
    "domain": "example.co.uk",
    "timestamp": "2025-04-08T12:00:00"
  }
]