ADD base domain information on Index page and API endpoint

README.md

@@ -85,11 +85,39 @@ Where columns are:
 4. Record Type (A, AAAA, MX, CNAME, TXT, etc.)
 5. Record Data (IP address, hostname, or other data depending on record type)
 
+## Domain Base Name Detection
+
+The application includes functionality to identify base domains from fully qualified domain names, including handling of multi-part TLDs like ".co.uk" or ".com.au".
+
+### Multi-Part TLD List
+
+The application uses a hardcoded list of common multi-part TLDs to correctly extract base domains (e.g., "example.co.uk" from "mail.example.co.uk").
+
+This list can be found in `main.py` as `MULTI_PART_TLDS`.
+
+### Updating the TLD List
+
+To ensure accurate domain parsing, you should periodically update the multi-part TLD list. The best sources for this information are:
+
+1. **Public Suffix List (PSL)**: The most comprehensive and authoritative source
+   - Website: https://publicsuffix.org/list/
+   - GitHub: https://github.com/publicsuffix/list
+   - This list is maintained by Mozilla and used by browsers and DNS applications
+
+2. **IANA's TLD Database**: The official registry of top-level domains
+   - Website: https://www.iana.org/domains/root/db
+
+3. **Commercial Domain Registrars**: Often provide lists of available TLDs
+   - Examples: GoDaddy, Namecheap, etc.
+
+For the most accurate and comprehensive implementation, consider implementing a parser for the Public Suffix List or using a library that maintains this list (e.g., `publicsuffix2` for Python).
+
 ## API Endpoints
 
 - `/api/uploads` - Get all uploads
-- `/api/slds` - Get all SLDs (Second Level Domains)
-- `/api/slds/{sld}` - Get domains by SLD
+- `/api/domains` - Get all domains
+- `/api/base-domains` - Get only unique base domains (e.g., example.com, example.co.uk) with simplified response format
+- `/api/domains/{domain}` - Get domains by name
 - `/api/dns` - Get all DNS records 
 - `/api/dns/types` - Get unique values for filters
 
@@ -100,8 +128,27 @@ You can filter the API results using the following query parameters:
 - `upload_id` - Filter by specific upload
 - `record_type` - Filter by DNS record type
 - `record_class` - Filter by DNS record class
-- `tld` - Filter by Top Level Domain
-- `sld` - Filter by Second Level Domain
 - `domain` - Search by domain name
+- `base_domains_only` - Only show base domains (e.g., example.com not mail.example.com)
+- `deduplicate` - For DNS records, control whether to show all records or deduplicate
 
-Example: `/api/dns?record_type=A&tld=com&upload_id=upload_20250408120000`
+Examples:
+- `/api/domains?base_domains_only=true` - Show only base domains
+- `/api/base-domains` - Get a simplified list of unique base domains
+- `/api/dns?record_type=A&domain=example.com&deduplicate=false` - Show all A records for example.com without deduplication
+
+### Response Format Examples
+
+1. Base Domains Endpoint (`/api/base-domains`):
+```json
+[
+  {
+    "domain": "example.com",
+    "timestamp": "2025-04-08T12:00:00"
+  },
+  {
+    "domain": "example.co.uk",
+    "timestamp": "2025-04-08T12:00:00"
+  }
+]
+```