Teleporter does not overwrite Domains

[JvdMaat]

Versions

• Docker 2022.12
• Pi-hole: 5.14.2
• AdminLTE: 5.18
• FTL: 5.20

Platform

• OS and version: Ubuntu 22.04.1 LTS
• Platform: Docker

Expected behavior

I have two piholes. I removed and added Domains (whitelist/blacklist) from one, and executed a teleporter export. On the second pihole which had the same Domains (white/blacklists) prior, I executed a teleporter import with all checkboxes checked except Audit Log. Expected the Domains to have been updated to what was on the other pihole since the "Clear existing data" checkbox was checked.

Actual behavior / bug

Domains do not get updated correctly. New domains do get added. Domains that were removed do not get removed on target. Domains that were changed from Status Enabled to Disabled do not update on target.

Initially after searching here, it was hinted this could be a locale issue. Docker runs in POSIX locale, while the source pihole (Raspberry Pi Zero W) runs UTF-8.
I changed the source to POSIX as well to have a level playing field, however this did not change the outcome.

Debug Token

• URL source pihole: https://tricorder.pi-hole.net/FquQbV5m/
• URL target pihole: https://tricorder.pi-hole.net/E9VSqEaY/

[yubiuser]

I can reproduce this. It's fairly easy: create a backup, make some changes to the domainlist and restore the backup. For me, deleted entries got re-added (this is Ok), state (enables/disabled) did not change after import (not Ok), newly added domains lost group assignment (not OK).

[github-actions]

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

[csm10495]

I'm hitting this as well. If I delete a domain in one then teleport it to the other, the domain still remains.

[gsgleason]

I've been troubleshooting the issue of an import not actually removing domains that do not appear in the archive being imported while the clear existing data option is selected.

Here is what I've found:

For domains, the POST request has some values that correspond to the checkbox inputs on the teleporter page. While processing the POST request, these get passed to archive_restore_table(), with possible values of 'table' as blacklist, regex_blacklist, whitelist, and regex_whitelist.

Line 91 of teleporter.php (https://github.com/pi-hole/web/blob/master/scripts/pi-hole/php/teleporter.php#L91) is the part of archive_restore_table() that will flush the tables, if requested, but since blacklist, regex_blacklist, whitelist, and regex_whitelist are not actually table names in the gravity database, no flush is performed.

What's interesting is that there is another function, archive_insert_into_table() , which handles text files in the archive (as opposed to json files). I'm guessing this is for an older version. This has a section starting on line 241 that will set $table to "domainlist" for whitelist, blacklist, and regex_blacklist (It's missing regex_whitelist for some reason; maybe it didn't exist for the version that used plain text files for the domain lists). When it gets to the part that flushes tables, it correctly flushes the domainlist table prior to inserting the new data.

What's also weird is that the archive_insert_into_table() uses another function flush_table(), whereas archive_restore_table() has it's own logic for flushing the requested table. I don't see why there are two places to do the same thing, and I don't understand why there are two functions to import the data depending on the file type.

This could be fixed many different ways; domainlist needs to be flushed based on the type of entry, depending on which domain list is being updated.

Here was my solution:

gsg@apu1d4:/var/www/html/admin/scripts/pi-hole/php$ diff teleporter.php teleporter.bak
135,136c135
<             $type = LISTTYPE_WHITELIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 0;
138,139c137
<             $type = LISTTYPE_BLACKLIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 1;
141,142c139
<             $type = LISTTYPE_REGEX_WHITELIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 2;
144,145c141
<             $type = LISTTYPE_REGEX_BLACKLIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 3;

I would rather refactor the two functions to both use flush_table(), but I feel like this is an adequate proof-of-concept.

[Exit24]

I'm having a similar issue using Pi-hole v5.18.3 FTL v5.25.2 Web Interface v5.21.

I have 2 piholes. I added a whitelist entry on pihole 1, and then I ran a teleporter backup. On pihole 2 I executed a teleporter import with all checkboxes checked. The new whitelist entry on pihole 1 doesn't appear on pihole 2 after the teleporter import.