Support static MAC address + fix clone VNET jail #725

tschettervictor · 2024-10-09T17:45:59Z

This PR adds some functionality, and some fixes.

support for static MAC VNET jails (bridged and vnet)
fix cloning VNET jails
simplify some sed commands when cloning jails
allow cloning VNET jails and regenerate MAC address for cloned jail

This PR has been tested with bridged VNET, VNET, and loopback jails. Steps below

Create jail
Ensure network works
Clone jail
Ensure network works on cloned jail
Ensure jail.conf is updated properly on cloned jail
Ensure rc.conf is updated properly on cloned jail

This commit will generate a static MAC address for each jail, based on the name of the jail. It will use the first half (xx:xx:xx) of the host MAC to avoid network clashes, and generate a random HEX string from the hashed name of the jail. It will then add that random 5 character HEX string in MAC format, and add an "a" and "b" for the host and jail respectively. This way a jail can retain it's MAC ID even if it is deleted and reinstalled, as long as the same name is retained.

Current implementation allows for cloning jails that were created using a static MAC. Also removed some unnecessary sed strings. These were simplified.

- fixed cloning of VNET bridged jails - added support for regenerating static MAC on cloned jails - simplified some sed commands to edit cloned jail.conf file Tested with bridged VNET, VNET, and loopback jails

- support static MAC on bridged and VNET jails - remove quotes around vnet.interface

tschettervictor · 2024-11-24T18:16:24Z

Bumping this to get some conversation going.
For me this is absolutely necessary to have as jails in my networks have MAC filtering applied and need to have a static MAC address.

Thoughts?

This reverts some changes the the "update_jailconf" function. The reason behind this revert is that if a jail somehow has the same name as a directory, then the previous commit would have changed the directory name as well, which would break stuff. The current code avoids all that and only replaces the necessary jail name value.

tschettervictor · 2024-12-04T15:22:58Z

I've reverted some edits to the update_jailconf function after a conversation at #666 regarding jail names. The original commit would have rename even a directory if it had the same name as the jail. The original code, however, handled it perfectly, by only renaming the instances it needed to, so I reverted that part.

The other parts seem fine though, regarding the static MAC and cloning jails, so I'm still pushing for this PR.

tschettervictor · 2024-12-06T20:11:00Z

I've modified this to include the function "generate_static_mac" in "common.sh" since it is only ever used when creating a jail.

When cloning a jail, the same principle is used, but since the "external_interface" variable is not passed to "clone.sh" it is handled like this: since the jail.conf file is copied over from the old jail, and then edited, we can simply "cat" it to retrieve the MAC prefix. Then, the same hashing method is applied to retrieve the suffix based on the new jail name.

I've tested the clone function quite extensively, even cloning jail into the double digit numbers just to make sure the sed commands properly replace values in "jail.conf" and "rc.conf"

bmac2 · 2024-12-08T01:58:14Z

so this PR ONLY changes mac address for vnet jails and bridged jails, not others. @tschettervictor, correct? It does not make any changes to a shared networking, which uses the mac address of the adapter for the mac address, all the same.

Did you intend it to touch shared networking devices?

tschettervictor · 2024-12-08T02:01:38Z

That is correct. Only VNET and bridged VNET are affected.

Shared network jails do not need to have a static MAC or any type of MAC really, since they use the host stack.

bmac2 · 2024-12-08T02:16:04Z

when I clone a vnet jail, it gets a new hwaddr. BUT if I reboot a cloned jail the hwaddr changes. It didn't on the original jail I cloned from, it was static. What am I missing?

tschettervictor · 2024-12-08T02:18:32Z

I'm not 100% sure what the hw address if for to be honest. But the actual MAC is the one found under "ether"

Iocage does it the same way.

tschettervictor · 2024-12-08T02:21:05Z

Here are 2 screenshot of an Iocage jail after rebooting it. The ether is the same, but the hw is always different.

bmac2 · 2024-12-08T02:27:13Z

PR tests OK for me. no issues I can find. @yaazkal is going to confirm my testing prior to merging this pr.

bmac2 · 2024-12-26T04:59:10Z

same error. not cloning jail correctly.

root@fbsd:/usr/local/share/bastille # bastille clone jail4 clonedjail4 192.168.1.238/24
Valid: (192.168.1.238/24).
Attempting to clone 'jail4' to clonedjail4...
grep: /usr/local/bastille/jails/clonedjail4/jail.conf: No such file or directory
Cloned 'jail4' to 'clonedjail4' successfully.
root@fbsd:/usr/local/share/bastille # bastille list all
JID State IP Address Published Ports Hostname Release Path
jail1 Up 192.168.1.155 - jail1 14.1-RELEASE /usr/local/bastille/jails/jail1/root
jail4 Up 192.168.1.159 - jail4 14.1-RELEASE /usr/local/bastille/jails/jail4/root
jail5 Up 192.168.1.234 - jail5 14.1-RELEASE /usr/local/bastille/jails/jail5/root
root@fbsd:/usr/local/share/bastille #

bmac2 · 2024-12-26T05:09:35Z

grep: /usr/local/bastille/jails/clonedjail4/jail.conf: No such file or directory

that is the error that is blowing it up. Mine is NOT copying the jail.conf file over, nor is it writing a new one. Looked at the code and did NOT see where it copies over or creates the new jail.conf file prior to checking for it.

also, just checked and it is not even creating a directory for the new cloned jail at all. So no path for it to follow to find a conf file.

bmac2 · 2024-12-26T05:21:55Z

from looking at the code, there is a copy command to copy the jail.conf when you are NOT using zfs, that is line 194. But looking above that in the zfs section I do not see it copying the jail.conf file over and creating the paths. I have zfs enabled on all my stuff. What am I missing in the section from line 167 to 185 where it should create the path and copy the jail.conf file over .

@tschettervictor @yaazkal

tschettervictor · 2024-12-26T14:01:25Z

ZFS clones the entire jail dataset and sends it to the new location, so the jail.conf file is included.

Can you share the output of "mount | grep yournewjailname?

tschettervictor · 2024-12-26T15:37:11Z

One thing to not. I had to let bastille create its own "bastille_zfs_prefix" which it then mounts at "bastille_prefix"

If this is not done, then it is not mounted at the right place, and bastille will not use "bastille_prefix" as its mount point.

This results in jails not ending up where bastille expects them to end up.

bmac2 · 2024-12-26T17:49:39Z

root@fbsd:/home/bmccormick # mount | grep jail4
/usr/local/bastille/releases/14.1-RELEASE on /usr/local/bastille/jails/jail4/root/.bastille (nullfs, local, noatime, read-only, nfsv4acls)
devfs on /usr/local/bastille/jails/jail4/root/dev (devfs)
root@fbsd:/home/bmccormick #

bmac2 · 2024-12-26T17:50:59Z

I let bastille create all the mount points. I did exactly ONLY the steps I listed a few comments ago.

bmac2 · 2024-12-26T17:53:53Z

jail4 is the one I am trying to clone. The NEW jail does not show any mount points nor does it create the directory for it at all

root@fbsd:/home/bmccormick # bastille clone jail4 clone4 192.168.1.153/24 Valid: (192.168.1.153/24). Attempting to clone 'jail4' to clone4... grep: /usr/local/bastille/jails/clone4/jail.conf: No such file or directory Cloned 'jail4' to 'clone4' successfully.

tschettervictor · 2024-12-26T18:06:19Z

What does ZFS list show?

I had one issue where it was trying to take snapshots when they were already created, and it errored out.

Also, maybe add "set -x" to the clone file to see what's going on.

bmac2 · 2024-12-26T18:28:02Z

root@fbsd:/home/bmccormick # zfs list
NAME USED AVAIL REFER MOUNTPOINT
zroot 2.50G 226G 96K /zroot
zroot/ROOT 2.50G 226G 96K none
zroot/ROOT/default 2.50G 226G 2.50G /
zroot/home 236K 226G 96K /home
zroot/home/bmccormick 140K 226G 140K /home/bmccormick
zroot/tmp 96K 226G 96K /tmp
zroot/usr 288K 226G 96K /usr
zroot/usr/ports 96K 226G 96K /usr/ports
zroot/usr/src 96K 226G 96K /usr/src
zroot/var 736K 226G 96K /var
zroot/var/audit 96K 226G 96K /var/audit
zroot/var/crash 96K 226G 96K /var/crash
zroot/var/log 248K 226G 248K /var/log
zroot/var/mail 104K 226G 104K /var/mail
zroot/var/tmp 96K 226G 96K /var/tmp
root@fbsd:/home/bmccormick #

tschettervictor · 2024-12-26T18:29:37Z

Your bastille mounts aren't even showing...

I'm on IRC now. How can we hook up?

bmac2 · 2024-12-26T18:32:41Z

ZFS options

bastille_zfs_enable="YES" ## default: ""
bastille_zfs_zpool="" ## default: ""
bastille_zfs_prefix="bastille" ## default: "${bastille_zfs_zpool}/bastille"
bastille_zfs_options="-o compress=lz4 -o atime=off" ## default: "-o compress=lz4 -o atime=off"

all default options, no changes from default

tschettervictor · 2024-12-26T18:33:40Z

You have to add your zpool there.

Here is my zfs list

root@bastille:~ # zfs list | grep bastille
zroot/bastille                        1.01G  33.7G      416K  /usr/local/bastille
zroot/bastille/backups                 384K  33.7G      384K  /usr/local/bastille/backups
zroot/bastille/cache                   210M  33.7G      384K  /usr/local/bastille/cache
zroot/bastille/cache/13.4-RELEASE      209M  33.7G      209M  /usr/local/bastille/cache/13.4-RELEASE
zroot/bastille/jails                  59.8M  33.7G      384K  /usr/local/bastille/jails
zroot/bastille/jails/jail1            19.8M  33.7G      400K  /usr/local/bastille/jails/jail1
zroot/bastille/jails/jail1/root       19.4M  33.7G     19.4M  /usr/local/bastille/jails/jail1/root
zroot/bastille/jails/jail2            19.8M  33.7G      400K  /usr/local/bastille/jails/jail2
zroot/bastille/jails/jail2/root       19.4M  33.7G     19.4M  /usr/local/bastille/jails/jail2/root
zroot/bastille/jails/jailvnet         19.8M  33.7G      400K  /usr/local/bastille/jails/jailvnet
zroot/bastille/jails/jailvnet/root    19.5M  33.7G     19.5M  /usr/local/bastille/jails/jailvnet/root
zroot/bastille/logs                    512K  33.7G      512K  /var/log/bastille
zroot/bastille/releases                765M  33.7G      384K  /usr/local/bastille/releases
zroot/bastille/releases/13.4-RELEASE   764M  33.7G      764M  /usr/local/bastille/releases/13.4-RELEASE
zroot/bastille/templates               384K  33.7G      384K  /usr/local/bastille/templates
root@bastille:~ #

tschettervictor · 2024-12-26T18:34:15Z

zroot/bastille is mounted at /usr/local/bastille

bmac2 · 2024-12-26T18:36:53Z

what is your bastille.conf ?

tschettervictor · 2024-12-26T18:38:15Z

ZFS options

bastille_zfs_enable="YES" ## default: ""
bastille_zfs_zpool="zroot" ## default: ""
bastille_zfs_prefix="bastille" ## default: "${bastille_zfs_zpool}/bastille"
bastille_zfs_options="-o compress=lz4 -o atime=off" ## default: "-o compress=lz4 -o atime=off"

Everything else is default.

bmac2 · 2024-12-26T18:39:32Z

ZFS options

bastille_zfs_enable="YES" ## default: ""
bastille_zfs_zpool="zroot" ## default: ""
bastille_zfs_prefix="bastille" ## default: "${bastille_zfs_zpool}/bastille"
bastille_zfs_options="-o compress=lz4 -o atime=off" ## default: "-o compress=lz4 -o atime=off"

mine and it is not creating mount points like yours

tschettervictor · 2024-12-26T18:41:26Z

You need to let bastille create "zroot/bastille" or it will not mount it correctly.

I recommend trashing it and restarting. Just zfs destroy -r zroot/bastille

Then bootstrap a release and it will do its thing automatically.

bmac2 · 2024-12-26T18:41:33Z

root@fbsd:/usr/local/etc/bastille # bastille list all
JID State IP Address Published Ports Hostname Release Path
jail1 Down 192.168.1.155 - jail1 14.1-RELEASE /usr/local/bastille/jails/jail1/root
jail2 Down 192.168.1.156 - jail2 14.1-RELEASE /usr/local/bastille/jails/jail2/root
jail3 Down 192.168.1.152,10.0.0.152 - jail3 14.1-RELEASE /usr/local/bastille/jails/jail3/root
jail4 Up 192.168.1.159 - jail4 14.1-RELEASE /usr/local/bastille/jails/jail4/root
jail5 Down 192.168.1.160 - jail5 14.1-RELEASE /usr/local/bastille/jails/jail5/root
jail6 Up 192.168.1.162 - jail6 14.1-RELEASE /usr/local/bastille/jails/jail6/root
jail7 Up 192.168.1.163 - jail7 14.1-RELEASE /usr/local/bastille/jails/jail7/root
root@fbsd:/usr/local/etc/bastille #

jail6 and jail7 were created after setting the zfs root setting. Still no love

tschettervictor · 2024-12-26T18:43:23Z

I had to completely restart after that happened to me. The problem is that if bastille does not create the zroot/bastille dataset, then it never gets mounted at the right place. This then affects everything going forward.

bmac2 · 2024-12-26T18:49:40Z

cannot create 'zroot/bastille/jails/jail7': parent does not exist
cannot create 'zroot/bastille/jails/jail7/root': parent does not exist
that is my error now

tschettervictor · 2024-12-26T18:50:59Z

Were you able to destroy zroot/bastille?

I had that error too before I completely wiped it.

bmac2 · 2024-12-26T18:54:22Z

the default settings are NOT to create a new zfs mount for each jail. The default action is to just put them in the
/usr/local/bastille/jails/ directory. When did creating a new zfs for each jail become default settings?? Do we want to enforce that?

@yaazkal

bmac2 · 2024-12-26T18:56:16Z

root@fbsd:/home/bmccormick # bastille create -V jail8 14.1-RELEASE 192.168.1.162/24 re0
Valid: (192.168.1.162/24).
Valid: (re0).
cannot create 'zroot/bastille/jails/jail8': parent does not exist
cannot create 'zroot/bastille/jails/jail8/root': parent does not exist

Creating a thinjail...

no IP address found for not set
[jail8]:
e0a_bastille1
e0b_bastille1

bmac2 · 2024-12-27T06:41:35Z

after fixing my zfs issues this PR tests correctly.

things tested.

Cloning jails, both vnet and normal jailes
static ether addresses holding after reboot of jail

Merging PR

yaazkal · 2024-12-27T12:08:35Z

thank you @bmac2 unfortunately I had no time to test it.
thank you @tschettervictor for the initiative.

tschettervictor added 8 commits October 8, 2024 16:21

support for -V option also

2560b43

spacing edit

cc75f45

beginning work to allow cloned jail with new static MAC feature

a9c1bae

Current implementation allows for cloning jails that were created using a static MAC. Also removed some unnecessary sed strings. These were simplified.

Uncomment for non-VNET jails

ee21616

Add see command to rename bridges interface

5fab649

support cloning of bridged VNET jails + static MAC jails

ca66263

- fixed cloning of VNET bridged jails - added support for regenerating static MAC on cloned jails - simplified some sed commands to edit cloned jail.conf file Tested with bridged VNET, VNET, and loopback jails

add support for static MAC on VNET jails

55203b2

- support static MAC on bridged and VNET jails - remove quotes around vnet.interface

tschettervictor mentioned this pull request Oct 9, 2024

[ENHANCEMENT] Allow for jails with static MAC addresses #716

Closed

tschettervictor added 2 commits November 10, 2024 15:31

Merge branch 'BastilleBSD:master' into master

8a6002d

Merge branch 'BastilleBSD:master' into master

dd0d805

tschettervictor added 2 commits December 4, 2024 08:18

Update clone.sh - remove duplicate line

9a157f2

tschettervictor added 3 commits December 6, 2024 11:39

Update common.sh - move generation of static mac to main functions

203af6c

Update common.sh - clean up static mac code

e055c87

Update clone.sh - final commit for static mac cleanup

6a3d675

tschettervictor added 5 commits December 8, 2024 13:21

Merge branch 'BastilleBSD:master' into master

a42d122

Merge branch 'BastilleBSD:master' into master

176e36d

typo

b127193

Merge branch 'BastilleBSD:master' into master

05d645e

Update clone.sh

3f0a430

bmac2 mentioned this pull request Dec 26, 2024

Update clone.sh - simplify "update_fstab" function #757

Closed

Merge changes from previous PR

a6e4902

bmac2 merged commit 43b839a into BastilleBSD:master Dec 27, 2024
1 check passed

yaazkal mentioned this pull request Jan 4, 2025

Fix generate_static_mac #781

Merged

Support static MAC address + fix clone VNET jail #725

Support static MAC address + fix clone VNET jail #725

Conversation

tschettervictor commented Oct 9, 2024

tschettervictor commented Nov 24, 2024

tschettervictor commented Dec 4, 2024

tschettervictor commented Dec 6, 2024

bmac2 commented Dec 8, 2024

tschettervictor commented Dec 8, 2024

bmac2 commented Dec 8, 2024

tschettervictor commented Dec 8, 2024

tschettervictor commented Dec 8, 2024

bmac2 commented Dec 8, 2024

bmac2 commented Dec 26, 2024

bmac2 commented Dec 26, 2024 • edited Loading

bmac2 commented Dec 26, 2024 • edited Loading

tschettervictor commented Dec 26, 2024

tschettervictor commented Dec 26, 2024

bmac2 commented Dec 26, 2024

bmac2 commented Dec 26, 2024

bmac2 commented Dec 26, 2024 • edited Loading

tschettervictor commented Dec 26, 2024 • edited Loading

bmac2 commented Dec 26, 2024

tschettervictor commented Dec 26, 2024

bmac2 commented Dec 26, 2024

ZFS options

tschettervictor commented Dec 26, 2024

tschettervictor commented Dec 26, 2024

bmac2 commented Dec 26, 2024

tschettervictor commented Dec 26, 2024

ZFS options

bmac2 commented Dec 26, 2024

ZFS options

tschettervictor commented Dec 26, 2024

bmac2 commented Dec 26, 2024

tschettervictor commented Dec 26, 2024

bmac2 commented Dec 26, 2024

tschettervictor commented Dec 26, 2024

bmac2 commented Dec 26, 2024

bmac2 commented Dec 26, 2024

bmac2 commented Dec 27, 2024

yaazkal commented Dec 27, 2024

bmac2 commented Dec 26, 2024 •

edited

Loading

bmac2 commented Dec 26, 2024 •

edited

Loading

bmac2 commented Dec 26, 2024 •

edited

Loading

tschettervictor commented Dec 26, 2024 •

edited

Loading