Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] network: Bridge interface name limit #786

Open
tschettervictor opened this issue Jan 3, 2025 · 52 comments
Open

[BUG] network: Bridge interface name limit #786

tschettervictor opened this issue Jan 3, 2025 · 52 comments
Labels
bug Something isn't working

Comments

@tschettervictor
Copy link
Collaborator

Just ran into this. Tried to create a jail named "bridgenomac" and ifconfig ran into an error because the resulting interface it tried to create and name was called "e11a_bridgenomac" which is apparently too long for ifconfig. See below...

ifconfig: ioctl SIOCSIFNAME (set name): File name too long
jail: bridgenomac: ifconfig epair11a up name e11a_bridgenomac: failed

But when I simply call the jail "bridgenoma" without the ending "c" then it works. So it seems there is a 14 character limit the the interface names.

I would vote to simply use the model employed by VNET which is to call the interfaces "bastille1 bastille2 bastille3 etc..."
We could just call the "e11a_epair e12a_epair" or simply avoid calling them any name, and just leaving them as "epair11a epari12a" and just have a better description for each interface.
@JRGTH @bmac2 @yaazkal

@tschettervictor tschettervictor added the bug Something isn't working label Jan 3, 2025
@tschettervictor tschettervictor changed the title [BUG] [BUG] network: Bridge interface name limit Jan 3, 2025
@bmac2
Copy link
Collaborator

bmac2 commented Jan 5, 2025

good catch that there is a limit on the name length. I for one had no clue ifconfig had a limit. Especially based on the long names linux gives interfaces. Makes me wonder if they rewrote the linux ifconfig to make it not have that limit.

@tschettervictor
Copy link
Collaborator Author

It's just as easy to simply use epair1a and epair1b as the names. That how VNET does it without bridges. Then simply add a description for the interface.

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 11, 2025

Are we ok with leaving the bridges name at "epair0...a|b" going forward?

It's not integrated yet, but I think this is a necessary change. Especially if one has long jail names linked to bridge interfaces.

Instead of "e5b_longjailnameher_a" it would simply be "epair5b" with a description. This is that same as the current VNET does it without the bridge interfaces.

@bmac2 @yaazkal @JRGTH

@bmac2
Copy link
Collaborator

bmac2 commented Jan 11, 2025

I am good with that

@JRGTH @yaazkal what do you guys think?? Will that break anything?

@tschettervictor
Copy link
Collaborator Author

Looks like we really have no choice.

@tschettervictor
Copy link
Collaborator Author

And it won't affect existing setups.

@2Belette
Copy link

I find the current way for Bastille to create epair name very convenient as it is much easier when doing some interface based monitoring... but at the same time I understand that for long jail name it makes some issue...

Maybe an alternative is to use the JID, at lease we have a simple way to correlate the epair name with the jail, and the JID size will always fit, something like epair_JIDa|epair_JIDb.

Without that it is a pain to lose the context even if there always be a way to find out but for me the current method with the name is very convenient

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 15, 2025

If we do implement this, we would add a description to the interface, just like the "e0a|b_bastilleX VNET interfaces do.

Would that be enough for you to be able to monitor the jails?

I don't think using the JID would work, because that can change on each reboot of the server and jails.

@2Belette

@2Belette
Copy link

Good point for JID and that it is not consistent.

I am not familiar with our current description, where can it be found ?
As long as we can retrieve easily the corresponding interface pair name with jails this should be ok, as on large installation that could be hundreds of them...

@tschettervictor
Copy link
Collaborator Author

It would be found by 'ifconfig epairXa'

epair0a: flags=8962<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vnet host interface for Bastille jail testjail

@2Belette
Copy link

Then it would be very nice to automatize/grep things !

@michael-o
Copy link
Contributor

epair is generic, does it make sense to have a prefix for better grepping?

@tschettervictor
Copy link
Collaborator Author

Open to ideas...
The current way will not work for me long term.

@2Belette
Copy link

@michael-o as long as the description is present to ifconfig we should be easily able to match , or maybe I am missing your point ?

@tschettervictor
Copy link
Collaborator Author

We need to figure this out here, as #808 implements this.

@2Belette
Copy link

2Belette commented Jan 15, 2025

I am testing the latest master to check the current behavior, and it is like it has changed from yesterday ?

I have for a jail test a e11a_test on my host, but in the jail got e11b_test but not anymore the vnet0 and no more _name in rc.conf with the corresponding ifconfig, my jail is not set with any IP address.

I have used : bastille create -B test 14.2-RELEASE 10.10.10.201/24 mybridge

EDIT : it seems to be related to some template modification I done, I am under investigation

EDIT2 : as soon as I got PKG in my base template I got the issue I explained above, can't I use it on base and need another template pass for PKG ?

@tschettervictor
Copy link
Collaborator Author

Can you share the template you are using?

@tschettervictor
Copy link
Collaborator Author

@michael-o Are we ok with naming the epairs simply epair?

We could also do "bastille_epair0" etc... but i feel that would be unnecessary if we include the description.

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 15, 2025

#813 Has the fix mentioned below.

@2Belette I'm seeing this too now. It looks like a new bug. It looks like the BASE templates is being pulled in first, and because you have PKG in that one, the IP, gateway, etc... are not yet added as per the VNET template. This then caused the PKG to fail, and the whole template command to fail.

I can solve this issue by not having the pkg command exit on error, but if that is preferable I don't know. Because the BASE template is being pulled before the VNET template, having the PKG command in there makes little sense...

@tschettervictor
Copy link
Collaborator Author

@2Belette Can you test the way #808 does epair names? It names the simply "epairX" and adds a description to identify each one with the jail it is tied to.

@2Belette
Copy link

Sorry it took some time to implement the monitoring with the current system.
Here is the result, with our current naming, which is working out of the box and getting the context per epair/jail with no change. (but it is an issue for long jail names..)

2025 01 15 22 45 56

I will test #808 and see if I have a way to use description instead of interface name...

@2Belette
Copy link

#813 Has the fix mentioned below.

@2Belette I'm seeing this too now. It looks like a new bug. It looks like the BASE templates is being pulled in first, and because you have PKG in that one, the IP, gateway, etc... are not yet added as per the VNET template. This then caused the PKG to fail, and the whole template command to fail.

I can solve this issue by not having the pkg command exit on error, but if that is preferable I don't know. Because the BASE template is being pulled before the VNET template, having the PKG command in there makes little sense...

Thanks, that something I am learning and not familar with, I am still trying to understand the best-practices in term of templates. The reason why I was using PKG on base is to include the basic pkg I am using on all the jails, but perhaps it is not a good idea and needs another specific template for that? I did it because I liked to have everything built with everything I need in one place.

@tschettervictor
Copy link
Collaborator Author

I'd create another one, and make a habit of running it after jail creation.

IMO it is best to leave default templates as they will be changed upon a 'pkg upgrade' if bastille has a new version.

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 15, 2025

"ifconfig epairXa | grep description | rev | awk '{print $1}' | rev"

Will get you the jail name linked to the epair interface.

Or if you want to grep the epair value from the jailname, then

"grep -E -m 1 "epair[0-9]+" ${bastille_jailsdir}/jailname/jail.conf"

Will print the epair name of said jail.

Or simply "bastille config jailname get vnet.interface" this will print the "b" side of the epair.

@2Belette
Copy link

Thanks your commands are working fine.

My issue is more on the monitoring part where, even if I add --collector.netdev.address-info to node_exporter arguments it doesn't seem to export ifconfig description. I don't see it in the code https://github.com/prometheus/node_exporter/blob/master/collector/netdev_common.go

@JRGTH
Copy link
Collaborator

JRGTH commented Jan 16, 2025

Hi, I'm currently writing bastille setup ZFS activation stuff but just on evenings, hence can't really test all this new PR's atm, but I thing similar naming like eXb_bastilleX might work and the description, however I only speaking for simple jail networking here so can't comment or have any use atm for a single jails with multiple IP/IF/Bridge and let alone pf/ipfw so I know things get complicated further with all that.

Just to denote, I personally when coding this kind of stuff I avoid for special characters and spaces at all costs, in order to avoid unexpected issues with third party managers and database/metrics software with picky configuration languages.

I think jib can take care for users using bastille with standard/simple networking though, here is a picture of some test jails IF naming though the "Webmin Bastille Manager"(outdated) that I just use for monitoring purposes until I have time to revamp it with the latest features an additions:
Screenshot from 2025-01-16 01-35-57

Regards!

@michael-o
Copy link
Contributor

@michael-o as long as the description is present to ifconfig we should be easily able to match , or maybe I am missing your point ?

The reason is to be able to filter by interface name. You cannot filter by description and then determine the interface reliably. So to speak: Which interfaces are used by bastille only...

@2Belette
Copy link

I see and I agree.. It is relatively simple when greping but becomes very complicated when hundreds of interfaces and to monitor that...

Maybe we should start to ask / list what are our usecases.

For me, the only reason I need/see is to identify which interface(s) belong to which jail for troubleshooting and monitoring purposes, I honestly don't see anything else as from host perspective, watching ifconfig is not providing a lot, it is like a bunch of virtual patches connected to a bridge! But it is extremely valuable as soon as we want to monitor.

The easiest way for me is to keep what we do and cuting the length to 14 when it is exceeding.
So from the original example e11a_bridgenomac would be e11a_bridgenom

I think this should be acceptable as like @JRGTH I am trying to keep the jail name short, simple and without any special character, this would give 9 characters available if we keep eXXa_jail name

I like the description, but I can't find a way to grab it from the monitoring tools I am testing so far..

@tschettervictor
Copy link
Collaborator Author

The problem with shrinking the length will cause issues when cloning the jail. There is no way to tell what the old interface name was...

Would adding a -n switch to the create command help? This would tell create to act like, use jailname for interface name.

This would leave the default "epair" name, but use the jailname name if the -n is specified AND the jailname does not exceed the max length.

@2Belette
Copy link

l like the approach !

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 16, 2025

If no one has anything against it, I will implement.

This will only apply to bridge interfaces.

Current naming of VNET as show in @JRGTH image above will stay.

@tschettervictor
Copy link
Collaborator Author

@2Belette This is harder than I thought. This will making cloning somewhat of a pain.

Wondering if anyone has a better way?

@tschettervictor
Copy link
Collaborator Author

Would it be preferable to just be the name of the jail, instead of eXa_jailname?

@2Belette
Copy link

What about jails with multiple interfaces and potentially multiple interfaces on multiple bridges. We should have a way to differentiate them

@tschettervictor
Copy link
Collaborator Author

Good point.

So having it eXa_jailname is the best then?

What about having it on the VNET interfaces as well? How could we do that?

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 17, 2025

@2Belette I think it best to keep using the current way of naming interfaces, and if the jailname exceeds 15 characters, then use "epairX" as the name.

This will keep it simple for users everywhere, and not have to add unnecessary switches.

Are we agreed on this?

I have the above working locally here just fine. Now to integrate it into cloning too.

@2Belette
Copy link

I am perfectly fine with that as I keep my jails name short by purpose. So multiple interfaces would produce something like eXXa_jailname eYYa_jailname eZZa_jailname ?

@tschettervictor
Copy link
Collaborator Author

Correct.

I just need to do the rename then I'll create a PR.

@2Belette
Copy link

Just a small idea / question : would not be a good idea to add the description anyway (with or without using -n switch) ? I am thinking about multiple interfaces (and potentially into different bridges). adding a good description would help also to recognize from the host perspective, as eXXa_jailname is already providing the context (the jail) but XX or YY is somehow more obscure without going to the jail and figure it out. Don't know what is possible in term of description but if we have a way to put the name of the interface from the jail into the description this would give the name in the other end. Supporting alias would also help for example one alias in the jail is vtnet0 = vlan100, vlan100 in the description host for the same pair would help a lot to understand the relationship.

@tschettervictor
Copy link
Collaborator Author

Just a small idea / question : would not be a good idea to add the description anyway (with or without using -n switch) ? I am thinking about multiple interfaces (and potentially into different bridges). adding a good description would help also to recognize from the host perspective, as eXXa_jailname is already providing the context (the jail) but XX or YY is somehow more obscure without going to the jail and figure it out. Don't know what is possible in term of description but if we have a way to put the name of the interface from the jail into the description this would give the name in the other end. Supporting alias would also help for example one alias in the jail is vtnet0 = vlan100, vlan100 in the description host for the same pair would help a lot to understand the relationship.

Perhaps open an enhancement report about that.

I'll get this all functional (description is included) and focus on the rest later.

@2Belette
Copy link

Done #817

@tschettervictor
Copy link
Collaborator Author

@2Belette Im going to lay this out here, and if you still want it, I'll open a PR.

Basically if we do the naming thing as I suggested (if the character limit is exceeded, we use epair naming) it will work, but adds a bunch of additional code to "rename" and "clone".

This is because in cases like renaming and cloning jails, the name is changed, and we need to again check if the character limit exceeds the max. If it does, and the old jail (rename or clone) does not, or if the new jail does not exceed but the old one does, we need to "sed" and replace the values in jail.conf and rc.conf.

Again, I now have this working locally, and will PR it if you desire.

Personally I would leave the naming schema as "epairX" as it greatly simplifies things, and is consistent with the way the VNET (non-bridged) interfaces are named.

But again, I'm not the only one using this great software so...

@2Belette
Copy link

Thanks for your testing and explanation.

Perhaps another approach would be to leave the default epair but your -n would use the jailname but would just send an error if the jail name is longer than 15 characters and also puting this into the documentation, so there is a tradeoff with the option but anyone who wants to use it just has to make jailname < 15 characters. This would solve all the sed and further manipulation for cloning and so on ?

We just need to make the command failed/error when -n is used and jailname > 15 characters. (but maybe I am over simplifying what needs to be done in the background?)

@tschettervictor
Copy link
Collaborator Author

But what about cloning a jail that was created with -n, and also renaming it?

@2Belette
Copy link

In fact I don't see how this could be solved, apart from failing the command in the first place + explaining the limitation in the documentation so everyone is aware...

This would probably not be acceptable if it would be a default option, but as this has to be used with -n

But, yes this would not solve a potential issue where a user does that and forget when changing the name or cloning long time after the creation

@tschettervictor
Copy link
Collaborator Author

tschettervictor commented Jan 18, 2025

Like I said, I have it working locally, and if we are still wanting it, I will open a PR.

It's automated, and no option selection required.

It's not as bad as you think. The only issue is that we are always dealing with the actual epairs as well as their name.

So for each interface we need to grep for the number, and then sed each line individually.

@2Belette
Copy link

My vote is yes !

@bmac2
Copy link
Collaborator

bmac2 commented Jan 18, 2025

are we making something that is simply a document the length limit into a complicated solution. This seems a lot of code to maintain and a complicated solution to a simple tell people don't make huge long jailnames.

@michael-o @yaazkal @JRGTH What do you guys think??? Am I being a simpleton on this, or is this overkill???

@michael-o
Copy link
Contributor

are we making something that is simply a document the length limit into a complicated solution. This seems a lot of code to maintain and a complicated solution to a simple tell people don't make huge long jailnames.

@michael-o @yaazkal @JRGTH What do you guys think??? Am I being a simpleton on this, or is this overkill???

No opinion

@tschettervictor
Copy link
Collaborator Author

are we making something that is simply a document the length limit into a complicated solution. This seems a lot of code to maintain and a complicated solution to a simple tell people don't make huge long jailnames.

@michael-o @yaazkal @JRGTH What do you guys think??? Am I being a simpleton on this, or is this overkill???

It's actually not a lot of code. It really just "if name is longer, then use epair as the name"

@tschettervictor
Copy link
Collaborator Author

@2Belette can you please test #820 for me? I'll post instructions shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants