A Networkmanager built with coreDNS and coreDHCP with web gui
Core Network Management is designed to be a modern, scalable, distributed, network managing tool that makes statically assigned devices a thing of the past. The web application manages CoreDNS and CoreDHCP instances, and allows the easy management of DHCP static-mappings, meaning a host will get the same IP every time it makes a new request. The system also has the ability to automatically generate forward and reverse records for hosts, as well as update these records based on the host’s current DHCP license. The system also is able to allow roaming between some networks, while rejecting DHCP requests from all but specifically designated hosts on others. The system is also designed to minimize latency in a geographically distributed environment, allowing networks and domains to be assigned to locations.
CoreDNS popular cloud-native open source DNS server, which is written in Golang and designed to be scalable and easily extended by plugins. In this case, the CoreDNS server is configured to proxy all domains it is authoritative for to the DNS-Backend via gRPC. Responses from the DNS-backend are also cached, as these lookups are expensive, so CoreDNS is also configured to use an in-memory cache. If the built-in in-memory cache proves to be too small for the system’s workload, CoreDNS can also be easily configured to use an external REDIS cache. All other requests are recursively looked up against a public DNS server.
CoreDHCP is an up-and-coming cloud-native open source DHCP server, which is written in Golang, and designed to be easily extensible. This project is still a work in progress, and has a far smaller community (and therefore a smaller array of plugins) then CoreDNS, but it is still the best solution for this use-case. However, completing this project may involve contributing to this project and it’s plugins, and or even writing some custom plugins from scratch. One example of why this may be necessary: the REDIS plugin, which is required for this use-case, does not support IPv6. The intended functionality of this DHCP server is that it hands out DHCP based on the current leases in the REDIS database.
This is a custom Golang application that acts as a gRPC endpoint, and is responsible for answering requests from the CoreDNS nodes by looking up the DNS record in the CockroachDB database. gRPC is a high-performance RPC(Remote Procedure Call) framework, initially developed by Google, and allows the easy generation of interface code via Google’s ProtoBuf tool.
CockroachDB was chosen as it is a very scalable database, is designed for not only high availability via replication and synchronization, but also for geographically disparate instances, allowing for a single logical database that spans multiple physical regions, where different tables are “located” primarily in specific regions, keeping latency and synchronization overhead lower. This is the SQL database that holds all hosts, networks, domains, users, and groups. This database is read from by the DNS-backend nodes, and read from and written to from the API nodes.
Redis is an in-memory NO-SQL database, which in this case holds the “live” MAC to IP mappings for the DHCP servers. Because the database is clusterable, these can also be spread out geographically, though it also is ideal to configure it such that each redis instance would primarily hold records for in networks at it’s given geographical location. The records in this database are synthesized from the records in CockroachDB, and kept in sync by the API nodes, as they keep the differential changes in sync for each change they make (and that is the only node that makes changes to the databases). These Redis instances can also be used as a cache for the CoreDNS nodes, if the in-node cache proves to be too small for the scale that the system is running at.
The frontend node, which is a Golang webapp, is the primary web interface for the whole system. This node relies on the API endpoint nodes to send and receive information about the current state of the system, but is designed to be stateless itself, so it is more scalable. Through the web interface, hosts can be added, and DNS records can be changed, users can be managed, and logs can be viewed.
The API node, which is also a Golang application, is RESTful JSON API, and it is the only node that does any writing to the databases, so it handles read requests from many other nodes, and write requests from the frontend nodes.
NGINX is used here as a load balancer for the API and Frontend services, though for smaller deployments this is not needed, as simple High Availability failover can be achieved with Docker Swarm or Kubernetes, eliminating some complexity, as one instance of the service should be able to handle a fair amount of traffic.
Core-NetworkManager is designed to be cloud-native, so it should be able to scale horizontally and vertically - since the majority of the microservices are stateless, they can easily be replicated, and scaled out that way, with docker swarm or kubernetes handling the internal application interactions with a service mesh, and a set of NGINX instances handling the publicly exposed web endpoints. For the DNS and DHCP endpoints, those can be scaled by directing traffic at different physical locations to different instances, at a network level. Internal services, such as the DNS backend, Redis, and CockroachDB, and be scaled out as needed, with internal failover and replication being handled by Docker Swarm or Kubernetes. The only large foreseeable issue with long-term use and scalability is that the internal CockroachDB is used to hold all event logs (DHCP success/failure, DNS changes, Host IP assignments, permissions changes, etc). This could lead to the CockroachDB growing at a steady rate over time, which is not sustainable. One solution might be to throw out logs of a certain age, but even old logs can be very useful. Likely the better solution would be to either add a seperate log collection and handling system (perhaps Logstash as the collector and ElasticSearch and the DB and handler), or to add an interface that would allow the logs to be piped to an external centralized logging server. Since this application is designed for large organizations, the second option is probably most reasonable, though picking any specific server would limit the potential user-base of such a system.
The application is managed via a web portal, and all network and domain changes are made through that web portal. The other services of the application, DNS and DHCP, are accessed as those services would typically be accessed - over the network. The application provides the most control and benefit if it is used at the exclusive DNS and DHCP service for the organization, as it can then serve as a central network management for all networks and domains. The permission model allows for users and groups to have granular permissions on specific hosts, networks, and domains, which can allow for the secure segmentation and delegation of network management and maintenance responsibilities. The application is designed to be run in an on-premises distributed docker swarm or kubernetes cluster, though all but the DHCP service can be moved to the cloud if absolutely necessary, though this will increase latency, and decrease security.