I am attempting to create a LAN with a separate subnet for the VMs that I'm creating on my KVM host (using eth0 for access to host and eth1 for VM access).
So separate IP addresses for each interface would seem the easiest option. You'll need a router somewhere along the network path to "join" them together.
One of the VM's will be offering DHCP and DNS services to the rest of the VMs.
DHCP will require foprwarding to cross the subnet(s) - DNS is routable.
My concern is how to get internet from my ISP router to the one that I'm creating for the VM's.
So long as the router "knows" - i.e.: has a route to the target, it'll work.
At first I was thinking of bridging and simply hooking up every VM to a bridge on the host's eth1.
Bridging is not routing.
So I figured I need some kind of gateway.
Yes you do.
I'm trying that at the moment but not having much luck getting the gateway VM to access either my regular router or the other VM I have set up right now.
Does the router have an interface on each network and does it have the required routes?
I looked into using VLAN's, but that seemed more for large-scale deployments, and I don't know how it would work with a Cobbler/Spacewalk/DHCP deployment (but if needed I can cross that bridge when I come to it)
Generally, VLANs are the responsibility of the network - i.e.: it should be transparent to the hosts. You can make each host "VLAN aware" but this is much, much harder.
using a VM as a router and/or gateway (and just NAT that particular VM?)
A router is not necessarily a NAT device.
When I pinged outside of my LAN (say 220.127.116.11) I could traceroute to my ISP's router, but nothing further from there.
This does not necessarily mean all is lost. For example, ICMP ping/traceroute or TCP "pingf"/"traceroute" could simply be blocked by the network you have no (or limited) control over.
So now my VM's can access the internet via the first NIC (say eth0) and access the internal virtnet via the second (say eth1). Without being able to diagnose the router/gateway issue, that is the best config that I can come up with.
So I'll make something up (lack of data).
Say I have a VM called VM1, with:
192.168.2.0/24 via 192.168.2.1/24
18.104.22.168/16 via 22.214.171.124/16
default (0.0.0.0) via 192.168.2.254 # anything you don't know about, send here!
I also have a router:
192.168.2.0/24 via 192.168.2.254/24
default via <my_isp_ip_address> # anything you don't know about, send here!
Additionally I need some NAT as I'm using private class C addresses for Internet access.
That means eth1 is internal only (can't get to the internet) and eth0 can get to the internet.
Hopefully that's given you something to think about.