Routed Static Block With PPPoE & OPNsense

ByJohn Wuethrich

Routed Static Block With PPPoE & OPNsense

Routed Static Block With PPPoE & OPNsense 1
Yes motherland non soviat bandograph meter u in skew

If you are just getting started with Opnsense or first time with a static block This post is for you! When trying to figure this out a lot of form posts said it couldn’t be done this way and to use 1:1 NAT; so maybe even those around the block a few times will benefit. 

This post is a somewhat how-to for setting up a static block with routing on opnsense over pppoe with a modem in transparent bridge mode.(vdsl2 & centurylink) This Assumes you want to end up with a NATed DHCP Private LAN and a DHCP Public ipv4 routed LAN. Buckle in!

This site is actually served over the configuration I will discuss below. Opnsense is running as a virtual machine and the web server lives in the same physical box. believe it or not you pulled this from my 80/40 VDSL 2 connection.  Im going to cover what im actually running this on as its a oddball/ one of the more complex setups ive which was nesicateted by lack of hardware. Odd how you can own something but not touch it…. but we wont go into that :D. 

First lets define the desired End Result and whats needed to follow along:

Tutorial

its not my intent to cover basic install but you can find those instructions here

https://docs.opnsense.org/manual/install.html

 

NICs/Vlans/Interfaces (oh my) 

Desired Config:

  • WAN Interface
  • Public/Server Lan Subnet Routed
    • DHCP For your block (optional)
  • NATed LAN Interface

Prerequisites

  • Opnsense 19.x.x (might work on earlier releases but ymmv)
  • PPP DSL (Modem in transparent Bridge Mode / PPPoE)
  • Modem/Account Credentials For PPP/PPPOE
  • Two or Three NICS
    • A two plus port NIC is fine as well.
    • I do not recomend anything short of physical function pass through if running in a hyper visor.  
    • If everything plays nice VLANs can make two NIC ports or even one all one needs. YMMV with your drivers/hardware. 
    • Im using intel x540 t-2 blah blah blah and ixgbe plus ixgbevf. 
  • A block of Static IPs from your ISP
  • Patience ( or hair to pull)

    Maybe my google foo is sub par these days but when I first set out to do this every post/result I found was wrong or saying “I’ve never seen that work”. I’ve had it working in a few permutations now. Due to a lot of drama im working with minimial hardware avalible at the moment. my configuration reflects that…. It might just be easier to describe that first:

Obtaining PPP/PPPoE credentials 

Call support or Snoop at whats running in modem?

Find your providers Customer Service number

Request your PPP/PPPoE credentials. 

My experience with Century Link:

preface: when I got the right person on the phone, they imediately no questions asked gave me the credentials. I was calling for another reason having already snooped my modem (and at last retail job in double NAT having done the same lol) that said i wanted to know if they would give the info out so i asked on a unrelated call. YMMV but I was given the creds first ask with the right department.  

Im decently impressed with the actual service from CL… As in the performance of the pipe.

That said CL is kinda like if ma bell was hacked into pices and kids grew from her severed limbs then hacked apart again and multiple frankenstines were created and deaimated in the name of service… or er.. something like that but franado frankinstido...monopoly  this ma bell …has alzheimer’s. At least with customer service. 

what do i mean? with the different mergers and splits CL has ended up with DHCP in some regions PPPoE in others. Talking with different customerserservice reps and or sales ive gleamed there are several different billing systems possibly from different decades as well. With a biz account they literlay will not give you access to everything you pay for with out scheduling an “On boarding call” and none of the support short of this will know why your mycentruylink login doesnt work if you do as i do and blow this off. 

in short, some of the reps you will encounter are great and knowledgeable people. dont count on ever talking to the same one again. One night i ended up talking to support in an office in poland other times its been in the midwest and once in Arizona. in shorter… imho: calling centurylink for anything is like pulling teeth over the course of 6hrs per tooth with out Novocain 

that said: you will need the username@domain.tld and pass that your modem is currently using to get you online. one route to obtain this is calling centurylink (or your providers) customer service. 

tell them you are putting your modem in bridge mode and need the PPPoE user and Pass, be ready to write it down or type it and somewhere other than the credentials field in opnsense… i recommend google keep or docs, maybe an actual sticky note.

 

 

There is an alternative route to obtain these creds for most of you. This is shortened and adapted from a tutorial at Inaudible Discussion. it was written with a Zyxel C1000Z in mind but I found substituting ssh for telnet and zyxel for actionteck c3000a worked just fine. 

The jist: 

  •  ssh or telnet into your modem. Id go ssh. 
  • in my case the sticker on physical modem had the admin pass. the same for web config worked via ssh and admin for user.
  • I had a weird glitch where putty or bitvise would die if i resized before typing sh. something about the dumbed down shell not liking the info that the remote user resized… so once it connects id suggest sh  and enter  key to drop to ash shell. From there on in i could resize the window with out the connection dropping
  • type ps or top command.
  • look for something like “pppd-cppp0.1-D0-iptm0.1-u” 
  • record the PID (should be just numbers no alpha or characters)
  • type : cat proc/PID/cmdline
    • should return something like
      • pppd-cppp0.1-D0-iptm0.1-u”user@centurylink.net”-p”PASS”-f0-k-P”PASS”#
    • or possibly (according to tutorial) your pass is encoded and you will see something like this:
      • pppd-cppp0.1-iptm0.1-u”myusername@qwest.net”-p-jlFrVNtRMtU=-f0-D0-n1-L0-X120
      • if this is the case your pass is encoded in base 64 and spans everything after -p to the next – or in this case jlFrVNtRMtU=
      • google base 64 decoder to get back to what opnsense input requires

see here for full tutorial 

Interface Assignment

long and short is you need 3. this can be one physical NIC and 3 vlans, 3 physical NICS or mix and match.

you need the appearance/function of a NIC for:

  • Wan (Modem)
  • Lan (Public)
  • Lan (Nat)
Routed Static Block With PPPoE & OPNsense 2
how I have my interfaces set up

if the modem can be set to do pppoe on a vlan  you might even reuse its switch ports (i did until just today). my c3000a has a qirk where it only does the 2xx vlan when set to not connect on vlan you might have to play with yours. aka dont trust Ceuntrylink firmware to do what it says. 

im from this point assuming you set up your 3 interfaces/vlans/combo of via ssh and the “assign interfaces” while doing the setup. If not log back in via ssh and its easy to do again. 

now you should have a note pad with your modem credentials and nic assignments (or a better memory than i do … what were we talking about?) 

On a side note, if you are still using putty id highly suggest you check out bitvise ssh client  I still keep putty installed but its rare i use it these days. 

WAN Config

assuming you went with normal settings your box can be reached via browser at 192,168.1.1 Login and : 

  1. Navigate on the left nav menu to interfaces – > Assignments
  2. find the nic that connects to your modem (in bridge mode).
    1. if its already set (in the drop down box) for “wan” click the interface name “wan”
      1. if not and you have a wan interface select the correct interface (or vlan) from the drop menu
      2. click save 
    2. repeat from step 2
  3. if the interface is already enabled skip this step. else
    1. click Enable interface
    2. if you end up on assignments screen click “WAN” again
    3. Procede to next
  4. Check “block private”
  5. check “block bogon”
  6. under “ IPv4 Configuration Type” select PPPoE
  7. The only fields left to concern yourself with are
    1. Username
    2. Password
    3. we will come back to ipv6 in a min
  8. click save
  9. go back to assignments

Public Lan Config

(your /29 or other block)


From the “Interfaces -> Assignments” screen:

  1. click “opt1” 
  2. click enable 
  3. click save
  4. click “opt1”
  5. under “ IPv4 Configuration Type” select “DHCP”
  6. if you are a CentLack subscriber they indicated probably the top of your ip block is the gateway. for instance xxx.xxx.xxx.112 i recomend you make this interface ip xxx.xxx.xxx.111 /29
  7. click save
  8. reboot the router (because why not)
  9. you might find u have internet when it comes back up (nat)
  10. Ill finish this soon

Firewall Config (NAT)

first we will get rid of auto rules and then replace them with the same rules plus two more. feel free to harden this config later. this is just a basic wide open it will work presented as is. 

Switch to manual outbound:

  1.  Click Firewall -> NAT -> Outbound nat (on left hand menu)
  2. Select  “Manual outbound NAT rule generation
    (no automatic rules are being generated)”
  3. click save

Now we need rules

  1. assuming you are here —
    Firewall -> NAT -> Outbound nat (on left hand menu)
  2. click add (top right)
  3. Interface “WAN”
  4. TCP/IP Version : IPv4
  5. Proto: any
  6. source address: LAN net
  7. Translation/Target : WAN address
  8. Save
  9. duplicate this rule (click add again )
    1. change “Translation/port” to 500
    2. check “static-port”

this just replaced what went away when we went to manual. Now we need to exclude the public lan from NAT

  1. same config page 
    Firewall -> NAT -> Outbound nat (on left hand menu)

  2. click “add”
    •  The spread
      1. check “ Do not NAT”
      2. Interface “WAN”
      3. IPV4
      4. Proto: any
      5. Source address: PUBLANnet
      6. Dst: Any (port too)
      7. Translation Target: Interface address
  3. click save

Firewall Config (Rules)

make sure each interface has at least one rule in each direction. with 0 rules its effectively disabled. 

ill make this look pretty and finish this part soon

Routed Static Block With PPPoE & OPNsense 3
Routed Static Block With PPPoE & OPNsense 4

My First working Setup

I learned a bit to late my parenst have issues. I spent 8 months bleeding wondering wtf was happening in my last apt. lost the lease/couldnt renew and parents invite me back. around that time we figured out it was mold in their basement. thus began 2 years and counting of gaslighting. they paid to fix their house a year ago and are using the police (and lies to) to keep basicly everything that was ever mine. 

which is why this newish server is on the floor and the setup is so wack ($50 linksys playing wifi ap)

 

I was eventually able to get the centurylink modem to provide pppoe and act as a switch. This simplified it quite a bit and boosted speeds. As seen above half of the x540 is connected to the 100 megabit switch on the cheap linksys. When I was able to make the c3000a play switch this linksys just became wifi ap. I couldnt get wifi to reliably stay on in bridge mode. I am sure i can get it on via gui or more sure via ssh and i havent tried very hard to be honest… this was just quicker. 

 

The complexity came from wanting the c3000a modem to be a vlaned off switch or something like it because i didnt have my managed vlan enabled (802.11q ) gigabit switch. 

  • Hypervisor/DA BOX
    • Intel Xeon e5 2660v3
      • 32GB ECC DDR4
      • Nvidia 1050ti 4GB
      • Intel X540-t2 (technically i have 2 of these but its dual socket and I only have one socket populated…so the onboard one is useless unless someone in internet land wants to toss me… i can dream right 😀 )
    • Intel Clearlinux
      • Compatibility Module Off in Firmware
      • intel-iommu=on added to boot string
      • some arbitrary number of virtual functions in a pool for each physical port. pools rather…
        • ixgbe and ixgbevf modules @ play
    • ActionTech c3000a
      • Bridge Mode on vlan….VLAN201
        • well i think its CLs firmware but this particular box does vlan pppoe when its set not to and doesnt when it does… ymmv. its set to no vlan but i connect on 201. why? I needed the switch lol. It might work with out the vlan at all Ill try to clear this up soon. If it doesnt you wont be reading this till it does again 😀
      • Ethernet cord to laptop NIC hit 192.168.0.1
        • set ports 1 to 4 to their own subnet/private bridge
        • WAN  port (5) is on main subnet
          • Centurylink your firmware is shit
          • remember i needed the switch
            • I own two gigabit managed 8 ports but why i dont possess them is …. lets not go there.
      •  x540 NIC to Modem
          • 1 physical port into WAN on c3000a
          • The other into p1 on c3000a
          • The subneting in modem config might make the vlan irrelevant/redundant/extra shit in the frame but this setup has been rock solid since i figured it out
          • im doing SR-IOV in kvm pools for each physical port/function
  • KVM Guests
    • Router ( guess )
    • Webserver
    • Cam Server
    • PBX
    • Kitchen Sink Serv….err
  • All VFs on the physical port into wan port on modem is WAN
    • PPPoE on ixv0 vlan 201 (physical function 0 -> Modem WAN)
    • ixv0 vlan 40 public routed subnet pub lan/ server lan /dmz
      • take ur pick
      • x540 is not a switch so the wan on modem port plays one 
      • Hey camera guy…u getting this?
    • Everything on PF-1 is NAT LAN Land (Private Subnet)
    • I highly recomend another switch instead of this 

 

About the author

John Wuethrich administrator

Leave a Reply

avatar
  Subscribe  
Notify of
%d bloggers like this: