Greetings,
I'm trying to slowly migrate away from an existing edgerouter setup to a mikrotik one, and have point to point 0/0, ::0/0 /31 tunnels established just fine between the existing network running ospf for link state, with bgp connecting loopbacks.
When I added the mikrotik in, the peers try to establish, but never complete and I'm trying to figure out which side has the ospf bug here (leaning towards mikrotik here since I've made this work with other devices as well like ciscos / opnsense). I'd look at the packet exchange, but trying to get a clean packet dump from an encrypted tunnel on the wan side isn't exactly easy.
From what I see, the nodes see each other's hellos, and begin exchange, but the mikrotik side never sees the DD reply from the far side so it just continually fails to establish and times out.
What am I missing here, or did I find a bug in how mikrotik (or the edgerouters) are adhering to the OSPF RFCs?
Relevant log bits from ospf debug mikrotik side:
fw1csc2 mikrotik side below
fw1csc1 - er-x side - tried to get the relevant pieces, let me know if I missed something.
Basic diagram
I'm trying to slowly migrate away from an existing edgerouter setup to a mikrotik one, and have point to point 0/0, ::0/0 /31 tunnels established just fine between the existing network running ospf for link state, with bgp connecting loopbacks.
When I added the mikrotik in, the peers try to establish, but never complete and I'm trying to figure out which side has the ospf bug here (leaning towards mikrotik here since I've made this work with other devices as well like ciscos / opnsense). I'd look at the packet exchange, but trying to get a clean packet dump from an encrypted tunnel on the wan side isn't exactly easy.
From what I see, the nodes see each other's hellos, and begin exchange, but the mikrotik side never sees the DD reply from the far side so it just continually fails to establish and times out.
What am I missing here, or did I find a bug in how mikrotik (or the edgerouters) are adhering to the OSPF RFCs?
Relevant log bits from ospf debug mikrotik side:
Code:
09:47:17 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } receive DD from 192.168.17.3 Init Master More sequence 320209:47:17 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } send DD to 192.168.17.3 sequence 320209:47:22 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } send hello09:47:22 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } receive DD from 192.168.17.3 Init Master More sequence 320209:47:22 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } send DD to 192.168.17.3 sequence 320209:47:25 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } hello09:47:27 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } receive DD from 192.168.17.3 Init Master More sequence 320209:47:27 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } send DD to 192.168.17.3 sequence 320209:47:32 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } send hello09:47:32 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } receive DD from 192.168.17.3 Init Master More sequence 320209:47:32 route,ospf,packet ospf_v2 { version: 2 router-id: 192.168.50.1 } backbone_v4 { 0.0.0.0 } interface { p2p 192.168.17.2%wg1 } neighbor { router-id: 192.168.50.2 state: Exchange } send DD to 192.168.17.3 sequence 3202
fw1csc2 mikrotik side below
Code:
# model = RB5009UG+S+# serial number = [snip]/interface bridgeadd admin-mac=[snip] auto-mac=no comment=defconf name=bridgeadd name=lo1 protocol-mode=none/interface ethernetset [ find default-name=ether1 ] comment=WAN/interface wireguardadd comment=fw1csc1 listen-port=51821 mtu=1420 name=wg1/interface vlanadd interface=bridge name=vlan2 vlan-id=2/interface ethernet switch portset 7 mirror-egress=yes mirror-ingress=yes mirror-ingress-target=ether3/interface ethernet switchset 0 mirror-egress-target=ether3/interface listadd comment=defconf name=WANadd comment=defconf name=LANadd name=WG_tunadd include=LAN,WG_tun name=NoBlock/interface wireless security-profilesset [ find default=yes ] supplicant-identity=MikroTik/ip pooladd name=dhcp ranges=192.168.10.50-192.168.10.254add name=dhcp_prod ranges=192.168.11.50-192.168.11.250/ip dhcp-serveradd address-pool=dhcp interface=bridge lease-time=23h59m59s name=defconfadd address-pool=dhcp_prod interface=vlan2 lease-time=23h59m59s name=dhcp_prod/queue typeadd kind=fq-codel name=FQ_Codel/queue interfaceset ether1 queue=FQ_Codel/routing ospf instanceadd disabled=no name=ospf_v2add disabled=yes name=ospf_v3 version=3/routing ospf areaadd disabled=no instance=ospf_v2 name=backbone_v4add disabled=no instance=ospf_v3 name=backbone_v6/routing bgp templateset default as=65510 disabled=no output.no-client-to-client-reflection=yes routing-table=mainadd as=65010 disabled=no multihop=yes name=peer nexthop-choice=propagate output.keep-sent-attributes=yes .no-client-to-client-reflection=yes routing-table=main templates=\ default/certificate settingsset crl-download=yes crl-use=yes/interface bridge portadd bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether2add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether3 pvid=5add bridge=bridge comment=defconf ingress-filtering=no interface=ether4 trusted=yesadd bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether5add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether6 pvid=2add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether7add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether8add bridge=bridge comment=defconf interface=sfp-sfpplus1/ip neighbor discovery-settingsset discover-interface-list=LAN/ip settingsset tcp-syncookies=yes/ipv6 settingsset accept-router-advertisements=yes/interface bridge vlanadd bridge=bridge tagged=ether4,ether7 untagged=vlan2 vlan-ids=2/interface list memberadd comment=defconf interface=bridge list=LANadd comment=defconf interface=ether1 list=WANadd interface=lo1 list=LANadd interface=vlan2 list=LANadd interface=wg1 list=WG_tun/interface wireguard peersadd allowed-address=0.0.0.0/0 comment=fw1csc1 endpoint-address=fw1csc.[snip] endpoint-port=51820 interface=wg1 persistent-keepalive=25s public-key=\ "[snip]"/ip addressadd address=192.168.10.1/24 comment=defconf interface=bridge network=192.168.10.0add address=192.168.50.1 interface=lo1 network=192.168.50.1add address=192.168.11.1/24 comment=prod interface=vlan2 network=192.168.11.0add address=192.168.17.2/31 comment=fw1csc1 interface=wg1 network=192.168.17.2/ip cloudset ddns-enabled=yes/ip dhcp-clientadd comment=defconf interface=ether1 use-peer-dns=no/ip dhcp-server networkadd address=192.168.10.0/24 comment=defconf dns-server=192.168.10.1 domain=[snip] gateway=192.168.10.1 netmask=24add address=192.168.11.0/24 comment=prod dns-server=192.168.11.1 domain=[snip] gateway=192.168.11.1/ip dnsset allow-remote-requests=yes servers=8.8.8.8/ip dns staticadd address=192.168.10.1 comment=defconf name=router.lan/ip firewall filteradd action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untrackedadd action=accept chain=input comment=vpn_local_in in-interface-list=WG_tunadd action=accept chain=input comment="wireguard 1" dst-port=51821 protocol=udpadd action=drop chain=input comment="defconf: drop invalid" connection-state=invalidadd action=accept chain=input comment="defconf: accept ICMP" protocol=icmpadd action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!NoBlockadd action=accept chain=forward comment="defconf: accept in ipsec policy" disabled=yes ipsec-policy=in,ipsecadd action=accept chain=forward comment="defconf: accept out ipsec policy" disabled=yes ipsec-policy=out,ipsecadd action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related hw-offload=yesadd action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untrackedadd action=drop chain=forward comment="defconf: drop invalid" connection-state=invalidadd action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN/ip firewall natadd action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WANadd action=dst-nat chain=dstnat comment=dns dst-port=53 in-interface-list=WAN protocol=udp to-addresses=192.168.10.15 to-ports=53/ip serviceset telnet disabled=yesset ftp disabled=yesset www-ssl disabled=no/ip sshset host-key-size=4096 strong-crypto=yes/ipv6 addressadd address=::7a9a:18ff:feab:79ae eui-64=yes from-pool=global interface=bridgeadd eui-64=yes from-pool=global interface=*Eadd address=::874:bff:fed3:e88d eui-64=yes from-pool=global interface=lo1/ipv6 dhcp-clientadd interface=ether1 pool-name=global prefix-hint=::/48 request=address,prefix use-peer-dns=no/ipv6 firewall address-listadd address=::/128 comment="defconf: unspecified address" list=bad_ipv6add address=::1/128 comment="defconf: lo" list=bad_ipv6add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6add address=100::/64 comment="defconf: discard only " list=bad_ipv6add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6/ipv6 firewall filteradd action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untrackedadd action=drop chain=input comment="defconf: drop invalid" connection-state=invalidadd action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6add action=accept chain=input comment="defconf: accept UDP traceroute" port=33434-33534 protocol=udpadd action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp src-address=fe80::/10add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udpadd action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ahadd action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-espadd action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsecadd action=drop chain=input comment="defconf: drop everything else not coming from LAN" in-interface-list=!LANadd action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untrackedadd action=drop chain=forward comment="defconf: drop invalid" connection-state=invalidadd action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6add action=accept chain=forward comment="defconf: accept HIP" protocol=139add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udpadd action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ahadd action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-espadd action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsecadd action=drop chain=forward comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN/ipv6 ndadd advertise-dns=no hop-limit=64 interface=ether1 ra-preference=low reachable-time=5m/routing bfd configurationadd disabled=yes interfaces=WG_tun/routing bgp connectionadd as=65010 connect=yes disabled=no listen=yes local.address=192.168.50.1 .role=ebgp-peer multihop=yes name=fw1csc1 nexthop-choice=propagate output.keep-sent-attributes=yes \ .no-client-to-client-reflection=yes remote.address=192.168.50.3 .as=65030 routing-table=main templates=peer/routing ospf interface-templateadd area=backbone_v4 cost=10 disabled=no interfaces=lo1 passiveadd area=backbone_v6 cost=10 disabled=no interfaces=lo1 passiveadd area=backbone_v4 cost=50 disabled=no interfaces=WG_tun priority=1 type=ptp use-bfd=noadd area=backbone_v6 cost=50 disabled=no interfaces=WG_tun type=ptp use-bfd=no/system clockset time-zone-name=America/Chicago/system identityset name=fw1.[snip]/system loggingadd disabled=yes topics=ospf/system noteset show-at-login=no/system ntp clientset enabled=yes/system ntp serverset enabled=yes multicast=yes use-local-clock=yes/system routerboard settings# Firmware upgraded successfully, please reboot for changes to take effect!set auto-upgrade=yes/tool mac-serverset allowed-interface-list=LAN/tool mac-server mac-winboxset allowed-interface-list=LAN/tool romonset enabled=yes/tool romon portadd cost=200 disabled=no forbid=yes interface=ether1
fw1csc1 - er-x side - tried to get the relevant pieces, let me know if I missed something.
Code:
firewall { broadcast-ping disable group { port-group Wireguard { description "" port 51820-51822 } } name VPN_IN { default-action accept rule 10 { action accept description Estab log disable protocol all state { established enable invalid disable new disable related enable } } rule 20 { action drop description invalid log disable protocol all state { established disable invalid enable new disable related disable } } rule 30 { action accept description http(s) destination { port 80,443 } log disable protocol tcp_udp state { established disable invalid disable new enable related disable } } rule 40 { action accept description icmp log disable protocol icmp } rule 50 { action accept description bgp destination { address 192.168.50.0/24 port bgp } log disable protocol tcp source { address 192.168.50.0/24 } state { established disable invalid disable new enable related disable } } rule 60 { action accept description ssh destination { port 22 } log disable protocol tcp state { established disable invalid disable new enable related disable } } rule 61 { action accept description proxmox destination { port 8006 } log enable protocol tcp_udp state { established disable invalid disable new enable related disable } } } name VPN_LOCAL { default-action accept rule 10 { action accept description Estab log disable protocol all state { established enable invalid disable new disable related enable } } rule 20 { action drop description invalid log disable protocol all state { established disable invalid enable new disable related disable } } rule 30 { action accept description ospf log disable protocol ospf } rule 40 { action accept description dns destination { port 53 } log disable protocol tcp_udp } rule 50 { action accept description bgp destination { address 192.168.50.1 port bgp } log disable protocol tcp source { address 192.168.50.0/24 } state { established disable invalid disable new enable related disable } } } name WAN_LOCAL { default-action drop description "WAN to router" rule 10 { action accept description "Allow established/related" state { established enable related enable } } rule 20 { action accept description WireGuard destination { group { port-group Wireguard } } log disable protocol udp } rule 21 { action accept description "backup https" destination { port 8443 } log enable protocol tcp_udp state { established disable invalid disable new enable related disable } } }}interfaces { ethernet eth0 { address dhcp description Internet dhcp-options { default-route update default-route-distance 210 name-server no-update } dhcpv6-pd { no-dns pd 0 { interface switch0 { host-address ::1 prefix-id :0 service slaac } interface switch0.2 { host-address ::1 prefix-id :1 service slaac } prefix-length /60 } rapid-commit enable } duplex auto firewall { in { ipv6-name WANv6_IN name WAN_IN } local { ipv6-name WANv6_LOCAL name WAN_LOCAL } } interface wireguard wg0 { address 192.168.17.3/31 address fe80::250:55ff:fec0/64 description fw1csc2 firewall { in { ipv6-name VPNv6_IN name VPN_IN } local { ipv6-name VPNv6_IN name VPN_LOCAL } } ip { ospf { cost 50 dead-interval 40 hello-interval 10 priority 1 retransmit-interval 5 transmit-delay 1 } } listen-port 51820 mtu 1420 peer [snip] { allowed-ips 0.0.0.0/0 allowed-ips ::0/0 endpoint [snip].sn.mynetname.net:51821 persistent-keepalive 25 } private-key /config/auth/wg.key route-allowed-ips false } loopback lo { address 192.168.50.1/32 }}protocols { ospf { area 0.0.0.0 { area-type { normal } network 192.168.17.0/24 network 192.168.50.0/24 network 192.168.10.0/24 } log-adjacency-changes { } parameters { abr-type cisco router-id 192.168.50.1 } passive-interface default passive-interface-exclude wg0 }}
Basic diagram
Statistics: Posted by ikiris — Sat Jan 20, 2024 6:03 pm