NAT Reflection Issue for a VM running on TrueNAS
-
I freely acknowledge this is most likely a PICNIC (problem in chair, not in computer) error, so I am now seeking clarity to resolve it.
GIVEN:
GCI cable modem & Internet service
pfSense as a gateway and DHCP server (LAN DHCP IP range x.x.x.1/24
On pfSense, I see the gateway IP address and the WAN IP address, which are different addresses.
TrueNAS Scale as NAS (IP address x.x.x.3)
Virtualized server running under TrueNAS Scale (IP x.x.x.206
ARK Survival Evolved server running on VM (IP also x.x.x.206)
ARK server showing the WAN IP address as the public address
Ports required for ARK servers:
UDP 7777-7800 - Client & Peer ports
UDP 27015-27025 -Query ports
TCP 32330-32341 - RCON portsTo make rule creation a bit easier, I made use of aliases for the ARK server (x.x.x.206) and each range of ports as listed above
I have created (now) a variety of NAT rules to supposedly accept traffic from anywhere to the WAN address and redirect to the local IP address of the VM/ARK server.Testing for open ports in any of the above ranges on the WAN IP address shows none are open. I hope someone can help me see where I am going off in the weeds with this. I had no issue porting out a Plex server to the outside world, but I am struggling to get this one working. Any thoughts, suggestions, or ideas would be most welcome. I can provide more detailed information if needed.
-
@SlackerDude
So you have primarily a port forwarding issue, not NAT reflection in first line. Or do you want to access the server from inside your LAN using the public IP?I suspect, that the server doesn't respond. Incoming access might work, since you can access your Plex server from outside.
To investigate, sniff the traffic on the LAN interface using Packet Capture. You can use a port checker to initiate some traffic, but remember that these services normally only send TCP, no UDP packets.
You should see incoming packets on the LAN. If there are no respond packets, however, you problem might be on the server. -
@SlackerDude I have found port scanning tools to be quite inconsistent.
For fun I just tested two different online tools:
https://dnschecker.org/port-scanner. Which shows 2 out of 5 scanned and open ports being open, and I know for certain that the services work.
https://pentest-tools.com/network-vulnerability-scanning/port-scanner-online-nmap Claimed 0 ports were open out of the 5 I just tested... -
@viragomann
Sorry for the late response. I am attaching a screenshot of the packet capture on the LAN interface. I used the following checker:
you get signal https://www.yougetsignal.com/tools/open-ports/ -
@SlackerDude
So at least on port 32330 the server is responding. And hence I assume that this port is shown as opened.Did you also test other ports? The capture doesn't show any other access attempts from this public IP.
If you also checked other ports and they are not shown in the capture, sniff on the WAN to see if the packets are reaching your pfSense at all.
You can enter the currently checked port in the port filter to avoid noise in the result. -
@viragomann I have tested ports 7777, 27015, and 32330 specifically because those are the ones I have not figured out how to forward correctly. I have a Plex server that uses port 32400 and it tests well and shows as open. Additionally, I have tested from the perspective of my local ISP, as well as from within a VPN in an attempt to make it appear as though access was originating outside my ISP network.
UPDATE:
I have finally been able to confirm that all ports are correctly forwarded, as evidenced by my being able to see my servers from a computer outside my network in the Steam client. After logging into one of them, I can also confirm I see the other servers/maps to travel to from an obelisk.
I still cannot see the other servers/maps when logged into the game from inside my network. At least I have confirmed that much, which brings me back to how to make server-to-server travel possible from inside my network. I'm not sure if that involves enabling loopback/hairpin functionality in pfSense, NAT reflection, or using a VPN connection to make computers inside the network appear as if they are located somewhere else in the world. I apologize for the confusion this rambling is probably causing, but I feel that I may be closing in on the actual issue now, though I am still at a loss on how to resolve it.
-
@SlackerDude
NAT reflection enables your internal devices to access local services by using the public IP.
This is only really useful if you need to use the public IP, but not an public domain.If you use a public FQDN to access the service, however, it would by more reliable to add a host override to your local DNS.
This requires that your involved devices use the local DNS for host name resolution.By default, pfSense has the DNS resolver enabled and distributes its interface IP via DHCP.
So it depends on your application and devices, what's the best practice here.
-
@viragomann I apologize for my apparent stupidity here. I will try to refine what I am attempting to accomplish. I want to be able to access my ARK Survival Evolved servers from within my home network as if I were outside my home network. I have proven to myself that accessing them from outside my home network allows me to see all three servers, and once logged into one of them with the game client, I can see the other two servers to travel to from within the game itself. I cannot see the other two servers when logged into the game from inside my home network
That is the only remaining issue, and I'm not sure if a NAT Reflection rule, or series of rules (properly constructed) will accomplish that, or if some other solution might be more applicable.
Here is a screenshot of why I think it may be a NAT Reflection issue
-
You find the settings for NAT reflection under a different heading than Firewall NAT...
Go to System > Advanced > Firewall & NAT and scroll down to the Network Address Translation section. There you select Pure NAT and tick the boxes for NAT reflection, and click Save and apply. -
@Gblenn I have done that, and have attempted to create rules to allow access:
Enable NAT Reflection:
List of NAT Rules:
Example of one NAT Rule:
That got me to the point where I could see and access the servers from outside my home network, and travel from one to another within the game. However, I need to know what I missed to make that same full functionality available from inside my home network. -
@SlackerDude Ok so all of that looks great, and since you can access the game from an external source, that shouldn't be the issue...
It does look a bit strange though that one can see the actual port numbers and IP's in the List of rules. Rather than the aliases you have created? Not that it matters as long as the information is correct..
One thing to note wrt aliases, as they are all the same type (UDP) you could create one single alias for all the ports that you use for that one server. Just click Add port when you edit the alias and enter the next port or range and so on...
I'm not sure how automatic this really is, but what if the outbound rules are only automatically created when the rule is created/edited, Then if you try editing each of those rules and click save and apply changes and see if that changes anything?
Then when the firewall rules have reloaded, try again... it really should work fine also from inside the network the way you have it set up.
BTW, another thought, are you access the ARK server using it's internal IP?
-
@Gblenn I goofed when I uploaded that image. that was a screenshot I took before creating the alias for the various ranges of ports. This is what looks like as of today:
I separated the various ranges to cover a possible total of 8 servers, with the game/peer ports, query ports, and RCON ports grouped for simplicity's sake:
I will try your suggestion and edit/save each of them and follow up with a forced reload by way of simply rebooting pfSense, just to keep things honest. I will share the results in a bit. Thanks! -
@Gblenn I did consolidate the Aliases:
and created a new NAT rule:
as well as a WAN rule:
and then rebooted pfSense. I ended up with the same result, unfortunately:
I honestly don't know what to try next. -
@SlackerDude Hmm, that is strange... so a few other ideas and checks then.
You mentioned that Plex has been working fine for you, so if you could try accessing Plex from inside your network, using a web browser (publicIP:32400).
That will tell you if NAT reflection is actually working or not. And if that doesn't work, try reloading the Plex rule as well by making an edit (or just save) and Apply.The picture you are showing seems like it's from a server management SW,
Have you tried starting up the game and entering your external IP to access your server that way? I'm thinking that the Server SW may have a bit more intelligence to it and could detect going back to itself.Also, do you really need, or want, to open the RCON ports? Not that it would make a difference here, but those are for server management aren't they?
-
@Gblenn Great question! I can access my Plex server from inside my home network only by a private IP address and can access it from a public IP only when I am outside my network, so I guess that indicates the NAT reflection is not working.
I do not require the RCON ports, as I access and administer the ARK servers via an RDP connection inside my home network, and so can eliminate them.
Okay, so it seems the thing I am missing is a working NAT reflection rule. I have tried various rules, but have so far failed to get a positive result.
-
@SlackerDude
You can try NAT reflection + proxy if pure NAT doesn't work.
Also consider that your firewall rules have to allow the access.If NAT reflection works properly, can be investigated by sniffing the traffic on the internal interface.
You should see request packets from your client to the public WAN IP. These have to be forwarded by pfSense, where the source is turned into the pfSense interface IP (LAN maybe) and the destination into the servers IP.
So the server should respond to pfSense and pfSense should forward responses to the client, while the source is turned into WAN IP. -
@viragomann said in NAT Reflection Issue for a VM running on TrueNAS:
@SlackerDude
You can try NAT reflection + proxy if pure NAT doesn't work.
Also consider that your firewall rules have to allow the access.Perhaps you know how this works, wrt the automatic creation of outbound rules? If you have a bunch of NAT rules already, and THEN tick the boxes in Setup > Advanced > Firewall & NAT. Will all of the rules then be rebuilt and the relevant outbound rules will be created? Or will that only apply to new rules created after those functions have been turned on?
I'm thinking the rules may need to be redone, or at least edited and saved for that to happen? I don't see any reason Pure NAT shouldn't work?>
-
@Gblenn
The rules are not visible in the GUI, as far as I know. But they should be listed in /tmp/rules.debug, I think.I had no success with pure NAT for simple HTTP traffic years ago, when the server and client were within the same network segment. But it worked with proxy though.
I know, the proxy mode is not necessary these days anymore and I've disabled it in my settings, but I'd still suggest to try it if there is no joy otherwise. -
@viragomann Exactly, it's not easy to troubleshoot when you can't even find the rules. I assume that when a rule is created, it will check if those options are set, and then create the outbound rule. But is the reverse also true? If you tick the boxes, is there a check to see if any rules exist for which outbound rules need to be created. And if I remove those options, will my outbound rules be removed?
There is a section in /tmp/rules.debug that starts out with:
'# Outbound NAT rules (automatic)
'# Subnets to NAT
table <tonatsubnets> {
etc.And I can see references to my NAT rules that I have in place, including Plex...
So I did some testing... and if I uncheck the boxes, the list changes only slightly, but I can verify that I am no longer able to access the server I tested towards, using my public IP (fqdn).
What's more, after turning reflective NAT back on, I had to reboot the server in order for it to be accessible again. I suppose I could kill all the states instead, but that's what I did.
So, @SlackerDude, try restarting your server(s) and see if you can access Plex using your external IP?
-
@Gblenn said in NAT Reflection Issue for a VM running on TrueNAS:
I assume that when a rule is created, it will check if those options are set, and then create the outbound rule. But is the reverse also true? If you tick the boxes, is there a check to see if any rules exist for which outbound rules need to be created. And if I remove those options, will my outbound rules be removed?
As far as I know, the rule set is rewritten every time you apply changes or a service like pfBlockerNG.
If you run "pfctl -sa" in the shell or on Diagnostics > Command Prompt you get the NAT rules only. This is probably more meaningful than tmp/rules.debug.