Agent operations
Q: I have installed the agents successfully, but I didn’t see it on UI Sensor Monitoring page.
A: An agent is required to register with backend server running within cluster before it could start operating. When an agent is not shown on UI page, most likely it’s because the registration has failed. There are a few things we could check to see why a registration failed:
-
Check if the connection between the agent and the backend server is working properly
-
Check if the curl request could be sent to backend server properly
-
Check HAProxy access and backend server logs to see if the registration request made it to the server
-
Check the error return from curl request in the log file
Q: The agent is installed and I could find in on UI page. However, the “SW Ver” column shows “initializing” instead of a version string.
A: After the initial agent is installed and registered with the backend server, it would take another 30 minutes for the agent to report its version.
Q: The agent is upgraded properly, but the “SW Ver” fields still show the old version after a long time (like several hours).
A: After the agent is upgraded successfully, it will try to send a curl request to report its current running version and check for new version in the same request. It is possible that the request couldn’t make it to the backend, due to several reason:
-
The request is timed out, couldn’t get the response in time
-
The network is facing problem, agent couldn’t connect to backend servers
Q: I have an agent running on RHEL/CentOS-6.x and it is working properly. I am planning to upgrade the OS to RHEL/CentOS-7.x. Would the agent still work after the upgrade?
A: currently we do not support the scenario in which the OS has been upgraded, especially upgrading the major releases. In order to have the agent work after OS upgrade, do the following steps:
-
Uninstall the existing agent software
-
Clean up all files, including certs
-
Go to UI, delete the agent entry
-
Upgrade the OS to the desired version
-
Install the agent software on the new OS
Q: I have an agent running on RHEL/CentOS-6.x and it is working properly. I am planning to rename the host. Would the agent still work after rename/reboot?
A: An agent identity is calculated based on the host’s uniqueness, including hostname and bios-uuid. Changing hostname changes the host’s indentify. It is recommended to do the following:
-
Uninstall the existing agent software
-
Clean up all files, including certs
-
Go to UI, delete the old agent entry
-
Rename the host and reboot
-
Install the agent software again
Q: On Windows host, firewall deviation was caused by adding/deleting/modifying a rule. How do I find the rule?
A: On deviation detection, agent logs the last 15 seconds of firewall events to “C:\Windows\System32\config\systemprofile\AppData\Roaming\tet\firewall_events”. Rule that caused deviation will be found in the latest file created as policy_dev_<policy id>_<timestamp>.txt
Q: I have installed the agent on a Windows host successfully. Why do I not see any reported flows from the sensor?
A: Npcap is required to collect flows on a Windows host. Ten seconds after the agent is installed successfully, it will install Npcap. If the sensor does not report flows after several minutes, check if the agent and the backend server is connected and if Npcap is installed properly on the NPCAP Issues.
Q: I have installed the agent on Windows host, 2008 R2, successfully. Why does the system clock drift when tetsensor service is running?
A: This is a known problem with Go and Windows 2008 R2. For more information, see Golang and Win2008 R2.
The process, tet-main.exe, running as a part of tetsensor service, is built using Go Version 1.15. That is why the system clock drifts when the tetsensor service is running.
This issue occurs when Windows 2008 R2 workload is configured to use the external NTP server or Domain Controller as NTP server.
The possible work around :
-
Periodially force NTP to sync the clock: w32tm /resync /force
-
Disable tet-main.exe manually.
-
Run cmd.exe with “administrator” privilege.
-
Run regedit.exe
-
Go to “HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\TetSensor”
-
Double click on “ImagePath”
-
Edit value, remove tet-main.exe
before “C:\Program Files\Cisco Tetration\TetSenEngine.exe” TetSensor TetSen.exe “-f sensor_config” tet-main.exe ” ” TetUpdate.exe
after “C:\Program Files\Cisco Tetration\TetSenEngine.exe” TetSensor TetSen.exe “-f sensor_config” TetUpdate.exe
-
Restart tetsensor service
Disable tet-main.exe after every time agent is upgraded.
-
-
Remove external NTP server configuration:
-
Run command : w32tm /config /update /manualpeerlist: /syncfromflags:manual /reliable:yes
-
Restart Windows Time Service, W32Time
-
Q: Why does agent rehoming fail when the internal DNS does not resolve public domains or when hosts use a proxy for external domain resolution?
A: When the internal DNS does not resolve public domains or hosts use proxy for external domain resolution, agent rehoming fails because of Web Services Security (WSS) name resolution validation failure.
To resolve this issue, perform one of the following:
-
Use the Sensor Virtual IP (VIP) instead of the Sensor VIP FQDN during the agent rehoming configuration.
-
Update the hosts file on the workloads with the Sensor VIP and FQDN. You can find the hosts file at:
-
Windows:
C:\Windows\System32\drivers\etc\hosts
-
Linux/Unix:
/etc/hosts
-
-
Add a temporary DNS record for the Sensor VIP FQDN in the internal DNS server.
Q: Why do agents connected to SaaS clusters and running on Windows and Linux workloads communicate with the Instance Metadata Service (IMDS), at the IP address 169.254.169.254?
A: The Secure Workload SaaS cluster needs to determine whether the workload belongs to a cloud provider environment, such as Azure, AWS, GCP. The agent queries Instance Metadata Service (IMDS), at the IP address 169.254.169.254 to obtain this information. Only the agents connected to SaaS clusters and running on Windows and Linux workloads will communicate with the IMDS. Once installed, the agent attempts the query only once. In versions up to 3.10 (1.1), the agent attempts the query every time the agent starts.