Troubleshooting SSL/TLS Connections
As discussed in the previous section, it is important to configure your explicit or transparent web proxy to bypass SSL/TLS decryption for agent communications. If the bypass is not configured, these proxies might attempt to decrypt
SSL/TLS traffic by sending its own certificate to the agent. Because the agent only uses its local CA to validate the certificate, these proxy certificates will cause connection failures.
Symptoms include agent failing to register to the cluster, agent not checking-in, agent not sending flows, and/or agent not receiving enforcement configuration (if enforcement is enabled).
|
Troubleshooting steps below are assuming default installation paths were used. Windows: C:Program FilesCisco Tetration Linux: /usr/local/tet. If you installed your agents in a different location, substitute that location in the instructions. |
SSL/TLS Connection issues are reported in the agent logs. To verify if there are SSL errors in the logs, run the following commands for the associated issue being observed.
Registration, check-in
Linux
grep "NSS error" /usr/local/tet/log/check_conf_update.log
Windows (PowerShell)
get-content "C:\Program Files\Cisco Tetration\logs\TetUpdate.exe.log" | select-
˓→string -pattern "curl failed SSL peer certificate"
Flows
Most of the SSL/TLS connection issues seen are during the initial connection and registration of the agent. Sending flows relies on the registration to be complete before attempting to connect. SSL/TLS errors seen here would be the result of the sensorVIP IPs being allowed but not the collector IPs.
Linux
grep "SSL connect error" /usr/local/tet/log/tet-sensor.log
Windows (PowerShell)
get-content "C:\Program Files\Cisco Tetration\logs\WindowsSensor*.log" | select-
˓→string -pattern "Certificate verification error"
Enforcement
Linux
grep "Unable to validate the signing cert" /usr/local/tet/log/tet-enforcer.log
Windows (PowerShell)
get-content "C:\Program Files\Cisco Tetration\logs\WindowsSensor*.log" | select-
˓→string -pattern "Handshake failed"
If an SSL error is seen in the log checks above you can verify what certificate is being sent to the Agents with the following commands.
Explicit Proxy - where a proxy is configured in user.cfg
Linux
curl -v -x http://<proxy_address>:<port> https://<sensorVIP>:443
Windows (PowerShell)
cd "C:\Program Files\Cisco Tetration"
.\curl.exe -kv -x http://<proxy_address>:<port> https://<sensorVIP>:443
Transparent Proxy - No user.cfg proxy configuration required. It’s a proxy configured between all HTTP(S) traffic from agent to the internet.
Linux
openssl s_client -connect <sensorVIP from TaaS Portal>:443 -CAfile /usr/local/tet/
˓→cert/ca.cert
Windows (PowerShell)
cd C:\Program Files\Cisco Tetration
.\openssl.exe s_client -connect <sensorVIP from TaaS Portal>:443 -CAfile cert\ca.cert
You are looking for the following in the openssl s_client respose
Verify return code: 0 (ok)
If you see an error, examine the certificate. An example certificate (chain) should include only the following cert (CN IP is an example):
Certificate chain
0 s:/C=US/ST=CA/L=San Jose/O=Cisco Systems, Inc./OU=Tetration, Insieme BU/CN=129.146.
˓→155.109
i:/C=US/ST=CA/L=San Jose/O=Cisco Systems, Inc./OU=Tetration Analytics/CN=Customer CA
If you see additional certificates, then there is possibly a Web decrypting proxy between the agent and Secure Workload. Contact your security or network group and verify if the proxy bypass is configured using the listed IPs from the Configuring IP traffic for Agent Communications section.
Windows sensor installation script fails on Windows 2016 servers: Error message that might appear “The underlying connection was closed: An unexpected error occurred on a receive.” Possible reason might be the SSL/TLS versions set in PowerShell.
To check the SSL/TLS versions running, run the following command:
[Net.ServicePointManager]::SecurityProtocol
If the output from the above command is:
Ssl3, Tls
Then use the below command to change the allowed protocols and retry the installation:
[Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Ssl3,
˓→Tls,Tls11,Tls12'