1. Sourcegraph Help Center
  2. Troubleshooting Guides
  3. Diagnosis

Triaging TLS/SSL Issues

Warren Gifford
Updated

Subject

This guide explores common debugging tools for identifying issues setting up TLS/SSL with HTTPS certs, and provisioning Sourcegraph a secure URL

 

Problem

Customers may run into TLS/SSL issues for a variety of reasons. Often admins run into these issues when first deploying Sourcegraph as they attempt to provision a new URL to the instance and configure HTTPS certs via our built in methods or their Cloud provider/internal devOps Infrastructure. More often than not we won't have access to their infrastructure but we can still help them identify the issues.

 

TLS/Docs and Standard Methods

General guidance on each deployment style can be found in our docs here: https://docs.sourcegraph.com/admin/http_https_configuration

Note: customers often have highly customized https cert provisioning, make sure to build context and ask about how and where customers have enabled https certs!


Instance specific diagnostic steps follow --

Kubernetes/Helm

In our Standard Kubernetes TLS certs are governed by an Ingress Controller, using the ingress-nginx standard. The following bullets are some things to check when diagnosing TLS/SSL issues in k8s deployments:

  • logs from the frontend service! kubectl -n namespace logs sourcegraph-frontend-0
  • Have a look at the sourcegraph-frontend.Ingress.yaml file found in deploy-sourcegraph
  • Have a look at the mandatory.yaml found in deploy-sourcegraph
  • Get the describe outputs for the frontend service, and the sourcegraph frontend Ex: kubectl -n <namespace> describe ingress sourcegraph-frontend
  • Check if the SSL certificate is valid and has not expired. You can use the openssl command to check the SSL certificate:
    openssl x509 -in <certificate-file> -text -noout
  • It is also possible that the certificate file is not located in the expected path or there is a permission issue preventing the `ingress-nginx-controller` pod from accessing the certificate file.

This general ingress-nginx debug doc may be helpful

Docker-Compose

Docker compose instances by default provides TLS via builtin cert generation from Caddy:

  • Have a look at the caddy portion of their docker-compose.yaml file from deploy-sourcegraph-docker
  • Check logs from caddy and sourcegraph-frontend. docker logs caddy
  • Enter the caddy container and use the caddy environ command to check the setup docker exec -it caddy sh -c 'caddy environ'
  • Inside the caddy container certs can be manually inspected in /caddy-storage/data/caddy

Single Container

Single container Sourcegraph instances use Nginx to provide https certs. Largely this information and configuration is stored in the automatically generate .sourcegraph directory:

  • See if you can get a look at the customers nginc.conf file found inside the container at etc/sourcegraph/nginx.conf  or on the host machine at ~/.sourcegraph/config/nginx.conf
  • Find out the flags used by the admin for their docker run initialization of the instance

 

Helm

Single container Sourcegraph instances use Nginx to provide https certs. Largely this information and configuration is stored in the automatically generate .sourcegraph directory:

  • See if you can get a look at the customers nginc.conf file found inside the container at etc/sourcegraph/nginx.conf  or on the host machine at ~/.sourcegraph/config/nginx.conf
  • Find out the flags used by the admin for their docker run initialization of the instance

 

General Cert and Domain Debugging

If a customer has provisioned an IP URL and HTTPS cert, and its still encountering issues accessing their Sourcegraph instance the following commands may be helpful in identifying the issue:

  • curl the instance with the -i flag, Ex:  
    # curl -i cse-aws-test.sgdev.org
    HTTP/1.1 308 Permanent Redirect
    Connection: close
    Location: https://cse-aws-test.sgdev.org/
    Server: Caddy
    Date: Fri, 30 Dec 2022 03:40:26 GMT
    Content-Length: 0
  • Use the dig or host command to verify DNS to IP association on the host machine 
    # dig cse-aws-test.sgdev.org

    ; <<>> DiG 9.10.6 <<>> cse-aws-test.sgdev.org
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58804
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 512
    ;; QUESTION SECTION:
    ;cse-aws-test.sgdev.org. IN A

    ;; ANSWER SECTION:
    cse-aws-test.sgdev.org. 300 IN A 3.133.49.142

    ;; Query time: 195 msec
    ;; SERVER: 2001:558:feed::1#53(2001:558:feed::1)
    ;; WHEN: Thu Dec 29 19:47:50 PST 2022
    ;; MSG SIZE rcvd: 67

    # host cse-aws-test.sgdev.org
    cse-aws-test.sgdev.org has address 3.133.49.14
  • You can manually inspect a cert using the openssl tool to verify it is provisioned for the correct domain name. The following example is for our cse-aws cert generated by caddy: 
    λ ~/ openssl x509 -in cse-aws.crt -noout -text
    Certificate:
    Data:
    Version: 3 (0x2)
    Serial Number:
    04:15:31:64:14:b5:c5:e6:da:6d:28:be:cd:29:62:87:5e:01
    Signature Algorithm: sha256WithRSAEncryption
    Issuer: C=US, O=Let's Encrypt, CN=R3
    Validity
    Not Before: Dec 5 02:46:11 2022 GMT
    Not After : Mar 5 02:46:10 2023 GMT
    Subject: CN=cse-aws-test.sgdev.org
    Subject Public Key Info:
    Public Key Algorithm: id-ecPublicKey
    Public-Key: (256 bit)
    pub:
    04:22:7e:bc:a9:48:27:56:e7:88:de:2c:e9:7e:21:
    20:cb:a5:23:ff:2c:ea:b1:38:2a:c3:1d:3b:63:96:
    f2:59:4f:fc:89:09:f1:64:7f:8c:e3:7e:a4:7e:59:
    07:62:61:b5:6c:20:45:8a:04:59:35:90:45:11:bb:
    18:d2:d8:e7:b6
    ASN1 OID: prime256v1
    NIST CURVE: P-256
    X509v3 extensions:
    X509v3 Key Usage: critical
    Digital Signature
    X509v3 Extended Key Usage:
    TLS Web Server Authentication, TLS Web Client Authentication
    X509v3 Basic Constraints: critical
    CA:FALSE
    X509v3 Subject Key Identifier:
    DF:CA:D3:0B:09:39:3D:D3:12:EC:F6:61:99:DF:00:BC:01:2F:C5:16
    X509v3 Authority Key Identifier:
    keyid:14:2E:B3:17:B7:58:56:CB:AE:50:09:40:E6:1F:AF:9D:8B:14:C2:C6

    Authority Information Access:
    OCSP - URI:http://r3.o.lencr.org
    CA Issuers - URI:http://r3.i.lencr.org/

    X509v3 Subject Alternative Name:
    DNS:cse-aws-test.sgdev.org
    X509v3 Certificate Policies:
    Policy: 2.23.140.1.2.1
    Policy: 1.3.6.1.4.1.44947.1.1.1
    CPS: http://cps.letsencrypt.org

    1.3.6.1.4.1.11129.2.4.2:
    ......v.z2.T..-. .8.R....p2..M;.+.:W.R.R.....d(......G0E. ..jT.OY....}...uO.#...)>......4w.!..........Tc..bT.^.n........&.FX..v..>.$..M.u.9..X.l].B.z.5.....%........d*......G0E. ..L........z.Ft.9_.$.....X...j...!..M~S..Hs..
    ..jY..x.Gp..D....%..S
    Signature Algorithm: sha256WithRSAEncryption
    8f:66:07:fa:46:db:05:bb:f1:06:f0:de:8a:97:14:ad:39:b9:
    47:3d:75:6f:c5:8c:92:93:8e:c6:dc:2f:c5:1c:82:9f:c4:67:
    0e:fe:a3:8b:9c:28:d6:41:ed:2e:ef:57:e7:f6:30:e4:de:31:
    28:38:07:c1:f9:8c:5e:06:d5:de:63:b7:b6:c8:ad:56:ea:6b:
    2d:f0:9b:ef:0c:41:ca:68:c9:67:41:b7:1e:cb:0c:aa:22:27:
    f9:ca:dd:18:08:a9:f8:f4:dc:c8:67:f3:c4:8d:f7:51:5b:88:
    11:ea:50:a1:c3:71:f2:2b:94:f9:86:30:cc:ba:3a:eb:b4:ef:
    14:14:67:f5:45:1a:5a:40:90:76:bd:dd:1e:55:af:57:dc:6e:
    e5:73:0d:0c:ab:00:f4:68:3f:c6:33:2a:52:f6:44:21:b0:aa:
    98:36:fb:eb:c4:69:d7:6e:e5:10:19:a5:65:8f:ad:5c:42:b0:
    90:e9:f6:a2:f4:96:45:33:3c:a1:8f:28:99:0a:21:68:98:d4:
    21:f4:7d:fb:c6:23:98:0e:07:e7:59:ba:a8:48:97:e6:9c:f5:
    02:3a:51:de:6f:d0:17:ce:40:02:21:c9:02:15:d0:b5:33:75:
    eb:e9:12:49:3c:42:1b:de:2c:15:dd:36:b4:99:24:63:5d:7c:
    b9:86:92:a

 

This is a big subject if you've more to add please do!

Was this article helpful?
0 out of 0 found this helpful

Articles in this section