The SSLError in Python requests happens when trying to access a web page with an untrusted or invalid SSL certificate. This error breaks the SSL handshake between your code and the remote server, preventing any data from being transmitted securely.
In this comprehensive guide, we'll cover everything you need to know to troubleshoot and fix SSL certificate errors in Python requests.
What Causes SSLError in Python Requests?
There are a few common triggers for the SSLError in requests:
- Expired certificate: The site's SSL certificate has expired and needs to be renewed. Python requests will reject expired certs.
- Domain mismatch: The domain you're requesting doesn't match the domain on the SSL certificate. Requests will throw an error for certificate mismatches.
- Self-signed certificate: The site is using a self-signed certificate not signed by a trusted certificate authority. Python requests doesn't trust self-signed certs.
- Untrusted CA: The SSL certificate is signed by a certificate authority that isn't trusted by your Python environment.
- Problematic cipher suites: The server only supports insecure cipher suites that requests will not use.
- Revoked certificate: The SSL certificate has been revoked by the issuing CA, meaning it should no longer be trusted.
- Expired or changed intermediate certificates: Issues with intermediate certs in the chain can also trigger SSL errors.
Any of these situations will lead to requests rejecting the site's SSL certificate and throwing a
SSLError. The specific error message will provide clues on the exact cause.
How to Diagnose Python Requests SSL Issues
Debugging SSL issues can be tricky. How do you track down the root cause? Here are some effective troubleshooting steps:
Check the Error Message
First look at theException message – it may provide hints on the issue:
SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
Here the error indicates a TLS version incompatibility issue. Compare against common SSLError messages to isolate potential causes like expired certs, domain mismatch, untrusted CAs, etc.
Enable Requests DEBUG logging
The best way to diagnose is enabling DEBUG logs in requests to see the SSL handshake:
import logging import requests logging.basicConfig() logging.getLogger().setLevel(logging.DEBUG) requests.get('https://expired.badssl.com')
This prints verbose request details:
...checks for domain match ...using TLS version TLSv1.2 ...adding CA /etc/ssl/certs/CustomCA.pem ...certificate expired error ...SSL handshake failed
The debug logs provide the smoking gun of exactly why SSL validation failed.
Use Online SSL Scanners
Check OS and Python CA Stores
Many SSL problems come down to missing Certificate Authority certificates needed to validate the cert chain. Use Python's
ssl.get_default_verify_paths() to check your OS and Python CA file paths. Ensure these files exist and contain necessary root and intermediate certificates.
Review Server Configuration
Check the affected server's SSL certificate, keys, protocols, ciphersuites, etc. Look for expired certificates, mixed HTTP/HTTPS content, vulnerable TLS versions, and weak ciphers. These server-side problems frequently trigger SSL handshakes errors. With some debugging legwork, you can get to the bottom of nearly any SSLError.
Quick Fix: Disabling Certificate Verification
Once you've diagnosed the issue, what's the quickest fix? You can simply disable all SSL certificate verification in requests using
This forces requests to skip validation and allows sites with any SSL problem to work. However, disabling verification is extremely dangerous and defeats the purpose of SSL encryption. Without verification, the connection is susceptible to man-in-the-middle attacks.
Only use this approach for public test APIs – never for transmitting user credentials or sensitive information! Now let's explore safer ways to resolve SSLErrors.
Using a Custom CA Certificate Bundle
To fix verification issues while staying secure, you can provide a custom PEM file containing any missing CA certificates required to validate the site's cert chain:
custom_ca_bundle = '/path/to/custom_ca_cert.pem' requests.get(url, verify=custom_ca_bundle)
If the built-in requests bundle is missing a necessary intermediate or root CA, this allows augmenting the trusted certificates. For private certs, exporting the site certificate to PEM format and specifying it as the custom CA works. Always prefer using a proper chain of trust over disabling verification entirely.
Updating the Certifi CA Store
Requests relies on the certifi package and its CA bundle for verifying certificates:
import requests print(requests.certs.where()) /usr/local/lib/python3/dist-packages/certifi/cacert.pem
Many SSLErrors arise from outdated CA certificates in certifi. You can update to the latest CA store with
pip install certifi -U and potentially resolve verification issues caused by expired certificates. Automate certifi updates to stay on top of the latest certificate authorities.
Overriding Cipher Suites
Try forcing requests to use modern, secure cipher suites if you get SSL handshake errors regarding incompatible cipher suites:
from requests.packages.urllib3.util.ssl_ import DEFAULT_CIPHERS requests.get(url, ssl_ciphers=DEFAULT_CIPHERS)
This bypasses outdated servers only supporting old insecure ciphers. Other libraries like
httpx also support overriding cipher suites. Compare your current OpenSSL cipher list against the Mozilla recommended configurations to identify mismatches.
Using the SSLContext Class for Low-Level Control
For full control over the SSL handshake, create a custom
SSLContext in Python's built-in
# Create the SSL context ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) ctx.load_verify_locations('/path/to/ca_certs.pem') # Force certificate validation ctx.verify_mode = ssl.CERT_REQUIRED # Disable insecure protocols ctx.options |= ssl.OP_NO_SSLv2 ctx.options |= ssl.OP_NO_SSLv3 # Restrict cipher suites ctx.set_ciphers('ECDHE+AESGCM:!RC4:!3DES') requests.get(url, ssl_context=ctx)
This handles everything from trust stores, certificate validation, security protocols, and cipher suites. Utilize the low-level control of SSLContext to configure requests exactly as needed to fix validation issues.
Using Framework-Specific SSL Settings
If using a Python web scraping framework like Scrapy, selenium, or Beautiful Soup, use their SSL verification toggles:
yield scrapy.Request(url, callback=self.parse, ssl_verify=False)
options = webdriver.ChromeOptions() options.set_capability("acceptSslCerts", True) driver = webdriver.Chrome(options=options)
soup = BeautifulSoup(html, "html.parser", verify=False)
These bypass SSLErrors when scraping. Only disable for public sites without credentials.
Ignoring Specific Error Codes
In rare cases, you may need to ignore certain SSL error codes to allow requests to proceed:
from requests.packages.urllib3.exceptions import InsecureRequestWarning requests.packages.urllib3.disable_warnings(InsecureRequestWarning) try: response = requests.get(url) except requests.exceptions.SSLError as err: if err.args.reason == 'CERTIFICATE_VERIFY_FAILED': print("Ignoring error for site") else: raise
This is highly dangerous but can be necessary as a last resort for certain sites. Know the risks before ignoring specific codes!
Automating Certificate Renewals
Many SSL issues arise from forgetful site owners not renewing expired certificates. Use Let's Encrypt clients like Certbot to automate free certificate generation and renewals. Services like SSLMate also help manage certificates at scale.
Set up monitoring to check expirations and force renewals before certificates lapse. Proper certificate lifecycle management prevents so many SSLErrors.
Leveraging SSL Troubleshooting Tools
$ openssl s_client -connect expired.badssl.com:443 ... verify error:num=10:certificate has expired
Learn to use them to inspect certificates, handshake failures, mismatches, expiration dates, and more.
The Dangers of Disabling Certificate Verification
It's worth reiterating how dangerous disabling SSL certificate verification can be:
- No encryption: Disabling verification turns SSL into plain unencrypted HTTP. Sensitive data is transmitted in the clear over the internet for anyone to intercept.
- MITM attacks: There is no protection against man-in-the-middle attacks that modify traffic and steal credentials or data. An attacker can impersonate the site.
- Host forgery: Your code could get tricked into talking to a fake imposter domain rather than the real origin. This compromises security and integrity.
- User distrust: Visitors will get scary certificate warnings undermining confidence and harming your brand reputation.
Only disable verification for public test sites as a temporary workaround, never live production services transmitting private user data.
Robustly handling SSL certificates is a crucial skill for building bulletproof Python services. With some diligent debugging and configuration, you can squash frustrating requests SSLErrors for good.
Hopefully, this deep dive has given you lots of helpful tips and tricks for keeping those Python requests running smoothly across even the most complex SSL environments. Happy coding!