Troubleshooting

The first step of troubleshooting issues in programs using PycURL is identifying which piece of software is responsible for the misbehavior. PycURL is a thin wrapper around libcurl; libcurl performs most of the network operations and transfer-related issues are generally the domain of libcurl.

setopt-Related Issues

setopt is one method that is used for setting most of the libcurl options, as such calls to it can fail in a wide variety of ways.

TypeError: invalid arguments to setopt

This usually means the type of argument passed to setopt does not match what the option expects. Recent versions of PycURL have improved error reporting when this happens and they also accept more data types (for example tuples in addition to lists). If you are using an old version of PycURL, upgrading to the last version may help troubleshoot the situation.

The next step is carefully reading libcurl documentation for the option in question and verifying that the type, structure and format of data you are passing matches what the option expects.

pycurl.error: (1, '')

An exception like this means PycURL accepted the structure and values in the option parameter and sent them on to libcurl, and libcurl rejected the attempt to set the option.

Until PycURL implements an error code to symbol mapping, you have to perform this mapping by hand. Error codes are found in the file curl.h in libcurl source; look for CURLE_OK. For example, error code 1 means CURLE_UNSUPPORTED_PROTOCOL.

libcurl can reject a setopt call for a variety of reasons of its own, including but not limited to the requested functionality not being compiled in or being not supported with the SSL backend being used.

Transfer-Related Issues

If your issue is transfer-related (timeout, connection failure, transfer failure, perform hangs, etc.) the first step in troubleshooting is setting the VERBOSE flag for the operation. libcurl will then output debugging information as the transfer executes:

>>> import pycurl
>>> curl = pycurl.Curl()
>>> curl.setopt(curl.VERBOSE, True)
>>> curl.setopt(curl.URL, 'https://www.python.org')
>>> curl.setopt(curl.WRITEDATA, open('/dev/null', 'w'))
>>> curl.perform()
* Hostname www.python.org was found in DNS cache
*   Trying 151.101.208.223...
* TCP_NODELAY set
* Connected to www.python.org (151.101.208.223) port 443 (#1)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 696 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL re-using session ID
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*      server certificate verification OK
*      server certificate status verification SKIPPED
*      common name: www.python.org (matched)
*      server certificate expiration date OK
*      server certificate activation date OK
*      certificate public key: RSA
*      certificate version: #3
*      subject:
*      start date: Sat, 17 Jun 2017 00:00:00 GMT
*      expire date: Thu, 27 Sep 2018 12:00:00 GMT
*      issuer: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert SHA2 Extended Validation Server CA
*      compression: NULL
* ALPN, server accepted to use http/1.1
> GET / HTTP/1.1
Host: www.python.org
User-Agent: PycURL/7.43.0.1 libcurl/7.52.1 GnuTLS/3.5.8 zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
Accept: */*

< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< X-Clacks-Overhead: GNU Terry Pratchett
< Via: 1.1 varnish
< Fastly-Debug-Digest: a63ab819df3b185a89db37a59e39f0dd85cf8ee71f54bbb42fae41670ae56fd2
< Content-Length: 48893
< Accept-Ranges: bytes
< Date: Thu, 07 Dec 2017 07:28:32 GMT
< Via: 1.1 varnish
< Age: 2497
< Connection: keep-alive
< X-Served-By: cache-iad2146-IAD, cache-ewr18146-EWR
< X-Cache: HIT, HIT
< X-Cache-Hits: 2, 2
< X-Timer: S1512631712.274059,VS0,VE0
< Vary: Cookie
< Strict-Transport-Security: max-age=63072000; includeSubDomains
<
* Curl_http_done: called premature == 0
* Connection #1 to host www.python.org left intact
>>>

The verbose output in the above example includes:

• DNS resolution

• SSL connection

• SSL certificate verification

• Headers sent to the server

• Headers received from the server

If the verbose output indicates something you believe is incorrect, the next step is to perform an identical transfer using curl command-line utility and verify that the behavior is PycURL-specific, as in most cases it is not. This is also a good time to check the behavior of the latest version of libcurl.