Qt Reference Documentation

Contents

Web FTP Client Example

Files:

The Web FTP Client example shows how to add support for a new protocol to QtWebKit-based applications.

An FTP client displaying the contents of the ftp.qt.nokia.com site.

Introduction

The QtWebKit module presents many ways to integrate the worlds of native desktop and mobile applications and the Web, making it possible for developers to extend and combine features found in Qt and WebKit to create new ones. In this article, we examine the use of Qt's network access API with WebKit and show how to turn QWebView into a simple FTP client.

In the Web Plugin Example, we extended Qt's WebKit integration by showing how to add custom widgets to Web pages. In the article, we used QNetworkRequest to ask for content for display in a widget, and we obtained the data returned by the server by reading from the corresponding QNetworkReply.

Qt's network access API is a technology that aims to replace much, but not all, of the functionality provided by the QHttp(obsolete) and QFtp classes. Although the network access API is a Qt-specific technology, the QtWebKit module integrates this Qt technology with WebKit to enable customization of the browser engine by Qt application developers. It also means that we can control how the browser engine obtains and renders content.

Since QNetworkRequest and QNetworkReply are designed to provide a reusable abstraction for network operations, it seems obvious to use these classes to add FTP support to browsers written using QtWebKit. To do this, we first need to examine the network access classes before we see how the QtWebKit module uses them to manage network operations.

Network Access

The central class in Qt's network access API is QNetworkAccessManager. This class performs the work of dispatching requests to remote servers and handling incoming replies. Applications typically construct an instance of this class and use it for all high level network communication.

Applications create QNetworkRequest objects, each of them specifying a URL where the request is to be sent and containing meta-data that will be understood by the server. Each request is dispatched by passing it to a function in the network manager — there are different functions corresponding to different kinds of operations, such as get(), put() and post(). Each of these functions returns a QNetworkReply object which is used to obtain the content sent in the reply, as well as any meta-data that describes it.

The QtWebKit module provides the QWebPage class which represents the content displayed in a QWebView widget. Behind the scenes, this class uses a default network access manager to handle network communication. This default manager works perfectly well for fetching content over HTTP from http:// URLs, but only supports fetching of files over FTP when using ftp:// URLs.

Fortunately, QWebPage provides the setNetworkAccessManager() function that allows the default manager to be replaced with one with more features. This lets us add improved support for FTP quite easily if we can write a new manager that supports ftp:// URLs.

The process of replacing the manager and using a new one with an existing QWebPage object can be broken up into three steps:

  1. Creating a new QNetworkAccessManager subclass.
  2. Creating a new QNetworkReply subclass to deal with the FTP protocol.
  3. Setting the new manager on the QWebPage.

Additionally, to provide a reasonable user experience, we should also handle content that the browser engine cannot display. To do this, we create a custom Downloader object. We will briefly return to this topic later.

Creating a New Network Manager

Replacing an existing network manager for a QWebPage is conceptually simple: we subclass QNetworkAccessManager and reimplement its createRequest() function to check for URLs with the ftp scheme. However, we want to ensure that the manager uses any existing cache and proxy settings that may have been set up for the existing manager used by the QWebPage.

To keep the existing proxy and cache, we give our network manager a constructor that accepts the old manager as an argument. In the constructor, we reuse the settings from the old manager.

 NetworkAccessManager::NetworkAccessManager(QNetworkAccessManager *manager, QObject *parent)
     : QNetworkAccessManager(parent)
 {
     setCache(manager->cache());
     setCookieJar(manager->cookieJar());
     setProxy(manager->proxy());
     setProxyFactory(manager->proxyFactory());
 }

The createRequest() function is used to create and dispatch requests to remote servers for each of the different kinds of operation that the API presents to the developer. Since we are only interested in performing simple fetches of resources using the ftp scheme, we filter out other schemes and other kinds of operation, delegating the task of handling these to the default implementation.

 QNetworkReply *NetworkAccessManager::createRequest(
     QNetworkAccessManager::Operation operation, const QNetworkRequest &request,
     QIODevice *device)
 {
     if (request.url().scheme() != "ftp")
         return QNetworkAccessManager::createRequest(operation, request, device);

     if (operation == GetOperation)
         // Handle ftp:// URLs separately by creating custom QNetworkReply
         // objects.
         return new FtpReply(request.url());
     else
         return QNetworkAccessManager::createRequest(operation, request, device);
 }

Here, we construct and return an instance of the FtpReply class. This class performs most of the work of handling the FTP protocol.

Creating a Custom Reply

The network access API is designed to be simple to use: we set up a request, dispatch it using the network manager, and obtain a QNetworkReply object. If we are not interested in the reply's meta-data, we can simply read the data using its readAll() function because QNetworkReply is a QIODevice subclass.

In order to keep the API so simple, however, we need to perform some work behind the scenes. In this case, that means that we must perform a series of communications with the FTP server. Fortunately, we can use the existing implementation provided by QFtp to perform the low level work.

In the FtpReply class, we need to reimplement four functions in the API to ensure that it will work correctly. These functions, abort(), bytesAvailable(), isSequential(), readData(), rely on the rest of the implementation to fill a QByteArray with data and use an integer offset to track how much has been read from the device by the browser.

 class FtpReply : public QNetworkReply
 {
     Q_OBJECT

 public:
     FtpReply(const QUrl &url);
     void abort();
     qint64 bytesAvailable() const;
     bool isSequential() const;

 protected:
     qint64 readData(char *data, qint64 maxSize);

 private slots:
     void processCommand(int command, bool error);
     void processListInfo(const QUrlInfo &urlInfo);
     void processData();

 private:
     void setContent();
     void setListContent();

     QFtp *ftp;
     QList<QUrlInfo> items;
     QByteArray content;
     qint64 offset;
     QStringList units;
 };

The processCommand(), processListInfo and processData() slots handle interaction with the FTP server. The private setContent() and setListContent() functions are used to add meta-data to the reply and compose HTML for the browser to display.

Two of the private variables hold information about the data obtained from the FTP server: items is updated to contain information about each file found at a given URL, and content contains the raw data obtained from the server. The offset variable is used to track how much data has been read by the browser from the reply.

In the constructor, we construct a QFtp object and connect the signals and slots that form the basis of the interaction with the FTP server. The high level communication is reported by the commandFinished() signal. New data from the server is reported by the QFtp::readyRead()} signal. Individual items in an FTP directory listing are reported by the listInfo() signal.

 FtpReply::FtpReply(const QUrl &url)
     : QNetworkReply()
 {
     ftp = new QFtp(this);
     connect(ftp, SIGNAL(listInfo(QUrlInfo)), this, SLOT(processListInfo(QUrlInfo)));
     connect(ftp, SIGNAL(readyRead()), this, SLOT(processData()));
     connect(ftp, SIGNAL(commandFinished(int, bool)), this, SLOT(processCommand(int, bool)));

     offset = 0;
     units = QStringList() << tr("bytes") << tr("K") << tr("M") << tr("G")
                           << tr("Ti") << tr("Pi") << tr("Ei") << tr("Zi")
                           << tr("Yi");

     setUrl(url);
     ftp->connectToHost(url.host());
 }

We also initialize the offset into the data that represents the number of bytes that the browser has read from the reply. Additionally, we define a list of units for use with the setListContent() function. The last two tasks performed in the constructor are to set the URL of the reply so that the browser can tell where it came from, and to connect to the FTP server.

Fetching Data from the Server

All communication with the server is handled by the processCommand() slot, which acts on responses from the server and tells us when a command we have issued has completed. This slot performs the task of logging in to the server when connection has occurred (the ConnectToHost command has completed), asking for a list of files when logged in (Login has completed), preparing a page with a listing when all file information has been received (List has completed), and setting the current content for the reply when data has been fetched from the server (Get has completed).

 void FtpReply::processCommand(int, bool err)
 {
     if (err) {
         setError(ContentNotFoundError, tr("Unknown command"));
         emit error(ContentNotFoundError);
         return;
     }

     switch (ftp->currentCommand()) {
     case QFtp::ConnectToHost:
         ftp->login();
         break;

     case QFtp::Login:
         ftp->list(url().path());
         break;

     case QFtp::List:
         if (items.size() == 1)
             ftp->get(url().path());
         else
             setListContent();
         break;

     case QFtp::Get:
         setContent();

     default:
         ;
     }
 }

The result of the List command is handled by looking at the number of items obtained from the server. The items themselves are recorded by the processListInfo() slot. When a List command is complete, we can count the number of items received and determine whether or not we should create a file listing, or try to fetch the file instead by invoking a Get command.

 void FtpReply::processListInfo(const QUrlInfo &urlInfo)
 {
     items.append(urlInfo);
 }

Since the reply will only be used once, we can simply append items to a list and never bother to clear it.

The processData() slot simply appends data obtained from the FTP server to the QByteArray containing the content to be supplied to the browser.

 void FtpReply::processData()
 {
     content += ftp->readAll();
 }

Data is appended to the content array until the connection to the FTP server is closed, either by the reply or by the server itself. One of the ways in which this happens is when a Get command completes. At this point, the setContent() function is called from within the processCommand() function.

 void FtpReply::setContent()
 {
     open(ReadOnly | Unbuffered);
     setHeader(QNetworkRequest::ContentLengthHeader, QVariant(content.size()));
     emit readyRead();
     emit finished();
     ftp->close();
 }

Here, we prepare the reply for use by the browser by opening it for unbuffered reading and setting the header that reports the amount of data held by the reply. We emit signals that indicate that the network operation has finished and that it has data to be read. Since we are no longer interested in the FTP server, we close the connection to it.

Preparing Content for the Reader

Another way in which the reply closes the connection to the server is when the setListContent() function is called from the processCommand() function. Most of the implementation of this function involves transforming the information about the items held in the reply's private items variable to HTML.

 void FtpReply::setListContent()
 {
     QUrl u = url();
     if (!u.path().endsWith("/"))
         u.setPath(u.path() + "/");

     QString base_url = url().toString();
     QString base_path = u.path();

     open(ReadOnly | Unbuffered);
     QString content(
         "<html>\n"
         "<head>\n"
         "  <title>" + Qt::escape(base_url) + "</title>\n"
         "  <style type=\"text/css\">\n"
         "  th { background-color: #aaaaaa; color: black }\n"
         "  table { border: solid 1px #aaaaaa }\n"
         "  tr.odd { background-color: #dddddd; color: black\n }\n"
         "  tr.even { background-color: white; color: black\n }\n"
         "  </style>\n"
         "</head>\n\n"
         "<body>\n"
         "<h1>" + tr("Listing for %1").arg(base_path) + "</h1>\n\n"
         "<table align=\"center\" cellspacing=\"0\" width=\"90%\">\n"
         "<tr><th>Name</th><th>Size</th></tr>\n");

     QUrl parent = u.resolved(QUrl(".."));

     if (parent.isParentOf(u))

         content += QString("<tr><td><strong><a href=\"" + parent.toString() + "\">"
             + tr("Parent directory") + "</a></strong></td><td></td></tr>\n");

     int i = 0;
     foreach (const QUrlInfo &item, items) {

         QUrl child = u.resolved(QUrl(item.name()));

         if (i == 0)
             content += QString("<tr class=\"odd\">");
         else
             content += QString("<tr class=\"even\">");

         content += QString("<td><a href=\"" + child.toString() + "\">"
                            + Qt::escape(item.name()) + "</a></td>");

         qint64 size = item.size();
         int unit = 0;
         while (size) {
             qint64 new_size = size/1024;
             if (new_size && unit < units.size()) {
                 size = new_size;
                 unit += 1;
             } else
                 break;
         }

         if (item.isFile())
             content += QString("<td>" + QString::number(size) + " "
                                + units[unit] + "</td></tr>\n");
         else
             content += QString("<td></td></tr>\n");

         i = 1 - i;
     }

     content += QString("</table>\n"
                        "</body>\n"
                        "</html>\n");

     this->content = content.toUtf8();

     setHeader(QNetworkRequest::ContentTypeHeader, QVariant("text/html; charset=UTF-8"));
     setHeader(QNetworkRequest::ContentLengthHeader, QVariant(this->content.size()));
     emit readyRead();
     emit finished();
     ftp->close();
 }

Once the HTML description of the files has been composed in a QString, we convert it to a UTF-8 encoded set of bytes which we store in the reply's private content variable. In this case, the QByteArray holds HTML instead of file data. We set the reply's headers to indicate that it contains UTF-8 encoded HTML with a certain length, and we emit the readyRead() and finished() signals to let the browser know that it can start reading the content.

Supplying Data to the Browser

We reimplement four QIODevice functions to provide basic read-only behavior, simply supplying the data held in the content array.

We do not support aborting of the reply, so our abort() implementation is empty.

 void FtpReply::abort()
 {
 }

Similarly, we do not support random access reading, so isSequential() is reimplemented to always return true.

 bool FtpReply::isSequential() const
 {
     return true;
 }

The bytesAvailable() function returns the total number of bytes held by the reply minus the value of offset, which is the number of bytes we have already supplied to the reader.

 qint64 FtpReply::bytesAvailable() const
 {
     return content.size() - offset + QIODevice::bytesAvailable();
 }
 qint64 FtpReply::readData(char *data, qint64 maxSize)
 {
     if (offset < content.size()) {
         qint64 number = qMin(maxSize, content.size() - offset);
         memcpy(data, content.constData() + offset, number);
         offset += number;
         return number;
     } else
         return -1;
 }

The readData() reimplementation tries to return as much data to the reader as it will allow, copying bytes directly to the appropriate location in memory. The offset variable is updated to keep track of how many bytes have been read.

Enabling the Protocol

Now that we have an FTP-enabled network manager and a reply that can handle communication with FTP servers, we can now enable the manager for a given QWebPage. We derive the FtpView class from QWebView and configure its behavior in its constructor.

As we mentioned earlier, we pass the original network manager to the newly-created manager and pass the new manager to the QWebPage belonging to the browser. This enables our network manager for the content it displays.

 FtpView::FtpView()
 {
     QNetworkAccessManager *oldManager = page()->networkAccessManager();
     NetworkAccessManager *newManager = new NetworkAccessManager(oldManager, this);
     page()->setNetworkAccessManager(newManager);

     page()->setForwardUnsupportedContent(true);
     downloader = new Downloader(this, newManager);

     connect(page(), SIGNAL(unsupportedContent(QNetworkReply *)),
             downloader, SLOT(saveFile(QNetworkReply *)));
     connect(page(), SIGNAL(downloadRequested(const QNetworkRequest &)),
             downloader, SLOT(startDownload(const QNetworkRequest &)));

     connect(this, SIGNAL(urlChanged(const QUrl &)),
             this, SLOT(updateWindowTitle(const QUrl &)));
 }

We also go to some effort to handle content that WebKit does not natively support, using a Downloader helper class to manage this and files that the user downloads via the browser's Save Link... context menu entry.

In the example's main() function, we perform the usual steps to initialize our Qt application. We choose an appropriate starting URL for the FtpView widget to open before running the application's event loop.

 int main(int argc, char *argv[])
 {
     QApplication app(argc, argv);

     FtpView view;
     view.setUrl(QUrl("ftp://ftp.qt.nokia.com"));
     view.show();

     return app.exec();
 }

Summary

As we have seen, enabling support for another protocol and URL scheme in QtWebKit is a fairly simple process involving the creation of a network manager and custom reply object. The implementation challenges are mostly related to how the protocol is handled by the custom QNetworkReply subclass where, in our case, we need to issue the appropriate commands in the correct order to obtain data from the FTP server.

We also need to ensure that that the reply emits the appropriate signals to inform the browser that it has content to be read. Our implementation is intentionally simple, only notifying the browser with the readyRead() signal when all the content is ready to read and emitting the finished() signal to indicate that communication is complete; a more sophisticated approach would interleave the commands sent to the server with the emission of signals, allowing the browser to read content as data is obtained from the FTP server.

The reply also needs to be open for reading. Forgetting to call the open() function is a common error to make when dealing with devices, but in this case it is the reply's responsibility to open itself. It must indicate how much content it has for the browser to read. As we have seen, this is done by setting the reply's ContentLengthHeader header with the appropriate value. With this information available, the browser can read from the reply when the content becomes available, displaying a directory listing or downloading content depending on the type of data supplied.

We can use the approach described in this article to enable support for other protocols by writing or extending a network manager to handle URL schemes such as mailto, sip, news, file and ldap. Applications that integrate Web content with information from other sources can also provide custom URL schemes as long as care is taken not to use an existing public scheme.

© 2015 The Qt Company Ltd. Documentation contributions included herein are the copyrights of their respective owners.


The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation.

Documentation sources may be obtained from www.qt-project.org.


Qt and respective logos are trademarks of The Qt Company Ltd in Finland and/or other countries worldwide. All other trademarks are property of their respective owners. Privacy Policy