Available in NetTalk Desktop, NetTalk Server and NetTalk Apps. Applies to Desktop & Server (ABC or Clarion) apps.

• Introduction
• Before Init
• Before Connecting
• Connecting
• FTP Over TLS
• Control Template
• Commands
• Properties
• Done
• ErrorTrap
• Directory Listings
• Progress Control
• Examples
• FAQ

FTP is one of the oldest protocols on the internet. It is used to transfer files between and FTP Client and an FTP Server. NetTalk includes an FTP Client Class which allows you to build FTP Client functionality into your Clarion program.

FTP is different to most other protocols because it involves two connections between the server and the client. The first connection is known as the Control Connection and commands, and responses, flow on this connection. However when data needs to flow, a second connection, known as the Data Connection is created. The actual file then travels on the Data connection.

Originally the data connection was opened by the server connecting to the client. While this worked well at the beginnings of the internet, it requires special network configuration to allow this to happen. As the internet matured this reverse-connection because a security hazard and most companies don't support it.

In order to extend the life of the FTP protocol a new approach, allowing the client to create the data connection to the server, was invented. This approach is known as Passive mode. In builds before 8.29 the default for the object was Active mode. From build 8.29 the default is passive mode.

As security became a higher and higher priority the need to make the FTP connection secure became important. The FTP protocol was thus further extended to add support for TLS (FTPS and FTPES) and support for SSH (SFTP). NetTalk supports FTP over TLS (FTPS and FTPES) but does not support FTP over SSH.

NetTalk provides two classes which are used in the FTP support. The NetFtpClientControl class handles the control connection, and the NetFtpClientData class handles the data connection.

The FTP client is always a Window procedure, because NetTalk communications are asynchronous and thus need an ACCEPT loop, which in turn needs a window structure. If the procedure is completely automatic then the window itself can be hidden from the user.

This section gets you going as quickly as possible to add FTP functionality to your application.

1. Make sure the Activate NetTalk Global extension has been added to the application.
2. Make sure the Activate StringTheory Global Extension has been added to the application.
3. Check out the ftpDemo app in the \examples\nettalk\ftp\FTP Functions folder. This contains some example procedures which you may be able to use in your application either "as is" or with modifications. See the Examples section below for more on this example.
4. If you want to construct an FTP procedure of your own, then the following basic steps are pretty much required;

 
1. Create a procedure based on the Window template
2. Add a NetTalk Object Extension to the procedure, Object Name ThisFtpControl and Base Class NetFTPClientControl. On the Settings tab set the Data Object to ThisFTPData
3. Add a NetTalk Object Extension to the procedure, Object Name ThisFtpData and Base Class NetFTPDataControl. On the Settings tab set the Control Object to ThisFTPControl
4. Either set the Passive mode setting on the ThisFTPControl Settings tab, or, in embed code, set the thisFtpControl.PassiveMode property before the call to ThisFTPControl.init. If this step is omitted then the connection will default to Passive mode.
5. Add some trigger event to the window (usually a button). Add code to this button which primes all the required parameters (see Before Connecting below) and then calls the first command you wish to execute (see Commands below.)
6. IIn the ThisFTPControl.Done method, add code to execute when a command completes. (See Done below.)
7. IIn the ThisFTPControl.ErrorTrap method add code to deal with errors - often culminating in a Post(Event:CloseWindow).

 

The fundamental way the object works changes when it is in passive mode or active mode. Thus it is necessary to set the PassiveMode property before the control object .INIT method is called. The other properties can be set later, before the first command, but the PassiveMode property must be set very early. The default value is 1, so if this property is not set, or is set too late, then the FTP object will assume that passive mode is required.

As with most connections you connect to a server by specifying a host name and port number. If the server is not in passive mode you should also set that and if the server is to connect over TLS then that needs to be set as well.

The server address is placed in the ServerHost property. The server port number is placed in the ControlPort property.

If the server supports passive mode then the PassiveMode property is set to True. This is the default, so you don't have to set it. If the server does not support passive mode then you need to set this property to False.

If you wish to connect over TLS then call the SetFtpType method with one of the possible equates;
Ftp:NoTLS
Ftp:ImplicitTLSControl
Ftp:ImplicitTLS
Ftp:ExplicitTLS
The SetFTPType method takes the Data connection object as the second parameter.

FTP is an authenticated protocol, meaning that a user name and password are required before you can start transferring files. Some servers allow the use of a "generic" login, called an anonymous login, where you can use some documented login and password, however it's important to note that even these sites do require a login and password to be passed to the server.

The user login is placed in the property called User, and the password is placed in the property called Password. If these change during a connection then set the property OpenNewControlConnection to force the current control connection to close and a new one to be opened with the next command.

If you are connecting with TLS then all the normal TLS properties apply to both the control connection and the data connection.

Example

This is an example of setting all the above settings before the Control object is Initialized. In the code below thisFtpControl is the control object, and thisFtpData is the data object.

thisFtpControl.serverHost = 'ftp.capesoft.com'
thisFtpControl.ControlPort = 21
thisFtpControl.PassiveMode = true
thisFtpControl.SetFtpType(FTP:ImplicitTLS,thisFtpData)
thisFtpControl.User = 'someone'
thisFtpControl.Password = '1234'
thisFtpControl.openNewControlConnection = true

FtpControl.TLSCertificateOptions.CertificateFile = ''
FtpControl.TLSCertificateOptions.PrivateKeyFile = ''
FtpControl.TLSCertificateOptions.DontVerifyRemoteCertificateCommonName = true
FtpControl.TLSCertificateOptions.DontVerifyRemoteCertificateWithCARoot = true
FtpControl.TLSCertificateOptions.CARootFile = 'CA_Roots.pem'

FtpData.TLSCertificateOptions.CertificateFile = ''
FtpData.TLSCertificateOptions.PrivateKeyFile = ''
FtpData.TLSCertificateOptions.DontVerifyRemoteCertificateCommonName = true
FtpData.TLSCertificateOptions.DontVerifyRemoteCertificateWithCARoot = true
FtpData.TLSCertificateOptions.CARootFile = 'CA_Roots.pem'

There is no need to explicitly connect to the server. The object will automatically connect as required when a command is executed.

[ This section of the document has been updated to be used with version 8.29 or later ]

TLS over FTP is done in one of two ways, Explicit and Implicit.

Implicit TLS (sometimes referred to as FTPS)  is the simpler of the two, each connection to the FTP server is done over an TLS connection. The client program (ie NetTalk) assumes the server is listening using TLS, and simply applies TLS to the connection from the very beginning. Because this approach is simpler, it is also faster. Since most FTP-TLS servers that do Explicit TLS also do Implicit TLS it's usually best to try this approach first. Implicit TLS is included in builds 6.06 and later. To place the FTP objects in TLS mode you should use the SetFtpType method with one of the following 4 equates;

Ftp:NoTLS
Ftp:ImplicitTLSControl
Ftp:ImplicitTLS
Ftp:ExplicitTLS

For example;
 
thisFtpControl.SetFtpType(Ftp:ImplicitTLS,thisFtpData)

Note that the second parameter is the Data Object.

These should be set After the call to thisFtpControl.Init, but before any other FTP commands are called.

For Implicit TLS you will probably also need to set the PORT property to something other than the default of 21. The default for Implicit TLS is port 990, however it may be something else (and equally, servers running on Port 990 are not necessarily TLS servers.) Explicit TLS servers may use port 21, or they may use some other port.

ExplicitTLS (informally referred to as FTPES) is slightly different. In this mode the connection starts off as normal (ie plain-text) but before the Login the client explicitly asks to "go to" TLS mode. This asking is repeated at other times, such as when data connections for uploading or downloading need to occur. Support for Explicit TLS was included in NetTalk version 8.29 and later.

SFTP is something completely different, it is the File transfer extension to SSH and is not supported at this time.
For more information on the flavors of FTP see Wikipedia.

From version 8.29 the control template has been deprecated. It still exists, and still works, but is not a recommended approach for use anymore. See the JumpStart section for more information on adding FTP procedures to your application.

FFTP is designed to be a remote-file-system, meaning that the files are organized with directories, and sub-directories and so on. While you will primarily be sending and receiving files there are a number of commands you can use to navigate and manage the server. The commands supported by the FTP object are listed here;

FTP Command	NetTalk Method	Comments
abor	Aborting()	Sends an Abort command to the server. This terminates any file being uploaded or downloaded. The CommandQueue is emptied. No call to Done or ErrorTrap for the existing, or waiting commands will be called. A call to Done with the _Command property set to aborting will be made instead.
appe	AppendToFile(LocalFileName, RemoteFileName)	Similar to the PutFile, but does an append instead of an overwrite. It transfers the local file specified by LocalFileName from the local machine to the FTP Server, and stores it as RemoteFileName. If a file with the same path and name as RemoteFileName already exists, the transferred data is appended to the end of the existing file. If no such file exists, a new file is created. Will either call Done or ErrorTrap or success or failure.
cdup	ChangeDirUp()	Changes the remote current working directory to one level higher in the directory tree. Will either call Done or ErrorTrap or success or failure.
cwd	ChangeDir(DirectoryName)	Changes the remote current working directory to DirectoryName. Will either call Done or ErrorTrap or success or failure.
dele	DeleteFile(RemoteFileName)	Deletes the remote file specified by RemoteFileName. Will either call Done or ErrorTrap or success or failure.
list	GetDirListing(DirectoryName)	Retrieves a list of contents of the remote directory specified by DirectoryName and stores them in self.DirListingQ property. To get contents of the current working directory, pass an empty string as the parameter. For more information on the DirListingQ property see Directory Listings below. By setting the FullListing property to false, you will receive a list of file or directory names only. When FullListing is true (the default), full information on the files will be retrieved. Note that DirectoryName can also contain a remote file name. In this case, current information on the specific file will be returned. Note: Many FTP Servers are case sensitive Examples: GetDirListing ('') ! current dir GetDirListing ('mydirectory') ! dir GetDirListing ('myfile.exe') ! file GetDirListing ('m*') ! file or dir GetDirListing ('m*.exe') ! file(s) Will either call Done or ErrorTrap on success or failure.
size	GetSize(RemoteFileName)	Gets the size of a specific file on the server. The result is placed in the ExpectedFileSize property. Note that not all servers support this call - if it fails ErrorTrap will not be called. If it does fail then the code will check the current DirListingQ to get a size from there. In all cases the Done method is called once the command has completed.
mkd	MakeDir(DirectoryName)	Pass the name of the new remote directory to be created as a parameter. Will either call Done or ErrorTrap or success or failure.
noop	Noop()	Not usually called by the program. Used internally. NOOP is a non-operational command designed to do nothing. It is useful for preventing a FTP Control Connection from closing.
pwd	GetCurrentDir()	Returns the current directory name on the server. The reply is placed in the CurrentDirectory property. Will either call Done or ErrorTrap or success or failure.
quit	Quit()	This method will allow for all the data transfer in process to complete and then disconnect you from the FTP Server. Will either call Done or ErrorTrap on success or failure.
retr	GetRemoteFile(RemoteFileName,LocalFileName)	Transfers the remote file specified by RemoteFileName from the FTP Server to the local machine and stores it in LocalFileName.  If a file with the same path and name as LocalFileName already exists, it will be overwritten by the received file.  A new file is created, otherwise. Will either call Done or ErrorTrap or success or failure.
rnfr / rnto	Rename(RemoteName, NewName)	Renames the remote file specified by RemoteName to NewName.  This method can also be used to rename a directory. Will either call Done or ErrorTrap or success or failure.
rmd	RemoveDir(directory)	Removes the remote directory specified by DirectoryName. Will either call Done or ErrorTrap on success or failure.
stop	Stop()	Immediately terminates the data connection.
stor	PutFile(LocalFileName,RemoteFileName)	Transfers the local file specified by LocalFileName from the local machine to the FTP Server, and stores it as RemoteFileName.  If a file with the same path and name as RemoteFileName already exists, it will be overwritten by the file being transferred.  If the file does not already exist, then it is created. Will either call Done or ErrorTrap on success or failure.
	PutFile (StringTheory pBinData,RemoteFileName)	An alternative version of PutFile. This one takes a StringTheory object as the first parameter instead of a local file name. The contents of the object are then stored on the server under the RemotefileName.
syst	System	Gets a OS type from the server. the result is placed in the SystemType property.
type	SetType(Type)	The type is either A for ascii, or I for binary. Not usually called by the program, is handled internally. Will either call Done or ErrorTrap on success or failure.

A number of properties affect how the FTP Client will behave. Other properties are the result of commands.

Property	Description
BinaryTransfer	If set to true then all file transfers will be in binary mode. If false transfers will be in ASCII mode. The default is binary mode, and there is almost never a requirement to transfer in ASCII mode.
BytesLeftToReceive	When receiving a file, contains the amount of the file which still has to arrive.
BytesLeftToSend	When sending a file, contains the number of bytes that have not been sent yet.
_Command	Contains the command (as called in the command list) which has just completed. Typically used in the Done or ErrorTrap methods. Is also set at the start of the calls to PutFile or GetRemoteFile. Note that this property is always in lower case.
CurrentDirectory	Set after a call to the GetCurrentDir command.
DirListingQ	A queue containing the remote directory contents after a call to GetDirListing. See Directory Listings for more information.
ExpectedFileSize	The size of the file being retrieved. Populated by a call to GetSize, which in turn will use the existing DirListingQ property if the server does not support the SIZE command.
ForwardSlashes	UNIX operating systems use the forward slash (/) instead of the backslash (\) when separating directories and file names. Since FTP originated on UNIX most FTP servers (even when running on windows) follow this convention. If this property is true then all backslashes in remote file name parameters will automatically be converted to forward slashes before executing the command.
PassiveMode	This needs to be set before the object INIT method is called. (In other words, it's called much earlier than any other setting). The default value is true.
ProgressControl	The Use Equate of a progress bar control on the window. Can be used to display the progress of file uploads and downloads. See Progress Control for more information.
SystemType	A text string containing the System Type as identified by the server. A command to fetch this is called automatically when logging into the server. See System.

As motioned earlier the communications between the server and the client is asynchronous. This means that you send a command, but you cannot immediately test for the response to that command. Rather when the command completes successfully the .Done method will be called. So if you wish to add more commands at this point then you can do so.

Inside the done method you can test the ._command property which will return the name of the command which just completed. This property is in lower case, and matches the NetTalk Method Name in the table above.

If the command is unsuccessful then the .ErrorTrap method is called instead of the .Done method.

Example

case self._command
of 'makedir'
  self.changedir(somedir)
end

Important: Some commands that you call will generate other commands internally. For example when doing a Putfile a SetType command is executed first. In this case you'll get a call to Done first for SetType and then later on for PutFile. So your code in the Done method should be aware that more calls to Done may be called than commands you have executed.

The ErrorTrap method is called when an error occurs. when this happens the Done method will not be called.

The format of the directory listing is not specified by the protocol. So the exact format of the text which is returned depends on the specific server, and specific OS of the server. NetTalk does a reasonable job of deformating this reply and placing the result in a property called .DirListingQ.

The Queue is declared as follows;

Net:FTPDirListingQType       Queue, Type
Name                            string(FILE:MAXFILENAME)
Shortname                       string(13) ! Never used - just here because of the DirListing structure
Date                            long
Time                            long
Size                            long
Attrib                          byte
NoOfLinks                       long
Owner                           string(50)
GroupOwnership                  string(50)
AccessPermissions               string(50)
                             end

In your program you can declare your own queue of this type and as long as it has the same structure, you can set the object to use your queue rather than the built-in one.  For example, in the program code;

MyQueue                      Queue
Name                           string(FILE:MAXFILENAME)
Shortname                      string(13)
Date                           long
Time                           long
Size                           long
Attrib                         byte
NoOfLinks                      long
Owner                          string(50)
GroupOwnership                 string(50)
AccessPermissions              string(50)
                            end

thisFtpControl.DirListingQ &= MyQueue

This means that the call to GetDirListing will result in your queue being automatically populated with the result. Since the queue is declared in your procedure it is also easy to assign it to a LIST control on the window.

Unknown format of directory listing

If the object is unable to interpret the reply from the server then an error will be generated "Unknown format of directory listing".  It is quite unusual to find a server that uses a format that NetTalk can't figure it out.

If you don't want this error to be called then put the following code in the virtual method _FigureOutDirFormat() after the parent call:
if self._DirListingFormat = 0
  self._DirListingFormat = 255
end

You will need to figure out the directory listing yourself in this case. The way that it works is that the _ProcessGetDirListing() method is called when data is received. If there is not enough information to figure out the format then the _DirFormatSet property is set to zero and the data is stored.

When there is sufficient information from the server the _DirFormatSet property is set to 1 and _FigureOutDirFormat() is called. If this method manages to figure out the directory format then it sets the _DirListingFormat property, otherwise it sets it to zero.

This is what is happening in this case - enough data has been received to figure out the format, but the format itself is unknown. In order to add support for the custom format add code to the _FigureOutDirFormat() embed to process the directory listing after the parent call (the parent method could not figure it out, which is what caused the error).

The _FigureOutDirFormat() method is passed a string which contains the directory list (what you would see printed out in a terminal if you were connecting to the FTP server on the command line and the LIST command was issued to the server).

NetTalk allows you to process the listing and set the _DirListingFormat to a number between 200 and 255, which means that it is in a custom format. You then need to process this string in the _FillDirListingQ_FormatCustom() virtual method that NetTalk provides for you.

It might be helpful to have a look at the source code for the methods such as _FillDirListingQ_Format01() to see how they process the string into the queue and then write your own FillDirListingQ_FormatCustom() to process your specific server's response. You might find that it is quite close to one of the currently supported formats. The full source code for the FTP objects is in the NetFtp.clw and NetFtp.inc files in your \Clarion\Accessory\Libsrc\win folder.

Some of the operations, specifically PutFile and GetRemoteFile can take some time to complete. The class has a property called .ProgressControl which can be set to the use equate of a progress bar on the screen. If this property is set then the control (ie a progress bar) will be updated as the file upload, or download, happens.

For example

thisFtpControl.ProgressControl = ?Progress1

Three additional properties are available if you wish to display the actual values. They are;
.ExpectedFileSize
.BytesLeftToSend
.BytesLeftToReceive

NetDemo.App

FTPDemo.App

FTPLegacy.App

1. How do I upload multiple files using FTP?

You can call .PutFile multiple times. The commands will be queued and executed one at a time.