NetTalk Email
Introduction
Almost all programs can benefit from Sending and/or
Receiving emails. NetTalk makes it easy to add simple, or sophisticated,
email handling abilities directly into your application.
Emails are sent using the SMTP protocol.
Emails are received using either the POP3 or IMAP protocol.
Upgrading
In NetTalk 10 (specifically 10.22 and later) some of
the properties changed from STRING values to STRINGTHEORY objects. Because
assigning a value to an object using the = sign causes a GPF, the names of
the properties have changed. This means that if you have used the old
properties in your embed code then you will get a compile error.
If you get a compile error in your code then you can replace the old
property name, and syntax, with the new property name and syntax.
These changes affect Email Sending (SMTP), Email Receiving (POP3 and IMAP)
classes.
| Old Syntax |
New Syntax |
| x = email.WholeMessageLen |
x = email.WholeMessageST.Length() |
| s = email.WholeMessage |
s = email.WholeMessageST.GetValue() |
| x = email.MessageTextLen |
x = email.MessageTextST.Length() |
| s = email.MessageText |
s = email.MessageTextST.GetValue() |
| email.MessageText = s |
email.MessageTextST.SetValue(clip(s)) |
| x = email.MessageHTMLLen |
x = email.MessageHTMLST.Length() |
| s = email.MessageHTML |
s = email.MessageHTMLST.GetValue() |
| email.MessageHTML = s |
email.MessageHtmlST.SetValue(clip(s)) |
| if Imap.wholeMessageLen |
if Imap.wholeMessageST.Length() |
| x = email.BufferLen |
x = email.BufferST.Length() |
| s = email.Buffer |
s = email.BufferST.GetValue() |
The prototype for the following methods have changed;
| MethodName |
Change |
| EmbedImages |
First parameter has changed from a *string
to a StringTheory object. |
The following methods are no longer required, and if you have called them
in your embed code then you can just remove the call;
| MethodName |
| SetRequiredMessageSize |
JumpStart
Sending Emails
The example in
\clarion\examples\NetTalk\Email\Demo\ABC is the best email
example. And you can import various dictionary tables and procedures
directly from there to speed things along. References to "the example"
in the text below are referring to this example. A Legacy version of
this example is in
\clarion\examples\NetTalk\Email\Demo\Legacy
Quick Method
- Import NetTalkEmail.Dctx into
your dictionary
- Add the NetTalk Global Extension (and StringTheory Global
Extension) to your app
- On the NetTalk Global Extension, Procedures Tab, click on the SendEmail
button
What Just Happened?
In order to be able to send emails from your
application there are really only a few basic things to do;
- Allow the user to enter server connection settings somewhere so
that your program knows which server to send to.
- Optionally, create a EmailLog table
in your dictionary, and add a browse on this to your app.
- Create a SendEmail procedure which
you can call from anywhere in your code.
- Add a call to SendEmail in all the
various places where you wish to send emails.
Server Connection Settings
TIP: You can
import the
NetTalkEmail.dctx file into
your dictionary. This adds two tables,
EmailSettings
and
EmailLog to your
dictionary. The
EmailSettings table
contains the fields below.
A number of settings are used when connecting to a server. These
settings can vary from one installation to the next so it's important
to allow the user to capture these. Once set (on one workstation) the
settings are usually valid for all the workstations so (assuming the
settings are visible to all machines) no per-workstation setup is
required.
You could add these settings to your existing shared settings table or
you could create a new table. The settings are as follows:
- EmailServer (String 255)
This is the URL of the server. It's sometimes an IP address (eg
192.168.1.1) or a DNS name (eg mail.capesoft.com)
- EmailPort (Long) (A long.
The default value should be 25. Values of 587, 990 and 465 are
also commonly used.
- AlwaysSSL (Long).
A boolean value (0 or 1) this is set to 1 if the connection to the
server is always over TLS
- StartTLS (Long)
A boolean value (0 or 1) this is set to 1 if the connection starts
normally but then a STARTTLS command switches it to TLS.
- SSLMethod (Long)
The TLS version to use when connecting to the server. The default
is TLS 1.2, 1.1 or 1.0 (in that order).
- UserName (String 255)
The user name for the user on the server.
- Password (String 255)
The password for this user when connecting to the server
- FromAddress (String 255)
The default FROM address of the "person" sending the email. All
emails must have a valid FROM address (ie the server must
recognize the FROM address) or the chance of the email being
successful is very small.
Once you have the fields in your table you should give the user
some way to change these settings. They will need to be different
with each site you install, so the user will need to have access to
them.
Tip: On one of our internet
machines we run a mail server. In our shipping programs we default
all the above settings to that (internet) mail server. This
increases the chances of the email sending "just working"
immediately after the program is installed. The customer can change
the settings later if they like. Of course if they've blocked the
outgoing port in their firewall then they won't be able to send via
our server, but then they will have their own internal server they
can route the mail through.
EmailLog Table
TIP: You can
import the
NetTalkEmail.dctx file into
your dictionary. This adds two tables,
EmailSettings
and
EmailLog to your
dictionary. The
EmailLog
table contains the fields below.
Keeping a record of outgoing emails is very useful. Exposing this
record to the customer adds value to your program and also allows them
(or you) to audit what emails were sent, and when.
While you can store any fields you like in this table, the following
are recommended;
- Guid (String 16)
- User (String 255)
- Date (Long / Date)
- Time (Long / Time)
- SendDate (Long / Date)
- SendTime (Long / Time)
- ToEmail (String 255)
- FromEmail (String 255)
- Subject (String 255)
- CC (String 255)
- BCC (String 255)
- Attachment (String 1024)
- EmbedList (String 1024)
- MessageText (MEMO 64000)
- MessageHTML (MEMO 64000)
- Status (Long)
Global Variables
When logging outgoing emails it's useful to know
the current user's Name and Email Address. (ie the name and address of
the user currently using the program.) There are many systems for
managing users, so the easiest way to support this is to add two
global variables to your program. These can be set when a user logs
in, (you will need to do that in your embed code) and then used in the
SendEmail procedure when a mail is sent.
The global variable names are
- Glo:UserEmail (String 255)
- Glo:UserName (String 255)
You can change these names if you need to - then just change where
they are used in the
SendEmail procedure.
UpdateEmailSettings Procedure
The example contains a sample procedure which just
edits a single record in a table. Of course you may have added the
fields to an existing table, in which case just use your existing form
to set the settings.
BrowseEmailLog Procedure
An example of this procedure is in the example.
This procedure is optional. It allows users to see what emails have
been sent, and if desired, resend an email that did not get through.
This procedure is a very standard browse on the table - there are no
special NetTalk extensions here.
The only bit of hand-code in the procedure revolves around the Resend
button.
SendEmail Procedure
The SendEmail procedure is a normal procedure in
your app tree, so you have a lot of control over what the window looks
like, and how it works. If you are logging emails then this is a great
procedure to add the logging code to.
The SendEmail procedure has the following prototype;
(EmailParametersGroup pEmail),Long,Proc
The parameters are set to;
(EmailParametersGroup pEmail)
the NetTalk Send Email control template has been added to the window.
Note the template settings for this, as this is where you use a
combination of the email server settings, and the parameters to
construct the email. See the example for some ideas.
The example contains an interface which makes debugging email problems
easier. It allows you to see the conversation between the program and
the email server as it occurs. The log of the most recent conversation
is also written to a text file for you. Plus of course each outgoing
email is logged to the EmailLog table.
You can of course edit this procedure to make it work, or look,
different so that it better matches your application.
Calling the SendEmail Procedure
The SendEmail procedure can be acalled from any
other procedure. It might be a button, or a menu item, or it might
happen in source code when some event occurs. To call SendEmail follow
these steps:
- Create a local variable in the procedure;
parms group(EmailParametersGroup).
- Where you want to send the email, prime the components of the
parms parameter
clear(parms)
parms.pEmailTo = 'bruce@capesoft.com'
parms.pEmailFrom = glo:UserEmail
parms.pEmailSubject = 'Great Release'
parms.pEmailCC = ''
parms.pEmailBCC = ''
parms.pEmailFileList = ''
parms.pEmailEmbedList = ''
parms.pEmailMessageTextST.SetValue('Well done on a great
release')
parms.pEmailMessageHTMLST.SetValue('<html>...</html>')
- Call the SendEmail procedure. You can do this on another thread;
START(SendEmail,25000,parms)
or on the same thread (in which case you get a returned value;
result = SendEmail(parms)
Receiving Emails
IMAP
Non-ASCII Characters
For English speaking users, who exclusively use the
Roman alphabet from A to Z there is no issue here, and this section in the
documentation can be skipped.
For non-English speaking users though, you may need to deal with non-roman
characters. You will have some choices in how you do that though, and so
those choices need to be communicated to the class so that it knows what
you are doing. The default approach though is the one most likely to be
used by your program, so it is probable that you will not need to do
anything yourself.
A number of the class properties could be subject to these foreign
characters. These include any email address list (
From,
To,
CC,
BCC),
the
Subject and of course the
MessageText
and
MessageHTML. It is assumed that whichever
approach you take to handling foreign characters, this approach is
consistent across all these properties.
The two primary approaches to encoding non-ASCII text are to either use
Unicode, or codepages (aka charsets). Since Clarion (up to version 10)
cannot display, or get user entry, in Unicode, most programs would use the
codepage approach.
Codepages (Charsets)
This is the default approach for non-ASCII
characters. The codepage property will default to the setting (
system{prop:charset})
used in your program, so it is likely there is nothing you will need to
do. If your system charset is set then the class will assume that all
outgoing properties are in that charset, and all incoming properties
need to be converted to that charset.
You are also able to set the
codepage property
yourself to any
st:CP_ value, as listed in
StringTheory.Inc.
The
CodePage property will only be used if
the
encoding property is not set to either
st:EncodeUtf8 or
st:EncodeUtf16
(see
Unicode below).
Unicode
If your email values are already formatted in
Unicode then you MUST set the encoding
property before calling the SendEmail method.
It should be set to either st:EncodeUtf8 or
st:EncodeUtf16 (depending on how you have
formatted your strings.)
SendEmail Control Template
The SendEmail Control Template lets you easily create
your own SendEmail procedure. It places a lot of fields on the window for
you, but you can obviously remove the ones you do not want to see. In the
Demo example all the fields, except the actual Send button have been
removed.
The Control template also acts as the "Glue" between the incoming
parameters, the application-wide email settings, and the SendEmail object.
The template settings should be filled in carefully.
General Tab
Object Name
Defaults to ThisSendEmail. You can change this if you like.
Base Class
Defaults to NetEmailSend. You can change this to a derived class if you
have written one.
Settings Tab
Server
Usually taken from your global setting or settings table.
Port
Usually taken from your global setting or settings table.
User
Usually taken from your global setting or settings table.
Password
Usually taken from your global setting or settings table.
Helo
Usually left blank, then the domain from the FROM address is used.
AlwaysTLS
Usually taken from your global setting or settings table.
StartTLS
Usually taken from your global setting or settings table.
CA Root File
Usually a fixed value,
'caroot.pem'. The
caroot.pem file will be copied to the
application folder for you and should be distributed with your
application.
To
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailTo.
From
Usually set to
loc:EmailFrom. This field is
calculated in code based on what the user entered, on the parameters,
and the global setting.
CC
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailCC.
BCC
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailBCC.
Subject
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailSubject.
Attachments List
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailFileList.
Embeds List
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailEmbedList.
Text
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailMessageText.
HTML
Could be the field on the Window, if the field is there, else set it to
pEmail.pEmailMessageHTML.
Show Messages
If this is on the a
MESSAGE window is
called when the email is sent (or if it fails to send.)
Change Cursor While Sending
If this is on then the cursor will change to a
Cursor:Wait
while the email is being sent.
Suppress Error Messages
If this option is on then unexpected error messages will not appear in a
MESSAGE window.
EmailParametersGroup
A group structure has been defined to easily move
multiple fields between procedures. Usually this is used when calling the
SendEmail procedure. You don't have to populate all the fields in the
group, the ones you don't need can be left blank.
| Name |
Type |
Description |
| pEmailFrom |
String(4096) |
The from address of the person sending the email. |
| pEmailTo |
String(4096) |
A semi-colon separated list of email addresses that the email is
going To. |
| pEmailSubject |
String(4096) |
The subject line of the email. |
| pEmailCC |
String(4096) |
A semi-colon separated list of email addresses who will receive
a cc of the mail. |
| pEmailBcc |
String(4096) |
A semi-colon separated list of email addresses who will receive
a bcc of the mail. |
| pEmailFileList |
String(8192) |
A comma separated list of files to attach to the email. |
| pEmailEmbedList |
String(8192) |
A comma separated list of images that have been embedded in the
email. (If left blank this will be constructed by the SendEmail
object as long as the images are accessible.) |
| pEmailMessageText |
String(100000) |
The "text" contents of the email itself. |
| pEmailMessageHtml |
String(100000) |
the "html" contents of the email itself. |
| pEmailSettingsPath |
String(255) |
not used |
| pEmailTest |
Long |
Allows the SendEmail procedure to differentiate between a "test
mode" and "real mode". Not used by the object, but supplied for
your own use in your SendEmail procedure. |
| pHelo |
String(255) |
Passed to the Helo property of the object. this is passed to the
server as the EHLO command. Some servers require this. |
| pSSL |
Long |
Set the connection to be Always TLS. (Formerly known as Always
SSL). This is NOT usually used if the StartTLS command is
used. Typically set in SendEmail from a global setting, but this
allows it to be overridden by the caller. |
| pStartTLS |
Long |
Indicated the connection will start as plain, but switch to TLS
after connecting. Some servers prefer this approach, some prefer
AlwaysTLS. Typically set in SendEmail from a global setting, but
this allows it to be overridden by the caller. |
| pSign |
Long |
Digitally signs the email if the
necessary certificate is installed. |
| pHide |
Long |
Indicates to the Email Sending window if it should be visible or
not. |
| pServer |
String(255) |
Typically set in SendEmail from a global setting, but this
allows it to be overridden by the caller. |
| pPort |
Long |
Typically set in SendEmail from a global setting, but this
allows it to be overridden by the caller. |
| pEmailReplyTo |
String(255) |
The Reply To address (if different to the From address) |
| pStmpUser |
String(255) |
Typically set in SendEmail from a global setting, but this
allows it to be overridden by the caller. |
| pPassword |
String(255) |
Typically set in SendEmail from a global setting, but this
allows it to be overridden by the caller. |
| pSSLMethod |
Long |
Allows you to override the default allowed TLS versions for the
connection. See Levels. |
| pEmailFileNameList |
String(8192) |
|
| pEmailImagePath |
String(255) |
|
| pUserData |
StringTheory |
Not used by NetTalk. Added as a convenience to allow you to pass
in other data to the SendEmail procedure. Use pUserDataString
below in preference to this if starting a new thread. |
| pUserId |
String(255) |
Not used by NetTalk. Added as a convenience to allow you to pass
in other data to the SendEmail procedure. |
| pOAuth |
StringTheory |
(deprecated) - use pToken instead |
| pToken |
String(4096) |
An OAuth Token to be used to authenticate the user. Use this
over the pOAuth property above, as it is easier to pass a string
in a group when starting a thread. |
| pUserDataString |
String(1024) |
Not used by NetTalk. Added as a convenience to allow you to pass
in other data to the SendEmail procedure. Use this in preference
to the pUserData above if starting a new thread. |
|
|
|
OAuthAuthentication
Some email systems, (notably Microsoft Office365 /
Outlook,) are moving away from SMTP authentication (user & password)
to OAuth Authentication (user & token). In order to understand how
this works it's necessary to be familiar with OAuth (see
here
for details on doing OAuth through NetTalk.)
The idea is that the OAuthLogin procedure provides a token - and this
token is then used in the EmailParametersGroup. The SendEmail procedure
will then use this token to complete the authentication.
While in concept simple (once you have the OAuth sorted out) the details
can be quite specific. Some hints are included below for specific email
providers. These hints are accurate at time of writing, but can be changed
by the service, so refer to your service documentation as well
Office365 / Outlook
- When registering your app with Microsoft, and setting the
permissions required (ie the "scope") look under the Microsoft Graph
API, and allow permissions on SMTP.Send and (if necessary for
receiving) POP.AccessAsUser.All.
- Email server is at outlook.office.com, port 587, StartTLS, TLS 1.2
or higher. (Email server at smtp.office365.com also appears to
work.)
- The Scope setting (when doing the OAuth Login) must be one (or
more) of
- IMAP:
https://outlook.office.com/IMAP.AccessAsUser.All
- POP:
https://outlook.office.com/POP.AccessAsUser.All
- SMTP:
https://outlook.office.com/SMTP.Send
- The Authorize URL (when doing the OAuth Login) must be
https://login.microsoftonline.com/common/oauth2/v2.0/authorize
- The pUser and Token field are passed to the SendEmail procedure in
the SendEmailParameters group. The pPassword field should be blank.
Constructing HTML Emails
Text Version
Some mail readers, by design or user preference, do
not display the HTML part of an email. So even if you populate the MessageHTML property, you should also populate
the MessageText property when creating an
email. If the client does not display HTML they will display the MessageText part.
HTML Version
Different mail clients use different HTML rendering
engines to display emails. Notably Outlook uses the Microsoft Word
engine (which is not as advanced as the engine used by IE.) Some clients
will remove any links to external CSS, and even CSS collected into a
<style> section.
For this reason the HTML used in emails needs to be very simple, and
very "old school". If you do send more complex HTML then testing is
definitely your friend.
There are numerous posts on the internet regarding best practices in
this space - some recommended ones are;
Embedded Images
When constructing the HTML it is often desirable to
embed images in it. This is usually done using the HTML
<img>
tag. For example;
<img src="https://capesoft.com/images/logo.png"
>
This embeds a link to the image in the HTML. The image itself is not
included in the HTML. Rather the image is fetched from the server when
the user opens the email
[1].
Unfortunately many users have images of this sort turned off in their
email client, so the user never fetches, and never sees, the image.
There is a way though to embed the image in the HTML so that it is
included in the email itself, and no call to the web server is required.
You need to create an
<img> tag,
setting the source to a local resource. For example;
<img src="c:\images\logo.png" >
or
<img src="logo.png" >
If the HTML is constructed in this way, then you need to make a call to
the
ProcessHTMLForImages method;
thisEmail.ProcessHTMLForImages(rootfolder)
The
Rootfolder parameter allows you to
specify a location on the disk. This location will be used by any images
which do not have a path set (such as
<img
src="logo.png" >)
This method should be called after the
MessageHTML
property has been set, and before the call to
Send.
[1] The use of external images
in an email message is often used my marketing folks to track when an
email was read. This has the side effect of validating email addresses
for spammers. For this reason most email clients are set to not load
external images by default. Since most users leave the default as is, or
even explicitly turn images off, this approach of using external images
has largely fallen out of favor.
Digitally Signing Emails
When you send an email to a person, you can digitally
sign it. If you do this, and the client email program the receiver is
using supports digital signatures, then they will be able to verify that
you sent the email, and the content of the email has been unaltered since
you sent it.
Note that a Digital Signature is different to the simple Signature option
that many email clients provide - the feature that adds simple text to the
end of your emails. That is a different thing, and completely unrelated to
Digital Signatures.
Digital Signatures require that you acquire and install a special kind of
certificate, known as an Email Certificate, on the sending computer. This
certificate is specific to the FROM address that you will be using to send
your emails from. These certificates are widely available and are usually
free. For example
Comodo offers a free certificate, but
Googling offers many possible choices.
Once you have acquired your certificate it needs to be imported into the
Windows Certificate Store on the sending computer. You can run the
Certificate Manager by running the
CertMgr.Msc
program. Go to the
Personal / Certificates
area, right-click on the
Folder and
Select All Tasks / Import from the menu. Follow
the wizard to import the file. (Note that some web pages for acquiring the
certificate install the certificate into the store for you automatically,
so this step may not be needed.)
Certificates do expire, typically after 12 months, so you will need to
renew them every year.
To digitally sign an email, when sending, set the property
OptionsSigned
to one of
Net:NoSign (the default),
Net:Sign or
Net:TrySign.
If set to
Net:Sign, and the certificate does
not exist, then the mail will not get sent. If it is set to Net:TrySign,
and the certificate does not exist, then the mail will be sent
unsigned. For example;
net.OptionsSigned = Net:Sign
While there are other properties you can set at this point, related to
digital signatures, it is unlikely that you will want to change them from
their default values. These options, and default values are;
| Property |
Default Value |
| OptionsSignedProtocol |
application/x-pkcs7-signature |
| OptionsSignedMicAlg |
sha256 |
| OptionsSigName |
smime.p7s |
In some cases (if the client is unable to process a SHA-256 hash) it may
be necessary to change the
OptionsSignedMicAlg property
to 'sha1'. However this should only be done if problems with old email
clients is encountered.
The FROM address for the email is automatically used to identify the
certificate to use. If an appropriate certificate is not found, and
OptionsSigned is set to
Net:Sign, then
ErrorTrap will be
called and the email will not be sent. If
OptionsSigned
is set to
Net:TrySign then the email
will be sent unsigned.
FAQ
- Gmail : Enable Less Secure Apps
- Secure Email Stopped Working
- Someone just tried to sign into
your Google Account....
- 550 Relaying Prohibited
- External Exchange Failure
- EML File
- Connecting to Yahoo Mail
Gmail : Enable Less Secure Apps
Secure Email Stopped Working
My email program, which has been working for a
long time, using an old build of NetTalk is now getting errors and is
unable to send. The error number was -69 which means the SSL Remote
Certificate Failed Verification.
If you are connecting to a server via SSL (specifically SSL v3) then you
may encounter this problem. SSLv3 was the default level for SSL used for
earlier versions of NetTalk. SSL v3 was however broken in October of
2014, and was declared deprecated by the Internet Engineering Task Force
(IETF) in
RFC
7568 in June of 2015. As a consequence of the attacks in October
2014 many servers immediately moved to stop supporting SSLv3 and this
was accelerated by the RFC in June 2015.
The solution is to change the connection your program uses from SSLv3 to
TLS. If you are on a recent NetTalk version (NetTalk7 or later) then
make sure you are on the latest build and this should be done for you
automatically. (The internal default changed from SSLv3 to TLS in build
8.30 and 7.40)
If you are using an older NetTalk, or if you have specifically set the
SSLMethod property for a class (so it is not just the default value)
then you will need to set the property manually. In your code add;
net.SSLMethod =
NET:SSLMethodTLSv1
If you are on NetTalk 8 or later then use the
NET:SSLMethodTLS
equate as this covers all TLS versions, and defaults to the highest
level of security.
If you are using NetTalk 7 then you can use
NET:SSLMethodTLSv1, NET:SSLMethodTLSv1_1
or
NET:SSLMethodTLSv1_2 equates.
NetTalk 6 and earlier does not support anything other than
NET:SSLMethodTLSv1.
Warning: It is probable that at some
point servers will turn off support for TLSv1 at which point programs
earlier than NT build 7.40 will cease working. An update to NetTalk 8
(or later) is thus recommended to avoid problems when this happens.
550 Relaying Prohibited
"I am getting 550
Relaying Prohibited Errors when trying to send
email. What's going on?"
Relaying problems are all to do with your Email Server. Basically it
doesn't like who you are sending emails
From. There
are a couple of things you can do to try and resolve this:
- Authenticate with the server using the AuthUser and Password
properties
- Make sure the FROM address you are sending from is a valid one,
one which the server accepts.
External Exchange Failure
"I am using Exchange Server. Internal email
sent from NetTalk works fine, but my external emails are never
delivered."
Go to Internet mail service properties - routing tab - switch "reroute
incoming smtp mail" to on.
EML file
"How do I create an EML file when sending
emails"
The WholeMessage property contains the
email in EML format. So you can just save this property as a file. This
is easy with StringTheory.
For example;
str StringTheory
str.SetValue(net.WholeMessage)
str.SaveFile('something.eml')
Connecting to Yahoo Email
Thanks to Ray Rippey for providing the following
information.
Sign in and go to your Account security page.
Click Generate app password or Manage app passwords.
Select your app from the drop down menu and click Generate.
Follow the instructions below the password.
Click Done.
Use this app password (as Password) and your email address (as User) to
sign in to your email app.
Class Reference
NetEmailSend
NetEmailReceive
NetIMAP
[End of this document]
