- General Information
          
        
- Designing your SQL Database
          
        
- In Depth
          
        
- Other Information
          
        
- Can't find what you're looking for?
          
        
 
    Introduction (essential
        reading when converting from TPS to SQL)
     Using SQL with Clarion is not difficult, but there are
      some things to know before you start. If you have no SQL experience, then
      it is advised that you start with some basic tutorials to get yourself a
      little more familiar with SQL. In terms of using SQL with Clarion, there
      are often some mindsets that might need to be broken, as Clarion can be
      very forgiving in terms of database design and implementation with
      Flat-File systems such as TPS. Once you are familiar with some of these
      concepts, implementing SQL in Clarion is not difficult. FM3 does
      everything it can to make the conversion process as quick and easy as
      possible. There are however some added things to know when using FM3, as
      it sometimes needs a little more information from you regarding the
      backend it is working with. All these things are discussed below.
      File formats / databases:
      Firstly, it is important to understand some of the fundamental differences
      between using an SQL backend as oppose to a Flat-File format.
      
Flat-files
      
        - TopSpeed
- Clarion
- Btrieve*
Client - Server
      
        - Btrieve*
- SoftVelocity’sIP Driver
      This is essentially a layer on top of flat files. Server handles /
      controls access to the files.
      
SQL / Relation Database Management System (RDMS)
      
        - MsSQL
- Oracle
- FireBird
- MySQL
- PostgresSQL
- PervasiveSQL*
- Sybase
      This is essentially a layer on top of Client-Server. Data access and
      manipulation is done using Structured Query Language (SQL). SQL Databases
      are generally described as relational database management systems.
      
      * The Btrieve driver can be Client Server or Flat File. Pervasive have
      also built an SQL layer to this file format.
      
Choosing your Database / File Driver:
      Currently, we only discuss the pros and cons of Supported File Drivers and
      Databases in FM3.
      
        
          
            | Driver | Pros | Cons | 
          
            | BTrieve | Free | Limited to a 4K record size, Does not return helpful error
              information for FM3. | 
          
            | Clarion | 
 | Prone to corrupt, unbuilt keys and data files. | 
          
            | Topspeed | Free, Good Performance | Prone to corrupt files. | 
          
            | MsSQL | Good performance, Stored Procedures and Triggers, Good name,
              Tried and tested | Fairly Expensive, Regular security patches | 
          
            | Oracle | Good performance, Stored Procedures and Triggers, Good name | Very Expensive, Needs a qualified DBA, i.e. a bit difficult for
              the untrained to manage. | 
          
            | MySQL | Good performance | Not Free | 
          
            | Firebird | Good Performance, Optionally Embedded, Stored Procedures and
              Triggers, Free | Key size limitation of 254K under default characterset, Keys
              cannot have mixed sort orders, but this can be easily worked
              around. | 
          
            | Sybase/ASA | Good performance, Stored Procedures and Triggers, Good name,
              Tried and tested | Fairly Expensive (although cheaper than MSS) | 
          
            | PostgreSQL | Good performance, Stored Procedures and Triggers, Free | Text based variables don't handle non-printable characters.
              Requires binary strings to be set to BINARY in the dct. | 
        
      
    Clarion / SQL Fundamentals
      (essential reading when converting from TPS to
        SQL)
    
      Questions to ask before you start:
      Who is Master?
      The Clarion Dictionary
      
      or
      
        
        SQL Database Backend
      
      The very idea of File Manager 3 implies that the Clarion Dictionary is the
      master of the backend structure. In other words, you make changes in the
      dictionary, and FM3 makes the relevant changes to the backend. In SQL
      circles this could be severely frowned upon, especially if there is an
      existing database, and / or a DataBase Administrator. Read on for further
      discussion on this...
      
Who is in charge of the Database backend?
      You, the developer
      
      or
      
      
      
Some DBA or other 3rd party
      
      In some SQL environments, you the developer are not permitted to change
      anything on the backend, especially without permission. If there is a
      Database Administrator, or another person in charge, they are not going to
      like the idea of FM3. In circumstances like this, it may not be the best
      solution to use FM3.
      
Do I have an existing database?
      Existing TPS
      
      or
      
        
        Existing SQL
      
      If you have an existing TPS database, then you'll need to read the
      documentation on 
Converting TPS to SQL. If you
      have an existing SQL database, and are applying FM3, then you may find
      that FM3 will want to make some changes to the structure, even if nothing
      has changed. To understand more of this, read the 
Datatypes
      section. The Clarion synchronizer / table import in the dictionary does
      not necessarily pick the best matched datatypes for your app, and again,
      treats the backend as master instead of the dictionary.
      
Am I converting from Flat-files (eg: TPS)?
      Converting Existing Data 
      
      As mentioned above, FM3 makes this a breeze, however, you must follow the
      instructions laid out in the 
Converting to SQL
      section.
      
Design
      ALL tables must contain a PRIMARY Key!
      Good design suggests each table must contain a unique record identifier,
      commonly known as a primary key. This should ideally be a single field key
      of integer type. You may choose to auto-increment this field, or set it at
      insert time. However, nulls and zero should not be allowed. While it is
      not illegal to use other datatypes as your primary key, a numeric datatype
      often provides the fastest data access to the table. This is also helpful
      when declaring relationships. It is also not illegal to use
      multi-component keys as your primary key, but this can also be untidy and
      slow things down. In other words, create a simple fast primary key, and
      declare other indexes for all your other purposes. A Globally Unique
      Identifier (GUID) is most often a string / char datatype, and while this
      could be a primary key, it is recommended you rather declare it as a
      separate unique key. This means relationships can still be declared on the
      Primary Key of the parent table. If you are using CapeSoft's
      
        Replicate, and indeed any other type of replication, you may
      required to use a multi-component Primary key. For example: PK_Customers
      key(CUS:Site,CUS:CustomerID).
      
Ensure all Primary, Unique, and Foreign keys are uniquely named
        throughout the database.
      Different flavors of SQL don't allow duplicate names for certain objects.
      For example, TPS users may like to name all there primary keys
      "PrimaryKey". This will fail in most SQL backends. A good naming
      convention is to use the tablename, or the fieldnames as part of the
      keyname, e.g.: PK_Customers, FK_Countries, LastNameKey etc. You could also
      add the file prefix to the external name attribute to ensure uniqueness,
      e.g.: CUS_LastNameKey.
      
Ensure that all object names do not use Reserved Keywords.
      Be sure not to name your tables, fields, and keys with reserved SQL
      keywords, such as "Date". If you are converting from an existing
      Dictionary, then you can effectively get around this by using the Full
      Path Name, and External Name attributes to declare non reserved names. A
      good suggestion is to use your file prefix and an underscore, e.g.
      "PRE_Date".
      
      
There's no such thing as "Exclude Empty
        Keys" in SQL
      You need to turn this dictionary key option off for all your unique keys.
      This is good practice in flat-files as well.
      
      This option in the dictionary editor is such a bad idea. Seriously. It
      causes so much heartache when converting TPS data to SQL. In SQL
      nullability is on the _field_ not the key. In tps, blank fields (no such
      thing as null in TPS) are excluded from the key table (which is like a
      separate file within the tps file). In SQL, there is no option to exclude
      elements of a key from that key if they are blank or null. So nulls/blanks
      are included in the key. If it is a single element key, then you can only
      have one null and one blank (or zero - not the same as null in SQL) value
      in a key.
      
      So when you come to exporting your data into SQL, you need to either: Make
      the key non-unique, or remove records that cause a duplicate key error
      (preferably before exporting to SQL). The easiest way is to make the key
      non-unique - as there are essentially (because of that confounded "Exclude
      Empty Keys" switch in the dct) duplicates in the key - although these
      "duplicates" were permitted in TPS.
      
In SQL case-sensitive sorting is on the Field level
      In the clarion dct, case-sensitivity is defined in the key - but
      (depending on the default collation of your SQL database) - sort orders
      are on the field. This is called the field collation. In order to maintain
      an efficient sort order (or to maintain uniqueness across a key that is
      case sensitive) you need to tell FM3 which fields you need to collate in
      case-sensitive order. See the
      
CaseSensitive
      field user option in the main FM3 docs for details on how to do this.
      
Define Foreign Keys on Integer type fields.
      If you have followed the recommendation for Primary Keys, then this should
      fall into place nicely. Again, performance will take a hit if you use
      character type data as your foreign key links. Also, this keeps it clean
      and simple. For example, if you have a Customer table that relates to a
      Countries table. Build it something like this:
      
      Code example:
      CLARION: 
      
        
          
            | Customers |  | file,pre(CUS),driver('MSSQL') | 
          
            | PK_Customers |  | key(CustomerID),primary,nocase | 
          
            | FK_Countries |  | key(CountryID),nocase |  | 
          
            | record |  | record | 
          
            | CustomerID |  |  | long |  | 
          
            | CountryID |  |  | long |  | 
          
            | LastName |  |  | string(40) |  | 
          
            | FirstName |  |  | string(60) |  | 
          
            |  |  | end |  |  | 
          
            |  |  | end |  | 
        
      
      
      
        
          
            | Countries |  | file,pre(COU),driver('MSSQL') | 
          
            | PK_Countries |  | key(CountryID),primary,nocase | ! Links to Customers
                FK_Countries | 
          
            | record |  | record | 
          
            | CountryID |  |  | long |  | 
          
            | Country |  |  | string(100) | 
          
            |  |  | end | 
          
            |  |  | end |  | 
        
      
      
      One of the biggest benefits of this style of design is that you should
      never ever need to alter those fields - i.e., the primary and foreign key
      fields. This means FM3 can easily and safely upgrade the files, even when
      your relationships reside on an SQL backend. This also ensures data
      integrity.
      
Recommendations and Tips
      The Owner string
      
        - Use a global variable for this attribute. In the dictionary, place
          an exclamation mark in front of the variable name: e.g.: !GLO:dbOwner
- For FM3, use a String, not a CString for your owner variable.
- Use the FM3:Connect To SQL Backend
          extension template for creating a connect procedure for priming your
          owner variable.
The SQL Database
      
        - You do not need to create the tables on the SQL backend. FM3 will do
          this for you.
- You need to create the database on the SQL Backend, and setup a
          user.
- The SQL User login must be the dbowner of the database (MsSQL).
You don't need to worry
        about this next bit as it's now taken care of in the DLL - unless you
        don't want your application to use a sysadmin login. If you don't want
        your application to use SysAdmin rights, then you must create the linked
        server manually, and then login with a user with dbOwner rights.
      For complete automation of FM3 administration, the user should also be a
      member of the SysAdmin role (MsSQL). This is, however, not necessary if
      you can create the linkedserver manually as follows:
      
      FM3 needs to perform administrative tasks on the SQL backend. It therefore
      needs some level of administration rights. These rights can be limited to
      dbowner by completing the following configuration of the MsSQL Server. Two
      tasks must be completed on your MsSQL Server for FM3 to continue it's job.
      The server must be a linked server, and the server option DATA ACCESS must
      be set to true.
      
      
To Add the MsSQL Server as a Linked Server
      
        - From your SQL Query Analyser, execute this script: 
 EXEC sp_addlinkedserver
            '<YourSQLServerName>','SQL Server' where
          <YourSQLServerName> is the name of your SQL Server.
- For more information, see the Microsoft SQL Server Books Online.
To set the DATA ACCESS option to TRUE
        - From your SQL Query Analyser, execute this script: 
 EXEC sp_serveroption
            '<YourSQLServerName>', 'DATA ACCESS', 'TRUE' where
          <YourSQLServerName> is the name of your SQL Server.
- For more information, see the Microsoft SQL Server Books Online.
 
    SQL Data Types
      (essential reading when converting from TPS to
        SQL)
    
      Dates and Times
        Memos
        Reals
        Arrays (Dimensioned fields)
        Dynamic and Static Indexes
        Oracle DataType
          Matching
        Large String Data
        MSSQL - UniqueIdentifier
        Binary strings (see the
        PostgreSQL section of the docs)
        Computed fields
      Dates and Times
      Did You Know?
      Some of the SQL drivers DO NOT directly support the DATE and TIME
      datatypes in Clarion. In some drivers DATE may be used, but there are some
      things to watch out for and think about when using DATEs and TIMEs.
      
      
Note: Sybase (ASA), PostgreSQL, Firebird,
      MSSQL 2008 
do support the native DATE and TIME
      datatypes, so you won't need to follow the below mentioned suggestions.
      Also MSSQL 2005 allows a DATE to be mapped directly to a datetime (and
      likewise a time) - so you can use those as is.
      
What to do:
      Here are 4 options for Dates and Times. The bottom line is there are pros
      and cons to all the options! FM3 supports all 4 options. These points are
      provided for your information to help you make the best design choice for
      your scenario!
      
      
Option 1: Use a GROUP over STRING(8) containing a DATE field and a
        TIME field (only required for MySQL, Oracle and MSSQL users ).
      
      MSSQL users: You'll only need to use this option if you find yourself in
      one (or more) of the following situations:
      
        - Using MSSQL 2000 or previous.
- Your Date and/or time is part of a unique key (or your TIME is part
          of any index).
- You have other applications (probably non-Clarion) using the
          database that require a datetime (with both portions populated).
      This is the default import behaviour for most SQL date datatypes into a
      Clarion dictionary. This is all very well, but nasty if you're converting
      an existing app to SQL. Apart from going and updating all the structures
      in the dictionary, you'll have to make changes to your code within the
      program to handle the new structure, and process the data as required.
      Though you can still set up individual date or time key, a date+time key
      will not be possible! The up side is that by using the standard backend
      date, other apps will also be able to use the same info. The fact of the
      matter: if you have to build an app for an existing database with existing
      data, you're going to have to go with this structure, and handcode any
      necessary processing!
      
      
NB: You'll also need to add the following
      user options as shown for the specific fields. These are constants and you
      can add them as they appear in the table below. BDE will add these
      useroptions in when doing the conversion.
      
      
 Code example:
 
      
      Code example:
      
      CLARION:
      
        
          
            | 
 | Example Field Name | Data Type | UserOption to add | 
          
            |  | MyDateTime | string(8) | UnRealField | 
          
            |  | MyDateTime_Group | group,over(MyDateTime) | 
 | 
          
            |  | MyDateTime_Date |  | date | RealField | 
          
            |  | MyDateTime_Time |  | time | RealField | 
          
            |  |  | end | 
 | 
        
      
      
      SQL:
 (mssql)
      
      
      Option 2: Use a DATE and a TIME datatype.
      
      In MSSQL 2005 and up (have not tested all backends), you may match a DATE
      to a DATETIME, and a TIME to a DATETIME. For DATE, it is happy to leave
      the time portion = 0, but in TIME, the date portion defaults to today's
      date. This will not make keys and indexes very useful! Other applications
      accessing your data will also need some explanation.
      
      
Code example:
      
      CLARION:
      
      
      SQL: (mssql)
      
        
          
            |  | MyDate | datetime | 
          
            |  | MyTime | datetime | 
        
      
      
        Important Note: This option should only be used for DATEs, not
      TIMEs. If used for TIME (without the DATE portion) - you will be plagued
      with the "Another user updated this record" error. If you have single
      TIMEs then rather create the group, over and an extra DATE field. You can
      then initialize the DATE portion to 0 so that all your local indexes
      (using the TIME field) will be consistent. This is not an FM3 issue, but a
      Clarion limitation.
      
      
Option 3: Use a LONG datatype.
      
      Well, every Clarion programmer will jump for joy at this idea, ... BUT, is
      it really a good solution? Well, you would have to analyze the situation
      completely, and especially think of what may come up in the future! For
      example, if you are building an app that uses it's own database, and no
      other frontend apps or tools look at it, or care what's in it, then you
      have no problem. But, what about those "likely-to-happen" scenarios, like
      building a dynamic website from the data, or an intranet? Of course, if
      you're in charge of it all you can build those with Clarion, and you have
      no problems! But, if a VB guy ever came along and needed to use the data
      for his application, oh boy - I guess at this point you could start a "Why
      Clarion is the best RDA tool" argument! Otherwise, if you, and only you
      (or any other Clarion programmer) are going to be using the data - joy joy
      and more joy - keys are a breeze, date calculations are a breeze,
      formatting display is a breeze, and it's only 4 bytes of data each!
      <sigh>I love Clarion!</sigh>
      
      
Code example:
      
      CLARION:
      
      
      SQL: (mssql)
      
        
          
            |  | MyDate/font> | int | 
          
            |  | MyTime | int | 
        
      
      
      Note: FM3 now optionally creates a utility
      MSSQL Stored Procedure (
ds_ConvertClarionDateTime)) for
      converting your Clarion date time integer values to a valid mssql
      datatype! This will benefit other applications using your data, by
      allowing them to retrieve the datetime datatype version of your data. 
Click here for more info.
      
      
Option 4: Use a STRING or CSTRING datatype.
      
      Hmm, sounds like a reasonable suggestion? Let's think about it. For
      indexing purposes you are forced to use a "yyyymmdd" and "hhmmss" format
      (you could of course use "-" or ":" separators). No problem, but if you
      need to do calculations, you'll have to deformat all the strings first -
      so, maybe not such a big deal, but remember any other programs using the
      data will also have to deformat it for their use, and they don't
      necessarily have the blessing of a DEFORMAT function! It also uses more
      space than LONGs - 8 bytes for date and 6 bytes for time (and more if you
      use the separators) instead of 4 bytes each! 
      
      
Code example:
      
      CLARION:
      
        
          
            |  | MyDate | string(8) | ! or cstring(9) | 
          
            |  | MyTime | string(6) | ! or cstring(7) | 
        
      
      
      SQL: (mssql)
      
        
          
            |  | MyDate | char(8) | /* or varchar(8) */ | 
          
            |  | MyTime | char(6) | /* or varchar(6) */ | 
        
      
      Memos
      Did You Know?
      Memos are NOT directly supported in Clarion's SQL drivers.
      
What to do:
      Either:
      
      Use the ForceSQLDataType = Text 
field
        user option
      (if the backend you are using supports the text character or equivalent,
      which is an equivalent for a MEMO).
      
      Or:
      
      Convert all your MEMOs to STRINGs or CSTRINGs.
      
Reals
      Did You Know?
      The native MSSQL driver does not directly support the REAL datatype.
      
What to do:
      Leave it as REAL. For Oracle, change to SREAL.
      
      If you import a float from MSSQL to the dct, it generates a Clarion REAL.
      For this reason, FM3 handles the creation of a float datatype in MSSQL.
      
Arrays (Dims)
      Did you know?
      The native SQL drivers do not directly support the DIMs. Prior to
      upgrading to SQL, you must first convert your DIMmed fields to an array
      over a group in your last TPS version. In your SQL version, wherever you
      use the array in code, you need to either declare a DIMmed variable over
      the group, or re-code to use the new fields in the group. The idea is that
      your DIM fields are overed over a group of values in the TPS version. FM3
      handles this structure. 
      
      
Code example:
      
      CLARION:
      
      
        
          
            |  | MyDim_Group | group | 
          
            |  |  | MyDim_1 |  | string(20) | 
          
            |  |  | MyDim_2 |  | string(20) | 
          
            |  |  | MyDim_3 |  | string(20) | 
          
            |  | MyDim | string(20),dim(3),over(MyDim_Group) | 
        
      
      
      SQL:
      
        
          
            |  | MyDim_1 | char(20) |  | 
          
            |  | MyDim_2 | char(20) |  | 
          
            |  | MyDim_3 | char(20) |  | 
        
      
      
      Dynamic and Static Indexes
      Did you know?
      The Clarion native SQL drivers do not support Dynamic and Static Indexes.
      
What to do:
      Change them to keys, or delete them.
      
      If you have any dynamic or static indexes defined in your dictionary, you
      will need to delete them, or change them to keys.
      
      
Oracle
        DataType Matching
      Did you know?
      Clarion Datatypes do not match Oracle Datatypes directly.
      
What to do:
      Follow the advice and recommendation below.
      
      This may be obvious for most SQL datatypes, but in Oracle we have found it
      to be significantly different. For example, if you import an oracle table
      into your dictionary, you may get a lot of PDECIMAL fields you weren't
      expecting. This is because, depending on who is master, the datatype must
      be "big enough" for the other end! i.e.. The max value contained in a
      Number(3) can be 999, but that's to big for a BYTE which can has a max of
      255, so a SHORT is declared. But, going the other way around, the max
      value of a SHORT is 32,767 which is too big for a number(3), so Number(5)
      is created! With FM3, we needed a rule, so we don't go around in circles
      converting datatypes endlessly! We decided that in FM3's case, the
      Dictionary is the master, and therefore the backend must be "big enough"
      to handle any data passed from the frontend. Here a couple of tables
      showing the import and create matches:
      
      Table import from Oracle:
      
        
          
            |  | Number(1) | byte |  | 
          
            |  | Number(2) | byte |  | 
          
            |  | Number(3) | short |  | 
          
            |  | Number(4) | short |  | 
          
            |  | Number(5) | long |  | 
          
            |  | Number(6) | long |  | 
          
            |  | Number(7) | long |  | 
          
            |  | Number(8) | long |  | 
          
            |  | Number(9) | long |  | 
          
            |  | Number(10) | pdecimal(11) |  | 
          
            |  | Number(11) | pdecimal(11) |  | 
          
            |  | Number(12) | pdecimal(13) |  | 
        
      
      
      FM3's creation to Oracle:
      
        
          
            |  | byte | number(3) |  | 
          
            |  | short | number(5) |  | 
          
            |  | long | number(10) |  | 
          
            |  | pdecimal(x) | number(x) |  | 
          
            |  | pdecimal(x,y) | number(x,y) |  | 
        
      
      
      Large String Data
        (greater than record or datatype limit)
      Did you know?
      Most SQL databases have a record size limit. In some circumstances, this
      can be changed. Please see your SQL Documentation.
      
What to do:
      Follow the advice and recommendation below.
      
      As with Clarion, large data is not stored within the record. The drivers
      do not support Memos, but FM3 will automatically create TEXT fields in
      MSSQL and CLOB fields in ORACLE to handle data larger than their
      respective record size limits. Please note, you will need to use STRINGs
      or CSTRINGs in the Clarion definition. At the moment FM3 does not take the
      whole record size into account. It will only apply TEXT and CLOB to
      individual STRINGs or CSTRINGs greater than 8k and 4k respectively. 
      
      
Code example:
      
      CLARION:
      
        
          
            |  | MyLargeString | cstring(10000) |  | 
        
      
      
      SQL:
      
        
          
            |  | MyLargeString | text | (mssql) | 
          
            |  | MyLargeString | clob | (oracle) | 
        
      
      MSSQL -
        UniqueIdentifier
      Did you know?
      FM3 supports the MSSQL datatype - uniqueidentifier. SQL Server does not
      allow this field to be renamed (after it is created) - so you need to name
      this field correctly the first time that it is added to your dictionary.
      
What to do:
      
        - Your field must be a cstring 37 in your dictionary.
- You need to enter the following in your user options for the ROWGUID
          field:
          
            
              |  | ForceSQLDataType | uniqueidentifier ROWGUIDCOL
                  NOT NULL DEFAULT NewID() |  | 
          
        
        
        In theory you can allow nulls in a ROWGUID field, but invariably you
        would want to make this part of a constraint (normally primary) of the
        table, in which case you will not be permitted to use NULL values. If
        you would like to allow NULLs in the ROWGUID, then you can enter the
        following instead:
        
        
          
            
              |  | ForceSQLDataType | uniqueidentifier NULL
                  ROWGUIDCOL DEFAULT NewID() |  | 
          
        
        - You need to add the | READONLY attribute to the field's external
          name in the dictionary. If you do not do this, you will get a 22005
          error when inserting a record into the SQL table. 
 
 NOTE: You must use a fully
            qualified external name as well as the READONLY attribute. You
            cannot simply add the | READONLY attribute to the external name
            field by itself. This will result in an error 47 when trying to use
            the file. You will then by unable to rename the ROWGUID field, as a
            ROWGUID field is not permitted to be altered.
Image and Text types vs Binary data
      In TPS files, all string data is stored as binary data. Each byte can
      contain any value from 0 to 255. In SQL though, all datatypes can be
      categorized into either binary or text type data types. This means that
      strings can be correlated to chars 95% of the time, as they will contain
      valid text data (like addresses, names, descriptions, etc.). However, in
      flat-files, strings can also be used to contain binary data (non-printable
      characters) – which makes them non-compatible with text based data types.
      Where strings are used in TPS files for storing binary data, these fields
      must be converted to binary based variable type in order to allow the SQL
      backend to store this data correctly.
      
      You need to identify each variable that contains binary data and force the
      data type to convert to binary. On the Field User Options for those
      fields, enter:
      
      ForceSQLDataType                         binary
      
      A Caveat with this approach: Clarion does not handle binary data in
      browses very well. An alternative approach to this (depending on your
      backend - PostgreSQL supports this) - is to use a larger stringsize in
      your dictionary, because PostgreSQL will store non-printable characters in
      a text datatype using more than one byte (up to 3 bytes for some
      characters). If you have a string(20) in your dictionary, then restrict
      the data entry to 20, bump the size up to 60 and this will be a better
      working solution in PostgreSQL.
      
      
Note: GUIDs will be handled later on – so
      leave these as they are at this stage.
      
Computed Fields
      If you require a computed field on the backend, and you would like to use
      the computed field in your application, then you'll need to add that to
      your dct. You'll need to use the ForceSQLdatetype field user option to set
      this up so that FM3 creates the field correctly, and does not try to alter
      it (once created):
      
      ForceSQLDataType                         AS <expression>
      
      Then in the External name field option:
      
      External Name                                Balance Due | READONLY
      
      Where <expression> is an expression using the names of the fields
      required for the calculation. Note: The calculation field should be added
      after the other fields (used in the calculation) have been added.
    
 
    Field Management(essential
        reading when converting from TPS to SQL)
     The standard behaviour of FM3 follows the Clarion
      default which implies you can have fields in your SQL tables that don't
      exist in the dictionary. Therefore, by default FM3 does not remove any
      fields and keys. This ensures no valuable data is lost. If you would like
      FM3 to ensure the backend matches your dictionary (i.e., the Dictionary is
      the Master of the fields), use the DctMasterFields user option.
      
      This is normally not necessary (with FM3), as your file will not error 47
      if there are extra fields or fields with longer dimensions (like if you
      change a string(200) to a string(100)) - but some folks like to keep it
      this way. The downside with this is that if other parties use the database
      (or the clients build in server side triggers for reporting or whatever)
      then those fields will be used, and when you no longer need those fields
      and delete them from your dct, then FM3 will delete them - making other
      applications not work (because the fields have disappeared).
      
      The difference with SQL as apposed to Topspeed, is that in Topspeed your
      file structure in your dct had to match the file structure in the db
      otherwise Error47. Extra fields are a no-no, different length fields, etc.
      But this rule falls away with SQL as you can have fields on the backend
      that aren't in your dct and your program will happily open them without
      knowing the difference (as to whether they are there are not). The
      question you should be asking is Why do I want DctMasterFields on, rather
      than why do I want it off. Obviously you need the fields in the backend
      that are declared in your dictionary.
      
      There are Dictionary, File and Field level user options available. They
      are DctMasterFields for dictionary and file level, and DctMasterField at
      field level. This simply means "The Dictionary is Master of the Field(s)".
      
      
        
          
            | Property | Value | 
          
            | DctMasterFields | 1 | 
        
      
      
      This setting is overridden at each level. In other words, this option set
      at field level overrides the file level option, which in turn overrides
      the Dictionary level option. So, if for example you set a certain file
      with this option on, you can then override that setting for a specific
      field by setting the field level option to 0. You must also be aware
      though, that if you add Key level options, and the key is deleted in due
      course, FM3 will not know what it's setting may have been, and default to
      the file or dct level setting to handle the management thereof.
      
      For more information on how to use User Options in FM3, see the
      
Dictionary User Option Reference.
      
      
Note: Even if you have set the
      DctMasterFields, FM3 will only do structure changes on the datatables if
      there is an error opening the tables. This means that if other fields are
      added to the backend, these won't be deleted, unless FM3 detects an error.
      If you would like to enforce a structure change, then on the FM3 connect
      window (at runtime), you can check the 
Force FM3 Full Data
        Structure Comparison checkbox.
    
Key Management  (essential
        reading when converting from TPS to SQL)
     Keys in your Clarion dictionary are not necessarily
      all vital indexes on your SQL Backend. The Clarion docs state that for
      SQL, it is not vital for your keys to match exactly. Often Clarion
      programmers will create keys for sorting purposes etc., which are not
      necessary as indexes on the SQL Backend. In fact, for optimal SQL
      performance, it's better not to create all the indexes. Most SQL Tools
      have a utility, or an automatic Index Optimizing Wizard. In MsSQL for
      example, there is the Index Tuning Wizard. This typically studies the
      statements the server receives and suggests the indexes needed to optimize
      the database based on the queries.
      
      For this reason, FM3 does not by default create all the keys as indexes on
      your SQL backend. Here's what it does. It creates all Primary Keys on the
      backend, as this is both advised and essential. It will also then create
      Unique Indexes on the backend based on your Unique keys in the dictionary.
      Just as it won't create any other keys, it will also not manage or drop
      any of the other keys declared in your dictionary. You may have keys in
      your dictionary that are never defined on the backend. You may also have
      indexes on the backend that are never defined as keys in your dictionary.
      
      Because File Manager 3 is an automatic database management tool however,
      we have added functionality to optionally create and manage certain, or
      all of the keys you define in your dictionary. We use the ever common User
      Option as an interface for this. We have Dictionary, File, and Key level
      user options available. They are DctMasterKeys for dictionary and file
      level, and DctMasterKey at key level. This simply means "The Dictionary is
      Master of the Keys".
      
      
        
          
            | Property | Value | 
          
            | DCTMasterKeys | 1 | 
        
      
      
      This setting is overridden at each level. In other words, this option set
      at key level overrides the file level option, which in turn overrides the
      Dictionary level option. So, if for example you set a certain file with
      this option on, you can then override that setting for a specific key by
      setting the key level option to 0. You must also be aware though, that if
      you add Key level options, and the key is deleted in due course, FM3 will
      not know what it's setting may have been, and default to the file or dct
      level setting to handle the management thereof.
      
      Currently still functional, but phasing out to obsolete is the CreateKey
      User Option. This is an exact synonym for DCTMasterKeys.
      
      For more information on how to use User Options in FM3, see the
      
Dictionary User Option Reference.
      
      
Note: Even if you have set the
      DCTMasterKeys, FM3 will only do structure changes on the datatables if
      there is an error opening the tables. This means that if other keys are
      added to the backend, these won't be deleted, unless FM3 detects an error.
      If you would like to enforce a structure change, then on the FM3 connect
      window (at runtime), you can check the 
Force FM3 Full Data
        Structure Comparison checkbox.
      
      
Note: For MSSQL backends - there is an
      issue with MSSQL2000, whereby it struggles to create statistics on large
      table structures (of a few hundred columns/fields). In this case, FM3
      could struggle to create indices on the backend, which will result in only
      primary and unique keys being created (instead of all the indices, if
      DCTMasterKeys is set). Here's a note from Developer.* Blogs:
      
      The scenario is that you have a gigantic SQL Server 2000 table--gigantic
      not necessarily because it has a lot of rows, but gigantic because it has
      a ton of
      
columns. I've inherited a couple tables like this on different
      projects, tables with two or three 
hundred columns. (Don't look
      at me, I didn't create these tables.)
      
      It's not uncommon to need to add a new index to these kinds of ridiculous
      tables to cover a query. So you go to try and add your new index and you
      get an error like this:
      
      Cannot create more than 249 nonclustered indices or column statistics on
      one table.
      
      You may be frustrated by this initially because you know for a fact that
      there are not 249 indices on this table (or maybe you do...but you've got
      bigger problems in that case). What's really blocking you, though, are not
      too many indices, but too many auto-computed statistics, which are like
      temporary indices created by SQL Server for columns that don't have
      indices. What you have to do to add your new index is get rid of one of
      these auto statistics. Here's how to do that:
      
      For some reason, the "Manage Statistics" dialog box is in SQL Query
      Analyzer, not Enterprise Manager. Go to the Tools menu in Query Analyzer
      and choose Manage Statistics. Choose the database and the table, then
      scroll down to select a "statistic" from the list for a column you don't
      care about (that is, one that is seldom or never used in a WHERE clause).
      Use the Delete button to delete it.
      
      Now try adding your new index again. It should work.
      
      
Note: one step in the above instructions
      that I may have skipped is to go to the Properties dialog for the database
      in Enterprise Manager and turn off Auto Compute Statistics and Auto Create
      Statistics. When I solved this for myself, I had turned these off, and
      then I turned them back on after I deleted a statistic and added the new
      index.
      
      This is not a CapeSoft site, so the link may not be valid:
      
Solution for
        SQL Server Cannot create more than N nonclustered indices Error
        developer. Blogs
      
      Note: PostgreSQL users, check out the
      section:
      
PostgreSQL - Decrementing
        Indices
    Connect to the SQL backend
    
      Introduction
      In terms of FM3, the most vital part of your entire application is the
      Connection procedure. This is where your application makes it's first
      connection to the SQL backend, and allows FM3 to perform it's
      initialization process, which includes collecting some vital information
      for FM3's File Management responsibilities. The Connect procedure happens
      really early on in your program, and it is important that no upgradable
      files are opened before or during this procedure. FM3 therefore uses it's
      own system file to connect to the backend at this point. You can use one
      the 
 template utility to
      import this procedure into your application.
      
Connection Issues
      
        - Errorcode 90 or failure to connect: 
          
            - Make sure that the variable you entered in the SQL_Connect
              window for the owner is the same that you used in the owner for
              your SQL tables, and that your owner variable (in the dictionary)
              is prefixed with !.
- Make sure that you have entered the necessary Connection details
              in the connect window.
- Make sure that you are using the correct user name and password,
              as well as the other details required.
- You may need to specify the machine and server name (depending
              on the backend):
 <ComputerName>\<ServerName> - don't use localhost as
              this sometimes will not consolidate a connection.
- You may need to execute the following stored procedures (SQL
              Express) to add the necessary roles to the server:
 
 EXEC sp_addsrvrolemember 'demo',
                'sysadmin'
 
 as well as:
 
 EXEC sp_addrolemember 'db_owner', 'demo'
 
- Error creating the FM3 system file: Make sure that
          the user that you are using to connect to the database has sufficient
          rights to create /open and modify tables.
- FM3 cannot detect any connection properties for this file
            (or database name not found in connect string): You have
          either not given FM3 the correct connection information, or your user
          rights disallow access.
          
            - Ensure you have setup your user access rights successfully.
- Ensure that your Connection Properties are setup correctly in
              the FM3:ConnectToSQLBackend Extension.
- Ensure that the owner variable that the file is set to use in
              the dictionary (or in your Multi-Proj Driver Substitution setup)
              is exactly the same as that set in the FM3 SQL_Connect control
              template.
 
- Connect window re-opening: This is normally caused
          by setting the filename at runtime. You may find you have a variable
          that is used for the filename, and the variable is set after FM3 is
          initialized.
- Connect OK from remote, but not from a local machine:
          You need to install the native agent for SQL 2005 on the client PC,
          which can be downloaded from: 
http://www.microsoft.com/downloads/details.aspx?FamilyID=d09c1d60-a13c-4479-9b91-9e8b9d835cdc&DisplayLang=en
- The standard Clarion Connect window appears after the FM3
            one does at runtime. This is normally caused by the owner
          in the dictionary (or a particular file in the dictionary) being
          different to the owner variable set in the FM3 Connect controls
          template on the SQL_Connect window. These 2 variables must be the same
          in all your files maintained by FM3 (in SQL). You may also have the
          command line parameter /ShowConnectErrors in the shortcut calling your
          application. Remove that.
- Not Connecting to PostgreSQL? You might be using a
          different ODBC driver, the newer PostgreSQL unicode driver is:
 PostgreSQL ODBC Driver(UNICODE)
 In the FM3 Local Embeds|Setup PostgreSQL connect string embed point,
          set:
 LOC:ODBCDriver = 'PostgreSQL ODBC Driver(UNICODE)'
Advanced Topics on Customisation of the Connect window
      The purpose of the Connection process is to connect to a SQL Backend.
      While the user has security roles and privileges assigned by a SQL
      Admoninstrator, this is _not_ program level security. It should not
      necessarily be used as your program user login. 
      
      Let me explain further... In FM3's case, it is vital that the SQL user
      used for the program has at least dbowner rights. This allows some
      administrative tasks to be performed, which, let's face it, FM3 is in the
      business of doing. You don't necessarily want your program user to have
      that level of control himself. Therefore, it is possibly useful to 'hide
      the connection info' from the user, and rather only let him log in to the
      program with his assigned username and password. This is done using
      
        SecWin for example.
      
      Of course, if you don't need that level of Program security, then by all
      means, prompt for the Connection info. This obviously also gives the
      flexibility of connecting to any valid supported database at runtime. 
      
      Ok, so how then does FM3's new Control Template allow me to accomplish all
      this? Firstly, there are embed points in all the right places, so this is
      a good start. It is however important to understand each step of FM3's
      Connect Process, and ensure that if any customizations are done, that FM3
      still connects effortlessly, and collects the necessary information needed
      for File Management.
      
      One thing which many will ask, is how can I store the info in a different
      file... i.e., not an INI file! Well, that is a good question, and it is
      fairly obvious that an INI file does not hold any encryption or protection
      from being read. Saving Connection info in there is not always the best
      choice. Although, there may be many apps that can live with that. Don't
      forget, that you don't have to make it obvious to anyone else where the
      info is stored. You can tell FM3 on the local extension to use woods.dat
      for example! Ok, I just picked an arbitrary name out of the sky for that
      one, but you get the drift. woods.dat is not obviously an ini file. Nor is
      it obvious that is stores Connection info.
      
      Anyway, the fact is, it is still plain text, and not good enough for all
      scenarios, so... option two is this... write your own 'encryption /
      decryption code' and call it from the embed points of the put and get ini
      routines. So someone finds your ini file which says
      Password=f739kss823KSJLC((&FDLA@#%K ... and that won't be much help to
      them.
      
      The next option is to write to the registry. You can code this yourself,
      but adding code to the embed points of the get and put INI settings
      routines in the connect procedure. Remember to either EXIT before the FM3
      generated code is executed, or omit() the FM3 generated code for that
      routine.
      
      Option 4 is 
not recommended, but if you
      understand the limitations, and the other options, or combination of
      options are not good enough for you, then you can do the following. If you
      want to use an encrypted tps file to store your settings, you must
      understand that this TPS file can never be upgraded by FM3. The Connect
      procedure happens before the rest of FM3's initialization code, and
      therefore is not yet manageable. If you change the structure, you're on
      your own! In other words, if you use this approach, which is 
"not
        recommended", and you send us a support email complaining of an
      error 47, we will send you a link to this very paragraph! And the solution
      is... you'll have to convert it yourself. Now hopefully, this will be a
      very rare occurrence, because a file holding this kind of info, should not
      hold much more than that, and should rarely need to be upgraded.
      So, now that everyone understands, here is how you do it. In the get and
      put ini routines, use omit() around the FM3 generated code, and then add
      your own file handling. Alternatively, add your code in the first embed
      point of the routine, and remember to exit afterwards.
      
      Please understand that it is not always necessary to make things more
      complicated than they are, and that this control template, and amount of
      generated code means your program works as is! ie, setup and support
      should drop to a minimum! If you make changes, be sure you understand what
      is going on, and don't try to omit or workaround any of FM3's code, unless
      absolutely necessary.
      
      The next issue which is also fairly common, is when using TPS and SQL
      files in your program, is it absolutely necessary to call the Connect
      Procedure? Quite simply, the answer is yes. This does not mean you have to
      prompt for user input... the is an Auto Logon switch, and you can
      programmatically set the necessary connection variables. If you don't run
      the Connect proc, then your program could continue to manage TPS files,
      but would bomb out when the SQL files are accessed. FM3 initialization
      code must happen at program startup. If you are brave, and wish to block
      any SQL files being opened if Connection did not take place, then that's
      up to you to program.
      
      This new template is fully Multi-Proj compatible, and works very nicely
      with Multi-Proj's Driver Substitution. I will be adding a Multi-Proj / FM3
      example in the next little while for those who would like to see how
      powerful the combination is!
    
 
    Converting to SQL
    
      Introduction
      If you are not familiar with FM2 or FM3, please tryout the
      
example apps before attempting a SQL
      conversion. Familiarise yourself with FM2's AutoUP feature by trying out
      the 
 Jump Start example for TPS. Once
      familiar with AutoUP, follow these simple instructions for converting to
      SQL.
      
Before you start
      Basic Rule
      
      Always apply FM2/3 to your flat-file application, before attempting a
      conversion to SQL.
      
      If you have not had FM2/3 previously installed and applied to your
      program, then follow the steps below.
      
        - Before changing your dictionary, follow the steps laid out in the
          Adding FM3 to your Application section.
- Compile and run your TPS driven FM3 enabled program.
- Now the UPG.tps file exists, containing valid file structures for
          each file.
- Send out this update to all your clients to allow each client to
          build a upg.tps file.
      If you have already been using FM2/3 in your application, or have
      completed the steps above, then a upg.tps file should already exist, and
      you can follow these instructions.
      
        - Make your dictionary SQL compliant (see the Clarion
            / SQL Fundamentals and SQL Data Types
          sections of this document). Compile and run your application (you will
          need to ship this exe to your clients, so that the TPS data is at the
          latest version before upgrading to SQL).
 
 Either:
 
- Generally the best approach is to use Multi-Proj's driver
          substitution feature to use the same application and dictionary - with
          multiple exe outputs to support different database types. In this way
          you can keep supporting your TPS application without leaving your TPS
          clients in the cold. 
 
 Or:
 
- Alternatively, in the Dictionary editor, change the driver to the
          relevant SQL driver.
- Set the Owner attribute with either your connection string, or a
          string variable. Read about SQL Owner Strings.
- Set the Full Path Name for SQL (e.g.: owner.myfilename). Read about
          SQL TableNames.
- On the Options tab, add the string user option OldName.
- Increase the Version Number (if
          using BDE - this will be done automatically)
 
 Continued for both options:
 
- If your TPS file was encrypted with an owner string, then you need
          to use the OldOwner user option to migrate the
          data to the SQL backend
- If you have had FM2 applied, follow these steps to Convert
            to FM3.
- On the Global Extension, select the
          SQL driver, and leave the the previously used driver(s) also checked.
- Run the 'Import the SQL Connect procedure' template utility. You'll
          need to set the Global Owner variable in the SQL_Connect window's
          "Connect To SQL Backend" Control Template prompts. To do this, double
          click on the SQL_Connect window and click on the "Connect To SQL
          Backend" control template to bring up the prompts and enter the Global
          Owner variable in the field provided on the General tab.
- In your FM3 global extension template, on the Auto Up tab, set the
          "SQL Connect procedure" to 'SQL_Connect' (i.e. the new procedure that
          the template utility just created).
 
- Compile and run your application.
 
    Miscellaneous Topics
    
      SQL Owners
      
        
          
            | Driver | SQL Owner string | Example | 
          
            | MSSQL | Server, DatabaseName, UserName, Password | CRUNCHIE, CapeSoftDB, Scott, Tiger | 
          
            | SQL Express | Machine\Server, DatabaseName, UserName, Password | CRUNCHIE\CRUNCHIE, CapeSoftDB, Scott, Tiger | 
          
            | Oracle | UserName/Password@Protocol:DatabaseName | Scott/Tiger@2:CapeSoftDB | 
          
            | 
 | UserName@Protocol:DatabaseName,Password | Scott@2:CapeSoftDB,Tiger | 
          
            | MySQL | DSN, UserName, Password (See
              DSN-less Connections below) | MyDSN,Scott,Tiger | 
          
            | Firebird | DSN, USERNAME, Password (See
              DSN-less Connections below) | MyDSN,SCOTT,Tiger (note UPPERCASE) | 
          
            | Sybase/ ASA | DatabaseName,UserName,Password;ENG=Server;tcpip={host=111.222.333.444:5555} | CapeSoftDB,Scott,Tiger;ENG=CapeSoftDB;TCPIP={host=127.0.0.1:49152} | 
          
            | PostgreSQL | DSN, UserName, Password (See
              DSN-less Connections below) | MyDSN,Scott,Tiger | 
        
      
      SQL TableNames
      
        
          
            | Driver | Table name | Example | 
          
            | MsSQL | TableOwner.TableName | dbo.Customers | 
          
            | Oracle | UserName.TableName | Scott.Customers | 
          
            | Oracle XPress (10g) | TableName | Customers | 
          
            | MySQL | DatabaseName.TableName | CapeSoftDB.Customers | 
          
            | Firebird | TABLENAME  (Case-sensitive - UPPERCASE recommended) | CUSTOMERS | 
          
            | Sybase / ASA | UserName.TableName | Scott.Customers | 
          
            | PostgreSQL | SchemaName.TableName | public.customers | 
        
      
      Check out the 
HotTips section for more Database
      specific topics.
      
      
ODBC
      How to set up an ODBC Data Source Name
        (DSN)
      This section will show you how to set up a Data Source Name for your ODBC
      connections. It also gives examples of a DSN-less connection.
      
      The exact steps may vary slightly between operating systems.
      
        - Start > Settings > Control Panel.
- Administrative Tools. (2K, XP, 2003, Vista)
- Double click on ODBC Data Sources.
- Select the System tab.
        
        
        - Click on Add.
- Select the driver you wish to use, and click Finish.
- Type in a descriptive name e.g.: 'MyApp_dsn' , and follow the
          prompts for setting up your driver's DSN.
      This name can now be used as your connection string. 
      
How to set up an DSN-less Connections
        (DriverConnect)
      A DSN-less connection does not require the above steps, but rather a
      connection string, semi-colon separated with the driver and connection
      details:
      
      Here are 2 examples:
      
      
DRIVER=MySQL ODBC 3.51
        Driver;SERVER=MyServer;USER=MyUserName;PASSWORD=MyPassword;DATABASE=MyDB;
      
      DRIVER=SQL
        Server;SERVER=SQLBox;UID=Bill;PWD=micro$oft;DATABASE=Northwind;WSID=BillsWorkStation;APP=Microsoft
        Open Database Connectivity;
     
    Oracle TableSpaces
     File Manager 3 now supports the creation of tables
      into a designated Oracle TableSpace. It is advised that you create this
      TableSpace manually before allowing FM3 to create the tables. This ensures
      your full control of the TableSpace definition. That said though, FM3 will
      create the TableSpace for you if it does not exist. To use this feature,
      we've added String User Option which can be set at File or Dictionary
      level. This user option takes the form of a comma separated string as
      TableSpaceName, TableSpaceDataFile, TableSpaceSize. The TableSpaceDataFile
      and TableSpaceSize parameters are optional, but FM3 defaults these to the
      default Oracle Data Folder, and 50M size.
      
      
        
          
            | TableSpace | MyTableSpace,C:\Oracle\OraData\MyTableSpace\MyTableSpace.ora,100M | 
        
      
      Here is an example of the code generated by FM3 to create the TableSpace:
      
        
          
            | Create TableSpace "MyTableSpace" Logging
 Datafile 'MyTableSpace.dbf'
 Size 50M
 Extent Management Local
 | 
        
      
      
      Please note that the File definition overrides the Dictionary setting.
      i.e. If you have a dictionary level TableSpace option set, and you have a
      file level TableSpace option, the file level user option will be used for
      that file.
      
      Please note too, that the directory for the datafile setting must exist!
      
      We have also added the ability to create your indexes and keys in a
      separate TableSpace. This is done in much the same way as the TableSpace
      option, although this can be set at Key, File and Dictionary level. Again,
      the Key user option has higher priority than the file, which has higher
      priority over the dictionary level user option. 
      
        
          
            | IndexSpace | MyTableSpace,C:\Oracle\OraData\MyIndexSpace\MyIndexSpace.ora,100M | 
        
      
    Accessing Multiple
      Databases in one program at the same time
    
      File Manager 3 can only manage one database at a time. (Managing in this
      context refers to upgrading files etc.)
      
      In other words, if your program connects to database A, then FM3 will only
      upgrade the tables on that database.
      While FM3 is connected to database A, your program will still be able to
      access other databases in the normal way.
      The Owner of the FM3 table (e.g.: gMsSqlFile) dictates which database will
      be managed. Thus in order to manage a second database, you must restart
      the program, setting the owner appropriately.
      
      If you would like to connect to another SQL database simultaneously, then
      you will need to suppress the FM3 conversion for those specific tables
      (this excludes flat files like topspeed and dat files, etc.). Click
      
here to find out more information on how
      to do this.
    
Hot Tips
    
      General SQL
      
      MSSQL
      
      Firebird
      
      PostgreSQL
      
      General SQL
      SQL Backend
        Case-Sensitivity
      FM3 Supports both Case-Sensitive, and Case-Insensitive SQL Backends. But,
      for Clarion 5, FM3 assumes the backend is case-insensitive (MSSQL Default
      Install). Clarion 5 only supports a Case-Insensitive backend. 
      
      
        Converting TPS Zero and Blank data to NULL in your SQL database
      TPS does not store true NULL values. Numeric data is stored as zero, and
      empty character data is blank (spaces). In order to use the functionality
      of NULL in a SQL backend, we have implemented a template option to set
      your Zero or Blank data to NULL in the backend during conversion. You will
      find this on the new SQL Advanced tab on the Global Extension, default
      off. When off, FM3 converts Zero data to zero data and blank data to blank
      data (spaces) in your sql backend (not null).
      
      You can now use this option at all levels - Template / Dictionary level,
      File level, Field level. Just the ZeroNull to the User Options tab in the
      dictionary:
      
      
ZeroNull 1
      
      ZeroNull can be set to 1 (true) or 0 (false). Please note that a Field
      User Option overrides a File User Option for that field, and File level
      ZeroNull overrides a Template / Dictionary level option for that
      particular file.
      
      Are you not sure whether you need to use NULLs or not? Check out this
      article:
        http://www.sqlservercentral.com/columnists/mcoles/fourrulesfornulls.asp
      
      Server side
        Auto-incrementing
      In flat files, your application has to handle auto-numbering for your
      auto-incrementing keys. You can still use this in SQL, but there are some
      disadvantages to this approach - especially if multiple applications are
      writing to the database simultaneously.
      
      In order to effectively implement Server-side auto-incrementing, you need
      to do the following to your table in your dictionary:
      
        - If you are converting from TPS to SQL, then you need to remove the
          Auto Number attribute from the key. Server-side
          auto-incrementing is field based, not key based.
- On the field that you will be auto-numbered, go to
          the Options tab, and enter 2 User Options there: AutoNumber = 1, and
          IsIdentity = 1.
 
  
 
- On the file properties, if you are wanting to
          mimic client-side autonumbering (for child records) - then you need to
          add the EmulateAutoNumKey = 1 to the File User options. Take note:
          this is not optimal for strict sequential numbering (like for
          invoices) - as cancelling an insert can create sequential gaps.
- On the file properties, in the Drive options field
          (on the General tab), you need to enter the /AUTOINC select statement
          that will be used to select the next value. The value will determine 
        
          
            | SQL Backend | Driverstring to add | 
          
            | PostgreSQL | /AUTOINC='SELECT
              currval(''schemaname.tablename_fieldname_seq'')' | 
          
            | MySQL | /AUTOINC='SELECT LAST_INSERT_ID()' | 
          
            | MSSQL | /AUTOINC=SELECT SCOPE_IDENTITY() | 
          
            | MSSQL (old) | /AUTOINC=SCOPE_IDENTITY() | 
        
      
      NOTE: At the time of writing this, this is a Connection level driver
      option, so it's important you add it to the FM3 connect file driver
      options (in the global extension template). If you're using MultiProj's
      Driver Substitution and you're substituting to SQL, then enter the driver
      string in the "Send" field of the "Override Driver Possibilities" option.
      
      A note for PostgreSQL users:
      
      PostgreSQL handles auto-incrementing differently from other databases. A
      sequence is not automatically invoked. It is a default constraint which is
      used only if no value is passed. Clarion (bydefault) passes a value -
      other SQL backends ignore this value when inserting a record, whereas
      PostgreSQL will apply this passed value - thus not auto-incrementing the
      field.
      
      There are 2 ways to handle this:
      
        - Call the nextval(Sequence) function before you insert the record,
          and assign that value to the field. The advantage of this is that you
          already know what the value is, so in the case of child tables you are
          set for good. 
- Make that field readonly in the dictionary, such that the query
          generator will not pass a value in the insert/update queries.
MSSQL
      Utility Stored Procedures
      The first in this section is a MSSQL Stored Procedure for returning
      Clarion date and time integer values in MSSQL's datetime datatype. For
      example, it is in many ways nicer to store dates and times as a Clarion
      long (eg: 73838 = Feb 25, 2003 and 5400001 = 15:00), as it makes keys and
      mathematical calculations much easier. But, this data would be useless to
      any non-Clarion program, unless converted to meaningful data. This is what
      ds_ConvertClarionDateTime does. It takes 2 input parameters, 1 output
      parameter, and a return status. You can optionally convert on a date value
      or time value, but must pass a value of 0 (zero) into the "unwanted"
      parameter.
      
      Example: (Calling from SQL Query Analyser)
      
      
        
          
            | Example | 
          
            | declare @rs int              -- return
                status declare @cd int              --
                clarion date
 declare @ct int              --
                clarion time
 declare @sdt datetime        --
                sql datetime
 set @cd = 73842              --
                March 1, 2003 - 0 if "blank"
 set @ct = 5400001            -- 15:00 - 0 if "blank"
 exec @rs
                = ds_ConvertClarionDateTime
                @cd,@ct,@sdt  output
 if @rs = -1
                                 -- Both date and time parameters are zero
 print 'Zero Date and Time!'
 if @rs = 1                   --
                Integer value is greater than valid clarion date
 print 'Invalid Clarion Date!'
 if @rs = 2                   -- Integer value is greater than
                valid clarion time
 print 'Invalid Clarion Time!'
 select @sdt as 'Returned DateTime'
 | 
        
      
      
        MSSQL: Importing and Exporting MDF and LDF
        files
      Importing a MSSQL database from an LDF and MDF file:
      
      
        - Open the SQL Management Studio.
- Right-click 'Databases' and select 'Attach' from the drop down menu.
- Click Add and locate the MDF file that contains the database
          structures.
- Click OK - if there is an error locating the corresponding LDF file,
          then use the ellipses button corresponding to the LDF file to locate
          the correct LDF file.
      Exporting an MSSQL database to an LDF and MDF file:
      
      If you're struggling with an aspect of FM3 - where FM3 is not maintaining
      your database correctly, then we could request a database dump - which
      consists of an MDF file and an LDF file, together with your dictionary and
      the UPG.tps file that corresponds with that dump. This is how it's done:
      
        - Open the SQL Management Studio.
- Right-click on the Database that requires exporting and select
          Properties.
- On the Files Properties window make a note of the mdf and ldf files
          used for the database.
- Return to the main window - Right-click the database and select
          detach from the Tasks menu.
- The files that you need to send are the mdf and ldf files indicated
          in step 3.
        MSSQL: Restoring a backup database to a new one (on a different server)
      This is very simple, but if you don't know the couple of steps, it can be
      very frustrating.
      
        - Copy the backup file (<databasename>.bak) to a place on the
          server pc (preferably).
- in Management Studio: Create a blank database on the server.
- Right click on Databases, and select Restore database.
- Select the new database you just created from the To database drop
          list, and then enter the filename you copied in the From device field
          (not that simple - but following the steps is pretty straight
          forward).
- Check the Restore checkbox next to the filename in the list when it
          appears.
- Click on the options (in the Select a page list) - and check the
          Overwrite the existing database checkbox (this is the magic you're
          after - omitting this step will lead to lots of frustration).
- Now click the OK button - and make some coffee/tea/etc. while it
          performs the restoration.
        Connecting to your MSSQL Server using a non-Standard port
      You cannot do this directly using the Clarion MSSQL Driver, but you can do
      this by creating an alias in the MSSQL Client manager. If you want to
      create a registry entry in your app you can do with:
      
PUTREG(REG_LOCAL_MACHINE,'SOFTWARE\Microsoft\MSSQLServer\Client\ConnectTo','MyServer','DBMSSOCN,211.68.105.49,2433')
      
      and then you can use the next owner attribute,
      
      'MyServer,<database>,<uid>,<pwd><;LANGUAGE=language><;APP=name><;WSID=name>'
      
What FM3 does not
        support in MSSQL
      
        - Schemas. At this stage you have to use the default schema 'dbo'.
How to prevent
        duplicate records appearing in a browse
      
        - Add a Manual Sort order with main the Sort Column/Field and also the
          Record Primary Key as a Secondary then the Duplicates disappear and
          Column Header Sorting works.
 
 Also ensure that your primary key fields are added as hotfields (and
          into the view) if they are not already in the browse.
Firebird
      Firebird Tips
      These notes were created using Firebird v1.5.2 CR1 (which is Firebird
      version 1.5.2.4634).
      
Downloading Firebird
      There are two websites for the Firebird downloads, but the first is more
      useful:
      
      You’ll need to download and install three items:
      
        - The Firebird install for Windows
 For example: Firebird-1.5.2.4634-1_RC1-Win32.exe
 
 Tip: Turn on the Copy Firebird
            client library to <system> folder option during
          installation.
- The Firebird ODBC drivers:
 For example: Firebird_ODBC_1.2.0.69-Win32.exe
 
 You will need these ODBC drivers installed on each machine that will
          be running your application so that it can connect to your database.
An admin tool for managing your databases.
        I recommend HK-Software’s IB Expert (www.h-k.de)
        A free personal edition is available and does the job very well.
        For example: ibep_2004.10.30.1_full.exe
        
        Tip: The first time you run this app it
        displays an error. Just try again.
      
SYSDBA User & Password
      The default username and password is:
      
      Username:
      
SYSDBA
      
      Password:
      
masterkey
      
      Creating Your Firebird
        Database
      FM3 requires that you have already created a (blank) database in your SQL
      backend. (FM3 will create all the tables and keys* for you, you just need
      to create the database).
      
      * Not all keys are created by FM3, just the ones it needs, but you can
      override this.
      
      Make sure Firebird is running (you can start it from your Control Panel).
      
      Using IB Expert create a database such as the one below (Database | Create
      Database):
      
      
 Note
      
      Note: You must use 
        Dialect 3.
      
Note: If you are running firebird as a
      service, then the file path must be accessible to the service.
      
      After creating the database (or if you have already created it), you can
      view the database, with the following registration information:
      
      
 
      
      You’ll then see your Database listed in the IB Expert database list. If
      you double click on it, you’ll see the database properties:
      
      
 
      
      GDS32.dll
      Without this DLL your Firebird applications won’t work. After installation
      you should find it in your Windows\System32 folder. It’s probably worth
      shipping a copy of this DLL with your application.
      
      
Key Limitations in Size
      Firebird Indices (keys) have a limit of 254 characters, depending on the
      character set, so it may be less. You may have to shorten some of your
      keys or fields in order for Firebird tables to be made.
      
      
Key Sort Orders
      Each Firebird Index can only contain either all ascending or all
      descending Key Sort Orders.
      
      NOTE: At present, FM3 will only create and maintain ascending keys in
      Firebird.
      
Key Limitations in
        Duplication
      Firebird doesn’t like have two unique Indices with the same fields in
      them. For example the following are not allowed:
      
      DateKey: Date, Share
      ShareKey: Share, Date
      
      You’ll need to either add or remove fields from the keys or make one of
      the keys non-unique.
      
Connecting to a remote Firebird Database (in an FM3 enabled
        application):
      In the Connect window:
      
        - Make sure that you've entered the username in uppercase, 
- The password is case-sensitive and must match that of the created
          database.
- ODBC Backend must be FireBird (In the configure section)
- The key is getting the Server and Database Name correct. The 
            Database Name is the relative name of the .GDB file that
          contains the database. The Server name is either the
          resolved machine name or IP Address. The Port is not
          used. Here's an example:
 
      
      PostgreSQL
      PostgreSQL - Binary Strings
      The char and varchar character types in Postgres are quite rigorous in the
      use of non-characters used. In Clarion, a string is basically a binary
      collection of ASCII bytes - which can normally be equated to a char type
      in Postgres because of the nature of data stored in strings (addresses,
      names, comments, etc.). However, because of the potential that Clarion
      possesses in the Topspeed driver to store non-printable characters (or
      binary characters) in strings, this can mean that equating a string to a
      char in Postgres can have adverse effects - i.e. your TPS data string will
      not be converted into the Postgres database. This is particularly
      prevalent in GUIDs (used with Capesoft's Replicate) which will mostly
      contain binary data.
      
      There're a few ways of working around this issue:
      
        - Use the ds_SetMakeGUIDsBinary(1) function before the FM3 init code
          runs. This will basically ensure that all GUIDs in all your tables
          will be created as type bytea (or the equivalent of a binary string in
          Clarion) - without making any changes to your dictionary. This will
          cater for the use of GUIDs in your dictionary. You only need this for
          existing applications that used Replicate v2.10 or less at any stage.
          Applications that only started using Replicate at Replicate version
          2.11 or above, are generated with GUIDs that are legal text
          characters.
- Use the | BINARY dictionary switch in the external name (see the
          dictionary help on External name options) to inform FM3 that this is a
          binary string and should be created as 'bytea' on the backend, as
          apposed to the default char type. This will cater for non-GUID fields
          that contain binary data.
      The following characters are unusable in text based datatypes (like chars,
      varchars, etc.) in PostgreSQL:
      
        - chr(0) - this is stored as chr(32)
- chr(10) - this is stored as chr(13)
- Range: chr(128) to chr(255) - these cannot be stored legally (in one
          byte) - some of these chars are stored in 2 or 3 bytes.
PostgreSQL - Clarion
        Version support
      Why does FM3 only support Clarion6 and up? FM3 uses a generated
      maintenance file to connect to the database and query system tables. There
      is one for each driver supported (MSS, TPS, ODBC, etc.) - the ODBC file is
      called gODBCFile. The ODBC generated file contains uppercase field names,
      which are necessary for FireBird support. In Clarion6, the external name
      is treated as case insensitive - thus ds_field1 is equated to DS_FIELD1,
      but in Clarion5.5 and below this is not the case. Field names are created
      as lower case, but when queried, the case is expected as upper, and so the
      field names don't match (hence error 47). This makes multiple ODBC backend
      support impossible in previous versions of Clarion.
      
PostgreSQL -
        Decrementing Indices
      PostgreSQL does not support decrementing Indices. For more discussion on
      this topic, please see:
      
        http://archives.postgresql.org/pgsql-general/2002-03/msg00092.php
      Peculiarities in PostgreSQL
      Turning off the CR/LF conversion (to LF only).
      
      By default, PostgreSQL converts the CR/LF to LF character when inserting
      and upgrading records. This means that insertions into the database get
      added with just an LF. However, this can be problematic for fields with
      existing data (with CR/LF charset) - because the data is read and arrives
      at your clarion application with the CR/LF charset, but when it is added
      back, the record is encoded to just the LF and compared to a reget (which
      has the CR/LF character). This will then be marked as different from the
      original, and so a "This record was changed by another station. Those
      changes will now be displayed. Use the Ditto Button or Ctrl+H to recall
      your changes" error.
      
      In order to bypass this error, you need to turn the CR/LF conversion off
      in the odbc driver.
      
      If you are connecting via a normal connection (i.e. non-DSN) then you need
      to add the LFConversion=0 to the driver string (which you can insert on
      the AutoUp tab).
      
      If you have a dsn connection, then you do this as follows:
      
      
 
      
      PostgreSQL
        - Getting things going, Drivers and resources
      What you'll need:
      
      Some steps to get you going:
      
        - Install the server on the database machine (and run it, pretty
          straight forward)
- Setup the accountname details on the server machine database.
- On the client machine, install the GUI interface to the server.
- The crucial thing is to grant access on the Server side (for the
          client). You do this in the pg_hba.conf file
 (which is in the C:\Program Files\PostgreSQL\8.1\data directory by
          default, or wherever you installed the Server).
 
 You need to put the following line in:
 
 host all all 192.168.2.3/32 trust
 
 for access for your local machine.
 
 It seems like 0 works as a wildcard, so 192.168.2.0/0 would allow the
          same machine and others to connect.
 
 If you get a Can't Obtain databases error when running the Client
          application, then you've set this up incorrectly. The first time I did
          it, I didn't put the /32 in. It's really fussy about that.
- The next crucial thing to do on the server side (for the client) in
          granting access to the client, is to modify the postgresql.conf file
          to tell PostgreSQL which addresses to allow access to the server. 
 
 Open the postgresql.conf file (which is in the C:\Program
          Files\PostgreSQL\8.1\data directory by default, or wherever you
          installed the Server) with a text editor.
 
 You need to put the following line in:
 
 listen_addresses = '*'
- Install the ODBC driver on the Client machine. For FM3 (at the
          writing of this) you'll need the PostgreSQL Unicode
          ODBC driver (32 bit one whether on 32 bit or 64 bit OS systems). Make
          sure that this one is installed when you install the ODBC driver. If
          you already have a PostgreSQL driver installed (but not the PostgreSQL
          Unicode driver) - then you'll need to run the upgrade.bat file that is
          included in the ODBC install zip. Otherwise, you can simply run the
          msi install file.
- It's a good idea to setup a DSN on the client machine to test the
          connection (the first time you work with PostgreSQL): 
            http://www.sevainc.com/access/PostgreSQL_ODBC_Configuration.html
          (not a CapeSoft link, so may not be valid in future). NOTE: If you are
          using a 64 bit OS, then you must run the 
            C:\Windows\SysWOW64\odbcad32.exe to register a DSN (not the
          odbcad32.exe, which by default will point to the 64 bit DSN manager -
          I know, it's the wrong name - don't ask, I don't make the rules).
Note: You may have some issues when
      connecting to a different machine (i.e. where the Server resides on a
      different machine to the client application). Anti-virus could be the
      culprit or your Firewall. The best solution would be to first try and
      allow the PostgreSQL port - by default 5432 - failing that, to disable the
      Firewall and/or Anti-Virus temporarily during your test.
    
 
    Clarion / SQL Related Links and Resources
    
      
        - If you haven't already, read this document! To
            Top.
- Dan Pressnell of Toolwares
          has written some excellent Articles entitled "The Better SQL Series". These can be found on
          the  IceTips
          site.
- ClarionMag
          is an excellent resource containing many articles by many Clarion
          programmers. Note: This is a paid for subscription based resource.
- ClarionLive
          is also an excellent resource for Clarion Programmers.
- And, one of the best places to find information are the SoftVelocity
          newsgroups.