Functions in DLLs and shared libraries

Introduction

VFX Forth supports calling external API calls in dynamic link libraries (DLLs) for Windows and shared libraries in Linux and other Unix-derived operating systems. Various API libraries export functions in a variety of methods mostly transparent to programmers in languages such as C, Pascal and Fortran. Floating point data is supported for use with Lib\Ndp387.fth.

Before a library function can be used, the library itself must be declared, e.g.


LIBRARY: Kernel32.dll

Access to functions in a library is provided by the EXTERN: syntax which is similar to a C style function prototype, e.g.


EXTERN: int PASCAL SendMessage(
  HWND hwnd, DWORD mesg, WPARAM wparam, LPARAM lparam
);

This can be used to prototype the function SendMessage from the Microsoft Windows API, and produces a Forth word SendMessage.

  SendMessage  \ hwnd mesg wparam lparam -- int

For Linux and other Unices, the same notation is used. The default calling convention is nearly always applicable. The following example shows that definitions can occupy more than one line. It also indicates that some token separation may be necessary for pointers:


Library: libc.so.6

Extern: int execve(
  const char * path,
  char * const argv[],
  char * const envp[]
);

This produces a Forth word execve.

  execve       \ path argv envp -- int

The parser used to separate the tokens is not ideal. If you have problems with a definition, make sure that * tokens are white-space separated. Formal parameter names, e.g. argv above are ignored. Array indicators, [] above, are also ignored when part of the names.

The input types may be followed by a dummy name which is discarded. Everything on the source line after the closing ')' is discarded.

From VFX Forth v4.3 onwards, the use of PASCAL is not required as PASCAL is the default calling convention in the Windows version. The default for the Linux and OS X versions is "C". The default is always used unless overridden in the declaration.

Format


EXTERN: <return> [ <callconv> ] <name> '(' <arglist> ')' ';'

<return>    := { <type> [ '*' ] |  void }
<arg>       := { <type> [ '*' ] [ <name> ] }
<args>      := { [ <arg>, ]* <arg> }
<arglist>   := { <args> | void }        Note: "void, void" etc. is illegal.
<callconv>  := { PASCAL | WINAPI | STDCALL | "PASCAL" | "C" }
<name>      := <any Forth acceptable namestring>
<type>      := ... (see below, "void" is a valid type)

Note that during searches <name> is passed to the operating system exactly as it is written, i.e. case sensitive. The Forth name is case-insensitive.

As a standard Forth's string length for dictionary names is only guaranteed up to 31 characters for portable source code, very long API names can cause problems. Therefore the word AliasedExtern: allows separate specification of API and Forth names (see below). AliasedExtern: also solves problems when API functions only differ in case or their names conflict with existing Forth word names.

Calling Conventions

In the discussion caller refers to the Forth system (below the application layer and callee refers to a a function in a DLL or shared library. The EXTERN: mechanism supports three calling conventions.

C-Language: "C"
Caller tidies the stack-frame. The arguments (parameters) which are passed to the library are reordered. This convention can be specified by using "C" after the return type specifier and before the function name. For Linux and most Unix-derived operating systems, this is the default.
Pascal language: "PASCAL"
Callee removes arguments from the stack frame. This is invisible to the programmer at the application layer The arguments (parameters) which are passed to the library are not reordered. This convention is specified by "PASCAL" after the return type specifier and before the function name.
Windows API: WINAPI | PASCAL | STDCALL
In nearly all cases (but not all), calls to Windows API functions require C style argument reversal and the called function cleans up. Specify this convention with PASCAL, WinAPI or StdCall after the return type specifier and before the function name. For Windows, this is the default.

Unless otherwise specified, the Forth system's default convention is used. Under Windows this is WINAPI and under Linux and other Unices it is "C".

Promotion and Demotion

The system generates code to either promote or demote non-CELL sized arguments and return results which can be either signed or unsigned. Although Forth is an un-typed language it must deal with libraries which do have typed calling conventions. In general the use of non-CELL arguments should be avoided but return results should be declared in Forth with the same size as the C or PASCAL convention documented.

Argument Reversal

The default calling convention for the host operating system is used. The right-most argument/parameter in the C-style prototype is on the top the Forth data stack. When calling an external function the parameters are reordered if required by the operating system; this is to enable the argument list to read left to right in Forth source as well as in the C-style operating system documentation.

Under certain conditions, the order can be reversed. See the words "C" and "PASCAL" which define the order for the operating system. See L>R and R>L which define the Forth stack order with respect to the arguments in the prototype.

C comments in declarations

Very rudimentary support for C comments in declarations is provided, but is good enough for the vast majority of declarations.

Comments can be // ... or /* ... */,
Comments must be at the end of the line,
Comments are treated as extending to the end of the line,
Comments must not contain the ')' character.

The example below is taken from a SQLite interface.


Extern: "C" int sqlite3_open16(
  const void * filename,  /* Database filename [UTF-16] */
  sqlite3 ** ppDb         /* OUT: SQLite db handle */
);

Controlling external references

1 value ExternWarnings? \ -- n
Set this true to get warning messages when an external reference is redefined.

0 value ExternRedefs? \ -- n
If non-zero, redefinitions of existing imports are permitted. Zero is the default for VFX Forth so that redefinitions of existing imports are ignored.

1 value LibRedefs? \ -- n
If non-zero, redefinitions of existing libraries are permitted. Non-zero is the default for VFX Forth so that redefinitions of existing libraries and OS X frameworks are permitted. When set to zero, redefinitions are silently ignored.

1 value InExternals? \ -- n
Set this true if following import definitions are to be in the EXTERNALS vocabulary, false if they are to go into the wordlist specified in CURRENT. Non-Zero is the default for VFX Forth.

: InExternals \ --
External imports are created in the EXTERNALS vocabulary.

: InCurrent \ --
External imports are created in the wordlist specified by CURRENT.

Library Imports

In VFX Forth, libraries are held in the EXTERNALS vocabulary, which is part of the minimum search order. Other Forth systems may use the CURRENT wordlist.

For turnkey applications, initialisation, release and reload of required libraries is handled at start up.

variable lib-link \ -- addr
Anchors the chain of dynamic/shared libraries.

variable lib-mask \ -- addr
If non-zero, this value is used as the mode for dlopen() calls in Linux and OS X.

: init-libs \ --
Release and reload the required libraries.

: find-libfunction \ z-addr -- address|0
Given a zero terminated function name, attempt to find the function somewhere within the already active libraries.

: .Libs \ --
Display the list of declared libraries.

: #BadLibs \ -- u
Return the number of declared libraries that have not yet been loaded.

: .BadLibs \ --
Display a list of declared libraries that have not yet been loaded.

: Library: \ "<name>" -- ; -- loadaddr|0
Register a new library by name. If LibRedefs? is set to zero, redefinitions are silently ignored. Use in the form:

  LIBRARY: <name>

Executing <name> later will return its load address. This is useful when checking for libraries that may not be present. After definition, the library is the first one searched by import declarations.

: topLib \ libfn --
Make the library structure the top/first in the library search order.

: firstLib \ "<name>" --
Make the library first in the library search order. Use during interpretation in the form:

  FirstLib <name>

to make the library first in the search order. This is useful when you know that there may be several functions of the same name in different libraries.

: [firstLib] \ "<name>" --
Make the library first in the library search order. Use during compilation in the form:

  [firstLib] <name>

to make the library first in the search order. This is useful when you know that there may be several functions of the same name in different libraries.

Mac OS X extensions

The phrase Framework <name.framework> creates two Forth words, one for the library access, the other to make that library top in the search order. For example:

  framework Cocoa.framework

produces two words

  Cocoa.framework/Cocoa

  Cocoa.framework

The first word is the library definition itself, which behaves in the normal VFX Forth way, returning its load address or zero if not loaded. The second word forces the library to be top/first in the library search order. Thanks to Roelf Toxopeus.

As of OSX 10.7, FRAMEWORK (actually dlopen()) will search for frameworks in all the default Frameworks directories:

/Library/Frameworks
/System/Library/Frameworks
~/Library/Frameworks

: framework \ --
Build the two framework words. See above for more details. If LibRedefs? is set to zero, redefinitions are silently ignored.

Function Imports

Function declarations in shared libraries are compiled into the EXTERNALS vocabulary. They form a single linked list. When a new function is declared, the list of previously declared libraries is scanned to find the function. If the function has already been declared, the new definition is ignored if ExternRedefs? is set to zero. Otherwise, the new definition overrides the old one as is usual in Forth.

In VFX Forth, ExternRedefs? is zero by default.

variable import-func-link \ -- addr
Anchors the chain of imported functions in shared libraries.

: ExternLinked \ c-addr u -- address|0
Given a string, attempt to find the named function in the already active libraries. Returns zero when the function is not found.

: init-imports \ --
Initialise Import libraries. INIT-IMPORTS is called by the system cold chain.

defer preExtCall \ --
Windows only. A hook provided for debugging and extending external calls without floating point parameters or return items. It is executed at the start of the external call before any parameter processing.

defer postExtCall \ --
Windows only. A hook provided for debugging and extending external calls without floating point parameters or return items. It is executed at the end of the external call after return data processing.

defer preFPExtCall \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. . It is executed at the start of the external call before any parameter processing.

defer postFPExtCall \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. It is executed at the end of the external call after return data processing.

: InExternals \ --
External imports are created in the EXTERNALS vocabulary.

: InCurrent \ --
External imports are created in the wordlist specified by CURRENT.

: Extern: \ "text" --
Declare an external API reference. See the syntax above. The Forth word has the same name as the function in the library, but the Forth word name is not case-sensitive. The length of the function's name may not be longer than a Forth word name.

: AliasedExtern: \ "forthname" "text" --
Like EXTERN: but the declared external API reference is called by the explicitly specified forthname. The Forth word name follows and then the API name. Used to avoid name conflicts, e.g.

AliasedExtern: saccept int accept( HANDLE, void *, unsigned int *);

which references the Winsock accept function but gives it the Forth name SACCEPT. Note that here we use the fact that formal parameter names are optional.

: LocalExtern: \ "forthname" "text" --
As AliasedExtern:, but the import is always built into the CURRENT wordlist.

: extern \ "text" --
An alias for EXTERN:.

: ExternVar \ "<name>" -- ; ExternVar <name>
Used in the form

  ExternVar <name>

to find a variable in a DLL or shared library. When executed, <name> returns its address.

: AliasedExternVar \ "<forthname>" "<dllname>" --
Used in the form

  AliasedExternnVar <forthname> <varname>

to find a variable in a DLL or shared library. When executed, <forthname> returns its address.

: .Externs \ -- ; display EXTERNs
Display a list of the external API calls.

: #BadExterns \ -- u
Silently return the number of unresolved external API calls.

: .BadExterns \ --
Display a list of any external API calls that have not been resolved.

: func-loaded? \ xt -- addr|0
Given the XT of a word defined by EXTERN: or friends, returns the address of the DLL function in the DLL, or 0 if the function has not been loaded/imported yet.

Pre-Defined parameter types

The types known by the system are all found in the vocabulary TYPES. You can add new ones at will. Each TYPE definition modifies one or more of the following VALUEs. )

argSIZE

Size in bytes of data type.

argDEFSIGN

Default sign of data type if no override is supplied.

argREQSIGN

Sign OverRide. This and the previous use 0 = unsigned and 1 = signed.

argISPOINTER

1 if type is a pointer, 0 otherwise

Each TYPES definition can either set these flags directly or can be made up of existing types.

Note that you should explicitly specify a calling convention for every function defined.

Calling conventions

: "C" \ --
Set Calling convention to "C" standard. Arguments are reversed, and the caller cleans up the stack.

: "PASCAL" \ --
Set the calling convention to the "PASCAL" standard as used by Pascal compilers. Arguments are not reversed, and the called routine cleans up the stack. This is not the same as PASCAL below.

: PASCAL \ --
Set the calling convention to the Windows PASCAL standard. Arguments are reversed in C style, but the called routine cleans up the stack. This is the standard Win32 API calling convention. N.B. There are exceptions! This convention is also called "stdcall" and "winapi" by Microsoft, and is commonly used by Fortran programs. This is not the same as "PASCAL" above.

: WinApi \ --
A synonym for PASCAL.

: StdCall \ --
A synonym for PASCAL.

: VC++ \ --
Defines the calling convention as being for a C++ member function which requires "this" in the ECX register. The function must be defined with an explicit this pointer (void * this). Because exported VC++ member functions can have either "C" or "PASCAL" styles, the this pointer must be positioned so that it is leftmost when reversed (C/WINAPI/StdCall style) or is rightmost when not reversed ("PASCAL" style). See also the later section on interfacing to C++ DLLs.

: R>L \ --
By default, arguments are assumed to be on the Forth stack with the top item matching the rightmost argument in the declaration so that the Forth parameter order matches that in the C-style declaration. R>L reverses this.

: L>R \ --
By default, arguments are assumed to be on the Forth stack with the top item matching the rightmost argument in the declaration so that the Forth parameter order matches that in the C-style declaration. L>R confirms this.

Basic Types

: unsigned \ --
Request current parameter as being unsigned.

: signed \ --
Request current parameter as being signed.

: int \ --
Declare parameter as integer. This is a signed 32 bit quantity unless preceeded by unsigned.

: char \ --
Declare parameter as character. This is a signed 8 bit quantity unless preceeded by unsigned.

: void \ --
Declare parameter as void. A VOID parameter has no size. It is used to declare an empty parameter list, a null return type or is combined with * to indicate a generic pointer.

: * \ --
Mark current parameter as a pointer.

: ** \ --
Mark current parameter as a pointer.

: *** \ --
Mark current parameter as a pointer.

: const ; \ --
Marks next item as constant in C terminology. Ignored by VFX Forth.

: int32 \ --
A 32bit signed quantity.

: int16 \ --
A 16 bit signed quantity.

: int8 \ --
An 8 bit signed quantity.

: uint32 \ --
32bit unsigned quantity.

: uint16 \ --
16bit unsigned quantity.

: uint8 \ --
8bit unsigned quantity.

: LongLong \ --
A 64 bit signed or unsigned integer. At run-time, the argument is taken from the Forth data stack as a normal Forth double with the top item on the top of the data stack.

: LONG int ;
A 32 bit signed quantity.

: SHORT \ --
For most compilers a short is a 16 bit signed item, unless preceded by unsigned.

: BYTE \ --
An 8 bit unsigned quantity.

: float \ --
32 bit float.

: double \ --
64 bit float.

: bool1 \ --
One byte boolean.

: bool4 \ --
Four byte boolean.

: ... \ --
The parameter list is of unknown size. This is an indicator for a C varargs call. Run-time support for this varies between operating system implementations of VFX Forth. Test, test, test.

Windows Types

The following parameter types are non "C" standard and are used by Windows in function declarations. They are all defined in terms of existing types.

: OSCALL PASCAL ;
Used for portable code to avoid three sets of declarations. For Windows, this is a synonym for PASCAL and under Linux and other Unices this is a synonym for "C".

: DWORD unsigned int ;
32 bit unsigned quantity.

: WORD unsigned int 2 to argSIZE ;
16 bit unsigned quantity.

: HANDLE void * ;
HANDLEs under Windows are effectively pointers.

: HMENU handle ;
A Menu HANDLE.

: HDWP handle ;
A DEFERWINDOWPOS structure Handle.

: HWND handle ;
A Window Handle.

: HDC handle ;
A Device Context Handle.

: HPEN handle ;
A Pen Handle.

: HINSTANCE handle ;
An Instance Handle.

: HBITMAP handle ;
A Bitmap Handle.

: HACCEL handle ;
An Accelerator Table Handle.

: HBRUSH handle ;
A Brush Handle.

: HMODULE handle ;
A module handle.

: HENHMETAFILE handle ;
A Meta File Handle.

: HFONT handle ;
A Font Handle.

: HRESULT DWORD ;
A 32bit Error/Warning code as returned by various COM/OLE calls.

: LPPOINT void * ;
Pointer to a POINT structure.

: LPACCEL void * ;
Pointer to an ACCEL structure.

: LPPAINTSTRUCT void * ;
Pointer to a PAINTSTRUCT structure.

: LPSTR void * ;
Pointer to a zero terminated string buffer which may be modified.

: LPCTSTR void * ;
Pointer to a zero terminated string constant.

: LPCSTR void * ;
Another string pointer.

: LPTSTR void * ;
Another string pointer.

: LPDWORD void * ;
Pointer to a 32 bit DWORD.

: LPRECT void * ;
Pointer to a RECT structure.

: LPWNDPROC void * ;
Pointer to a WindowProc function.

: PLONG long * ;
Pointer to a long (signed 32 bit).

: ATOM word ;
An identifier used to represent an atomic string in the OS table. See RegisterClass() in the Windows API for details.

: WPARAM dword ;
A parameter type which used to be 16 bit but under Win32 is an alias for DWORD.

: LPARAM dword ;
Used to mean LONG-PARAMETER (i.e. 32 bits, not 16 as under Win311) and is now effectively a DWORD.

: UINT dword ;
Windows type for unsigned INT.

: BOOL int ;
Windows Boolean type. 0 is false and non-zero is true.

: LRESULT int ;
Long-Result, under Win32 this is basically an integer.

: colorref DWORD ;
A packed encoding of a color made up of 8 bits RED, 8 bits GREEN, 8 bits BLUE and 8 bits ALPHA.

: SOCKET dword ;
Winsock socket reference.

: __in ;
Microsoft header annotation.

: __inout ;
Microsoft header annotation.

: __out ;
Microsoft header annotation.

: __in_opt ;
Microsoft header annotation.

: __inout_opt ;
Microsoft header annotation.

: __out_opt ;
Microsoft header annotation.

Linux Types

: OSCALL "C" ;
Used for portable code to avoid three sets of declarations. For Windows, this is a synonym for PASCAL and under Linux this is a synonym for "C".

: FILE uint32 ;
Always use as FILE * stream.

: DIR uint32 ;
Always use as DIR * stream.

: size_t uint32 ;
Linux type for unsigned INT.

: off_t uint32 ;
Linux type for unsigned INT.

: int32_t int32 ;
Synonym for int32.

: int16_t int16 ;
Synonym for int16.

: int8_t int8 ;
Synonym for int8.

: uint32_t uint32 ;
Synonym for uint32.

: uint16_t uint16 ;
Synonym for uint16.

: uint8_t uint8 ;
Synonym for uint8.

: time_t uint32 ;
Number of seconds since midnight UTC of January 1, 1970.

: clock_t uint32 ;
Processor time in terms of CLOCKS_PER_SEC.

: pid_t int32 ;
Process ID.

: uid_t uint32 ;
User ID.

: mode_t uint32 ;
File mode.

Mac OS X Types

: OSCALL "C" ;
Used for portable code to avoid three sets of declarations. For Windows, this is a synonym for PASCAL and under OS X this is a synonym for "C".

: FILE uint32 ;
Always use as FILE * stream.

: DIR uint32 ;
Always use as DIR * stream.

: size_t uint32 ;
Unix type for unsigned INT.

: off_t uint32 ;
Unix type for unsigned INT.

: int32_t int32 ;
Synonym for int32.

: int16_t int16 ;
Synonym for int16.

: int8_t int8 ;
Synonym for int8.

: uint32_t uint32 ;
Synonym for uint32.

: uint16_t uint16 ;
Synonym for uint16.

: uint8_t uint8 ;
Synonym for uint8.

: time_t uint32 ;
Number of seconds since midnight UTC of January 1, 1970.

: clock_t uint32 ;
Processor time in terms of CLOCKS_PER_SEC.

: pid_t int32 ;
Process ID.

: uid_t uint32 ;
User ID.

: mode_t uint32 ;
File mode.

Compatibility words

These words are mainly for users converting code from other Forth systems.

This section provides shared library imports in the form:

  function: foo  ( a b c d -- x )

where the brackets must be space delimited.

: FUNCTION: \ "<name>" "<parameter list>" --
Generate a reference to an external function. The Forth name is the same as the name of the external function. Use in the form:

 function: foo1 ( a b c d -- }

 function: foo2 ( a b c d -- e }

 function: foo3 ( a b c d -- el eh }

The returned value may be 0, 1 or 2 items corresponding to void, int/long and long long on most 32 bit systems.

: ASCALL: \ "<synonym-name>" "<name>" "<parameter list>" --
Generate a reference to an external function. The Forth name is not the same as the name of the external function. Use in the form:

 ascall: forthname funcname ( a b c d -- e }

: GLOBAL: \ "<name>" --
Generate a reference to an external variable. Use in the form:

 global: varname

Using the Windows hooks

The hooks preExtCall and postExtCall are DEFERred words into which you can plug actions that will be run before and after any external call. They are principally used:

To save and restore the NDP state when using screen and printer drivers that do not obey all the Windows rules.
To save and restore the NDP state and you want the NDP state preserved regardless of any consequences. Although this is safe, the system overhead is greater than that of preserving your floats in variables or locals as required.
Installing error handlers that work with nested callbacks.

The hooks preFPExtCall and postFPExtCall are compiled into calls with floating point parameters or return values. They do not affect the NDP state.

The examples below illustrate both actions.

Deferred words and variables

defer preExtCall \ --
Windows only. A hook provided for debugging and extending external calls. It is executed at the start of the external call before any parameter processing.

defer postExtCall \ --
Windows only. A hook provided for debugging and extending external calls. It is executed at the end of the external call after return data processing.

defer preFPExtCall \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. It is executed at the start of the external call before any parameter processing.

variable XcallSaveNDP? \ -- addr
Set true when imports must save and restore the NDP state. Windows only. From build 2069 onwards, the default behaviour for Windows includes saving and restoring the FPU state. This can be inhibited by clearing XcallSaveNDP? before execution.

variable abort-code \ -- addr
Holds error code for higher level routines, especially RECOVERY below. Windows versions only.

variable aborting? \ -- addr
Holds a flag to indicate whether error recovery should be performed by a calling routine.

defer xcall-fault \ -- ; handles errors in winprocs
Used by application code in the DEFERred words preExtCall and postExtCall above to install user-defined actions.

Default versions

code PreExtern  \ -- ; R: -- sys
\ Clears the abort code and saves the NDP state if XcallSaveNDP?
\ is set.
  mov   dword ptr abort-code , # 0      \ no previous abort code
  cmp     [] XcallSaveNDP? , # 0        \ Win: required
  nz, if,
    pop     eax
    lea     esp, -/fsave [esp]
    fsave   0 [esp]
    push    eax
  endif,
  ret
end-code
assign preExtern to-do preExtCall

code PostExtern \ -- ; R: sys --
\ Restore the NDP state if XcallSaveNDP? is set and test the
\ abort code.
  cmp   [] XcallSaveNDP? , # 0          \ required
  nz, if,
    pop     eax
    frstor  0 [esp]
    lea     esp, /fsave [esp]
    push eax
  endif,
  \ Detecting faults in nested callbacks.
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code
assign postExtern to-do postExtCall

code PreFPExtern        \ -- ; R: -- sys ; SFP006
\ Clears the abort code.
  mov   dword ptr abort-code , # 0      \ no previous abort code
  ret
end-code
assign preFPExtern to-do preFPExtCall

code PostFPExtern       \ -- ; R: sys -- ; SFP006
\ Test the abort code.
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code
assign postFPExtern to-do postFPExtCall

: DefaultExterns \ --
Set the default PRE and POST EXTERN handlers.

Protected EXTERNs

Protected EXTERNs allow VFX Forth to recover when a crash occurs inside a Windows call and the Forth registers have been corrupted. For example

  255 0 GetCurrentDirectory

will crash because an address of zero is invalid. Protected EXTERNs save the Forth registers befor making the call so that exception handlers can restore VFX Forth to a known state.

code PreProtExtern      \ -- ; R: -- sys
\ Clears the abort code and saves the NDP state if XcallSaveNDP?
\ is set.
  mov    edx, # XcallBuffer             \ where the saved data goes
  mov    eax, 0 [esp]                   \ return address
  sub    eax, # 6                       \ xt of EXTERN (call [] prexx)
  mov    0 scb.xt [edx], eax            \ save it
  lea    eax, 4 [esp]                   \ RSP on entry
  mov    0 scb.esp [edx], eax
  mov    0 scb.ebp [edx], ebp
  mov    0 scb.esi [edx], esi
  mov    0 scb.edi [edx], edi

  mov   dword ptr abort-code , # 0      \ no previous abort code
  cmp     [] XcallSaveNDP? , # 0        \ Win: required
  nz, if,
    pop     eax                         \ return address
    lea     esp, -/fsave [esp]
    fsave   0 [esp]
    push    eax
  endif,
  ret
end-code

code PostProtExtern     \ -- ; R: sys --
\ Restore the NDP state if XcallSaveNDP? is set and test the
\ abort code.
  mov   dword ptr XcallBuffer scb.xt , # 0  \ reset Extern in progress
  cmp   [] XcallSaveNDP? , # 0          \ required
  nz, if,
    pop     eax                         \ return address
    frstor  0 [esp]
    lea     esp, /fsave [esp]
    push eax
  endif,
  \ Detecting faults in nested callbacks.
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code

code PreProtFPExtern    \ -- ; R: -- sys ; SFP006
\ Clears the abort code.
  mov    edx, # XcallBuffer             \ where the saved data goes
  mov    eax, 0 [esp]                   \ return address
  sub    eax, # 6                       \ xt of EXTERN (call [] prexx)
  mov    0 scb.xt [edx], eax            \ save it
  lea    eax, 4 [esp]                   \ RSP on entry
  mov    0 scb.esp [edx], eax
  mov    0 scb.ebp [edx], ebp
  mov    0 scb.esi [edx], esi
  mov    0 scb.edi [edx], edi

  mov   dword ptr abort-code , # 0      \ no previous abort code
  ret
end-code

code PostProtFPExtern   \ -- ; R: sys -- ; SFP006
\ Test the abort code.
  mov   dword ptr XcallBuffer scb.xt , # 0      \ reset Extern in progress
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code

: ProtectedExterns \ --
Set the protected PRE and POST EXTERN handlers.

Interfacing to C++ DLLs

Caveats

These notes were written after testing on Visual C++ v6.0. Don't blame us if the rules change!

Example code

The example code may be found in the directory EXAMPLES\VC++. Because of the inordinate amount of time we spent wandering around inside debuggers to get this far, we recommend that you adopt a cooperative and investigative attitude when requesting technical support on this topic.

Accessing constructors and destructors

Example code for accessing the constructor of class is provided in TRYCPP.FTH which accesses the class DllTest in DLLTEST.CPP.

Since C++ is supposed to provide a higher level of abstraction, apparently simple operations may generate reams of code. So it is with the equivalent of

  pClass = new SomeClass;

The actual code generated may/will be a call to a function new to generate an object structure (not a single cell) followed by passing the return value from new to the class constructor.

The class constructor (in C++ CDllTest::CDllTest()) is not normally exported from C++ without some extra characters being added to the name. For example, the reference to it in the example code is:

  extern: PASCAL void * ??0CDLLTest@@QAE@XZ( void );

This function is not directly callable because it has to be passed the result of the new operator. To solve this problem DLLTest.dll contains a helper function CallNew which is passed the address of the constructor for the class. This is redefined as NEW for normal use.


\ C++ Helpers
extern: PASCAL void * CallNew( void * );
extern: PASCAL void CallDelete( void * this);


\ CDLLTest class specific
extern: PASCAL void * ??0CDLLTest@@QAE@XZ( void );

0 value CDLLTest        \ -- class|0

: InitCDLLTest  \ -- ; initialise the CPP interface
  ['] ??0CDLLTest@@QAE@XZ func-loaded? -> CDLLTest
;

: New   \ class -- *obj|0
  CallNew
;

: Delete        \ *obj --
  CallDelete
;

The word INITCDLLTEST gets the address of the constructor for the class, and NEW then runs the CallNew function which executes the C++ new operator and calls the constructor. Unfortunately, you will have to do this for each class that use in the DLL. What is returned by CallNew is an object pointer. This is not the object itself, but the address of another (undocumented) data structure. It can be used as the this pointer for all following member function calls.

Once you have finished with the onject, you must relase its resources using the delete method (the destructor). This is implemented in VC++ by passing the object pointer to the delete function. This is performed by the CallDelete function exported from the DLL. Again, the Forth word DELETE provides syntactic sugar by just calling CallDelete.

Accessing member functions

A Visual C++ member function exported from a DLL requires the "this" pointer in the ECX register. This can be achieved using the following form:


extern: VC++ PASCAL BOOL TestWindow1( void * this, char * ch, int n, int nbyte );

The function must be defined with an explicit this pointer (void * this). Because exported VC++ member functions can have either C or "PASCAL" styles, the this pointer must be positioned so that it is leftmost when reversed (C/PASCAL/WINAPI/StdCall/APIENTRY style) or is rightmost when not reversed ("PASCAL" style).


extern: PASCAL VC++ BOOL GetHello( void * this, char * buff, int len );
extern: PASCAL VC++ BOOL TestWindow1( void * this, char * ch, int n, int nbyte );
extern: PASCAL VC++ BOOL TestWindow2( void * this, void * pvoid, int ndword, int nlong );

0 value CDLLTest        \ -- constructor/class

: InitCDLLTest  \ --;  Initialise the CPP interface
  ['] ??0CDLLTest@@QAE@XZ func-loaded? to CDLLTest
;

create Magic#  $AAAA5555 ,      \ -- addr

#64 buffer: StringBuff  \ -- ; buffer for GetHello

: TestCDLLTest  \ -- ; test CDLLTest interface
  InitCDLLTest  CDLLTest if
    cr ." Initialisation succeeded"
    CDLLTest new ?dup if
      cr ." new succeeded"
      dup StringBuff #64 GetHello drop
      cr ." GetHello returns: " StringBuff .z$
      dup Magic# 4 5 TestWindow1 drop
      dup Magic# #20 #30 TestWindow2 drop
      delete
      cr ." delete done"
    else
      cr ." new failed"
    endif
  else
    cr ." Initialisation failed"
  endif
;

Please note that the actual code in TRYCPP.FTH may/will be different as we extend the facilities. See the source code itself!

Accessing third party C++ DLLs

Most third party C++ DLLs are provided with C header files which define the interfaces. Study of these will provide the information you need to determine how to access them.

For simple C++ classes, the DllTest.dll file can be used to provide constructor and destructor access. Note that classes with multiple constructors will export these as functions with the same basic name differentiated by the name mangling.

The DLL Fth2VC60.dll contains new and delete access for use with other DLLs. Note that the third party DLLs must be compatible with VC++ v6.0. The example file EXAMPLES\VC++\USECPP.FTH demonstrates using Fth2VC60.dll.


library: Fth2VC60.dll

extern: PASCAL void * FTH2CPPNew( void * constructor);
extern: PASCAL void FTH2CPPDelete( void * this);

: New           \ *class -- *obj|0
  FTH2CPPNew
;

: Delete        \ *obj --
  FTH2CPPDelete
;

If you are using an incompatible compiler or DLL, create a similar support DLL for that compiler. You can use the source code for Fth2VC60.dll as an example.

Changes at v4.3

The guts of the EXTERN: mechanism have been rewritten to provide more features and to support more operating systems.

Additional C types

The following C data types are now supported:

Float - a 32 bit floating point item.
Double - a 64 bit floating point item
LongLong - a 64 bit integer. These are taken from and returned to the Forth data stack as Forth double numbers with the high portion topmost on the stack.

Floating point numbers are taken from the NDP floating point unit. This is directly compatible with the the Forth floating point pack in Lib\Ndp387.fth.

More Operating Systems

The requirements of newer operating systems, especially those for 64-bit operation, are more stringent for things like data alignment. Consequently the underlying mechanism has changed.

Miscellaneous

These notes are probably only relevant for code that has carnal knowledge of the VFX Forth internals.

The XCALL primitive has been removed and is replaced by NXCALL.
The compile time code generation is completely different and there is no centralised despatch mechanism.
Some faciliies provided by Lib\Win32\SafeCallback.fth are now built in to the Windows system. You must use the new version of SafeCallback.fth.

The word NXCALL is provided for constructing your own import mechanisms, but it only deals with single-cell arguments and provides no type safety at all. It is used internally by VFX Forth in the first stage build of a console-mode kernel.

code NXCALL \ i*x addr i -- res
Calls the operating system function at addr with i arguments, returning the result res. As far as the operating systems is concerned, i*x appear on the CPU return stack pointed to by ESP, and the return value is taken from EAX. After executing NXCALL the return value res is the contents of the EAX register.