How it works now:
User defines an external function:
noreturn ExitProcess(u32 uExitCode);
void main() {
ExitProcess(0);
}
Then program is compiled like: vox main.vx C:\Windows\System32\kernel32.dll
The compiler then reads main.vx
and C:\Windows\System32\kernel32.dll
files.
Compiler parses the kernel32.dll
file and reads all exported functions into the hashmap of external symbols.
When compiling main.vx
module, compiler sees ExitProcess
external function definition and looks into external symbol hashmap and resolves to the symbols that was registered from kernel32.dll
.
When executable is generated compiler creates import table and inserts an entry for kernel32.dll
library with single imported symbol ExitProcess
.
Bads:
- All external symbols are dumped into the same namespace, which has potential for collisions.
- Compiler needs to read potentially big file into memory, which takes time.
- User needs to provide library path on CLI.
- When cross-compiling you need to have access to
kernel32.dll
file.
Goods:
- You can provide any library or host module that exports
ExitProcess
and will resolve by name. - No attributes needed
Now we can see all the interesting requirements:
- Namespaced name lookups (each external module has its own namespace and external symbol specifies namespace for lookup)
- Rebindability (when module name specified in the source code may differ from file name)
- Cross-platformity (Ideal: no .def files need to be distributed with the source; semi-ideal: .def files)
- Performance (Ideal: no file reads needed; semi-ideal: read file that only contains exported symbol names (
.def
)) - Reusing the same mechanism to bind not only to dynamic/shared library, but to any external module
- Minimizing CLI switches (by default file name should be assumed the same as external module name, since it is the most common scenario with system libraries)
Why is rebindability useful? .dll
/.so
file name can be different between the projects using the module that defines external function. Also it gives possibility to reuse the Vox module in embedded context.
Proposed solution:
@extern(module, "kernel32")
noreturn ExitProcess(u32 uExitCode);
- Use an attribute to mark external function
- That attribute must specify the name of external module. External module shall be a string, because file name may contain chars that are invalid in the identifiers
- Default: name of external module is used as a name of
.so
/.dll
library when compiling as an executable..dll
/.so` suffix will be added as (and if) required by the target plaform. - CLI switch is introduced to change the default name of external module to a custom one
--externModule=<external_mod>:<file_name>
- If non-namespaced lookups are forbidden, then all external functions must have the attribute attached
- External module name must not contain
.so
/.dll
suffix, as the same definition may be reused on different platforms and in embedded environment where it is bound to host defined module. - No dll is passed on the CLI
- In case user decides to name function diffently that it is exported in the external module another attribute can be specified.
@mangle("<name_to_lookup>")
@mangle("ExitProcess")
@extern(module, "kernel32")
noreturn exit(u32 uExitCode);
Now the example can look like:
@extern(module, "kernel32")
noreturn ExitProcess(u32 uExitCode);
void main() {
ExitProcess(0);
}
And compiled with vox main.vx
.
Compiler will create import entry for ExitProcess
in kernel32.dll
library.
To rebind the kernel32
external module user can do vox main.vx --externModule=kernel32:custom.dll
Edit:
After thinking more about the chosen solution, I found 1 case where passing .dll
file to the compiler is benefitial.
When compiler has a list of symbols from the dll it can verify that all references from @extern(module, "mod_name")
map to an existing symbol within the dll. Of course this means that exactly the same dll file must be loaded at load-time.
It's a small benefit, but it can be useful when the bindings are initially created to verify that everything is correct during build-time. Adding such an option on top of extern(module)
shouldn't be too hard.