runtime/jit-rt: revive and migrates the whole thing to OrcJIT v2

[liushuyu]

This pull request migrates the dynamic compile system to LLVM OrcJIT v2 (the newer "v2" version from post-LLVM 16 with opaque pointers, not the older "v2" version from the LLVM 12 ~ 15 era).

The implementation is currently in very rough shape, and I am not entirely sure what I am doing. However, the "Hello World" example in the documentation has started to work now.

[kinke]

Hehe, looks like one's motivated, nice! - In case you haven't seem them, there are quite a few lit-tests for this feature: https://github.com/ldc-developers/ldc/tree/master/tests/dynamiccompile. No idea if anything can still be salvaged from #3184, Ivan's WIP from 5 (!) years ago by now.

[liushuyu]

Hehe, looks like one's motivated, nice! - In case you haven't seem them, there are quite a few lit-tests for this feature: https://github.com/ldc-developers/ldc/tree/master/tests/dynamiccompile. No idea if anything can still be salvaged from #3184, Ivan's WIP from 5 (!) years ago by now.

I see. I did not know there was another pull request doing the same thing from 5 years ago. Now that I see they made very similar changes before, I realize I duplicated a fair amount of changes (but this seems like a form of cross-validation? Two independent implementations look very similar).

[liushuyu]

there are quite a few lit-tests for this feature: https://github.com/ldc-developers/ldc/tree/master/tests/dynamiccompile.

Now, 100% of those LIT tests are passing (with LLVM 18).

[kinke]

Very nice progress! - I've pushed 2 commits to reduce the remaining failures on aarch64.

[kinke]

In my Windows VM, the tests don't hang as they do for CI (with a pushed Win32 fix wrt. __chkstk, now on 32-bit Windows too). Instead, some dynamic-compile tests pass (on both 32 and 64-bit Windows), others fail with errors like:

• dynamiccompile/simd_simple_opt.d:
  • 64-bit:

    Dynamic compiler fatal: Symbol not found in jitted code: "�.jit_bind" (".jit_bind")
    

  • 32-bit:

    Incorrect number of arguments passed to called function!
      call x86_stdcallcc void @"\01__D3ldc15dynamic_compile__T4bindTPFG16hZG32hTxG16hZQyFQvxQlZ7wrapperFSQCpQCo__TQCaTQByTxQBqZQCnFQClxQCcZ7ContextQCzZQCy"(ptr inreg noalias align 1 %3, ptr byval(%"ldc.dynamic_compile.bind!(ubyte[32] function(ubyte[16]), const(ubyte[16])).bind.Context") align 4 %4) #0
    LLVM ERROR: Broken module found, compilation aborted!
    

• dynamiccompile/simple.d:
  • 64-bit:

    Dynamic compiler fatal: Symbol not found in jitted code: "�_D6simple3bazFZv" ("_D6simple3bazFZv")
    

  • 32-bit:

    Dynamic compiler fatal: Symbol not found in jitted code: "�__D6simple3barFZi" ("__D6simple3barFZi")
    

Looks like there's some trouble with the \1 (�) mangle prefix on Windows, which is used to make LLVM not apply any extra mangling (e.g., extra leading underscore on Win32, and vectorcall/stdcall suffix on Win64).

Edit: We do use \1.jit_bind in general for all targets (https://github.com/liushuyu/ldc/blob/9531ab3828e6b4d1fe675cfcc15085d4ce948155/runtime/jit-rt/cpp-so/bind.cpp#L47-L48), but it's only a problem on Windows. So maybe not name/mangle-related after all, but some other Windows-specific issue.

[liushuyu]

dynamiccompile/simple.d:

• 64-bit:

  Dynamic compiler fatal: Symbol not found in jitted code: "�_D6simple3bazFZv" ("_D6simple3bazFZv")
  

I have tried to debug this problem on a Windows 11 installation with Visual Studio 2022 (Community Edition).
It seems like either our dynamic compile mechanism setup LLVM in the wrong way, or there is an LLVM bug on Windows: the symbols were not materialized when they should be (LLVM on Unix platforms will materialize the JIT symbols during the lookup, i.e. lazily compile/resolve the symbols).

[liushuyu]

On Ubuntu 20.04 aarch64:

********************
Unexpectedly Passed Tests (1):
  LDC :: dynamiccompile/throw.d

But it's still broken on macOS aarch64 unfortunately

[liushuyu]

It seems like I may have discovered several bugs in LLVM that are exclusive to Windows. I have currently worked around them because I am not entirely sure if they are "bugs" or "intended behaviors".

[liushuyu]

The CI failures are due to network issues; a retry should fix them.

[thewilsonator]

retry should fix them.

(restarted)

[liushuyu]

retry should fix them.

(restarted)

Thanks! All CI tests passed now

[thewilsonator]

Did you resolve #4774 (comment) ?

[kinke]

I've pushed a commit merging the two very similar optimizer.cpp versions; a mechanical refactoring without any intended functional changes. There's probably more room for follow-up refactorings, such as moving setRtCompileVar() to another jitrt-specific module.

[liushuyu]

I've pushed a commit merging the two very similar optimizer.cpp versions; a mechanical refactoring without any intended functional changes. There's probably more room for follow-up refactorings, such as moving setRtCompileVar() to another jitrt-specific module.

Thanks!

Hopefully, by switching to the more stable easy LLJIT API, we can last longer before another need for refactoring comes.

[thewilsonator]

Anything else to be done here?