Compilation for Linux, MacOS, and Windows #17

kuniss · 2023-03-05T21:07:39Z

applied matrix compilation and testing as suggested by the GitHub action for setting up dlang
MacOS compilation and testing worked out of the box, Windows alternative needed some adaptations...
adjustments to get internal D compiler invocation for generated compiler to run on Windows
enhancements to test helper functions to get tests passed on Windows
adjustments to some tests where Windows command line is completely different to Unix-like OS
renamed GitHub action build file d.yml to build.yml as later we will add a release.yml file

An example of an successful run may be found at here.

@linkrope, I created a PR to allow you to check style of the D source code adaptations, I did and to take the advantages of the GitHub PR tooling.

of the DMD compiler imvocation to enable compilation on Windows.

to get tests passed on Windows. Added Windows specific helper function asSingleLineDosInput is for inline EAG source code tests under Windows.

as Windows is very different here to Linux and MacOS and any Unix-kind OS.

and the according tests.

as later we want to add a "release.yml".

as already contained in example tests.

kuniss · 2023-12-23T18:26:11Z

@linkrope Merry Christmas! 😄
Did you see this PR?
I would add to it the executable publication as asset.
Probably I will use https://github.com/marketplace/actions/upload-assets-to-a-release for the publication...

which adds binaries to a published GitHub release.

as "latest" fails. linkrope did it too for dmd. The versino of ldc was looked up at https://github.com/ldc-developers/ldc/releases matching the version of Frontend, druntime and Phobos to that of dmd.

by ignoring all tags pushes as trigger.

as it is a power shell running by default, not a dos shell.

assuming those are the tags from a release publication. Otherwise, with '*' pattern, build.yml is not triggered at all.

to be ignored according to https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-excluding-branches-and-tags

get missed before. Maybe, another order works better...

as excluding version tags let any git push skip the build. Not clear why.

kuniss · 2023-12-27T23:12:29Z

Seems, I got the publishing of the 3 different gamma executables working too.
It builds on my fork as visible in this build.
And it gets published to the release page which triggered the build.

The only thing, I did not get managed is to avoid duplicate on tag pushing. It does not work as described at the GitHub Actions documentation...

as in master, see ci.yml.

src/gamma/main.d

.github/workflows/d.yml

.github/workflows/release.yml

@linkrope

as suggested by @linkrope and using it build().

as discussed on PR here: https://github.com/linkrope/gamma/pull/17/files/11c93a9229e028f778573de87b4f478dda96f508#r1437757527

from ci.yml, to refernce them later there and in release.yml

Misused due to a misunderstanding.

based on reusable workflows used at ci.yml and passing built binaries from build job using download-artifact action.

as it all happens on Linux, no Windows runner involved anymore here as in the first workflow version.

and within twice, once for the tag pushed for release and second for the release event. This was found in https://stackoverflow.com/questions/70743715/how-do-i-configure-a-github-actions-workflow-so-it-does-not-run-on-a-tag-push

into upload.yml to have the same abstractions levels for jobs in release.yml

as it shows an operational error on push. Seems, this token is implicitely inherited.

kuniss · 2024-01-04T22:45:12Z

I guess, I found a good solution now. Please review it.

The release binary upload to the release page is now done using the upload/download GH Actions, as it is intended, I guess.
For the result see

a CI build: https://github.com/linkrope/gamma/actions/runs/7415578645
a release build: https://github.com/kuniss/gamma/actions/runs/7415515452
a resulting release page with binaries: https://github.com/kuniss/gamma/releases/tag/v0.3.0-gamma.3

I set the uploaded artifact redemption period for a build to 1 day, as it is mainly needed for upload when a release is published. But we could extend it to 90 days max. What do you think?
On a release page the uploaded artifacts should stay forever...

@linkrope

as suggested by @linkrope in linkrope#17 (comment). Not tested yet on Windows as locally not available, currently. Runs wel on Linux.

linkrope · 2024-01-06T19:09:00Z

src/gamma/main.d

@@ -250,3 +254,11 @@ void build(string[] fileNames, string outputDirectory)
    if (status)
        exit(status);
 }
+
+string executableName(const string name)


Make it explicitly private:

private string executableName(const string name)

For my curiosity, why only this method, and not the others in main.d too?
I guess, I "copied" it from the other methods above.

Done with 49328b8.

linkrope · 2024-01-06T19:16:38Z

src/gamma/main.d

+        return name ~ ".exe";
+    else
+        return name;
+}


Get rid of this red "minus in circle": in VS Code, enable "insertFinalNewline"

I do not see the red circle well in my light theme. 😉

Thank for the configuration hint regarding "insertFinalNewline"! Didn't know it.

Done with 49328b8.

@linkrope

as suggested by @linkrope.

kuniss · 2024-01-06T22:15:01Z

@linkrope, I just encountered, that dub build is building a debug version:

denis@lisp:~/git/gamma$ dub build
    Starting Performing "debug" build using /usr/bin/dmd for x86_64.

Is my interpretation right?
If so, shouldn't we build a non-debug version when releasing?

linkrope · 2024-01-07T00:09:16Z

@linkrope, I just encountered, that dub build is building a debug version:
denis@lisp:~/git/gamma$ dub build
    Starting Performing "debug" build using /usr/bin/dmd for x86_64.
Is my interpretation right? If so, shouldn't we build a non-debug version when releasing?

See: https://dub.pm/dub-reference/buildtypes/#default-examples

("optimize", "inline", debugInfo") would be nice: fast code, still with helpful diagnostics.

Note that array bounds check is still enabled in release mode. There is a special build type for release builds with no bounds check. Similiarly, if you believe that contracts (pre -conditions, post-conditstions, invariants) are as valuable as bounds checks, then you should avoid release mode, where contracts are skipped. And use just "optimize" and "inline" instead. The "debug" is not about speed but about size: wouldn't you like to have debug symbols available to understand a backtrace in case of a fatal error?

kuniss · 2024-01-07T15:16:11Z

("optimize", "inline", debugInfo") would be nice: fast code, still with helpful diagnostics.

Note that array bounds check is still enabled in release mode. There is a special build type for release builds with no bounds check. Similiarly, if you believe that contracts (pre -conditions, post-conditstions, invariants) are as valuable as bounds checks, then you should avoid release mode, where contracts are skipped. And use just "optimize" and "inline" instead. The "debug" is not about speed but about size: wouldn't you like to have debug symbols available to understand a backtrace in case of a fatal error?

I tend to use --build=release-debug. That way, we would have bound checks and symbol informations in case of exceptions. As far as I understood, we do not have pre-, post-conditions and invariant specifications either. So, we do not need to check them, isn't it?

Or should we define our own build type in dub.json?

    "buildTypes": {
        "release-gamma": {
            "buildOptions": ["debugMode", "optimize", "inline", "debugInfo"]
        }
    }

kuniss · 2024-01-07T16:20:56Z

    "buildTypes": {
        "release-gamma": {
            "buildOptions": ["debugMode", "optimize", "inline", "debugInfo"]
        }
    }

Tried this in my PR detached branch and got a Linux binary that is bigger, the other binaries are smaller, see this release page.

I guess, we should run the verification tests with the same build type and LCD compiled to be sure it works, isn't it?

kuniss · 2024-01-07T16:45:21Z

I guess, we should run the verification tests with the same build type and LCD compiled to be sure it works, isn't it?

I tried this in this build run on the same branch as above: https://github.com/kuniss/gamma/actions/runs/7439461088
Works!

Now I'm thinking: Shouldn't we use LCD at ci.yml too? I mean, at the end we are interested in checking LCD compiled gamma is working well. That it is working well with DMD, the "developer" should check locally, isn't it?

linkrope · 2024-01-08T21:08:42Z

Now I'm thinking: Shouldn't we use LCD at ci.yml too? I mean, at the end we are interested in checking LCD compiled gamma is working well. That it is working well with DMD, the "developer" should check locally, isn't it?

This has to wait until "dmd" is no longer hard-coded in main.d? Otherwise, it could be tricky to install both compilers and use ldc to run the test, while dmd is used to compile the generated compiler.

@linkrope

as suggested by @linkrope in linkrope#17 (comment). Not tested yet on Windows as locally not available, currently. Runs wel on Linux.

@linkrope

as suggested by @linkrope in linkrope#17 (comment).

kuniss · 2024-04-13T21:30:14Z

I'm going to close this PR as it was substituted by PR #18 which has in general the same content but a newer origin base.

@linkrope

* Added Windows specific modifications of the DMD compiler imvocation to enable compilation on Windows. * Added executableName() for Windows' ".exe" extension as suggested by @linkrope and using it build(). * Get rid of OS specific file path separators as suggested by @linkrope in #17 (comment). * Enabled matrix build for Linux, MaxOS, and Windows and the according tests. * Renamed build.yml to ci.yml as discussed on PR here: https://github.com/linkrope/gamma/pull/17/files/11c93a9229e028f778573de87b4f478dda96f508#r1437757527 * Refactored out reusable workflows from ci.yml, to refernce them later there and in release.yml * Added uploading of built executables * Avoid CI workflow is running on release publications and within twice, once for the tag pushed for release and second for the release event. This was found in https://stackoverflow.com/questions/70743715/how-do-i-configure-a-github-actions-workflow-so-it-does-not-run-on-a-tag-push * Improved matrix names in GH Actions GUI * Refactored out upload workflow into upload.yml to have the same abstractions levels for jobs in release.yml * Attempt to optimize the built artifacts using a specific build type. * Switched to single workflow in build.yml for CI and release builds. * Get rid of DMD compiler at all for build. gamma now supports to use any $DC defined D compiler for target compiler compiling.

kuniss added 8 commits March 5, 2023 21:25

Added Windows specific modifications

1d0a6f5

of the DMD compiler imvocation to enable compilation on Windows.

Added Windows specifc modidications and helper function

1eec841

to get tests passed on Windows. Added Windows specific helper function asSingleLineDosInput is for inline EAG source code tests under Windows.

Added Windows specific test code alternatives

5bc752d

as Windows is very different here to Linux and MacOS and any Unix-kind OS.

Enabled matrix build for Linux, MaxOS, and Windows

ea14db9

and the according tests.

Renamed build script to "build.yml"

dcfb072

as later we want to add a "release.yml".

Removed unittest job

3e0d2c5

as already contained in example tests.

Switched to ldc-latest for faster code.

17a0e69

Swicthed back to DMD for testing.

acde0f7

kuniss and others added 11 commits December 27, 2023 19:11

Added a release worflow

6084930

which adds binaries to a published GitHub release.

Pinned dmd and ldc versions

0cb4ecf

as "latest" fails. linkrope did it too for dmd. The versino of ldc was looked up at https://github.com/ldc-developers/ldc/releases matching the version of Frontend, druntime and Phobos to that of dmd.

Pinned dmd and ldc versions

4ebccca

as "latest" fails. linkrope did it too for dmd. The versino of ldc was looked up at https://github.com/ldc-developers/ldc/releases matching the version of Frontend, druntime and Phobos to that of dmd.

Give executables unique names before upload

2467e84

Avoid GH releases are triggering build.yml

52150bf

by ignoring all tags pushes as trigger.

Corrected release exe renaming for Windows

3a6aafa

as it is a power shell running by default, not a dos shell.

Ignore tags starting with 'v' only

e6be91e

assuming those are the tags from a release publication. Otherwise, with '*' pattern, build.yml is not triggered at all.

One asterisk is enough for matching a tag

f1f16b6

to be ignored according to https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-excluding-branches-and-tags

Force a push to trigger a build

6d2ebe2

get missed before. Maybe, another order works better...

Revert back to original push event specification

5befcc2

as excluding version tags let any git push skip the build. Not clear why.

Ensure test is running before release upload.

405d93a

Upgraded dscanner to version 0.14.0

11c93a9

as in master, see ci.yml.

linkrope requested changes Dec 28, 2023

View reviewed changes

src/gamma/main.d Outdated Show resolved Hide resolved

src/gamma/main.d Outdated Show resolved Hide resolved

.github/workflows/d.yml Outdated Show resolved Hide resolved

.github/workflows/release.yml Outdated Show resolved Hide resolved

kuniss and others added 7 commits December 30, 2023 18:05

Added executableName() for Windows' ".exe" extension

9255ab6

as suggested by @linkrope and using it build().

Renamed build.yml to ci.yml

aac3c2a

as discussed on PR here: https://github.com/linkrope/gamma/pull/17/files/11c93a9229e028f778573de87b4f478dda96f508#r1437757527

Merge branch 'master' into cross-compilation

50d8851

Refactored out reusable workflows

2f39e92

from ci.yml, to refernce them later there and in release.yml

Referenced reusable workflows

fb4287e

Define compiler versions as workflow parameters

c071bc8

Added uploading of built executables

5425d17

kuniss added 9 commits January 3, 2024 22:15

Corrected source path for upload step

60c44b0

Misused due to a misunderstanding.

Bumped actions/upload-artifact to v4

4b544ed

Rewritten release workflow

79e239d

based on reusable workflows used at ci.yml and passing built binaries from build job using download-artifact action.

Corrected the renaming for the Windows binary

54388a8

as it all happens on Linux, no Windows runner involved anymore here as in the first workflow version.

Avoid CI workflow is running on release publications

4e54fcd

and within twice, once for the tag pushed for release and second for the release event. This was found in https://stackoverflow.com/questions/70743715/how-do-i-configure-a-github-actions-workflow-so-it-does-not-run-on-a-tag-push

Improved matrix names in GH Actions GUI

287aa96

Improved formating

1839c5b

Refactored out upload workflow

b534bcc

into upload.yml to have the same abstractions levels for jobs in release.yml

Removed secret declaration for GITHUB_TOKEN

ae44c3e

as it shows an operational error on push. Seems, this token is implicitely inherited.

Get rid of OS specific file path separators

de88bc1

as suggested by @linkrope in linkrope#17 (comment). Not tested yet on Windows as locally not available, currently. Runs wel on Linux.

linkrope reviewed Jan 6, 2024

View reviewed changes

linkrope approved these changes Jan 6, 2024

View reviewed changes

Made locally only used method private

49328b8

as suggested by @linkrope.

kuniss added a commit to kuniss/gamma that referenced this pull request Apr 11, 2024

Get rid of OS specific file path separators

b2909b6

as suggested by @linkrope in linkrope#17 (comment). Not tested yet on Windows as locally not available, currently. Runs wel on Linux.

kuniss added a commit to kuniss/gamma that referenced this pull request Apr 13, 2024

Get rid of OS specific file path separators

4b9d114

as suggested by @linkrope in linkrope#17 (comment).

kuniss mentioned this pull request Apr 13, 2024

Cross compilation for Linux, MacOS, and Windows #18

Merged

kuniss closed this Apr 13, 2024

kuniss deleted the cross-compilation branch July 10, 2024 21:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compilation for Linux, MacOS, and Windows #17

Compilation for Linux, MacOS, and Windows #17

kuniss commented Mar 5, 2023

kuniss commented Dec 23, 2023

kuniss commented Dec 27, 2023

kuniss commented Jan 4, 2024

linkrope Jan 6, 2024

kuniss Jan 6, 2024

kuniss Jan 7, 2024

linkrope Jan 6, 2024

kuniss Jan 6, 2024

kuniss Jan 6, 2024

kuniss Jan 7, 2024

kuniss commented Jan 6, 2024

linkrope commented Jan 7, 2024

kuniss commented Jan 7, 2024 •

edited

Loading

kuniss commented Jan 7, 2024

kuniss commented Jan 7, 2024 •

edited

Loading

linkrope commented Jan 8, 2024

kuniss commented Apr 13, 2024

Compilation for Linux, MacOS, and Windows #17

Compilation for Linux, MacOS, and Windows #17

Conversation

kuniss commented Mar 5, 2023

kuniss commented Dec 23, 2023

kuniss commented Dec 27, 2023

kuniss commented Jan 4, 2024

linkrope Jan 6, 2024

Choose a reason for hiding this comment

kuniss Jan 6, 2024

Choose a reason for hiding this comment

kuniss Jan 7, 2024

Choose a reason for hiding this comment

linkrope Jan 6, 2024

Choose a reason for hiding this comment

kuniss Jan 6, 2024

Choose a reason for hiding this comment

kuniss Jan 6, 2024

Choose a reason for hiding this comment

kuniss Jan 7, 2024

Choose a reason for hiding this comment

kuniss commented Jan 6, 2024

linkrope commented Jan 7, 2024

kuniss commented Jan 7, 2024 • edited Loading

kuniss commented Jan 7, 2024

kuniss commented Jan 7, 2024 • edited Loading

linkrope commented Jan 8, 2024

kuniss commented Apr 13, 2024

kuniss commented Jan 7, 2024 •

edited

Loading

kuniss commented Jan 7, 2024 •

edited

Loading