Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build in docker #2841

Closed
wants to merge 2 commits into from
Closed

Build in docker #2841

wants to merge 2 commits into from

Conversation

kopp
Copy link

@kopp kopp commented Dec 16, 2024

Add scripts to have all build steps run in a docker to make building more reproducible.

@kopp
Copy link
Author

kopp commented Dec 16, 2024

unfortunately currently building the js for the python plugin fails with the following error:

% cd bindings/kepler.gl-jupyter/js
% bash build_in_docker.sh

removed 54 packages, and changed 102 packages in 3s

11 packages are looking for funding
  run `npm fund` for details
➤ YN0065: Yarn will periodically gather anonymous telemetry: https://yarnpkg.com/advanced/telemetry
➤ YN0065: Run yarn config set --home enableTelemetry 0 to disable

➤ YN0000: · Yarn 4.4.0
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0s 289ms
➤ YN0000: ┌ Post-resolution validation
➤ YN0060: │ eslint is listed by your project with version 5.16.0 (p89060), which doesn't satisfy what eslint-config-uber-es2015 and other dependencies request (but they have non-overlapping ranges!).
➤ YN0060: │ react is listed by your project with version 18.2.0 (pf32e1), which doesn't satisfy what react-side-effect (via react-helmet) and other dependencies request (but they have non-overlapping ranges!).
➤ YN0060: │ webpack is listed by your project with version 4.47.0 (p43939), which doesn't satisfy what node-polyfill-webpack-plugin and other dependencies request (but they have non-overlapping ranges!).
➤ YN0002: │ keplergl-jupyter@workspace:. doesn't provide prettier (pccd90), requested by eslint-plugin-prettier.
➤ YN0002: │ keplergl-jupyter@workspace:. doesn't provide typescript (p23fad), requested by react-dev-utils and other dependencies.
➤ YN0086: │ Some peer dependencies are incorrectly met by your project; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code.
➤ YN0086: │ Some peer dependencies are incorrectly met by dependencies; run yarn explain peer-requirements for details.
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0013: │ 1785 packages were added to the project (+ 483.53 MiB).
➤ YN0000: └ Completed in 18s 946ms
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed in 0s 780ms
➤ YN0000: · Done with warnings in 20s 266ms

> [email protected] clean
> rimraf dist/ && rimraf ../keplergl/static/

npm notice
npm notice New major version of npm available! 8.19.2 -> 10.9.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.2
npm notice Run npm install -g [email protected] to update!
npm notice

> [email protected] build:lab
> NODE_OPTIONS=--openssl-legacy-provider rimraf babel/ && mkdir babel && babel lib --out-dir babel

Successfully compiled 15 files with Babel.
 Hash: 8f19357d08cf8f8071bc7e9d9ff0d5178eee7447c599669f060ea6bccf2114fb500a09402e89d55d
Version: webpack 4.47.0
Child
    Hash: 8f19357d08cf8f8071bc
    Time: 88ms
    Built at: 12/16/2024 10:00:34 AM
     2 assets
    Entrypoint main = extension.js extension.js.map
    [0] ./lib/extension.js 966 bytes {0} [built]

    ERROR in extension.js from Terser
    Error: error:0308010C:digital envelope routines::unsupported
        at new Hash (node:internal/crypto/hash:69:19)
        at Object.createHash (node:crypto:133:10)
        at /keplergl/node_modules/terser-webpack-plugin/dist/index.js:217:37
        at Array.forEach (<anonymous>)
        at TerserPlugin.optimizeFn (/keplergl/node_modules/terser-webpack-plugin/dist/index.js:160:259)
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:7:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1409:36
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1405:32
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at Compilation.seal (/keplergl/node_modules/webpack/lib/Compilation.js:1342:27)
        at /keplergl/node_modules/webpack/lib/Compiler.js:675:18
        at /keplergl/node_modules/webpack/lib/Compilation.js:1261:4
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at Compilation.finish (/keplergl/node_modules/webpack/lib/Compilation.js:1253:28)
        at /keplergl/node_modules/webpack/lib/Compiler.js:672:17
        at eval (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1185:12
        at /keplergl/node_modules/webpack/lib/Compilation.js:1097:9
        at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Child
    Hash: 7e9d9ff0d5178eee7447
    Time: 131559ms
    Built at: 12/16/2024 10:02:46 AM
     2 assets
    Entrypoint main = index.js index.js.map
      [47] (webpack)/buildin/global.js 472 bytes {0} [built]
      [57] ./node_modules/react-intl/lib/index.js + 36 modules 166 KiB {0} [built]
           |    37 modules
      [69] ./node_modules/reselect/es/index.js + 1 modules 8.85 KiB {0} [built]
           |    2 modules
     [106] ./node_modules/redux/es/redux.js + 1 modules 27.4 KiB {0} [built]
           |    2 modules
     [275] ./node_modules/react-redux/es/index.js + 27 modules 48.6 KiB {0} [built]
           |    28 modules
     [442] ./node_modules/apache-arrow/Arrow.dom.mjs + 126 modules 564 KiB {0} [built]
           |    127 modules
     [660] ./lib/log.js 248 bytes {0} [built]
    [1391] ./lib/index.js 263 bytes {0} [built]
    [1404] ./lib/keplergl-plugin.js 7.62 KiB {0} [built]
    [1405] external "@jupyter-widgets/base" 42 bytes {0} [built]
    [1406] ./lib/keplergl/kepler.gl.js 8.84 KiB {0} [built]
    [1432] min-document (ignored) 15 bytes {0} [built]
    [2523] ./lib/keplergl/store.js 2.61 KiB {0} [built]
    [2524] ./lib/keplergl/utils.js 3.32 KiB {0} [built]
    [2525] ./package.json 4.05 KiB {0} [built]
        + 2706 hidden modules

    ERROR in index.js from Terser
    Error: error:0308010C:digital envelope routines::unsupported
        at new Hash (node:internal/crypto/hash:69:19)
        at Object.createHash (node:crypto:133:10)
        at /keplergl/node_modules/terser-webpack-plugin/dist/index.js:217:37
        at Array.forEach (<anonymous>)
        at TerserPlugin.optimizeFn (/keplergl/node_modules/terser-webpack-plugin/dist/index.js:160:259)
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:7:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1409:36
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1405:32
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at Compilation.seal (/keplergl/node_modules/webpack/lib/Compilation.js:1342:27)
        at /keplergl/node_modules/webpack/lib/Compiler.js:675:18
        at /keplergl/node_modules/webpack/lib/Compilation.js:1261:4
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at Compilation.finish (/keplergl/node_modules/webpack/lib/Compilation.js:1253:28)
        at /keplergl/node_modules/webpack/lib/Compiler.js:672:17
        at eval (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1185:12
        at /keplergl/node_modules/webpack/lib/Compilation.js:1097:9
        at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Child
    Hash: c599669f060ea6bccf21
    Time: 104073ms
    Built at: 12/16/2024 10:02:18 AM
     1 asset
    Entrypoint main = index.js
      [47] (webpack)/buildin/global.js 472 bytes {0} [built]
      [57] ./node_modules/react-intl/lib/index.js + 36 modules 166 KiB {0} [built]
           |    37 modules
      [69] ./node_modules/reselect/es/index.js + 1 modules 8.85 KiB {0} [built]
           |    2 modules
     [106] ./node_modules/redux/es/redux.js + 1 modules 27.4 KiB {0} [built]
           |    2 modules
     [275] ./node_modules/react-redux/es/index.js + 27 modules 48.6 KiB {0} [built]
           |    28 modules
     [442] ./node_modules/apache-arrow/Arrow.dom.mjs + 126 modules 564 KiB {0} [built]
           |    127 modules
     [660] ./lib/log.js 248 bytes {0} [built]
    [1391] ./lib/embed.js 538 bytes {0} [built]
    [1404] ./lib/keplergl-plugin.js 7.62 KiB {0} [built]
    [1405] external "@jupyter-widgets/base" 42 bytes {0} [built]
    [1406] ./lib/keplergl/kepler.gl.js 8.84 KiB {0} [built]
    [1432] min-document (ignored) 15 bytes {0} [built]
    [2523] ./lib/keplergl/store.js 2.61 KiB {0} [built]
    [2524] ./lib/keplergl/utils.js 3.32 KiB {0} [built]
    [2525] ./package.json 4.05 KiB {0} [built]
        + 2706 hidden modules

    ERROR in index.js from Terser
    Error: error:0308010C:digital envelope routines::unsupported
        at new Hash (node:internal/crypto/hash:69:19)
        at Object.createHash (node:crypto:133:10)
        at /keplergl/node_modules/terser-webpack-plugin/dist/index.js:217:37
        at Array.forEach (<anonymous>)
        at TerserPlugin.optimizeFn (/keplergl/node_modules/terser-webpack-plugin/dist/index.js:160:259)
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:7:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1409:36
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1405:32
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at Compilation.seal (/keplergl/node_modules/webpack/lib/Compilation.js:1342:27)
        at /keplergl/node_modules/webpack/lib/Compiler.js:675:18
        at /keplergl/node_modules/webpack/lib/Compilation.js:1261:4
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at Compilation.finish (/keplergl/node_modules/webpack/lib/Compilation.js:1253:28)
        at /keplergl/node_modules/webpack/lib/Compiler.js:672:17
        at eval (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1185:12
        at /keplergl/node_modules/webpack/lib/Compilation.js:1097:9
        at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Child
    Hash: 14fb500a09402e89d55d
    Time: 102068ms
    Built at: 12/16/2024 10:02:16 AM
     1 asset
    Entrypoint main = main.js
       [0] external {"root":"React","commonjs2":"react","commonjs":"react"} 42 bytes {0} [built]
      [14] external {"root":"styled","commonjs2":"styled-components","commonjs":"styled-components"} 42 bytes {0} [built]
      [43] external {"root":"ReactDOM","commonjs2":"react-dom","commonjs":"react-dom"} 42 bytes {0} [built]
      [47] (webpack)/buildin/global.js 472 bytes {0} [built]
      [56] external {"root":"ReactIntl","commonjs2":"react-intl","commonjs":"react-intl"} 42 bytes {0} [built]
     [103] external {"root":"Redux","commonjs2":"redux","commonjs":"redux"} 42 bytes {0} [built]
     [267] external {"root":"ReactRedux","commonjs2":"react-redux","commonjs":"react-redux"} 42 bytes {0} [built]
     [437] ./node_modules/apache-arrow/Arrow.dom.mjs + 126 modules 564 KiB {0} [built]
           |    127 modules
     [679] ./lib/keplergl/store.js 2.61 KiB {0} [built]
    [1342] ./lib/log.js 248 bytes {0} [built]
    [1382] ./lib/keplergl/main.js 1.31 KiB {0} [built]
    [1422] min-document (ignored) 15 bytes {0} [built]
    [2489] external {"root":"Helmet","commonjs2":"react-helmet","commonjs":"react-helmet"} 42 bytes {0} [built]
    [2490] ./lib/keplergl/kepler.gl.js 8.84 KiB {0} [built]
    [2491] ./lib/keplergl/utils.js 3.32 KiB {0} [built]
        + 2617 hidden modules

    WARNING in configuration
    The 'mode' option has not been set, webpack will fallback to 'production' for this value. Set 'mode' option to 'development' or 'production' to enable defaults for each environment.
    You can also set it to 'none' to disable any default behavior. Learn more: https://webpack.js.org/configuration/mode/

    ERROR in main.js from Terser
    Error: error:0308010C:digital envelope routines::unsupported
        at new Hash (node:internal/crypto/hash:69:19)
        at Object.createHash (node:crypto:133:10)
        at /keplergl/node_modules/terser-webpack-plugin/dist/index.js:217:37
        at Array.forEach (<anonymous>)
        at TerserPlugin.optimizeFn (/keplergl/node_modules/terser-webpack-plugin/dist/index.js:160:259)
        at AsyncSeriesHook.eval [as callAsync] (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:7:1)
        at AsyncSeriesHook.lazyCompileHook (/keplergl/node_modules/tapable/lib/Hook.js:154:20)
        at /keplergl/node_modules/webpack/lib/Compilation.js:1409:36
        at eval (eval at create (/keplergl/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
        at /keplergl/node_modules/html-webpack-plugin/lib/cached-child-compiler.js:267:53
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    Child HtmlWebpackCompiler:
         1 asset
        Entrypoint HtmlWebpackPlugin_0 = __child-HtmlWebpackPlugin_0
        [0] ./node_modules/html-webpack-plugin/lib/loader.js!./template/keplergl-html.ejs 5.6 KiB {0} [built]

@kopp kopp mentioned this pull request Dec 16, 2024
Merged
@lixun910
Copy link
Collaborator

"Error: error:0308010C:digital envelope routines::unsupported" looks familiar:

This error typically occurs with Node.js versions 17 and above due to changes in the OpenSSL configuration. To fix this: run the command "export NODE_OPTIONS=--openssl-legacy-provider" before starting the build

If you did not try it: run it before build_in_docker.sh?

@kopp
Copy link
Author

kopp commented Dec 20, 2024

Ok, I had another look at the code and wrote down what I understood at #2848 . Could you, @lixun910 , please have a look at that? It is probably not perfect and not complete -- but my current understanding. If you think that this makes sense, I'd also try to add Dockerfiles that 'codify' my understanding.

After using the node version that kepler.gl it self uses, I do not get the errors above -- but now I die due to low memory when webpack runs (my machine has 'only' 12GB -- how much do I need?), so it looks like I cannot compile the example, i.e. cannot test my own instructions. I'll try to get my hand on a machine with more memory...

@lixun910 lixun910 deleted the branch keplergl:xli-kepler-jupyter-034-a1 December 27, 2024 05:09
@lixun910 lixun910 closed this Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kopp @lixun910