Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File "C:\emsdk-main\upstream\emscripten\tools\cache.py", line 68, in lock acquire_cache_lock(reason) #22941

Closed
Benman2785 opened this issue Nov 15, 2024 · 15 comments · Fixed by #22946

Comments

@Benman2785
Copy link

Benman2785 commented Nov 15, 2024

latest emscripten installed today
VS2022 installed today

commands for compiling:

emcmake cmake -S. -Bbuild-em -DCMAKE_BUILD_TYPE=Release
embuilder build sdl2 sdl2_ttf sdl2_image sdl2_image_jpg sdl2_image_png
cmake --build build-em -j $(getconf _NPROCESSORS_ONLN)

File "C:\emsdk-main\upstream\emscripten\tools\cache.py", line 68, in lock
acquire_cache_lock(reason)
File "C:\emsdk-main\upstream\emscripten\tools\cache.py", line 42, in acquire_cache_lock
assert 'EM_CACHE_IS_LOCKED' not in os.environ, f'attempt to lock the cache while a parent process is holding the lock ({reason})'
AssertionError: attempt to lock the cache while a parent process is holding the lock (sysroot\lib\wasm32-emscripten\thinlto\libSDL2.a)

googled it and found same error in 2020 and 2023 -> all said its fixed on windows -> but it isnt

actually first time i didnt use

embuilder build sdl2 sdl2_ttf sdl2_image sdl2_image_jpg sdl2_image_png

but i restarted pc

full log: emsdk-err.log

@sbc100
Copy link
Collaborator

sbc100 commented Nov 15, 2024

Which version of emsdk are you using?

I can't seem to reproduce this running ./embuilder build sdl2 sdl2_ttf sdl2_image sdl2_image_jpg sdl2_image_png locally:

$ ./embuilder build sdl2 sdl2_ttf sdl2_image sdl2_image_jpg sdl2_image_png --force
embuilder:INFO: building sdl2
cache:INFO: deleting cached file: /usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2.a
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2.a... (this will be cached in "/usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2.a" for subsequent builds)
system_libs:INFO: compiled 118 inputs in 2.18s
cache:INFO:  - ok
embuilder:INFO: ...success. Took (2.31s)
embuilder:INFO: building sdl2_ttf
cache:INFO: deleting cached file: /usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_ttf.a
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2_ttf.a... (this will be cached in "/usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_ttf.a" for subsequent builds)
system_libs:INFO: compiled 1 inputs in 1.72s
cache:INFO:  - ok
embuilder:INFO: ...success. Took (1.81s)
embuilder:INFO: building sdl2_image
cache:INFO: deleting cached file: /usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_image.a
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2_image.a... (this will be cached in "/usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_image.a" for subsequent builds)
system_libs:INFO: compiled 19 inputs in 0.33s
cache:INFO:  - ok
embuilder:INFO: ...success. Took (0.42s)
embuilder:INFO: building sdl2_image_jpg
cache:INFO: deleting cached file: /usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_image_jpg.a
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2_image_jpg.a... (this will be cached in "/usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_image_jpg.a" for subsequent builds)
system_libs:INFO: compiled 19 inputs in 0.34s
cache:INFO:  - ok
embuilder:INFO: ...success. Took (0.43s)
embuilder:INFO: building sdl2_image_png
cache:INFO: deleting cached file: /usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_image_png.a
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2_image_png.a... (this will be cached in "/usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2_image_png.a" for subsequent builds)
system_libs:INFO: compiled 19 inputs in 0.39s
cache:INFO:  - ok
embuilder:INFO: ...success. Took (0.49s)
embuilder:INFO: Built 5 targets in (5.48s)

Can you confirm, that that command is that one that fails for you?

@sbc100
Copy link
Collaborator

sbc100 commented Nov 15, 2024

I can reproduce this now:

$ EMCC_CORES=1 ./emcc test/hello_world.c -flto -sUSE_SDL=2 -sUSE_PTHREADS=1 -sUSE_SDL_IMAGE=2 -sSDL2_IMAGE_FORMATS=png
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/lto/libSDL2_image_png.a... (this will be cached in "/usr/local/google/home/sbc/dev/wasm/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libSDL2_image_png.a" for subsequent builds)
Traceback (most recent call last):
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/emcc.py", line 1637, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/emcc.py", line 1630, in main
    ret = run(args)
          ^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/emcc.py", line 675, in run
    linker_inputs = phase_compile_inputs(options, state, newargs, input_files)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/emcc.py", line 1039, in phase_compile_inputs
    cmd = get_clang_command() + inputs
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/emcc.py", line 1000, in get_clang_command
    return compiler + get_cflags(state.orig_args, use_cxx) + compile_args
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/emcc.py", line 464, in get_cflags
    ports.add_cflags(cflags, settings)
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/tools/ports/__init__.py", line 580, in add_cflags
    port.get(Ports, settings, shared)
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/tools/ports/libpng.py", line 49, in get
    return [shared.cache.get_lib(get_lib_name(settings), create, what='port')]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/tools/cache.py", line 150, in get_lib
    return get(name, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/tools/cache.py", line 168, in get
    with lock(shortname):
  File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/tools/cache.py", line 68, in lock
    acquire_cache_lock(reason)
  File "/usr/local/google/home/sbc/dev/wasm/emscripten/tools/cache.py", line 42, in acquire_cache_lock
    assert 'EM_CACHE_IS_LOCKED' not in os.environ, f'attempt to lock the cache while a parent process is holding the lock ({reason})'
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: attempt to lock the cache while a parent process is holding the lock (sysroot/lib/wasm32-emscripten/lto/libpng.a)
emcc: error: subprocess 1/19 failed (returned 1)! (cmdline: /usr/local/google/home/sbc/dev/wasm/emscripten/emcc -c /usr/local/google/home/sbc/dev/wasm/emscripten/cache/ports/sdl2_image/SDL_image-release-2.6.0/IMG.c -o /usr/local/google/home/sbc/dev/wasm/emscripten/cache/ports-builds/sdl2_image/IMG.c.o -g -sSTRICT -Werror -flto=full -O2 -I/usr/local/google/home/sbc/dev/wasm/emscripten/cache/ports/sdl2_image/SDL_image-release-2.6.0 -O2 -sUSE_SDL=2 -Wno-format-security -DLOAD_PNG -sUSE_LIBPNG)

@sbc100
Copy link
Collaborator

sbc100 commented Nov 15, 2024

Can you try running ./embuilder build sdl2_image_png --lto. It seems to work as a workaround to build the library up front.

@sbc100
Copy link
Collaborator

sbc100 commented Nov 15, 2024

I think this is essentially the same issue as #20204

sbc100 added a commit to sbc100/emscripten that referenced this issue Nov 16, 2024
Because some possible dependencies of these libraries have `-mt`
variants (specifically png and harfbuzz) we also needs to declare `-mt`
variant of these libraries.

The reason for this is a little complex: When we find the set of
transitive dependencies of given library we select the correct variant
of each library.  This means that if we are building with `-pthread`
then we select and include the `-mt` variants of all dependencies.
However if libsdl2_image or libsdl2_ttf themselves then need to be built
we end up building them without `-pthread` which means that emcc
subprocesses will try to select and build the single threaded variants
of the dependencies.  This is mostly a serious problem because we don't
allow for nested calls to emcc (we assume all dependencies have been
already built before we try to build a given library and the we error
out with `EM_CACHE_IS_LOCKED` if that is not the case).

Testing this fix requires the cache to be setup just right so I'm not
sure its worth it.

Fixes: emscripten-core#22941, emscripten-core#20204
sbc100 added a commit to sbc100/emscripten that referenced this issue Nov 16, 2024
Because some possible dependencies of these libraries have `-mt`
variants (specifically png and harfbuzz) we also needs to declare `-mt`
variant of these libraries.

The reason for this is a little complex: When we find the set of
transitive dependencies of given library we select the correct variant
of each library.  This means that if we are building with `-pthread`
then we select and include the `-mt` variants of all dependencies.
However if libsdl2_image or libsdl2_ttf themselves then need to be built
we end up building them without `-pthread` which means that emcc
subprocesses will try to select and build the single threaded variants
of the dependencies.  This is mostly a serious problem because we don't
allow for nested calls to emcc (we assume all dependencies have been
already built before we try to build a given library and the we error
out with `EM_CACHE_IS_LOCKED` if that is not the case).

Testing this fix requires the cache to be setup just right so I'm not
sure its worth it.

Fixes: emscripten-core#22941, emscripten-core#20204
@Benman2785
Copy link
Author

error persists even with --lto or --lto --force

@sbc100
Copy link
Collaborator

sbc100 commented Nov 18, 2024

A fix is in flight: #22946

sbc100 added a commit to sbc100/emscripten that referenced this issue Nov 18, 2024
Because some possible dependencies of these libraries have `-mt`
variants (specifically png and harfbuzz) we also needs to declare `-mt`
variant of these libraries.

The reason for this is a little complex: When we find the set of
transitive dependencies of given library we select the correct variant
of each library.  This means that if we are building with `-pthread`
then we select and include the `-mt` variants of all dependencies.
However if libsdl2_image or libsdl2_ttf themselves then need to be built
we end up building them without `-pthread` which means that emcc
subprocesses will try to select and build the single threaded variants
of the dependencies.  This is mostly a serious problem because we don't
allow for nested calls to emcc (we assume all dependencies have been
already built before we try to build a given library and the we error
out with `EM_CACHE_IS_LOCKED` if that is not the case).

Testing this fix requires the cache to be setup just right so I'm not
sure its worth it.

Fixes: emscripten-core#22941, emscripten-core#20204
@sbc100 sbc100 closed this as completed in 26b1978 Nov 18, 2024
@Benman2785
Copy link
Author

emscripten_bug.txt

still cant compile :/

@sbc100
Copy link
Collaborator

sbc100 commented Nov 19, 2024

emscripten_bug.txt

still cant compile :/

I'm having trouble replicating. Are you sure you are using the fix from #22946?

@sbc100
Copy link
Collaborator

sbc100 commented Nov 19, 2024

I was able to replicate the your error before #22946 landed but not after so I think maybe you don't have the fix. Did you do emsdk install tot? Can you try running that again to make sure you have the fix?

@Benman2785
Copy link
Author

i did: emsdk update && emsdk install latest

will try with install tot

@Benman2785
Copy link
Author

emsdk.bat update
emsdk.bat install latest
emsdk.bat install tot
** until here all is fine **
embuilder build sdl2 sdl2_ttf sdl2_image sdl2_image-jpg sdl2_image-png --lto --force
** i need *-jpg & *-png as _ is wrong **
** now i try to compile **
emcmake cmake -S. -Bbuild-em -DCMAKE_BUILD_TYPE=Release
cmake --build build-em -j

still cant compile - but get way further -> now it the untested code of DevilutionX can be the problem -> will look into it when i have more time

thank you :)

@sbc100
Copy link
Collaborator

sbc100 commented Nov 19, 2024

Great news. Glad that old bug is finally fixed.

@Benman2785
Copy link
Author

Benman2785 commented Nov 19, 2024

btw - you should also edit your manual: emsdk.bat should NEVER been in an admin CMD :)
because if you use VisualStudio and have installed emscripten via admin cmd there are problems too

@sbc100
Copy link
Collaborator

sbc100 commented Nov 19, 2024

btw - you should also edit your manual: emsdk.bat should NEVER been in an admin CMD :) because if you use VisualStudio and have installed emscripten via admin cmd there are problems too

Perhaps you could elaborate? What kind of problems? Could you perhaps open new bug in the emsdk repo for that?

@Benman2785
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants