Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't build #64

Open
olivierlaporte opened this issue Feb 21, 2023 · 10 comments
Open

Can't build #64

olivierlaporte opened this issue Feb 21, 2023 · 10 comments

Comments

@olivierlaporte
Copy link

Hello I try to reproduce the work produced by arbdevml for issue #49.

I fork the repository.
I have docker, make installed.

I reproduce the described steps :

  • on vosk-browser folder
make builder

I have the following output :

#7 [ 4/30] RUN git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579
#7 sha256:d72b762a9137ae3da9126377d52f4ac1e5fb4134afc31851f4a093636254bbdc
#7 0.474 Cloning into '.'...
#7 19.32 fatal: reference is not a tree: 6417ac1dece94783e80dfbac0148604685d27579
#7 ERROR: executor failed running [/bin/sh -c git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579]: exit code: 128
------
 > [ 4/30] RUN git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579:
------
executor failed running [/bin/sh -c git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579]: exit code: 128

I think the kaldi project was updated and the git hash does not exists anymore

=> I check the rest of the Dockerfile file and i see that a clone of an inria repository is needed... but this repository seems to be not accessible ?

Someone can help me because i really want to help and have the voice fingerprinting feature with spk model that i have already experiment on python distro ?
@simonfn
Copy link

simonfn commented Feb 21, 2023

Hello, I got the same issue. I tried to checkout to another commit by replacing "6417ac1dece94783e80dfbac0148604685d27579" to "9af2c5c16389e141f527ebde7ee432a0c1df9fb9", which is the commit done around the same date as the commit from "vosk/". But it fails to apply the "kaldi.patch".

#10 [ 6/30] RUN git apply -v /tmp/kaldi.patch
#10 1.862 /tmp/kaldi.patch:230: trailing whitespace.
#10 1.862
#10 1.862 /tmp/kaldi.patch:232: trailing whitespace.
#10 1.862
#10 1.862 /tmp/kaldi.patch:269: trailing whitespace.
#10 1.862
#10 1.862 Checking patch src/Makefile...
#10 1.862 error: while searching for:
#10 1.862
#10 1.862 # Reset the default goal, so that the all target will become default
#10 1.862 .DEFAULT_GOAL :=
#10 1.862 all: $(SUBDIRS) matrix/test
#10 1.862       -echo Done
#10 1.862
#10 1.862 mklibdir:
#10 1.862
#10 1.862 error: patch failed: src/Makefile:41
#10 1.862 error: src/Makefile: patch does not apply
#10 1.862 Checking patch src/configure...
#10 1.863 Hunk #1 succeeded at 1026 (offset 4 lines).
#10 1.863 Checking patch src/ivector/ivector-extractor.cc...
#10 1.863 Checking patch src/makefiles/default_rules.mk...
#10 1.863 Checking patch src/makefiles/wasm_clapack.mk...
#10 1.863 Checking patch tools/Makefile...
#10 1.864 error: while searching for:
#10 1.864   # This new OS path is confirmed working on Windows 10 / Cygwin64.
#10 1.864   openfst_add_CXXFLAGS = -g -O2 -Wa,-mbig-obj
#10 1.864 else
#10 1.864   openfst_add_CXXFLAGS = -g -O3 -msse -msse2
#10 1.864 endif
#10 1.864
#10 1.864 openfst-$(OPENFST_VERSION)/Makefile: openfst-$(OPENFST_VERSION)
#10 1.864       cd openfst-$(OPENFST_VERSION)/ && \
#10 1.864         autoreconf -i && \
#10 1.864       ./configure --prefix=`pwd` $(OPENFST_CONFIGURE) CXX="$(CXX)" \
#10 1.864               CXXFLAGS="$(openfst_add_CXXFLAGS) $(CXXFLAGS)" \
#10 1.864
#10 1.864 error: patch failed: tools/Makefile:78
#10 1.864 error: tools/Makefile: patch does not apply
#10 1.864 Checking patch tools/extras/check_dependencies.sh...
#10 1.864 Checking patch tools/openfst.patch...
#10 1.864 /tmp/kaldi.patch:269: new blank line at EOF.
#10 1.864 +
#10 ERROR: executor failed running [/bin/sh -c git apply -v /tmp/kaldi.patch]: exit code: 1
------
 > [ 6/30] RUN git apply -v /tmp/kaldi.patch:
#10 1.864         autoreconf -i && \
#10 1.864       ./configure --prefix=`pwd` $(OPENFST_CONFIGURE) CXX="$(CXX)" \
#10 1.864               CXXFLAGS="$(openfst_add_CXXFLAGS) $(CXXFLAGS)" \
#10 1.864
#10 1.864 error: patch failed: tools/Makefile:78
#10 1.864 error: tools/Makefile: patch does not apply
#10 1.864 Checking patch tools/extras/check_dependencies.sh...
#10 1.864 Checking patch tools/openfst.patch...
#10 1.864 /tmp/kaldi.patch:269: new blank line at EOF.
#10 1.864 +
------
ERROR: failed to solve: executor failed running [/bin/sh -c git apply -v /tmp/kaldi.patch]: exit code: 1
make: *** [Makefile:12: builder] Error 1

Any idea on which commit is the right one ?

@olivierlaporte
Copy link
Author

simonfn I have done the same workaround and I have the same error.

I think repositories have changed since last summer and if you have not the builder installed, you cannot release.

@ccoreilly
Copy link
Owner

I guess I'll have to switch to forks of the repositories which I still have around.

In the meantime you should be able to pull the builder image:

docker pull ghcr.io/ccoreilly/vosk-wasm-builder:0.0.6

@simonfn
Copy link

simonfn commented Feb 22, 2023

I tried to build the binary from the v.0.0.8 with the docker image from the v.0.0.6, but it fails with :

94 warnings and 1 error generated.
em++: error: '/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=29 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -std=c++17 -Wno-unused-function -I/opt/include -I/opt/kaldi/src -I/opt/kaldi/tools/openfst/include -I/io/src -O3 /io/src/../vosk/src/recognizer.cc -c -o /tmp/emscripten_temp_eh0ac_s_/recognizer_2.o' failed (returned 1)
make: *** [Makefile:90: /io/src/../build/release/vosk.bc] Error 1
make: Leaving directory '/io/src'
make: *** [Makefile:19: binary] Error 2

The v.0.0.7 and v.0.0.6 seems to work with the same build environnement with some warnings when compiling .

@olivierlaporte
Copy link
Author

I will try on my side and give feedbacks. Thanks a lot @ccoreilly.

@ccoreilly ccoreilly mentioned this issue Feb 23, 2023
Open
@ccoreilly
Copy link
Owner

@olivierlaporte @simonfn could you test the branch in #66 ?

@simonfn
Copy link

simonfn commented Feb 23, 2023

@ccoreilly #66 worked for me, although it took quite some times to build the image

@olivierlaporte
Copy link
Author

olivierlaporte commented Feb 23, 2023 •
edited
Loading

It's also ok on my side. @ccoreilly Thanks you for your quick response and your very quick patch !

And like @simonfn building the image is quite long but it works.

git checkout -b make-builder-reproducible origin/make-builder-reproducible
time make builder

Step 31/32 : RUN rm -rf /tmp/libarchive
 ---> Running in 76862ebc8b05
Removing intermediate container 76862ebc8b05
 ---> ebe4d5360042
Step 32/32 : WORKDIR /opt
 ---> Running in 06b31cbed381
Removing intermediate container 06b31cbed381
 ---> f4353dc52c41
Successfully built f4353dc52c41
Successfully tagged ghcr.io/ccoreilly/vosk-wasm-builder:0.0.7
Successfully tagged ghcr.io/ccoreilly/vosk-wasm-builder:latest

time make binary

/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:979:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
59 warnings generated.
em++: warning: generating an executable with an object extension (.bc).  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
mkdir -p /io/src/../build/release/
cd /io/src/../build/release/; \
/emsdk/upstream/emscripten/em++ -o vosk.js /io/src/../build/release/vosk.bc --pre-js /io/src/vosk.pre.js --post-js /io/src/vosk.post.js --bind -std=c++17 -s ALLOW_MEMORY_GROWTH=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s MODULARIZE=1 -s EXPORT_NAME=LoadVosk -s ENVIRONMENT=worker -s EXPORTED_RUNTIME_METHODS=['FS','IDBFS'] -lidbfs.js -O3 -s SINGLE_FILE=1  /opt/kaldi/src/base/kaldi-base.bc /opt/kaldi/src/cudamatrix/kaldi-cudamatrix.bc /opt/kaldi/src/decoder/kaldi-decoder.bc /opt/kaldi/src/feat/kaldi-feat.bc /opt/kaldi/src/fstext/kaldi-fstext.bc /opt/kaldi/src/gmm/kaldi-gmm.bc /opt/kaldi/src/hmm/kaldi-hmm.bc /opt/kaldi/src/ivector/kaldi-ivector.bc /opt/kaldi/src/lat/kaldi-lat.bc /opt/kaldi/src/lm/kaldi-lm.bc /opt/kaldi/src/matrix/kaldi-matrix.bc /opt/kaldi/src/nnet3/kaldi-nnet3.bc /opt/kaldi/src/online2/kaldi-online2.bc /opt/kaldi/src/transform/kaldi-transform.bc /opt/kaldi/src/tree/kaldi-tree.bc /opt/kaldi/src/util/kaldi-util.bc /opt/kaldi/tools/clapack-wasm/CLAPACK-3.2.1/lapack.a /opt/kaldi/tools/clapack-wasm/CLAPACK-3.2.1/libcblaswr.a /opt/kaldi/tools/clapack-wasm/CBLAS/lib/cblas.a /opt/kaldi/tools/clapack-wasm/f2c_BLAS-3.8.0/blas.a /opt/kaldi/tools/openfst/lib/libfst.a /opt/kaldi/tools/openfst/lib/libfstngram.a /opt/lib/libarchive.so
warning: undefined symbol: _ZN5kaldi5rnnlm21RnnlmComputeStateInfoC1ERKNS0_35RnnlmComputeStateComputationOptionsERKNS_5nnet34NnetERKNS_8CuMatrixIfEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm22ReadSparseWordFeaturesERNSt3__213basic_istreamIcNS1_11char_traitsIcEEEEiPNS_12SparseMatrixIfEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm26KaldiRnnlmDeterministicFst5ClearEv (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm26KaldiRnnlmDeterministicFstC1EiRKNS0_21RnnlmComputeStateInfoE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: do_fio (referenced by top-level compiled C/C++ code)
warning: undefined symbol: e_wsfe (referenced by top-level compiled C/C++ code)
warning: undefined symbol: i_nint (referenced by top-level compiled C/C++ code)
warning: undefined symbol: popen (referenced by top-level compiled C/C++ code)
warning: undefined symbol: pow_dd (referenced by top-level compiled C/C++ code)
warning: undefined symbol: pow_ri (referenced by top-level compiled C/C++ code)
warning: undefined symbol: r_sign (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_cat (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_cmp (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_copy (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_wsfe (referenced by top-level compiled C/C++ code)
rsync -r /io/src/../build/release/vosk.js /io/src/../lib/src/vosk-wasm.js
rsync -r vosk.d.ts /io/src/../lib/src/vosk-wasm.d.ts
make: Leaving directory '/io/src'

@arbdevml
Copy link

Has anyone successfully implemented this functionality,
if yes, could someone share the experience, please?
Thank you for your answers.

@nshmyrev
Copy link

Has anyone successfully implemented this functionality, if yes, could someone share the experience, please? Thank you for your answers.

It works just fine. You'd better describe what is your trouble in detail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@olivierlaporte @nshmyrev @ccoreilly @arbdevml @simonfn