Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

加载中文维基百科数据要多久 #55

Open
xwt0016 opened this issue Jan 30, 2024 · 0 comments
Open

加载中文维基百科数据要多久 #55

xwt0016 opened this issue Jan 30, 2024 · 0 comments

Comments

@xwt0016
Copy link

xwt0016 commented Jan 30, 2024

FAISS.from_documents加载wiki.zh-sim.txt 45w数据，2个小时还没加载完，这正常吗，大概要多长时间？
听说faiss向量检索是基于内存的，langchain调faiss只支持indexflatip或者eindexflatl2作为索引方式。45w条数据检索时大约需要多大内存呢？
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@xwt0016