Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: CI issues #2388

Closed
1 task done
JinHai-CN opened this issue Dec 17, 2024 · 8 comments
Closed
1 task done

[Bug]: CI issues #2388

JinHai-CN opened this issue Dec 17, 2024 · 8 comments
Assignees
@small-turtle-1
Labels
bug Something isn't working

Comments

@JinHai-CN
Copy link
Contributor

JinHai-CN commented Dec 17, 2024 •
edited
Loading

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Actual behavior and How to reproduce it

Issue 1:

https://github.com/infiniflow/infinity/actions/runs/12369825011/job/34522532395

libc++abi: terminating due to uncaught exception of type infinity::UnrecoverableException@infinity_exception: Unable to download object: infinity/DELTA.24852, reason: NoSuchKey: Object does not exist@src/storage/io/s3_client_minio.cpp:39

Issue 2:

https://github.com/infiniflow/infinity/actions/runs/12369825011/job/34522532051

test_insert_checkpoint

Issue 3: Fixed by #2438

https://github.com/infiniflow/infinity/actions/runs/12373037137/job/34532562827

test_compact_restart_repeatedly

Issue 4: #2389
https://github.com/infiniflow/infinity/actions/runs/12372434436/job/34530626398?pr=2386

E infinity.common.InfinityException: (3037, 'Import file format error: No value in column num index 3 in CSV of row number: 2932, current row has column count: 3@src/executor/operator/physical_import.cpp:830@src/scheduler/fragment_context.cpp:1434')

Issue 5: fixed by #2428
https://github.com/infiniflow/infinity/actions/runs/12477264989/job/34823214511

[13:35:45.899] [142855] [critical] Error: last_full_ckp_ts 51472 >= max_commit_ts 57366 or last_ckp_ts 57368 > max_commit_ts 57366@src/storage/wal/wal_manager.cpp:512

Issue 6:
https://github.com/infiniflow/infinity/actions/runs/12512575496/job/34906086369


[14:49:45.340] [155072] [critical] Error: merging text_index ft_0000000000000000_6000 ft_000000000000a000... chunk_index_entry ft_000000000000a000 base_rowid expects to be 0000000000006000@src/storage/meta/entry/table_entry.cpp:952
[14:49:45.340] [155072] [critical] 0, cmake-build-debug/src/infinity(___interceptor_backtrace+0x46) [0x55aef2468776]
[14:49:45.340] [155072] [critical] 1, cmake-build-debug/src/infinity(+0x16f8d49) [0x55aef2f40d49]
[14:49:45.340] [155072] [critical] 2, cmake-build-debug/src/infinity(+0x16fa055) [0x55aef2f42055]
[14:49:45.340] [155072] [critical] 3, cmake-build-debug/src/infinity(+0x25a9e46) [0x55aef3df1e46]
[14:49:45.340] [155072] [critical] 4, cmake-build-debug/src/infinity(+0x2152ec1) [0x55aef399aec1]
[14:49:45.340] [155072] [critical] 5, cmake-build-debug/src/infinity(+0x21560c6) [0x55aef399e0c6]
[14:49:45.340] [155072] [critical] 6, cmake-build-debug/src/infinity(+0x21575fa) [0x55aef399f5fa]
[14:49:45.340] [155072] [critical] 7, cmake-build-debug/src/infinity(+0x2157595) [0x55aef399f595]
[14:49:45.340] [155072] [critical] 8, cmake-build-debug/src/infinity(+0x215756d) [0x55aef399f56d]
[14:49:45.340] [155072] [critical] 9, cmake-build-debug/src/infinity(+0x2157236) [0x55aef399f236]
[14:49:45.340] [155072] [critical] 10, cmake-build-debug/src/infinity(+0xc72b29) [0x55aef24bab29]
[14:49:45.340] [155072] [critical] 11, /lib64/libpthread.so.0(+0x7ea5) [0x7f8a8a004ea5]
[14:49:45.340] [155072] [critical] 12, /lib64/libc.so.6(clone+0x6d) [0x7f8a89609b0d]
[14:49:45.341] [155072] [critical] Error: TerminateHandler: Unhandled Exception: merging text_index ft_0000000000000000_6000 ft_000000000000a000... chunk_index_entry ft_000000000000a000 base_rowid expects to be 0000000000006000@src/storage/meta/entry/table_entry.cpp:952
[14:49:45.341] [155072] [critical] 0, cmake-build-debug/src/infinity(___interceptor_backtrace+0x46) [0x55aef2468776]
[14:49:45.341] [155072] [critical] 1, cmake-build-debug/src/infinity(+0x16f8d49) [0x55aef2f40d49]
[14:49:45.341] [155072] [critical] 2, cmake-build-debug/src/infinity(+0xcb69a1) [0x55aef24fe9a1]
[14:49:45.341] [155072] [critical] 3, cmake-build-debug/src/infinity(+0x66d4b03) [0x55aef7f1cb03]
[14:49:45.341] [155072] [critical] 4, cmake-build-debug/src/infinity(+0x66d6de6) [0x55aef7f1ede6]
[14:49:45.341] [155072] [critical] 5, cmake-build-debug/src/infinity(+0x66d6dd0) [0x55aef7f1edd0]
[14:49:45.341] [155072] [critical] 6, cmake-build-debug/src/infinity(+0x16fa13d) [0x55aef2f4213d]
[14:49:45.341] [155072] [critical] 7, cmake-build-debug/src/infinity(+0x25a9e46) [0x55aef3df1e46]
[14:49:45.341] [155072] [critical] 8, cmake-build-debug/src/infinity(+0x2152ec1) [0x55aef399aec1]
[14:49:45.341] [155072] [critical] 9, cmake-build-debug/src/infinity(+0x21560c6) [0x55aef399e0c6]
[14:49:45.341] [155072] [critical] 10, cmake-build-debug/src/infinity(+0x21575fa) [0x55aef399f5fa]
[14:49:45.341] [155072] [critical] 11, cmake-build-debug/src/infinity(+0x2157595) [0x55aef399f595]
[14:49:45.341] [155072] [critical] 12, cmake-build-debug/src/infinity(+0x215756d) [0x55aef399f56d]
[14:49:45.341] [155072] [critical] 13, cmake-build-debug/src/infinity(+0x2157236) [0x55aef399f236]
[14:49:45.341] [155072] [critical] 14, cmake-build-debug/src/infinity(+0xc72b29) [0x55aef24bab29]
[14:49:45.341] [155072] [critical] 15, /lib64/libpthread.so.0(+0x7ea5) [0x7f8a8a004ea5]
[14:49:45.341] [155072] [critical] 16, /lib64/libc.so.6(clone+0x6d) [0x7f8a89609b0d]
@JinHai-CN JinHai-CN added the bug Something isn't working label Dec 17, 2024
@yangzq50
Copy link
Contributor

yangzq50 commented Dec 27, 2024 •
edited
Loading

Fixed in #2416

https://github.com/infiniflow/infinity/actions/runs/12503536686/job/34884005644

Runner name: 'inf118-94a10a20f1ab'
Random log file name: 869yz0pz

[20:24:46.275] [86192] [critical] Error: Invalid MergeFlag: 1@src/storage/wal/catalog_delta_entry.cpp:261
[20:24:46.275] [86192] [critical] 0, cmake-build-debug/src/infinity(___interceptor_backtrace+0x46) [0x55643cd5f5c6]
[20:24:46.275] [86192] [critical] 1, cmake-build-debug/src/infinity(+0x16f2b79) [0x55643d837b79]
[20:24:46.275] [101741] [trace] Task: 0 of Fragment: 0 is completed
[20:24:46.275] [86192] [critical] 2, cmake-build-debug/src/infinity(+0x16f3e85) [0x55643d838e85]
[20:24:46.275] [101741] [trace] All tasks in fragment: 0 are completed
[20:24:46.275] [86192] [critical] 3, cmake-build-debug/src/infinity(+0x1117f26) [0x55643d25cf26]
[20:24:46.275] [86192] [critical] 4, cmake-build-debug/src/infinity(+0x112e922) [0x55643d273922]
[20:24:46.275] [86192] [critical] 5, cmake-build-debug/src/infinity(+0x112d924) [0x55643d272924]
[20:24:46.275] [86192] [critical] 6, cmake-build-debug/src/infinity(+0x23561e9) [0x55643e49b1e9]
[20:24:46.275] [86192] [critical] 7, cmake-build-debug/src/infinity(+0x1d5638b) [0x55643de9b38b]
[20:24:46.275] [86192] [critical] 8, cmake-build-debug/src/infinity(+0x1d57e1a) [0x55643de9ce1a]
[20:24:46.275] [86192] [critical] 9, cmake-build-debug/src/infinity(+0x1d57db5) [0x55643de9cdb5]
[20:24:46.275] [86192] [critical] 10, cmake-build-debug/src/infinity(+0x1d57d8d) [0x55643de9cd8d]
[20:24:46.275] [86192] [critical] 11, cmake-build-debug/src/infinity(+0x1d57a56) [0x55643de9ca56]
[20:24:46.275] [86192] [critical] 12, cmake-build-debug/src/infinity(+0xc6c979) [0x55643cdb1979]
[20:24:46.275] [86192] [critical] 13, /lib64/libpthread.so.0(+0x7ea5) [0x7f3b68159ea5]
[20:24:46.275] [86192] [critical] 14, /lib64/libc.so.6(clone+0x6d) [0x7f3b6775eb0d]

@yangzq50 yangzq50 mentioned this issue Dec 30, 2024
Merged
2 tasks
@yangzq50
Copy link
Contributor

yangzq50 commented Dec 30, 2024 •
edited by JinHai-CN
Loading

Both are fixed by #2431

Error when running python/parallel_test/test_insert_delete_parallel.py

[16:38:04.289] [279304] [critical] Error: DumpByline: Cannot find segment index entry with id: 0@src/storage/compaction_process.cpp:221

$2 = 0x1701109 <infinity::PrintStacktrace@infinity_exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)+601 at /home/yzq/GitHub/infinity_update_py/src/common/utility/exception.cpp:34>
$3 = 0x1702415 <infinity::UnrecoverableError@infinity_exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*, unsigned int)+1093 at /home/yzq/GitHub/infinity_update_py/src/common/utility/exception.cpp:71>
$4 = 0x215e867 <infinity::CompactionProcessor@compaction_process::DoDumpByline(infinity::DumpIndexBylineTask@bg_task*)+4007 at /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:221>
$5 = 0x216038d <infinity::CompactionProcessor@compaction_process::Process()+5165 at /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:297>
$6 = 0x216104a <infinity::CompactionProcessor@compaction_process::Start()::$_0::operator()() const+58 at /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:65>
$7 = 0x2160fe5 <std::__1::__invoke[abi:ne180100]<infinity::CompactionProcessor@compaction_process::Start()::$_0>(infinity::CompactionProcessor@compaction_process::Start()::$_0&&)+21 at /usr/lib/llvm-18/include/c++/v1/__type_traits/invoke.h:344>
$8 = 0x2160fbd <_ZNSt3__116__thread_executeB8ne180100INS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEZN8infinityW18compaction_process19CompactionProcessor5StartEvE3$_0JETpTnmJEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE+29 at /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:194>
$9 = 0x2160c86 <std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, infinity::CompactionProcessor@compaction_process::Start()::$_0> >(void*)+278 at /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:202>
$10 = 0xc78bfd <asan_thread_start(void*)+109>

SUMMARY: AddressSanitizer: heap-use-after-free /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/segment_index_entry.cppm:106:57 in infinity::SegmentIndexEntry@segment_index_entry::table_index_entry() const

    #0 0x5862a5b12f41 in infinity::SegmentIndexEntry@segment_index_entry::table_index_entry() const /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/segment_index_entry.cppm:106:57
    #1 0x5862a5222221 in infinity::WalChunkIndexInfo@wal_entry::WalChunkIndexInfo(infinity::ChunkIndexEntry@chunk_index_entry*) /home/yzq/GitHub/infinity_update_py/src/storage/wal/wal_entry.cpp:211:49
    #2 0x5862a6566771 in infinity::WalChunkIndexInfo@wal_entry* std::__1::construct_at[abi:ne180100]<infinity::WalChunkIndexInfo@wal_entry, infinity::ChunkIndexEntry@chunk_index_entry*&, infinity::WalChunkIndexInfo@wal_entry*>(infinity::WalChunkIndexInfo@wal_entry*, infinity::ChunkIndexEntry@chunk_index_entry*&) /usr/lib/llvm-18/include/c++/v1/__memory/construct_at.h:41:46
    #3 0x5862a656670c in infinity::WalChunkIndexInfo@wal_entry* std::__1::__construct_at[abi:ne180100]<infinity::WalChunkIndexInfo@wal_entry, infinity::ChunkIndexEntry@chunk_index_entry*&, infinity::WalChunkIndexInfo@wal_entry*>(infinity::WalChunkIndexInfo@wal_entry*, infinity::ChunkIndexEntry@chunk_index_entry*&) /usr/lib/llvm-18/include/c++/v1/__memory/construct_at.h:49:10
    #4 0x5862a65666e0 in void std::__1::allocator_traits<std::__1::allocator<infinity::WalChunkIndexInfo@wal_entry>>::construct[abi:ne180100]<infinity::WalChunkIndexInfo@wal_entry, infinity::ChunkIndexEntry@chunk_index_entry*&, void, void>(std::__1::allocator<infinity::WalChunkIndexInfo@wal_entry>&, infinity::WalChunkIndexInfo@wal_entry*, infinity::ChunkIndexEntry@chunk_index_entry*&) /usr/lib/llvm-18/include/c++/v1/__memory/allocator_traits.h:305:5
    #5 0x5862a6566548 in infinity::WalChunkIndexInfo@wal_entry* std::__1::vector<infinity::WalChunkIndexInfo@wal_entry, std::__1::allocator<infinity::WalChunkIndexInfo@wal_entry>>::__emplace_back_slow_path<infinity::ChunkIndexEntry@chunk_index_entry*&>(infinity::ChunkIndexEntry@chunk_index_entry*&) /usr/lib/llvm-18/include/c++/v1/vector:1491:3
    #6 0x5862a65624e1 in infinity::WalChunkIndexInfo@wal_entry& std::__1::vector<infinity::WalChunkIndexInfo@wal_entry, std::__1::allocator<infinity::WalChunkIndexInfo@wal_entry>>::emplace_back<infinity::ChunkIndexEntry@chunk_index_entry*&>(infinity::ChunkIndexEntry@chunk_index_entry*&) /usr/lib/llvm-18/include/c++/v1/vector:1511:13
    #7 0x5862a6536be8 in infinity::SegmentIndexEntry@segment_index_entry::AddWalIndexDump(infinity::ChunkIndexEntry@chunk_index_entry*, infinity::Txn@txn*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>>) /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/segment_index_entry.cpp:402:17
    #8 0x5862a61ff118 in infinity::CompactionProcessor@compaction_process::DoDumpByline(infinity::DumpIndexBylineTask@bg_task*) /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:223:30
    #9 0x5862a6200b4c in infinity::CompactionProcessor@compaction_process::Process() /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:297:25
    #10 0x5862a6201809 in infinity::CompactionProcessor@compaction_process::Start()::$_0::operator()() const /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:65:41
    #11 0x5862a62017a4 in decltype(std::declval<infinity::CompactionProcessor@compaction_process::Start()::$_0>()()) std::__1::__invoke[abi:ne180100]<infinity::CompactionProcessor@compaction_process::Start()::$_0>(infinity::CompactionProcessor@compaction_process::Start()::$_0&&) /usr/lib/llvm-18/include/c++/v1/__type_traits/invoke.h:344:25
    #12 0x5862a620177c in void std::__1::__thread_execute[abi:ne180100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::CompactionProcessor@compaction_process::Start()::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::CompactionProcessor@compaction_process::Start()::$_0>&, std::__1::__tuple_indices<...>) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:193:3
    #13 0x5862a6201445 in void* std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::CompactionProcessor@compaction_process::Start()::$_0>>(void*) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:202:3
    #14 0x5862a4d188ac in asan_thread_start(void*) asan_interceptors.cpp.o
    #15 0x716faf09ca93 in start_thread nptl/pthread_create.c:447:8
    #16 0x716faf129c3b in clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

0x515000832a40 is located 64 bytes inside of 464-byte region [0x515000832a00,0x515000832bd0)
freed by thread T333 here:
    #0 0x5862a4d59c61 in operator delete(void*) (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xcb8c61) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)
    #1 0x5862a6633a33 in std::__1::default_delete<infinity::SegmentIndexEntry@segment_index_entry>::operator()[abi:ne180100](infinity::SegmentIndexEntry@segment_index_entry*) const /usr/lib/llvm-18/include/c++/v1/__memory/unique_ptr.h:66:5
    #2 0x5862a663404b in std::__1::__shared_ptr_pointer<infinity::SegmentIndexEntry@segment_index_entry*, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>::__shared_ptr_default_delete<infinity::SegmentIndexEntry@segment_index_entry, infinity::SegmentIndexEntry@segment_index_entry>, std::__1::allocator<infinity::SegmentIndexEntry@segment_index_entry>>::__on_zero_shared() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:228:3
    #3 0x5862a4d6c357 in std::__1::__shared_count::__release_shared[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:157:7
    #4 0x5862a4d6c2b8 in std::__1::__shared_weak_count::__release_shared[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:186:25
    #5 0x5862a52db9f5 in std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>::~shared_ptr[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:648:17
    #6 0x5862a5306668 in std::__1::pair<unsigned int const, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>::~pair() /usr/lib/llvm-18/include/c++/v1/__utility/pair.h:80:29
    #7 0x5862a5306644 in void std::__1::__destroy_at[abi:ne180100]<std::__1::pair<unsigned int const, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, 0>(std::__1::pair<unsigned int const, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>*) /usr/lib/llvm-18/include/c++/v1/__memory/construct_at.h:67:11
    #8 0x5862a53065f8 in void std::__1::allocator_traits<std::__1::allocator<std::__1::__tree_node<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, void*>>>::destroy[abi:ne180100]<std::__1::pair<unsigned int const, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, void, void>(std::__1::allocator<std::__1::__tree_node<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, void*>>&, std::__1::pair<unsigned int const, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>*) /usr/lib/llvm-18/include/c++/v1/__memory/allocator_traits.h:316:5
    #9 0x5862a62e56c2 in std::__1::__tree<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, std::__1::__map_value_compare<unsigned int, std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, std::__1::less<unsigned int>, true>, std::__1::allocator<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>>>::destroy(std::__1::__tree_node<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, void*>*) /usr/lib/llvm-18/include/c++/v1/__tree:1548:5
    #10 0x5862a62fad44 in std::__1::__tree<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, std::__1::__map_value_compare<unsigned int, std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>, std::__1::less<unsigned int>, true>, std::__1::allocator<std::__1::__value_type<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>>>::~__tree() /usr/lib/llvm-18/include/c++/v1/__tree:1539:3
    #11 0x5862a62d4534 in std::__1::map<unsigned int, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>, std::__1::less<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>>>>::~map[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/map:1135:112
    #12 0x5862a66c85bf in infinity::TableIndexEntry@table_index_entry::~TableIndexEntry() /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/table_index_entry.cppm:54:15
    #13 0x5862a66d5574 in void std::__1::__destroy_at[abi:ne180100]<infinity::TableIndexEntry@table_index_entry, 0>(infinity::TableIndexEntry@table_index_entry*) /usr/lib/llvm-18/include/c++/v1/__memory/construct_at.h:67:11
    #14 0x5862a66d5558 in void std::__1::allocator_traits<std::__1::allocator<infinity::TableIndexEntry@table_index_entry>>::destroy[abi:ne180100]<infinity::TableIndexEntry@table_index_entry, void, void>(std::__1::allocator<infinity::TableIndexEntry@table_index_entry>&, infinity::TableIndexEntry@table_index_entry*) /usr/lib/llvm-18/include/c++/v1/__memory/allocator_traits.h:316:5
    #15 0x5862a66d54d5 in void std::__1::__shared_ptr_emplace<infinity::TableIndexEntry@table_index_entry, std::__1::allocator<infinity::TableIndexEntry@table_index_entry>>::__on_zero_shared_impl[abi:ne180100]<std::__1::allocator<infinity::TableIndexEntry@table_index_entry>, 0>() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:284:5
    #16 0x5862a66d4ea4 in std::__1::__shared_ptr_emplace<infinity::TableIndexEntry@table_index_entry, std::__1::allocator<infinity::TableIndexEntry@table_index_entry>>::__on_zero_shared() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:287:78
    #17 0x5862a4d6c357 in std::__1::__shared_count::__release_shared[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:157:7
    #18 0x5862a4d6c2b8 in std::__1::__shared_weak_count::__release_shared[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:186:25
    #19 0x5862a50d7075 in std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>::~shared_ptr[abi:ne180100]() /usr/lib/llvm-18/include/c++/v1/__memory/shared_ptr.h:648:17
    #20 0x5862a5b1cad4 in void std::__1::__destroy_at[abi:ne180100]<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, 0>(std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>*) /usr/lib/llvm-18/include/c++/v1/__memory/construct_at.h:67:11
    #21 0x5862a5b1ca98 in void std::__1::allocator_traits<std::__1::allocator<std::__1::__list_node<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, void*>>>::destroy[abi:ne180100]<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, void, void>(std::__1::allocator<std::__1::__list_node<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, void*>>&, std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>*) /usr/lib/llvm-18/include/c++/v1/__memory/allocator_traits.h:316:5
    #22 0x5862a5b1ca19 in void std::__1::__list_imp<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, std::__1::allocator<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>>>::__delete_node[abi:ne180100]<>(std::__1::__list_node<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, void*>*) /usr/lib/llvm-18/include/c++/v1/list:576:5
    #23 0x5862a5b1c843 in std::__1::__list_imp<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, std::__1::allocator<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>>>::clear() /usr/lib/llvm-18/include/c++/v1/list:635:7
    #24 0x5862a5b1b664 in std::__1::__list_imp<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, std::__1::allocator<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>>>::~__list_imp() /usr/lib/llvm-18/include/c++/v1/list:622:3
    #25 0x5862a66ee634 in std::__1::list<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>, std::__1::allocator<std::__1::shared_ptr<infinity::TableIndexEntry@table_index_entry>>>::~list() /usr/lib/llvm-18/include/c++/v1/list:667:28
    #26 0x5862a66ebd2f in infinity::EntryList@entry_list<infinity::TableIndexEntry@table_index_entry>::~EntryList() /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/entry_list.cppm:48:7
    #27 0x5862a66f18c8 in infinity::TableIndexMeta@table_index_meta::~TableIndexMeta() /home/yzq/GitHub/infinity_update_py/src/storage/meta/table_index_meta.cppm:37:14
    #28 0x5862a66f1908 in infinity::TableIndexMeta@table_index_meta::~TableIndexMeta() /home/yzq/GitHub/infinity_update_py/src/storage/meta/table_index_meta.cppm:37:14
    #29 0x5862a50e6602 in std::__1::default_delete<infinity::TableIndexMeta@table_index_meta>::operator()[abi:ne180100](infinity::TableIndexMeta@table_index_meta*) const /usr/lib/llvm-18/include/c++/v1/__memory/unique_ptr.h:66:5

previously allocated by thread T332 here:
    #0 0x5862a4d593e1 in operator new(unsigned long) (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xcb83e1) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)
    #1 0x5862a653142f in infinity::SegmentIndexEntry@segment_index_entry::NewIndexEntry(infinity::TableIndexEntry@table_index_entry*, unsigned int, infinity::Txn@txn*) /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/segment_index_entry.cpp:138:61
    #2 0x5862a66bcf74 in infinity::TableIndexEntry@table_index_entry::GetOrCreateSegment(unsigned int, infinity::Txn@txn*, std::__1::shared_ptr<infinity::SegmentIndexEntry@segment_index_entry>&) /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/table_index_entry.cpp:161:31
    #3 0x5862a664ea88 in infinity::TableEntry@table_entry::MemIndexInsertInner(infinity::TableIndexEntry@table_index_entry*, infinity::Txn@txn*, unsigned int, std::__1::vector<infinity::AppendRange@data_access_state, std::__1::allocator<infinity::AppendRange@data_access_state>>&) /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/table_entry.cpp:740:39
    #4 0x5862a6648d01 in infinity::TableEntry@table_entry::MemIndexInsert(infinity::Txn@txn*, std::__1::vector<infinity::AppendRange@data_access_state, std::__1::allocator<infinity::AppendRange@data_access_state>>&) /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/table_entry.cpp:720:21
    #5 0x5862a6648065 in infinity::TableEntry@table_entry::AppendData(unsigned long, void*, unsigned long, infinity::BufferManager@buffer_manager*, bool) /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/table_entry.cpp:499:9
    #6 0x5862a63f26a5 in infinity::Catalog@catalog::Append(infinity::TableEntry@table_entry*, unsigned long, void*, unsigned long, infinity::BufferManager@buffer_manager*, bool) /home/yzq/GitHub/infinity_update_py/src/storage/meta/catalog.cpp:401:25
    #7 0x5862a5134255 in infinity::TxnTableStore@txn_store::PrepareCommit(unsigned long, unsigned long, infinity::BufferManager@buffer_manager*) /home/yzq/GitHub/infinity_update_py/src/storage/txn/txn_store.cpp:441:5
    #8 0x5862a513ad66 in infinity::TxnStore@txn_store::PrepareCommit(unsigned long, unsigned long, infinity::BufferManager@buffer_manager*) /home/yzq/GitHub/infinity_update_py/src/storage/txn/txn_store.cpp:703:22
    #9 0x5862a50b9aec in infinity::Txn@txn::CommitBottom() /home/yzq/GitHub/infinity_update_py/src/storage/txn/txn.cpp:602:16
    #10 0x5862a528d59b in infinity::WalManager@wal_manager::Flush() /home/yzq/GitHub/infinity_update_py/src/storage/wal/wal_manager.cpp:373:22
    #11 0x5862a52c3149 in infinity::WalManager@wal_manager::Start()::$_0::operator()() const /home/yzq/GitHub/infinity_update_py/src/storage/wal/wal_manager.cpp:121:37
    #12 0x5862a52c30e4 in decltype(std::declval<infinity::WalManager@wal_manager::Start()::$_0>()()) std::__1::__invoke[abi:ne180100]<infinity::WalManager@wal_manager::Start()::$_0>(infinity::WalManager@wal_manager::Start()::$_0&&) /usr/lib/llvm-18/include/c++/v1/__type_traits/invoke.h:344:25
    #13 0x5862a52c30bc in void std::__1::__thread_execute[abi:ne180100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::WalManager@wal_manager::Start()::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::WalManager@wal_manager::Start()::$_0>&, std::__1::__tuple_indices<...>) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:193:3
    #14 0x5862a52c2d85 in void* std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::WalManager@wal_manager::Start()::$_0>>(void*) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:202:3
    #15 0x5862a4d188ac in asan_thread_start(void*) asan_interceptors.cpp.o

Thread T334 created by T0 here:
    #0 0x5862a4d00735 in pthread_create (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xc5f735) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)
    #1 0x5862a4eafec8 in std::__1::__libcpp_thread_create[abi:ne180100](unsigned long*, void* (*)(void*), void*) /usr/lib/llvm-18/include/c++/v1/__threading_support:317:10
    #2 0x5862a61f9b67 in std::__1::thread::thread<infinity::CompactionProcessor@compaction_process::Start()::$_0, void>(infinity::CompactionProcessor@compaction_process::Start()::$_0&&) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:212:14
    #3 0x5862a61f987e in infinity::CompactionProcessor@compaction_process::Start() /home/yzq/GitHub/infinity_update_py/src/storage/compaction_process.cpp:65:25
    #4 0x5862a50959bb in infinity::Storage@storage::AdminToWriter() /home/yzq/GitHub/infinity_update_py/src/storage/storage.cpp:311:25
    #5 0x5862a509bde9 in infinity::Storage@storage::SetStorageMode(infinity::StorageMode@wal_manager) /home/yzq/GitHub/infinity_update_py/src/storage/storage.cpp:662:28
    #6 0x5862a4f81532 in infinity::InfinityContext@infinity_context::ChangeServerRole(infinity::NodeRole, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, unsigned short) /home/yzq/GitHub/infinity_update_py/src/main/infinity_context.cpp:147:59
    #7 0x5862a4f8710e in infinity::InfinityContext@infinity_context::InitPhase2() /home/yzq/GitHub/infinity_update_py/src/main/infinity_context.cpp:105:39
    #8 0x5862a4d5c9d6 in main /home/yzq/GitHub/infinity_update_py/src/bin/infinity_main.cpp:279:33
    #9 0x716faf02a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #10 0x716faf02a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #11 0x5862a4c7ff74 in _start (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xbdef74) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)

Thread T333 created by T0 here:
    #0 0x5862a4d00735 in pthread_create (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xc5f735) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)
    #1 0x5862a4eafec8 in std::__1::__libcpp_thread_create[abi:ne180100](unsigned long*, void* (*)(void*), void*) /usr/lib/llvm-18/include/c++/v1/__threading_support:317:10
    #2 0x5862a5e05577 in std::__1::thread::thread<infinity::BGTaskProcessor@background_process::Start()::$_0, void>(infinity::BGTaskProcessor@background_process::Start()::$_0&&) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:212:14
    #3 0x5862a5e0524b in infinity::BGTaskProcessor@background_process::Start() /home/yzq/GitHub/infinity_update_py/src/storage/background_process.cpp:39:25
    #4 0x5862a5095702 in infinity::Storage@storage::AdminToWriter() /home/yzq/GitHub/infinity_update_py/src/storage/storage.cpp:299:20
    #5 0x5862a509bde9 in infinity::Storage@storage::SetStorageMode(infinity::StorageMode@wal_manager) /home/yzq/GitHub/infinity_update_py/src/storage/storage.cpp:662:28
    #6 0x5862a4f81532 in infinity::InfinityContext@infinity_context::ChangeServerRole(infinity::NodeRole, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, unsigned short) /home/yzq/GitHub/infinity_update_py/src/main/infinity_context.cpp:147:59
    #7 0x5862a4f8710e in infinity::InfinityContext@infinity_context::InitPhase2() /home/yzq/GitHub/infinity_update_py/src/main/infinity_context.cpp:105:39
    #8 0x5862a4d5c9d6 in main /home/yzq/GitHub/infinity_update_py/src/bin/infinity_main.cpp:279:33
    #9 0x716faf02a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #10 0x716faf02a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #11 0x5862a4c7ff74 in _start (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xbdef74) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)

Thread T332 created by T0 here:
    #0 0x5862a4d00735 in pthread_create (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xc5f735) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)
    #1 0x5862a4eafec8 in std::__1::__libcpp_thread_create[abi:ne180100](unsigned long*, void* (*)(void*), void*) /usr/lib/llvm-18/include/c++/v1/__threading_support:317:10
    #2 0x5862a5287ad7 in std::__1::thread::thread<infinity::WalManager@wal_manager::Start()::$_0, void>(infinity::WalManager@wal_manager::Start()::$_0&&) /usr/lib/llvm-18/include/c++/v1/__thread/thread.h:212:14
    #3 0x5862a528775c in infinity::WalManager@wal_manager::Start() /home/yzq/GitHub/infinity_update_py/src/storage/wal/wal_manager.cpp:121:21
    #4 0x5862a50953bb in infinity::Storage@storage::AdminToWriter() /home/yzq/GitHub/infinity_update_py/src/storage/storage.cpp:287:15
    #5 0x5862a509bde9 in infinity::Storage@storage::SetStorageMode(infinity::StorageMode@wal_manager) /home/yzq/GitHub/infinity_update_py/src/storage/storage.cpp:662:28
    #6 0x5862a4f81532 in infinity::InfinityContext@infinity_context::ChangeServerRole(infinity::NodeRole, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, unsigned short) /home/yzq/GitHub/infinity_update_py/src/main/infinity_context.cpp:147:59
    #7 0x5862a4f8710e in infinity::InfinityContext@infinity_context::InitPhase2() /home/yzq/GitHub/infinity_update_py/src/main/infinity_context.cpp:105:39
    #8 0x5862a4d5c9d6 in main /home/yzq/GitHub/infinity_update_py/src/bin/infinity_main.cpp:279:33
    #9 0x716faf02a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #10 0x716faf02a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #11 0x5862a4c7ff74 in _start (/home/yzq/GitHub/infinity_update_py/cmake-build-debug/src/infinity+0xbdef74) (BuildId: 57afa0cdbf0388bdcf4a94636d2da82c3ad4df4b)

SUMMARY: AddressSanitizer: heap-use-after-free /home/yzq/GitHub/infinity_update_py/src/storage/meta/entry/segment_index_entry.cppm:106:57 in infinity::SegmentIndexEntry@segment_index_entry::table_index_entry() const
Shadow bytes around the buggy address:
  0x515000832780: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x515000832800: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x515000832880: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x515000832900: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa
  0x515000832980: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x515000832a00: fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd
  0x515000832a80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x515000832b00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x515000832b80: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa
  0x515000832c00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x515000832c80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==255487==ABORTING

@yangzq50
Copy link
Contributor

yangzq50 commented Dec 30, 2024 •
edited by JinHai-CN
Loading

Fixed by @small-turtle-1 in #2419

https://github.com/infiniflow/infinity/actions/runs/12543993246/job/34975912122

[17:09:03.376] [219594] [critical] Error: TerminateHandler: Unhandled Exception: Data read error: Read magic number which length isn't 0.@src/storage/buffer/file_worker/data_file_worker.cpp:157

$1 = 0x470e29 <infinity::PrintStacktrace@infinity_exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)+41 at /infinity/src/common/utility/exception.cpp:35>
$2 = 0x1ffbf0 <(anonymous namespace)::TerminateHandler()+224 at /infinity/src/bin/infinity_main.cpp:216>
$3 = 0x16df543 <std::__terminate(void (*)())+3>
$4 = 0x16e19a6
$5 = 0x16e1990 <__cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*)>
$6 = 0x4716d6 <infinity::RecoverableError@infinity_exception(infinity::Status@status, char const*, unsigned int)+822 at /usr/local/include/c++/v1/string:1868>
$7 = 0x11cc367 <infinity::DataFileWorker@data_file_worker::ReadFromMmapImpl(void const*, unsigned long)+279 at /usr/local/include/c++/v1/__memory/unique_ptr.h:263>
$8 = 0x5d3f73 <infinity::FileWorker@file_worker::Mmap()+179 at /infinity/src/storage/buffer/file_worker/file_worker.cpp:219>
$9 = 0x5cec79 <infinity::BufferObj@buffer_obj::Load()+89 at /infinity/src/storage/buffer/buffer_obj.cpp:80>
$10 = 0x6448e0 <infinity::VectorBuffer@vector_buffer::Initialize(infinity::BufferManager@buffer_manager*, infinity::BlockColumnEntry@block_column_entry*, unsigned long, unsigned long)+240 at /infinity/src/storage/column_vector/vector_buffer.cpp:138>
$11 = 0x644610 <infinity::VectorBuffer@vector_buffer::Make(infinity::BufferManager@buffer_manager*, infinity::BlockColumnEntry@block_column_entry*, unsigned long, unsigned long, infinity::VectorBufferType@vector_buffer)+144 at /infinity/src/storage/column_vector/vector_buffer.cpp:69>
$12 = 0x5d4b87 <infinity::ColumnVector@column_vector::Initialize(infinity::BufferManager@buffer_manager*, infinity::BlockColumnEntry@block_column_entry*, unsigned long, infinity::ColumnVectorTipe@column_vector, infinity::ColumnVectorType@column_vector, unsigned long)+407 at /usr/local/include/c++/v1/__memory/shared_ptr.h:576>
$13 = 0x6f8430 <infinity::BlockColumnEntry@block_column_entry::GetColumnVectorInner(infinity::BufferManager@buffer_manager*, infinity::ColumnVectorTipe@column_vector, unsigned long)+720 at /infinity/src/storage/meta/entry/block_column_entry.cpp:192>
$14 = 0x6f85e1 <infinity::BlockColumnEntry@block_column_entry::GetConstColumnVector(infinity::BufferManager@buffer_manager*, unsigned long)+17 at /infinity/src/storage/meta/entry/block_column_entry.cpp:171>
$15 = 0x5a24eb <infinity::BlockColumnIter@block_column_iter<true>::BlockColumnIter(infinity::BlockColumnEntry@block_column_entry*, infinity::BufferManager@buffer_manager*, unsigned long, unsigned long)+43 at /usr/local/include/c++/v1/new:271>
$16 = 0x7361e0 <std::__1::vector<infinity::BlockColumnIter@block_column_iter<true>, std::__1::allocator<infinity::BlockColumnIter@block_column_iter<true> > >::__emplace_back_slow_path<infinity::BlockColumnEntry@block_column_entry*, infinity::BufferManager@buffer_manager*&, unsigned long&, unsigned long&>(infinity::BlockColumnEntry@block_column_entry*&&, infinity::BufferManager@buffer_manager*&, unsigned long&, unsigned long&)+240 at /usr/local/include/c++/v1/vector:991>
$17 = 0x735f8a <infinity::BlockIter@block_iter<true>::BlockIter(infinity::BlockEntry@block_entry const*, infinity::BufferManager@buffer_manager*, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)+218 at /usr/local/include/c++/v1/vector:1511>
$18 = 0x737144 <infinity::SegmentIter@segment_iter<true>::Next()+260 at /usr/local/include/c++/v1/optional:359>
$19 = 0x73da57 <_ZN8infinityW10data_store9DataStoreINS_W14vec_store_type19PlainL2VecStoreTypeIfEEjE6AddVecITkNS_W11hnsw_common19DataIteratorConceptINT_12QueryVecTypeET0_EENS_W12segment_iter23CappedOneColumnIteratorIfLb1EEEEENSt3__14pairImmEEOS9_+311 at /usr/local/include/c++/v1/optional:359>
$20 = 0x728bee <std::__1::__variant_detail::__visitation::__base::__dispatcher<2ul>::__dispatch[abi:ne180100]<std::__1::__variant_detail::__visitation::__variant::__value_visitor<infinity::SegmentIndexEntry@segment_index_entry::RebuildChunkIndexEntries(infinity::TxnTableStore@txn_store*, infinity::SegmentEntry@segment_entry*)::$_1>&&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)0, infinity::KnnHnsw@hnsw_alg<infinity::PlainCosVecStoreType@vec_store_type<float>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainIPVecStoreType@vec_store_type<float>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainL2VecStoreType@vec_store_type<float>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainCosVecStoreType@vec_store_type<unsigned char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainIPVecStoreType@vec_store_type<unsigned char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainL2VecStoreType@vec_store_type<unsigned char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainCosVecStoreType@vec_store_type<signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainIPVecStoreType@vec_store_type<signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainL2VecStoreType@vec_store_type<signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::LVQCosVecStoreType@vec_store_type<float, signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::LVQIPVecStoreType@vec_store_type<float, signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::LVQL2VecStoreType@vec_store_type<float, signed char>, unsigned int>*, decltype(nullptr)> const&>(std::__1::__variant_detail::__visitation::__variant::__value_visitor<infinity::SegmentIndexEntry@segment_index_entry::RebuildChunkIndexEntries(infinity::TxnTableStore@txn_store*, infinity::SegmentEntry@segment_entry*)::$_1>&&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)0, infinity::KnnHnsw@hnsw_alg<infinity::PlainCosVecStoreType@vec_store_type<float>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainIPVecStoreType@vec_store_type<float>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainL2VecStoreType@vec_store_type<float>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainCosVecStoreType@vec_store_type<unsigned char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainIPVecStoreType@vec_store_type<unsigned char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainL2VecStoreType@vec_store_type<unsigned char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainCosVecStoreType@vec_store_type<signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainIPVecStoreType@vec_store_type<signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::PlainL2VecStoreType@vec_store_type<signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::LVQCosVecStoreType@vec_store_type<float, signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::LVQIPVecStoreType@vec_store_type<float, signed char>, unsigned int>*, infinity::KnnHnsw@hnsw_alg<infinity::LVQL2VecStoreType@vec_store_type<float, signed char>, unsigned int>*, decltype(nullptr)> const&)+158 at /infinity/src/storage/meta/entry/segment_index_entry.cpp:920>
$21 = 0x72415c <infinity::SegmentIndexEntry@segment_index_entry::RebuildChunkIndexEntries(infinity::TxnTableStore@txn_store*, infinity::SegmentEntry@segment_entry*)+2828 at /usr/local/include/c++/v1/__memory/unique_ptr.h:246>
$22 = 0x7638f0 <infinity::TableEntry@table_entry::OptimizeIndex(infinity::Txn@txn*)+528 at /usr/local/include/c++/v1/__tree:198>
$23 = 0x68b77c <infinity::CompactionProcessor@compaction_process::ScanAndOptimize()+684 at /usr/local/include/c++/v1/__iterator/wrap_iter.h:53>
$24 = 0x68cd26 <infinity::CompactionProcessor@compaction_process::Process()+1782 at /usr/local/include/c++/v1/string:1833>
$25 = 0x68d71a <std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, infinity::CompactionProcessor@compaction_process::Start()::$_0> >(void*)+42 at /usr/local/include/c++/v1/__memory/unique_ptr.h:263>

JinHai-CN pushed a commit that referenced this issue Jan 2, 2025
@yangzq50
Fix UnrecoverableError: Invalid MergeFlag: 1 (#2416)
d48860a 
### What problem does this PR solve?

Fix `bool TxnTableStore::CheckConflict(const TxnTableStore
*other_table_store) const`

Issue link:#2388

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
@JinHai-CN
Copy link
Contributor Author

JinHai-CN commented Jan 6, 2025 •
edited
Loading

Fixed by #2431
https://github.com/infiniflow/infinity/actions/runs/12627717736/job/35182741775?pr=2427

=================================================================
==90948==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x51500064fac0 at pc 0x5e242f56e63c bp 0x7e39a29eedf0 sp 0x7e39a29eede8
READ of size 8 at 0x51500064fac0 thread T334
#0 0x5e242f56e63b in infinity::SegmentIndexEntry@segment_index_entry::Flush(unsigned long) /infinity/src/storage/meta/entry/segment_index_entry.cpp:804:23
#1 0x5e242e1f5d77 in infinity::AddSegmentIndexEntryOp@catalog_delta_entry::FlushDataToDisk(unsigned long) /infinity/src/storage/wal/catalog_delta_entry.cpp:1016:98
#2 0x5e242f4252f7 in infinity::Catalog@catalog::SaveDeltaCatalog(unsigned long, unsigned long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) /infinity/src/storage/meta/catalog.cpp:1143:45
#3 0x5e242e0bc53d in infinity::Txn@txn::DeltaCheckpoint(unsigned long, unsigned long&) /infinity/src/storage/txn/txn.cpp:711:20
#4 0x5e242e2afc76 in infinity::WalManager@wal_manager::DeltaCheckpointInner(infinity::Txn@txn*) /infinity/src/storage/wal/wal_manager.cpp:554:19
#5 0x5e242e2acd80 in infinity::WalManager@wal_manager::Checkpoint(bool) /infinity/src/storage/wal/wal_manager.cpp:475:9
#6 0x5e242ee25429 in infinity::BGTaskProcessor@background_process::Process() /infinity/src/storage/background_process.cpp:129:39
#7 0x5e242ee26a79 in infinity::BGTaskProcessor@background_process::Start()::$_0::operator()() const /infinity/src/storage/background_process.cpp:39:41
#8 0x5e242ee26a14 in decltype(std::declval<infinity::BGTaskProcessor@background_process::Start()::$_0>()()) std::__1::__invoke[abi:ne180100]<infinity::BGTaskProcessor@background_process::Start()::$_0>(infinity::BGTaskProcessor@background_process::Start()::$_0&&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
#9 0x5e242ee269ec in void std::__1::__thread_execute[abi:ne180100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::BGTaskProcessor@background_process::Start()::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::BGTaskProcessor@background_process::Start()::$_0>&, std::__1::__tuple_indices<...>) /usr/local/include/c++/v1/__thread/thread.h:193:3
#10 0x5e242ee266b5 in void* std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, infinity::BGTaskProcessor@background_process::Start()::$_0>>(void*) /usr/local/include/c++/v1/__thread/thread.h:202:3
#11 0x5e242dd15138 in asan_thread_start(void*) /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_interceptors.cpp:239:28
#12 0x7e3b6fd11ea4 in start_thread pthread_create.c
#13 0x7e3b6f316b0c in clone (/lib64/libc.so.6+0xfeb0c) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25)

0x51500064fac0 is located 2160 bytes after 464-byte region [0x51500064f080,0x51500064f250)
allocated by thread T151 here:
#0 0x5e242dd549cd in operator new(unsigned long) /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_new_delete.cpp:86:3
#1 0x5e242f560e5f in infinity::SegmentIndexEntry@segment_index_entry::NewIndexEntry(infinity::TableIndexEntry@table_index_entry*, unsigned int, infinity::Txn@txn*) /infinity/src/storage/meta/entry/segment_index_entry.cpp:138:61
#2 0x5e242f84162f in infinity::TableIndexEntry@table_index_entry::PopulateEntirely(infinity::SegmentEntry@segment_entry*, infinity::Txn@txn*, infinity::PopulateEntireConfig@segment_index_entry const&) /infinity/src/storage/meta/entry/table_index_entry.cpp:354:56
#3 0x5e242f7c2c71 in infinity::TableEntry@table_entry::Import(std::__1::shared_ptr<infinity::SegmentEntry@segment_entry>, infinity::Txn@txn*) /infinity/src/storage/meta/entry/table_entry.cpp:435:32
#4 0x5e242e13de61 in infinity::TxnTableStore@txn_store::Import(std::__1::shared_ptr<infinity::SegmentEntry@segment_entry>, infinity::Txn@txn*) /infinity/src/storage/txn/txn_store.cpp:159:19
#5 0x5e242e0adefb in infinity::Txn@txn::Import(infinity::TableEntry@table_entry*, std::__1::shared_ptr<infinity::SegmentEntry@segment_entry>) /infinity/src/storage/txn/txn.cpp:115:18
#6 0x5e242fd44ca2 in infinity::PhysicalImport@physical_import::SaveSegmentData(infinity::TableEntry@table_entry*, infinity::Txn@txn*, std::__1::shared_ptr<infinity::SegmentEntry@segment_entry>) /infinity/src/executor/operator/physical_import.cpp:1913:10
#7 0x5e242fd32589 in infinity::PhysicalImport@physical_import::ImportCSV(infinity::QueryContext@query_context*, infinity::ImportOperatorState@operator_state*) /infinity/src/executor/operator/physical_import.cpp:572:13
#8 0x5e242fd30e42 in infinity::PhysicalImport@physical_import::Execute(infinity::QueryContext@query_context*, infinity::OperatorState@operator_state*) /infinity/src/executor/operator/physical_import.cpp:96:13
#9 0x5e242ee1b1aa in infinity::FragmentTask@fragment_task::OnExecute() /infinity/src/scheduler/fragment_task.cpp:85:58
#10 0x5e242e030dc3 in infinity::TaskScheduler@task_scheduler::RunTask(infinity::FragmentTask@fragment_task*) /infinity/src/scheduler/task_scheduler.cpp:190:15
#11 0x5e242e030610 in infinity::TaskScheduler@task_scheduler::Schedule(infinity::PlanFragment@plan_fragment*, infinity::BaseStatement const*) /infinity/src/scheduler/task_scheduler.cpp:155:17
#12 0x5e242ec27ebe in infinity::QueryContext@query_context::QueryStatement(infinity::BaseStatement const*) /infinity/src/main/query_context.cpp:264:21
#13 0x5e242ebcea01 in infinity::Infinity@infinity::Import(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, infinity::ImportOptions@query_options) /infinity/src/main/infinity.cpp:946:45
#14 0x5e242e9b93c1 in infinity::InfinityThriftService@infinity_thrift_service::Import(infinity_thrift_rpc::CommonResponse&, infinity_thrift_rpc::ImportRequest const&) /infinity/src/network/infinity_thrift_service.cpp:380:42
#15 0x5e24319237d9 in infinity_thrift_rpc::InfinityServiceProcessor::process_Import(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) /infinity/src/network/infinity_thrift/InfinityService.cpp:8849:17
#16 0x5e243191a8dd in infinity_thrift_rpc::InfinityServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, int, void*) /infinity/src/network/infinity_thrift/InfinityService.cpp:8428:5
#17 0x5e24319878d7 in apache::thrift::TDispatchProcessor::process(std::__1::shared_ptr<apache::thrift::protocol::TProtocol>, std::__1::shared_ptr<apache::thrift::protocol::TProtocol>, void*) /infinity/third_party/thrift/lib/cpp/src/thrift/TDispatchProcessor.h:121:12
#18 0x5e2433e0e991 in apache::thrift::server::TConnectedClient::run() /infinity/third_party/thrift/lib/cpp/src/thrift/server/TConnectedClient.cpp:61:24
#19 0x5e2433dcdb40 in apache::thrift::concurrency::ThreadManager::Task::run() /infinity/third_party/thrift/lib/cpp/src/thrift/concurrency/ThreadManager.cpp:195:18
#20 0x5e2433dc551f in apache::thrift::concurrency::ThreadManager::Worker::run() /infinity/third_party/thrift/lib/cpp/src/thrift/concurrency/ThreadManager.cpp:310:19
#21 0x5e2433e00bd9 in apache::thrift::concurrency::Thread::threadMain(std::__1::shared_ptr<apache::thrift::concurrency::Thread>) /infinity/third_party/thrift/lib/cpp/src/thrift/concurrency/Thread.cpp:28:23
#22 0x5e2433e00097 in decltype(std::declval<void (*)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>)>()(std::declval<std::__1::shared_ptr<apache::thrift::concurrency::Thread>>())) std::__1::__invoke[abi:ne180100]<void (*)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>>(void (*&&)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>&&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
#23 0x5e2433dfff7d in void std::__1::__thread_execute[abi:ne180100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void (*)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void (*)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>>&, std::__1::__tuple_indices<2ul>) /usr/local/include/c++/v1/__thread/thread.h:193:3
#24 0x5e2433dffb95 in void* std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void (*)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>>>(void*) /usr/local/include/c++/v1/__thread/thread.h:202:3
#25 0x5e242dd15138 in asan_thread_start(void*) /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_interceptors.cpp:239:28

Thread T334 created by T0 here:
#0 0x5e242dcfd211 in pthread_create /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_interceptors.cpp:250:3
#1 0x5e242deab4d8 in std::__1::__libcpp_thread_create[abi:ne180100](unsigned long*, void* (*)(void*), void*) /usr/local/include/c++/v1/__threading_support:317:10
#2 0x5e242ee23447 in std::__1::thread::thread<infinity::BGTaskProcessor@background_process::Start()::$_0, void>(infinity::BGTaskProcessor@background_process::Start()::$_0&&) /usr/local/include/c++/v1/__thread/thread.h:212:14
#3 0x5e242ee2311b in infinity::BGTaskProcessor@background_process::Start() /infinity/src/storage/background_process.cpp:39:25
#4 0x5e242e0942ad in infinity::Storage@storage::AdminToWriter() /infinity/src/storage/storage.cpp:299:20
#5 0x5e242e09aa59 in infinity::Storage@storage::SetStorageMode(infinity::StorageMode@wal_manager) /infinity/src/storage/storage.cpp:663:28
#6 0x5e242df7d882 in infinity::InfinityContext@infinity_context::ChangeServerRole(infinity::NodeRole, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, unsigned short) /infinity/src/main/infinity_context.cpp:147:59
#7 0x5e242df8345e in infinity::InfinityContext@infinity_context::InitPhase2() /infinity/src/main/infinity_context.cpp:105:39
#8 0x5e242dd57f36 in main /infinity/src/bin/infinity_main.cpp:281:33
#9 0x7e3b6f23a554 in __libc_start_main (/lib64/libc.so.6+0x22554) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25)

Thread T151 created by T0 here:
#0 0x5e242dcfd211 in pthread_create /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_interceptors.cpp:250:3
#1 0x5e242deab4d8 in std::__1::__libcpp_thread_create[abi:ne180100](unsigned long*, void* (*)(void*), void*) /usr/local/include/c++/v1/__threading_support:317:10
#2 0x5e2433dff565 in std::__1::thread::thread<void (*)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>&, void>(void (*&&)(std::__1::shared_ptr<apache::thrift::concurrency::Thread>), std::__1::shared_ptr<apache::thrift::concurrency::Thread>&) /usr/local/include/c++/v1/__thread/thread.h:212:14
#3 0x5e2433dfeac9 in apache::thrift::concurrency::Thread::start() /infinity/third_party/thrift/lib/cpp/src/thrift/concurrency/Thread.h:130:48
#4 0x5e2433db66e8 in apache::thrift::concurrency::ThreadManager::Impl::addWorker(unsigned long) /infinity/third_party/thrift/lib/cpp/src/thrift/concurrency/ThreadManager.cpp:361:16
#5 0x5e2433dc0d38 in apache::thrift::concurrency::SimpleThreadManager::start() /infinity/third_party/thrift/lib/cpp/src/thrift/concurrency/ThreadManager.cpp:567:5
#6 0x5e242ea848f1 in infinity::PoolThriftServer@thrift_server::Init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, int, int) /infinity/src/network/thrift_server.cpp:108:20
#7 0x5e242dd5bafc in (anonymous namespace)::StartThriftServer() /infinity/src/bin/infinity_main.cpp:80:24
#8 0x5e242dd5b970 in main::$_0::operator()() const /infinity/src/bin/infinity_main.cpp:259:9
#9 0x5e242dd5b954 in decltype(std::declval<main::$_0&>()()) std::__1::__invoke[abi:ne180100]<main::$_0&>(main::$_0&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
#10 0x5e242dd5b914 in void std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne180100]<main::$_0&>(main::$_0&) /usr/local/include/c++/v1/__type_traits/invoke.h:419:5
#11 0x5e242dd5b8ec in std::__1::__function::__alloc_func<main::$_0, std::__1::allocator<main::$_0>, void ()>::operator()[abi:ne180100]() /usr/local/include/c++/v1/__functional/function.h:169:12
#12 0x5e242dd5a108 in std::__1::__function::__func<main::$_0, std::__1::allocator<main::$_0>, void ()>::operator()() /usr/local/include/c++/v1/__functional/function.h:311:10
#13 0x5e242de94aeb in std::__1::__function::__value_func<void ()>::operator()[abi:ne180100]() const /usr/local/include/c++/v1/__functional/function.h:428:12
#14 0x5e242de94a14 in std::__1::function<void ()>::operator()() const /usr/local/include/c++/v1/__functional/function.h:981:10
#15 0x5e242df83d3d in infinity::InfinityContext@infinity_context::StartThriftServers() /infinity/src/main/infinity_context.cpp:540:9
#16 0x5e242dd57d7e in main /infinity/src/bin/infinity_main.cpp:269:33
#17 0x7e3b6f23a554 in __libc_start_main (/lib64/libc.so.6+0x22554) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25)

SUMMARY: AddressSanitizer: heap-buffer-overflow /infinity/src/storage/meta/entry/segment_index_entry.cpp:804:23 in infinity::SegmentIndexEntry@segment_index_entry::Flush(unsigned long)
Shadow bytes around the buggy address:
0x51500064f800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064f880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064f900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064f980: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064fa00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x51500064fa80: fa fa fa fa fa fa fa fa[fa]fa fa fa fa fa fa fa
0x51500064fb00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064fb80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064fc00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064fc80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x51500064fd00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable:           00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone:       fa
Freed heap region:       fd
Stack left redzone:      f1
Stack mid redzone:       f2
Stack right redzone:     f3
Stack after return:      f5
Stack use after scope:   f8
Global redzone:          f9
Global init order:       f6
Poisoned by user:        f7
Container overflow:      fc
Array cookie:            ac
Intra object redzone:    bb
ASan internal:           fe
Left alloca redzone:     ca
Right alloca redzone:    cb
==90948==ABORTING

This was referenced Jan 6, 2025
Merged
Merged
@small-turtle-1
Copy link
Contributor

small-turtle-1 commented Jan 7, 2025 •
edited
Loading

pr #2431 fix:
#2388 (comment)
#2388 (comment)

@yangzq50
Copy link
Contributor

yangzq50 commented Jan 7, 2025

In unittest, InfinityContext::instance().storage()->memindex_tracer() returned nullptr

https://github.com/infiniflow/infinity/actions/runs/12645309717/job/35234245125

==242138==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000030 (pc 0x55defd90f90c bp 0x7f33b2feaf90 sp 0x7f33b2fead20 T56)
==242138==The signal is caused by a READ memory access.
==242138==Hint: address points to the zero page.
    #0 0x55defd90f90c in infinity::MemIndexTracer@memindex_tracer::IncreaseMemoryUsage(unsigned long) /infinity/src/storage/tracer/memindex_tracer.cppm:89:41
    #1 0x55deffc83385 in infinity::BaseMemIndex@base_memindex::IncreaseMemoryUsageBase(unsigned long) /infinity/src/storage/tracer/base_memindex.cpp:27:22
    #2 0x55deff081534 in infinity::MemoryIndexer@memory_indexer::IncreaseMemoryUsage(unsigned long) /infinity/src/storage/invertedindex/memory_indexer.cpp:441:19
    #3 0x55deff086850 in infinity::MemoryIndexer@memory_indexer::GetOrAddPosting(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) /infinity/src/storage/invertedindex/memory_indexer.cpp:403:9
    #4 0x55deff090a04 in infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const /infinity/src/storage/invertedindex/memory_indexer.cpp:150:106
    #5 0x55deff090997 in decltype(std::declval<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>())) std::__1::__invoke[abi:ne180100]<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
    #6 0x55deff090937 in std::__1::shared_ptr<infinity::PostingWriter@posting_writer> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<infinity::PostingWriter@posting_writer>, false>::__call[abi:ne180100]<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) /usr/local/include/c++/v1/__type_traits/invoke.h:411:12
    #7 0x55deff0908f7 in std::__1::__function::__alloc_func<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1, std::__1::allocator<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1>, std::__1::shared_ptr<infinity::PostingWriter@posting_writer> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)>::operator()[abi:ne180100](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) /usr/local/include/c++/v1/__functional/function.h:169:12
    #8 0x55deff08f0ab in std::__1::__function::__func<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1, std::__1::allocator<infinity::MemoryIndexer@memory_indexer::Insert(std::__1::shared_ptr<infinity::ColumnVector@column_vector>, unsigned int, unsigned int, bool)::$_1>, std::__1::shared_ptr<infinity::PostingWriter@posting_writer> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) /usr/local/include/c++/v1/__functional/function.h:311:10
    #9 0x55deff00010e in std::__1::__function::__value_func<std::__1::shared_ptr<infinity::PostingWriter@posting_writer> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)>::operator()[abi:ne180100](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const /usr/local/include/c++/v1/__functional/function.h:428:12
    #10 0x55defefe1027 in std::__1::function<std::__1::shared_ptr<infinity::PostingWriter@posting_writer> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const /usr/local/include/c++/v1/__functional/function.h:981:10
    #11 0x55defefde11f in infinity::ColumnInverter@column_inverter::GeneratePosting() /infinity/src/storage/invertedindex/column_inverter.cpp:222:23
    #12 0x55deff07ee33 in infinity::MemoryIndexer@memory_indexer::CommitSync(unsigned long) /infinity/src/storage/invertedindex/memory_indexer.cpp:247:44
    #13 0x55deff095d93 in infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0::operator()(int) const /infinity/src/storage/invertedindex/memory_indexer.cpp:184:33
    #14 0x55deff095be8 in decltype(std::declval<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&>()(std::declval<int>())) std::__1::__invoke[abi:ne180100]<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&, int>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&, int&&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
    #15 0x55deff095808 in std::__1::__packaged_task_func<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0, std::__1::allocator<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>, void (int)>::operator()(int&&) /usr/local/include/c++/v1/future:1442:10
    #16 0x55deff0a615d in std::__1::__packaged_task_function<void (int)>::operator()(int) const /usr/local/include/c++/v1/future:1593:10
    #17 0x55deff0a5ed3 in std::__1::packaged_task<void (int)>::operator()(int) /usr/local/include/c++/v1/future:1754:5
    #18 0x55deff098382 in std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int)::operator()(int) const /infinity/third_party/ctpl/ctpl_stl.h:262:17
    #19 0x55deff098358 in decltype(std::declval<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>()(std::declval<int>())) std::__1::__invoke[abi:ne180100]<std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int)&, int>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&, int&&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
    #20 0x55deff0982bc in void std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne180100]<std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int)&, int>(std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int)&, int&&) /usr/local/include/c++/v1/__type_traits/invoke.h:419:5
    #21 0x55deff098294 in std::__1::__function::__alloc_func<std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int), std::__1::allocator<std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int)>, void (int)>::operator()[abi:ne180100](int&&) /usr/local/include/c++/v1/__functional/function.h:169:12
    #22 0x55deff096950 in std::__1::__function::__func<std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int), std::__1::allocator<std::__1::future<decltype(fp(0))> ctpl::thread_pool::push<infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0>(infinity::MemoryIndexer@memory_indexer::Commit(bool)::$_0&&)::'lambda'(int)>, void (int)>::operator()(int&&) /usr/local/include/c++/v1/__functional/function.h:311:10
    #23 0x55defd20d16b in std::__1::__function::__value_func<void (int)>::operator()[abi:ne180100](int&&) const /usr/local/include/c++/v1/__functional/function.h:428:12
    #24 0x55defd20c49a in std::__1::function<void (int)>::operator()(int) const /usr/local/include/c++/v1/__functional/function.h:981:10
    #25 0x55defd20bc58 in ctpl::thread_pool::set_thread(int)::'lambda'()::operator()() const /infinity/third_party/ctpl/ctpl_stl.h:289:25
    #26 0x55defd20ba14 in decltype(std::declval<ctpl::thread_pool::set_thread(int)::'lambda'()>()()) std::__1::__invoke[abi:ne180100]<ctpl::thread_pool::set_thread(int)::'lambda'()>(ctpl::thread_pool::set_thread(int)::'lambda'()&&) /usr/local/include/c++/v1/__type_traits/invoke.h:344:25
    #27 0x55defd20b9ec in void std::__1::__thread_execute[abi:ne180100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ctpl::thread_pool::set_thread(int)::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ctpl::thread_pool::set_thread(int)::'lambda'()>&, std::__1::__tuple_indices<...>) /usr/local/include/c++/v1/__thread/thread.h:193:3
    #28 0x55defd20b665 in void* std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ctpl::thread_pool::set_thread(int)::'lambda'()>>(void*) /usr/local/include/c++/v1/__thread/thread.h:202:3
    #29 0x55defcbe7718 in asan_thread_start(void*) /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_interceptors.cpp:239:28
    #30 0x7f33f4c36ea4 in start_thread pthread_create.c
    #31 0x7f33f423bb0c in clone (/lib64/libc.so.6+0xfeb0c) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /infinity/src/storage/tracer/memindex_tracer.cppm:89:41 in infinity::MemIndexTracer@memindex_tracer::IncreaseMemoryUsage(unsigned long)
Thread T56 created by T0 here:
    #0 0x55defcbcf7f1 in pthread_create /root/llvm-project-18.1.8.src/compiler-rt/lib/asan/asan_interceptors.cpp:250:3
    #1 0x55defcd724e8 in std::__1::__libcpp_thread_create[abi:ne180100](unsigned long*, void* (*)(void*), void*) /usr/local/include/c++/v1/__threading_support:317:10
    #2 0x55defd20b1d7 in std::__1::thread::thread<ctpl::thread_pool::set_thread(int)::'lambda'()&, void>(ctpl::thread_pool::set_thread(int)::'lambda'()&) /usr/local/include/c++/v1/__thread/thread.h:212:14
    #3 0x55defd2013c0 in ctpl::thread_pool::set_thread(int) /infinity/third_party/ctpl/ctpl_stl.h:304:40
    #4 0x55defd1ff5e8 in ctpl::thread_pool::resize(int) /infinity/third_party/ctpl/ctpl_stl.h:171:31
    #5 0x55defe483b1d in infinity::InfinityContext@infinity_context::SetIndexThreadPool() /infinity/src/main/infinity_context.cpp:518:28
    #6 0x55defe47e729 in infinity::InfinityContext@infinity_context::ChangeServerRole(infinity::NodeRole, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, unsigned short) /infinity/src/main/infinity_context.cpp:242:13
    #7 0x55defe482aee in infinity::InfinityContext@infinity_context::InitPhase2() /infinity/src/main/infinity_context.cpp:105:39
    #8 0x55defcdd5686 in BufferObjTest_test1_Test::TestBody() /infinity/src/unit_test/storage/buffer/buffer_obj.cpp:92:43
    #9 0x55df02955028 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /infinity/third_party/googletest/googletest/src/gtest.cc:2599:10
    #10 0x55df029046d1 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /infinity/third_party/googletest/googletest/src/gtest.cc:2635:14
    #11 0x55df028c8ca5 in testing::Test::Run() /infinity/third_party/googletest/googletest/src/gtest.cc:2674:5
    #12 0x55df028ca337 in testing::TestInfo::Run() /infinity/third_party/googletest/googletest/src/gtest.cc:2853:11
    #13 0x55df028cb495 in testing::TestSuite::Run() /infinity/third_party/googletest/googletest/src/gtest.cc:3012:30
    #14 0x55df028e9ec6 in testing::internal::UnitTestImpl::RunAllTests() /infinity/third_party/googletest/googletest/src/gtest.cc:5870:44
    #15 0x55df02955b48 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /infinity/third_party/googletest/googletest/src/gtest.cc:2599:10
    #16 0x55df02908d96 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /infinity/third_party/googletest/googletest/src/gtest.cc:2635:14
    #17 0x55df028e9299 in testing::UnitTest::Run() /infinity/third_party/googletest/googletest/src/gtest.cc:5444:10
    #18 0x55defcc293e0 in RUN_ALL_TESTS() /infinity/third_party/googletest/googletest/include/gtest/gtest.h:2293:73
    #19 0x55defcc29376 in main /infinity/src/unit_test/main.cpp:19:12
    #20 0x7f33f415f554 in __libc_start_main (/lib64/libc.so.6+0x22554) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25)

==242138==ABORTING

JinHai-CN pushed a commit that referenced this issue Jan 7, 2025
@small-turtle-1
Fix checkpoint ts. (#2428)
60afe8d 
### What problem does this PR solve?

Fix checkpoint ts bug.

Issue link:#2388 Issue 5.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
JinHai-CN pushed a commit that referenced this issue Jan 7, 2025
@small-turtle-1
Fix dump deleted segment index in dump task. (#2431)
5a1179a 
### What problem does this PR solve?

Issue link:#2388

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
@small-turtle-1 small-turtle-1 mentioned this issue Jan 9, 2025
Merged
1 task
@small-turtle-1
Copy link
Contributor

pr #2438 fix issue 3

JinHai-CN pushed a commit that referenced this issue Jan 9, 2025
@small-turtle-1
Fix ts bug. (#2438)
d6537f8 
### What problem does this PR solve?

Fix checkpoint ts bug.

Issue link:#2388

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
@small-turtle-1
Copy link
Contributor

issue 4 has been fixed by #2389

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@small-turtle-1 small-turtle-1

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@JinHai-CN @yangzq50 @small-turtle-1