Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

basic test error #1

Open
FrancisLeeHa opened this issue Aug 5, 2020 · 8 comments
Open

basic test error #1

FrancisLeeHa opened this issue Aug 5, 2020 · 8 comments
Assignees
@xjtuwxg
Labels
bug Something isn't working enhancement New feature or request

Comments

@FrancisLeeHa
Copy link

FrancisLeeHa commented Aug 5, 2020 •
edited
Loading

Hello. When I run mvx basic test on arm, I got this error on x86 kernel, It seems to be the filename or command error, but I don`t know how to fix it. Would you please help me, thanks!
I make this experiment on qume
popcorn@arm:~/mvx$ LD_PRELOAD=./loader.so ./basic

popcorn@x86:~/mvx$ [42121.379789] SYSC_open: filename open.c, ret 0
[42121.383889] SYSC_open: ret -2 (0xfffffffe)
[42121.387782] mvx_update_fd_vtab: Should not happen for fd id -2
[42121.407778] follower ret -9 != 18 from master
[42121.423786] [MVX Violation] kernel/popcorn/mvx_engine.c:328 mvx_follower_wait_exec
[42121.427780] mvx_update_fd_vtab: Should not happen for fd id -1
[42121.443900] follower ret -9 != 18 from master
[42121.455883] [MVX Violation] kernel/popcorn/mvx_engine.c:225 follower_writev
[42121.459780] follower ret -9 != 63 from master
[42121.463961] ------------[ cut here ]------------
[42121.464863] kernel BUG at fs/read_write.c:1007!
[42121.465599] invalid opcode: 0000 [#6] SMP
[42121.466357] Modules linked in: msg_socket
[42121.467057] CPU: 0 PID: 27729 Comm: open Tainted: G D 4.4.137-popcorn+ #1
[42121.467773] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
[42121.467773] task: ffff8800bb78c580 ti: ffff8800b9134000 task.ti: ffff8800b9134000
[42121.467773] RIP: 0010:[] [] SyS_writev+0x1c7/0x220
[42121.467773] RSP: 0018:ffff8800b9137e90 EFLAGS: 00010202
[42121.467773] RAX: 000000000000007f RBX: 0000000000000001 RCX: 0000000000000000
[42121.467773] RDX: 000000000000007f RSI: 0000000000000000 RDI: ffff8800b9137f1f
[42121.467773] RBP: ffff8800bac55780 R08: 0000000000000004 R09: ffff8800b9137ea0
[42121.467773] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000007f
[42121.467773] R13: 000000000000007f R14: 0000000000000000 R15: 0000000000000000
[42121.467773] FS: 00007ffff7ffe288(0000) GS:ffff88013fc00000(0000) knlGS:0000000000000000
[42121.467773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[42121.467773] CR2: 0000000000000003 CR3: 00000000bb74a000 CR4: 0000000000360670
[42121.467773] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[42121.467773] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[42121.467773] Stack:
[42121.467773] 000000000000003f ffffffff813fc25e 0000000000000000 0000000000000000
[42121.467773] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[42121.467773] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[42121.467773] Call Trace:
[42121.467773] [] ? __schedule+0x1ce/0x730
[42121.467773] [] ? entry_SYSCALL_64_fastpath+0x1e/0x8e
[42121.467773] Code: c3 76 2a 48 c1 e0 04 4d 89 e5 48 8d 7c 24 10 48 01 e8 48 83 78 08 7f 4c 0f 46 68 08 48 8b 30 44 89 ea e8 2d a1 13 00 85 c0 74 c6 <0f> 0b 48 89 ef e8 cf 94 ff ff 48 8b
[42121.467773] RIP [] SyS_writev+0x1c7/0x220
[42121.467773] RSP
[42121.467777] ---[ end trace 08c469fbdb08ec3f ]---
@xjtuwxg
Copy link
Member

xjtuwxg commented Aug 5, 2020

Hi, you need to build the basic on both x86 and arm nodes, using the same version of musl-gcc. Also, make sure to move the basic binary to the same location on each node.

@FrancisLeeHa
Copy link
Author

Hi, you need to build the basic on both x86 and arm nodes, using the same version of musl-gcc. Also, make sure to move the basic binary to the same location on each node.

Thanks! But I just build the basic on both x86 and arm and using the same version of musl-gcc

popcorn@arm:~/mvx$ /usr/local/musl/bin/musl-gcc --version 
gcc (Debian/Linaro 4.9.2-10) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

popcorn@x86:~/mvx$  /usr/local/musl/bin/musl-gcc --version
gcc (Debian 4.9.2-10) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Also， I just put them to the same location on each node, but it still run with error.

  • this is arm output
popcorn@arm:~/mvx$ pwd
/home/popcorn/mvx
popcorn@arm:~/mvx$ ls
Makefile  epoll    lighttpd       musl-1.2.0         open    tiny.c
basic     epoll.c  lighttpd.conf  musl-1.2.0.tar.gz  open.c
basic.c   info.h   loader.so      mvx_syscall.c      tiny
popcorn@arm:~/mvx$ LD_PRELOAD=./loader.so ./basic
Issue an MVX init syscall.
[ 1691.235831] 
[ 1691.235831] === HeterSec syscall ===
[ 1691.235977] Arg0: 0x1(1)
[ 1691.236098] [ 0] argv[0]: ./basic. arg_size 7
=== [MVX Unit Test] Testing 2 syscalls: ===
writev, read

Writting to stderr
[ 1691.249273] fd 2, count 20, ret 20
Input your name (type "quit" to exit): 
test
5 char(s) read, buf: test

^C
popcorn@arm:~/mvx$
  • this is x86 output
popcorn@x86:~/mvx$ pwd
/home/popcorn/mvx
popcorn@x86:~/mvx$ ls
basic    epoll.c   lighttpd.conf  musl-1.2.0         open    tiny.c
basic.c  info.h    loader.so      musl-1.2.0.tar.gz  open.c
epoll    lighttpd  Makefile       mvx_syscall.c      tiny
popcorn@x86:~/mvx$ [ 1726.536003] follower ret -9 != 44 from master
[ 1726.537720] fd 2, count 20, ret -9
[ 1726.538878] [**MVX Violation**] kernel/popcorn/mvx_engine.c:254 follower_write
[ 1726.540680] follower ret -9 != 14 from master
[ 1726.543477] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[ 1726.545314] follower ret -9 != 20 from master
[ 1726.548628] follower ret -9 != 40 from master
[ 1730.223155] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[ 1730.225316] follower ret -9 != 5 from master
[ 1730.226863] ------------[ cut here ]------------
[ 1730.228359] kernel BUG at fs/read_write.c:1007!
[ 1730.229297] invalid opcode: 0000 [#2] SMP 
[ 1730.229297] Modules linked in: msg_socket
[ 1730.229297] CPU: 1 PID: 707 Comm: basic Tainted: G      D         4.4.137-popcorn+ #1
[ 1730.229297] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
[ 1730.229297] task: ffff8800ba32c640 ti: ffff8800bb734000 task.ti: ffff8800bb734000
[ 1730.229297] RIP: 0010:[<ffffffff811616d7>]  [<ffffffff811616d7>] SyS_writev+0x1c7/0x220
[ 1730.229297] RSP: 0018:ffff8800bb737e90  EFLAGS: 00010202
[ 1730.229297] RAX: 000000000000007f RBX: 0000000000000001 RCX: 0000000000000000
[ 1730.229297] RDX: 000000000000007f RSI: 0000000000000000 RDI: ffff8800bb737f1f
[ 1730.229297] RBP: ffff8800b8815660 R08: 0000000000000007 R09: ffff8800bb737ea0
[ 1730.229297] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000007f
[ 1730.229297] R13: 000000000000007f R14: 0000000000000000 R15: 0000000000000000
[ 1730.229297] FS:  00007ffff7ffe288(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
[ 1730.229297] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1730.229297] CR2: 0000000000000006 CR3: 00000000bb4ee000 CR4: 0000000000360670
[ 1730.229297] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1730.229297] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 1730.229297] Stack:
[ 1730.229297]  0000000000000005 ffffffff813fc25e 0000000000000000 0000000000000000
[ 1730.229297]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 1730.229297]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 1730.229297] Call Trace:
[ 1730.229297]  [<ffffffff813fc25e>] ? __schedule+0x1ce/0x730
[ 1730.229297]  [<ffffffff813ff90a>] ? entry_SYSCALL_64_fastpath+0x1e/0x8e
[ 1730.229297] Code: c3 76 2a 48 c1 e0 04 4d 89 e5 48 8d 7c 24 10 48 01 e8 48 83 78 08 7f 4c 0f 46 68 08 48 8b 30 44 89 ea e8 2d a1 13 00 85 c0 74 c6 <0f> 0b 48 89 ef e8 cf 94 ff ff 48 8b  
[ 1730.229297] RIP  [<ffffffff811616d7>] SyS_writev+0x1c7/0x220
[ 1730.229297]  RSP <ffff8800bb737e90>
[ 1730.254822] ---[ end trace fddde1203d3b9a64 ]---
popcorn@x86:~/mvx$

And if I run it separately, it will run correctly.

  • this is arm
popcorn@arm:~/mvx$ ./basic 
=== [MVX Unit Test] Testing 2 syscalls: ===
writev, read

Writting to stderr
Input your name (type "quit" to exit): 
test
5 char(s) read, buf: test

Input your name (type "quit" to exit): 
quit
5 char(s) read, buf: quit

popcorn@arm:~/mvx$
  • this is x86
 popcorn@x86:~/mvx$ ./basic 
=== [MVX Unit Test] Testing 2 syscalls: ===
writev, read

Writting to stderr
Input your name (type "quit" to exit): 
test
5 char(s) read, buf: test

Input your name (type "quit" to exit): 
exit
5 char(s) read, buf: exit

Input your name (type "quit" to exit): 
quit
5 char(s) read, buf: quit

popcorn@x86:~/mvx$

How to fix it? thanks

@xjtuwxg
Copy link
Member

xjtuwxg commented Aug 6, 2020

Hi, could you enable the debug information especially on the x86 side, and share us the log?
make menuconfig --> Popcorn Distributed Execution Support --> [*] MVX engine; [*] Syscall server; [*] Task migration

Also, could you share us the binaries you have (basic.x86 basic.arm)?

@FrancisLeeHa
Copy link
Author

FrancisLeeHa commented Aug 6, 2020 •
edited
Loading

OK, I will share the binaries now and log later, I send binaries to your email [email protected], thanks!

@FrancisLeeHa
Copy link
Author

I have the log now

  • on arm
popcorn@arm:~/mvx$ LD_PRELOAD=./loader.so ./basic 
Issue an MVX init syscall.
[  333.494540] 
[  333.494540] === HeterSec syscall ===
[  333.494688] Arg0: 0x1(1)
[  333.495377] [ 0] argv[0]: ./basic. arg_size 7
[  333.495505] [MVX] mvx_prepare_init_req: Retrieved argc 1, cmdline size 8, cmdline ./basic, exe path /home/popcorn/mvx/basic
[  333.495690] [MVX] mvx_server_start_mvx: [268] exe_path /home/popcorn/mvx/basic. arg_start 0x7fffffffeed9, arg_end 0x7fffffffeee1
[  333.495871] [MVX] mvx_server_start_mvx: size 2070; dst nid 0
[  333.502394] [MVX] MVX master variant [PID:268] init ...
=== [MVX Unit Test] Testing 2 syscalls: ===
[  333.503957] [MVX] 
[  333.503957] [ 0] syscall 66: <SYS_writev>
[  333.504252] [MVX] master waits for follower return on syscall 66
[  333.534449] [MVX] --> handle_mvx_reply: ret 0x2c(44), follower syscall 20
[  333.534822] [MVX] 
[  333.534822] SYSC_writev: mvx [268]. fd 1, vlen 2. MVX master.
[  333.535102] [MVX] SYSC_writev: buf: === [MVX Unit Test] Testing 2
[  333.535220] [MVX] SYSC_writev: buf:  syscalls: ===
writev, read

[  333.535844] [MVX] 
[  333.535844] [ 1] syscall 66: <SYS_writev>
[  333.535996] [MVX] master waits for follower return on syscall 66
[  333.544207] [MVX] --> handle_mvx_reply: ret 0xe(14), follower syscall 1
[  333.544681] [MVX] 
[  333.544681] SYSC_writev: mvx [268]. fd 1, vlen 2. MVX master.
[  333.544997] [MVX] SYSC_writev: buf: writev, read
[  333.545157] [MVX] SYSC_writev: buf: 
[  333.545157] 
Writting to stderr
[  333.549648] fd 2, count 20, ret 20
[  333.549755] [MVX] 
[  333.549755] [ 2] syscall 64: <SYS_write>
[  333.549907] [MVX] master waits for follower return on syscall 64
[  333.553855] [MVX] --> handle_mvx_reply: ret 0x14(20), follower syscall 20
[  333.554736] [MVX] SYSC_write: mvx [268] Writting to stderr
[  333.554736] 
[  333.554870] [MVX] Not support SYSC_write yet.
Input your name (type "quit" to exit): 
[  333.555260] [MVX] 
[  333.555260] [ 3] syscall 66: <SYS_writev>
[  333.555389] [MVX] master waits for follower return on syscall 66
[  333.559381] [MVX] --> handle_mvx_reply: ret 0x28(40), follower syscall 20
[  333.559860] [MVX] 
[  333.559860] SYSC_writev: mvx [268]. fd 1, vlen 2. MVX master.
[  333.560175] [MVX] SYSC_writev: buf: Input your name (type "quit" to exit): 
[  333.560402] [MVX] SYSC_writev: buf: 
test[  345.775783] [MVX] 
[  345.775783] [ 4] syscall 63: <SYS_read>
[  345.776341] [MVX] master_read: fd 0, flag 2, len 5, retval 5.
[  345.776651] [MVX] master waits for follower return on syscall 63

[  345.788424] [MVX] --> handle_mvx_reply: ret 0x5(5), follower syscall 20
[  345.788887] [MVX post syscall] SYSC_read: ret 5
5 char(s) read, buf: test
[  345.790600] [MVX] 
[  345.790600] [ 5] syscall 66: <SYS_writev>
[  345.790888] [MVX] master waits for follower return on syscall 66
  • on x86
popcorn@x86:~/mvx$ [  409.701426] [MVX] clone_mvx_thread: MVX helper thread [675] finished launching MVX process, bin path: /home/popcorn/mvx/basic.
[  409.704798] [MVX] argv[0] ./basic (7)
[  409.706156] [MVX] MVX follower variant [PID:676] init ...
[  409.708168] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 44 (master)
[  409.711781] [MVX] Cannot open STDIN file: -13.
[  409.712826] [MVX] [676] init_mvx_follower_process: Finished the Follower MVX process initialization.
[  409.716864] [MVX] mvx_process_fork: Finished MVX process setup. ret 0
[  409.731424] [MVX] 
[  409.731825] [MVX] Follower waits msg on
[  409.731825] [ 0] syscall: <SYS_writev>
[  409.733575] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  409.734889] follower ret -9 != 44 from master
[  409.735805] [MVX] 
[  409.735805] SYSC_writev: mvx [676]. fd 1, vlen 2. MVX follower.
[  409.737515] [MVX] SYSC_writev: buf: === [MVX Unit Test] Testing 2
[  409.738398] [MVX] SYSC_writev: buf:  syscalls: ===
[  409.739157] fd 2, count 20, ret -9
[  409.739841] [MVX] 
[  409.740064] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 14 (master)
[  409.741413] [MVX] Follower waits msg on
[  409.741413] [ 1] syscall: <SYS_write>
[  409.742974] [**MVX Violation**] kernel/popcorn/mvx_engine.c:254 follower_write
[  409.744118] [MVX] syscall 1: <SYS_write>. flag 2. master msg syscall 20
[  409.745140] follower ret -9 != 14 from master
[  409.745935] [MVX] SYSC_write: mvx [676] Writting to stderr
[  409.745935] 
[  409.746657] [MVX] Not support SYSC_write yet.
[  409.747053] [MVX] 
[  409.747245] [MVX] Follower waits msg on
[  409.747245] [ 2] syscall: <SYS_writev>
[  409.753264] [MVX] --> handle_mvx_message: syscall #1, flag 2, len 0, retval 20 (master)
[  409.754293] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[  409.754957] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 1
[  409.755587] follower ret -9 != 20 from master
[  409.756159] [MVX] 
[  409.756159] SYSC_writev: mvx [676]. fd 1, vlen 2. MVX follower.
[  409.757033] [MVX] SYSC_writev: buf: writev, read
[  409.757033] 
[  409.757033] Input your name (type "quit" to exit): 
[  409.758033] [MVX] SYSC_writev: buf: 
[  409.758333] [MVX] 
[  409.758547] [MVX] Follower waits msg on
[  409.758547] [ 3] syscall: <SYS_writev>
[  409.759359] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 40 (master)
[  409.760348] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  409.760989] follower ret -9 != 40 from master
[  409.761409] [MVX] 
[  409.761409] SYSC_writev: mvx [676]. fd 1, vlen 2. MVX follower.
[  409.762224] [MVX] SYSC_writev: buf: your name (type "quit" to exit): 
[  409.762799] [MVX] SYSC_writev: buf: 
[  409.763099] [MVX] 
[  409.763309] [MVX] Follower waits msg on
[  409.763309] [ 4] syscall: <SYS_writev>
[  421.981646] [MVX] --> handle_mvx_message: syscall #0, flag 2, len 5, retval 5 (master)
[  421.983961] [MVX] --> buf: test
[  421.983961] 
[  421.985666] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[  421.987810] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 0
[  421.989817] follower ret -9 != 5 from master
[  421.991080] [MVX] 
[  421.991080] SYSC_writev: mvx [676]. fd 1, vlen 1. MVX follower.
[  421.994065] ------------[ cut here ]------------
[  421.995408] kernel BUG at fs/read_write.c:1007!
[  421.996813] invalid opcode: 0000 [#1] SMP 
[  421.997596] Modules linked in: msg_socket
[  421.997596] CPU: 1 PID: 676 Comm: basic Not tainted 4.4.137-popcorn+ #3
[  421.997596] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
[  421.997596] task: ffff88013ad36a80 ti: ffff88013aeb8000 task.ti: ffff88013aeb8000
[  421.997596] RIP: 0010:[<ffffffff8124f04d>]  [<ffffffff8124f04d>] SyS_writev.cold.4+0xc8/0xca
[  421.997596] RSP: 0018:ffff88013aebbe90  EFLAGS: 00010202
[  421.997596] RAX: 000000000000007f RBX: ffff8800bad66820 RCX: 0000000000000000
[  421.997596] RDX: 000000000000007f RSI: 0000000000000000 RDI: ffff88013aebbf1f
[  421.997596] RBP: 0000000000000001 R08: 0000000000000007 R09: ffff88013aebbea0
[  421.997596] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[  421.997596] R13: 000000000000007f R14: 0000000000000000 R15: 0000000000000000
[  421.997596] FS:  00007ffff7ffe288(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
[  421.997596] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  421.997596] CR2: 0000000000000006 CR3: 000000013ad52000 CR4: 0000000000360670
[  421.997596] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  421.997596] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  421.997596] Stack:
[  421.997596]  0000000000000005 ffffffff813fcb3e 0000000000000000 0000000000000000
[  421.997596]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  421.997596]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  421.997596] Call Trace:
[  421.997596]  [<ffffffff813fcb3e>] ? __schedule+0x1ce/0x730
[  421.997596]  [<ffffffff8140020a>] ? entry_SYSCALL_64_fastpath+0x1e/0x8e
[  421.997596] Code: 83 c4 01 48 c7 c6 00 91 41 81 48 c7 c7 6b df 57 81 e8 48 3c ed ff 49 63 c4 48 39 c5 77 ab 48 89 df e8 e8 c1 f0 ff e9 95 2c f1 ff <0f> 0b 48 8b 35 3a 5c 3f 00 48 c7 c7  
[  421.997596] RIP  [<ffffffff8124f04d>] SyS_writev.cold.4+0xc8/0xca
[  421.997596]  RSP <ffff88013aebbe90>
[  422.020294] ---[ end trace 900084b06eee6e9e ]---
[  422.021356] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 26 (master)

@FrancisLeeHa
Copy link
Author

What a magic! It can run basic mvx after I try it again and again

  • on arm
popcorn@arm:~/mvx$ LD_PRELOAD=./loader.so ./basic 
Issue an MVX init syscall.
[  703.469623] 
[  703.469623] === HeterSec syscall ===
[  703.469755] Arg0: 0x1(1)
[  703.469879] [ 0] argv[0]: ./basic. arg_size 7
[  703.469953] [MVX] mvx_prepare_init_req: Retrieved argc 1, cmdline size 8, cmdline ./basic, exe path /home/popcorn/mvx/basic
[  703.470102] [MVX] mvx_server_start_mvx: [271] exe_path /home/popcorn/mvx/basic. arg_start 0x7fffffffeed9, arg_end 0x7fffffffeee1
[  703.470250] [MVX] mvx_server_start_mvx: size 2070; dst nid 0
[  703.470668] [MVX] MVX master variant [PID:271] init ...
=== [MVX Unit Test] Testing 2 syscalls: ===
[  703.470917] [MVX] 
[  703.470917] [ 0] syscall 66: <SYS_writev>
[  703.471039] [MVX] master waits for follower return on syscall 66
[  703.480244] [MVX] --> handle_mvx_reply: ret 0x2c(44), follower syscall 20
[  703.480478] [MVX] 
[  703.480478] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  703.480608] [MVX] SYSC_writev: buf: === [MVX Unit Test] Testing 2
[  703.480690] [MVX] SYSC_writev: buf:  syscalls: ===
writev, read

[  703.480910] [MVX] 
[  703.480910] [ 1] syscall 66: <SYS_writev>
[  703.481033] [MVX] master waits for follower return on syscall 66
[  703.488854] [MVX] --> handle_mvx_reply: ret 0xe(14), follower syscall 20
[  703.489027] [MVX] 
[  703.489027] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  703.489141] [MVX] SYSC_writev: buf: writev, read
[  703.489204] [MVX] SYSC_writev: buf: 
[  703.489204] 
Writting to stderr
[  703.491273] fd 2, count 20, ret 20
[  703.491370] [MVX] 
[  703.491370] [ 2] syscall 64: <SYS_write>
[  703.491513] [MVX] master waits for follower return on syscall 64
[  703.497649] [MVX] --> handle_mvx_reply: ret 0x14(20), follower syscall 20
[  703.497854] [MVX] SYSC_write: mvx [271] Writting to stderr
[  703.497854] 
[  703.497963] [MVX] Not support SYSC_write yet.
Input your name (type "quit" to exit): 
[  703.498176] [MVX] 
[  703.498176] [ 3] syscall 66: <SYS_writev>
[  703.498296] [MVX] master waits for follower return on syscall 66
[  703.498920] [MVX] --> handle_mvx_reply: ret 0x28(40), follower syscall 20
[  703.499064] [MVX] 
[  703.499064] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  703.499174] [MVX] SYSC_writev: buf: Input your name (type "quit" to exit): 
[  703.499265] [MVX] SYSC_writev: buf: 
test
[  705.966089] [MVX] 
[  705.966089] [ 4] syscall 63: <SYS_read>
[  705.966393] [MVX] master_read: fd 0, flag 2, len 5, retval 5.
[  705.966660] [MVX] master waits for follower return on syscall 63
[  705.978652] [MVX] --> handle_mvx_reply: ret 0x5(5), follower syscall 20
[  705.978995] [MVX post syscall] SYSC_read: ret 5
5 char(s) read, buf: test
[  705.979420] [MVX] 
[  705.979420] [ 5] syscall 66: <SYS_writev>
[  705.979638] [MVX] master waits for follower return on syscall 66
[  705.985690] [MVX] --> handle_mvx_reply: ret 0x1a(26), follower syscall 20
[  705.985970] [MVX] 
[  705.985970] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  705.986148] [MVX] SYSC_writev: buf: 5 char(s) read, buf: 
[  705.986252] [MVX] SYSC_writev: buf: test

[  705.986482] [MVX] 
[  705.986482] [ 6] syscall 66: <SYS_writev>
[  705.986645] [MVX] master waits for follower return on syscall 66
[  705.987503] [MVX] --> handle_mvx_reply: ret 0x1(1), follower syscall 20
[  705.987703] [MVX] 
[  705.987703] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  705.987853] [MVX] SYSC_writev: buf: 
[  705.987926] [MVX] SYSC_writev: buf: 
Input your name (type "quit" to exit): 
[  705.988159] [MVX] 
[  705.988159] [ 7] syscall 66: <SYS_writev>
[  705.988298] [MVX] master waits for follower return on syscall 66
[  705.988877] [MVX] --> handle_mvx_reply: ret 0x28(40), follower syscall 20
[  705.989071] [MVX] 
[  705.989071] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  705.989220] [MVX] SYSC_writev: buf: Input your name (type "quit" to exit): 
[  705.989535] [MVX] SYSC_writev: buf: 
haha[  709.093173] [MVX] 
[  709.093173] [ 8] syscall 63: <SYS_read>
[  709.093767] [MVX] master_read: fd 0, flag 2, len 5, retval 5.
[  709.094028] [MVX] master waits for follower return on syscall 63

[  709.107712] [MVX] --> handle_mvx_reply: ret 0x5(5), follower syscall 20
[  709.108136] [MVX post syscall] SYSC_read: ret 5
5 char(s) read, buf: haha
[  709.108657] [MVX] 
[  709.108657] [ 9] syscall 66: <SYS_writev>
[  709.108929] [MVX] master waits for follower return on syscall 66
[  709.115427] [MVX] --> handle_mvx_reply: ret 0x1a(26), follower syscall 20
[  709.115854] [MVX] 
[  709.115854] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  709.116140] [MVX] SYSC_writev: buf: 5 char(s) read, buf: 
[  709.116301] [MVX] SYSC_writev: buf: haha

[  709.116659] [MVX] 
[  709.116659] [10] syscall 66: <SYS_writev>
[  709.116917] [MVX] master waits for follower return on syscall 66
[  709.118215] [MVX] --> handle_mvx_reply: ret 0x1(1), follower syscall 20
[  709.118457] [MVX] 
[  709.118457] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  709.118606] [MVX] SYSC_writev: buf: 
[  709.118672] [MVX] SYSC_writev: buf: 
Input your name (type "quit" to exit): 
[  709.118925] [MVX] 
[  709.118925] [11] syscall 66: <SYS_writev>
[  709.119066] [MVX] master waits for follower return on syscall 66
[  709.120015] [MVX] --> handle_mvx_reply: ret 0x28(40), follower syscall 20
[  709.120268] [MVX] 
[  709.120268] SYSC_writev: mvx [271]. fd 1, vlen 2. MVX master.
[  709.120420] [MVX] SYSC_writev: buf: Input your name (type "quit" to exit): 
[  709.120529] [MVX] SYSC_writev: buf:
  • on x86
popcorn@x86:~/mvx$ [  779.711388] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 44 (master)
[  779.713785] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  779.715727] follower ret -9 != 44 from master
[  779.717734] [MVX] 
[  779.717734] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  779.719528] [MVX] 
[  779.720091] [MVX] Follower waits msg on
[  779.720091] [37] syscall: <SYS_writev>
[  779.721750] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 14 (master)
[  779.722992] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  779.724122] follower ret -9 != 14 from master
[  779.725067] [MVX] clone_mvx_thread: MVX helper thread [681] finished launching MVX process, bin path: /home/popcorn/mvx/basic.
[  779.726910] [MVX] 
[  779.726910] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  779.728407] [MVX] 
[  779.728715] [MVX] Follower waits msg on
[  779.728715] [38] syscall: <SYS_writev>
[  779.729567] [MVX] argv[0] ./basic (7)
[  779.730114] [MVX] MVX follower variant [PID:682] init ...
[  779.731659] [MVX] --> handle_mvx_message: syscall #1, flag 2, len 0, retval 20 (master)
[  779.732242] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[  779.732243] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 1
[  779.732244] follower ret -9 != 20 from master
[  779.732249] [MVX] 
[  779.732249] SYSC_writev: mvx [678]. fd 1, vlen 1. MVX follower.
[  779.732310] ------------[ cut here ]------------
[  779.732311] kernel BUG at fs/read_write.c:1007!
[  779.732317] invalid opcode: 0000 [#2] SMP 
[  779.732323] Modules linked in: msg_socket
[  779.732332] CPU: 1 PID: 678 Comm: basic Tainted: G      D         4.4.137-popcorn+ #3
[  779.732333] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
[  779.732345] task: ffff8800bb4f4b00 ti: ffff8800bb734000 task.ti: ffff8800bb734000
[  779.732386] RIP: 0010:[<ffffffff8124f04d>]  [<ffffffff8124f04d>] SyS_writev.cold.4+0xc8/0xca
[  779.732390] RSP: 0018:ffff8800bb737e90  EFLAGS: 00010202
[  779.732391] RAX: 000000000000007f RBX: ffff8800b94e6400 RCX: 0000000000000000
[  779.732391] RDX: 000000000000007f RSI: 0000000000000000 RDI: ffff8800bb737f1f
[  779.732392] RBP: 0000000000000001 R08: 0000000000000006 R09: ffff8800bb737ea0
[  779.732392] R10: ffffffff817d4d5c R11: 00000000000002c9 R12: 0000000000000000
[  779.732393] R13: 000000000000007f R14: 0000000000000000 R15: 0000000000000000
[  779.732394] FS:  00007ffff7ffe288(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
[  779.732394] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  779.732395] CR2: 0000000000000005 CR3: 00000000bac7c000 CR4: 0000000000360670
[  779.732398] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  779.732398] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  779.732399] Stack:
[  779.732400]  0000000000000014 ffff8800bb4f4b60 0000000000000000 0000000000000000
[  779.732401]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  779.732402]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  779.732407] Call Trace:
[  779.732431]  [<ffffffff8140020a>] ? entry_SYSCALL_64_fastpath+0x1e/0x8e
[  779.732448] Code: 83 c4 01 48 c7 c6 00 91 41 81 48 c7 c7 6b df 57 81 e8 48 3c ed ff 49 63 c4 48 39 c5 77 ab 48 89 df e8 e8 c1 f0 ff e9 95 2c f1 ff <0f> 0b 48 8b 35 3a 5c 3f 00 48 c7 c7  
[  779.732450] RIP  [<ffffffff8124f04d>] SyS_writev.cold.4+0xc8/0xca
[  779.732451]  RSP <ffff8800bb737e90>
[  779.733824] ---[ end trace 900084b06eee6e9f ]---
[  779.738344] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 40 (master)
[  779.738349] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  779.738349] follower ret -9 != 40 from master
[  779.738352] [MVX] 
[  779.738352] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  779.738391] [MVX] 
[  779.738391] [MVX] Follower waits msg on
[  779.738391] [39] syscall: <SYS_writev>
[  779.767430] [MVX] Cannot open STDIN file: -13.
[  779.767885] [MVX] [682] init_mvx_follower_process: Finished the Follower MVX process initialization.
[  779.769193] [MVX] 
[  779.769410] [MVX] Follower waits msg on
[  779.769410] [ 0] syscall: <SYS_writev>
[  779.769700] [MVX] mvx_process_fork: Finished MVX process setup. ret 0
[  782.207582] [MVX] --> handle_mvx_message: syscall #0, flag 2, len 5, retval 5 (master)
[  782.209919] [MVX] --> buf: test
[  782.209919] 
[  782.211279] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[  782.213400] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 0
[  782.215317] follower ret -9 != 5 from master
[  782.216293] [MVX] 
[  782.216293] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  782.217921] [MVX] 
[  782.218357] [MVX] Follower waits msg on
[  782.218357] [ 1] syscall: <SYS_writev>
[  782.220281] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 26 (master)
[  782.221882] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  782.223813] follower ret -9 != 26 from master
[  782.225106] [MVX] 
[  782.225106] SYSC_writev: mvx [682]. fd 1, vlen 2. MVX follower.
[  782.227123] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 1 (master)
[  782.227128] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  782.227128] follower ret -9 != 1 from master
[  782.227131] [MVX] 
[  782.227131] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  782.227243] [MVX] 
[  782.227243] [MVX] Follower waits msg on
[  782.227243] [ 2] syscall: <SYS_writev>
[  782.228595] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 40 (master)
[  782.228598] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  782.228599] follower ret -9 != 40 from master
[  782.228600] [MVX] 
[  782.228600] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  782.228623] [MVX] 
[  782.228643] [MVX] Follower waits msg on
[  782.228643] [ 3] syscall: <SYS_writev>
[  782.241296] [MVX] SYSC_writev: buf: === [MVX Unit Test] Testing 2
[  782.242036] [MVX] SYSC_writev: buf:  syscalls: ===
[  782.242669] [MVX] 
[  782.242931] [MVX] Follower waits msg on
[  782.242931] [ 4] syscall: <SYS_writev>
[  785.335182] [MVX] --> handle_mvx_message: syscall #0, flag 2, len 5, retval 5 (master)
[  785.337514] [MVX] --> buf: haha
[  785.337514] 
[  785.338886] [**MVX Violation**] kernel/popcorn/mvx_engine.c:225 follower_writev
[  785.341006] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 0
[  785.342947] follower ret -9 != 5 from master
[  785.344809] [MVX] 
[  785.344809] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  785.347299] [MVX] 
[  785.347607] [MVX] Follower waits msg on
[  785.347607] [ 5] syscall: <SYS_writev>
[  785.350055] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 26 (master)
[  785.351163] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  785.353500] follower ret -9 != 26 from master
[  785.355014] [MVX] 
[  785.355014] SYSC_writev: mvx [682]. fd 1, vlen 2. MVX follower.
[  785.357740] [MVX] SYSC_writev: buf: g 2[  785.358102] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 1 (master)
[  785.358107] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  785.358108] follower ret -9 != 1 from master
[  785.358112] [MVX] 
[  785.358112] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  785.358221] [MVX] 
[  785.358222] [MVX] Follower waits msg on
[  785.358222] [ 6] syscall: <SYS_writev>
[  785.359913] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 40 (master)
[  785.359918] [MVX] syscall 20: <SYS_writev>. flag 2. master msg syscall 20
[  785.359918] follower ret -9 != 40 from master
[  785.359921] [MVX] 
[  785.359921] SYSC_writev: mvx [680]. fd 1, vlen 0. MVX follower.
[  785.360055] [MVX] 
[  785.360056] [MVX] Follower waits msg on
[  785.360056] [ 7] syscall: <SYS_writev>

[  785.371539] [MVX] SYSC_writev: buf:  syscalls: ===
[  785.372105] [MVX] 
[  785.372342] [MVX] Follower waits msg on
[  785.372342] [ 8] syscall: <SYS_writev>

But it still have kernel bug on x86
[ 779.732311] kernel BUG at fs/read_write.c:1007!
What`s problem?

@xjtuwxg
Copy link
Member

xjtuwxg commented Aug 7, 2020

Hey, thanks for the log. This is actually a bug/enhancement. The problem is caused by the hardcoded pseudo STDIN/STDOUT file locations in kernel/popcorn/mvx_server.c:

 35 /* Pseudo file locations for follower fd 0, 1 and 2. */                         
 36 #define MVX_STDIN_LOC           "/dev/pts/0"
 37 #define MVX_STDOUT_LOC          "/dev/pts/0"

So, if you use ssh to connect to the VMs, and do it on the ssh client it should work.

popcorn@x86:~/mvx$ [ 409.701426] [MVX] clone_mvx_thread: MVX helper thread [675] finished launching MVX process, bin path: /home/popcorn/mvx/basic.
[ 409.704798] [MVX] argv[0] ./basic (7)
[ 409.706156] [MVX] MVX follower variant [PID:676] init ...
[ 409.708168] [MVX] --> handle_mvx_message: syscall #20, flag 2, len 0, retval 44 (master)
[ 409.711781] [MVX] Cannot open STDIN file: -13.

@xjtuwxg xjtuwxg added bug Something isn't working enhancement New feature or request labels Aug 7, 2020
@xjtuwxg xjtuwxg self-assigned this Aug 7, 2020
@FrancisLeeHa
Copy link
Author

Thanks!
When I use the command create the VMs, I just get into the terminals that I can`t exit (if quit by force command, VMs will be shutdown too). The TTY is ttyS0, which is not pts/0

popcorn@x86:~$ ps
  PID TTY          TIME CMD
  618 ttyS0    00:00:00 bash
  663 ttyS0    00:00:00 ps

I just keep this two terminal and use ssh connect the VMs, now the tty change to pts/0

popcorn@arm:~/mvx$ ps
  PID TTY          TIME CMD
  232 pts/0    00:00:00 bash
  252 pts/0    00:00:00 ps

And the mvx demo run correctly.

Why can`t I inter pts/0 TTY just when I create the VMs?
And how can I create the VMs running on background? So that I will ssh to login instead of using this ttyS0 TTY

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xjtuwxg xjtuwxg

Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xjtuwxg @FrancisLeeHa