Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log all config files for reference #206

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

narasimhan-v
Copy link
Collaborator

This commit:

  • Creates a folder with timestamp for results
  • Creates a file with command line used to trigger the wrapper
  • Logs all test configs
  • Logs input file and no_run configs
  • Logs wrapper file to that folder
  • Pushes job results to that folder

Signed-off-by: Narasimhan V [email protected]

@narasimhan-v
Copy link
Collaborator Author

# ./avocado-setup.py --run-suite host_io_fc_fvt 
20:10:14 INFO    : Check for environment
20:10:15 INFO    : 
20:10:15 INFO    : Running Host Tests Suite io_fc_fvt_fs_mark_fs_mark
20:10:15 INFO    : Running: /usr/local/bin/avocado run avocado-misc-tests/io/disk/fs_mark.py -m /root/sim/tests/tests/avocado-misc-tests/io/disk/fs_mark.py.data/fs_mark.yaml --force-job-id f1ca33262941356a47ef238c16266e18c7780fc6  --job-results-dir /root/sim/tests/26-4-2020_20_10_14/results
/usr/local/lib/python3.6/site-packages/avocado_framework-78.0-py3.6.egg/avocado/plugins/run.py:287: FutureWarning: The following arguments will be changed to boolean soon: sysinfo, output-check, failfast, keep-tmp and ignore-missing-references.
  FutureWarning)
Error running method "pre_tests" of plugin "fetchasset": 'Attribute' object has no attribute 'id'
JOB ID     : f1ca33262941356a47ef238c16266e18c7780fc6
JOB LOG    : /root/sim/tests/26-4-2020_20_10_14/results/job-2020-04-26T20.10-f1ca332/job.log
 (1/3) avocado-misc-tests/io/disk/fs_mark.py:FSMark.test;dir-disk-ext4-1d3d: ERROR: 'FSMark' object has no attribute 'link' (1.70 s)
 (2/3) avocado-misc-tests/io/disk/fs_mark.py:FSMark.test;dir-disk-xfs-8b3d: ERROR: 'FSMark' object has no attribute 'link' (1.32 s)
 (3/3) avocado-misc-tests/io/disk/fs_mark.py:FSMark.test;dir-disk-btrfs-209e: CANCEL: btrfs is not supported with                                 RHEL 7.4 onwards (0.99 s)
RESULTS    : PASS 0 | ERROR 2 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1
JOB TIME   : 4.42 s
JOB HTML   : /root/sim/tests/26-4-2020_20_10_14/results/job-2020-04-26T20.10-f1ca332/results.html
20:10:21 INFO    : 
20:10:21 INFO    : 
20:10:21 INFO    : Running Host Tests Suite io_fc_fvt_fs_mark_fs_mark.2
20:10:21 INFO    : Running: /usr/local/bin/avocado run avocado-misc-tests/io/disk/fs_mark.py -m /root/sim/tests/tests/avocado-misc-tests/io/disk/fs_mark.py.data/fs_mark.yaml --force-job-id c1d04f0b8f914cd272fcd55131cbb8d844669287  --job-results-dir /root/sim/tests/26-4-2020_20_10_14/results
/usr/local/lib/python3.6/site-packages/avocado_framework-78.0-py3.6.egg/avocado/plugins/run.py:287: FutureWarning: The following arguments will be changed to boolean soon: sysinfo, output-check, failfast, keep-tmp and ignore-missing-references.
  FutureWarning)
Error running method "pre_tests" of plugin "fetchasset": 'Attribute' object has no attribute 'id'
JOB ID     : c1d04f0b8f914cd272fcd55131cbb8d844669287
JOB LOG    : /root/sim/tests/26-4-2020_20_10_14/results/job-2020-04-26T20.10-c1d04f0/job.log
 (1/3) avocado-misc-tests/io/disk/fs_mark.py:FSMark.test;dir-disk-ext4-1d3d: ERROR: 'FSMark' object has no attribute 'link' (1.31 s)
 (2/3) avocado-misc-tests/io/disk/fs_mark.py:FSMark.test;dir-disk-xfs-8b3d: ERROR: 'FSMark' object has no attribute 'link' (1.31 s)
 (3/3) avocado-misc-tests/io/disk/fs_mark.py:FSMark.test;dir-disk-btrfs-209e: CANCEL: btrfs is not supported with                                 RHEL 7.4 onwards (0.79 s)
RESULTS    : PASS 0 | ERROR 2 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1
JOB TIME   : 3.79 s
JOB HTML   : /root/sim/tests/26-4-2020_20_10_14/results/job-2020-04-26T20.10-c1d04f0/results.html
20:10:25 INFO    : 
20:10:25 INFO    : Summary of test results can be found below:
TestSuite                                                                   TestRun    Summary             
 
host_io_fc_fvt_fs_mark_fs_mark                                              Run        Successfully executed
/root/sim/tests/26-4-2020_20_10_14/results/job-2020-04-26T20.10-f1ca332/job.log
|  total 3  ||  pass 0  ||  cancel 1  ||  errors 2  ||  failures 0  ||  skip 0  |

 
host_io_fc_fvt_fs_mark_fs_mark.2                                            Run        Successfully executed
/root/sim/tests/26-4-2020_20_10_14/results/job-2020-04-26T20.10-c1d04f0/job.log
|  total 3  ||  pass 0  ||  cancel 1  ||  errors 2  ||  failures 0  ||  skip 0  |

20:10:25 INFO    : Results and Configs logged at: /root/sim/tests/26-4-2020_20_10_14
20:10:25 INFO    : Removing temporary mux dir

Truncated tree structure for idea

# tree /root/sim/tests/26-4-2020_20_10_14
/root/sim/tests/26-4-2020_20_10_14
├── avocado-wrapper.log
├── command.txt
├── no_run_tests
│   └── no_run_tests.conf
├── results
│   ├── job-2020-04-26T20.10-c1d04f0
│   ├── job-2020-04-26T20.10-f1ca332
│   └── latest -> job-2020-04-26T20.10-c1d04f0
└── test_configs
    └── io_fc_fvt.cfg

@narasimhan-v
Copy link
Collaborator Author

narasimhan-v commented Apr 26, 2020

@sathnaga Please paste results from guest run.

@lgtm-com
Copy link

lgtm-com bot commented Apr 26, 2020

This pull request introduces 1 alert and fixes 2 when merging dfe837c into d14a603 - view on LGTM.com

new alerts:

  • 1 for Syntax error

fixed alerts:

  • 1 for Unused local variable
  • 1 for Module is imported with 'import' and 'import from'

@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch from dfe837c to 50fcf02 Compare April 26, 2020 16:18
@lgtm-com
Copy link

lgtm-com bot commented Apr 26, 2020

This pull request introduces 1 alert when merging 50fcf02 into d14a603 - view on LGTM.com

new alerts:

  • 1 for Unused import

@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch from 50fcf02 to 96f72e1 Compare April 26, 2020 20:31
@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch from 96f72e1 to 278d2f8 Compare April 27, 2020 05:49
@sathnaga
Copy link
Collaborator

@narasimhan-v Thanks for the change, this is a much needed for debugging and useful while archives test results for longer reference.

Yet to review the complete change, one concern by glancing I have is that you are trying to change
the current result folder path, which would break the all the assumptions we have in all other places.
we expect, job-* to be present in the user given outputdir, or results dir in base path, with this we introduce an another child folder, we need to re-think on that, in general log file,config selection am fine with, and one more thing, we need think of multiple config files in single cmdline, I guess the current implementation will support that, just incase if you want to re-check.

@narasimhan-v
Copy link
Collaborator Author

which would break the all the assumptions we have in all other places.
we expect, job-* to be present in the user given outputdir, or results dir in base path,

Previous:
<output-dir>/results/job-*

With this commit:
<output-dir>/<timestamped-folder>/results/job-*

Would need suggestions on this, because parsing of this new folder structure is not going to be easier, especially in an automated result parsing environment.

and one more thing, we need think of multiple config files in single cmdline, I guess the current implementation will support that, just incase if you want to re-check.

That is already verified for hosts. Would need your help in verifying for guest cfg, and let me know if there are more cfg files needed to be logged for guests.

@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch 2 times, most recently from 34a5b23 to 55381c6 Compare April 27, 2020 13:57
@sathnaga
Copy link
Collaborator

sathnaga commented May 6, 2020

@narasimhan-v pls rebase

@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch from 55381c6 to 9ae61f6 Compare May 6, 2020 07:46
@narasimhan-v
Copy link
Collaborator Author

@sathnaga done

@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch 2 times, most recently from 389a3f0 to ac08b57 Compare May 6, 2020 07:51
@lgtm-com
Copy link

lgtm-com bot commented May 6, 2020

This pull request introduces 2 alerts when merging ac08b57 into 8225de2 - view on LGTM.com

new alerts:

  • 2 for Variable defined multiple times

@narasimhan-v narasimhan-v force-pushed the logging_configs_results branch from ac08b57 to 5720b82 Compare May 6, 2020 08:39
This commit:
* Creates a folder with timestamp for logs
* Creates a file with command line used to trigger the wrapper
* Logs all test configs
* Logs input file and no_run configs
* Logs wrapper log to that folder

Signed-off-by: Narasimhan V <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants