diff --git a/README.md b/README.md index 9ad0db14..619d2b61 100644 --- a/README.md +++ b/README.md @@ -9,20 +9,27 @@ for the US Core Implementation Guide [v7.0.0](https://hl7.org/fhir/us/core/STU7/). This test kit provides testing for US Core Servers. -Visit the [US Core Test Kit Manual](https://github.com/inferno-framework/us-core-test-kit/wiki) for more information on using this test kit. +Visit the [US Core Test Kit +Documentation](https://github.com/inferno-framework/us-core-test-kit/wiki) for +more information on using this test kit. -## Instructions +## Getting Started -It is highly recommended that you use [Docker](https://www.docker.com/) to run -these tests. This test kit requires at least 10 GB of memory are available to Docker. +The Inferno on HealthIT.gov platform hosts a [public +instance](https://inferno.healthit.gov/test-kits/us-core/) of this test +kit that developers and testers are welcome to use. However, users are +encouraged to download and run this tool locally. -- Clone this repo. -- Run `setup.sh` in this repo. -- Run `run.sh` in this repo. -- Navigate to `http://localhost`. The US Core test suite will be available. +The quickest way to run this test kit locally is with [Docker](https://www.docker.com/). -See the [Inferno Documentation](https://inferno-framework.github.io/docs/) -for more information on running Inferno. +- Install Docker +- Clone this repository, or download an [official release](/releases). +- Run `./setup.sh` within the test kit directory to download necessary dependencies +- Run `./run.sh` within the test kit directory to start the application +- Navigate to `http://localhost` + +See the [Inferno Documentation](https://inferno-framework.github.io/docs/getting-started-users.html#running-an-existing-test-kit) +for more information on running Inferno locally. ## License diff --git a/docs/Overview.md b/docs/Overview.md index 01d53906..0401836c 100644 --- a/docs/Overview.md +++ b/docs/Overview.md @@ -1,4 +1,3 @@ -# US Core Test Kit Overview The US Core Test Kit provides server testing for several versions of US Core. Tests are designed to simulate a realistic client that would like to access all @@ -15,7 +14,7 @@ certification criterion, that may reference US Core. This guide is targeting users who would like to run the US Core Test Kit without being the context of another Implementation Guide or certification criterion. -# Testing Options +## Testing Options Users of the test kit are presented with two options: @@ -29,13 +28,13 @@ Please note that US Core states that at least version SMART App Launch v2.0 is used in US Core 7; however, these tests allow testers to select SMART App Launch v1 for any US Core version. -# Test Prerequisites +## Test Prerequisites Systems do not need to load a specific set of data to run these tests. However, the tests do require that the system under test has a set of data that is representative of the data that the system will be handling. -# Conformance Criteria +## Conformance Criteria Systems do NOT need to pass all tests within this Test Kit to be considered 'US Core Conformant'. US Core allows systems to support a subset of content @@ -45,13 +44,13 @@ relevant to a given system. Tests are divided in a logical way that should alig with common boundaries associated with real-world systems; e.g. you may run tests relevant to a certain profile while skipping tests for other profiles. -# Test Groups +## Test Groups This test kit is organized into the following high-level test groups. The system under test can choose which test groups are relevant to their system. -## 1 SMART App Launch +### 1 SMART App Launch This group provides two tests: an EHR launch test and a standalone launch test, based on the SMART App Launch guide. When used, a bearer token is stored @@ -59,7 +58,7 @@ that can be used in the US Core FHIR API test section. Read the description provided in each test with instructions on their use. -## 2 US Core FHIR API +### 2 US Core FHIR API This group contains the majority of the US Core tests that are applicable to the US Core Implementation Guide. Users may either run all tests, or select a @@ -76,7 +75,7 @@ directly reference a patient ID (e.g.; Organization, Practitioner) should not be run individually. They can only be run when the entire group US Core FHIR API group is run. -## 3 US Core SMART Granular Scopes (US Core v6+, SMART App Launch v2+) +### 3 US Core SMART Granular Scopes (US Core v6+, SMART App Launch v2+) This group tests the specific implementation details surrounding use of granular SMART App Launch scopes within US Core. \ No newline at end of file diff --git a/docs/README.md b/docs/README.md index 4a1148c1..18b8ca57 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,31 +1,23 @@ -# US Core Test Kit Manual - -The US Core Test Kit is an automated testing tool for servers -implementing the US Core FHIR Implementation Guide. It contains server -tests for v3.1.1, v4.0.0, v5.0.1, v6.1.0, and v7.0.0 of US Core. - -> *To update this manual, please update the associated files within the -`/docs/` directory of this git repository. Manual updates to this wiki will be discarded -by GitHub on each push to the `main` branch.* - -# Installation - -Information on downloading and running this application, including prerequisites -and system requirements, are provided in the repository [README](../). - -# Using this Test Kit -* [Test Kit Overview](Overview): An overview of the Test Kit and its tests. - -# Maintaining this Test Kit -Developers contributing to this Test Kit should be familiar with authoring -Inferno Framework test suites. The following guides provide additional -information about the design and implementation of this Test Kit to aid +The **US Core Test Kit** is an automated conformance testing tool for servers +implementing the US Core FHIR Implementation Guide. It contains server tests for +several versions of US Core, including v3.1.1, v4.0.0, v5.0.1, v6.1.0, and +v7.0.0 of US Core. The following documentation describes how to use and +contribute to this test kit. + +## Using this Test Kit +* [Getting Started](https://github.com/inferno-framework/us-core-test-kit/?tab=readme-ov-file#getting-started) +* [Test Kit Overview](Overview.md): An overview of the Test Kit and its tests. + +## Contributing this Test Kit +Developers contributing to this test kit should be familiar with [authoring +Inferno Framework test suites](https://inferno-framework.github.io/docs/writing-tests/). The following guides provide additional +information about the design and implementation of this test kit to aid contributing to these tests: -* [Technical Overview](Technical-Overview): An overview of the technical design of this Test Kit. -* [Standards Update Guide](Version-Update-Guide): A step by step guide on updating this test kit to support new versions of US Core. +* [Technical Overview](Technical-Overview.md): An overview of the technical design of this Test Kit. +* [Version Update Guide](Version-Update-Guide.md): A step by step guide on updating this test kit to support new versions of US Core. -# Support +## Support For questions or issues with this Test Kit, please reach out to the Inferno team on the [#Inferno FHIR Zulip diff --git a/docs/Technical-Overview.md b/docs/Technical-Overview.md index 562b973b..aa759a74 100644 --- a/docs/Technical-Overview.md +++ b/docs/Technical-Overview.md @@ -1,14 +1,13 @@ -# Background - -This guide orients developers to the technical design of this Test Kit. While -the Inferno Framework has documentation on how to create tests, additional -documentation on the organization of this test kit is necessary due to the -complexity involved in developing tests for the US Core guide. +This technical overview orients developers to the technical design of this Test +Kit. While the [Inferno Framework has +documentation](https://inferno-framework.github.io) on how to create tests, +additional documentation on the organization of this test kit is necessary due +to the complexity involved in developing tests for the US Core guide. Prior to reviewing this document, the developer is expected to be familiar -with the US Core Test Kit and building tests with the Inferno Framework. +with the US Core Test Kit and building tests with the [Inferno Framework](https://inferno-framework.github.io). -# Test Design Principles and Features +## Test Design Principles and Features Tests for this Test Kit have been designed with the following principles: @@ -46,7 +45,7 @@ of Ruby development before attempting to alter these tests. This Test Kit also uses RSpec to "unit test" components of these tests, and developers are expected to learn the basics of RSpec as well. -# US Core Test Kit Project Source Code Structure: +## US Core Test Kit Project Source Code Structure: Below is a description of the source code structure for the US Core Test Kit: @@ -78,9 +77,9 @@ Importantly, code within the `generated` folder is generated using the generation script and should *not* be manually edited. Instead, changes to the generator should be made in the `generator` folder. -# Related Systems and Dependencies +## Related Systems and Dependencies -## Reference Server +### Reference Server The Inferno Reference Server serves as a reference implementation server, supporting the US Core Implementation Guide, SMART App Launch, and Bulk Data @@ -94,7 +93,7 @@ Inferno Reference Server to ensure it conforms to the latest Implementation Guide. For detailed instructions on how to load data into the server database, please refer to the README.md file on the GitHub repository. -## SMART App Launch Test Kit +### SMART App Launch Test Kit The Inferno US Core Test Kit incorporates a test group from the SMART App Launch Test Kit. The purpose of the SMART App Launch Test Kit is to confirm a server's @@ -108,14 +107,14 @@ in response to any potential ambiguity that results from the combination of FHIR, US Core and SMART App Launch, particularly when relevant to certification activities in the US. -# Testing Code Changes +## Testing Code Changes This test kit includes comprehensive "self testing" functionality to provide confidence that the tests perform as expected. Prior to committing changes to this test kit, developers should ensure that both RSpec tests and End-to-End tests pass. -## RSpec Tests +### RSpec Tests The test kit contains many "unit" tests within the `spec` directory. These tests are written in RSpec, and can be run with the following command: @@ -128,7 +127,7 @@ that the code base achieves 100% test coverage; instead, the team has followed a common sense approach to testing components that 1) are complicated or 2) are likely to change. -## End-to-End testing +### End-to-End testing Besides the unit tests provided within this Test Kit, after each update the tests should be validated against a complete server implementation @@ -137,10 +136,10 @@ and contains data that passes all of these tests. Note that if test changes have been made that require data on the Reference Server to change as well, that data set needs to be updated. -# Updating the Test Kit +## Updating the Test Kit **To update this test kit in response to a new version of the US Core Implementation -Guide, please see the [Version Update Guide](Version-Update-Guide). The Version +Guide, please see the [Version Update Guide](Version-Update-Guide.md). The Version Update Guide provides a detailed walkthrough of the components of the Test Kit that are changed during this process.** @@ -151,7 +150,7 @@ certification activities for implementers, or even worse, encourage improper implementations just to make incorrect tests pass. Therefore, the tests should be updated to include the test fixes as soon as possible. -## Updating Generating Tests +### Updating Generating Tests The main testing logic for these tests are generated from machine-readable data provided by each version of the US Core Implementation Guide. The code for @@ -169,7 +168,7 @@ To regenerate the files, run: Tests will be created for each version of the implementation guide provided in the `lib/us_core_test_kit/igs` directory. -## Updating shared component tests +### Updating shared component tests While the generator creates code for each suite, the generated code typically references shared component tests. These component tests reside in the @@ -181,7 +180,7 @@ Tests that are version specific reside in the `lib/us_core_test_kit/custom_groups` directory. -## Overriding US Core IG Machine-readable Content +### Overriding US Core IG Machine-readable Content Occasionally, the US Core authors publish incorrect content within the implementation guide that affects the generation of tests, or validation of @@ -204,7 +203,7 @@ in: Please see those files for examples of how to modify the behavior of the generator based on issues in the published implementation guide. -## Overriding HL7 Validator Behavior +### Overriding HL7 Validator Behavior The HL7 Validator is used to validate resources at runtime. Occasionally, the HL7 Validator will report validation errors that are considered to be out of @@ -217,7 +216,7 @@ only applies to a certain version of US Core, you can update the `lib/us_core_test_kit/generator/suite_generator.rb` file by adding a version-specific exception. -# Unusual Implementation Details +## Unusual Implementation Details While code in this Test Kit is intended to be as simple and as easy-to-understand as possible, sometimes unanticipated testing requirements are introduced that diff --git a/docs/Version-Update-Guide.md b/docs/Version-Update-Guide.md index 4a4f4808..f7681653 100644 --- a/docs/Version-Update-Guide.md +++ b/docs/Version-Update-Guide.md @@ -1,5 +1,3 @@ -# Background - This guide walks developers through the process of creating tests for new versions of the US Core Implementation Guide. While this test kit provides tools to support the creation of these tests based on machine-readable content @@ -10,13 +8,13 @@ be automated. Prior to reviewing this document, the developer is expected to be familiar with using the US Core Test Kit and building test kits using the Inferno Framework. -Please review the [Technical Overview](Technical-Overview) document for a +Please review the [Technical Overview](Technical-Overview.md) document for a high-level overview of the test kit's design. This document should be updated after each new version of the US Core Implementation guide is incorporated into the Test Kit. -# Versions of US Core Test Kit to Support +## Supporting US Core Versions This test kit should be updated to provide testing for all versions of the US Core Implementation Guide that is approved for certification within the @@ -41,7 +39,7 @@ release. A general convention that this test kit has taken is to name suites that refer to a ballot using "-ballot" appended to the end of the suite ID. Once the final version is released, the ballot version is deleted. -# How to Update the Test Kit +## Updating the Test Kit Tests for a new version of the US Core Implementation Guide can be created for this Test Kit using the following steps: diff --git a/docs/_Footer.md b/docs/_Footer.md new file mode 100644 index 00000000..2825697e --- /dev/null +++ b/docs/_Footer.md @@ -0,0 +1,3 @@ +_Test Kit documentation is stored within the `/docs` folder of this git repository +and is automatically copied to the wiki with each update to the `main` branch. +Do not change content within the wiki directly as changes will be overwritten._ \ No newline at end of file diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md new file mode 100644 index 00000000..8218c1f0 --- /dev/null +++ b/docs/_Sidebar.md @@ -0,0 +1,9 @@ +**[Documentation Home](Home)** + +**Using this Test Kit** + - [Getting Started](https://github.com/inferno-framework/us-core-test-kit/?tab=readme-ov-file#getting-started) + - [Test Kit Overview](Overview.md) + +**Contributing to this Test Kit** + - [Technical Overview](Technical-Overview) + - [Version Update Guide](SVAP-Update-Guide) \ No newline at end of file