Use lazy file lists in AppTree

[presidentbeef]

This makes finding files a little faster. On large apps, I've observed improvement around 9%.

[dryrunsecurity]

DryRun Security Summary

The pull request improves the serialization and deserialization of Brakeman security scanner objects, focusing on the Tracker and AppTree classes to enhance their handling of file paths, symlinks, and overall marshalling capabilities.

Expand for full summary

Summary:

The code changes in this pull request focus on improving the serialization and deserialization (marshalling) of various objects used in the Brakeman security scanner for Ruby on Rails applications. The changes primarily target the Brakeman::Tracker, Brakeman::AppTree, and related classes, ensuring that these objects can be properly serialized and deserialized for storage, transportation, or caching purposes.

While the changes do not directly introduce any obvious security vulnerabilities, it's important to review the usage of these serialized objects to ensure that no sensitive information is exposed and that the serialization process is implemented securely. Improper serialization and deserialization can sometimes lead to security issues, such as deserialization attacks, if not handled correctly.

Additionally, the changes to the Brakeman::AppTree class, which is responsible for finding and handling Ruby files in an application's directory structure, are focused on improving the handling of symlinks and the inclusion/exclusion of certain directories (such as vendor). These changes can indirectly benefit the security analysis performed by Brakeman by allowing it to process larger projects more efficiently and accurately.

Files Changed:

1. lib/brakeman/tracker.rb: The changes add a new marshallable method to the Brakeman::Tracker class, ensuring that the Tracker object can be properly serialized and deserialized.
2. test/test.rb: The changes update the Brakeman::Rescanner class to use the marshallable method when creating a deep copy of the @original object, ensuring that the copy is fully serializable.
3. lib/brakeman/app_tree.rb: The changes add the marshallable method to the Brakeman::AppTree class, converting certain instance variables to arrays to make them serializable. The glob_files method is also optimized for performance and updated to handle symlinks.
4. test/tests/app_tree.rb: The changes add support for handling symlinks and controlling the inclusion/exclusion of the vendor directory when searching for Ruby files. The changes also introduce the engine_paths and additional_libs_path options to include files from custom locations.

Code Analysis

We ran 9 analyzers against 4 files and 0 analyzers had findings. 9 analyzers had no findings.

View PR in the DryRun Dashboard.