Speed.AI - LLM Edition is an Android app designed to benchmark Large Language Models (LLMs). It measures key performance indicators such as tokens per second and CPU/GPU/RAM usage. The benchmarks are conducted using a standardized dataset for all models, ensuring consistent input conditions. Additionally, the app allows for custom conversations with any supported model, providing benchmarking results afterward.
This app is built on the MLC LLM Android app. Therefore, the instructions and steps for building and running the app are identical to those of the MLC LLM app. To get started, follow this documentation.
This text is clear but can be made a bit smoother and more precise. Here's an improved version:
In the Android project, set the backend address in local.properties under API_ADDRESS and assign a base64-encoded 32-byte (AES-256) value to API_KEY. Ensure that this key matches the one used in the backend.
Here are some screenshots of the app in action:
