⚡Model Performance

Key Factors Affecting Performance

1. System Capabilities Model performance is influenced by your machine’s hardware and specifications. Make sure your device meets the minimum system requirements for optimal performance.

2. Model Selection

   The complexity of the model you choose can have a significant impact on performance. For guidance on choosing the best model for your needs, see How to choose and download a model?

3. Model Configuration Fine-tuning the model’s settings can have a noticeable impact on performance as well. For a full guide on configuration options, visit our Advanced Model Settings page.

For a better understanding of how well the model is performing, have a look at the statistics below the main input on the chat page.

Parameter	Description
Memory	The Memory usage section in the sidebar displays how much memory your operating system and running applications are currently using. This serves as a helpful indicator of AI model performance on your device.
GPU	The GPU (Graphics Processing Unit) is used to accelerate the model's processing, especially for tasks involving complex computations. A more powerful GPU can improve the model's speed and responsiveness.
TPS	TPS (Tokens per Second) measures how fast the model processes text. It shows the number of tokens generated per second based on task complexity and hardware. A TPS above 1 indicates good performance, while below 1 is slow.
Context size	Context size measures how much text the language model considers when processing questions and generating responses. It determines the amount of conversation history the model can use. When the context reaches its limit, older messages are automatically removed to make space for new ones.