Model Performance
Last updated
Was this helpful?
Last updated
Was this helpful?
System Capabilities Model performance is influenced by your machine’s hardware and specifications. Make sure your device meets the for optimal performance.
Model Selection
The complexity of the model you choose can have a significant impact on performance. For guidance on choosing the best model for your needs, see
Model Configuration Fine-tuning the model’s settings can have a noticeable impact on performance as well. For a full guide on configuration options, visit our Advanced Model Settings page.
For a better understanding of how well the model is performing, have a look at the statistics below the main input on the chat page.
Memory
The Memory usage section in the sidebar displays how much memory your operating system and running applications are currently using. This serves as a helpful indicator of AI model performance on your device.
GPU
The GPU (Graphics Processing Unit) is used to accelerate the model's processing, especially for tasks involving complex computations. A more powerful GPU can improve the model's speed and responsiveness.
TPS
TPS (Tokens per Second) measures how fast the model processes text. It shows the number of tokens generated per second based on task complexity and hardware. A TPS above 1 indicates good performance, while below 1 is slow.
Context size
Context size measures how much text the language model considers when processing questions and generating responses. It determines the amount of conversation history the model can use. When the context reaches its limit, older messages are automatically removed to make space for new ones.
