LM Studio is user -friendly and available in binary format for Windows and Mac (Linux version is being developed). It supports many models compatible with the GGML Tensor Library from the LLAMA.CPP project and requires 16GB of RAM.
Let's explore how easy and easy to use LM Studio!
What is LM Studio?
LM Studio is a multi -platform Desktop application that allows you to search, create and test many different LLMs. This application provides user -friendly interface and many features to serve both beginners and advanced users.
The main features of LM Studio
- User interface:
- LM Studio presented the same interface as chatgpt, making it easy to interact with different models.
- The interface includes options to discover models, load them localities and interact with them in conversation.
- Discover and choose the model:
- The “Discover” part allows you to discover many different llm from the Hugging Face storage.
- You can search for models based on specific criteria and view their details, including the number of parameters, architecture and author.
- Load and model testing:
- After selecting a model, you can download and load the local model.
- The application provides conveniently to switch between different models and test their capabilities.
- System requirements and customization:
- You can determine the system requirements to affect the output of the model.
- Customize other parameters such as the length of the response, the stop chain, etc.
- Model Runtime:
- LM Studio supports many different runtime can improve the performance of the models.
- The application provides compatible Runtime information, allowing you to download and install them.
- Developer regime:
- For advanced users, developer mode provides additional features and installation.
- Discover options such as server configuration, API endpoint and diary.
Use models from the chat panel
After installation, LM Studio facilitates the download of models from Hugging Face Hub, including preset options.
For example, we can download the Zephyr 7B β model, which is adjusted to the GGUF format of LLAMA.CPP.
Activating and loading the model into LM Studio is very simple.
After that, you can start using the model right from the chat control panel, without an internet connection.
The right control panel is displayed and allows to modify the default preset for the model. Use of memory and useful deduction data is displayed in the title of the window and below the chat control panel.
Other models, such as Codellama Instruct 7B, are also available to download and use.
LM Studio also highlights new models and versions from Hugging Face, turning it into an invaluable tool to explore and test the latest releases.
Access models by API
A remarkable feature of LM Studio is the ability to create Local Inference Servers with just one click.
Optional Automatic Prompt Formatting Simplify Prompt construction to match the model format of the model. API is displayed in accordance with Openai format.
The following is an example of how to call the end point by curl:
curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{ "role": "system", "content": "You are an AI assistant answering Tech questions" },
{ "role": "user", "content": "What is Java?" }
],
"temperature": 0.7,
"max_tokens": -1,
"stream": false
}'Response to provide required information:
{
"id": "chatcmpl-iyvpdtqs1qzlv6jqkmdt9",
"object": "chat.completion",
"created": 1699806651,
"model": "~/.cache/lm-studio/models/TheBloke/zephyr-7B-beta-GGUF/zephyr-7b-beta.Q4_K_S.gguf",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Java is a high-level, object-oriented
programming language that was first released by Sun
Microsystems in 1995. It is now owned by Oracle Corporation.
Java is designed to be platform independent, meaning that it
can run on any operating system that has a Java Virtual
Machine (JVM) installed. Java's primary applications are in
the development of desktop applications, web applications,
and mobile apps using frameworks such as Android Studio,
Spring Boot, and Apache Struts. Its syntax is similar to
C++, but with added features for object-oriented programming
and memory management that make it easier to learn and use
than C++. Java's popularity is due in part to its extensive
library of pre-written code (known as the Java Class
Library) which makes development faster and more efficient."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 0,
"completion_tokens": 166,
"total_tokens": 166
}
}This feature supports a lot in integrated testing with frontend such as chatbot or working process solutions like Flows.
Although not an open source, LM Studio is a powerful addition to your local toolkit, allowing you to easily test and apply models from Hugging Face. User -friendly interface and diverse features make it an essential resource for anyone who wants to explore the world of large language models.
