Connect with us

Hi, what are you looking for?

Tech

We can also run chatbots natively on new AMD hardware!

We can also run chatbots natively on new AMD hardware!

Technology News – Following Nvidia's conversation with RTX comes AMD's GPT-based compiled chatbot powered by a large language model (LLM) that requires relatively modern hardware.

In February, we also reported on Chat With RTX, which can be run locally on Nvidia GeForce RTX 4000 and 3000 cards with the help of TensorRT-LLM, producing a local data packet. Now we get something similar from AMD, but it can run not only on their video cards. If you have an AI-capable CPU from AMD, you won't have to buy one of the latest Radeon 7000 series video cards to run the technology.

AMD Ryzen 7000 and 8000 APUs can use the XDNA NPU (AI accelerator), and Radeon RX 7000 cards can use AI acceleration cores. The company announced in a blog post how to run the chatbot on your computer. Download the appropriate version of LM Studio (for AMD Ryzen processors, for AMD Radeon RX 7000 series graphics cards) and run it. On the Search tab, copy the following search term depending on what you want to run. For Mistral 7b, you need “TheBloke/OpenHermes-2.5-Mistral-7B-GGUF” and select it from the results on the left, it will likely be first on the list. To run LLAMA v2 7b, search for “TheBloke/Llama-2-7B-Chat-GGUF”. You can also try other models: On the right panel, scroll to the Q4 KM model file and download it. AMD recommends this for most Ryzen MI models. Switch to the chat tab and wait for it to start.

The situation is a little different if our device does not have such a processor and we instead use an AMD Radeon 7000 video card. In this case, the GPU Offload option must be selected on the right panel, which must then be converted to the maximum value. Then we check the selected GPU type to see if it indicates AMD ROCm. Then we can start using the chatbot.

AI has become a part of our lives in every way, which is why we're seeing expansion from Nvidia and AMD.

source: WCCFTech, AMD, LM Studio, LM Studio Rocom

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like

Top News

In a harrowing incident that has shaken the community of Lewiston, Maine, a series of shootings on Wednesday evening resulted in a tragic loss...

Top News

President Joe Biden’s abrupt departure from a speech on the U.S. economy at the White House on Monday sent a ripple of speculation and...

Tech

A dangerous application appeared in the Apple App Store disguised as a known program. 24.hu reported the Based on TechCrunch article. Dangerous app in...

World

Chinese scientists have discovered a little-known type of ore containing a rare earth metal highly sought after for its superconducting properties. The ore, called...

Copyright © 2024 Campus Lately.