AI-powered coding assistants are the main talk in the developer world for a while, there’s no denying that. I can’t count the times I’ve read somewhere the AI will replace developers in the next X years. You’ve probably seen tools like GitHub Copilot, ChatGPT, or Tabnine popping up everywhere.

They promise to boost productivity, help with debugging, and even teach you new coding techniques. Sounds amazing, right? But like anything, AI-powered coding assistants have their downsides too. So, let’s talk about what makes them great—and where they might fall short.

Why AI Coding Assistants Are a Game-Changer

Obviously, one of the biggest advantages of using an AI assistant is the time it saves. Instead of writing the same repetitive boilerplate code over and over and over again, you can generate it in seconds. Need a quick function to parse JSON? AI has you covered. Easy peasy! Stuck on how to structure your SQL query? Just ask. This means less time spent on the boring stuff and more time on actual problem-solving.

AI is also a fantastic debugging tool. It can analyze your code, catch potential issues, and suggest fixes before you even run it. Instead of spending hours combing through error messages and Stack Overflow threads, you get quick, relevant suggestions that help you move forward faster.

And let’s not forget about learning. If you’re picking up a new language or framework, an AI assistant can guide you with real-time examples, explain unfamiliar syntax, and even generate sample projects. It’s like having a 24/7 coding mentor who doesn’t judge your questions.

Beyond just speed and learning, AI can actually help improve code quality. It can suggest best practices, helps format your code, and even recommends refactoring when your code gets messy. Plus, if you’re working in a team, it can assist with keeping code style consistent and even generate useful commit messages or documentation. Wouldn’t it be cool if we could plug the AI into our pipeline and make sure that all rules are being followed?

The downsides no one(everyone) talks about?

As cool as AI coding assistants are, you don’t need to be a genius to see that they’re far from perfect. One of the biggest concerns I personally see is over-reliance. If you’re constantly relying on AI to write your code, do you really understand what’s happening under the hood? This can be a problem when something breaks, and you don’t know how to fix it because you never really wrote the thing in the first place I’m sure you love reading someone else’s codebase and debugging that <3

Another issue is that AI-generated code isn’t always optimized or even correct! It might suggest something that works but isn’t efficient, secure, or maintainable. If you blindly accept AI suggestions without reviewing them, you could end up with a mess of inefficient or buggy code.

Then there’s the question of security. AI assistants are trained on huge datasets, and sometimes they can generate code that includes security vulnerabilities. If you’re working on sensitive stuff, you have to be extra careful about what code you’re using and where it’s coming from.

Let’s talk privacy! Many AI coding tools rely on cloud-based processing, meaning your code might be sent to external servers for analysis. If you’re working on proprietary or confidential code, you need to be aware of the risks and check the privacy policies of the tools you’re using.

And finally, while AI can make you more productive, it can also be a bit of a crutch. Some developers might start relying too much on AI for even basic things, which can slow down their growth and problem-solving skills in the long run.

So, Should You Use One?

AI coding assistants are undeniably powerful tools, but they work best when used wisely. They’re great for boosting productivity, helping with debugging, and learning new technologies—but they shouldn’t replace actual coding knowledge and problem-solving skills. Think of them as a really smart assistant, not a replacement for your own expertise.

If you use AI responsibly—review its suggestions, stay mindful of security risks, and make sure you’re still learning and improving as a developer it can be a fantastic addition to your workflow, just don’t let it do all the thinking for you

Still interested and want to start using AI in your daily work? Enter bolt.diy

bolt.diy is the open source version of Bolt.new (previously known as oTToDev and bolt.new ANY LLM), which allows you to choose the LLM that you use for each prompt! Currently, you can use OpenAI, Anthropic, Ollama, OpenRouter, Gemini, LMStudio, Mistral, xAI, HuggingFace, DeepSeek, or Groq models 

bolt.diy was originally started by Cole Medin but has quickly grown into a massive community effort to build the one of the open source AI coding assistants out there.

What do I need to get this deployed?

Well, just Terraform and a Linode account.
In the backend we will deploy a VM with a GPU attached, install bolt.diy, ollama and ask it to write some code! Maybe a simple Tic-Tac-Toe game?

Ideally you would run your bolt.diy deployment on a separate machine from the machine running the model, but for our use case, current deployment model is more than enough.

Like most of the things on this blog, guess what we’re gonna use? Yes! IaC!!!

Here’s a link to the Github repository containing the Terraform code.

Code will do the following:

1. Deploy a GPU based instance in Akamai Connected Cloud
2. Use cloud-init to install the following:
   • curl
   • wget
   • nodejs
   • npm
   • nvtop – great tool to monitor your GPU usage
   • Nvidia drivers
3. Deploy and configure a firewall which will allow SSH and bolt.diy access from your IP.
4. Configure bolt and ollama to run as a Linux service. For ollama service, we are always making sure we have a model downloaded and created with 32K context size.

How do you deploy it?

Just fill in your Linode API token and the desired region, Linode token and your IP address in variables.tf file and run the following commands:

git clone https://github.com/aslepcev/linode-bolt.diy
cd linode-bolt.diy
#Fill in the variables.tf file now
terrafom init
terraform plan
terraform apply

After a short 5-6 minute wait, everything should be deployed and ready to use. Go ahead and visit the IP address of your VM on the port 5173.

Example url: http://172.233.246.209:5173

Make sure that Ollama is selected as a provider and you’re off to the races!

What can it do?

Well, it really depends on the model we are running. With the RTX 4000 Ada GPU, we can comfortably run a 14B parameter model with 32K context size which is “ok” for smaller and simpler stuff.

I tested it out with a simple task of creating a Tic-Tac-Toe game in NodeJS. It got the functionality right the first time, but it looked like something only a mother could love

I just told it to make it a bit prettier and add some color; these were the results I got:

Interestingly, during the coding process, it made a mistake which it managed to identify and fix all on its own! All I did was press the “Ask Bolt” button.

Also, here’s a fully functioning Space Invaders alike game which it also wrote

What if I want to run a larger model? 32B parameters or even larger?

That’s very easy! Since Ollama can use multiple GPU’s, all we need to do is scale up the VM we are using to the one which includes two or more GPU’s. Akamai offers maximum of 4 GPU’s per VM which brings up to 80 GB of VRAM which we can use to run our model. I will not experiment with larger models in this blog post; this is something we will benchmark and try out in the future.

Cheers! Alex.

P.S – parts of this post were written by bolt.diy

The post Your own AI coding assistant running on Akamai cloud! first appeared on Architect the cloud.

Before we dive into installation and real usage, let’s talk about DeepSeek-R1 a bit and why it’s a big thing.

The team behind DeepSeek-R1-Zero took a cool approach to advancing large language models by skipping supervised fine-tuning (SFT) entirely and jumping straight into reinforcement learning (RL).

This resulted in a model capable of self-verification, reflection, and generating intricate chain-of-thoughts (CoTs) to solve complex problems. What’s borderline groundbreaking here is that this is the first open research to show RL alone can incentivize reasoning in LLMs, without the need for SFT. Development pipeline for DeepSeek-R1 includes two RL stages to improve reasoning patterns and align with human preferences, along with two SFT stages to seed the model’s reasoning and non-reasoning capabilities.

They didn’t stop at advancing reasoning in large models—they also focused on making smaller models more powerful! By distilling the reasoning patterns of larger models into smaller ones, they achieved better performance than what RL alone could achieve on small models.

Using DeepSeek-R1’s reasoning data, they fine-tuned several dense models and demonstrated exceptional benchmark results. To further benefit the community, they’ve open-sourced distilled checkpoints ranging from 1.5B to 70B parameters, based on the Qwen2.5 and Llama3 series.

What is distillation you may ask?

In the context of AI, distillation (short for knowledge distillation) is a technique used to transfer knowledge from a larger, more complex model (called the teacher) to a smaller, simpler model (called the student). The goal is to make the smaller model perform as well as—or almost as well as—the larger model, while requiring significantly fewer resources (e.g., memory, computation).

How Distillation Works:

1. Training the Teacher Model:
   A large model is trained first. This model is usually highly expressive, capable of capturing complex patterns in the data, but it may be too computationally intensive for practical deployment.
2. Soft Labels as Knowledge:
   Instead of just training the student model on the original dataset, the student learns from the soft predictions of the teacher model. These predictions include probability distributions over all possible outcomes (rather than binary correct/incorrect labels), which contain richer information about the relationships between classes.
3. Student Model Training:
   The student model is trained to mimic the teacher’s behavior using a combination of:
   • The soft predictions from the teacher model (knowledge transfer).
   • The original ground-truth data.

Why Distillation is Useful:

• Efficiency: Smaller models are faster, require less memory, and can run on devices with limited computational power (like mobile phones or edge devices).
• Scalability: By distilling large models into smaller ones, researchers and developers can deploy AI more broadly while maintaining strong performance.
• Accessibility: Distilled models often serve as the foundation for open-source projects, making advanced AI capabilities more accessible to the community.

Ok, let’s get to the interesting part and deploy our model. We can break down the entire deployment in 4 easy steps.

Step 1 – Deploy a GPU VM in Linode. I’m gonna deploy the smallest GPU VM available “RTX4000 Ada x1 Small” which costs 350$ per month.

Step 2 – Connect using SSH to your instance and install Nvidia drivers

Since I’m using Debian 12 as my OS, installing Nvidia drivers was quite easy following official instructions – https://wiki.debian.org/NvidiaGraphicsDrivers

After the driver has been install, I suggest you also install “nvtop” using:

apt install nvtop

nvtop is a cool utility which can monitor the usage of your GPU. Think of it like a “top” command, but for GPU’s.

Step 3 – Install Ollama

Installing Ollama is extremely simple, all you need to do it run a single command.

curl -fsSL https://ollama.com/install.sh | sh

Step 4 – Download and run the DeepSeek-R1 model into Ollama. Since the GPU I will be using has 20 GB of VRAM, we can fit a 14B model without any issues.

–verbose flag tells Ollama to display performance metrics after the generation has been completed.

ollama run deepseek-r1:14b --verbose

Let’s ask a simple question and see how it performs: What is distilling in the context of AI?

<think>

Hmm, distillation… I think it’s related to machine learning, specifically in training
models. Maybe it’s about transferring knowledge from one model to another? Let me recall.
Oh right, model distillation is the process where you take a complex, large model and
create a smaller, more efficient version that retains the same performance.

Wait, there are different types of distillation too. I should mention those. There’s
knowledge distillation where you transfer knowledge from a teacher model to a student
model. Then there’s parameter-efficient methods like fine-tuning or adapter-based
approaches. Also, pruning and quantization techniques come under distilling in terms of
optimizing models.

I need to make sure the explanation is clear but not too technical. The user might be
someone new to AI or looking to understand optimization techniques. They probably want a
practical understanding without getting bogged down by jargon.

Maybe I should start with the general idea, then break it down into types, and explain
each briefly. Also, highlighting why it’s important—like efficiency, scalability,
deployment on devices—would be helpful for them to grasp its value.

I should keep the response structured but conversational, avoiding any markdown. Just
plain, easy-to-understand English. Let me put that together.
</think>

In the context of AI and machine learning, distilling refers to a technique where
knowledge or information from one model is transferred to another, often simpler or
smaller model. This process aims to create a more efficient version of the original model
while preserving its performance.

Key Ideas:

1. Model Distillation:

• It involves training a “student” model to mimic the behavior or decisions of a
  larger, more complex “teacher” model.
• The teacher model is typically pre-trained on a large dataset and has high accuracy
  but may be computationally expensive or difficult to deploy.

1. Types of Distillation:

• Knowledge Distillation: The teacher model’s predictions (soft labels) are used to
  train the student model, which can lead to better generalization compared to using only
  hard labels.
• Parameter-Efficient Methods: Techniques like fine-tuning, adapter-based
  approaches, or prompt tuning allow the transfer of knowledge without copying all
  parameters from the teacher model.
• Pruning and Quantization: These are techniques used to reduce the size of a model
  by removing unnecessary weights (pruning) or reducing the precision of the weights
  (quantization).

1. Applications:

• Distillation is widely used to deploy AI models on edge devices where computational
  resources are limited.
• It also helps in scaling down large models for faster inference while maintaining
  performance.

In summary, distilling in AI is about making complex models more efficient and accessible
by transferring their knowledge to simpler or smaller models.

total duration: 19.004002094s
load duration: 12.267119ms
prompt eval count: 105 token(s)
prompt eval duration: 48ms
prompt eval rate: 2187.50 tokens/s
eval count: 615 token(s)
eval duration: 18.938s
eval rate: 32.47 tokens/s

One major difference between DeepSeek-R1 and majority of other generative AI models is that DeepSeek will show you the “train of thought” it used to get to the answer, not just the answer itself.

Once we have the model running locally, we can turn it into our own personal AI assistant by using Open-WebUI or turn it into your own coding assistant using Bolt.diy.

Ollama also has it’s on API which you can use to embed the model into your own applications.

Possibilities are endless!

BTW, we can even run a 32B parameter model, but the performance drops to 10 tokens per second and spills a bit into the RAM and CPU, but nothing drastic. Model is still fully usable and comparable to ChatGPT on a busy day.

Results you can see on the image are based on the request to build me “Space Invaders” game in HTML.

Until next time where we will take a look into bolt.diy and deploy our own coding assistant!

Alex!

The post DeepSeek-R1 on Akamai Connected Cloud first appeared on Architect the cloud.

We leveraged just two dual-core VMs and a Nodebalancer to process a pretty impressive number(40K) of movie reviews in just 20 minutes. However, deploying in a single region can lead to latency/availability issues and doesn’t fully utilize the global reach of modern cloud infrastructure Akamai Connected Cloud offers.

Also, single region deployments are kinda boring

In this post, we’ll expand our deployment by setting up virtual machines in all available Linode regions and replacing the current Nodebalancer with Akamai’s new Cloud Load Balancer (currently in beta access).

What is Akamai’s new Cloud Load Balancer you may ask? It’s really cool peace of tech.

Think of it like an umbrella over the internet; it gives you the possibility to load balance your workloads across ANY location; it can be on prem, Akamai’s cloud, some other hyper-scaler, heck, it can even be your home IP address if you want to

As long as it can be reached over the internet, Cloud Load Balancer can use it to deliver the request.

Joking aside, here’s a more official description of the service:

The Akamai Cloud Load Balancer (ACLB) (formerly referred to as Akamai Global Load Balancer) is a layer 4 and 7 load balancer that distributes traffic based on performance, weight, and content (HTTP headers, query strings, etc.). The ACLB is multi-region, independent of Akamai Delivery, and built for East-West and North-South traffic. Key features include multi-region/multi-cloud load balancing and method selection.

Why Scale Globally?

Scaling out our application across multiple regions has several benefits:

1. Reduced Latency: By having servers closer to our users, we can significantly reduce the time it takes for requests to travel back and forth.
2. High Availability: Distributing the load across multiple regions ensures that if one region goes down, well, we kinda don’t care, our app stays online
3. Better Performance: I mean, you can’t beat physics; simply having the possibility to do compute closer to the user improves performance and user experience.

Step-by-Step Deployment Guide

Ok, let’s get into the meaty part; codebase from our previous post hasn’t changed significantly; only thing which we changed is that we’re not hardcoding out region anymore, but we are fetching the list of available regions from Linode API and deploying an instance in each region.

In the code which fetches the regions, you will notice that I commented out the authentication part.

Some regions are available only to authenticated users; if you’re one of those, just uncomment those few lines and the full region list will be returned to you.

Let’s start with terraform plan and see what we will create.

terraform plan

Ok, 25 instances, just what we expect since Akamai has 25 compute regions currently publicly available.

Let’s proceed with terraform apply

terraform apply

I don’t know about you, but I always nerd out on seeing a bunch of servers popping up in the console

In a minute or two we should have all instances deployed. After the instances have been deployed, cloud-init will kick off and install DeepSparse server with an Nginx proxy in front (check out the previous post or the YAML file in the repo for more details).

Awesome! After we’ve got the infrastructure up and running, last step is to add our nodes to the Cloud load balancer pool; at the moment we will need to do some Clickops CLB service is currently in beta so IaC support isn’t out yet.

First step is creating a Cloud Load Balancer by clicking the “Create Cloud Load Balancer” button and giving it a name.

During the beta period, Cloud Load Balancer is deployed only in 5 locations. This number will grow drastically once the service goes GA.

With Akamai’s Cloud Load Balancer, everything starts with a “Configuration“. Let’s create one by pressing “Add Configuration” button.

We will configure our load balancer to “listen” on both HTTP and HTTPS. Once we selected HTTPS as our protocol, we need to add a certificate.

In order to do that, we need to prepare our certificate and private key which we will paste into the configuration field. In this case I will use a self-signed certificate.

At this stage we will only cover the configuration for the HTTPS protocol, HTTP is really easy and won’t bother wasting your time on it.

We need to paste in the certificate & the key, enter the SNI hostname and press “Create and Add” button.

After we’ve got the configuration and the certificate added, we need to create a “Route“. Let’s click on “Create a New HTTP Route” button and give it a name.

Great, we’ve created a route, but the route is currently empty and it doesn’t route anything. We will come back to this a bit later.

Next step is to save our configuration and click on “Service Targets” tab.

This is the place where we will define our target groups and origin servers. Click on “New Service Target” button

Next steps are quite explanatory, we need to give it a name and add the nodes which we want to load balance across.

Remember, this can be one of the existing Linode instance or it can be ANY IP address which can be reached via the internet.

This is the step where I could really use IaC support, we need to add all 25 servers by using ClickOps to our “Endpoints” list.

This is also the place where you can also select the load balancing algorithm which will be used to balance requests between the nodes. At the moment there are 5 of them available:

• Round Robin
• Least Request
• Ring Hash
• Random
• Maglev

Last step in “Service Target” configuration is to set the path and host header we will use for the health checks on the nodes and click on “Save Service Target” button.

We’re almost there, I promise

Final step is to go back to the “Routes” tab, click on the route which we’ve created earlier and click on Edit button.

In the rule configuration we will enter the hostname which we want to match upon and select our “Service target” from the dropdown.

We can also do advanced request matching based on path, header, method, regex or query string but for now we will use path prefix.

Ok, we have configured our Cloud Load balancer; final step to get our application running is to create a CNAME record for my subdomain “clbtest.slepcevic.net” and point it to “MDEE053110.mesh.akadns.net” (visible in the Summary page of the load balancer).

Let’s go ahead and visit our website. We see that our DeepSparse API server is happily responding and ready to receive requests! Woohoo!

Yes, it’s that easy. In less than 10 minutes we have deployed a globally distributed application on Akamai Connected Cloud. Once IaC support for Cloud Load Balancer is rolled out, we can bring this time down to 5 minutes without any problems.

Ok, 25 regions is cool, but that isn’t truly global is it?

Yes, you’re right; with 25 regions we have covered the large majority of the global (internet) population. Can we do better? For sure! Welcome Gecko!

Gecko?

Akamai’s new initiative, code-named Gecko, is set to revolutionize cloud computing by integrating cloud capabilities directly into Akamai’s extensive edge network. This move aligns perfectly with Akamai’s strategy to provide high-performance, low-latency, and globally scalable solutions. By embedding compute capabilities at or VERY near the edge, Gecko aims to deliver workloads closer to users, devices, and data sources than ever before.

What Does Gecko Mean for Our Deployment in the future?

Gecko’s will enable us to deploy our sentiment analysis application in hundreds of new locations worldwide, including traditionally hard-to-reach areas. This means extremely reduced latency, improved performance, and enhanced availability for users across the world.

The Benefits of Deploying on Gecko

1. Ultra-Low Latency: By running our workloads even closer to end-users, we can drastically reduce the time it takes to process and respond to requests.
2. Global Reach: With Gecko, we can deploy in cities and regions where traditional cloud providers struggle to reach, ensuring a truly global presence.
3. Scalability and Flexibility: With Akamai’s large compute footprint, we can scale out our application to tens of thousands of nodes across hundreds of locations.
4. Consistent Experience: Let’s be real, if you’re running a global application, you’re most probably dealing with multiple providers; with Gecko we can consolidate all of your workloads and location coverage with a single provider. Just the operational benefits of that should be enough to “tickle” your brain into considering it for your application.

Want to try it yourself?

Have fun Go ahead and clone the repo from https://github.com/slepix/neuralmagic-globalLinode and sign up to receive beta access to the new Cloud Load balancer on https://www.linode.com/green-light/ .

NeuralMagic running across all Linode (including Gecko) regions in the next post? Perhaps

Cheers! Alex

The post Deploying a GLOBAL sentiment Analysis service using DeepSparse and Akamai Connected Cloud first appeared on Architect the cloud.

DeepSparse is a sparsity-aware inference runtime that delivers GPU-class performance on commodity CPUs, purely in software, anywhere.

GPUs Are Not Optimal – Machine learning inference has evolved over the years led by GPU advancements. GPUs are fast and powerful, but they can be expensive, have shorter life spans, and require a lot of electricity and cooling.

Other major problems with GPU’s, especially if you’re thinking in the context of Edge computing, is that they can’t be packed as densely and are power ineffective compared to CPU’s; not to mention availability these days.

Since Akamai recently partnered up with Neural Magic, I’ve decided to write a quick tutorial on how to easily get started with running a simple DeepSparse sentiment analysis workload.

In case you want more about Akamai and Neural Magic’s partnership, make sure to watch this excellent video from TFiR. It will also give you a great summary of Akamai’s Project Gecko.

What is Sentiment analysis?

Sentiment analysis (also known as opinion mining or emotion AI) is the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine. 

Why is DeepSparse cool? Because I’m doing analysis of 40 thousands movie reviews in 20 minutes using only TWO DUAL CORE Linode VM’s. Mind officially blown.

Let’s do some math here; rounding it up to 120 thousand processed reviews an hour, with 2 instances and a load balancer, we can process over 86 million requests a month which will cost you a staggering 82$ .

If you’re doing that on other cloud providers, you’re paying a five digit monthly bill for that pleasure.

Want to try it yourself? It’s easy!

If you want to try it out on Linode, follow instructions below.

If you want to check out Neural Magic DeepSparse repo, head out here.

Step 1. Clone the Repository.

Open your terminal or command prompt and run the following command:

git clone https://github.com/slepix/neuralmagic-linode

This code will deploy 2 x Dedicated 4 GB virtual machines and a Nodebalancer. It will also install Neural Magic’s DeepSparse runtime as a Linux service and install & configure Nginx to proxy requests to DeepSparse server listening on 127.0.0.1:5543.

WARNING: THIS IS NOT PRODUCTION GRADE SERVER CONFIGURATION!

It’s just a POC! Secure your servers and consult Neural Magic documentation if you want to go to production.

Step 2. – Terraform init

Navigate to the repo using the following command:

cd neuralmagic-linode

If you haven’t already installed Terraform on your machine, you can download it from the official Terraform website and follow the installation instructions for your operating system.

Step 3.

Initialize Terraform by running:

terraform init

Step 4. – Configure your Linode token

Open variables.tf file and paste in your Linode token. If you don’t know how to create a Linode PAT, check this article here. It should look similar like the picture. You can also adjust the region while you’re here

Token in the picture is not valid. It's just an example. 

Step 5 – Run Terraform apply

After configuring your variables, you can apply the Terraform configuration by running:

terraform apply

Terraform will show you a plan of the changes it intends to make.

Review the plan carefully, and if everything looks good, type “yes" and press Enter to apply the changes. Give it 5-6 minutes to finish everything and by visiting your Nodebalancer IP, you should be presented with a landing page for DeepSparse server API.

Step 6.

After the installation is done, it’s finally time to send some data to our API and see how it performs.

We can do that by using curl or invoke-webrequest if you’re on Windows and using Powershell.

CURL:

sentence="Neural Magic & Akamai are cool!"
nodebalancer="172.233.34.110" #PUT YOUR NODEBALANCER IP HERE
curl -X POST http://$nodebalancer/v2/models/sentiment_analysis/infer -H "Content-Type: application/json" -d "{\"sequences\": \"$sentence\"}"

PowerShell:

$sentence = "Neural Magic & Akamai are cool!"
$nodebalancer = "172.233.34.110"

$path = "v2/models/sentiment_analysis/infer"
$api = "http://$nodebalancer/$path"
$body = @{
   sequences = $sentence
} | ConvertTo-Json

(Invoke-WebRequest -Uri $api -Method Post -ContentType "application/json" -Body $body -ErrorAction Stop).content

In both cases make sure to paste in the IP address of the Nodebalancer you deployed and modify the sentence as you wish.

Benchmark time!

In the repository, I’ve included a file called movies.csv and three files; two PowerShell and one Python file.

movies.zip – unzip this one in the same folder where your benchmark scripts are.

analyze.ps1 – PowerShell based benchmark, sends requests in serial – not performant.

panalyze.ps1 – PowerShell based benchmark, sends requests in parallel – better performant

pypanalyze.py – Python based benchmark, sends requests in parallel – best performer (doh!) <-use this

All you need to do to in order to kick off a benchmark is to update the the URL variable with your Nodebalancer IP and you’re off to the races.

Does it scale?

Yes! For kicks I’ve added a third node and the same job finished in 825 seconds. Feel free to add as many nodes as you like and see what numbers you can get. Additionally, you can play with the number of workers in the Python file.

Note 1: python script has been written with the help of ChatGPT :) Results matched with my PowerShell version against verified smaller sample size(check note 2), so I'm gonna call it good :)
	 
Note 2: PowerShell versions don't handle some comments as they should and end up sending garbage to the API. Happens in 3% of the cases. Most probably some encoding/character issue which I couldn't be bothered to fix :)

Note3: Movies.csv file has been generated by using data from https://kaggle.com/

Cheers,

Alex.

The post Sentiment analysis of 40 thousand movie reviews in 20 minutes using Neural Magic’s DeepSparse inference runtime and Linode virtual machines. first appeared on Architect the cloud.